Skip to Main Content

Java APIs

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Maximum Throughput -Multi Threading

843810Jun 24 2008
Hi,

My requirement is to improve the batch execution speed by using maximum resouces of system.

Present batch execution.
1. We read no of records from DB in a java a program which needs to be tranfered to third party system.
2. For every 350 (put it in to Vector) records we start new Thread which in turn communicate with third party front end system(legasy) and process 350 records 1 by one,which inturn process and update its DB.
3. here we are invoking a Thread for every 350 records,here the problem is system proceesed 6000 records in 135 min and other day it took same time 135 min to process 12000 records.

here the diff is no of threds invoked to process records, say 6000/350=17 threads , 12000/350=34, batch is getting struck when it reaches140+ Threads, hence we have monitored and can conclude that our Env supports onlly 130 Concurrent threads.

Now i would like to pass the denominator (350) dynamically based on no of records to be processed.
ex: if there are only 350 records to process todays batch i will invoke maximum threads 130 and shre the the records on equal ratio to process, hence i will get maximum throughput.

Here challenge is I should make sure there should always 130 threads running in my env when batch starts, when ever some threads complete its work goes dead stage, i have to find how many threads are alive (Thread.activeCount()) after every some time intervel (Here I would like to use timer) , and invoke those no of threads agin (130-livecont) till the end of the batch.

I woul like to parameterise no of threads to be invoked and Time intervel to check for live thread ount.

can any one shed the light on above implementaion.

my requirement is:
1. needs to find out live count of threads after every 1 min (say), how to use timer or some thing else can be done?
2. Thread.liveCount() is enough to track live threds or any other work around?
3. is it cause any memory issues like Gorbase Collection etc...

Here I am providing code sample that i am using presently.
/**
	 * ThreadInvoker.java
	 */
package MultiThred;

import java.util.Vector;

public class ThreadInvoker 
{
	
	public ThreadInvoker() 
	{		
	}
	public static void main(String[] args) 
	{		
		Vector vectObj = new Vector(350,1);
		MyThread myThread;
		
		int rowSize=0;
		rowSize=no of recordsFeatched to transfer;
		
		int rowsSentToProcess=0;
		int recCnt=0;
		
		vectObj.add(pos,recDVO);
		recCnt ++;
		
		for(int i=0;i<rowSize;i++)
		{
		   if(vectObj.size() == 350 || vectObj.size() == (rowSize - rowsSentToProcess))
		   {
			    myThread=new MyThread(vectObj);
				rowsSentToProcess += vectObj.size();
				vectObj = new Vector(350,1);
				recCnt = 0;
		   }
		}

	}

}


/**
 * MyThread.java
 *
 */
package MultiThred;

import java.util.Vector;

public class MyThread extends Thread
{
	/**
	 * 
	 */
	Vector v = new Vector();
	static protected int totalRec = 0;
	
	public MyThread(Vector _vThread)
	{
		v = _vThread;
		start();
		
	}
	
	public void run()
	{
		for (int i = 0 ; i < v.size() ; i++)
		{
			objRecord = (Record) v.get(i);
			printlog("****************Record aBout to be Processed*********** - " + i);
			
			//Transfering to other sys
			objSomeSys.calltoTranfer(objRecord);
		}
			printlog("Ending the Run method in MyThread ");
			
			totalRec = totalRec + v.size();
			
			printlog("Total Records transferred is " + totalRec);
			printlog("Ending the Run method in MyThread");
		
	}

}
Thanks in Antisipation...
Ananth
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Jul 22 2008
Added on Jun 24 2008
0 comments
211 views