I have developed an application that does 4 selects against a oracle database creating 4 resultsets. It than trys to cycle through the resultsets writing the records to a .csv file.
When the result sets start to get too big, more than a total of 50,000 records I get the following error. I have to believe it is some type of memory or time out issue with tomcat or apachie.
Here is the code that I use to write the result sets to a csv file.
String filepath = csvfilepath+bgcsvString+".csv";
PrintWriter pw = new PrintWriter(new FileOutputStream(filepath));
int rSizeList[] = {rsSize3,rsSize5,rsSize4};
ResultSet rsList[] = {mailaddRS,remfunitRS,remfmasterRS};
//mailadd and remf
for (int x=0; x<rsList.length; x++){
if(rSizeList[x]>0){
ResultSet tmpRS = rsList[x];
//get column names and print them to the file
ResultSetMetaData rsmd = tmpRS.getMetaData();
int numberOfColumns = rsmd.getColumnCount();
//write fields names
String rec = "";
for (int i=0; i < fieldNames.length; i++){
rec +='\"'+fieldNames[i]+'\"';
rec+=",";
}
if (rec.endsWith(",")) rec=rec.substring(0, (rec.length()-1));
pw.println(rec);
if(tmpdebugOn>3)out.println("tmp size" +rSizeList[x]);
//write values from result set to file
tmpRS.beforeFirst();
while(tmpRS.next()){
rec = "";
for (int i=1; i < (numberOfColumns+1); i++){
if((rsmd.getColumnTypeName(i)).equals(charfld)) rec +="\""+tmpRS.getString(i)+"\",";
if((rsmd.getColumnTypeName(i)).equals(numfld)) rec +="\""+tmpRS.getInt(i)+"\",";
}
if (rec.endsWith(",")) rec=rec.substring(0,(rec.length()-1));
if(tmpdebugOn>3)out.println("testing "+rec+"<br>");
pw.println(rec);
}
}
}
//close file
pw.close();
Date endcsvTime = new Date();
endcsvString = endcsvTime.toString();
Is there a better way of doing this or some way of streaming it?
ERROR
org.apache.jasper.JasperException
root cause
javax.servlet.ServletException
at org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:536)
at org.apache.jsp.areateamtest_jsp._jspService(areateamtest_jsp.java:426)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:193)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:309)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:387)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:673)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:615)
at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:786)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
at java.lang.Thread.run(Thread.java:534)