New to Java

NOTE
I "think" I might have the answer. There must be metadata stored with each fetch. I am going to do a run using: "+clearWarnings()+."
This test will take about 5-minutes....

I am:
(1) reading in one column per row from a mysql table.
(2) the column is a utf-8 encoded String.
(3) the String can be upto 256 characters long.
(4) I have about 80,000 rows to process.
If the integer value of a character falls within a range, I store it in an output table;

My program seems to work fine.
After 5 minutes of running, it inserts 3500 characters into the output table.
3500 is a reasonable result.
However, the program does eventually crash with the following stack trace:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2928)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1571)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1666)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2988)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2917)
at com.mysql.jdbc.Statement.execute(Statement.java:529)
at alpha.DataUtil.filterCharacters(DataUtil.java:26) // <-- line #26
at alpha.Main.main(Main.java:11)
Java Result: 1

static void filterCharacters() throws Exception {
  Connection connQuery  = DriverManager.getConnection("jdbc:mysql://localhost/alpha", "root", "xxx");
  Connection connInsert = DriverManager.getConnection("jdbc:mysql://localhost/beta", "root", "xxx");

  Statement stmt = connQuery.createStatement();
  ResultSet rslt = stmt.executeQuery("select longstring from documents;");
  rslt.next();

  while(!rslt.isLast()) {
    String longString = rslt.getString(1);
    for(int i = 0; i < longString.toCharArray().length; i++) {
      char c = longString.charAt(i);
      if(Util.isCharacterOfInterest(c)) {
        Statement st = connInsert.createStatement();
        try {
          st.execute("INSERT INTO interestingCharacters(character) VALUES('" + c + "');"); // <-- line #26
        } catch(SQLException e) { continue; } // catch/ignore primary key collision
      }
    }
    rslt.next();
  }
}

I don't see where a build-up of unreleased objects might be.
And, I thought ResultSet is designed to handle huge data fetches.

Edited by: azjp on May 19, 2011 1:07 PM