Skip to Main Content

Java Database Connectivity (JDBC)

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Problem with a select with hundred thousand records.

843854Sep 7 2004 — edited Sep 11 2004
Hello,

I have a mysql database. I have a table with the following fields:
time long primary key, open double, high double, low double, close double, volume int.
It stores stock prices.

I store one record for a 1 minute price bar.
I use the following code to read the bars:

String sql = "select * from bar1min_"+assetId+" where time >= '"+date1+"' and time <= '"+date2+"'";
Connection conn=DBUtil.getConnection(dbhost);
if(conn == null) return null;
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql);
while (rs.next()) {
bar.time=Long.parseLong(rs.getString("time"));
bar.O = rs.getDouble("open");
bar.H = rs.getDouble("high");
bar.L = rs.getDouble("low");
bar.C = rs.getDouble("close");
bar.V = rs.getInt("volume");
marketStart=asset.thisDayMarketStart(bar.time);
OneMinuteBarLoader.createBars(barss, bar, newbar, lastBarTime,interval,marketStart,marketEnd,stra);
}

I have several problems:

1.- If I select a period of two years (350.000 records) it works but it takes more than 1 minute
2.- with three years (500.000 records) it doesn't read a single record (No error is displayed).
3.- When it works it uses many mb of ram when commercial products of this kind use just 1mb for several years.
4.- When I do the same select from a dos windows it reads the 500.000 records in 0.24 sec(without displaying them of course)

Which would be the best way to define the table so the load/store is faster??? Could I load/store one bar in the same format I have in my class bar to make it faster??.

Can anyone please help me?

Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Oct 9 2004
Added on Sep 7 2004
19 comments
660 views