Problem with a select with hundred thousand records.
843854Sep 7 2004 — edited Sep 11 2004Hello,
I have a mysql database. I have a table with the following fields:
time long primary key, open double, high double, low double, close double, volume int.
It stores stock prices.
I store one record for a 1 minute price bar.
I use the following code to read the bars:
String sql = "select * from bar1min_"+assetId+" where time >= '"+date1+"' and time <= '"+date2+"'";
Connection conn=DBUtil.getConnection(dbhost);
if(conn == null) return null;
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql);
while (rs.next()) {
bar.time=Long.parseLong(rs.getString("time"));
bar.O = rs.getDouble("open");
bar.H = rs.getDouble("high");
bar.L = rs.getDouble("low");
bar.C = rs.getDouble("close");
bar.V = rs.getInt("volume");
marketStart=asset.thisDayMarketStart(bar.time);
OneMinuteBarLoader.createBars(barss, bar, newbar, lastBarTime,interval,marketStart,marketEnd,stra);
}
I have several problems:
1.- If I select a period of two years (350.000 records) it works but it takes more than 1 minute
2.- with three years (500.000 records) it doesn't read a single record (No error is displayed).
3.- When it works it uses many mb of ram when commercial products of this kind use just 1mb for several years.
4.- When I do the same select from a dos windows it reads the 500.000 records in 0.24 sec(without displaying them of course)
Which would be the best way to define the table so the load/store is faster??? Could I load/store one bar in the same format I have in my class bar to make it faster??.
Can anyone please help me?