Writing, Multiple Threads, large files?
807601Jun 18 2008 — edited Jun 18 2008I am working on a simulation. In the simulation, I have some data models (up to 10). These data models need some rather lengthy calculations performed on them. To do the calculations, I run up to three models at a time, each in a separate thread. Back when the simulation was designed to take in only one model, writing out the calculated data left me with a 40mb CSV file. That was fine when it was one model but certainly not ideal when it is possible to have 10 models.
Will it even be possible to use the three threads to write out that much data as it is calculated to three separate files?
If it is possible, is there any way to cut down that file size without losing data? The output file would not have to be human-readable. Or is there a way to program in some sort of compression after they are written?
Any advice is greatly appreciated.