Skip to Main Content

Java Programming

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

FileInputStream and what sizes of files it's capable of handling

807606Apr 3 2007 — edited Apr 3 2007
Hey all -

I've got a few enormous data files (7gigs, 9 gigs, etc...) that I'm working with. I've got a program that will just blindly copy the first N bytes of an input file into a separate file. For my purposes, it's easier for me to work with the first ~3 gigabytes of one of my big data files than working with the whole 7 or 9 gigs.

I'm using a FileInputStream and all appears to go well using the 7 gig file. I can copy out the first 2 gigs into a separate file and everything works great.

However, when I try to do the same on the 9 gigabyte file, the copying stops after just shy of 1 gigabyte (~930mb or so). My loop is still going from 0 bytes to 2 gigabytes, no actual bytes are read from the input file after ~930 megabytes.

I threw in some print statements and it appears that at the beginning of program execution, the FileInputStream.available() call returns only 993718656 (~900 megs) when I'm running on the 9 gigabyte file.

On the 7 gigabyte file, FileInputStream.available() first returns 2147483647 - considerably larger than what .avaialble() starts at on the 9 gigabyte file.

I've read in a few posts that the .available() call tends to be unreliable. I'm not using it by choice here, but more as a method of exploring why my program only copies the first ~900megs instead of the full amount I'd like copied.

Additionally, if theres a better Java.io type I should be using for working with monstrous files, please let me know!

Thanks in advance,
Brad
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on May 1 2007
Added on Apr 3 2007
18 comments
3,870 views