Skip to Main Content

Java Security

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Reading and encrypting a file in chunks

843811Jul 8 2010 — edited Jul 9 2010
Hi there.

I am writing a program in Java to encrypt files. I would like it to be able to encrypt files in pieces, so it can do very large files without trying (and failing) to read the entire file into memory first. Here's what I'm doing:
public static void performCrypto(boolean encrypt) throws IOException
{
	final int MEGABYTE = 1024 * 1024;
		
	File myfile = new File("myfile");
	File outfile = new File("outfile");
	InputStream in = new FileInputStream(myfile);
	OutputStream out = new FileOutputStream(outfile);
		
	long length = myfile.length();
	long currentByte = 0; // for keeping track of where we are in the file, as well as progress monitoring
		
	while (currentByte < length) // while more of the file can be read
	{
		int amountLeft = (int) (length - currentByte);
		int blockUnit = Math.min(MEGABYTE, amountLeft);
		byte[] read = new byte[blockUnit]; // read either one MB, or however much is left in the file
		in.read(read);

		byte[] digest = encrypt ? encrypt(read) : decrypt(read);
		out.write(digest);

		currentByte += blockUnit;
	}
		
	in.close();
	out.close();
}
	
private static byte[] encrypt(byte[] read)
{
	// do encryption with Twofish
	return null;
}
	
private static byte[] decrypt(byte[] read)
{
	// do decryption with Twofish
	return null;
}
Encryption works fine and dandy, just as expected. However, when I try to decrypt the files I encrypt, I run into an error thrown by the BouncyCastle Twofish implementation: "pad block corrupted". Here's the stack trace:
org.bouncycastle.crypto.InvalidCipherTextException: pad block corrupted
	at org.bouncycastle.crypto.paddings.PKCS7Padding.padCount(Unknown Source)
	at org.bouncycastle.crypto.paddings.PaddedBufferedBlockCipher.doFinal(Unknown Source)
	at MyClass.decrypt(Twofish.java: 45)
	at MyClass.performCrypto(TwofishEncryptor.java:29)
	at Driver.main(TwofishEncryptor.java:51)
I didn't include the actual Twofish BouncyCastle call for simplicity, but it isn't particularly important. Files that I encrypt with size less than 1 mb decrypt perfectly. I suspect this might have something to do with BouncyCastle's Twofish adding redundancy bits or some such thing, which is throwing off the decryption after reading it back in, since for each 1MB read in, there might be some K number of extra bits. Looks like my approach will only work if the digest is the same bitlength as the input.

Any ideas how I could encrypt and decrypt files in chunks, so I don't have to rely on the user's machine (and JVM) having enough space for the entire file in memory?
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Aug 6 2010
Added on Jul 8 2010
6 comments
1,944 views