Skip to Main Content

Java Programming


For appeals, questions and feedback, please email

How to design a Java desktop application for processing a file with a size of up to 100GB?

User_I6DXROct 28 2022

A requirement of a standalone Java application is that it shall be able to process a data file with a size of up to 100 GB. The file contains single one line of data. The data type is the IBM dataset. And the same data can be processed multiple times with various parameters.
The challenge is that I can't load all of the data into memory. That occurs at different stages. Reading the data from a file is one. Presenting the processed data on the GUI (JavaFX) is another. And there are one or two in between.
I am thinking of using an embedded database, such as MongoDB to hold the data. How shall I build such an app to meet the requirement?

Post Details
Added on Oct 28 2022