I am leveraging vdbench to generate raw data workloads to drives inside of a python based software application. Each drive under test spawns an independent vdbbench JVM to generate workloads to each drive. This means there can be up to 8 concurrent instances of vdbench running simultaneously on a system. Since updating to vdbench 5.03, i've seen a new failure mode where each independent JVM fails simultaneously with the following error message:
14:31:36.020 Receiving unexpected EOFException from slave: localhost-0
14:31:36.020 This means that this slave terminated prematurely.
14:31:36.020 This thread will go to sleep for 5 seconds to allow
14:31:36.020 slave termination to be properly recognized.
From the local-host0.stdout:
14:31:36.006 14:31:36.006 Unable to find bucket. Are we having a Harry Belafonte moment here? -83671
14:31:36.006 14:31:36.006
14:31:36.006
14:31:36.006 Vdbench JNI abort: Unable to find bucket. Are we having a Harry Belafonte moment here?
14:31:36.006
14:31:36.020 Abort
Here are the command line parameters i am passing to start the vdbench instance inside each python thread:
arg_list = ['java', '-client', '-Xmx512m', '-Xms64m', '-cp',
'{0}:{0}classes:{0}vdbench.jar'.format(vdb_wrapper.vdbench_dir),
'Vdb.Vdbmain', '-d25', '-f',self.vdb_file,'-p',str(port),'-o',
self.output_path,
]
I saw in another thread that the EOFException could be caused by lack of memory resources, but i am pretty confident this is not the case. With 8 drives simultaneously running my worst workload, I utilize about 4GB of system memory, with 6GB physical on the system.