We're starting to use some NFS based Repositories to hold our VM disk images and have been noticing that disk access in the VM to the virtual disk ( say /tmp for example) isn't exactly fast. We're seeing write speeds for multiple 200MB files at 40-120MB/sec. If we do writes directly to NFS (/mnt/nfsdir) we can reach speeds in excess of 800MB/sec! (we have 10Gbit network interlinks).
We did some tests with 'iozone' directly to NFS and it seems that the problem is largely defined by the 'record size' being used during writes. If the record size is sufficiently large, our NFS server preforms very well, but for record sizes below 64k, we get the speeds observed inside the VM talking to the VM's virtual disk. We tested from both a VM and the OVM (DOM_0) and got similar results. This suggests something about how Oracle VM is writing data back to the VM disk image.
I'm putting the question out there about what others are seeing for VM disk performance numbers. As a benchmark, I found that a VM I have hosted at Digital Ocean reliably gets 600MB/sec disk writes.
--
Ray Frush
Colorado State University.