Oracle VM server 3.0.2 disk performance
I've suspected Oracle VM from very slow disk operations.
I've done a test on dum0 and domU (paravirtualized OEL5U7 x86 template from edelivery) with following two commands:
1) time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 && sync )
2) time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 oflag=direct && sync )
Below you can find the results.
My question is: *"Is this expected io performance for paravirtualized guest?"*
dom0 vs. domU
1) *6.5* seconds vs. *105* seconds for 1000MB
2) *27* seconds vs. *118* seconds for 1000MB
HW:
- Dell r710 2x4core, 64GB RAM, 2x300GB 15krpm
dom0:
- Oracle VM 3.0.2
- ocfs2, mount options (rw,heartbeat=none)
- only server installation without manager for testing purposes
- xm dmesg
(XEN) Xen version 4.0.2-OVM (mockbuild@(none)) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) Thu Sep 22 16:11:15 EDT 2011
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: dom0_mem=1825M
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 00000000000a0000 (usable)
(XEN) 0000000000100000 - 00000000cf679000 (usable)
(XEN) 00000000cf679000 - 00000000cf68f000 (reserved)
(XEN) 00000000cf68f000 - 00000000cf6ce000 (ACPI data)
(XEN) 00000000cf6ce000 - 00000000d0000000 (reserved)
(XEN) 00000000e0000000 - 00000000f0000000 (reserved)
(XEN) 00000000fe000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000001030000000 (usable)
(XEN) ACPI: RSDP 000F0A90, 0024 (r2 DELL )
(XEN) ACPI: XSDT 000F0B94, 009C (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: FACP CF6B3F9C, 00F4 (r3 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: DSDT CF68F000, 3D5A (r1 DELL PE_SC3 1 INTL 20050624)
(XEN) ACPI: FACS CF6B6000, 0040
(XEN) ACPI: APIC CF6B3478, 015E (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: SPCR CF6B35D8, 0050 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: HPET CF6B362C, 0038 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: DMAR CF6B3668, 01C0 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: MCFG CF6B38C4, 003C (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: WD__ CF6B3904, 0134 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: SLIC CF6B3A3C, 0024 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: ERST CF692EDC, 0270 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: HEST CF69314C, 03A8 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: BERT CF692D5C, 0030 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: EINJ CF692D8C, 0150 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: SRAT CF6B3BC0, 0370 (r1 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: TCPA CF6B3F34, 0064 (r2 DELL PE_SC3 1 DELL 1)
(XEN) ACPI: SSDT CF6B7000, 4E44 (r1 INTEL PPM RCM 80000001 INTL 20061109)
(XEN) System RAM: 65526MB (67098724kB)
(XEN) Reserving non-aligned node boundary @ mfn 0x830000
(XEN) Domain heap initialised DMA width 32 bits
(XEN) Processor #32 6:12 APIC version 21
(XEN) Processor #0 6:12 APIC version 21
(XEN) Processor #34 6:12 APIC version 21
(XEN) Processor #2 6:12 APIC version 21
(XEN) Processor #50 6:12 APIC version 21
(XEN) Processor #18 6:12 APIC version 21
(XEN) Processor #52 6:12 APIC version 21
(XEN) Processor #20 6:12 APIC version 21
(XEN) Processor #33 6:12 APIC version 21
(XEN) Processor #1 6:12 APIC version 21
(XEN) Processor #35 6:12 APIC version 21
(XEN) Processor #3 6:12 APIC version 21
(XEN) Processor #51 6:12 APIC version 21
(XEN) Processor #19 6:12 APIC version 21
(XEN) Processor #53 6:12 APIC version 21
(XEN) Processor #21 6:12 APIC version 21
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 1, version 32, address 0xfec80000, GSI 32-55
(XEN) Enabling APIC mode: Phys. Using 2 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3059.102 MHz processor.
(XEN) Initing memory sharing.
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) EPT supports 2MB super page.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Total of 16 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) TSC is reliable, synchronization unnecessary
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 32 KiB.
(XEN) Brought up 16 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x6d2000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000001006000000->0000001008000000 (459008 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80002000->ffffffff806d2000
(XEN) Init. ramdisk: ffffffff806d2000->ffffffff81092a00
(XEN) Phys-Mach map: ffffea0000000000->ffffea0000390800
(XEN) Start info: ffffffff81093000->ffffffff810934b4
(XEN) Page tables: ffffffff81094000->ffffffff810a1000
(XEN) Boot stack: ffffffff810a1000->ffffffff810a2000
(XEN) TOTAL: ffffffff80000000->ffffffff81400000
(XEN) ENTRY ADDRESS: ffffffff80002000
(XEN) Dom0 has maximum 16 VCPUs
(XEN) Scrubbing Free RAM: .....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 168kB init memory.
(XEN) ioapic_guest_write: apic=1, pin=23, irq=55
(XEN) ioapic_guest_write: new_entry=0001a028
(XEN) ioapic_guest_write: old_entry=00010000 pirq=55
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 && sync )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 4.87025 seconds, 215 MB/s
real 0m6.424s
user 0m0.008s
sys 0m0.760s
time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 oflag=direct && sync )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 26.5861 seconds, 39.4 MB/s
real 0m26.705s
user 0m0.004s
sys 0m1.100s
domU:
- OVM_OL5U7_X86_PVM_10GB template
- only this one VM running in server
- cat vm.cfg
bootloader = '/usr/bin/pygrub'
disk = ['file:/u01/running_pool/tmp/OVM_OL5U7_X86_PVM_10GB/System.img,xvda,w']
memory = '2048'
name = 'tmp3'
vcpus = 2
on_crash = 'restart'
on_reboot = 'restart'
vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0']
vif = ['type=netfront, mac=00:16:3e:3b:26:a9, bridge=xenbr1']
timer_mode = 0
time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 && sync )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 93.2052 seconds, 11.3 MB/s
real 1m45.583s
user 0m0.003s
sys 0m2.602s
time ( dd if=/dev/zero of=test.zero bs=1024k count=1000 oflag=direct && sync )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 118.337 seconds, 8.9 MB/s
real 1m58.646s
user 0m0.001s
sys 0m0.372s
The performance of 64bit PVM is similar.
The performance is even worse then emulated HVM.
With smaller block sizes e.g. 8kb the throughput is about 300kb/s.
We work at proof of concept ... hardpartition the hardware to be able to correctly license oracle database.
We don't try the performance of "phy:/dev/xxx" for ASM volumes yet, hope will be better.
It's a paint that you need the whole infrastructure (VM manager, shared disks,) for one Oracle VM server version 3 compared to Oracle VM server version 2.2.
If you can compare the performance of your hardware and write your opinions to concept (running oracle database on oracle VM), it would be very helpfull.
Thank you and regards.
Karel