Hi Henk,
I am trying to use vdbench (first time to be honest) to create a baseline for our databases on a new array.
The use case is:
Random IOs going to 16 raw devices, IO size varies between 16K – 256K:
Will appreciate if you can confirm that I am doing it correctly:
- Currently, I am doing a test with 8 devices. to add additional 16 (or even more), can I just add the lines as separate lines starting with “sd” in the same input file? Or should I use a different host?
- Why is the i/o rate less than 20K in the vdbench output? is it because the number of disks underneath the luns can't do more IOPS, and hence the max number of IOPS is determined based upon the output and is always less than about 20K. Why is thread count not making any difference – whether I increase it or decrease it.
- Qdepth appears to be very high, is that the reason that I see a very high await in iostat output indicating that either the SAN switches or the FA ports on the storage array are not able to accept the data fast enough? is write response time in vdbench ~ svctime in iostat, and qdepth in vdbench ~ await in iostat?
- By changing the iorate to “max”, the i/o rate column numbers remain almost same but the write response went up drastically.
- Why is resp max column significantly higher than write resp ?
Input file:
./vdbench -f input2.txt
Input2.txt:
sd=sd1,lun=/dev/mapper/mpathzp1,openflags=o_direct,threads=80
sd=sd2,lun=/dev/mapper/mpathyp1,openflags=o_direct,threads=80
sd=sd3,lun=/dev/mapper/mpathabp1,openflags=o_direct,threads=80
sd=sd4,lun=/dev/mapper/mpathaap1,openflags=o_direct,threads=80
sd=sd5,lun=/dev/mapper/mpathrp1,openflags=o_direct,threads=80
sd=sd6,lun=/dev/mapper/mpathep1,openflags=o_direct,threads=80
sd=sd7,lun=/dev/mapper/mpathqp1,openflags=o_direct,threads=80
sd=sd8,lun=/dev/mapper/mpathpp1,openflags=o_direct,threads=80
wd=wd1,sd=*,xfersize=(16k,20,32k,20,64k,20,128k,20,256k,20),rdpct=60
rd=run1,wd=*,iorate=20000,elapsed=600,interval=1
vdbench output(iorate:20000):
Aug 05, 2016 interval i/o MB/sec bytes read resp read write resp resp queue cpu% cpu%
rate 1024**2 i/o pct time resp resp max stddev depth sys+u sys
13:35:36.048 121 16730.00 1625.84 101902 59.93 23.990 37.647 3.566 569.341 49.173 403.2 13.5 10.3
13:35:37.047 122 16817.00 1628.48 101539 60.03 24.015 37.086 4.384 545.139 47.820 398.2 13.6 10.7
13:35:38.047 123 15945.00 1545.72 101649 60.19 24.086 37.682 3.525 541.637 49.245 388.5 13.7 10.6
13:35:39.047 124 17060.00 1655.38 101745 60.46 23.454 35.725 4.694 566.040 45.682 397.7 13.5 10.4
13:35:40.046 125 15739.00 1505.86 100324 60.20 23.907 37.471 3.389 554.418 48.718 376.7 12.5 9.6
13:35:41.047 126 16136.00 1561.27 101456 60.28 23.707 37.362 2.987 562.409 50.570 386.8 14.2 10.7
13:35:42.046 127 16027.00 1532.67 100275 59.14 25.432 40.954 2.961 585.973 52.993 408.7 13.3 9.9
13:35:43.045 128 16013.00 1567.36 102635 59.90 24.682 38.929 3.398 653.538 50.517 389.9 13.6 10.7
13:35:44.046 129 16885.00 1623.81 100840 59.96 23.396 36.409 3.905 553.656 47.645 394.3 14.0 10.7
13:35:45.045 130 17072.00 1656.55 101746 59.92 23.498 36.016 4.781 561.743 45.840 400.9 14.5 11.3
13:35:46.045 131 17611.00 1717.56 102265 59.99 23.611 35.703 5.481 562.726 43.825 412.7 14.5 11.3
13:35:47.046 132 16732.00 1605.61 100621 59.92 22.274 34.941 3.341 529.253 45.692 373.4 13.5 10.5
13:35:48.086 133 18160.00 1789.52 103328 59.46 23.353 35.805 5.089 565.872 44.713 426.5 14.0 10.8
13:35:49.046 134 15653.00 1516.53 101590 59.89 24.837 39.613 2.772 544.242 51.620 391.9 14.0 11.0
13:35:50.047 135 16840.00 1641.66 102220 59.33 24.332 38.958 2.991 594.722 50.972 410.9 14.2 10.7
Iostat output on the host (iorate=20000)
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
dm-19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-22 0.00 0.00 1240.00 806.20 120448.00 79174.40 195.12 75.70 36.76 0.49 100.00
dm-23 0.00 0.00 1212.00 821.80 120748.80 81347.20 198.74 79.89 39.33 0.49 100.00
dm-24 0.00 0.00 1230.40 790.40 123100.80 78822.40 199.84 79.16 39.13 0.49 100.00
dm-25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-34 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-44 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-46 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-49 0.00 0.00 1224.80 812.40 122265.60 79497.60 198.08 30.95 15.16 0.49 100.00
dm-50 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-53 0.00 0.00 1198.80 811.00 119731.20 81500.80 200.25 5.84 2.91 0.45 91.34
dm-55 0.00 0.00 1216.40 781.80 120128.00 79068.80 199.38 26.54 13.23 0.50 100.00
dm-56 0.00 0.00 1227.00 832.80 125244.80 82262.40 201.48 27.60 13.41 0.49 100.00
dm-54 0.00 0.00 1198.20 810.40 119670.40 81436.80 200.25 5.84 2.90 0.45 91.30
dm-51 0.00 0.00 1224.80 812.40 122211.20 79484.80 198.01 30.96 15.15 0.49 100.00
dm-57 0.00 0.00 1217.80 781.80 120067.20 79072.00 199.18 26.56 13.22 0.50 100.08
dm-58 0.00 0.00 1226.00 832.40 125072.00 82224.00 201.41 27.62 13.41 0.49 100.04
dm-39 0.00 0.00 1212.80 821.00 120816.00 81187.20 198.65 79.89 39.33 0.49 100.00
dm-30 0.00 0.00 1238.00 809.40 124556.80 79990.40 199.81 74.90 36.70 0.49 100.00
dm-38 0.00 0.00 1241.60 807.20 120816.00 79273.60 195.32 75.72 36.71 0.49 100.02
dm-41 0.00 0.00 1230.60 790.00 122665.60 78729.60 199.34 79.19 39.13 0.50 100.04
dm-37 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vdbench output(iorate:max):
Aug 05, 2016 interval i/o MB/sec bytes read resp read write resp resp queue cpu% cpu%
rate 1024**2 i/o pct time resp resp max stddev depth sys+u sys
15:41:27.052 31 19175.00 1849.69 101149 59.74 16.638 18.883 13.306 283.778 19.028 319.4 16.0 11.8
15:41:28.049 32 18735.00 1815.23 101596 59.71 16.948 19.097 13.764 436.044 21.122 319.4 16.2 11.9
15:41:29.049 33 19222.00 1828.67 99755 59.95 16.650 19.089 12.998 299.752 20.304 319.4 15.9 11.4
15:41:30.048 34 18698.00 1828.48 102540 60.02 17.078 19.325 13.705 373.329 21.460 319.4 15.5 11.4
15:41:31.048 35 19043.00 1839.98 101316 60.15 16.810 19.144 13.288 337.942 20.857 319.4 15.7 11.8
15:41:32.049 36 19014.00 1841.13 101533 59.80 16.694 19.295 12.824 553.030 22.040 319.5 15.4 11.6
15:41:33.048 37 18949.00 1816.27 100506 60.47 16.884 19.253 13.260 386.569 21.620 319.3 15.2 11.1
15:41:34.050 38 18969.00 1836.95 101543 60.19 16.909 19.570 12.886 545.720 22.704 319.8 15.5 11.6
15:41:35.049 39 18742.00 1826.52 102189 59.75 16.955 19.705 12.874 522.537 22.196 319.3 15.8 11.8
15:41:36.049 40 19370.00 1840.20 99617 59.32 16.427 19.433 12.043 399.247 23.059 319.4 16.2 12.1
15:41:37.049 41 18754.00 1831.94 102427 59.32 16.920 20.507 11.689 537.596 26.708 319.4 17.2 11.7
15:41:38.049 42 18730.00 1821.06 101949 60.32 17.216 20.443 12.312 556.103 26.041 319.5 15.4 11.5
15:41:39.050 43 18887.00 1835.30 101892 60.20 16.958 19.989 12.373 442.003 23.961 319.4 15.3 11.5
15:41:40.050 44 19725.00 1911.20 101599 60.26 16.754 20.220 11.497 535.593 27.025 332.1 17.0 11.7
15:41:41.049 45 17984.00 1746.77 101847 61.02 17.103 20.014 12.546 529.905 25.025 306.6 15.7 11.7
15:41:42.047 46 18799.00 1814.66 101218 60.68 16.950 19.825 12.515 516.635 24.999 319.4 20.8 12.0
15:41:43.048 47 19004.00 1828.38 100883 60.33 16.888 19.945 12.238 496.948 24.577 319.4 16.1 11.6
15:41:44.048 48 18755.00 1832.86 102473 59.66 16.963 20.078 12.354 523.144 24.948 319.4 16.3 11.8
15:41:45.047 49 18987.00 1839.98 101614 60.45 16.770 19.489 12.615 548.294 24.614 319.4 16.0 12.0
Iostat output on the host (with iorate:max)
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
dm-22 0.00 0.00 750.80 482.00 74035.20 47680.00 197.46 39.92 32.82 0.81 100.00
dm-23 0.00 0.00 639.60 446.60 63481.60 44524.80 198.87 39.94 37.24 0.92 100.00
dm-24 0.00 0.00 693.40 460.20 68019.20 44883.20 195.74 39.92 35.06 0.87 100.00
dm-25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-34 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-44 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-46 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-49 0.00 0.00 1680.40 1108.60 169443.20 109328.00 199.91 39.84 14.30 0.36 100.02
dm-50 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-53 0.00 0.00 3274.80 2181.20 322224.00 219974.40 198.75 39.68 7.27 0.18 100.02
dm-55 0.00 0.00 1768.40 1205.40 176374.40 121622.40 200.41 39.84 13.41 0.34 100.02
dm-56 0.00 0.00 1759.40 1183.00 174912.00 117827.20 198.98 39.83 13.54 0.34 100.02
dm-54 0.00 0.00 3275.60 2180.40 322220.80 219897.60 198.72 39.70 7.27 0.18 100.10
dm-51 0.00 0.00 1678.00 1111.00 168992.00 109603.20 199.78 39.83 14.29 0.36 100.02
dm-57 0.00 0.00 1766.40 1207.40 176243.20 121827.20 200.46 39.83 13.41 0.34 100.02
dm-58 0.00 0.00 1757.00 1185.20 174662.40 118230.40 199.10 39.83 13.54 0.34 100.02
dm-39 0.00 0.00 638.40 448.00 63158.40 44544.00 198.27 39.96 37.24 0.92 100.06
dm-30 0.00 0.00 732.60 480.40 72774.40 47699.20 198.64 39.93 33.26 0.82 100.02
dm-38 0.00 0.00 750.00 482.80 73984.00 47651.20 197.33 39.93 32.81 0.81 100.02
dm-41 0.00 0.00 693.20 460.20 68166.40 44960.00 196.16 39.92 35.06 0.87 100.02