Hello, experts
Recently I've deployed successfully Oracle RAC 18c for Linux x86_64 on my VirtualBox 5.2.18 on Oracle Linux 7.5 (only test environment).
At that time I used ASMFD to initialize those shared disks (not using multipathd to bind those disks),
[root@adccrac1 ~]# export ORACLE_HOME=/u01/app/18.0.0/grid
[root@adccrac1 ~]# export PATH=$PATH:$ORACLE_HOME/bin
[root@adccrac1 ~]# export ORACLE_BASE=/tmp
[root@adccrac1 ~]# /u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG1 /dev/sdb --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG2 /dev/sdc --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG3 /dev/sdd --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label MGMT /dev/sde --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label DATA1 /dev/sdf --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label DATA2 /dev/sdg --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label ARCH /dev/sdh --init
/u01/app/18.0.0/grid/bin/asmcmd afd_label FTRA /dev/sdi –-init
[root@adccrac1 ~]# cd /dev/oracleafd/disks/
[root@adccrac1 disks]# ls -lrht
total 32K
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 CRSDG3
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 CRSDG2
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 CRSDG1
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 ARCH
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 MGMT
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 FTRA
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 DATA2
-rw-rw-r-- 1 grid oinstall 9 Sep 7 10:07 DATA1
Afterwards I get ready to add another disk "sdj" to use multipathd to bind it, the following are some steps,
(1) halt two nodes RAC,
(2) using "VBoxManage" to create another disk and changing it to be shareable and providing it to two nodes,
# VBoxManage createhd -filename /root/"VirtualBox VMs"/asm_disks/data3.vdi -size 10240 -format VDI -variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: fc7d2707-616e-49e0-bcff-24540ffffaf1
# VBoxManage modifyhd /root/"VirtualBox VMs"/asm_disks/data3.vdi --type shareable
#
# VBoxManage storageattach adccrac1 --storagectl "SATA" --port 12 --device 0 --type hdd --medium /root/"VirtualBox VMs"/asm_disks/data3.vdi --mtype shareable
#
# VBoxManage storageattach adccrac2 --storagectl "SATA" --port 12 --device 0 --type hdd --medium /root/"VirtualBox VMs"/asm_disks/data3.vdi --mtype shareable
#
(3) startup two nodes RAC,
(4) configure multipathd service,
# ssh adccrac1
root@adccrac1's password:
Last login: Thu Sep 6 16:52:16 2018 from gateway
[root@adccrac1 ~]# rpm -qa | grep device-mapper
device-mapper-event-libs-1.02.146-4.0.1.el7.x86_64
device-mapper-multipath-libs-0.4.9-119.el7.x86_64
device-mapper-1.02.146-4.0.1.el7.x86_64
device-mapper-libs-1.02.146-4.0.1.el7.x86_64
device-mapper-event-1.02.146-4.0.1.el7.x86_64
device-mapper-multipath-0.4.9-119.el7.x86_64
device-mapper-persistent-data-0.7.3-3.el7.x86_64
[root@adccrac1 ~]# systemctl status multipathd
● multipathd.service - Device-Mapper Multipath Device Controller
Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
Active: inactive (dead)
Condition: start condition failed at Thu 2018-09-06 17:11:58 CST; 15h ago
ConditionPathExists=/etc/multipath.conf was not met
[root@adccrac1 ~]# ls -lrht /etc/multipath.conf
ls: cannot access /etc/multipath.conf: No such file or directory
[root@adccrac1 ~]# mpathconf --help
usage: /usr/sbin/mpathconf <command>
Commands:
Enable: --enable
Disable: --disable
Only allow certain wwids (instead of enable): --allow <WWID>
Set user_friendly_names (Default y): --user_friendly_names <y|n>
Set find_multipaths (Default y): --find_multipaths <y|n>
Load the dm-multipath modules on enable (Default y): --with_module <y|n>
start/stop/reload multipathd (Default n): --with_multipathd <y|n>
select output file (Default /etc/multipath.conf): --outfile <FILE>
[root@adccrac1 ~]# mpathconf --enable
[root@adccrac1 ~]#
[root@adccrac1 ~]# ls -lrht /etc/multipath
multipath/ multipath.conf
[root@adccrac1 ~]# ls -lrht /etc/multipath.conf
-rw------- 1 root root 2.4K Sep 7 08:55 /etc/multipath.conf
[root@adccrac1 ~]# /usr/lib/udev/scsi_id -gud /dev/sdj
1ATA_VBOX_HARDDISK_VBfc7d2707-f1faff0f
[root@adccrac1 ~]# vi /etc/multipath.conf
multipaths {
multipath {
wwid 1ATA\_VBOX\_HARDDISK\_VBfc7d2707-f1faff0f
alias mpathdata
}
}
[root@adccrac1 ~]# systemctl start multipathd.service
[root@adccrac1 ~]# systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-09-07 09:58:15 CST; 2s ago
Process: 10670 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)
Process: 10666 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)
Process: 10664 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS)
Main PID: 10672 (multipathd)
CGroup: /system.slice/multipathd.service
└─10672 /sbin/multipathd
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio4: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio5: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio6: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio7: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio8: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio9: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vmb: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: ofsctl: HDIO_GETGEO failed with 25
Sep 07 09:58:15 adccrac1 multipathd[10672]: oracleafd/admin: HDIO_GETGEO failed with 25
Sep 07 09:58:16 adccrac1 multipathd[10672]: path checkers start up
At this very moment as you can see above it seems to be some errors, next to I observe the directory "/dev/mapper",
[root@adccrac1 ~]# cd /dev/mapper/
[root@adccrac1 mapper]# ls -lrht
total 0
crw------- 1 root root 10, 236 Sep 7 10:07 control
There is no disk on "/dev/mapper", I re-scan disks via this command "multipath -v2",
[root@adccrac1 ~]# multipath -v2
Sep 07 10:25:55 | asm/.asm_ctl_spec: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg0: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg1: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg2: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg3: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg4: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg5: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg6: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg7: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vbg8: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vdbg: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio0: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio1: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio10: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio11: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio12: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio13: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio14: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio15: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio16: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio17: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio18: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio19: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio2: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio20: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio21: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio22: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio23: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio24: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio25: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio26: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio27: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio28: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio29: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio3: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio30: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio31: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio4: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio5: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio6: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio7: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio8: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vio9: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | asm/.asm_ctl_vmb: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | ofsctl: HDIO_GETGEO failed with 25
Sep 07 10:25:55 | oracleafd/admin: HDIO_GETGEO failed with 25
Could you help me for troubleshooting this weird issue?
Thanks in advance.
Best Regards
Quanwen Zhao