Skip to Main Content
Forums

Infrastructure Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Add Server to Server Pool failed

958350Aug 28 2012 — edited Sep 4 2012
Hi, All!
When you try to add a server to the pool, is an error ... Tell me what could be wrong?

Hardware:
Server: IBM x3550m3 - OVMSERVER1 ip 192.168.1.247, 10.20.2.1
Server: IBM x3550m3 - OVMSERVER2 ip 192.168.1.248, 10.20.2.2
Storage: IBM DS3524 - SAS link
Storage disk:
HS 1.5Gb
HC 2.5Gb
OVMPOOL 100Mb
Sofrware:
Oracle VM 3.1.1

OVMPOOL mount as /opt on OVMSERVER1 and mount as nfs-partitions
mount:
/dev/sdd3 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
*/dev/sdc1 on /opt type ext3 (rw)*
/dev/sdd1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
xenfs on /proc/xen type xenfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
configfs on /sys/kernel/config type configfs (rw)
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
none on /var/lib/xenstored type tmpfs (rw)
*192.168.1.247:/opt/OVMS on /nfsmnt/b40bd57f-1ab1-4d03-9254-0151b6f635f7 type nfs (rw,addr=192.168.1.247)*
/dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b type ocfs2 (rw,_netdev,heartbeat=global)
/dev/mapper/360080e50002d797a0000051e501bc836 on /OVS/Repositories/0004fb00000300008b9920aae151469d type ocfs2 (rw,heartbeat=none)
/dev/mapper/360080e50002d797a00000523501bc85c on /OVS/Repositories/0004fb00000300007c5a44a941e15d32 type ocfs2 (rw,heartbeat=none)

exportfs:
/opt/OVMS 192.168.1.247(rw,sync,no_root_squash,no_subtree_check) 192.168.1.248(rw,sync,no_root_squash,no_subtree_check) 10.20.2.1(rw,sync,no_root_squash,no_subtree_check) 10.20.2.2(rw,sync,no_root_squash,no_subtree_check)

dmesg OVMSERVER2
Aug 28 08:18:37 OVMSERVER2 twisted: [-] Log opened.
Aug 28 08:18:37 OVMSERVER2 twisted: [-] twistd 8.2.0 (/usr/bin/python 2.4.3) starting up.
Aug 28 08:18:37 OVMSERVER2 twisted: [-] reactor class: twisted.internet.selectreactor.SelectReactor.
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor] Rescanning all plugins
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.xen_plugin] Starting plugin process /usr/lib/python2.4/site-packages/monitor/plugins/xen_plugin.py
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.xen_plugin] Plugin process /usr/lib/python2.4/site-packages/monitor/plugins/xen_plugin.py launched, PID 5181
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.xen_plugin] Process 5181 started, launching watchdog with gracetime 3600
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.oel] Starting plugin process /usr/lib/python2.4/site-packages/monitor/plugins/oel.py
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.oel] Plugin process /usr/lib/python2.4/site-packages/monitor/plugins/oel.py launched, PID 5182
Aug 28 08:18:37 OVMSERVER2 twisted: [monitor.plugin.oel] Process 5182 started, launching watchdog with gracetime 3600
Aug 28 08:18:38 OVMSERVER2 kernel: FS-Cache: Loaded
Aug 28 08:18:38 OVMSERVER2 kernel: FS-Cache: Netfs 'nfs' registered for caching
Aug 28 08:18:38 OVMSERVER2 kernel: device-mapper: nfs: version 1.0.0 loaded
Aug 28 08:18:38 OVMSERVER2 multipathd: dm-4: add map (uevent)
Aug 28 08:18:39 OVMSERVER2 kernel: OCFS2 DLM 1.8.0
Aug 28 08:18:39 OVMSERVER2 kernel: ocfs2: Registered cluster interface o2cb
Aug 28 08:18:39 OVMSERVER2 kernel: OCFS2 DLMFS 1.8.0
Aug 28 08:18:39 OVMSERVER2 kernel: OCFS2 User DLM kernel interface loaded
Aug 28 08:18:39 OVMSERVER2 o2cb.init: online a2b7d59e374df2ba
Aug 28 08:18:39 OVMSERVER2 kernel: o2hb: Heartbeat mode set to global
Aug 28 08:18:43 OVMSERVER2 kernel: o2net: Connection to node OVMSERVER1 (num 0) at 10.20.2.1:7777 shutdown, state 7
Aug 28 08:18:51 OVMSERVER2 last message repeated 4 times
Aug 28 08:18:51 OVMSERVER2 kernel: o2hb: Heartbeat started on region 0004FB00000500006352ACB0D04C104B (dm-4)
Aug 28 08:18:51 OVMSERVER2 o2hbmonitor: Starting
Aug 28 08:18:51 OVMSERVER2 kernel: o2cb: This node is not connected to nodes: 0.
Aug 28 08:18:51 OVMSERVER2 kernel: o2cb: Cluster check failed. Fix errors before retrying.
Aug 28 08:18:51 OVMSERVER2 kernel: (mount.ocfs2,5633,5):ocfs2_dlm_init:3001 ERROR: status = -22
Aug 28 08:18:51 OVMSERVER2 kernel: (mount.ocfs2,5633,5):ocfs2_mount_volume:1883 ERROR: status = -22
Aug 28 08:18:51 OVMSERVER2 kernel: ocfs2: Unmounting device (252,4) on (node 0)
Aug 28 08:18:51 OVMSERVER2 kernel: (mount.ocfs2,5633,5):ocfs2_fill_super:1240 ERROR: status = -22
Aug 28 08:18:53 OVMSERVER2 kernel: o2net: Connection to node OVMSERVER1 (num 0) at 10.20.2.1:7777 shutdown, state 7
Aug 28 08:18:53 OVMSERVER2 kernel: o2hb: Region 0004FB00000500006352ACB0D04C104B (dm-4) is now a quorum device
Aug 28 08:18:55 OVMSERVER2 kernel: o2net: Connection to node OVMSERVER1 (num 0) at 10.20.2.1:7777 shutdown, state 7
Aug 28 08:18:56 OVMSERVER2 kernel: o2cb: This node is not connected to nodes: 0.
Aug 28 08:18:56 OVMSERVER2 kernel: o2cb: Cluster check failed. Fix errors before retrying.
Aug 28 08:18:56 OVMSERVER2 kernel: (mount.ocfs2,5710,9):ocfs2_dlm_init:3001 ERROR: status = -22
Aug 28 08:18:56 OVMSERVER2 kernel: (mount.ocfs2,5710,9):ocfs2_mount_volume:1883 ERROR: status = -22
Aug 28 08:18:56 OVMSERVER2 kernel: ocfs2: Unmounting device (252,4) on (node 0)
Aug 28 08:18:56 OVMSERVER2 kernel: (mount.ocfs2,5710,9):ocfs2_fill_super:1240 ERROR: status = -22
Aug 28 08:18:57 OVMSERVER2 kernel: o2net: Connection to node OVMSERVER1 (num 0) at 10.20.2.1:7777 shutdown, state 7
Aug 28 08:19:01 OVMSERVER2 last message repeated 2 times
Aug 28 08:19:01 OVMSERVER2 kernel: o2cb: This node is not connected to nodes: 0.
Aug 28 08:19:01 OVMSERVER2 kernel: o2cb: Cluster check failed. Fix errors before retrying.
Aug 28 08:19:01 OVMSERVER2 kernel: (mount.ocfs2,5808,5):ocfs2_dlm_init:3001 ERROR: status = -22
Aug 28 08:19:01 OVMSERVER2 kernel: (mount.ocfs2,5808,5):ocfs2_mount_volume:1883 ERROR: status = -22
Aug 28 08:19:01 OVMSERVER2 kernel: ocfs2: Unmounting device (252,4) on (node 0)
Aug 28 08:19:01 OVMSERVER2 kernel: (mount.ocfs2,5808,5):ocfs2_fill_super:1240 ERROR: status = -22
Aug 28 08:19:02 OVMSERVER2 o2cb.init: offline ocfs2 0
Aug 28 08:19:03 OVMSERVER2 kernel: o2net: Connection to node OVMSERVER1 (num 0) at 10.20.2.1:7777 shutdown, state 7
Aug 28 08:19:35 OVMSERVER2 last message repeated 16 times
Aug 28 08:19:41 OVMSERVER2 last message repeated 3 times
Aug 28 08:19:43 OVMSERVER2 kernel: o2net: No connection established with node 0 after 60.0 seconds, giving up.
Aug 28 08:20:43 OVMSERVER2 kernel: o2net: No connection established with node 0 after 60.0 seconds, giving up.

Oracle VM Manager log:

Job Construction Phase
----------------------
begin()
Appended operation 'Server Role Update' to object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'.
Appended operation 'Server Join Server Pool' to object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'.
Appended operation 'Server Pool Member Update' to object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'.
Appended operation 'Server Cluster Configuration Update' to object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'.
Appended operation 'Server Cluster Configure' to object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'.
Appended operation 'Server Cluster Join' to object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'.
commit()
Completed Step: COMMIT

Objects and Operations
----------------------
Object (IN_USE): [ServerPool] 0004fb0000020000a2b7d59e374df2ba (OVMPOOL)
Operation: Server Pool Member Update
Object (IN_USE): [Server] ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)
Operation: Server Role Update
Operation: Server Join Server Pool
Operation: Server Cluster Configure
Operation: Server Cluster Join
Object (IN_USE): [Server] fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)
Operation: Server Cluster Configuration Update
Object (IN_USE): [Cluster] a2b7d59e374df2ba

Job Running Phase at 16:08 on Tue, Aug 28, 2012
----------------------------------------------
Job Participants: []


Actioner
--------
Starting operation 'Server Pool Member Update' on object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'
Completed operation 'Server Pool Member Update' completed with direction ==> LATER
Starting operation 'Server Role Update' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Role Update' completed with direction ==> DONE
Starting operation 'Server Join Server Pool' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Join Server Pool' completed with direction ==> LATER
Starting operation 'Server Cluster Configuration Update' on object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'
Completed operation 'Server Cluster Configuration Update' completed with direction ==> LATER
Starting operation 'Server Pool Member Update' on object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'
Completed operation 'Server Pool Member Update' completed with direction ==> DONE
Starting operation 'Server Cluster Configure' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Cluster Configure' completed with direction ==> LATER
Starting operation 'Server Cluster Configuration Update' on object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'
Completed operation 'Server Cluster Configuration Update' completed with direction ==> LATER
Starting operation 'Server Cluster Join' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Cluster Join' completed with direction ==> LATER
Starting operation 'Server Cluster Configuration Update' on object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'
Completed operation 'Server Cluster Configuration Update' completed with direction ==> LATER
Starting operation 'Server Join Server Pool' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Join Server Pool' completed with direction ==> DONE
Starting operation 'Server Cluster Configure' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Cluster Configure' completed with direction ==> LATER
Starting operation 'Server Cluster Configuration Update' on object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'
Completed operation 'Server Cluster Configuration Update' completed with direction ==> DONE
Starting operation 'Server Cluster Join' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Completed operation 'Server Cluster Join' completed with direction ==> LATER
Starting operation 'Server Cluster Configure' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: OVMSERVER2 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.1.248:8899/api/2 configure_server_for_cluster nfs 192.168.1.247:/opt/OVMS 0004fb00000500006352acb0d04c104b b40bd57f-1ab1-4d03-9254-0151b6f635f7, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['mount', '/dev/mapper/ovspoolfs', '/poolfsmnt/0004fb00000500006352acb0d04c104b'] failed (1): stderr: mount.ocfs2: Invalid argument while mounting /dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b. Check 'dmesg' for more information on this error.
stdout:
Tue Aug 28 08:32:23 MSK 2012
Tue Aug 28 08:32:23 MSK 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.ClusterAction.configureServerForCluster(ClusterAction.java:88)
at com.oracle.ovm.mgr.op.physical.ServerClusterConfigure.configureCluster(ServerClusterConfigure.java:139)
at com.oracle.ovm.mgr.op.physical.ServerClusterConfigure.action(ServerClusterConfigure.java:58)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.1.248:8899/api/2 configure_server_for_cluster nfs 192.168.1.247:/opt/OVMS 0004fb00000500006352acb0d04c104b b40bd57f-1ab1-4d03-9254-0151b6f635f7, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['mount', '/dev/mapper/ovspoolfs', '/poolfsmnt/0004fb00000500006352acb0d04c104b'] failed (1): stderr: mount.ocfs2: Invalid argument while mounting /dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b. Check 'dmesg' for more information on this error.
stdout:
Tue Aug 28 08:32:23 MSK 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
... 30 more


FailedOperationCleanup
----------
Starting failed operation 'Server Cluster Configure' cleanup on object 'OVMSERVER2'
Complete rollback operation 'Server Cluster Configure' completed with direction=OVMSERVER2

Rollbacker
----------
Executing rollback operation 'Server Pool Member Update' on object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'
Complete rollback operation 'Server Pool Member Update' completed with direction=LATER
Executing rollback operation 'Server Cluster Configure' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Complete rollback operation 'Server Cluster Configure' completed with direction=DONE
Executing rollback operation 'Server Join Server Pool' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Complete rollback operation 'Server Join Server Pool' completed with direction=LATER
Executing rollback operation 'Server Cluster Configuration Update' on object 'fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)'
Complete rollback operation 'Server Cluster Configuration Update' completed with direction=DONE
Executing rollback operation 'Server Pool Member Update' on object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'
Complete rollback operation 'Server Pool Member Update' completed with direction=LATER
Executing rollback operation 'Server Join Server Pool' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Complete rollback operation 'Server Join Server Pool' completed with direction=DONE
Executing rollback operation 'Server Role Update' on object 'ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)'
Complete rollback operation 'Server Role Update' completed with direction=DONE
Executing rollback operation 'Server Pool Member Update' on object '0004fb0000020000a2b7d59e374df2ba (OVMPOOL)'
Complete rollback operation 'Server Pool Member Update' completed with direction=DONE

Objects To Be Rolled Back
-------------------------
Object (IN_USE): [ServerPool] 0004fb0000020000a2b7d59e374df2ba (OVMPOOL)
Object (IN_USE): [Server] ba:16:46:6e:ed:7c:3d:d7:9c:76:35:78:12:36:4f:79 (OVMSERVER2)
Object (IN_USE): [Server] fa:2d:de:e2:ee:af:3b:1b:a3:cc:ab:74:8a:32:c7:ef (OVMSERVER2)
Object (IN_USE): [Cluster] a2b7d59e374df2ba


Write Methods Invoked
-------------------
Class=InternalJobDbImpl vessel_id=12038 method=addTransactionIdentifier accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=addServer accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=lock accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=addServerRole accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=addServerRole accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=addServerRole accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=addServerInternal accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setServerPool accessLevel=6
Class=ClusterDbImpl vessel_id=499 method=allocateSlotForServer accessLevel=6
Class=ClusterDbImpl vessel_id=499 method=addServer accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=lock accessLevel=6
Class=ServerDbImpl vessel_id=420 method=lock accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setAsset accessLevel=6
Class=ServerDbImpl vessel_id=420 method=reconfigureCluster accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCluster accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setAsset accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setAsset accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setCompletedStep accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setAssociatedHandles accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentJobOperationComplete accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=420 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=setCurrentJobOperationComplete accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=420 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=420 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentJobOperationComplete accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setProgressMessage accessLevel=6
Class=ServerDbImpl vessel_id=420 method=setCurrentJobOperationComplete accessLevel=6
Class=ServerDbImpl vessel_id=420 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setFailedOperation accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=nextJobOperation accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=ServerDbImpl vessel_id=420 method=nextJobOperation accessLevel=6
Class=ClusterDbImpl vessel_id=499 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=420 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=InternalJobDbImpl vessel_id=12038 method=setTuringMachineFlag accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=ServerDbImpl vessel_id=9544 method=nextJobOperation accessLevel=6
Class=ServerPoolDbImpl vessel_id=493 method=nextJobOperation accessLevel=6
Completed Step: ROLLBACK
Job failed commit (internal) due to OVMAPI_4010E Attempt to send command: dispatch to server: OVMSERVER2 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.1.248:8899/api/2 configure_server_for_cluster nfs 192.168.1.247:/opt/OVMS 0004fb00000500006352acb0d04c104b b40bd57f-1ab1-4d03-9254-0151b6f635f7, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['mount', '/dev/mapper/ovspoolfs', '/poolfsmnt/0004fb00000500006352acb0d04c104b'] failed (1): stderr: mount.ocfs2: Invalid argument while mounting /dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b. Check 'dmesg' for more information on this error.
stdout:
Tue Aug 28 08:32:23 MSK 2012
Tue Aug 28 08:32:23 MSK 2012
com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: OVMSERVER2 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.1.248:8899/api/2 configure_server_for_cluster nfs 192.168.1.247:/opt/OVMS 0004fb00000500006352acb0d04c104b b40bd57f-1ab1-4d03-9254-0151b6f635f7, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['mount', '/dev/mapper/ovspoolfs', '/poolfsmnt/0004fb00000500006352acb0d04c104b'] failed (1): stderr: mount.ocfs2: Invalid argument while mounting /dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b. Check 'dmesg' for more information on this error.
stdout:
Tue Aug 28 08:32:23 MSK 2012
Tue Aug 28 08:32:23 MSK 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.ClusterAction.configureServerForCluster(ClusterAction.java:88)
at com.oracle.ovm.mgr.op.physical.ServerClusterConfigure.configureCluster(ServerClusterConfigure.java:139)
at com.oracle.ovm.mgr.op.physical.ServerClusterConfigure.action(ServerClusterConfigure.java:58)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
at sun.reflect.GeneratedMethodAccessor408.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.1.248:8899/api/2 configure_server_for_cluster nfs 192.168.1.247:/opt/OVMS 0004fb00000500006352acb0d04c104b b40bd57f-1ab1-4d03-9254-0151b6f635f7, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['mount', '/dev/mapper/ovspoolfs', '/poolfsmnt/0004fb00000500006352acb0d04c104b'] failed (1): stderr: mount.ocfs2: Invalid argument while mounting /dev/mapper/ovspoolfs on /poolfsmnt/0004fb00000500006352acb0d04c104b. Check 'dmesg' for more information on this error.
stdout:
Tue Aug 28 08:32:23 MSK 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
... 30 more


----------
End of Job
----------
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Oct 2 2012
Added on Aug 28 2012
#oracle-virtualization, #oracle-vm-server-for-x86, #server
7 comments
1,687 views