Hello experts,
I am trying to troubleshoot a communication issue between oracle grid OMS and an oracle agent.
Based on my initial investigation, I have arrived at the conclusion that the problem is at OS/networking level and not with OMS or grid agent software (hence posting in linux forum...)
The original problem is that the grid agent can not communicate with OMS
[oracle@dbserver ~]$ emctl pingOMS
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
pingOMS failed. Time out while reading the response.
At OS level, ping (from agent machine to OMS machine) works:
[oracle@dbserver ~]$ ping -c 3 omsserver
PING omsserver (aa.b.cc.dd) 56(84) bytes of data.
64 bytes from omsserver (aa.b.cc.dd): icmp_seq=1 ttl=57 time=3.79 ms
64 bytes from omsserver (aa.b.cc.dd): icmp_seq=2 ttl=57 time=3.65 ms
64 bytes from omsserver (aa.b.cc.dd): icmp_seq=3 ttl=57 time=3.69 ms
--- omsserver ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 3.652/3.715/3.796/0.092 ms
But curl does not work:
[oracle@dbserver ~]$ curl -ik -m 10 -v https://omsserver:1159/em/upload
* About to connect() to omsserver port 1159
* Trying aa.b.cc.dd... Timeout
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host
There is no firewall on OMS machine to block incoming connections:
[root@omsserver ~]# iptables -L -n -v --line-numbers
Chain INPUT (policy ACCEPT 614K packets, 638M bytes)
num pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 640K packets, 614M bytes)
num pkts bytes target prot opt in out source destination
[root@dbserver ~]# iptables -L -n -v --line-numbers
Chain INPUT (policy ACCEPT 614K packets, 638M bytes)
num pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 640K packets, 614M bytes)
num pkts bytes target prot opt in out source destination
The OMS server machine and grid agent machine are on different subnets.
My knowledge in linux and networking is quite limited but I have been told (by a linux admin) that it might need an explicit static route entry added to OMS server machine
in order for agent machine to be able to talk to OMS server machine (it was done in the past and worked...).
Both OMS and agent machines have RHEL 5.5 and kernel is 2.6.18-194.el5
My questions are:
1) Why would a ping work but other programs (like emctl or curl) won't work?
2) What can be the reason that communication from OMS machine to grid agent machine works but not the other way (even when there is no firewall blocking on either machines)?
3) Is it possible that routing affects the communication? If yes, how?
Thanks in advance.