Oracle Listener is not starting in IBM AIX 5.3 Clusterware Problem
Hi Guys,
I have Two AIX box on with two CPU and another with One CPU.
IBM installed HACMP Cluster in the AIX box (Active Passive cluster). When there is problem in the Production Server the cluster is moving perfectly to the failover, all the mount points are all perfectly moved. But Oracle Listener is not starting. I'm getting the below errors.
Both the servers are having 16G of RAM. the only difference is Production has Two CPU and Failover has One CPU. My oracle listener is not all starting what will be the possible cause.
IBM is telling that their part is over in this as all mount points are moved and the cluster is moved.
What will the possible cause for this... Whether we need to install Oracle Clusterware on both the AIX Box. We removed the HACMP cluster and tried to mount the mount points and tried starting the listener, on this point I'm able to start the listener successfully.
It is necessary to install the Oracle Clusterware??
Please Help me in this regard.
#
# su - oraprd
TSALFAIL:oraprd> enc
ksh: enc: not found.
TSALFAIL:oraprd> env
_=/usr/bin/env
LANG=en_US
THREAD=NOPS
LOGIN=oraprd
G_BROKEN_FILENAMES=1
PATH=/oracle/PRD/102_64/bin:/oracle/PRD:/usr/sap/PRD/SYS/exe/run:/usr/bin:/etc
usr/sbin:/usr/ucb:/oracle/PRD/bin:/usr/bin/X11:/sbin:.
NLS_LANG=AMERICAN_AMERICA.UTF8
ORACLE_BASE=/oracle
LC__FASTMSG=true
LOGNAME=oraprd
MAIL=/usr/spool/mail/oraprd
DSMI_CONFIG=/usr/tivoli/tsm/client/api/bin64/dsm.opt
dbs_ora_tnsname=PRD
ORACLE_SID=PRD
LOCPATH=/usr/lib/nls/loc
PS1=TSALFAIL:oraprd>
DB_SID=PRD
DIR_LIBRARY=/usr/sap/PRD/SYS/exe/run
USER=oraprd
AUTHSTATE=files
dbs_ora_schema=SAPSR3
SHELL=/usr/bin/ksh
ODMDIR=/etc/objrepos
TIMEOUT=600
DSMI_LOG=/usr/tivoli/tsm/client/api/bin64
SAPDATA_HOME=/oracle/PRD
TMOUT=600
HOME=/oracle/PRD
TERM=ansi
MAILMSG=[YOU HAVE NEW MAIL]
ORACLE_HOME=/oracle/PRD/102_64
PWD=/oracle/PRD
TZ=IST-05:30:00
dbms_type=ORA
SAPSYSTEMNAME=PRD
DSMI_DIR=/usr/tivoli/tsm/client/api/bin64
A__z=! LOGNAME="*TMOUT
NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lib/nls/msg/%L/%N.cat
LIBPATH=/usr/lib:/lib:/usr/sap/PRD/SYS/exe/run:/oracle/PRD/102_64/lib
TSALFAIL:oraprd>
TSALFAIL:oraprd> more listener.log
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production on 22-JUL-
2009 16:57:23
Copyright (c) 1991, 2005, Oracle. All rights reserved.
System parameter file is /oracle/PRD/102_64/network/admin/listener.ora
Log messages written to /oracle/PRD/102_64/network/log/listener.log
Trace information written to /oracle/PRD/102_64/network/trace/listener.trc
Trace level is currently 0
Started with pid=790662
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD.WORLD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.0.0.6)(PORT=1527)))
Listener completed notification to CRS on start
TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
22-JUL-2009 16:57:23 * 12531
TNS-12531: TNS:cannot allocate memory
22-JUL-2009 16:57:23 * 12531
TNS-12531: TNS:cannot allocate memory
22-JUL-2009 16:57:23 * 12531
TNS-12531: TNS:cannot allocate memory
22-JUL-2009 16:57:23 * 12531
TSALFAIL:oraprd>
TSALFAIL:oraprd>
TSALFAIL:oraprd> lsnrctl
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production on 22-JUL-
2009 17:04:14
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL>
LSNRCTL> status
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL> start
Starting /oracle/PRD/102_64/bin/tnslsnr: please wait...
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production
System parameter file is /oracle/PRD/102_64/network/admin/listener.ora
Log messages written to /oracle/PRD/102_64/network/log/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD.WORLD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.0.0.6)(PORT=1527)))
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
^C
TSALFAIL:oraprd>
TSALFAIL:oraprd> lsnrctl
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production on 22-JUL-
2009 17:06:25
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL> show trc_level
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL>
TSALFAIL:oraprd> lsnrctl
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production on 22-JUL-
2009 17:06:25
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL> show trc_level
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL>
LSNRCTL>
LSNRCTL> set trc_level ADMIN
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL>
LSNRCTL> start
Starting /oracle/PRD/102_64/bin/tnslsnr: please wait...
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production
System parameter file is /oracle/PRD/102_64/network/admin/listener.ora
Log messages written to /oracle/PRD/102_64/network/log/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD.WORLD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD)))
Error listening on: (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=TSALFAIL)(
PORT=1527))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist
IBM/AIX RISC System/6000 Error: 13: Permission denied
Listener failed to start. See the error message(s) above...
LSNRCTL> status
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL> start
Starting /oracle/PRD/102_64/bin/tnslsnr: please wait...
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production
System parameter file is /oracle/PRD/102_64/network/admin/listener.ora
Log messages written to /oracle/PRD/102_64/network/log/listener.log
Trace information written to /oracle/PRD/102_64/network/trace/listener.trc
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD.WORLD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD)))
Error listening on: (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=TSALPRD)(P
ORT=1527))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist
IBM/AIX RISC System/6000 Error: 13: Permission denied
Listener failed to start. See the error message(s) above...
LSNRCTL>
LSNRCTL> stop
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL> start
Starting /oracle/PRD/102_64/bin/tnslsnr: please wait...
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.2.0 - Production
System parameter file is /oracle/PRD/102_64/network/admin/listener.ora
Log messages written to /oracle/PRD/102_64/network/log/listener.log
Trace information written to /oracle/PRD/102_64/network/trace/listener.trc
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD.WORLD)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=PRD)))
Error listening on: (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=TSALPRD)(P
ORT=1527))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist
IBM/AIX RISC System/6000 Error: 13: Permission denied
Listener failed to start. See the error message(s) above...
LSNRCTL> show trc_level
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD.WORLD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(PROTOCOL=IPC)(KEY=PRD))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 2: No such file or directory
Connecting to (ADDRESS=(COMMUNITY=SAP.WORLD)(PROTOCOL=TCP)(HOST=10.0.0.6)(PORT=1
527))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
IBM/AIX RISC System/6000 Error: 79: Connection refused
LSNRCTL>
Regards,
Vijayaraghavan K