Oracle Database Discussions

Hi guys,

I am having a strange problem with the my production server.

suddenly my production database is hanging out. I'm pasted the alert log file from the production database below. Kindly help me in this regard.

Each and every time I'm restarting the Production DB.

Log files showing TNS Operation Timed Out. But my network connectivity is good.

Alert Log File

Thread 1 advanced to log sequence 15365
Current log# 5 seq# 15365 mem# 0: G:\ORACLE9I\LOGFILE\REDO05.LOG
Sun Oct 05 13:36:39 2008
Thread 1 advanced to log sequence 15366
Current log# 4 seq# 15366 mem# 0: G:\ORACLE9I\LOGFILE\REDO04.LOG
Sun Oct 05 14:48:38 2008
Thread 1 advanced to log sequence 15367
Current log# 1 seq# 15367 mem# 0: G:\ORACLE9I\LOGFILE\REDO01.LOG
Sun Oct 05 15:30:44 2008
Thread 1 advanced to log sequence 15368
Current log# 2 seq# 15368 mem# 0: G:\ORACLE9I\LOGFILE\REDO02.LOG
Sun Oct 05 15:37:35 2008
Thread 1 cannot allocate new log, sequence 15369
All online logs needed archiving
Current log# 2 seq# 15368 mem# 0: G:\ORACLE9I\LOGFILE\REDO02.LOG
Dump file g:\oracle9i\bdump\alert_jsl.log
Sun Oct 05 15:55:59 2008
ORACLE V9.2.0.4.0 - Production vsnsta=0
vsnsql=12 vsnxtr=3
Windows 2000 Version 5.2 Service Pack 1, CPU type 586
Sun Oct 05 15:55:59 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
SCN scheme 2
Using log_archive_dest parameter default value
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 9.2.0.4.0.
System parameters with non-default values:
processes = 200
sessions = 225
timed_statistics = TRUE
shared_pool_size = 629145600
sga_max_size = 1259415684
large_pool_size = 25165824
java_pool_size = 25165824
control_files = E:\oracle9i\controlfile\control01.ctl, E:\oracle9i\controlfile\control02.ctl, F:\oracle9i\controlfile\control03.ctl
db_block_size = 8192
db_cache_size = 528482304
compatible = 9.2.0.4.0
remote_archive_enable = TRUE
log_archive_start = TRUE
log_archive_dest_1 = LOCATION=H:\oracle9i\archive\mandatory
log_archive_dest_2 = SERVICE=JSLSTBY
log_archive_dest_state_2 = ENABLE
standby_archive_dest = H:\oracle9i\archive
log_archive_format = jsl_%T_%S.arc
log_buffer = 524288
db_file_multiblock_read_count= 16
standby_file_management = AUTO
fast_start_mttr_target = 1200
undo_management = AUTO
undo_tablespace = UNDOTBS1
undo_retention = 10800
remote_login_passwordfile= EXCLUSIVE
db_domain =
instance_name = jsl
job_queue_processes = 4
background_dump_dest = G:\oracle9i\bdump
user_dump_dest = G:\oracle9i\udump
core_dump_dest = G:\oracle9i\cdump
sort_area_size = 524288
db_name = jsl
open_cursors = 600
star_transformation_enabled= FALSE
query_rewrite_enabled = FALSE
pga_aggregate_target = 419430400
aq_tm_processes = 1
PMON started with pid=2
LGWR started with pid=4
CKPT started with pid=5
SMON started with pid=6
DBW0 started with pid=3
RECO started with pid=7
CJQ0 started with pid=8
QMN0 started with pid=9
Sun Oct 05 15:56:11 2008
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=10
ARC0: Archival started
Sun Oct 05 15:56:12 2008
ARCH: STARTING ARCH PROCESSES COMPLETE
Sun Oct 05 15:56:12 2008
ARC0: Thread not mounted
ARC1 started with pid=11
Sun Oct 05 15:56:13 2008
ARC1: Archival started
ARC1: Thread not mounted
Sun Oct 05 15:56:14 2008
alter database mount exclusive
Sun Oct 05 15:56:20 2008
Successful mount of redo thread 1, with mount id 1986804254.
Sun Oct 05 15:56:20 2008
Database mounted in Exclusive Mode.
Completed: alter database mount exclusive
Sun Oct 05 15:56:20 2008
alter database open
Sun Oct 05 15:56:20 2008
Beginning crash recovery of 1 threads
Sun Oct 05 15:56:21 2008
Started first pass scan
Sun Oct 05 15:56:21 2008
Completed first pass scan
18355 redo blocks read, 1274 data blocks need recovery
Sun Oct 05 15:56:21 2008
Started recovery at
Thread 1: logseq 15368, block 2124, scn 0.0
Recovery of Online Redo Log: Thread 1 Group 2 Seq 15368 Reading mem 0
Mem# 0 errs 0: G:\ORACLE9I\LOGFILE\REDO02.LOG
Sun Oct 05 15:56:27 2008
Completed redo application
Sun Oct 05 15:56:27 2008
Ended recovery at
Thread 1: logseq 15368, block 20479, scn 0.663631261
1274 data blocks read, 1212 data blocks written, 18355 redo blocks read
Crash recovery completed successfully
Sun Oct 05 15:56:27 2008
LGWR: Primary database is in CLUSTER CONSISTENT mode
Sun Oct 05 15:56:27 2008
ARCH: Evaluating archive log 3 thread 1 sequence 15364
ARCH: Beginning to archive log 3 thread 1 sequence 15364
Creating archive destination LOG_ARCHIVE_DEST_2: 'JSLSTBY'
ARCH: Error 12535 Creating archive log file to 'JSLSTBY'
Sun Oct 05 15:56:48 2008
Errors in file g:\oracle9i\udump\jsl_ora_2260.trc:
ORA-12535: TNS:operation timed out

Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15364.ARC'
ARCH: Completed archiving log 3 thread 1 sequence 15364
Sun Oct 05 15:56:51 2008
LGWR: Primary database is in CLUSTER CONSISTENT mode
Thread 1 advanced to log sequence 15369
Thread 1 opened at log sequence 15369
Current log# 3 seq# 15369 mem# 0: G:\ORACLE9I\LOGFILE\REDO03.LOG
Successful open of redo thread 1.
Sun Oct 05 15:56:52 2008
SMON: enabling cache recovery
Sun Oct 05 15:56:52 2008
ARC0: Evaluating archive log 5 thread 1 sequence 15365
Sun Oct 05 15:56:53 2008
Undo Segment 1 Onlined
Undo Segment 2 Onlined
Undo Segment 3 Onlined
Undo Segment 4 Onlined
Undo Segment 5 Onlined
Undo Segment 6 Onlined
Undo Segment 7 Onlined
Undo Segment 8 Onlined
Undo Segment 9 Onlined
Sun Oct 05 15:56:53 2008
Undo Segment 10 Onlined
Successfully onlined Undo Tablespace 1.
Sun Oct 05 15:56:53 2008
SMON: enabling tx recovery
Sun Oct 05 15:56:53 2008
Database Characterset is WE8ISO8859P1
Sun Oct 05 15:56:54 2008
ARC0: Beginning to archive log 5 thread 1 sequence 15365
Sun Oct 05 15:56:54 2008
Creating archive destination LOG_ARCHIVE_DEST_2: 'JSLSTBY'
Sun Oct 05 15:56:55 2008
replication_dependency_tracking turned off (no async multimaster replication found)
Completed: alter database open
Sun Oct 05 15:57:11 2008
Restarting dead background process QMN0
QMN0 started with pid=9
Sun Oct 05 15:57:15 2008
ARC0: Error 12535 Creating archive log file to 'JSLSTBY'
Sun Oct 05 15:57:15 2008
Errors in file g:\oracle9i\bdump\jsl_arc0_3096.trc:
ORA-12535: TNS:operation timed out

Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15365.ARC'
ARC0: Completed archiving log 5 thread 1 sequence 15365
Sun Oct 05 15:57:15 2008
ARC1: Evaluating archive log 4 thread 1 sequence 15366
ARC1: Beginning to archive log 4 thread 1 sequence 15366
Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15366.ARC'
ARC1: Completed archiving log 4 thread 1 sequence 15366
Sun Oct 05 15:57:17 2008
ARC0: Evaluating archive log 1 thread 1 sequence 15367
ARC0: Beginning to archive log 1 thread 1 sequence 15367
Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15367.ARC'
ARC0: Completed archiving log 1 thread 1 sequence 15367
Sun Oct 05 15:57:17 2008
ARC1: Evaluating archive log 2 thread 1 sequence 15368
ARC1: Beginning to archive log 2 thread 1 sequence 15368
Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15368.ARC'
ARC1: Completed archiving log 2 thread 1 sequence 15368
Sun Oct 05 16:02:44 2008
Errors in file g:\oracle9i\bdump\jsl_arc0_3096.trc:
ORA-12535: TNS:operation timed out

Sun Oct 05 16:08:12 2008
Errors in file g:\oracle9i\bdump\jsl_arc0_3096.trc:
ORA-12535: TNS:operation timed out

Sun Oct 05 16:10:37 2008
Thread 1 advanced to log sequence 15370
Current log# 5 seq# 15370 mem# 0: G:\ORACLE9I\LOGFILE\REDO05.LOG
Sun Oct 05 16:10:38 2008
ARC1: Evaluating archive log 3 thread 1 sequence 15369
ARC1: Beginning to archive log 3 thread 1 sequence 15369
Creating archive destination LOG_ARCHIVE_DEST_1: 'H:\ORACLE9I\ARCHIVE\MANDATORY\JSL_001_15369.ARC'
ARC1: Completed archiving log 3 thread 1 sequence 15369
Sun Oct 05 16:13:40 2008
Errors in file g:\oracle9i\bdump\jsl_arc0_3096.trc:
ORA-12535: TNS:operation timed out



jsl_arc0_3096 Trace file
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
*** 2008-10-05 15:57:15.312
ARC0: Error 12535 Creating archive log file to 'JSLSTBY'
*** 2008-10-05 15:57:15.328
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:02:44.156
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:02:44.171
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:08:12.500
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:08:12.531
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:13:40.515
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:13:40.531
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:19:09.078
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:19:09.093
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:24:37.531
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:24:37.562
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:30:05.984
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:30:06.015
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:35:34.484
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:35:34.500
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:41:02.828
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:41:02.843
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:46:30.078
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:46:30.093
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:51:58.531
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:51:58.546
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 16:57:26.781
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 16:57:26.796
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 17:02:55.125
Error 12535 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Error 12535 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'JSLSTBY'
Heartbeat failed to connect to standby 'JSLSTBY'. Error is 12535.
*** 2008-10-05 17:02:55.140
kcrrfail: dest:2 err:12535 force:0
ORA-12535: TNS:operation timed out
*** 2008-10-05 17:08:23.359

Thanks