Hi,
ENV: 11gR2 on Filesystem OL 6.2 64-bit
The instance terminated and I found following messages in the alert log file:
Thu Mar 27 02:35:47 2014
Thread 1 cannot allocate new log, sequence 466
Private strand flush not complete
Current log# 3 seq# 465 mem# 0: /u05/oradata/DB/redo3a.log
Current log# 3 seq# 465 mem# 1: /u06/oradata/DB/redo3b.log
Thread 1 advanced to log sequence 466 (LGWR switch)
Current log# 4 seq# 466 mem# 0: /u05/oradata/DB/redo4a.log
Current log# 4 seq# 466 mem# 1: /u06/oradata/DB/redo4b.log
Thu Mar 27 02:35:53 2014
Archived Log entry 463 added for thread 1 sequence 465 ID 0xa29e0fd4 dest 1:
Thu Mar 27 03:44:00 2014
Time drift detected. Please check VKTM trace file for more details.
Thu Mar 27 06:50:36 2014
Time drift detected. Please check VKTM trace file for more details.
Thu Mar 27 06:52:38 2014
PMON (ospid: 5095): terminating the instance due to error 495
Instance terminated by PMON, pid = 5095
============= contents of VKTM trace file - START ===========
Trace file /u01/app/oracle/diag/rdbms/db/DB/trace/DB_vktm_5110.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0.3/db01
System name: Linux
Node name: server
Release: 2.6.32-300.3.1.el6uek.x86_64
Version: #1 SMP Fri Dec 9 18:57:35 EST 2011
Machine: x86_64
VM name: VMWare Version: 6
Instance name: DB
Redo thread mounted by this instance: 0 <none>
Oracle process number: 4
Unix process pid: 5110, image: oracle@<server>(VKTM)
*** 2014-03-21 11:41:52.559
*** SESSION ID:(225.1) 2014-03-21 11:41:52.559
*** CLIENT ID:() 2014-03-21 11:41:52.559
*** SERVICE NAME:() 2014-03-21 11:41:52.559
*** MODULE NAME:() 2014-03-21 11:41:52.559
*** ACTION NAME:() 2014-03-21 11:41:52.559
kstmmainvktm: succeeded in setting elevated priority
highres_enabled
*** 2014-03-21 11:41:52.559
VKTM running at (1)millisec precision with DBRM quantum (100)ms
[Start] HighResTick = 1395416512559467
kstmrmtickcnt = 0 : ksudbrmseccnt[0] = 1395416512
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (1107148)usec at (1395420766148222) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (3229823)usec at (1395723624369920) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (1360201)usec at (1395726319223046) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (13771467)usec at (1395821053614109) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (1346752)usec at (1395851825790091) whereas (1000000) is allowed
*** 2014-03-26 12:37:06.790
kstmchkdrift (kstmhighrestimecntkeeper:lowres): Time stalled at 1395851826
kstmchkdrift (kstmhighrestimecntkeeper:lowres): Stall, backward drift ended at 1395851827 drift: 1
*** 2014-03-26 12:37:51.564
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (3264482)usec at (1395851871564711) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (12688664)usec at (1395862536639694) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (14625561)usec at (1395864123470429) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (13487852)usec at (1395881237330267) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (16040064)usec at (1395882841905867) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (7040206)usec at (1395890899897798) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (3362359)usec at (1395906240249424) whereas (1000000) is allowed
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (20075769)usec at (1395917436942244) whereas (1000000) is allowed
============= contents of VKTM trace file - END ===========
Any idea?
Best regards
Message was edited by: user130038 Sorry forgot to mention that this database server is a "VMWare virtual Machine" :-(
Message was edited by: user130038
Message was edited by: user130038
=================================
Contents of trace file
==== START====
Trace file /u01/app/oracle/diag/rdbms/db/DB/trace/DB_pmon_5095.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0.3/db01
System name: Linux
Node name: server
Release: 2.6.32-300.3.1.el6uek.x86_64
Version: #1 SMP Fri Dec 9 18:57:35 EST 2011
Machine: x86_64
VM name: VMWare Version: 6
Instance name: DB
Redo thread mounted by this instance: 1
Oracle process number: 2
Unix process pid: 5095, image: oracle@server (PMON)
*** 2014-03-27 06:52:38.362
*** SESSION ID:(113.1) 2014-03-27 06:52:38.364
*** CLIENT ID:() 2014-03-27 06:52:38.364
*** SERVICE NAME:(SYS$BACKGROUND) 2014-03-27 06:52:38.364
*** MODULE NAME:() 2014-03-27 06:52:38.364
*** ACTION NAME:() 2014-03-27 06:52:38.364
Background process GEN0 found dead
Oracle pid = 5
OS pid (from detached process) = 5114
OS pid (from process state) = 5114
dtp = 0x6000c150, proc = 0x8e0976b08
Dump of memory from 0x000000006000C150 to 0x000000006000C1B0
06000C160 00000000 00000000 00000000 304E4547 [............GEN0]
06000C170 00000200 00000000 000013FA 00000000 [................]
06000C180 00001607 00000000 AE3D3700 00007F72 [.........7=.r...]
06000C190 000013FA 00000000 00001607 00000000 [................]
06000C1A0 00000001 19983B71 00040081 00000000 [....q;..........]
Dump of memory from 0x00000008E0976B08 to 0x00000008E0977BB8
8E0976B00 00000301 00000000 [........]
8E0976B10 00000302 00000000 00000000 00000000 [................]
8E0976B20 E0976B08 00000008 E0976B28 00000008 [.k......(k......]
8E0976B30 E0976B28 00000008 00000000 00000000 [(k..............]
8E0976B40 00000000 00000000 00000000 00000000 [................]
Repeat 1 times
8E0976B60 00000008 00000000 18DA4DD8 00000009 [.........M......]
8E0976B70 E0DBD370 00000008 18BC2E50 00000009 [p.......P.......]
8E0976B80 18E5E7B0 00000009 00000000 00000000 [................]
8E0976B90 18E5E848 00000009 18E5E848 00000009 [H.......H.......]
8E0976BA0 E0DBD350 00000008 00000701 00000010 [P...............]
8E0976BB0 E0B02C70 00000008 18BC2E50 00000009 [p,......P.......]
8E0976BC0 00000005 00000000 00000000 00000005 [................]
8E0976BD0 00000001 FFF2526A 17FAC018 00000009 [....jR..........]
8E0976BE0 00000000 00000000 00000000 00000000 [................]
Repeat 2 times
8E0976C10 18DA1D60 00000009 18DA4E30 00000009 [`.......0N......]
8E0976C20 00000000 00000000 00000000 00000000 [................]
Repeat 3 times
8E0976C60 00000000 00000000 00030000 00000000 [................]
Repeat 1 times
8E0976C80 00000000 00000000 00000000 00000000 [................]
8E0976C90 00000000 00000000 00040000 00000000 [................]
Repeat 1 times
8E0976CB0 00000000 00000000 00030000 00000000 [................]
Repeat 1 times
8E0976CD0 00000000 00000000 00050000 00000000 [................]
8E0976CE0 00000000 00000000 00030000 00000000 [................]
8E0976CF0 00000000 00000000 00000000 00000000 [................]
Repeat 4 times
8E0976D40 00000000 00000000 00040000 00000000 [................]
8E0976D50 00000000 00000000 00000000 00000000 [................]
Repeat 7 times
8E0976DD0 E0976DD0 00000008 E0976DD0 00000008 [.m.......m......]
8E0976DE0 00000000 00000000 00000000 00000000 [................]
8E0976DF0 E0976DF0 00000008 E0976DF0 00000008 [.m.......m......]
8E0976E00 00000000 00000000 00000000 00000000 [................]
Repeat 3 times
8E0976E40 00000023 00000024 0000000E 0000000F [#...$...........]
8E0976E50 E8929580 00000008 E8929580 00000008 [................]
8E0976E60 000067AF 00000000 00010000 00000000 [.g..............]
8E0976E70 00000000 00000000 00000000 00000000 [................]
Repeat 2 times
8E0976EA0 000013FA 00000000 00000000 00000000 [................]
8E0976EB0 00000000 00000000 00000000 00000000 [................]
Repeat 2 times
8E0976EE0 E0976B08 00000008 00000000 00000000 [.k..............]
8E0976EF0 00000000 00000000 00000000 00000000 [................]
Repeat 10 times
8E0976FA0 00000000 00000000 E0976FA8 00000008 [.........o......]
8E0976FB0 E0976FA8 00000008 00000000 00000000 [.o..............]
8E0976FC0 00000000 00000000 00000000 00000001 [................]
8E0976FD0 00000000 00000000 00000000 00070005 [................]
8E0976FD0 00000000 00000000 00000000 00070005 [................]
8E0976FE0 00000009 00000000 000013FA 00000000 [................]
8E0976FF0 00001607 00000000 AE3D3700 00007F72 [.........7=.r...]
8E0977000 000013FA 00000000 00001607 00000000 [................]
8E0977010 00000000 00000000 E0977020 00000008 [........ p......]
8E0977020 00000000 00000000 00000000 00000000 [................]
Repeat 2 times
8E0977050 00000000 00000000 00000003 FFFFFFFF [................]
8E0977060 00000000 00000000 00000000 00000000 [................]
Repeat 13 times
8E0977140 6361726F 0000656C 00000000 00000000 [oracle..........]
8E0977150 00000000 00000000 00000000 00000000 [................]
8E0977160 00000006 00000000 6C677079 646F6364 [........se]
8E0977170 62697462 72706F63 3130646F 00000000 [rver....]
8E0977180 00000000 00000000 00000000 00000000 [................]
Repeat 1 times
8E09771A0 00000000 00000000 00000014 00000000 [................]
8E09771B0 00000000 4E4B4E55 004E574F 00000000 [....UNKNOWN.....]
8E09771C0 00000000 00000000 00000000 00000000 [................]
8E09771D0 00000000 00000000 00000007 00000000 [................]
8E09771E0 34313135 00000000 00000000 00000000 [5114............]
8E09771F0 00000000 00000000 00000004 00000000 [................]
8E0977200 6361726F 7940656C 646C6770 62646F63 [oracle@se]
8E0977210 63626974 6F72706F 20313064 4E454728 [rver (GEN]
8E0977220 00002930 00000000 00000000 00000000 [0)..............]
8E0977230 00000022 00000000 304E4547 00000000 [".......GEN0....]
8E0977240 00000002 00000000 00000000 00000000 [................]
8E0977250 00000000 00000000 00000000 00000000 [................]
Repeat 8 times
8E09772E0 00000000 00000000 00000000 00000200 [................]
8E09772F0 00000000 00000000 00000000 00000000 [................]
8E0977300 00000000 00000000 E09F8FF8 00000008 [................]
8E0977310 188F9498 00000009 E8925AC8 00000008 [.........Z......]
8E0977320 00000000 00000000 00000000 00000000 [................]
8E0977330 F0D8CFF8 00000008 00000000 00000000 [................]
8E0977340 00000000 00000000 00000000 00000000 [................]
8E0977350 00000000 00000000 E0977358 00000008 [........Xs......]
8E0977360 E0977358 00000008 00050000 00000000 [Xs..............]
8E0977370 00000000 00000000 00088938 00000000 [........8.......]
8E0977380 000B5C88 00000000 000D8938 00000000 [.\......8.......]
8E0977390 00000000 00000000 00000000 00000000 [................]
8E09773A0 00000000 00000000 000007D8 00000000 [................]
8E09773B0 000000E0 00000000 000007D8 00000000 [................]
8E09773C0 00000000 00000000 00000000 00000000 [................]
Repeat 3 times
8E0977400 05000100 2527E000 00000000 01000000 [......'%........]
8E0977410 00000000 00000000 00000000 00000000 [................]
Repeat 1 times
8E0977430 00000000 00000000 00000000 00000001 [................]
8E0977440 00000000 00000000 00000000 00000000 [................]
Repeat 113 times
8E0977B60 00000000 00000000 E0977B68 00000008 [........h{......]
8E0977B70 E0977B68 00000008 00000000 00000000 [h{..............]
8E0977B80 E0977B80 00000008 E0977B80 00000008 [.{.......{......]
8E0977B90 533402F5 00000000 00000000 00000000 [..4S............]
8E0977BA0 00000000 00000000 00000000 00000000 [................]
8E0977BB0 00000000 00000000 [........]
kjzduptcctx: Notifying DIAG for crash event
----- Abridged Call Stack Trace -----
ksedsts()+461<-kjzdssdmp()+267<-kjzduptcctx()+232<-kjzdicrshnfy()+53<-ksuitm()+1332<-ksuxfd_check_dead()+395<-ksuxfd()+135<-ksuxfl()+425<-ksucln_dpc_main()+197<-ksucln_dpc()+29<-ksucln()+1609<-ksbrdp()+971<-opirip()+623<-opidrv()+603<-sou2o()+103<-opimai_real()+266
<-ssthrdmain()+252<-main()+201<-__libc_start_main()+253<-_start()+36
----- End of Abridged Call Stack Trace -----
*** 2014-03-27 06:52:38.768
PMON (ospid: 5095): terminating the instance due to error 495
======= END =====