ENV:single instance db 11.2.0.3 on 64bit windows 2008 server R2
no errors found in alert.log,oracle service goes down intermittently.
Following is the last lines in alert.log:
2013-05-29 20:54:59.890000 -06:00
Starting background process SMCO
SMCO started with pid=70, OS id=22636
2013-05-29 21:16:59.697000 -06:00
Time drift detected. Please check VKTM trace file for more details.
contents of the VKTM trace file:
Trace file C:\APP\ORACLE\DIAG1\diag\rdbms\vis\vis\trace\vis_vktm_25308.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Windows NT Version V6.1
CPU : 6 - type 8664, 6 Physical Cores
Process Affinity : 0x0x0000000000000000
Memory (Avail/Total): Ph:27412M/31899M, PhPgF:41016M/63797M+
Instance name: vis
Redo thread mounted by this instance: 0 <none>
Oracle process number: 4
Windows thread id: 25308, image: ORACLE.EXE (VKTM)
*** 2013-05-29 20:50:32.274
*** SESSION ID:(2017.1) 2013-05-29 20:50:32.274
*** CLIENT ID:() 2013-05-29 20:50:32.274
*** SERVICE NAME:() 2013-05-29 20:50:32.274
*** MODULE NAME:() 2013-05-29 20:50:32.274
*** ACTION NAME:() 2013-05-29 20:50:32.274
kstmmainvktm: succeeded in setting elevated priority
highres_enabled
*** 2013-05-29 20:50:32.274
VKTM running at (10)millisec precision with DBRM quantum (100)ms
[Start] HighResTick = 842214976721
kstmrmtickcnt = 0 : ksudbrmseccnt[0] = 1369882232
kstmchkdrift (kstmhighrestimecntkeeper:highres): Time jumped forward by (7478947)usec at (843798718318) whereas (1000000) is allowed
From windows event viewer,can found application error,following is the detail:
Log Name: Application
Source: Application Error
Date: 5/29/2013 9:16:52 PM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: SLC03OMO.us.oracle.com
Description:
Faulting application name: ORACLE.EXE, version: 11.2.0.3, time stamp: 0x4eb21f4b
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x000000006e8a00b6
Faulting process id: 0x3524
Faulting application start time: 0x01ce5ce060341504
Faulting application path: c:\app\oracle\product\11.2.0\bin\ORACLE.EXE
Faulting module path: unknown
Report Id: 5eec0ca0-c8d7-11e2-a59d-fc9cad5d59a7
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-05-30T03:16:52.000000000Z" />
<EventRecordID>33818</EventRecordID>
<Channel>Application</Channel>
<Computer>SLC03OMO.us.oracle.com</Computer>
<Security />
</System>
<EventData>
<Data>ORACLE.EXE</Data>
<Data>11.2.0.3</Data>
<Data>4eb21f4b</Data>
<Data>unknown</Data>
<Data>0.0.0.0</Data>
<Data>00000000</Data>
<Data>c0000005</Data>
<Data>000000006e8a00b6</Data>
<Data>3524</Data>
<Data>01ce5ce060341504</Data>
<Data>c:\app\oracle\product\11.2.0\bin\ORACLE.EXE</Data>
<Data>unknown</Data>
<Data>5eec0ca0-c8d7-11e2-a59d-fc9cad5d59a7</Data>
</EventData>
</Event>
can found another windows error reported almost the same time as the above application error:
Log Name: Application
Source: Windows Error Reporting
Date: 5/29/2013 9:17:14 PM
Event ID: 1001
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: SLC03OMO.us.oracle.com
Description:
Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: ORACLE.EXE
P2: 11.2.0.3
P3: 4eb21f4b
P4: StackHash_2264
P5: 0.0.0.0
P6: 00000000
P7: c0000005
P8: 000000006e8a00b6
P9:
P10:
Attached files:
C:\Windows\Temp\WER753D.tmp.appcompat.txt
C:\Windows\Temp\WER757D.tmp.WERInternalMetadata.xml
C:\Windows\Temp\WER757E.tmp.hdmp
C:\Windows\Temp\WER93C5.tmp.mdmp
These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_ORACLE.EXE_74241b6d51d6043166543f6b75dacacb44b54_cab_274194bc
Analysis symbol:
Rechecking for solution: 0
Report Id: 5eec0ca0-c8d7-11e2-a59d-fc9cad5d59a7
Report Status: 4
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Windows Error Reporting" />
<EventID Qualifiers="0">1001</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-05-30T03:17:14.000000000Z" />
<EventRecordID>33819</EventRecordID>
<Channel>Application</Channel>
<Computer>SLC03OMO.us.oracle.com</Computer>
<Security />
</System>
<EventData>
<Data>
</Data>
<Data>0</Data>
<Data>APPCRASH</Data>
<Data>Not available</Data>
<Data>0</Data>
<Data>ORACLE.EXE</Data>
<Data>11.2.0.3</Data>
<Data>4eb21f4b</Data>
<Data>StackHash_2264</Data>
<Data>0.0.0.0</Data>
<Data>00000000</Data>
<Data>c0000005</Data>
<Data>000000006e8a00b6</Data>
<Data>
</Data>
<Data>
</Data>
<Data>
C:\Windows\Temp\WER753D.tmp.appcompat.txt
C:\Windows\Temp\WER757D.tmp.WERInternalMetadata.xml
C:\Windows\Temp\WER757E.tmp.hdmp
C:\Windows\Temp\WER93C5.tmp.mdmp</Data>
<Data>C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_ORACLE.EXE_74241b6d51d6043166543f6b75dacacb44b54_cab_274194bc</Data>
<Data>
</Data>
<Data>0</Data>
<Data>5eec0ca0-c8d7-11e2-a59d-fc9cad5d59a7</Data>
<Data>4</Data>
</EventData>
</Event>
and following is the log of windows service control manager:
Log Name: System
Source: Service Control Manager
Date: 5/29/2013 9:17:15 PM
Event ID: 7034
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SLC03OMO.us.oracle.com
Description:
The OracleServiceVIS service terminated unexpectedly. It has done this 3 time(s).
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" />
<EventID Qualifiers="49152">7034</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8080000000000000</Keywords>
<TimeCreated SystemTime="2013-05-30T03:17:15.208132800Z" />
<EventRecordID>88710</EventRecordID>
<Correlation />
<Execution ProcessID="512" ThreadID="5532" />
<Channel>System</Channel>
<Computer>SLC03OMO.us.oracle.com</Computer>
<Security />
</System>
<EventData>
<Data Name="param1">OracleServiceVIS</Data>
<Data Name="param2">3</Data>
</EventData>
</Event>
Contents in Report.wer(a window event dump file):
Version=1
EventType=APPCRASH
EventTime=130143574121222854
ReportType=2
Consent=1
ReportIdentifier=5eec0ca1-c8d7-11e2-a59d-fc9cad5d59a7
IntegratorReportIdentifier=5eec0ca0-c8d7-11e2-a59d-fc9cad5d59a7
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=ORACLE.EXE
Sig[1].Name=Application Version
Sig[1].Value=11.2.0.3
Sig[2].Name=Application Timestamp
Sig[2].Value=4eb21f4b
Sig[3].Name=Fault Module Name
Sig[3].Value=StackHash_2264
Sig[4].Name=Fault Module Version
Sig[4].Value=0.0.0.0
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=00000000
Sig[6].Name=Exception Code
Sig[6].Value=c0000005
Sig[7].Name=Exception Offset
Sig[7].Value=000000006e8a00b6
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.1.7600.2.0.0.274.10
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=2264
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=2264db07e74365624c50317d7b856ae9
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=875f
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=875fa2ef9d2bdca96466e8af55d1ae6e
UI[2]=c:\app\oracle\product\11.2.0\bin\ORACLE.EXE
UI[5]=Check online for a solution (recommended)
UI[6]=Check for a solution later (recommended)
UI[7]=Close
UI[8]=Oracle RDBMS Kernel Executable stopped working and was closed
UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.
UI[10]=&Close
LoadedModule[0]=c:\app\oracle\product\11.2.0\bin\ORACLE.EXE
LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\Windows\system32\kernel32.dll
LoadedModule[3]=C:\Windows\system32\KERNELBASE.dll
LoadedModule[4]=c:\app\oracle\product\11.2.0\bin\oraclient11.dll
LoadedModule[5]=c:\app\oracle\product\11.2.0\bin\oracore11.dll
LoadedModule[6]=c:\app\oracle\product\11.2.0\bin\oranls11.dll
LoadedModule[7]=c:\app\oracle\product\11.2.0\bin\oraunls11.dll
LoadedModule[8]=C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4927_none_88dce9872fb18caf\MSVCR80.dll
LoadedModule[9]=C:\Windows\system32\msvcrt.dll
LoadedModule[10]=c:\app\oracle\product\11.2.0\bin\orauts.dll
LoadedModule[11]=C:\Windows\system32\ole32.dll
LoadedModule[12]=C:\Windows\system32\GDI32.dll
LoadedModule[13]=C:\Windows\system32\USER32.dll
LoadedModule[14]=C:\Windows\system32\LPK.dll
LoadedModule[15]=C:\Windows\system32\USP10.dll
LoadedModule[16]=C:\Windows\system32\RPCRT4.dll
LoadedModule[17]=C:\Windows\system32\PSAPI.DLL
LoadedModule[18]=C:\Windows\system32\WS2_32.dll
LoadedModule[19]=C:\Windows\system32\NSI.dll
LoadedModule[20]=C:\Windows\system32\ADVAPI32.dll
LoadedModule[21]=C:\Windows\SYSTEM32\sechost.dll
LoadedModule[22]=C:\Windows\system32\WINMM.dll
LoadedModule[23]=c:\app\oracle\product\11.2.0\bin\oracommon11.dll
LoadedModule[24]=c:\app\oracle\product\11.2.0\bin\orasnls11.dll
LoadedModule[25]=c:\app\oracle\product\11.2.0\bin\orageneric11.dll
LoadedModule[26]=c:\app\oracle\product\11.2.0\bin\oraxml11.dll
LoadedModule[27]=C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4927_none_88dce9872fb18caf\MSVCP80.dll
LoadedModule[28]=c:\app\oracle\product\11.2.0\bin\oran11.dll
LoadedModule[29]=c:\app\oracle\product\11.2.0\bin\oranl11.dll
LoadedModule[30]=c:\app\oracle\product\11.2.0\bin\orantcp11.dll
LoadedModule[31]=c:\app\oracle\product\11.2.0\bin\oranldap11.dll
LoadedModule[32]=c:\app\oracle\product\11.2.0\bin\orannzsbb11.dll
LoadedModule[33]=C:\Windows\system32\WSOCK32.dll
LoadedModule[34]=c:\app\oracle\product\11.2.0\bin\oraldapclnt11.dll
LoadedModule[35]=c:\app\oracle\product\11.2.0\bin\orancrypt11.dll
LoadedModule[36]=C:\Windows\system32\iphlpapi.dll
LoadedModule[37]=C:\Windows\system32\WINNSI.DLL
LoadedModule[38]=c:\app\oracle\product\11.2.0\bin\orazt11.dll
LoadedModule[39]=C:\Windows\system32\OLEAUT32.dll
LoadedModule[40]=c:\app\oracle\product\11.2.0\bin\oranro11.dll
LoadedModule[41]=c:\app\oracle\product\11.2.0\bin\oranhost11.dll
LoadedModule[42]=c:\app\oracle\product\11.2.0\bin\orancds11.dll
LoadedModule[43]=c:\app\oracle\product\11.2.0\bin\orantns11.dll
LoadedModule[44]=c:\app\oracle\product\11.2.0\bin\oraztkg11.dll
LoadedModule[45]=C:\Windows\system32\Secur32.dll
LoadedModule[46]=C:\Windows\system32\SSPICLI.DLL
LoadedModule[47]=c:\app\oracle\product\11.2.0\bin\orapls11.dll
LoadedModule[48]=c:\app\oracle\product\11.2.0\bin\oraslax11.dll
LoadedModule[49]=c:\app\oracle\product\11.2.0\bin\oravsn11.dll
LoadedModule[50]=c:\app\oracle\product\11.2.0\bin\oraplp11.dll
LoadedModule[51]=c:\app\oracle\product\11.2.0\bin\orahasgen11.dll
LoadedModule[52]=C:\Windows\system32\NETAPI32.dll
LoadedModule[53]=C:\Windows\system32\netutils.dll
LoadedModule[54]=C:\Windows\system32\srvcli.dll
LoadedModule[55]=C:\Windows\system32\wkscli.dll
LoadedModule[56]=C:\Windows\system32\SAMCLI.DLL
LoadedModule[57]=C:\Windows\system32\LOGONCLI.DLL
LoadedModule[58]=c:\app\oracle\product\11.2.0\bin\oraocr11.dll
LoadedModule[59]=c:\app\oracle\product\11.2.0\bin\oraocrb11.dll
LoadedModule[60]=c:\app\oracle\product\11.2.0\bin\oraasmclnt11.dll
LoadedModule[61]=c:\app\oracle\product\11.2.0\bin\oracell11.dll
LoadedModule[62]=c:\app\oracle\product\11.2.0\bin\oraocrutl11.dll
LoadedModule[63]=C:\Windows\system32\imagehlp.dll
LoadedModule[64]=C:\Windows\system32\SHELL32.dll
LoadedModule[65]=C:\Windows\system32\SHLWAPI.dll
LoadedModule[66]=c:\app\oracle\product\11.2.0\bin\orasql11.dll
LoadedModule[67]=c:\app\oracle\product\11.2.0\bin\OCI.dll
LoadedModule[68]=c:\app\oracle\product\11.2.0\bin\libmmd.dll
LoadedModule[69]=c:\app\oracle\product\11.2.0\bin\oransgr11.dll
LoadedModule[70]=c:\app\oracle\product\11.2.0\bin\oraclsra11.dll
LoadedModule[71]=c:\app\oracle\product\11.2.0\bin\oradbcfg11.dll
LoadedModule[72]=c:\app\oracle\product\11.2.0\bin\orawwg.dll
LoadedModule[73]=C:\Windows\system32\IMM32.DLL
LoadedModule[74]=C:\Windows\system32\MSCTF.dll
LoadedModule[75]=C:\Windows\system32\dbghelp.dll
LoadedModule[76]=C:\app\oracle\product\11.2.0\BIN\ORAIMR11.Dll
LoadedModule[77]=C:\Windows\system32\mswsock.dll
LoadedModule[78]=C:\Windows\System32\wship6.dll
LoadedModule[79]=C:\app\oracle\product\11.2.0\bin\oranbeq11.dll
LoadedModule[80]=C:\Windows\system32\security.dll
LoadedModule[81]=C:\Windows\system32\DSROLE.DLL
LoadedModule[82]=C:\Windows\system32\activeds.dll
LoadedModule[83]=C:\Windows\system32\adsldpc.dll
LoadedModule[84]=C:\Windows\system32\WLDAP32.dll
LoadedModule[85]=C:\Windows\system32\ATL.DLL
LoadedModule[86]=C:\Windows\system32\credssp.dll
LoadedModule[87]=C:\Windows\system32\msv1_0.DLL
LoadedModule[88]=C:\Windows\system32\cryptdll.dll
LoadedModule[89]=C:\Windows\system32\cscapi.dll
LoadedModule[90]=C:\Windows\system32\SAMLIB.dll
LoadedModule[91]=C:\Windows\system32\NLAapi.dll
LoadedModule[92]=C:\Windows\system32\DNSAPI.dll
LoadedModule[93]=C:\Windows\System32\winrnr.dll
LoadedModule[94]=C:\Windows\system32\napinsp.dll
LoadedModule[95]=C:\Windows\System32\wshtcpip.dll
LoadedModule[96]=C:\Windows\system32\rasadhlp.dll
LoadedModule[97]=C:\Windows\System32\fwpuclnt.dll
LoadedModule[98]=C:\Windows\system32\CRYPTBASE.dll
File[0].CabName=AppCompat.txt
File[0].Path=WER753D.tmp.appcompat.txt
File[0].Flags=65538
File[0].Type=5
File[0].Original.Path=C:\Windows\Temp\WER753D.tmp.appcompat.txt
File[1].CabName=WERInternalMetadata.xml
File[1].Path=WER757D.tmp.WERInternalMetadata.xml
File[1].Flags=65538
File[1].Type=5
File[1].Original.Path=C:\Windows\Temp\WER757D.tmp.WERInternalMetadata.xml
File[2].CabName=memory.hdmp
File[2].Path=WER757E.tmp.hdmp
File[2].Flags=2097152
File[2].Type=3
File[2].Original.Path=C:\Windows\Temp\WER757E.tmp.hdmp
File[3].CabName=minidump.mdmp
File[3].Path=WER93C5.tmp.mdmp
File[3].Flags=2162690
File[3].Type=2
File[3].Original.Path=C:\Windows\Temp\WER93C5.tmp.mdmp
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=Oracle RDBMS Kernel Executable
AppPath=c:\app\oracle\product\11.2.0\bin\ORACLE.EXE