Hi
We have standalone OBIEE 11g environment prod.
today we have seen Presentation server crash more than 8 times in the span of 4 hours. below are the logs from sawlog.log. Please help in identifying the issue.
Thanks in Advance.
[OBIPS] [ERROR:31] [] [saw.views.evc.activate] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000ccaf9,0:1] [tid: 6556] [[
File:ruleexpresionbuilder.cpp
[2014-05-19T18:44:35.000+02:00] [OBIPS] [WARNING:16] [] [saw.odbc.statement.execute.timings] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cb6e5,0:1:9] [tid: 8552] SQLExecDirectW returned in 258290 msec(s).[[
File:odbcstatementimpl.cpp
PROCESS CRASHED at the same time
[2014-05-19T18:46:30.000+02:00] [OBIPS] [ERROR:31] [] [saw.httpserver.processrequest] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd213,0:1] [tid: 10264] The current xml is invalid with the following errors:
Bad xml instance!
[2014-05-19T18:46:50.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd515,0:1] [tid: 6216] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:46:50.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd515,0:1] [tid: 6216] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:46:50.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd515,0:1] [tid: 6216] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:46:50.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd515,0:1] [tid: 6216] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:46:59.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd787,0:1] [tid: 6616] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:rpcserver.cpp
[2014-05-19T18:47:04.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000cd8fc,0:1] [tid: 10944] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:50:34.000+02:00] [OBIPS] [WARNING:16] [] [saw.threadpool.indexusersaccount.performance] [ecid: ] [tid: ] ThreadPool<IndexUsersAccount:1>.Thread count: running 0, maximum 1 (1), idle 0. Jobs pending: 1. No available threads, creating new thread for job. Approaching configured maximum for total number of threads.[[
File:threadpool.cpp
Line:1258
[2014-05-19T18:50:44.000+02:00] [OBIPS] [ERROR:10] [] [saw.environment] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0371,0:1] [tid: 8684] Invalid Handle Of 'class saw::UserPrefs * __ptr64' Exception[[
File:handle.cpp
[2014-05-19T18:50:52.000+02:00] [OBIPS] [ERROR:1] [] [saw.securitysubsystem.checkauthentication.runimpl] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d043d,0:1:1] [tid: 3180] Odbc driver returned an error (SQLDriverConnectW).
State: 08004. Code: 10018. [NQODBC] [SQL_STATE: 08004] [nQSError: 10018] Access for the requested connection is refused.
[nQSError: 43113] Message returned from OBIS.
[nQSError: 13024] Successful completion of init block 'Authorization User Mgt' is required. (08004)[[
File:checkauthentication.cpp
[2014-05-19T18:51:00.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0571,0:1] [tid: 8684] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:51:00.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0571,0:1] [tid: 8684] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:51:00.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0571,0:1] [tid: 8684] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:rpcserver.cpp
[2014-05-19T18:51:41.000+02:00] [OBIPS] [ERROR:31] [] [saw.views.evc.activate] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d083a,0:1] [tid: 9444] [[
File:ruleexpresionbuilder.cpp
[2014-05-19T18:51:49.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0a70,0:1] [tid: 9444] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:51:49.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0a70,0:1] [tid: 9444] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:51:49.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0a70,0:1] [tid: 9444] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:rpcserver.cpp
[2014-05-19T18:52:02.000+02:00] [OBIPS] [WARNING:16] [] [saw.threadpool.indexusersaccount.performance] [ecid: ] [tid: ] ThreadPool<IndexUsersAccount:0>.Thread count: running 0, maximum 1 (1), idle 0. Jobs pending: 1. No available threads, creating new thread for job. Approaching configured maximum for total number of threads.[[
File:threadpool.cpp
[2014-05-19T18:52:06.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0c21,0:1] [tid: 11072] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:52:06.000+02:00] [OBIPS] [ERROR:16] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0c21,0:1] [tid: 11072] Fatal error during variable length response.[[
File:rpcserver.cpp
[2014-05-19T18:52:07.000+02:00] [OBIPS] [NOTIFICATION:1] [] [saw.security.userpopulationcache.populateapproleshierarchy] [ecid: 00iM5bTcnEXB5EECj7iOOo0002yS000000,0:2] [tid: 9660] Populating the application role hierarchy from backend.[[
File:userpopulationcache.cpp
[2014-05-19T18:52:14.000+02:00] [OBIPS] [ERROR:10] [] [saw.rpc.server.responder] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d0f9b,0:1] [tid: 10952] Assertion failure: nReqLen < 10*1024*1024 at line 141 of C:\ADE\aime_1\bifndn\analytics_web\main/project/webhttpserver/sawhttpserver.cpp[[
File:sawhttpserver.cpp
[2014-05-19T18:52:29.000+02:00] [OBIPS] [ERROR:31] [] [saw.views.evc.activate] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d1497,0:1] [tid: 11928] [[
File:ruleexpresionbuilder.cpp
[2014-05-19T18:53:01.000+02:00] [OBIPS] [ERROR:31] [] [saw.xmlDocumentCache.findDocument] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d20cb,0:1] [tid: 11928] Path not found: '\\?\E:\Apps\obiee\instances\instance1\tmp\OracleBIPresentationServicesComponent\coreapplication_obips1\sawvc\100b\9in5v44oi9h9are2vuhu70bbl6.sawxd'
The system cannot find the file specified.
[2014-05-19T18:53:04.000+02:00] [OBIPS] [ERROR:31] [] [saw.xmlDocumentCache.findDocument] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d20f9,0:1] [tid: 11928] Path not found: '\\?\E:\Apps\obiee\instances\instance1\tmp\OracleBIPresentationServicesComponent\coreapplication_obips1\sawvc\10de\e037m5q0obojdmjvtfvt61bqn2.sawxd'
The system cannot find the file specified.
[[
[2014-05-19T18:53:05.000+02:00] [OBIPS] [ERROR:31] [] [saw.views.evc.activate] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d20f9,0:1] [tid: 11928] [[
File:ruleexpresionbuilder.cpp
[2014-05-19T18:53:16.000+02:00] [OBIPS] [ERROR:31] [] [saw.httpserver.processrequest] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-00000000000d23d0,0:1] [tid: 5884] Unable to find the requested state identifer. Ensure that html content caching is not enabled for the content request for this page and try again. '167671154' (initial: '167686972', cleared: '167686972')[[
[2014-05-19T22:25:39.000+02:00] [OracleBIServerComponent] [ERROR:1] [] [] [ecid: 00iM5cmpocbB5EECj7iOOo0002bc000000] [tid: 22ac] [nQSError: 12002] Socket communication error at call=recv: (Number=10054) An existing connection was forcibly closed by the remote host.
[2014-05-19T22:25:34.000+02:00] [OBIPS] [ERROR:31] [] [saw.httpserver.processrequest] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-0000000000110974,0:1] [tid: 7060] Client no longer there.[[
[2014-05-19T22:25:34.000+02:00] [OBIPS] [ERROR:31] [] [saw.rpc.variablemos.finalize] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-0000000000110974,0:1] [tid: 7060] An error occurred during execution of "send". An established connection was aborted by the software in your host machine.
[2014-05-19T22:25:38.000+02:00] [OBIPS] [ERROR:1] [] [saw.securitysubsystem.checkauthentication.runimpl] [ecid: b291e43ee00dcdd6:-7f7dc1e9:146142b6536:-8000-0000000000110b45,0:1:1] [tid: 10164] Odbc driver returned an error (SQLDriverConnectW).
State: 08004. Code: 10018. [NQODBC] [SQL_STATE: 08004] [nQSError: 10018] Access for the requested connection is refused.