Database Software

Hi,

I encountered this on a production environment, the datastore was then became invalid, no connection to TimesTen available causing failures in our Java application. TimesTen version is 7.0.5 64-bit on RedHat Linux.

Wondering what the root cause and how to prevent it. Thanks in advance.


15:44:09.44 Err : : 23922: 17268/0x2aad67ef1820: Assertion failed: len > 0 [ttree.c:4.391:tnLookup():6884] PID 17268 (java) CONN 82 (java) 2012-03-08 15:44:09.433
15:44:09.47 Err : : 23922: 17268/0x2aad67ef1820: Data store marked invalid [ttree.c:4.391:tnLookup():6884] PID 17268 (java) CONN 82 (java) Context 0x2aad67ef1820
15:44:09.47 Warn: : 23922: 23925/0x194e2990: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.47 Err : : 23925: TT14000: TimesTen daemon internal error: subd: flusher thread failed in sb_dbLogFlusherSvc, tt error 994 (TT0994: Data store connection terminated. Please reconnect. -- file "dbAPI.c", lineno 9535, procedure "sb_dbLogFlusherSvc")
15:44:09.48 Err : REP: 29066: tt_data_1:receiver.c(2273): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.49 Err : REP: 29066: tt_data_1:receiver.c(1095): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.49 Err : REP: 29066: tt_data_1:repagent.c(1243): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.49 Err : : 23925: TT14000: TimesTen daemon internal error: subd: flusher thread failed to disconnect, tt error 0 (TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()").
15:44:09.49 Warn: : 23922: 17268/0x2aad78b0aa30: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.49 Warn: : 23922: 17268 ----------: Disconnecting from an old instance
15:44:09.49 Warn: : 23925: Stopping subdaemon worker for /opt/datastore/tt705/datastore/tt_data_1 because db is invalid.
15:44:09.49 Err : : 23925: subd: manager exiting due to invalidation
15:44:09.50 Warn: : 23922: 29066/0x490cdf0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.50 Warn: : 23922: 29066 ----------: Disconnecting from an old instance
15:44:09.50 Warn: : 23922: 29066/0x493fb10: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.50 Warn: : 23922: 29066 ----------: Disconnecting from an old instance
15:44:09.50 Err : REP: 29066: tt_data_1:repagent.c(3079): TT16005: Failed to disconnect from datastore '/opt/datastore/tt705/datastore/tt_data_1' for 'LOGFORCE' thread
15:44:09.50 Err : REP: 29066: tt_data_1:repagent.c(3079): TT846: TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()"
15:44:09.50 Warn: : 23922: 29066/0x4a44b20: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.50 Warn: : 23922: 29066 ----------: Disconnecting from an old instance
15:44:09.50 Err : REP: 29066: tt_data_1:repagent.c(1243): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.50 Err : REP: 29066: tt_data_1:repagent.c(3079): TT16005: Failed to disconnect from datastore '/opt/datastore/tt705/datastore/tt_data_1' for 'RECEIVER' thread
15:44:09.50 Err : REP: 29066: tt_data_1:repagent.c(3079): TT846: TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()"
15:44:09.51 Err : REP: 29066: tt_data_1:transmitter.c(8190): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.51 Err : REP: 29066: tt_data_1:repagent.c(1243): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:09.51 Err : REP: 29066: tt_data_1:repagent.c(3079): TT16005: Failed to disconnect from datastore '/opt/datastore/tt705/datastore/tt_data_1' for 'TRANSMITTER' thread
15:44:09.51 Err : REP: 29066: tt_data_1:repagent.c(3079): TT846: TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()"
15:44:09.52 Warn: : 23922: 8527/0xb35b0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.52 Warn: : 23922: 8527 ----------: Disconnecting from an old instance
15:44:09.55 Warn: SRV: 8527: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.66 Warn: : 23922: 4729/0xdfe50a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.66 Warn: : 23922: 4729 ----------: Disconnecting from an old instance
15:44:09.67 Warn: : 23922: 8511/0xe6f50a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.67 Warn: : 23922: 8511 ----------: Disconnecting from an old instance
15:44:09.70 Warn: : 23922: 10288/0x5ff0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.70 Warn: : 23922: 10288 ----------: Disconnecting from an old instance
15:44:09.70 Warn: : 23922: 23985/0x31f20a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.70 Warn: : 23922: 23985 ----------: Disconnecting from an old instance
15:44:09.71 Warn: SRV: 4729: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.71 Warn: SRV: 8511: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.73 Warn: : 23922: 29556/0x964b0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.73 Warn: : 23922: 29556 ----------: Disconnecting from an old instance
15:44:09.74 Warn: : 23922: 4716/0x193050a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.74 Warn: : 23922: 4716 ----------: Disconnecting from an old instance
15:44:09.74 Warn: : 23922: 3887/0xbb780a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.74 Warn: : 23922: 3887 ----------: Disconnecting from an old instance
15:44:09.77 Warn: SRV: 10288: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.81 Warn: SRV: 3887: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.81 Warn: : 23922: 16522/0x1cc900a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.81 Warn: : 23922: 16522 ----------: Disconnecting from an old instance
15:44:09.81 Warn: SRV: 23985: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.82 Warn: SRV: 29556: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.83 Warn: SRV: 4716: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.87 Warn: : 23922: 23973/0x46c30a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.87 Warn: : 23922: 23973 ----------: Disconnecting from an old instance
15:44:09.88 Warn: : 23922: 3902/0xb1670a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.88 Warn: : 23922: 3902 ----------: Disconnecting from an old instance
15:44:09.88 Warn: SRV: 3902: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.89 Warn: SRV: 16522: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.92 Warn: SRV: 23973: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.93 Warn: : 23922: 29575/0x14fc20a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.93 Warn: : 23922: 29575 ----------: Disconnecting from an old instance
15:44:09.95 Warn: : 23922: 23925 ------------------: subdaemon process exited
15:44:09.95 Warn: : 23922: 23925 exited while connected to data store '/opt/datastore/tt705/datastore/tt_data_1' shm 134809540 count=5
15:44:09.95 Warn: : 23922: daRecovery: subdaemon 23925, managing data store, failed: invalidate (failcode=202)
15:44:09.96 Warn: SRV: 29575: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:09.96 Warn: : 23922: 3894/0x41d20a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.96 Warn: : 23922: 3894 ----------: Disconnecting from an old instance
15:44:09.97 Warn: : 23922: 23977/0x1505d0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.97 Warn: : 23922: 23977 ----------: Disconnecting from an old instance
15:44:09.97 Warn: : 23922: 10276/0xb81d0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.97 Warn: : 23922: 10276 ----------: Disconnecting from an old instance
15:44:09.99 Warn: : 23922: 4712/0x19ca50a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:09.99 Warn: : 23922: 4712 ----------: Disconnecting from an old instance
15:44:09.99 Warn: SRV: 4712: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.00 Warn: : 23922: 16531/0x175920a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.00 Warn: : 23922: 16531 ----------: Disconnecting from an old instance
15:44:10.05 Warn: SRV: 10276: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.06 Warn: SRV: 3894: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.07 Warn: : 23922: 29567/0x190a80a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.07 Warn: : 23922: 29567 ----------: Disconnecting from an old instance
15:44:10.07 Warn: SRV: 23977: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.08 Warn: : 23922: 10280/0x36420a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.08 Warn: : 23922: 10280 ----------: Disconnecting from an old instance
15:44:10.10 Warn: SRV: 16531: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.11 Warn: SRV: 29567: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.12 Warn: SRV: 10280: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.16 Err : REP: 29066: tt_data_1:rephold.c(141): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:10.16 Err : REP: 29066: tt_data_1:repagent.c(1243): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:10.16 Warn: : 23922: 29066/0x4969bf0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.16 Warn: : 23922: 29066 ----------: Disconnecting from an old instance
15:44:10.16 Err : REP: 29066: tt_data_1:repagent.c(3079): TT16005: Failed to disconnect from datastore '/opt/datastore/tt705/datastore/tt_data_1' for 'REPHOLD' thread
15:44:10.16 Err : REP: 29066: tt_data_1:repagent.c(3079): TT846: TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()"
15:44:10.18 Err : REP: 29066: tt_data_1:receiver.c(1637): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:10.18 Err : REP: 29066: tt_data_1:repagent.c(1243): TT16012: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:10.18 Warn: : 23922: 29066/0x2aad480008c0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.18 Warn: : 23922: 29066 ----------: Disconnecting from an old instance
15:44:10.24 Warn: : 23922: 22348/0x1fa20a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.24 Warn: : 23922: 22348 ----------: Disconnecting from an old instance
15:44:10.29 Warn: SRV: 22348: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.35 Warn: : 23922: 17268 exited while connected to data store '/opt/datastore/tt705/datastore/tt_data_1' shm 134809540 count=92
15:44:10.35 Warn: : 23922: 29562/0x858b0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.35 Warn: : 23922: 29562 ----------: Disconnecting from an old instance
15:44:10.37 Warn: : 23922: 14964/0x42250a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.37 Warn: : 23922: 14964 ----------: Disconnecting from an old instance
15:44:10.37 Warn: : 23922: 16535/0x138840a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.37 Warn: : 23922: 16535 ----------: Disconnecting from an old instance
15:44:10.37 Warn: : 23922: 29653/0x1555b0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.37 Warn: : 23922: 29653 ----------: Disconnecting from an old instance
15:44:10.38 Warn: SRV: 16535: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.39 Warn: : 23922: 10284/0x17b7f0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.39 Warn: : 23922: 10284 ----------: Disconnecting from an old instance
15:44:10.39 Warn: SRV: 10284: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.40 Warn: SRV: 14964: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.41 Warn: : 23922: 4725/0x1b9380a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.41 Warn: : 23922: 4725 ----------: Disconnecting from an old instance
15:44:10.43 Warn: SRV: 29562: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.44 Warn: : 23922: 16519/0x14de30a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.44 Warn: : 23922: 16519 ----------: Disconnecting from an old instance
15:44:10.46 Warn: SRV: 29653: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.47 Warn: SRV: 4725: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.47 Warn: : 23922: 29650/0x1658d0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.47 Warn: : 23922: 29650 ----------: Disconnecting from an old instance
15:44:10.49 Warn: SRV: 16519: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.50 Warn: : 23922: 3898/0x1e7c70a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.50 Warn: : 23922: 3898 ----------: Disconnecting from an old instance
15:44:10.51 Warn: SRV: 3898: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.51 Warn: : 23922: 16526/0x18bc60a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.51 Warn: : 23922: 16526 ----------: Disconnecting from an old instance
15:44:10.51 Warn: SRV: 29650: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.53 Warn: : 23922: 29662/0x3fcd0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.53 Warn: : 23922: 29662 ----------: Disconnecting from an old instance
15:44:10.54 Warn: SRV: 16526: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.54 Warn: : 23922: 3890/0x1720a0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.54 Warn: : 23922: 3890 ----------: Disconnecting from an old instance
15:44:10.57 Warn: : 23922: 29657/0x1a1860a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.57 Warn: : 23922: 29657 ----------: Disconnecting from an old instance
15:44:10.57 Warn: : 23922: 22356/0x1110f0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.57 Warn: : 23922: 22356 ----------: Disconnecting from an old instance
15:44:10.58 Warn: : 23922: 22345/0x70a30a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.58 Warn: : 23922: 22345 ----------: Disconnecting from an old instance
15:44:10.58 Warn: : 23922: 8500/0x78290a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.58 Warn: : 23922: 8500 ----------: Disconnecting from an old instance
15:44:10.58 Warn: : 23922: 29666/0x26920a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.58 Warn: : 23922: 29666 ----------: Disconnecting from an old instance
15:44:10.58 Warn: : 23922: 14947/0x39020a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.58 Warn: : 23922: 14947 ----------: Disconnecting from an old instance
15:44:10.60 Warn: SRV: 29662: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.61 Warn: SRV: 14947: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.62 Err : REP: 29066: tt_data_1:repagent.c(3079): TT16005: Failed to disconnect from datastore '/opt/datastore/tt705/datastore/tt_data_1' for 'REPLISTENER' thread
15:44:10.62 Err : REP: 29066: tt_data_1:repagent.c(3079): TT846: TT0846: Data store connection invalid or not current -- file "dbAPI.c", lineno 3135, procedure "sb_dbDisconnect()"
15:44:10.62 Warn: : 23922: 14952/0x49970a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.62 Warn: : 23922: 14952 ----------: Disconnecting from an old instance
15:44:10.62 Warn: : 23922: 14960/0x76770a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.62 Warn: : 23922: 14960 ----------: Disconnecting from an old instance
15:44:10.63 Warn: SRV: 3890: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.63 Warn: SRV: 29666: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.64 Warn: SRV: 29657: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.65 Warn: SRV: 22356: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.65 Warn: SRV: 22345: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.65 Warn: SRV: 14952: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.66 Warn: SRV: 8500: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.67 Warn: SRV: 14960: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.81 Err : : 23922: repagent says it has failed to start: Data store is invalid. Replication Agent exiting but may be restarted by Timesten daemon (depending on restart policy)
15:44:10.89 Warn: : 23922: 10273/0x1b1460a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.89 Warn: : 23922: 10273 ----------: Disconnecting from an old instance
15:44:10.91 Warn: SRV: 10273: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.96 Warn: : 23922: 4720/0x56290a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.96 Warn: : 23922: 4720 ----------: Disconnecting from an old instance
15:44:10.96 Warn: SRV: 4720: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:10.98 Warn: : 23922: 23970/0x1d4530a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:10.98 Warn: : 23922: 23970 ----------: Disconnecting from an old instance
15:44:11.01 Warn: SRV: 23970: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.10 Warn: : 23922: 29571/0x1fd0d0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.10 Warn: : 23922: 29571 ----------: Disconnecting from an old instance
15:44:11.17 Warn: SRV: 29571: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.24 Warn: : 23922: 14956/0x159720a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.24 Warn: : 23922: 14956 ----------: Disconnecting from an old instance
15:44:11.25 Warn: : 23922: 22360/0xb5de0a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.25 Warn: : 23922: 22360 ----------: Disconnecting from an old instance
15:44:11.29 Warn: SRV: 14956: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.32 Warn: SRV: 22360: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.32 Warn: : 23922: 23981/0xf5b30a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.32 Warn: : 23922: 23981 ----------: Disconnecting from an old instance
15:44:11.38 Warn: SRV: 23981: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.38 Warn: : 23922: 8515/0x69940a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.38 Warn: : 23922: 8515 ----------: Disconnecting from an old instance
15:44:11.40 Warn: : 23922: 8495/0x1c0a30a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.40 Warn: : 23922: 8495 ----------: Disconnecting from an old instance
15:44:11.41 Warn: SRV: 8515: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:11.44 Warn: : 23922: 22352/0x176130a0: Forced Disconnect /opt/datastore/tt705/datastore/tt_data_1
15:44:11.44 Warn: : 23922: 22352 ----------: Disconnecting from an old instance
15:44:11.47 Warn: SRV: 22352: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:44:12.50 Warn: SRV: 8495: EventID=99| Data store connection terminated by TimesTen Server (because data store is invalid). Please reconnect.
15:45:54.96 Warn: : 23922: 29081/0x611f0f0: Recovery started
15:55:32.03 Warn: : 29081: subd: Warning identified in [sub.c: line 3188]
15:55:32.03 Warn: : 29081: subd: (Warning 20100): TT20100: This connection required recovery due to an improper shutdown -- file "db.c", lineno 11566, procedure "sbDbConnect"
15:55:32.03 Warn: : 29081: file "db.c", lineno 11566, procedure "sbDbConnect"
15:55:32.03 Warn: : 29081: subd: connect trouble, rc 2, reason 20100
15:55:32.03 Warn: : 29081: Warn 20100: TT20100: This connection required recovery due to an improper shutdown -- file "db.c", lineno 11566, procedure "sbDbConnect"
15:57:35.55 Err : REP: 3041: tt_data_1:receiver.c(10091): TT16084: Table: tt_data_1.SESSION. Failed to insert row for 'insert'
15:57:35.55 Err : REP: 3041: tt_data_1:receiver.c(10091): TT907: TT0907: Unique constraint (SESSION) violated at Rowid <0x000000012f62d8b0> -- file "hindex.c", lineno 2529, procedure "sbHixInsert"
15:57:35.55 Err : REP: 3041: tt_data_1:receiver.c(5439): TT16187: Transaction 1317905232/3501279703; Error: transient 0, permanent 1