Oracle Linux 5.4 with Oracle Grid and Db 11.2.0.4. No iptables running.
The following was working fine for sometime and suddenly stopped working. Not sure what changes were done, or what could have impacted the HTTPS raw listener endpoint.
Support notes were followed and applied, such as "How To Configure SSL For Oracle XML DB (Doc ID 942976.1)", "How To Enable The Secure HTTP Port (HTTPS) in XML DB (Doc ID 942945.1)", "How to Setup Native Oracle XML DB Web Services (Doc ID 444191.1)", and others.
The database instance successfully registers its ports with the local listener:
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xx)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=xx)(PORT=1443))(Presentation=HTTP)(Session=RAW))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xx)(PORT=8080))(Presentation=HTTP)(Session=RAW))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=xx)(PORT=2484)))
The native Oracle web service (PL/SQL function) works without an issue via HTTP (port 8080).
However, ANY access (irrespective of the complete URL content, or query string) on HTTPS fails with a "connection reset". Yes, wallet is configured, contains a user certificate and the rest - both sqlnet.ora and listener.ora (in the GI HOME/network/admin dir) are configured for this wallet. File and dir permissions are valid.
Chrome access for example shows the following error: "ERR_CONNECTION_RESET".
UTL_HTTP web client access from same db server (using same wallet with trusted certs) shows:
ORA-06512: at "SYS.UTL_HTTP", line 1128
ORA-29259: end-of-input reached
Running a SSL trace shows that the client reads from the tcp socket connected to port 1443, and then finds no data to read:
[12-JUN-2015 11:36:37:312] nzospLog: entry
[12-JUN-2015 11:36:37:312] [SSL WRITE] length = 61
[12-JUN-2015 11:36:37:312] --- Decoded Record [subtype = 3] ---
ClientHello[57]
client_version
TLSV1
random[32]
55 7A A8 25 6A 26 E8 AE 41 B8 89 3E 53 35 32 E5
82 69 F7 E4 D9 0F 4A 41 F8 99 2D BB 88 A5 87 94
session_id[0]
cipher_suites[18]
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_MD5
TLS_RSA_WITH_DES_CBC_SHA
TLS_RSA_EXPORT_WITH_RC4_40_MD5
TLS_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RENEGO_PROTECTION_REQUEST
compression_methods[1]
00
...snipped..
[12-JUN-2015 11:36:37:313] nsevfnt: cxd: 0x72967948 stage 0: NT events set:
READ
[12-JUN-2015 11:36:37:313] nsevfnt: cxd: 0x72967948 stage 0: NS events set:
INCOMING SEND
[12-JUN-2015 11:36:37:313] nsevrec: event is 0x2, on 0
[12-JUN-2015 11:36:37:313] nsevwait: 1 posted event(s)
[12-JUN-2015 11:36:37:313] nsevwait: exit (0)
[12-JUN-2015 11:36:37:313] nsevmute: entry
[12-JUN-2015 11:36:37:313] nsevmute: cid=0
[12-JUN-2015 11:36:37:313] nsevmute: normal exit
[12-JUN-2015 11:36:37:313] nsdo: entry
[12-JUN-2015 11:36:37:313] nsdo: cid=0, opcode=68, *bl=4096, *what=0, uflgs=0x0, cflgs=0x3
[12-JUN-2015 11:36:37:313] nsdo: rank=64, nsctxrnk=0
[12-JUN-2015 11:36:37:313] nsdo: nsctx: state=8, flg=0x2000400d, mvd=0
[12-JUN-2015 11:36:37:313] nsdo: reading from transport...
[12-JUN-2015 11:36:37:313] nttrd: entry
[12-JUN-2015 11:36:37:313] nttrd: exit
[12-JUN-2015 11:36:37:313] ntt2err: entry
[12-JUN-2015 11:36:37:313] ntt2err: Read unexpected EOF ERROR on 13
[12-JUN-2015 11:36:37:314] ntt2err: exit
[12-JUN-2015 11:36:37:314] nsdo: transport read error
[12-JUN-2015 11:36:37:314] nserror: entry
[12-JUN-2015 11:36:37:314] nserror: nsres: id=0, op=68, ns=12537, ns2=12560; nt[0]=507, nt[1]=0, nt[2]=0; ora[0]=0, ora[1]=0, ora[2]=0
I thought it could perhaps be due to a problem with the wallet (and/or certificates in it), used by the server (as configured in sqlnet.ora and listener.ora). So I have created a brand new wallet, with a new signed user certificate, with CA root chain as trusted certificates - but the problem remains exactly the same. Which kind of discards the actual certificates in the wallet as the cause.
Next up is Wireshark. But the deeper I am digging down this s/w stack, the deeper and darker the hole becomes.
Will appreciate ideas, comments, suggestions, and different angles, on how to tackle isolating this problem.
Thanks.