Configuration: two hosts with Oracle APEX 21.1.4, ORDS 24.1.1.r1201228 with two database mapped, Tomcat 9.0.88, Oracle JDK 17, nginx as a loadbalancer on a separate host.
Need help in troubleshooting a problem when an application developed with Oracle APEX stops working with HTTP 404 error after successful upgrading ORDS from 21.2 to 24.1. The application uses friendly urls. And, for some unknown reason, that's the problem because as soon as I disable this feature the error goes away. Another applications (APEX Builder including) that do not use friendly urls work ok.
The error I see in catalina.out looks like this one
2024-07-25T22:26:15.528Z WARNING Error occurred when sending response for:
404 GET https://<loadbalancer host>/ords/<database>/<workspace>/r/<application alias>/home
2024-07-25T22:26:15.528Z WARNING java.io.IOException: Broken pipe
org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:345)
at org.apache.catalina.connector.OutputBuffer.appendByteArray(OutputBuffer.java:750)
at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:671)
at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:380)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:358)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:103)
at oracle.dbtools.http.servlet.NormalizedServletResponse$NormalizedOutputStream.write(NormalizedServletResponse.java:924)
at java.base/java.io.OutputStream.write(OutputStream.java:127)
at oracle.dbtools.http.errors.ResponseRenderer$HTMLRenderer.writeHtml(ResponseRenderer.java:519)
at oracle.dbtools.http.errors.ResponseRenderer$HTMLRenderer.render(ResponseRenderer.java:348)
at oracle.dbtools.http.errors.ResponseRenderer.service(ResponseRenderer.java:143)
at oracle.dbtools.http.errors.ErrorPageFilter.doFilter(ErrorPageFilter.java:116)
at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
at oracle.dbtools.http.secure.ForceHttpsFilter.doFilter(ForceHttpsFilter.java:74)
at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
at oracle.dbtools.http.auth.ForceAuthFilter.doFilter(ForceAuthFilter.java:44)
at oracle.dbtools.http.filters.HttpFilter.doFilter(HttpFilter.java:47)
at oracle.dbtools.http.filters.FilterChainImpl.doFilter(FilterChainImpl.java:64)
at oracle.dbtools.http.filters.Filters.filter(Filters.java:67)
at oracle.dbtools.http.entrypoint.EntryPoint.service(EntryPoint.java:70)
at oracle.dbtools.http.entrypoint.EntryPointServlet.service(EntryPointServlet.java:130)
at oracle.dbtools.entrypoint.WebApplicationRequestEntryPoint.service(WebApplicationRequestEntryPoint.java:50)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1786)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
at java.base/java.lang.Thread.run(Thread.java:842)
Caused by: java.io.IOException: Broken pipe
at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:132)
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:97)
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:53)
at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:532)
at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:136)
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1423)
at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:775)
at org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(SocketWrapperBase.java:600)
at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:544)
at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:540)
at org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:73)
at org.apache.coyote.http11.Http11OutputBuffer.doWrite(Http11OutputBuffer.java:193)
at org.apache.coyote.Response.doWrite(Response.java:608)
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:333)
... 46 more
The ORDS upgrade itself was almost successful. All legacy config migrated with no issue and new config looks good for me. This new config was used to generate a war file that was deployed to Tomcat. All pools starts with no errors:
Mapped local pools from <path to config folder>/databases:
/ords/ => default => VALID
/ords/db1/ => db1 => VALID
/ords/db2/ => db2 => VALID
2024-07-25T22:21:20.909Z INFO Oracle REST Data Services initialized
Oracle REST Data Services version : 24.1.1.r1201228
Oracle REST Data Services server info: Apache Tomcat/9.0.88
Oracle REST Data Services java info: Java HotSpot(TM) 64-Bit Server VM 17.0.10+11-LTS-240
Any ideas what is wrong?