Integration

Hello,

We have found a memory (and performance) problem with Coherence Web 3.6.1 and Tomcat 6.+ when using jsp-tags.

The problem is that we see a lot of short lived objects being created under load (around 1.6GB/s for 10 threads in jmeter going against the same page as fast as possible). After some investigation we found the source and it was org.apache.catalina.util.DefaultAnnotationProcessor in Tomcat that does a lot of reflection. It is called from the generated JspServlet's when a instance of a jsptag is created to check for and call annotated methods (javax.annotation.PostConstruct, javax.annotation.PreDestroy and others).+

Normally, Jasper2 in Tomcat uses tagpooling so this call to the AnnotationProcessor is called once per tag when the init() method of the JspServlet is called.+

But when the jsp-page has the extends directive (to change the baseclass of JspServlet) tagpooling is disabled for that jsp. The result is that a new instace of the tag is created for each invocation.+

From org.apache.jasper.compiler.Generator.java

snippet from org.apache.jasper.compiler.Generator constructor
+/*+
* Temporary hack. If a JSP page uses the "extends" attribute of the
+* page directive, the _jspInit() method of the generated servlet class+
* will not be called (it is only called for those generated servlets
* that extend HttpJspBase, the default), causing the tag handler pools
* not to be initialized and resulting in a NPE. The JSP spec needs to
* clarify whether containers can override init() and destroy(). For
* now, we just disable tag pooling for pages that use "extends".
*/
+if (pageInfo.getExtends(false) == null) {+
isPoolingEnabled = ctxt.getOptions().isPoolingEnabled();
+} else {+
isPoolingEnabled = false;
+}+

The resulting generated JspServlets method to invoke a tag looks like this:

+private boolean jspxmeth_c_005fforEach_005f0(PageContext jspxpage_context)+
+throws Throwable {+
+PageContext pageContext = jspxpage_context;+
+JspWriter out = jspxpage_context.getOut();+
+// c:forEach+
+org.apache.taglibs.standard.tag.rt.core.ForEachTag jspxth_c_005fforEach_005f0 = new org.apache.taglibs.standard.tag.rt.core.ForEachTag();+
+org.apache.jasper.runtime.AnnotationHelper.postConstruct(_jsp_annotationprocessor, jspxth_c_005fforEach_005f0);+


Compare this to code for the same jsp when tagpooling is used:

+private boolean jspxmeth_c_005fforEach_005f0(PageContext jspxpage_context)+
+throws Throwable {+
+PageContext pageContext = jspxpage_context;+
+JspWriter out = jspxpage_context.getOut();+
+// c:forEach+
+org.apache.taglibs.standard.tag.rt.core.ForEachTag jspxth_c_005fforEach_005f0 = (org.apache.taglibs.standard.tag.rt.core.ForEachTag) _005fjspx_005ftagPool_005fc_005fforEach_0026_005fvarStatus_005fvar_005fitems.get(org.apache.taglibs.standard.tag.rt.core.ForEachTag.class);+



We have examined the code for the baseclass that Coherence uses when you instrument the war and it calls the jspInit() and_jspInit() method from init(ServletConfig cfg), so the NPE problem (as I understand it) would never occur with tagpooling turned on with this special baseclass. The init of the decompiled Coherence Web baseclass looks like this:

snippet from com.tangosol.coherence.servlet.api23.JspServlet.java
public final void init(ServletConfig cfg)
throws ServletException
+{+
+/* 75*/ azzert(cfg != null);+
+/* 78*/ SessionHelper helper = SessionHelper.ensureSessionHelper(cfg.getServletContext());+
+/* 79*/ cfg = helper.wrapServletConfig(cfg);+
+/* 81*/ m_helper = helper;+
+/* 82*/ m_cfg = cfg;+
+/* 84*/ jspInit();+
+/* 85*/ _jspInit();+
+}+

This is more or less the same as the default baseclass in Tomcat

snippet from org.apache.jasper.runtime.HttpJspBase
public final void init(ServletConfig config)
+ throws ServletException+
+{+
super.init(config);
+ jspInit();+
_jspInit();+
+}+


We solved this by patching Generator.java in Tomcat to accept a system property that forces tagpooling for jsps that has the extends directive.
After this we ran the same JMeter test again and the memory allocation went from ~1.6GB/s to ~300mb/s and the throughput (requests/s) doubled.

We have submitted the patch to SpringSource and hopefully this will make it into the trunk of Tomcat. Patch is below (we take no responsiblity for it, us at your own risk).

The other solution that we thought of was to remove the extends from the jsps, but that might break some stuff in Coherence Web. But since we always include the jsps from Servlets that are wrapped by Coherence Web, my guess is that it would work? Would be nice to get some feedback from Oracle on this and some examples if it is possible.




Index: java/org/apache/jasper/compiler/Generator.java
===================================================================
--- java/org/apache/jasper/compiler/Generator.java (revision 1100181)
+++ java/org/apache/jasper/compiler/Generator.java (working copy)
@@ -78,6 +78,12 @@
private static final String VAR_ANNOTATIONPROCESSOR =
System.getProperty("org.apache.jasper.compiler.Generator.VAR_ANNOTATIONPROCESSOR", "_jsp_annotationprocessor");

+ /* System property that makes it possible for jsp pages with the extends
+ * page directive to take advantage of tag pooling.
+ */
+ private static final boolean ENABLEPOOLINGFORJSPSUBCLASSES = Boolean.valueOf(
+ System.getProperty("org.apache.jasper.compiler.Generator.ENABLEPOOLINGFORJSPSUBCLASSES", "false"));
+
/* System property that controls if the requirement to have the object
* used in jsp:getProperty action to be previously "introduced"
* to the JSP processor (see JSP.5.3) is enforced.
@@ -3390,7 +3396,7 @@
* clarify whether containers can override init() and destroy(). For
* now, we just disable tag pooling for pages that use "extends".
*/
- if (pageInfo.getExtends(false) == null) {
+ if (pageInfo.getExtends(false) == null || ENABLEPOOLINGFORJSPSUBCLASSES) {
isPoolingEnabled = ctxt.getOptions().isPoolingEnabled();
} else {
isPoolingEnabled = false;


BR,
Carl Abramsson
www.hm.com