Spring MVC Slow on Google App Engine for Java?

We previously tried deploying Spring-based Java web applications on Google's App Engine (GAE). And I remember abandoning it due to the time it takes to handle a loading request.

Now, I was able to get some time to create a minimal Spring MVC web application and see how it performs on GAE. Below is the result. The web.xml loads the ContextLoaderListener and DispatcherServlet. The application context is empty (i.e. no beans). The dispatcher-servlet.xml contains a <context:component-scan /> tag and a ViewResolver. It takes about 6.75 seconds for a loading request (approximately cpu_ms=3600).

6.75 seconds for a loading request is probably not much. But the 1100 cpu_ms by slim3 is sure enviable. I also found this article helpful in explaining the cause of increase in loading request time.

Here's some detailed log entries:

javax.servlet.ServletContext log: Initializing Spring root WebApplicationContext
…
org.springframework.web.context.ContextLoader initWebApplicationContext: Root WebApplicationContext:
initialization completed in 2420 ms
…
javax.servlet.ServletContext log: Initializing Spring FrameworkServlet 'dispatcher'
org.springframework.web.servlet.FrameworkServlet initServletBean: FrameworkServlet 'dispatcher': initialization started
…
org.springframework.web.servlet.FrameworkServlet initServletBean: FrameworkServlet 'dispatcher':
initialization completed in 2995 ms

Looks like the DispatcherServlet eats up some processing time. If we just use the ContextLoader, and plain-vanilla servlets, the loading request time should be tolerable.

Google Closure Compiler with Maven Builds

In my previous post, I showed how we used Closure Library with our Maven-based project. More specifically, we used the Jetty Maven plugin to expose the JS code, and make it available in raw form for a fast and convenient code-run-debug cycle. This time, let me show how we used the Closure Compiler to compile and generate the minified JS file, and how we used it in our builds.

Running Closure Compiler

Running the compiler was a bit tricky. Since we didn't have just one single JS file, we needed to provide the compiler a list of JS files in the correct order (where providing classes are declared before the ones that required them). Good thing the library comes with Python scripts that does just that.

<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>exec-maven-plugin</artifactId>
 <executions>
  <execution>
   <id>calculate-dependencies</id>
   <phase>process-sources</phase>
   <goals>
    <goal>exec</goal>
   </goals>
   <configuration>
    <executable>python</executable>
    <arguments>
     <argument>src/main/js/closure-library/closure/bin/build/depswriter.py</argument>
     <argument>--root_with_prefix</argument>
     <!-- prefix is relative to location of closure-library/closure/goog/base.js -->
     <argument>src/main/js/myapp/ ../../../myapp/</argument>
     <argument>--output_file</argument>
     <argument>src/main/js/myapp/deps.js</argument>
    </arguments>
   </configuration>
  </execution>
  <execution>
   <id>compile-javascript</id>
   <phase>prepare-package</phase>
   <goals>
    <goal>exec</goal>
   </goals>
   <configuration>
    <executable>python</executable>
    <arguments>
     <argument>src/main/js/closure-library/closure/bin/build/closurebuilder.py</argument>
     <argument>--root</argument>
     <argument>src/main/js/closure-library</argument>
     <argument>--root</argument>
     <argument>src/main/js/myapp</argument>
     <argument>--output_mode</argument>
     <argument>compiled</argument>
     <argument>--compiler_jar</argument>
     <argument>src/lib/closure-compiler-20120430.jar</argument>
     <!-- WHITESPACE_ONLY, SIMPLE_OPTIMIZATIONS, ADVANCED_OPTIMIZATIONS -->
     <argument>--compiler_flags=--compilation_level=ADVANCED_OPTIMIZATIONS</argument>
     <!-- QUIET, DEFAULT, VERBOSE -->
     <argument>--compiler_flags=--warning_level=VERBOSE</argument>
     <argument>--namespace</argument>
     <argument>myapp</argument>
     <argument>--output_file</argument>
     <argument>${project.build.directory}/replaced-webapp/scripts/myapp.min.js</argument>
    </arguments>
   </configuration>
  </execution>
 </executions>
</plugin>

I've included the portion that calculates dependencies (from my previous post). Note that use of src/lib/closure-compiler-yyyymmdd.jar. You can download it from here, and extract the compiler.jar file. We just renamed it so we can track which version we're using. And in case of updates, just update the pom.xml.

Note also that we invoke the compiler during the prepare-package phase. This means that when you run jetty:run, the compiler does not get invoked. This makes it faster (and easier to repeat). But when you need to run/debug with the compiled/minified form (with advanced optimization), you run jetty:run-war.

The astute reader would notice that the minified file is currently not being referenced in any web page. This shall be explained in the next section.

Replacing Tags in Web Pages

Since the output of the compiler was one single JS file, we needed to modify our web pages (e.g. JSPs) to refer to it (and not the multiple files). Obviously, we could manually modify the JSP files. But we needed something that would automatically replace them during the build. We decided to use maven-replacer-plugin.

<plugin>
  <groupId>com.google.code.maven-replacer-plugin</groupId>
  <artifactId>maven-replacer-plugin</artifactId>
  <version>1.4.1</version>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>replace</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <ignoreMissingFile>true</ignoreMissingFile>
    <file>src/main/webapp/index.jsp</file>
    <outputFile>${project.build.directory}/replaced-webapp/index.jsp</outputFile>
    <regex>true</regex>
    <regexFlags>
      <regexFlag>MULTILINE</regexFlag>
      <regexFlag>DOTALL</regexFlag>
    </regexFlags>
    <replacements>
      <replacement>
        <token><!-- SCRIPTS.*END SCRIPTS --></token>
        <value><script src="<c:url value='/scripts/myapp.min.js' />" ></script></value>
      </replacement>
    </replacements>
  </configuration>
</plugin>

This effectively replaces the following:

<!-- SCRIPTS -->
<script src="/closure-library/closure/goog/base.js"></script>
<script src="/myapp/deps.js"></script>
<script src="/myapp/myapp.js"></script>
<!-- END SCRIPTS -->
<script>init();</script><!-- our JS code's entry point -->

with

<script src="<c:url value='/scripts/myapp.min.js' />"></script>
<script>init();</script><!-- our JS code's entry point -->

Note that the minified JS file is written to target/replaced-webapp/scripts. But this folder is never used to build the WAR. So, we need to modify our pom.xml to include this folder when building the WAR file.

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-war-plugin</artifactId>
 <configuration>
  <webResources>
   <resource>
    <directory>src/main/webapp</directory>
    <filtering>true</filtering>
   </resource>
   <resource>
    <directory>target/replaced-webapp</directory>
    <filtering>true</filtering>
    <includes>
     <include>**/*.jsp</include>
    </includes>
   </resource>
  </webResources>
 </configuration>
</plugin>

The above configuration effectively adds the target/replaced-webapp directory on to the WAR file. This adds a scripts folder with a minified JS file in it. It also contains the modified index.jsp file. When the web page references /scripts/myapp.min.js, it will find it.

Conclusion

It is great that we're able to debug in raw form. This provides an easier way to find which JS file/line is causing the error. We are also able to debug in minified form, and deploy the minified form. I hope this helps.

Integrating Google Closure Tools with Maven Builds

We wanted to use Google Closure Tools (compiler, plus the library) in our Maven-based project. The current approaches posted here and here were very good. But didn't quite satisfy our needs. As our JavaScript code base grew in size, and in complexity, we needed to run/debug our web pages with the JavaScript code in their raw form, apart from the minified/compiled mode. So, here's how we did it.

Project Structure (Maven's Standard Directory Layout)

Our project structure is as follows:

• src/main/java
• src/main/js
• src/main/js/closure-library
• src/main/js/myapp
• src/main/resources
• src/test/java
• src/test/resources
• src/main/webapp

It follows the standard directory layout. We placed our JavaScript (JS) code under src/main/js. We also placed the Closure Library underneath it. This made it easy to expose as additional web app root directories via the Jetty Maven Plugin. As will be explained later, this was crucial in being able to run the our JS code in raw form, which required the inclusion of base.js and our app's deps.js.

In our Subversion code repository, we linked to the Closure Library's code repository. A similar approach can be applied using git-svn mirror when using Git.

$ svn propget svn:externals
src/main/js/closure-library
    http://closure-library.googlecode.com/svn/trunk/

This external link provided src/main/js/closure-library/closure/goog/base.js to be part of our project structure.

Running/Debugging with Closure JavaScript in Raw Form

To have quick code-run-debug cycles, we couldn't afford to compile the JS code every time. So, we had to find a way to run our web pages with the following JS script tags.

<html>
<head>...</head>
<body>
<script src="closure-library/closure/goog/base.js"></script>
<script src="myapp/deps.js"></script>
<script src="myapp/main.js"></script>
<script>init();</script><!-- our JS code's entry point -->
</body>
</html>

To make this possible, we needed to expose the JS files on our servlet container. This was made easier by having our JS code and Closure Library under one folder.

<plugin>
  <groupId>org.mortbay.jetty</groupId>
  <artifactId>jetty-maven-plugin</artifactId>
  <configuration>
    <webAppConfig>
      <contextPath>/</contextPath>
      <baseResource 
          implementation="org.eclipse.jetty.util.resource.ResourceCollection">
        <resourcesAsCSV>src/main/webapp,src/main/js</resourcesAsCSV>
      </baseResource>
    </webAppConfig>
  </configuration>
</plugin>

Calculating Dependencies

Our JS code was not just one file. It had more than a dozen files. Just like the ones in the Closure Library, each class was placed in one file. Each had goog.provide and goog.require calls. For our JS classes to work, we needed to generate a dependency file. The Closure Library comes with its pre-generated deps.js file for its classes. We'll need one for our classes. We used the Python script (that comes with the Closure Library), closure/bin/build/depswriter.py. We chose to invoke it during the process-sources phase of the build. This made it easier when running the jetty:run command, as the dependency file will be re-generated before the servlet container is up and running.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>calculate-dependencies</id>
      <phase>process-sources</phase>
      <goals>
        <goal>exec</goal>
      </goals>
      <configuration>
        <executable>python</executable>
        <arguments>
          <argument>src/main/js/closure-library/closure/bin/build/depswriter.py</argument>
          <argument>--root_with_prefix</argument>
          <!-- prefix is relative to location of closure-library/closure/goog/base.js -->
          <argument>src/main/js/myapp/ ../../../myapp/</argument>
          <argument>--output_file</argument>
          <argument>src/main/js/maypp/deps.js</argument>
        </arguments>
      </configuration>
    </execution>
  </executions>
</plugin>

If there are no changes to the dependencies, you can modify your JS code, and just refresh/reload the web page, and the JS code is also reloaded. This provides a very fast and convenient code-run-debug cycle.

The resulting HTML should have loaded the JS files in the correct order. It should look something like this when viewed using your browser's debug mode (e.g. Firebug, or Webkit's Web Inspector).

<script src="/closure-library/closure/goog/base.js"></script>
<script type="text/javascript" src="http://localhost:8080/closure-library/closure/goog/deps.js"></script>
<script src="/myapp/deps.js"></script>
<script src="/myapp/main.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/disposable/idisposable.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/disposable/disposable.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/debug/error.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/string/string.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/asserts/asserts.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/array/array.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/../../../myapp/finder.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/../../../myapp/model.js"></script>
<script type="text/javascript">init();</script><!-- our JS code's entry point -->

It's interesting to point out how the Closure Library adds its deps.js file, and how it adds the dependencies (e.g. idisposable.js, error.js, string.js). The dependencies are due to the JS code requiring these, and the deps.js file adds in the corresponding JS <script> tags. Also note how our app's JS files are referenced — relative to the location of base.js. Look at lines 17-20. This is why we had to use the --root_with_prefix argument when calling depswriter.py. Otherwise, our JS code wouldn't even be loaded.

I'll post another follow-up on this topic where I'll explain how we compile, minify, and test the minified JS code.