Showing posts with label Jetty. Show all posts
Showing posts with label Jetty. Show all posts
Wednesday, November 7, 2012

Google Closure Compiler with Maven Builds

In my previous post, I showed how we used Closure Library with our Maven-based project. More specifically, we used the Jetty Maven plugin to expose the JS code, and make it available in raw form for a fast and convenient code-run-debug cycle. This time, let me show how we used the Closure Compiler to compile and generate the minified JS file, and how we used it in our builds.

Running Closure Compiler

Running the compiler was a bit tricky. Since we didn't have just one single JS file, we needed to provide the compiler a list of JS files in the correct order (where providing classes are declared before the ones that required them). Good thing the library comes with Python scripts that does just that.

<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>exec-maven-plugin</artifactId>
 <executions>
  <execution>
   <id>calculate-dependencies</id>
   <phase>process-sources</phase>
   <goals>
    <goal>exec</goal>
   </goals>
   <configuration>
    <executable>python</executable>
    <arguments>
     <argument>src/main/js/closure-library/closure/bin/build/depswriter.py</argument>
     <argument>--root_with_prefix</argument>
     <!-- prefix is relative to location of closure-library/closure/goog/base.js -->
     <argument>src/main/js/myapp/ ../../../myapp/</argument>
     <argument>--output_file</argument>
     <argument>src/main/js/myapp/deps.js</argument>
    </arguments>
   </configuration>
  </execution>
  <execution>
   <id>compile-javascript</id>
   <phase>prepare-package</phase>
   <goals>
    <goal>exec</goal>
   </goals>
   <configuration>
    <executable>python</executable>
    <arguments>
     <argument>src/main/js/closure-library/closure/bin/build/closurebuilder.py</argument>
     <argument>--root</argument>
     <argument>src/main/js/closure-library</argument>
     <argument>--root</argument>
     <argument>src/main/js/myapp</argument>
     <argument>--output_mode</argument>
     <argument>compiled</argument>
     <argument>--compiler_jar</argument>
     <argument>src/lib/closure-compiler-20120430.jar</argument>
     <!-- WHITESPACE_ONLY, SIMPLE_OPTIMIZATIONS, ADVANCED_OPTIMIZATIONS -->
     <argument>--compiler_flags=--compilation_level=ADVANCED_OPTIMIZATIONS</argument>
     <!-- QUIET, DEFAULT, VERBOSE -->
     <argument>--compiler_flags=--warning_level=VERBOSE</argument>
     <argument>--namespace</argument>
     <argument>myapp</argument>
     <argument>--output_file</argument>
     <argument>${project.build.directory}/replaced-webapp/scripts/myapp.min.js</argument>
    </arguments>
   </configuration>
  </execution>
 </executions>
</plugin>

I've included the portion that calculates dependencies (from my previous post). Note that use of src/lib/closure-compiler-yyyymmdd.jar. You can download it from here, and extract the compiler.jar file. We just renamed it so we can track which version we're using. And in case of updates, just update the pom.xml.

Note also that we invoke the compiler during the prepare-package phase. This means that when you run jetty:run, the compiler does not get invoked. This makes it faster (and easier to repeat). But when you need to run/debug with the compiled/minified form (with advanced optimization), you run jetty:run-war.

The astute reader would notice that the minified file is currently not being referenced in any web page. This shall be explained in the next section.

Replacing Tags in Web Pages

Since the output of the compiler was one single JS file, we needed to modify our web pages (e.g. JSPs) to refer to it (and not the multiple files). Obviously, we could manually modify the JSP files. But we needed something that would automatically replace them during the build. We decided to use maven-replacer-plugin.

<plugin>
  <groupId>com.google.code.maven-replacer-plugin</groupId>
  <artifactId>maven-replacer-plugin</artifactId>
  <version>1.4.1</version>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>replace</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <ignoreMissingFile>true</ignoreMissingFile>
    <file>src/main/webapp/index.jsp</file>
    <outputFile>${project.build.directory}/replaced-webapp/index.jsp</outputFile>
    <regex>true</regex>
    <regexFlags>
      <regexFlag>MULTILINE</regexFlag>
      <regexFlag>DOTALL</regexFlag>
    </regexFlags>
    <replacements>
      <replacement>
        <token><!-- SCRIPTS.*END SCRIPTS --></token>
        <value><script src="<c:url value='/scripts/myapp.min.js' />" ></script></value>
      </replacement>
    </replacements>
  </configuration>
</plugin>

This effectively replaces the following:

<!-- SCRIPTS -->
<script src="/closure-library/closure/goog/base.js"></script>
<script src="/myapp/deps.js"></script>
<script src="/myapp/myapp.js"></script>
<!-- END SCRIPTS -->
<script>init();</script><!-- our JS code's entry point -->

with

<script src="<c:url value='/scripts/myapp.min.js' />"></script>
<script>init();</script><!-- our JS code's entry point -->

Note that the minified JS file is written to target/replaced-webapp/scripts. But this folder is never used to build the WAR. So, we need to modify our pom.xml to include this folder when building the WAR file.

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-war-plugin</artifactId>
 <configuration>
  <webResources>
   <resource>
    <directory>src/main/webapp</directory>
    <filtering>true</filtering>
   </resource>
   <resource>
    <directory>target/replaced-webapp</directory>
    <filtering>true</filtering>
    <includes>
     <include>**/*.jsp</include>
    </includes>
   </resource>
  </webResources>
 </configuration>
</plugin>

The above configuration effectively adds the target/replaced-webapp directory on to the WAR file. This adds a scripts folder with a minified JS file in it. It also contains the modified index.jsp file. When the web page references /scripts/myapp.min.js, it will find it.

Conclusion

It is great that we're able to debug in raw form. This provides an easier way to find which JS file/line is causing the error. We are also able to debug in minified form, and deploy the minified form. I hope this helps.

Sunday, November 4, 2012

Integrating Google Closure Tools with Maven Builds

We wanted to use Google Closure Tools (compiler, plus the library) in our Maven-based project. The current approaches posted here and here were very good. But didn't quite satisfy our needs. As our JavaScript code base grew in size, and in complexity, we needed to run/debug our web pages with the JavaScript code in their raw form, apart from the minified/compiled mode. So, here's how we did it.

Project Structure (Maven's Standard Directory Layout)

Our project structure is as follows:

  • src/main/java
  • src/main/js
  • src/main/js/closure-library
  • src/main/js/myapp
  • src/main/resources
  • src/test/java
  • src/test/resources
  • src/main/webapp

It follows the standard directory layout. We placed our JavaScript (JS) code under src/main/js. We also placed the Closure Library underneath it. This made it easy to expose as additional web app root directories via the Jetty Maven Plugin. As will be explained later, this was crucial in being able to run the our JS code in raw form, which required the inclusion of base.js and our app's deps.js.

In our Subversion code repository, we linked to the Closure Library's code repository. A similar approach can be applied using git-svn mirror when using Git.

$ svn propget svn:externals
src/main/js/closure-library
http://closure-library.googlecode.com/svn/trunk/

This external link provided src/main/js/closure-library/closure/goog/base.js to be part of our project structure.

Running/Debugging with Closure JavaScript in Raw Form

To have quick code-run-debug cycles, we couldn't afford to compile the JS code every time. So, we had to find a way to run our web pages with the following JS script tags.

<html>
<head>...</head>
<body>
<script src="closure-library/closure/goog/base.js"></script>
<script src="myapp/deps.js"></script>
<script src="myapp/main.js"></script>
<script>init();</script><!-- our JS code's entry point -->
</body>
</html>

To make this possible, we needed to expose the JS files on our servlet container. This was made easier by having our JS code and Closure Library under one folder.

<plugin>
  <groupId>org.mortbay.jetty</groupId>
  <artifactId>jetty-maven-plugin</artifactId>
  <configuration>
    <webAppConfig>
      <contextPath>/</contextPath>
      <baseResource 
          implementation="org.eclipse.jetty.util.resource.ResourceCollection">
        <resourcesAsCSV>src/main/webapp,src/main/js</resourcesAsCSV>
      </baseResource>
    </webAppConfig>
  </configuration>
</plugin>

Calculating Dependencies

Our JS code was not just one file. It had more than a dozen files. Just like the ones in the Closure Library, each class was placed in one file. Each had goog.provide and goog.require calls. For our JS classes to work, we needed to generate a dependency file. The Closure Library comes with its pre-generated deps.js file for its classes. We'll need one for our classes. We used the Python script (that comes with the Closure Library), closure/bin/build/depswriter.py. We chose to invoke it during the process-sources phase of the build. This made it easier when running the jetty:run command, as the dependency file will be re-generated before the servlet container is up and running.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>calculate-dependencies</id>
      <phase>process-sources</phase>
      <goals>
        <goal>exec</goal>
      </goals>
      <configuration>
        <executable>python</executable>
        <arguments>
          <argument>src/main/js/closure-library/closure/bin/build/depswriter.py</argument>
          <argument>--root_with_prefix</argument>
          <!-- prefix is relative to location of closure-library/closure/goog/base.js -->
          <argument>src/main/js/myapp/ ../../../myapp/</argument>
          <argument>--output_file</argument>
          <argument>src/main/js/maypp/deps.js</argument>
        </arguments>
      </configuration>
    </execution>
  </executions>
</plugin>

If there are no changes to the dependencies, you can modify your JS code, and just refresh/reload the web page, and the JS code is also reloaded. This provides a very fast and convenient code-run-debug cycle.

The resulting HTML should have loaded the JS files in the correct order. It should look something like this when viewed using your browser's debug mode (e.g. Firebug, or Webkit's Web Inspector).

<script src="/closure-library/closure/goog/base.js"></script>
<script type="text/javascript" src="http://localhost:8080/closure-library/closure/goog/deps.js"></script>
<script src="/myapp/deps.js"></script>
<script src="/myapp/main.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/disposable/idisposable.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/disposable/disposable.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/debug/error.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/string/string.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/asserts/asserts.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/array/array.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/../../../myapp/finder.js"></script>
<script type="text/javascript"
  src="http://localhost:8080/closure-library/closure/goog/../../../myapp/model.js"></script>
<script type="text/javascript">init();</script><!-- our JS code's entry point -->

It's interesting to point out how the Closure Library adds its deps.js file, and how it adds the dependencies (e.g. idisposable.js, error.js, string.js). The dependencies are due to the JS code requiring these, and the deps.js file adds in the corresponding JS <script> tags. Also note how our app's JS files are referenced — relative to the location of base.js. Look at lines 17-20. This is why we had to use the --root_with_prefix argument when calling depswriter.py. Otherwise, our JS code wouldn't even be loaded.

I'll post another follow-up on this topic where I'll explain how we compile, minify, and test the minified JS code.

Tuesday, May 15, 2012

Decorating Error Pages with SiteMesh 2.4.x

I've encountered a problem when decorating error pages with SiteMesh 2.4.2. I thought that others may benefit from the fix that we've done. So, I'm explaining it here.

When using Jetty 7.x, and the following web.xml configuration, SiteMesh does not correctly decorate the HTTP 404 error page.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://java.sun.com/xml/ns/javaee"
    xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    version="2.5">

    <!-- Sitemesh filter -->
    <filter>
        <filter-name>sitemeshFilter</filter-name>
        <filter-class>com.opensymphony.sitemesh.webapp.SiteMeshFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>sitemeshFilter</filter-name>
        <url-pattern>/*</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
        <dispatcher>ERROR</dispatcher>
    </filter-mapping>

    <error-page>
        <error-code>404</error-code>
        <location>/WEB-INF/pages/not-found.jsp</location>
    </error-page>

</web-app>

I'm not sure if this problem occurs in Tomcat 6.x. From my tests, when a URL to a non-existent page is used (e.g. http://localhost/…/xxx), a 404 page is expected. This should be decorated by SiteMesh. But it's not.

After further investigation, when the SiteMesh filter configuration is limited to *.jsp (see below, line 12), all non-existing URLs that do not match *.jsp do get decorated. So, xxx does get decorated (as expected). But xxx.jsp does not! This inconsistency led to some work-arounds that included using "flash"-scoped variables and a HttpServletResponse#sendRedirect() call to redirect to the 404 page.

<?xml version="1.0" encoding="UTF-8"?>
<web-app ...>

    <!-- Sitemesh filter -->
    <filter>
        <filter-name>sitemeshFilter</filter-name>
        <filter-class>com.opensymphony.sitemesh.webapp.SiteMeshFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>sitemeshFilter</filter-name>
        <url-pattern>*.jsp</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
        <dispatcher>ERROR</dispatcher>
    </filter-mapping>

    <error-page>
        <error-code>404</error-code>
        <location>/WEB-INF/pages/not-found.jsp</location>
    </error-page>

</web-app>

This led me to look further into SiteMesh 2.4.2 code. After adding some debugging statements, I can see that the wrapped response was not used by Jetty in its HttpServletResponse#sendError() implementation. Thus, there was no content in SiteMesh's PageResponseWrapper. It was written directly to the original response, and not the wrapped response. So, in SiteMesh's point of view, there was no content to be decorated. But there is!

Given this finding, I looked for work-arounds. The result was some minor modification to the SiteMeshFilter and the following web.xml configuration.

<?xml version="1.0" encoding="UTF-8"?>
<web-app ...>

    <!-- Sitemesh filter -->
    <filter>
        <filter-name>sitemeshFilter</filter-name>
        <filter-class>com.opensymphony.sitemesh.webapp.SiteMeshFilter</filter-class>
    </filter>
    <filter>
        <filter-name>errorSitemeshFilter</filter-name>
        <filter-class>com.opensymphony.sitemesh.webapp.SiteMeshFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>errorSitemeshFilter</filter-name>
        <url-pattern>/*</url-pattern>
        <dispatcher>ERROR</dispatcher>
    </filter-mapping>
    <filter-mapping>
        <filter-name>sitemeshFilter</filter-name>
        <url-pattern>*.jsp</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
    </filter-mapping>

    <error-page>
        <error-code>404</error-code>
        <location>/WEB-INF/pages/not-found.jsp</location>
    </error-page>

</web-app>

The changes to the SiteMeshFilter allowed multiple instances of the filter to co-exist and act separately. One instance handles ERROR pages (due to sendError() calls). The other instance handles non-error pages. I used the filter's name to differentiate the two instances. When a non-existent URL is encountered, the filter that handles ERROR pages decorates properly. When a non-existent URL that matches *.jsp is encountered, the "sitemeshFilter" gets to filter it. But since it is not existent, the servlet container ends up calling sendError(404). As expected, the "sitemeshFilter" could not handle this properly. The good thing is, we have another "errorSitemeshFilter" that handles this.

public class SiteMeshFilter implements Filter {

    private FilterConfig filterConfig;
    private ContainerTweaks containerTweaks;
    private static final String ALREADY_APPLIED_KEY = "com.opensymphony.sitemesh.APPLIED_ONCE";
    private String alreadyAppliedKey;

    public void init(FilterConfig filterConfig) {
        this.filterConfig = filterConfig;
        alreadyAppliedKey = ALREADY_APPLIED_KEY + "-" + this.filterConfig.getFilterName();
        containerTweaks = new ContainerTweaks();
    }

    public void destroy() {
        filterConfig = null;
        containerTweaks = null;
    }

    /**
     * Main method of the Filter.
     *
     * Checks if the Filter has been applied this request. If not, parses the page
     * and applies {@link com.opensymphony.module.sitemesh.Decorator} (if found).
     */
    public void doFilter(ServletRequest rq, ServletResponse rs, FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequest request = (HttpServletRequest) rq;
        HttpServletResponse response = (HttpServletResponse) rs;
        ServletContext servletContext = filterConfig.getServletContext();

        // no changes
        . . .

        try {

            Content content = obtainContent(contentProcessor, webAppContext, request, response, chain);

            if (content == null) {
                return;
            }

            Decorator decorator = decoratorSelector.selectDecorator(content, webAppContext);
            decorator.render(content, webAppContext);

        } catch (IllegalStateException e) {
            // Some containers (such as WebLogic) throw an IllegalStateException when an error page is served.
            // It may be ok to ignore this. However, for safety it is propegated if possible.
            if (!containerTweaks.shouldIgnoreIllegalStateExceptionOnErrorPage()) {
                throw e;
            }
        } catch (RuntimeException e) {
            if (containerTweaks.shouldLogUnhandledExceptions()) {
                // Some containers (such as Tomcat 4) swallow RuntimeExceptions in filters.
                servletContext.log("Unhandled exception occurred whilst decorating page", e);
            }
            throw e;
        } catch (ServletException e) {
            request.removeAttribute(alreadyAppliedKey);
            throw e;
        }

    }

    // no changes
    . . .

    private boolean filterAlreadyAppliedForRequest(HttpServletRequest request) {
        if (request.getAttribute(alreadyAppliedKey) == Boolean.TRUE) {
            return true;
        } else {
            request.setAttribute(alreadyAppliedKey, Boolean.TRUE);
            return false;
        }
    }

}

This should work for error pages other than 404.

That's all. I hope this helps. Let me know if it works for you.

UPDATE: This is related to http://jira.opensymphony.com/browse/SIM-168.

UPDATE (2013 July): Thanks to Jeremy for pointing out the use of alreadyAppliedKey instead of ALREADY_APPLIED_KEY in the doFilter() method.