Debugging

One of the first inconveniences we encountered was that debugging from your IDE (Eclipse in this case) becomes rather difficult. In order to debug such integration tests, the following steps must be taken:

• Start the container (Jetty in this case) from Maven and set the MAVEN_OPTS environment variable to -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=y
• Start the debugger and attach to the remote JVM where the container is running.
• Make sure tests are pointing to the correct port since you may be running your integration tests against a different port than the standard port (see the Continuous Integration section below for why you would do this).

  This process of course repeats itself each time you need to restart the container, not to mention that you have to setup this process for each web application you are working with. While this process works perfectly well, it tends to be time consuming and often distracting from the real goal of fixing bugs.

  For further details on this process, you can visit the Jetty Wiki page dedicated to the subject.

Database

For a database backed, RESTful web service, a good integration test will start each time from a clean slate. That is, each method within a test class starts with the database in a known state. This can generally be acheived in two ways:

1. Use a local database like MySQL and re-initialize the database with DbUnit or by hand between each test run.
   • Requires everyone to have a local database installed (including your continuous integration server).
   • Does not run as fast as an in-memory database.
2. Use an in-memory database such as HSQLDB and REST calls to cleanup the database between tests.
   • Not practical since resetting the database can usually not be acheived through the exposed REST resources alone.
   • Cumbersome and slow.

   As can be seen from the bullets above, the best case scenario for quick integration tests would be to use an in-memory database. The problem we encountered was that because the container was being started by Maven, it runs in a different JVM than the JUnit tests. This means that if you try and create a connection to the in-memory database, two database will actually exist: one started from the container and one started form your test. As a result, the best and really only option was the local database installation, despite it's downfalls. As a result, every developer and the continuous integration server required a local MySQL database installation.

Continuous Integration

Runing integration tests during a continuous integration build is essential for such a product. However doing so brought up two major issues:

1. The continuous integration server needed to have a locally installed database just like each developer machine.
2. Each container setup to run on a continuous integration server needs to have it's own ports defined.

   For the first point, it's clear that using an in-memory database would null and void this issue. We have already addressed the reasons why an in-memory database is not reasonable when running the container and tests in separate JVMs, so we shall move on to the next point.

   When running in a shared environment like a continuous integration server, not only must each project that starts up a container need it's own ports, but any one project that happens to run its' integration tests at the same time will end up with port bind problems and the infamous: Address already in use . The only way to avoid this is to maintain a list of ports that each team and web application might be using on a particular continuous integration server. This can of course become cumbersome and will likely fall out of date quite rapidly. As for running two builds simultaneously, this is just something that we had to live with and were forced to sometimes see false build failures.

Debugging

Since the container now runs in the same JVM as your tests, debugging changes from a multi-step process to a walk in the park. We simply run our test classes (or even just a single test method) from Eclipse and we're ready to go. No need to attach remote debuggers or manage the container from Maven. Simple, easy and fast.

Database

Now that our integration tests are running in the same JVM as the container, we have the added bonus of sharing any object instances with the container. Jitr has built in support for Spring and sharing the web application's Spring application context with the integration tests. Since we were using DbUnit, we can now easily inject the same DataSource that the container has and pass this to DbUnit. This DataSource can now even be a connection to an in-memory database like HSQLDB since we share the exact same bean instance as the container. This made our integration tests faster still and made developer setups and maintenance easier than with a locally installed MySQL.

Continuous Integration

Jitr completely controls the container during your integration tests, so we added the ability for Jitr to choose a random, unused port instead of the default port that many other builds might also be using. In order to make the integration tests work properly though, they need to somehow be told of the port that the container finally ends up running on. Jitr does this by injecting the port and/or the base URI (host name, port and context path) directly into your test using special Jitr annotations. Once again, Jitr freed the developers from having to deal with the annoying complexity of integration testing and has alleviated any real chance of port conflicts on the shared continuous integration server.

System Integration Tests

Jetty is a very nice, fast and easy to use container for testing, as is HSQLDB a good database for testing, but neither of these was the target production software. As such, we needed to be able to also do an integration test run with a Cargo controlled Tomcat container running against a MySQL NDB cluster . To do this, a special "system integration tests" profile was added to the web application's Maven pom.xml .

        <profile>
            <id>systemIntegrationTests</id>
            <build>
                <plugins>
                    <!-- Surefire plugin -- set Jitr system properties -->
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>integration-test</id>
                                <phase>integration-test</phase>
                                <goals>
                                    <goal>test</goal>
                                </goals>
                                <configuration>
                                    <systemProperties>
                                        <jitr.mode>EXTERNAL</jitr.mode>
                                        <jitr.port>8081</jitr.port>
                                    </systemProperties>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    
                    <!-- Cargo plugin -- run for Jitr external integration tests -->
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                        <version>1.0</version>
                        <configuration>
                            <container>
                                <containerId>tomcat6x</containerId>
                                <zipUrlInstaller>
                                    <url>http://apache.mirroring.de/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip</url>
                                    <installDir>${cargo.install.tomcat6x.dir}</installDir>
                                </zipUrlInstaller>
                                <output>${cargo.container.log}</output>
                                <append>false</append>
                                <log>${cargo.log}</log>
                                <systemProperties>
                                    <log4j.configuration>file:./src/test/resources/log4j.xml</log4j.configuration>
                                </systemProperties>
                            </container>
                            <configuration>
                                <home>${cargo.home.tomcat6x.dir}</home>
                                <properties>
                                    <cargo.servlet.port>${container.port}</cargo.servlet.port>
                                </properties>
                            </configuration>
                        </configuration>
                        <executions>
                            <execution>
                                <id>start-container</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>start</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>stop-container</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>stop</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>

            <properties>
                <cargo.install.dir>${java.io.tmpdir}/cargo</cargo.install.dir>
                <cargo.install.tomcat6x.dir>${cargo.install.dir}/tomcat6x</cargo.install.tomcat6x.dir>
                <cargo.home.dir>${project.build.directory}/cargo</cargo.home.dir>
                <cargo.home.tomcat6x.dir>${cargo.home.dir}/tomcat6x</cargo.home.tomcat6x.dir>
                <cargo.log>${cargo.home.dir}/cargo.log</cargo.log>
                <cargo.container.log>${cargo.home.dir}/container.log</cargo.container.log>
            </properties>
        </profile>

When the Maven integration test phase is executed with this profile (mvn integration-test -PsystemIntegrationTests ), Jitr will not run the container and will use the port set through it's system property. Since we are now no longer running the container in the same JVM as the tests, our in-memory database solution would no longer work anyways, so using the MySQL NDB cluster was necessary in more ways than one. We can no longer share the web application's Spring application context, so Jitr also needs to load a test context configuration. This is defined on the test using Spring's standard @ContextConfiguration annotation. We are now completely setup for two type of integration tests and are able to run with either Jitr and an in-memory database, or from Maven with a MySQL NDB cluster.