Using mocking to keep development agile

We had built up a mocked service layer for our Wicket Tester unit tests (with Mockito). This enables our unit test to start any page or panel in our app with reasonable test data. However, when it came to development, we were still using the real service layer – which meant that as our application got bigger it took longer and longer to start and navigate to the page under development.

Why not use the mocked services during development too? Good idea, we thought. We added development entry points to our unit tests which allow us to start a minimal Jetty container to run the pages under development in a mocked service sandbox.

Turns out this works great – start time goes down from 10 seconds to 1 second and you have all the power of the mock scenarios already developed for unit tests during development.

This not only saves time during development, but encourages better unit tests (since you build better mock scenarios during development and can then reuse them for unit tests).

It also finally allows us to decouple application development from service development, meaning that the application developers can start immediately with application development and contribute to the definition of the service layers from a client perspective by creating mocks as needed.

Using Maven for production deployment

We use Maven to manage dependencies during development. This entails adding a pom.xml file to our Eclipse project which defines the jars which the application depends. Maven then takes care of fetching the right version of the jars from a number of repositories (central maven repository, vendor specific repositories, our own repository).

This works pretty well and its hard to imagine developing complex projects without this capability. However, when it comes to ensuring that an application is delivered to the production environment with all its dependencies, you’re pretty much on your own. You have to build either a war file or a jar-with-dependencies – both of which can be very tricky and lead to problems occurring in production which you never saw during development.

Additionally, our applications tend to have a lot of dependencies and the war files get huge.

So, we thought, why not just use Maven on production servers to fetch applications and their dependencies.

To do this we maintain a pom.xml file on the production server with the application listed as a dependency. We use the maven goals “versions:use-latest-releases” and “versions:commit” to update the pom file automatically to the latest release version. We then use the “dependency:build-classpath” goal to build a class path from the repository and finally run the application.

Abbreviated component paths in wicket tester

Wicket tester combined with Mockito is fantastic for unit testing your web application. However, developing the tests is harder than it should be because you have to refer to components by their path rather than their id.

For example, to clock the OK button on a page, I need something like:

tester.executeAjaxEvent(“border:form:pagetitlecontainer:ok”. “onclick”);

i.e. instead of “ok”, I must refer to the button by its full path “border:form:pagetitlecontainer:ok”. This seems tedious – if I know that I only have one component on the page with the id “ok”, why can’t I just use “ok”. Wicket tester should notice that the component is not available and traverse the component hierarchy for it. If its ambiguous then it can complain, but if not, then it should be OK.

To get around this limitation, we override the getComponentFromLastRenderedPage method of the WicketTester class as follows:

[sourcecode language=”java”]
public Component getComponentFromLastRenderedPage(final String path) {
// first check if we can find the component with the specified path – if not, check if its an abbreviated path
String fullPath = lookupPath(getLastRenderedPage(), path);
if (fullPath == null)
return null;

return super.getComponentFromLastRenderedPage(fullPath);

public String lookupPath(final MarkupContainer markupContainer, final String path) {
// try to look it up directly
if (markupContainer.get(path) != null)
return path;

// if that fails, traverse the component hierarchy looking for it
final List candidates = new ArrayList();
markupContainer.visitChildren(new IVisitor() {
private static final long serialVersionUID = 1L;

Set visited = new HashSet();

public Object component(Component c) {
if (!visited.contains(c)) {

if (c.getId().equals(path))
// if its unambiguous, then return the full path
if (candidates.isEmpty()) {
fail(“path: ‘” + path + “‘ not found for ” +
return null;
} else if (candidates.size() == 1) {
String pathToContainer = markupContainer.getPath();
String pathToComponent = candidates.get(0).getPath();
return pathToComponent.replaceFirst(pathToContainer + “:”, “”);
} else {
String message = “path: ‘” + path + “‘ is ambiguous for ” + Classes.simpleName(markupContainer.getClass()) + “. Possible candidates are: “;
for (Component c : candidates) {
message += “[” + c.getPath() + “]”;
return null;

The Palchinsky Principles

My holiday reading this year was “Adapt” by Tim Harford. One of the most interesting parts was about Peter Palchinsky, a great russian engineer who was an advisor to both the Tsar and to Stalin. Although his uncompromising honesty got him exiled to Siberia by the Tsar, pardoned again and finally murdered by Stalin’s secret police, he nevertheless had time to formulate 3 principles for innovation:

  1. Try lots of things, expecting many to fail.
  2. Make sure the failures are survivable.
  3. Learn from the failures.

As Tim Harford points out, this is roughly how evolution works in nature and it seems applicable to software development. Since customers don’t generally appreciate failures on their projects, this trial and error cycle needs to be carried out outside of production projects – in the “20%” projects we do on our own time.

Wicket needs a component catalog

I’m a big fan of Apache Wicket, but there’s an urgent need for a better-organized component-ecosystem around Wicket. To write a real web application you need a framework (that’s Wicket) and visual components (that’s JQuery, YUI, Recaptcha etc) and you need the two to work together smoothly.

A newcomer to Wicket should be impressed by the clean, component-oriented approach to producing the GUI for his application, but a production web application needs more than just HTML. So, inevitably, the next step is to try to find the stuff which will make your web-application look like all those other web 2.0 sites out there. However, very few of the Wicket components referenced from the Wicket home pages are “best-of-breed” and a certain amount of disappointment soon sets in.

Take the wicket example captcha implementation for example. There are two problems here: (a) the implementation produces a captcha image which is not only difficult for robots to read, but also pretty impossible for humans and (b) everybody else is already using recaptcha so why even bother distracting developers with yet-another home-spun implementation.

The Wicket community might react by saying “well, there are enough blogs describing how to use recaptcha with Wicket, so where’s the problem?”. Well the problem is that these blogs typically provide their description in a way which reflects their own expertise rather than that of the reader, which means that a novice Wicket programmer may or may not be able to follow the instructions to successfully integrate recaptcha into his application.

Or to put it another way, developers blogs are not a substitute for organized, tested and documented component libraries.

What could help, on the other hand, is a catalog of components, with ratings and a compatibility matrix indicating tested interoperability with other components/browsers etc.

By the way what prompted me to write this post was that yesterday I finally used a few of the Visural Wicket components (based on JQuery) and they are fantastic – well thought out APIs, styling etc. What I don’t understand is that mediocre-to-lousy versions of equivalent components (such as the Wicket extensions modal window or the wicketstuff lightbox) feature much more prominently on the Wicket sites, so the novice programmer is far more likely to use them (and be bitterly disappointed) than to use Visural Wicket (and be delighted).

Wicket adoption would likely be accelerated if the end-to-end experience of producing the first production web application involved a bit less trial-and-error and a bit more delight.

Shadowed variables in inner classes

Yesterday we had a bug caused by the following situation: a class X had a variable “user”. Class X had an inner, anonymous class extending class Y. Inside the inner class a reference was made to the “user” variable from the enclosing class X. Somebody then added a protected variable “user” to class Y. The java compiler silently updated the reference to “user” within the inner class from X.user to Y.user.

This is pretty nasty and I hoped that some of the code-checking tools would warn about about it, so I checked the code with PMD, Findbugs and Checkstyle. Unfortunately, none of them flagged the issue.

Deciphering anonymous class references like MyPage$1$1$2

If you work with anonymous classes a lot (as Wicket applications do), you sometimes get confronted with compiler/tool messages referring to anonymous (inner) classes with names like MyPage$1$1$2 (for instance in FindBugs reports). Its pretty tedious to figure out which class is meant. Maybe there’s a better way, but what I do is go to the target directory and call “javap MyPage$1$1$2.class”. This produces a decompiled version of the class which makes it pretty clear which class it is.

Wicket 1.4 and browser tabs

We had an amazingly annoying problem in a Wicket application. A specific user was continuously having problems with ajax controls on pages (search fields, auto-complete fields etc). The problems were caused by PageExpiredExceptions. We couldn’t understand why only this one user had these problems. This went on for ages, until today I found out that Wicket 1.4 sets a default limit of 5 page maps per session. This specific user typically worked with multiple browser tabs on the application and once he went over 5, some of the page maps got evicted and the ajax stuff started failing.

The solution was to call “getSessionSettings().setMaxPageMaps(100)” to allow up to 100 page maps per session.

monit – trust is good, control better

monit is a very cool system for keeping your linux servers working – highly recommended. With a few lines of configuration, you can have it check any aspect of your system and services and when problems occur, have it alert or take remedial actions (like restarting services, cleaning up log or temporary files etc).

For example, we had the problem that we are running clustered web apps (using Terracotta) in VMware VMs. The cluster nodes were being suspended regularly for backups and this caused them to be evicted from the cluster. A simple solution was to use monit to monitor the apps (via the same health check port we were using for the availability check for the HAProxy load balancer) and to restart the services if the health checkĀ fails (as happens after the VM is unsuspended after the backup).

Here’s the line from the monitrc file we use to monitor the health check port (9000 in this case):

check host webapp with address localhost
if failed port 9000 protocol http with timeout 2 seconds then exec “/webapp/”
The web app in this case is a java wicket app, running under jetty which also runs a health check on port 9000. A http query to this port causes the app to check its connection to its database and its cluster node status. If either fails, it returns a http error status. This health check is used by HAProxy (which takes failed nodes out of the load-balanced pool) and by monit (which restarts the services). This combination provides us with 100% uptime for these apps.

Java logging in development and production

Logging is tricky. Even some major open-source projects don’t do it correctly, so if you use their libraries, you end up with log files you didn’t ask for cluttering your machine.

Current best practices are to use a logging facade like commons-logging or slf4j to avoid these kind of problems by allowing libraries to conform to whatever logging strategy the application which uses the library is using. This means that if your app logs to myapp.log, then the library using slf4j will also log to myapp.log.

Here’s how we use slf4j in our projects:

Our libraries use slf4j-api – here’s the maven dependency you’ll need:

[code lang=”xml”]



Our applications use slf4j-log4j12 – here’s the maven dependency you’ll need:

[sourcecode lang=”xml”]



The code to log looks like this (its the same in libraries or applications):

[sourcecode lang=”java”]
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MyClass {
static Logger log = LoggerFactory.getLogger(MyClass.class);

public Application() {“some logging info”);

We want to have simple console logging during development. To do this we use a simple file containing only a console appender as shown below.

[sourcecode lang=”java”]
log4j.rootLogger=INFO, Stdout
log4j.appender.Stdout.layout.conversionPattern=%-5p – %-26.26c{1} – %mn

We put this file in a Settings folder in our home directory.

During development we need to get log4j to load this file so we get console logging. We do this by defining a system property “log4j.configuration” in the Eclipse Preferences/Installed JREs/Edit/Default VM Arguments (that way it applies for all development projects):

[sourcecode lang=”java”]

During production, we do this same thing, but this time we pass a log4j configuration with a rolling file appender as shown below:

[sourcecode lang=”java”]
log4j.rootLogger=INFO, R
log4j.appender.R.layout.ConversionPattern=%d %p %t %c – %m%n

So the startup in production looks something like this:

[sourcecode lang=”java”]
java -Dlog4j.configuration=file:///home/administrator/ -jar myapp.jar