blog.armstrongconsulting.com

I had a hard time figuring out what was required to cluster our Wicket apps. None of the Wicket books discuss it in any detail – presumably because its not considered a Wicket-specific topic. I bothered the guys in the ##Wicket IRC Channel for a while and Victor Igumnov helped me figure it out and provided me with all the Wicket and Jetty pieces I needed.

Before I start, I should explain that we’re running our Wicket apps with embedded Jetty (”don’t run your apps in Jetty, run Jetty in your apps”). We really like it because it essentially turns web apps into plain old java apps which you can develop, debug and deploy without all that tomcat clutter. So we wanted clustering solution which worked with embedded Jetty, preferably in an uncomplicated fashion.

My first idea was to use Hazelcast – we’re already using it for distributed locking and queuing (its a paragon of simplicity, exemplifying the “it just works” principle). However, I was unable to find exactly what I needed based on Hazelcast, so I followed Victor’s advice and used memcached instead.

You need three things to cluster a Wicket app:
(1) A load balancer – we use HAProxy (as does everybody else).
(2) A distributed map to store the application sessions – we’re using memcached here.
(3) A http session implementation which stores the session in the distributed map instead of locally.

Since the main point of clustering is fault-tolerance, you need to make sure that you avoid single points of failure in the system. Unfortunately, this means that you have to cluster not only your application, but also provide failover for HAProxy and for memcached. I won’t cover that here, but you just need to be aware that when you install those components, you’ll need to install at least two of each with failover. This is simple for memcached – for HAProxy its a bit harder, but at least you can reuse this infrastructure for multiple apps.

HAProxy is the load balancer which will take care of presenting one address for your app to clients, even though your app is running as a cluster of servers. I’m not going to explain how to configure HAProxy here except to say that you’ll need session affinity (aka sticky-sessions) turned on – this means that users stay on a server for a whole session unless the server goes down. Wicket prefers sticky-session load balancing, and its more efficient, so its best to use it.

Installing memcached is no problem. Just follow the instructions. The default configuration only allows connections from localhost, so you’ll need to allow access from the subnet containing the app servers, but otherwise there’s nothing to configure.

Then you need to have your http session persist itself to memcached instead of to the local machine. Frsi download Victor’s tcache distributed cache client library from http://github.com/victori/tcache. Then get Victor’s jetty-session-cache from http://github.com/victori/jetty-session-cache. Add the built jars to your project (or maven repository). Victor describes how to configure jetty via XML to use the MemcachedSessionManager. Since we confgure the embedded Jetty programatically, we needed the following line in our Jetty startup code:

 
context.setSessionHandler(
  new SessionHandler(
     new MemcachedSessionManager(memcachedServers, "my_app_session_store")));

where memcachedServers is a String[] containing a list of your memcached servers, like “server1:11211″, “server2:11211″.

You then need to tell Wicket to store its pages in the http session instead of the default disk-based page store. However, this apparently doesn’t work very efficiently, so the recommended approach is to use a custom page store which stores its pages directly to memcached instead of using the http session.

Victor has a solution here too, go to http://letsgetdugg.com/2010/02/07/clustering-wicket-for-fun-and-profit/ and follow the instructions to override your applications newSessionStore() and provide a memcached page store.

Now your Wicket app is clustered. You can try it out by accessing it via HAProxy, logging on to your app, killing the server with your app’s session and then continue using the app (for test purposes, I display the server running the app session, so I can see when it fails over).

I’d still like to replace memcached with Hazelcast (fully java-based embedded clustering in your app with no single point of failure)- maybe Victor will embrace Hazelcast and add it to the list of caches supported by tcache.

=================================
26.08.2010
I got a mail from Talip Ozturk of Hazelcast to say that Hazelcast supports the memcached protocol, so it should be possible to use a memcached session and page store implementation against a Hazelcast server. This would be fantastic, since it would eliminate the need to have multiple memcached servers – one less piece of infrastructure to worry about. I’ll try to get this working and let you know how I get on.

We are now deploying to our staging servers from our Maven repository using wget (the url is like ‘http://maven:8080/nexus/service/local/artifact/maven/redirect? r=snapshots&g=com.armstrongconsulting.controlcenter.server &a=controlcenterwebapp&v=LATEST&e=jar&c=jar-with-dependencies’.). The runnable jars (or wars) in the Maven repository are built, unit-tested and deployed to Maven by Continuum automatically. If any errors are found, the developer who made the offending commit to SVN is informed by Continuum. Works great, highly recommended. Thanks to Gabriel for setting it all up.

Our first approach was to create a DefaultFocusBehavior (as described by Julian Sinai) and attach it to the component which should receive the focus. The problem with this is that after a validation error during a submit, you usually want the focus to jump to the first invalid field, so the user can easily correct his erroneous input. The problem with the DefaultFocusBehavior is that the component to which you added it still has it and will try to grab the focus again. We created two methods for handling focus in our BasePage class (from which all of our application pages are inherited) – setFocus and setFocusToFirstInvalidComponent. The first one is called during page initialization and the second on validation failure. They share enough class variables which allow them to ensure that only one component has the DefaultFocusBehavior at a time.

I needed to validate a list of email addresses entered into a Wicket text area (a cc: list of email addresses). To do this, I created a simple form validator which breaks up the list from the text area into individual email addresses and then uses the EmailAddressValidator to validate them individually. It illustrates a few Wicket techniques: (a) how to validate a field containing several values which need to be validated individually and (b) how to use a Validator against a string instead of a form component. Anyway, here’s the code in case its of use to anyone.

class CCValidator extends AbstractFormValidator implements Serializable {
	private static final long serialVersionUID = 1L;
	private final TextArea<String> cc;
 
	public CCValidator(TextArea<String> cc) {
		this.cc = cc;
	}
 
	public FormComponent<?>[] getDependentFormComponents() {
		return new FormComponent[] { cc };
	}
 
	public void validate(Form<?> form) {
		String cc_entered = cc.getConvertedInput();
		String[] ccList = cc_entered.split("\r\n");
		for (String email : ccList) {
			if (!isValidEmailAddress(email)) {
				Map<String, Object> map = super.variablesMap();
				map.put("email_address", email);
				error(cc, "invalid_cc", map);
			}
		}
	}
 
	private boolean isValidEmailAddress(final String emailAddress) {
		Validatable<String> v = new Validatable<String>(emailAddress);
		EmailAddressValidator.getInstance().validate(v);
		return v.isValid();
	}
}

To use it, add it to the form as follows:

form.add(new CCValidator(cc));
// where cc is the TextArea where the cc list should be entered

We just started using JRebel on a wicket project with hibernate and Spring. JRebel (formerly known as JavaRebel) provides true hot-swap development under Eclipse – i.e. you never have to republish and wait for hibernate and Spring to reload everything – whenever you save a class JRebel notices and simply reloads the class into the JVM. This greatly increases productivity on larger projects and, perhaps more importantly, removes a major annoyance factor relative to other languages which handle this better.

Installation was a bit confusing because JRebel does not watch for changed classes in the directory where Eclipse puts them by default (at least that was the case for our installation of Eclipse on Mac OS X). To workaround this, you need to add a rebel.xml file to the WEB-INF/classes directory of your project as follows:


<?xml version="1.0" encoding="ISO-8859-1"?>
<application
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.zeroturnaround.com"
xsi:schemaLocation="http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd">
<classpath fallback="true">
<dir name="/Users/userx/Documents/workspace/projectx/build/classes"/>
</classpath>
</application>


where the “dir name” element is set to the directory where Eclipse writes the compiled .class files for your project.

In eclipse, you need to modify the launch configuration for tomcat and add the following to the arguments:


-noverify
-javaagent:/path/to/jrebel.jar
-Drebel.trace.log=true
-Drebel.wicket_plugin=true


Labels for checkboxes weren’t working as I expected in wicket panels (i.e. when I clicked the label, the checkbox didn’t react). Looking at the output html I saw that the id of the checkbox had been changed by wicket, so the “label for” was no longer associated with the checkbox. The solution was to use the label without the for (and nest the checkbox within the label tag).

Instead of:

<label for="option1">
  <input id="option1" type="checkbox" wicket:id="option1"/>Option 1
</label>

use:

<label>
  <input type="checkbox" wicket:id="option1"/>Option 1
</label>

We were wondering how to use Hibernate annotations to automatically create constraints (HTML “maxlength”) and validators (required fields, string length validation). After searching around, I found a wicket behavior for this in wicket-stuff from Ryan Sonnek. It was in wicket-stuff 1.3 and apparently had been removed from 1.4. Anyway, I patched it up a bit to make it work in wicket 1.4 and here it is (I also took the liberty of renaming it to AnnotatedConstraintBehavior in case we wanted to use it for annotations other than hibernate):

import java.lang.annotation.Annotation;
import java.lang.reflect.Field;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
 
import org.apache.wicket.AttributeModifier;
import org.apache.wicket.Component;
import org.apache.wicket.application.IComponentOnBeforeRenderListener;
import org.apache.wicket.behavior.AbstractBehavior;
import org.apache.wicket.markup.html.form.FormComponent;
import org.apache.wicket.model.IModel;
import org.apache.wicket.model.IPropertyReflectionAwareModel;
import org.apache.wicket.model.Model;
import org.apache.wicket.validation.validator.StringValidator;
import org.hibernate.validator.Length;
import org.hibernate.validator.NotNull;
 
/**
 * Configure a Wicket Component based on Hibernate annotations (@NotNull and @Length(min=x,max=y)).
 * 
 * Inspects the Model of a FormComponent and configures the Component according to the declared Hibernate Annotations used on the model object.
 * NOTE: This means the Component's Model mustbe known when configuring a Component.
 * 
 * This object can be used as a Behavior to configure a single Component.
 * NOTE: this object is stateless, and the same instance can be reused to configure multiple Components.
 * 
 * public class MyWebPage extends WebPage {
 * 	public MyWebPage() {
 *     TextField name = new TextField("id", new PropertyModel(user, "name");
 *     name.addBehavior(new AnnotatedConstraintBehavior());
 *     add(name);
 *   }
 * }
 * 
 * This object can also be used as a component listener that will automatically configure all FormComponents based on Hibernate annotations. This ensures that an entire application respects annotations without adding custom Validators or Behaviors to each FormComponent.
 * 
 * public class MyApplication extends WebApplication {
 * 	public void init() {
 * 		addPreComponentOnBeforeRenderListener(new AnnotatedConstraintBehavior());
 * 	}
 * }
 * 
 * @see http://jroller.com/page/wireframe/?anchor= hibernateannotationcomponentconfigurator
 * @see http ://jroller.com/page/wireframe/?anchor=hibernate_annotations_and_wicket
 */
@SuppressWarnings("serial")
public class AnnotatedConstraintBehavior extends AbstractBehavior implements IComponentOnBeforeRenderListener {
	@SuppressWarnings("unchecked")
	private static Map configs = new HashMap() {
		{
			put(NotNull.class, new HibernateAnnotationConfig() {
				public void onAnnotatedComponent(Annotation annotation, FormComponent component) {
					component.setRequired(true);
				}
			});
			put(Length.class, new HibernateAnnotationConfig() {
				public void onAnnotatedComponent(Annotation annotation, FormComponent component) {
					int max = ((Length) annotation).max();
					component.add(new AttributeModifier("maxlength", true, new Model(Integer.toString(max))));
					component.add(StringValidator.maximumLength(max));
				}
			});
		}
	};
 
	@Override
	public final void bind(Component component) {
		super.bind(component);
 
		configure(component);
	}
 
	public final void onBeforeRender(Component component) {
		if (!component.hasBeenRendered()) {
			configure(component);
		}
	}
 
	void configure(Component component) {
		if (!isApplicableFor(component)) {
			return;
		}
		FormComponent formComponent = (FormComponent) component;
		IPropertyReflectionAwareModel propertyModel = (IPropertyReflectionAwareModel) component.getDefaultModel();
		for (Annotation annotation : getAnnotations(propertyModel)) {
			Class<? extends Annotation> annotationType = annotation.annotationType();
			HibernateAnnotationConfig config = (HibernateAnnotationConfig) configs.get(annotationType);
			if (null != config) {
				config.onAnnotatedComponent(annotation, formComponent);
			}
		}
	}
 
	private Collection<Annotation> getAnnotations(IPropertyReflectionAwareModel propertyModel) {
		Field field = propertyModel.getPropertyField();
		if (field == null) {
			//Log.warn("Unable to find annotations for model: " + propertyModel);
			return Collections.emptyList();
		}
		return Arrays.asList(field.getAnnotations());
	}
 
	private boolean isApplicableFor(Component component) {
		if (!(component instanceof FormComponent)) {
			return false;
		}
		IModel model = component.getDefaultModel();
		if (null == model || !IPropertyReflectionAwareModel.class.isAssignableFrom(model.getClass())) {
			//Log.warn("No valid model is available for configuring Component: " + component);
			return false;
		}
 
		return true;
	}
 
	/**
	 * simple interface to abstract performing work for a specific annotation.
	 */
	private static interface HibernateAnnotationConfig {
		void onAnnotatedComponent(Annotation annotation, FormComponent component);
	}
}

To use it, you’ll need to have @Length(min=x,max=y) and/or @NotNull hibernate annotations on your objects. The behavior will find these in component models and react accordingly, adding the maxlength attributes to the markup, string validator and setting required.

I implemented a simple extension of the wicket PasswordTextField which provides you with an indication of the strength of the entered password (using regular expressions and returning “empty”, “weak”, “medium” or “strong”). Strong means at least 7 characters with mixed alphanumeric and non-alphanumeric characters. Medium means at least 6 characters with mixed alpha and non-alpha characters.

Originally I intended to make it a panel with a visual indicator (the usual spectrum from red through yellow to green), but then I figured that allowing the caller to make the styling decisions is better (if you want to display a spectrum, then add images to your styles and set the style on the component using an AttributeModifier – come to think of it, maybe I’ll do another post on how to do that…).

Anyway, here’s the component:

import org.apache.wicket.ajax.AjaxRequestTarget;
import org.apache.wicket.ajax.form.OnChangeAjaxBehavior;
import org.apache.wicket.markup.html.form.PasswordTextField;
import org.apache.wicket.model.IModel;
import org.apache.wicket.model.Model;
 
public abstract class PasswordTextFieldWithStrength extends PasswordTextField {
 private static final long serialVersionUID = 1L;
 
 public abstract void onUpdate(String passwordStrength, AjaxRequestTarget target);
 
 public PasswordTextFieldWithStrength(String id) {
  super(id);
 
  add(new OnChangeAjaxBehavior() {
   private static final long serialVersionUID = 1L;
 
   @Override
   protected void onUpdate(AjaxRequestTarget target) {
    String strength = getPasswordStrength(PasswordTextFieldWithStrength.this.getDefaultModelObjectAsString());
    PasswordTextFieldWithStrength.this.onUpdate(strength, target);
   }
  });
 }
 
 public String getStrength() {
  return getPasswordStrength(getDefaultModelObjectAsString());
 }
 
 private String getPasswordStrength(String password) {
  if (password.isEmpty())
   return "empty";
 
  String alphaPresent = ".*\\w.*"; // at least one alpha character
  String nonAlphanumericPresent = ".*[^a-zA-Z0-9].*"; // at least one non-alphanumeric character
  String nonAlphaPresent = ".*[^a-zA-Z].*"; // at least one non-alpha character
 
  if (password.matches(alphaPresent) && password.matches(nonAlphanumericPresent) && password.length() >= 7)
   return "strong";
 
  if (password.matches(alphaPresent) && password.matches(nonAlphaPresent) && password.length() >= 6)
   return "medium";
 
  return "weak";
 }
}

And here’s how to use it:

// put a label on the form which will display the strength of the entered password
final Label passwordStrengthLabel = new Label("passwordStrength");
passwordStrengthLabel.setOutputMarkupId(true);
form.add(passwordStrengthLabel);
 
PasswordTextFieldWithStrength passwordField = new PasswordTextFieldWithStrength("password") {
  private static final long serialVersionUID = 1L;
 
[caption id="attachment_33" align="aligncenter" width="492" caption="And here&#39;s how it looks in an application"]<img class="size-full wp-image-33" title="screenshot" src="http://blog.armstrongconsulting.com/wp-content/uploads/2009/07/Picture-31.png" alt="And here's how it looks in an application" width="492" height="234" />[/caption]
 
@Override
  public void onUpdate(String strength, AjaxRequestTarget target) {
    // strength will be one of "empty", "weak", "medium", "strong"
    // you can put it into a label, use it as a key to load a localized string, use it to construct a style etc
    passwordStrength = getLocalizer().getString("passwordStrength_" + strength, this);
    target.addComponent(passwordStrengthLabel);
  }
 };
form.add(passwordField);

Hibernate’s hbm2ddl.auto setting allows you to create, update, and validate a database schemas based on the mapping configuration. The create & update settings are great in development environments, since they will ensure your database schema will always be in sync with your model. We would not recommend these settings in an production environment because, there we need to have more fine grained control over database updates (hbm2ddl is too “magical” for production). Additionally, there may be updates to the data which cannot be handled by Hibernate’s hbm2ddl.auto implementation (e.g. content-related updates rather than structural).

Some people would say, “such database updates should not be done by the application, but rather by the database administrator”. However, for most applications, you want the code to maintain the database structure – otherwise you have terrible trouble keeping test and staging databases up to date.

To achieve this we “upgrade” our databases with our own implementation, using plain JDBC. A meta table in the schema holds a version number which gets checked during the application’s start up. If the version in the database does not match the code version, the application upgrades the database schema accordingly.

We need to have our own database upgrade execute before Hibernate intercepts. Otherwise Hibernate will complain that tables or columns which should be created by our upgrade procedure are not yet available.

This can be achieved by a bean, which implements the BeanFactoryPostProcessor.

package sample;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanFactoryPostProcessor;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
 
public class DatabaseVersion implements BeanFactoryPostProcessor {
 
 public static void checkDatabaseStructure() {
  /*
   * This method creates a generic JDBC connection,
   * checks the database structure
   * and upgrades it if applicable.
   */
 }
 
 @Override
 public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
  DatabaseVersion.chekDatabaseStructure();
 }
}

All what’s missing now is the definition of the bean in the applicationContext.xml file.

That’s it. The application context auto-detects BeanFactoryPostProcessor beans in their bean definitions and applys them before any other beans get created. If you are concearned about the order of loading, your bean can implement the “Ordered” interface in addition.

Many of our applications require scripting support (allowing users to create scripts to customize workflow within the application). Java provides very straightforward scripting via the javax.script script-engine library. A simple integration is shown below where a Javascript onSave method provided by the user is called, if available, passing a business-logic object “item”:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import javax.script.Invocable;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
 
public class Scripting extends Base {
 private ScriptEngineManager mgr;
 private ScriptEngine engine;
 private Invocable invocable;
 
 private boolean initialize (String script) {
  mgr = new ScriptEngineManager();
  engine = mgr.getEngineByName("js");
  try {
   engine.eval(script);
  } catch (ScriptException e) {
   warn("The script could not be loaded: " + e.getMessage());
   return false;
  }
  invocable = (Invocable)engine;
  return true;
 }
 
 public void onSave(Item item) {
  try {
   invocable.invokeFunction("onSave", item);
  } catch (ScriptException e) {
   warn("The onSave method of the script caused an error: " + e.getMessage());
  } catch (NoSuchMethodException e) {
   // no onSave method provided by the user - that's OK, just ignore it
  }
 }
}

Note: the javax.script library supports multiple scripting engines including javascript, python, groovy and java. Javascript was the easiest to get working because the Rhino Javascript engine is included in JDK 6.