Weird racing clock problem on VMWare Linux Guests

A while back we installed a new Dell server with a low-power Xeon 3GHz L3110 CPU to run some other essential network infrastructure. We chose the specific server configuration because it dissipates less than 30W while running 5 VMware VMs. It runs for hours on a UPS and doesn’t require cooling, so even if our server room air-conditioning were to die, this server should keep our network, firewalls, VPNs, DNS, DHCP and primary Terracotta server and a few other vital services up long enough for us to figure out what’s going on.

The server is running CentOS 5.5 and VMWare 1.0.10 – normally a very stable combination. However, we found that Linux guests running on this server were not keeping time properly. their clocks were running too fast – 50% too fast in fact. We finally figured out that the problem was caused by the fact that the CPU power-saving causes Linux to think that has a 2GHz processor instead of a 3GHz processor and this causes the 50% clock speedup in the Linux guests under VMWare. We disabled the demand-based power saving feature of the CPU in the BIOS and now it works correctly.

Adding AnnotatedConstraintBehavior to TextAreas

A while back I blogged about using an annotated constraint behavior to limit input in Wicket TextFields based on domain model annotations (http://blog.armstrongconsulting.com/?p=22). So if we annotate our domain models as follows:

@Length(max = 40)
private String firstName;

that gets automatically be translated into a maxlength=40 attribute on any text field bound to firstName (we use a form visitor on a custom form class to add the behavior to all text fields in the form).

That’s pretty cool, but unfortunately, it only works for text inputs because textarea doesn’t support the maxlength attribute. This really bothered me because its really important to limit the length of textareas since that’s where people try to paste in huge blocks of text and its annoying to only find out the limitation after submitting the form.

So today, I updated the behavior to also handle textareas. There’s a javascript workaround which simulates the maxlength behavior on textareas. I took the javascript from http://cf-bill.blogspot.com/2005/05/unobtrusive-javascript-textarea.html and in the renderHead of the behavior, I add the javascript.

Here’s the code – its in three files: AnnotatedConstraintBehavior.java (the behavior implementation), AnnotatedConstraintBehavior.js (the javascript needed to simulate maxlength on textareas) and ACForm.java (a form which does two extra things: (a) it validates its properties based on annotated constraints (like @NotNull and @Length)) and (b) it adds maxlength to text fields and textareas using the AnnotatedConstraintBehavior).

// AnnotatedConstraintBehavior.java
import java.io.Serializable;
import java.lang.annotation.Annotation;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import javax.validation.constraints.NotNull;

import org.apache.commons.validator.EmailValidator;
import org.apache.wicket.AttributeModifier;
import org.apache.wicket.Component;
import org.apache.wicket.ResourceReference;
import org.apache.wicket.application.IComponentOnBeforeRenderListener;
import org.apache.wicket.behavior.AbstractBehavior;
import org.apache.wicket.markup.html.IHeaderResponse;
import org.apache.wicket.markup.html.form.FormComponent;
import org.apache.wicket.model.AbstractPropertyModel;
import org.apache.wicket.model.IModel;
import org.apache.wicket.model.IPropertyReflectionAwareModel;
import org.apache.wicket.model.Model;
import org.apache.wicket.util.lang.PropertyResolver;
import org.apache.wicket.validation.IValidatable;
import org.apache.wicket.validation.validator.StringValidator;
import org.hibernate.validator.constraints.Email;
import org.hibernate.validator.constraints.Length;

/**
 * Configure a Wicket <code>Component</code> based on Hibernate annotations (@NotNull and @Length(min=x,max=y)).
 * <p>
 * Inspects the <code>Model</code> of a <code>FormComponent</code> and configures the <code>Component</code> according to the declared Hibernate Annotations used on the model object. <br />
 * <strong>NOTE:</strong> This means the <code>Component</code>'s <code>Model</code> <em>must</em> be known when {@link #configure(Component) configuring} a <code>Component</code>.
 * </p>
 *
 * <p>
 * This object can be used as a <code>Behavior</code> to configure a single <code>Component</code>. <br />
 * <strong>NOTE:</strong> this object is <em>stateless</em>, and the same instance can be reused to configure multiple <code>Component</code>s.
 * </p>
 *
 * <pre>
 * public class MyWebPage extends WebPage {
 * 	public MyWebPage() {
 *     TextField name = new TextField("id", new PropertyModel(user, "name");
 *     name.addBehavior(new AnnotatedConstraintBehavior());
 *     add(name);
 *   }
 * }
 * </pre>
 *
 * <p>
 * This object can also be used as a component listener that will automatically configure <em>all</em> <code>FormComponent</code>s based on Hibernate annotations. This ensures that an entire application respects annotations without adding custom
 * <code>Validator</code>s or <code>Behavior</code>s to each <code>FormComponent</code>.
 * </p>
 *
 * <pre>
 * public class MyApplication extends WebApplication {
 * 	public void init() {
 * 		addPreComponentOnBeforeRenderListener(new AnnotatedConstraintBehavior());
 * 	}
 * }
 * </pre>
 *
 * @see http://jroller.com/page/wireframe/?anchor= hibernateannotationcomponentconfigurator
 * @see http ://jroller.com/page/wireframe/?anchor=hibernate_annotations_and_wicket
 */
@SuppressWarnings("serial")
public class AnnotatedConstraintBehavior extends AbstractBehavior implements IComponentOnBeforeRenderListener {
	static org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(AnnotatedConstraintBehavior.class);

	@SuppressWarnings("unchecked")
	private static Map configs = new HashMap() {
		{
			put(NotNull.class, new AnnotationConfig() {
				public void onAnnotatedComponent(Annotation annotation, FormComponent component) {
					component.setRequired(true);
				}
			});
			put(Length.class, new AnnotationConfig() {
				public void onAnnotatedComponent(Annotation annotation, FormComponent component) {
					int max = ((Length) annotation).max();
					log.debug("adding maxlength=" + max + " attribute to " + component.getMarkupId());
					component.add(new AttributeModifier("maxlength", true, new Model(Integer.toString(max))));
					component.add(StringValidator.maximumLength(max));
				}
			});

			put(Email.class, new AnnotationConfig() {
				public void onAnnotatedComponent(Annotation annotation, FormComponent component) {
					component.add(new StringValidator() {
						@Override
						protected void onValidate(IValidatable<string> validatable) {
							if (!EmailValidator.getInstance().isValid(validatable.getValue())) {
								error(validatable);
							}

						}
					});
				}
			});

		}
	};

	@Override
	public void renderHead(IHeaderResponse iHeaderResponse) {
		super.renderHead(iHeaderResponse);
		iHeaderResponse.renderJavascriptReference(new ResourceReference(AnnotatedConstraintBehavior.class, "AnnotatedConstraintBehavior.js"));
	}

	@Override
	public final void bind(Component component) {
		super.bind(component);
		configure(component);
	}

	@Override
	public final void onBeforeRender(Component component) {
		if (!component.hasBeenRendered()) {
			configure(component);
		}
	}

	void configure(Component component) {
		if (!isApplicableFor(component)) {
			return;
		}
		FormComponent<?> formComponent = (FormComponent<?>) component;
		for (Annotation annotation : getAnnotations(component.getDefaultModel())) {
			Class<? extends Annotation> annotationType = annotation.annotationType();
			AnnotationConfig config = (AnnotationConfig) configs.get(annotationType);
			if (null != config) {
				config.onAnnotatedComponent(annotation, formComponent);
			}
		}
	}

	private Collection<annotation> getAnnotations(IModel<?> model) {

		try {
			// Only if a setter method is available we'll search for the
			// related field and find its Annotations.
			Method setter = ((IPropertyReflectionAwareModel) model).getPropertySetter();

			String name = setter.getName();

			if (name.startsWith("set") &amp;&amp; name.length() > 3) {
				name = name.substring(3, 4).toLowerCase() + name.substring(4);
			} else {
				return Collections.emptyList();
			}

			Object target = ((AbstractPropertyModel<?>) model).getTarget();
			Field field = PropertyResolver.getPropertyField(name, target);
			if (field == null) {
				return Collections.emptyList();
			}

			return Arrays.asList(field.getAnnotations());
		} catch (Exception ignore) {
			return Collections.emptyList();
		}
	}

	private boolean isApplicableFor(Component component) {
		if (!(component instanceof FormComponent<?>)) {
			return false;
		}
		IModel<?> model = component.getDefaultModel();
		if (model == null || !IPropertyReflectionAwareModel.class.isAssignableFrom(model.getClass())) {
			return false;
		}

		return true;
	}

	/**
	 * simple interface to abstract performing work for a specific annotation.
	 */
	private static interface AnnotationConfig extends Serializable {
		void onAnnotatedComponent(Annotation annotation, FormComponent<?> component);
	}
}
// AnnotatedConstraintBehavior.js
// javascript to simulate maxlength behavior on textareas
// source: http://cf-bill.blogspot.com/2005/05/unobtrusive-javascript-textarea.html

<script type="text/javascript">

 function textAreasInit(){
  var objs = document.getElementsByTagName("textarea");
  var oi = 0; //oi is object index
  var thisObj;

  for (oi=0;oi<objs.length;oi++) {
   thisObj = objs&#91;oi&#93;;
   // note that maxlength is case sensitve
   if (thisObj.getAttribute('maxlength')){
    thisObj.onkeyup = forceMaxLength;
   }
   thisObj.onchange = saveEntryValue;
  }
 }

 function forceMaxLength(){
  var maxLength = parseInt(this.getAttribute('maxlength'));
  if(this.value.length > maxlength){
   this.value = this.value.substring(0,maxlength);
  }
 }

function addEvent(elm, evType, fn, useCapture)
// addEvent and removeEvent
// cross-browser event handling for IE5+,  NS6 and Mozilla
// By Scott Andrew
{
  if (elm.addEventListener){
 elm.addEventListener(evType, fn, useCapture);
 return true;
  } else if (elm.attachEvent){
 var r = elm.attachEvent("on"+evType, fn);
 return r;
  } else {
 alert("Handler could not be removed");
  }
}

addEvent(window, "load", textAreasInit);
</script>
// ACForm.java
import java.io.Serializable;
import java.util.HashSet;
import java.util.Set;

import org.apache.wicket.Component;
import org.apache.wicket.Component.IVisitor;
import org.apache.wicket.behavior.AbstractBehavior;
import org.apache.wicket.markup.html.form.Form;
import org.apache.wicket.markup.html.form.TextArea;
import org.apache.wicket.markup.html.form.TextField;
import org.apache.wicket.model.IModel;
import org.wicketstuff.jsr303.PropertyValidation;

public class ACForm<t> extends Form<t> {
	private static final long serialVersionUID = 1L;

	static org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(Placeholder.class);

	public ACForm(String id, IModel<t> model) {
		super(id, model);
		add(new PropertyValidation());
	}

	public ACForm(String id) {
		super(id);
	}

	@Override
	protected void onBeforeRender() {
		super.onBeforeRender();
		visitChildren(TextField.class, new AddBehaviorVisitor(new AnnotatedConstraintBehavior()));

		// note: textarea doesn't actually support the maxlength attribute, but we add it anyway and enforce it with javascript (the AnnotatedConstraintBehavior adds it)
		visitChildren(TextArea.class, new AddBehaviorVisitor(new AnnotatedConstraintBehavior()));
	}
}

class AddBehaviorVisitor implements IVisitor<component>, Serializable {
	private static final long serialVersionUID = 1L;
	private final AbstractBehavior behavior;
	Set<component> visited = new HashSet<component>();

	public AddBehaviorVisitor(AbstractBehavior behavior) {
		this.behavior = behavior;
	}

	@Override
	public Object component(Component component) {
		if (!visited.contains(component)) {
			visited.add(component);
			component.add(behavior);
		}
		return IVisitor.CONTINUE_TRAVERSAL;
	}
}

Using Wicket’s CheckBoxMultipleChoice

When I started with Wicket, I remember fighting with CheckBoxMultipleChoice for hours trying to get it to work. I just used it again today and was surprised at how easy it is to use once you’re familiar with how Wicket works. So for anyone struggling with it, here’s how to do it.

Say we want to display a list of Projects associated with a User and we want to do it with the CheckBoxMultipleChoice. The projects which the user currently has access to should be checked initially. Checking or unchecking boxes will add or remove access to projects.

You can accomplish this with the following code:

add(new CheckBoxMultipleChoice("projects", new Model<project>(userProjects), allProjects));

Assuming you have a span with wicket:id=”projects” in your HTML, this will display a checkbox for each project found in the “allProjects” list. Those which are also in the “userProjects” list will be checked. After submitting the form, the “userProjects” list will hold those which were checked when the form was submitted.

Note: I have omitted a renderer – that’s because my Project class has a toString() method which returns its name. That’s allows Wicket to render it correctly without an explicit renderer.

Second thing to note is that “userProjects” must be an ArrayList of Projects, not a List (as your service method would normally return it). The reason for this is that the Model demands a Serializable class so the easiest thing is to define userProjects as follows:

ArrayList userProjects = new ArrayList(service.getUserProjects());

The choices parameter (allProjects) can be a List, so you can use it straight from your service method.

Hope that helps anyone trying to use CheckBoxMultipleChoice for the first time.

How to display an enum in a dropdown in Wicket

Its often useful to use an enum as the model for a dropdown (or radio group) in Wicket, but not immediately obvious how to handle the display of the values. Lets say I have an enum with three colors:

enum Color { RED, GREEN, BLUE };

I can hook this up to a dropdown with the following code:

<select wicket:id="color"/>

enum Color { RED, GREEN, BLUE };
private Color color = Color.GREEN;
add(new DropDownChoice<color>("color", new Model<color>(color), Arrays.asList(Color.values())));

However, that would display “RED”, “GREEN”, “BLUE”. I probably don’t want the dropdown to use the default toString() form of my enum declarations, but rather I want it to load translatable texts from property files.

To do this, I can use Wicket’s EnumChoiceRenderer as follows:

<select wicket:id="color"/>
enum Color { RED, GREEN, BLUE };
private Color color = Color.GREEN;
add(new DropDownChoice<color>("color", Arrays.asList(Color.values()), new EnumChoiceRenderer<color>(this)));
Color.RED=Red
Color.GREEN=Green
Color.BLUE=Blue

Now I get what I wanted – a dropdown displaying “Red”, “Green” and “Blue”.

Mockito

We just started using Mockito for unit testing of our Wicket applications. Its a great tool, allowing us to easily mock out our service layers on a test-by-test basis.

In the example below, we’re testing a page called ItemPage. It takes an id of an item to load – internally the page uses itemService.find(id) to load the item from the database. Using Mockito, we mock out the itemService.find call to return a test item.

public class TestItemPage extends WicketTestcase {
	private static final long serialVersionUID = 1L;

	@Test
	public void TestItemPage() {
		when(itemService.find(Mockito.anyInt())).thenAnswer(new Answer() {

			@Override
			public Item answer(InvocationOnMock invocation) throws Throwable {
				final int id = (Item) invocation.getArguments()[0];
				Item item = new Item(id);
				item.setTitle("test");
				item.setDeadline(new Date());
				return item;
			}
		});

		tester.startPage(new ItemPage(12345));
		tester.assertRenderedPage(ItemPage.class);
	}
}

Clustering wicket apps

After fooling around with other methods, we finally accepted the advice I got on the Wicket IRC channel and used Terracotta to cluster our Wicket-based apps running under Jetty. It turned out to be straightforward to implement.

The first thing to do was to add the Terracotta dependencies to our pom.xml.

<dependency>
	<groupId>org.terracotta.session</groupId>
	<artifactId>terracotta-session</artifactId>
	<version>1.1.1</version>
</dependency>

<dependency>
	<groupId>org.terracotta</groupId>
	<artifactId>terracotta-toolkit-1.1-runtime</artifactId>
	<version>2.0.0</version>
</dependency>

Then you just need to add a Terracotta filter to the jetty WebAppContext as follows:

FilterHolder tcFilterHolder = new FilterHolder(TerracottaJetty61xSessionFilter.class);
tcFilterHolder.setInitParameter("tcConfigUrl", "terracotta:9510,terracotta2:9510");
context.addFilter(tcFilterHolder, "/*", Handler.ALL);

That’s it. Terracotta will cluster the session (in the example we’re using two terracotta servers called “terracotta” and “terracotta2” – a main server and a standby).

We’re using a HAProxy load-balancer with session affinity to load-balance and failover the wicket apps. Note that we are only clustering the session and not the Wicket PageMapStore. This means that if the app fails over, the browser back-button will not work correctly after a failover. Since failover should only occur rarely, if ever, we don’t see the need to cluster the PageMapStore (although this is not hard either) and incur the network cost of replicating the PageMapStore.

Deploying with Continuum and Maven

We are now deploying to our staging servers from our Maven repository using wget (the url is like ‘http://maven:8080/nexus/service/local/artifact/maven/redirect? r=snapshots&g=com.armstrongconsulting.controlcenter.server &a=controlcenterwebapp&v=LATEST&e=jar&c=jar-with-dependencies’.). The runnable jars (or wars) in the Maven repository are built, unit-tested and deployed to Maven by Continuum automatically. If any errors are found, the developer who made the offending commit to SVN is informed by Continuum. Works great, highly recommended. Thanks to Gabriel for setting it all up.

Hazelcast distributed locks for easy fault tolerance

Hazelcast (hazelcast.com) provides an easy way to implement distributed locking to allow your applications to run multiple, fault-tolerant instances without worrying about issues related concurrent access to shared resources (like files, databases or whatever).

try {
	java.util.concurrent.locks.Lock lock = Hazelcast.getLock("mylock");
	while (true) {
		lock.lock();
		try {
			// do some work involving access to shared resources
		} finally {
			lock.unlock();
		}
	}
} finally {
	Hazelcast.shutdown();
}

We have an SMS server which retrieved mail messages from a POP3 mailbox, entered them to a database and then delivered via the Clickatell messaging gateway. Making it run multiple instances concurrently would have been a headache since it would involve various issues relating to transactions across the POP3 and the Clickatell interfaces. It was a whole lot easier to wrap the business logic with a hazelcast lock and let it run on two servers. The beauty of hazelcast is that it just works – since the default configuration uses multicast to detect other members of the cluster, there’s no additional infrastructure – just add the jar to your application and off you go. We might still implement Zookeeper or Terracotta in future, but both of them require more infrastructure (i.e. dedicated (virtual) servers) and configuration.
Postscript: I had occasional hangs with Hazelcast 1.8.4 which caused me to switch to Zookeeper. As expected, Zookeeper was a lot harder to use than Hazelcast – you need Zookeeper installed on 3 servers. There’s no official java client, just some recipes and I found an implementation of Zookeeper locks called Cages on google code. For a java developer, Hazelcast is obviously way easier to use. Anyway, after upgrading to Hazelcast version 1.8.5, the hang problem went away so I happily went back to using Hazelcast. We’re also now using the distributed queues – works great so far.

Wicket dropdowns with mismatched models

Sometimes you have a class containing an id of another (lets say a Machine class containing an owner id rather than an Owner object). The owner id is actually a primary key from the Owner table in the database. Now you want to display the Machine class in a wicket form with a dropdown to select the Owner.

In an ideal world, your Machine class would have “Owner getOwner()” and “setOwner (Owner owner)” methods and your MachineService class would have a static “List getOwners()” method and wicket’s DropDownChoice would just work with:

  final Form<machine> form = new Form<machine>("form", new CompoundPropertyModel<machine>(machine));
  add(form);
  form.add(new DropDownChoice("owner", MachineService.getOwners()));

However, in our Machine class, the owner property is an id of an Owner and not an Owner object. So how do we use the DropDownChoice without changing our Machine class? There are a few different approaches, but a simple one is to use a custom model with the DropDownChoice which converts between object and id representation, as follows:

  final Form<machine> form = new Form<machine>("form", new CompoundPropertyModel<machine>(machine));
  add(form);

  Model<owner> model = new Model<owner>() {
  private static final long serialVersionUID = 1L;
    @Override
    public Owner getObject() {
      if (machine.getOwner() == null)
	return null;
      return new Owner(machine.getOwner());
    }

    @Override
    public void setObject(Owner owner) {
      if (owner == null) {
        machine.setOwner(null);
      } else {
        machine.setOwner(owner.getOwnerNr());
      }
    }
 };
 form.add(new DropDownChoice<owner>("owner", model, ControlCenterService.getOwners()));

So now, when wicket needs to get the current selection, it calls getObject on your custom model and gets an object which it can compare. When it needs to set the selection, it calls setObject which writes the correct id into the machine object. Note: Your Machine class will need to implement toString() for the display in the dropdown. Additionally it will need a constructor which takes an id and an overridden equals() and hashCode() based on the id field.

Enforcing multiple levels of page access in Wicket

In Wicket when you want to require user login for some pages, you typically inherit the pages from a superclass called say ProtectedPage and then use a wicket SimplePageAuthorizationStrategy to protect these pages as follows:

getSecuritySettings().setAuthorizationStrategy(new SimplePageAuthorizationStrategy(ProtectedPage.class, LoginPage.class) {
  @Override
  protected boolean isAuthorized() {
    return ((ApplicationSession) Session.get()).getLoggedInUser() != null;
  }
});

This will cause any access to a protected page by a non-logged-in user to be redirected to the login page first.

In the real world, its likely however, that your application will also require administrator privileges for access to certain pages. To do this, you need to use a wicket CompoundAuthorizationStrategy as follows:

  CompoundAuthorizationStrategy pageAuthorizationStrategy = new CompoundAuthorizationStrategy();
  pageAuthorizationStrategy.add(new SimplePageAuthorizationStrategy(AdminPage.class, AccessDeniedPage.class) {
    @Override
     protected boolean isAuthorized() {
       return (((ApplicationSession) Session.get()).getLoggedInUser() != null) && (((ApplicationSession) Session.get()).getLoggedInUser().isAdministrator());
     }
   });
  pageAuthorizationStrategy.add(new SimplePageAuthorizationStrategy(ProtectedPage.class, LoginPage.class) {
    @Override
     protected boolean isAuthorized() {
       return ((ApplicationSession) Session.get()).getLoggedInUser() != null;
     }
   });
   getSecuritySettings().setAuthorizationStrategy(pageAuthorizationStrategy);

The above requires the user to be logged in to access any page inherited from ProtectedPage and to be both logged in AND be an administrator to access any page inherited from AdminPage.

Thanks to selckin for the tip!