All posts by roger

Corporate backup 2.0

Anyone who has a Mac is familiar with Time Machine, the almost magical, continuous backup capability of OSX. What many people may not know is that Time Machine is based on concepts which have been freely available for quite some time and which can easily be applied to corporate-wide backups. Because corporate backup is considered expensive to implement, many companies have outdated legacy backup systems based on tapes, tape-robots and offsite transport and storage of tapes. These systems are hopelessly outdated and can no longer keep up with the every increasing storage capacity of the disks they should be backing up and the decreasing backup time window in which backups should be completed.

We have approximately 3TB of data (consisting of about 40 databases, 200 virtual machines and hundreds of thousands of files) on our servers and workstations which need to be backed up. About a year ago we installed a comapny-wide backup to disk with offsite replication and versioning which has been providing us with continuous backup ever since. It continuously replicates a 4TB local RAID-6 disk-array offsite to a versioning 5TB RAID-6 disk array over a dedicated 4Mbps line, 24 hours a day, using rsync for the replication and snapshots based on Linux filesystem hard-links for versioning. Its implemented entirely on standard Linux components (zero license costs) and has been running without a glitch for over a year. Thanks to this system, we not only have an offsite backup of all business-critical data, but we can step back to any version of a database or virtual machine from yesterday, two days ago, four days ago, a week old, a month old etc. I can’t imagine why any company would still want to install a propietary backup system when such perfect technology is freely available.

Turnkey appliances

This blog is running on a Turnkey WordPress appliance (www.turnkeylinux.org). Virtual appliances are of course fantastic if they work – a fully configured, just-works server which you can download and provision in seconds (this blog took about 5 minutes to download, install on a VMWare Server virtual machine and get running). Until now, most appliances we tried had enough gotchas to make us return to manual installation on a generic distribution, but the Turnkey appliances seem to be perfect. Based on Ubuntu or Debian and with just enough stuff pre-installed to make them useful, while still being compact enough to compete with a manual installation. Thanks Turnkey!

Adding scripting to java applications

Many of our applications require scripting support (allowing users to create scripts to customize workflow within the application). Java provides very straightforward scripting via the javax.script script-engine library. A simple integration is shown below where a Javascript onSave method provided by the user is called, if available, passing a business-logic object “item”:

Note: the javax.script library supports multiple scripting engines including javascript, python, groovy and java. Javascript was the easiest to get working because the Rhino Javascript engine is included in JDK 6.

A sessionless login page for Wicket applications

We’re big fans of Apache Wicket, but as with most frameworks, sometimes the simplest things appear to be hard to do (or at least its hard to find out how to do them). Application session handling is great in Wicket, but I immediately ran into the problem that the problem that the login page of my application would timeout like any other page of the application. If the user logged out (at which point the login page is displayed), left the browser window open and then tried to use the same browser window to login again an hour later, he’d get a “sorry, your session has timed out, please login again”.  This message obviously makes no sense on the login page.

The solution (thanks Doug Donohue for the help on this) is to use a stateless form for the login page (which causes Wicket to only create a temporary session for the page) and when the user has successfully logged in, convert the session to a regular session.

The relevant code fragments are shown below:

Note however that you have to be very careful what components you use in a stateless page – otherwise you’ll suddenly find it to be stateful again (i.e. it will bind its session automatically and you’ll be back in the same situation). Basically anything which requires remembering a specific page instance (e.g. Ajax) will cause your page to become stateful.

There is some logic built into Wicket which should warn you when a page which you expect to be stateless becomes stateful, but it seems that in the latest versions of Wicket, these warnings are disabled. We ended up creating our own StatelessPage super-class which, in onBeforeRender, calls isPageStateless() and if that returns false, it runs through the components on the page and checks isStateless() for each and reports the wicket id for each component which is not stateless. That way, during development we can show a warning like “This page should be stateless, but isn’t because the following components are stateful: component1, component2…”