How to migrate large VMWare VMs to remote data-centers using rsync

Using VCenter’s migration over the internet to a remote data-center is often too slow for practical purposes – a 200GB VM may take several hours to move and that may be too much downtime.

KVM users typically use rsync to achieve this by first rsyncing the VM while it’s running and then stopping it and doing a final, differential rsync to transfer just the changes.

This isn’t so straightforward on VMWare for several reasons:

(a) There’s no built-in rsync on ESXi hypervisors

(b) The way in which VMWare writes to its VMDK files messes up rsync’s ability to do differential transfer. VMWare has something called CBT (changed block tracking), but rsync doesn’t know anything about it.

After some experimentation, I finally figured out how to do it and its pretty easy. You first need to create a snapshot on the VM you want to migrate. This localises all writes to the VM and allows rsync to do its job. You then do a full rsync (rsync –sparse -zaP user@source:/VM /destination/path/to/VM/) of the VM to the remote data-center while its running (this takes a long time, but you don’t care because you haven’t shut it down yet). When this is done, you shut the VM down (keeping the snapshot) and make a final rsync. This final rsync only transfers data written to the snapshot and is very quick (in my case, it was a 200GB VM which took 5 hours for the first rsync and 16 minutes for the final rsync).

Note: I used a linux VM running on the destination data-center to do the rsync, with the VM directory mounted with nfs on both sides.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.