Converting a VMWare workstation image to Xen

Recently I had to convert a VMWare virtual machine running Redhat Enterprise Linux 4 (vmclient running on vmhost) to it's own box. Due to the practical limitations of remote support, I decided that Xen might be a better solution than bare metal, without sacrificing performance. Since the server had been running on VMWare Workstation on a low memory box, the move to new hardware would boost performance no matter what I did. During my conversion experience, I discovered some useful resources scattered about the Internet. One of them was in German, which I don't speak, so I figured I would put things in one place for anyone challenged to do something like this in the future.

VMHost is a RHEL4 server with VMWare workstation running on it. First I found the directory containing the files for the server I want to clone. Looking in the /var/vmware/Virtual Machines directory, I see vmclient/vmclient.vmdk. This is the disk image. Fortunately, there is only a single image. I'm not sure what I would do if there were spanned images, which is one of the options you have with VMWare. I suppose you could append one to another, but I'd have to check into how best to do that.

I used qemu to convert from the vmware vmdk format to a raw disk image. Qemu is readily available at http://rpmfind.net. Convert the image by shutting down the VMWare session, then issuing the following:

qemu-img convert -f vmdk vmclient.vmdk -O raw vmclient.raw

Note that all the commands are on one line, so ignore any line breaks. This gives a disk image of vmclient in a format I can operate upon. For example do the following:

fdisk -l -u vmclient.raw

And the output will look something like this:

You must set cylinders.
You can do this from the extra functions menu.

Disk v-pt-dev1.raw: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id
System
vmclient.raw1 * 63 208844 104391 83
Linux
vmclient.raw2 208845 8385929 4088542+ 8e
Linux LVM

This tells me that I have a disk image with two primary partitions. Knowing that the first partition is /dev/sda1 or /boot (which I could tell with the df command inside the original virtual machine), I know that the second partition is the LVM partition containing both / and swap.

I moved this raw file over to the Xen host server (xenhost), which I have already installed with a generic XenServer 3.1 install (I like XenSource's distribution, but the concepts here should work with any Xen implementation). I create a new default RHEL 4.4 install using the Xen Client which comes with Xenserver 3.1. I had an NFS mount with all the RHEL4 CD content copied into it and pointed the install at that. But you could download a new Xen Template (XGT) for CEntOS and use that instead if you wish.

After creating the working RHEL install under Xen, I know I will need the /lib/modules/ directory in my final box, but my migration technique is going to destroy everything but /boot in the process, so I copied the modules directory to /boot temporarily. I then shut down the XenVM and ssh directly into xenhost.

Looking under /dev/mapper, I see the logical volumes (they have some absurdly long names, but end in sda, sdb, etc, depending on how many drives you set up). Since I have set up only one drive, I only have an sda logical volume. I want to save my /boot from this LV (with the modules directory already copied to it). After verifying with fdisk -l -u VG{whatever}.sda that the layout is the same, with the first 208844 blocks being /boot, I issued the following command:

dd if=VG{whatever}.sda bs=512 count=208844 of=xenclientboot.img

This pulled the /boot from the logical volume. I then overwrite the LV with my VMware image:

dd if=vmclient.raw of=VG{whatever}.sda

This will not yet boot because it does not have a paravirtualized kernel. At least on the box I am using. It may work on more recent CPUs which have hardware virtualization, in which case, saving the /boot may be unnecessary (though it may still be desirable for performance). So:

dd if=xenclientboot.img bs=512 count=208844 of=VG{whatever}.sda

This copies the correct /boot over. Boot the new Xen Client system, move the modules directory from /boot to it's correct place in /lib/modules, run depmod -a, then netconfig and configure your network stuff. You might also want to copy the xenmond binary from /usr/sbin in any other Xen installations, along with the xenmond start script in /etc/init.d/. This will allow the XenClient to monitor the running Xen Client. Remember to do chkconfig --add xenmond after you copy the file so that it will be added to the correct runlevels. Reboot, then things should work. I'd appreciate any feedback on clarifying things, or if you see any errors or better ways to do this. I'm always looking for better methods.