Xen Virtual Machine images.

Xen VM HOW-TO

How-To create an Ubuntu (Hoary Hedgehog) Host

• Install Ubuntu.

At the installation boot prompt type: server. This will give you a minimal installation without X and desktop

Server comes with a preselected packages list - if you want complete control of the installation, ala debian installer, use expert or server-expert

• Partitioning:

root partition 12 Gb
swap partition 2 Gb
LVM partition 146 Gb

• Install a build environment.

apt-get install build-essential automake autoconf libcurl3 libcurl3-dev

apt-get install python2.4-twisted python2.4-twisted-bin bridge-utils

apt-get install python2.4-dev modutils libncurses5-dev

• Get the xen source

Go to the xen downloads page: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads.html and get the src tarball you want: stable, testing or unstable.

Alternately:

wget http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-src.tgz (stable)

• Unpack in /usr/local/src (or wherever):

tar -zxvf xen-2.0.7-src.tgz

• chown root:root -R xen-2.0

• cd xen-2.0

• I am interested in the linux-2.6 dom0/domU and linux-2.4 domU kernels so:

rm -rf freebsd-5.3-xen-sparse netbsd-2.0-xen-sparse

• Edit the Makefile in the xen-2.0 directory to only build the required kernels. That is:

KERNELS ?= linux-2.6-xen0 linux-2.6-xenU linux-2.4-xenU

• Now to build the kernels. In the top level directory execute:

make kernels

Go away and get some coffee, this will take a while. Basically, issuing this command causes the build script to check for the required kernel source packages in the xen-2.0 directory. If they don't exist, it will download them from www.kernel.org (ie in this case it will download linux-2.6.11.12 and linux-2.4.30 kernel src). Once the source has been retrieved, it will patch the src and compile it with a xen .config file.This configuration is very basic.

• Compiling real kernels

Two options are available: manually configure the kernel .config or use a .config from elsewhere

manually configuring the kernel .config

change into the directory of the kernel you wish to configure - since we are configuring a linux dom0 host kernel:

cd linux-2.6.11-xen0

make ARCH=xen menuconfig

make ARCH=xen

Configure the kernel as required.

using an existing kernel .config

Alternately, you can cheat and just use a kernel .config from elsewhere - a prime example is the .config of the kernel with which you booted the system. Note that your life will be much nicer if you use a 2.6.11 .config : )

Ok, for this kernel config to work, it needs to be made xen-aware. So, copy the following lines verbatim to the top of the file.

#
CONFIG_XEN=y
CONFIG_ARCH_XEN=y
CONFIG_NO_IDLE_HZ=y
#
# XEN
#
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
# CONFIG_XEN_BLKDEV_FRONTEND is not set
# CONFIG_XEN_NETDEV_FRONTEND is not set
CONFIG_XEN_WRITABLE_PAGETABLES=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_X86=y
# CONFIG_X86_64 is not set
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y

Now you will have a .config file which is suitable to build a xen dom0 kernel. At the prompt execute:

make ARCH=xen oldconfig

Resolve any issues that may arise. Then:

make ARCH=xen

Note: If you see "error: isa_virt_to_bus_is_UNSUPPORTED undeclared" during compilation, this is because xen does not work with isa devices - remove them from the .config.

Repeat this procedure for each xen kernel you wish to build.

Now, get back in to the xen top level directory in order to make xen, the xen tools and install the kernels

• make and install the xen tools

make tools && make install-tools

• make and install xen

make xen && make install-xen

• install the newly built kernels

make install-kernels

• make the initrd's we need - note, if modules are needed in the initrd, they get added to "/etc/mkinitrd/modules" in the order they are required. For Scientific Linux 3.0.4 on an ext3 filesystem, we need "jbd, ext3"

mkinitrd -o /boot/initrd.img-2.6.11-xen0 -r /dev/sda1 2.6.11.12-xen0

mkinitrd -o /boot/initrd.img-2.6.11-xenU -r /dev/sda1 2.6.11.12-xenU

mkinitrd -o /boot/initrd.img-2.4.30-xenU -r /dev/sda1 2.4.30-xenU

• edit the grub menu, adding the following lines

title           Xen 2 / Linux 2.6
kernel          /boot/xen-2.0.7.gz dom0_mem=131072 noht
module          /boot/vmlinuz-2.6.11-xen0 ro root=/dev/sda1 ramdisk_size=32768 console=tty 2
module          /boot/initrd.img-2.6.11-xen0

• On Ubuntu Hoary with Python-2.4, there is a known problem with some essential directories not being created.

mkdir /var/lib/xen

mkdir /var/lib/xen/xend-db

mkdir /var/lib/xen/xend-db/domain

mkdir /var/lib/xen/xend-db/vnet

• Xen doesn't play nicely with the Linux TLS library

mv /lib/tls /lib/tls.disabled

• reboot into your new xen machine and see if if works!

• If you got this far, great! Now to start the xen daemon and see what happens.

/etc/init.d/xend start

xm list

If you see output similar to:

Name              Id  Mem(MB)  CPU  State  Time(s)  Console
Domain-0           0      123    0  r----   1161.3

then you are ready to start building a guest VM.

• finally, set xend to start on boot

cd /etc/rc2.d/

ln -sf ../init.d/xend S99xend

Using rpmstrap to build a base system

• homepage: http://hackers.progeny.com/~sam/rpmstrap/

• get and install it

cd /usr/local/src

wget http://hackers.progeny.com/~sam/rpmstrap/releases/rpmstrap-0.4.tar.bz2

tar -jxvf rpmstrap-0.4.tar.bz2

cd rpmstrap-0.4

Either install it or modify the rpmstrap script to look locally for it's lib directory.

• create the file list for Scientific Linnux 3.0.4

cp lib/scripts/centos3 lib/scripts/sl304

vi lib/scripts/sl304

remove the lines containing package names and replace with the following:

0:setup-2.5.27-1.noarch.rpm
1:filesystem-2.2.1-3.i386.rpm
2:basesystem-8.0-2.noarch.rpm
3:tzdata-2004e-1.EL.noarch.rpm
4:glibc-common-2.3.2-95.30.i386.rpm
5:libgcc-3.2.3-49.i386.rpm
6:glibc-2.3.2-95.30.i386.rpm
7:mktemp-1.5-18.i386.rpm
8:termcap-11.0.1-17.1.noarch.rpm
9:libtermcap-2.0.8-35.i386.rpm
10:bash-2.05b-29.0.3.i386.rpm
11:zlib-1.1.4-8.1.i386.rpm
12:chkconfig-1.3.11-0.3.i386.rpm
13:glib-1.2.10-11.1.i386.rpm
14:words-2-21.noarch.rpm
15:cracklib-2.7-22.i386.rpm
15:cracklib-dicts-2.7-22.i386.rpm
16:pcre-3.9-10.1.i386.rpm
17:libattr-2.2.0-1.i386.rpm
18:libacl-2.2.3-1.i386.rpm
19:laus-libs-0.1-66RHEL3.i386.rpm
20:iproute-2.4.7-11.30E.1.i386.rpm
21:popt-1.8.2-13.i386.rpm
22:coreutils-4.5.3-26.i386.rpm
22:db4-4.1.25-8.i386.rpm
22:dev-3.3.12.3-1.i386.rpm
22:e2fsprogs-1.32-15.1.i386.rpm
22:ethtool-1.8-3.3.i386.rpm
22:findutils-4.1.7-9.i386.rpm
22:gawk-3.1.1-9.i386.rpm
22:grep-2.5.1-24.1.i386.rpm
22:gpm-1.19.3-27.2.i386.rpm
22:info-4.5-3.i386.rpm
22:initscripts-7.31.18.EL-1.i386.rpm
22:iputils-20020927-11.30.1.i386.rpm
22:mingetty-1.06-1.i386.rpm
22:modutils-2.4.25-14.EL.i386.rpm
22:mount-2.11y-31.2.i386.rpm
22:ncurses-5.3-9.3.i386.rpm
22:net-tools-1.60-20E.1.i386.rpm
22:pam-0.75-62.i386.rpm
22:procps-2.0.17-13.i386.rpm
22:psmisc-21.3-1.RHEL.0.i386.rpm
22:sed-4.0.7-3.i386.rpm
22:shadow-utils-4.0.3-22.02.i386.rpm
22:sysklogd-1.4.1-12.3.i386.rpm
22:SysVinit-2.85-4.2.i386.rpm
22:util-linux-2.11y-31.2.i386.rpm
22:which-2.14-7.i386.rpm
23:glib2-2.2.3-2.0.i386.rpm
24:sl-release-3.0.4-7.4.noarch.rpm
25:readline-4.3-5.2.i386.rpm
26:less-378-12.i386.rpm
27:gzip-1.3.3-9.i386.rpm
28:tar-1.13.25-13.i386.rpm
29:lvm-1.0.8-9.i386.rpm
30:losetup-2.11y-31.2.i386.rpm
31:mkinitrd-3.5.13-1.i386.rpm
32:file-3.39-9.i386.rpm
33:kernel-2.4.21-27.0.2.EL.i686.rpm
34:hwdata-0.101.14-1.noarch.rpm
35:cpio-2.5-3.i386.rpm
36:MAKEDEV-3.3.12.3-1.i386.rpm
37:beecrypt-3.0.1-0.20030630.i386.rpm
38:bzip2-libs-1.0.2-11.i386.rpm
39:bzip2-1.0.2-11.i386.rpm
40:elfutils-libelf-0.91-3.i386.rpm
40:binutils-2.14.90.0.4-35.i386.rpm
41:elfutils-0.91-3.i386.rpm
42:gdbm-1.8.0-20.i386.rpm
43:gmp-4.1.2-5.i386.rpm
44:krb5-libs-1.2.7-38.i386.rpm
45:openssl-0.9.7a-33.12.i386.rpm
46:libxml2-2.5.10-7.i386.rpm
47:expat-1.95.5-6.i386.rpm
48:python-2.2.3-6.i386.rpm
49:libxml2-python-2.5.10-7.i386.rpm
49:perl-5.8.0-89.10.i386.rpm
49:perl-Filter-1.29-3.i386.rpm
49:patch-2.5.4-16.i386.rpm
50:rpmdb-SL-304-0.20050211.i386.rpm
50:rpm-build-4.2.3-13.i386.rpm
50:rpm-libs-4.2.3-13.i386.rpm
50:rpm-4.2.3-13.i386.rpm
51:rpm-python-4.2.3-13.i386.rpm
52:wget-1.8.2-15.i386.rpm
53:yum-conf-304-2.SL.noarch.rpm
53:yum-2.0.7-9.SL.noarch.rpm
54:nano-1.2.1-4.i386.rpm
55:openldap-2.0.27-17.i386.rpm
55:cyrus-sasl-2.1.15-10.i386.rpm
55:cyrus-sasl-md5-2.1.15-10.i386.rpm
56:libuser-0.51.7-1.EL3.3.i386.rpm
57:passwd-0.68-3.1.i386.rpm
58:jfsutils-1.1.2-2.i386.rpm
59:libstdc++-3.2.3-49.i386.rpm
59:apt-0.5.15cnc6-4.SL.i386.rpm
60:vim-minimal-6.3.046-0.30E.1.i386.rpm
61:iptables-1.2.8-12.3.i386.rpm
62:openssh-3.6.1p2-33.30.3.i386.rpm
62:openssh-clients-3.6.1p2-33.30.3.i386.rpm
62:openssh-server-3.6.1p2-33.30.3.i386.rpm
62:tcp_wrappers-7.6-34.1.i386.rpm

How-To create a Scientific Linux 3.0.4 guest

Assumptions:
1. You have successfully booted into Xen.
2. You wish to use logical volumes for the guest filesystems.
3. You have a dedicated partition for the logical volumes which will be created.

• Initialise and prepare the parition: assuming that the partition which will house the logical volumes is /dev/sda5

pvcreate /dev/sda5

vgcreate xen /dev/sda5

• create the logical volumes: 5Gb ext3 volume and 1Gb swap volume

lvcreate -L 5G -n /dev/xen/sl304-fs (or whatever you want to call the filesystem)

mkfs -t ext3 /dev/xen/sl304-fs

lvcreate -L 1G -n /dev/xen/sl304-sw

mkswap /dev/xen/sl304-sw

• bootsrap a base system

mkdir /mnt/slbase

mount -o loop /dev/xen/sl304-fs /mnt/slbase

Rpmstrap can either download the required packages or get them from a local source (ie disk). Since I chose to use a local source, I first copied all of the rpms off the installation disks.

mkdir /usr/local/src/sl304-rpms

cp all files into this directory

/usr/local/src/rpmstrap --local-source sl304-rpms/ sl304 /mnt/slbase/

In the command above, "sl304-rpms" is the directory containing the rpms, "sl304" is the script (found in rpmstrap-0.4/lib/scripts/) and "/mnt/slbase/" is where the blank filsystem is mounted. If everything is ok, this will "build" you a basic Scientific Linux 3.0.4 base system.

• customise

change to the root of the new filesystem:

chroot /mnt/slbase

rebuild the rpm database:

rpm --rebuilddb

edit the filesystem table:

vi /etc/fstab

/dev/sda1               /                       ext3    defaults        1 1
none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /dev/shm                tmpfs   defaults        0 0
none                    /proc                   proc    defaults        0 0
/dev/sda2               swap                    swap    defaults        0 0

edit the hosts file:

vi /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost

edit the network file:

vi /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=fully.qualified.domain.name:

edit the network interface file:

vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
IPADDR=192.168.10.10
NETMASK=255.255.255.0
NETWORK=192.168.10.1
BROADCAST=192.168.10.255
ONBOOT=yes
NAME=ethernet

or, if your host will get an address via dhcp:

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet

set up some apt sources - since I'm working with LCG middleware, I use the following:

vi /etc/apt/sources.list.d/cern.list

rpm http://linuxsoft.cern.ch  cern/slc305/i386/apt  os updates extras
rpm-src http://linuxsoft.cern.ch  cern/slc305/i386/apt  os updates extras
rpm http://grid-deployment.web.cern.ch/grid-deployment/gis apt/LCG_CA/en/i386 lcg
rpm http://grid-deployment.web.cern.ch/grid-deployment/gis apt/LCG-2_6_0/sl3/en/i386 lcg_sl3 lcg_sl3.updates

get out of the change root environment (ctrl^D):

exit chroot

copy the hosts resolv.conf file so that we have a DNS name service:

cp /etc/resolv.conf /mnt/slbase/etc/

copy the domU modules to the new filesystem:

cp -a /lib/modules/2.4.30-xenU/ slbase/lib/modules/

• umount the filesystem

umount /mnt/slbase

• create a config file for this virtual machine

vi /etc/xen/slbase

kernel="/boot/vmlinuz-2.4-xenU"
ramdisk="/boot/initrd.img-2.4.30-xenU"
disk=[ 'phy:xen/sl304-fs,sda1,w',
       'phy:xen/sl304-sw,sda2,w' ]
root="/dev/sda1"
memory=512
ip="192.168.10.10"
netmask="255.255.255.0"
gateway="192.168.10.1"
broadcast="192.168.10.255"
vif= [ 'mac=00:02:A5:0D:3D:AF, script=/root/xen/network-scripts/cfg-vm-route.sh, vifname=vif1.0' ]
hostname="lcg01.vpac.org"

Notes:

1. useful to assign a mac address to the guest VM rather than have a new one generated by Xen each time it boots up.
   
2. useful to assign the VM to a virtual interface (vif1.0 in this case). This is especially handy when using a routed private network (more later)
   

• start up the guest and see

xm create -c slbase