Date: Thu, 13 Jan 2005 16:08:03 +1300
>From: "Tom Hibbert" <tom@nsp.co.nz >
>To: <xen-devel@lists.sourceforge.net >
>Subject: [Xen-devel] Debian Sarge Root Raid + LVM + XEN install guide (LONG)

>Hello fellow xenophiles and happy new year!

>I've documented the install procedure for a prototype server here since I found no similar document anywhere on the net. It's a Sarge-based Domain0 on linux root raid from scratch, using LVM to store the data for the domU mail server and its mailstore. I humbly submit my notes in the hope that they are useful to some weary traveller.

>Have fun!

>
Debian Sarge XEN dom0 with Linux Root Raid and LVM

>
Hardware: 
           P4 3.2ghz LG775
>            Asus P5GD1-VM
            1gb DDR400 DRAM
>            2x80gb Seagate SATA disks

>
Reasons for using software raid (over Intel ICH raid or more expensive SCSI raid)
Speed
>Bonnie++ shows Linux Software Raid is MUCH faster than ICH5 (at least under Linux)
Reliability
>I have observed that frequent disk access with small files has destroyed ICH5 raid arrays in the past (at least under Linux)
Recovery
>I had a bad experience with the death of an Adaptec 3200S controller not long ago. The array was nonrecoverable because a replacement card could not be sourced in time. Additionally the firmware revision for the 3200s was unknown. (Recovery from controller death if even possible requires the same firmware revision as the original card, since that was not known we would have had to guess which takes time and time is money when you have a dead server)
Price
>Reduce cost of hardware to the client because we arent using expensive raid controllers
Prevalence
>It is much easier to source standard disks than it is to source SCSI disks (in the case of using SCSI raid controllers). It is also much easier to source a standard SATA controller than it is to source a RAID controller.

>
Reasons for using XEN
Recovery
>Putting all network services inside XEN virtual machines that can be backed up makes disaster recovery a non-brainer
Better utilisation of hardware
>Stacking virtual machines allows more efficient use of hardware (cost effectiveness)
It's just cooler :)
Methodology
Setting up the hardware - setting SATA to compatible mode
 Boot off Feather Linux USB key
Partition primary drive
Install base system
Chroot into base system
Install C/C++ development packages
Install XEN packages
Configure/build/install XEN Dom0 kernel
Install GRUB
Reboot to base system and set SATA to enhanced mode
Migrate system into RAID1 and test
Configure/build/install XEN DomU kernel
Configure LVM
Create DomU environment
Install services into DomU*
Configure XEN to boot DomU automatically
Testing*
Deployment*
>
* Not covered by this document
>
1. Setting up the hardware
Standard stuff here. Set the mode for SATA to Compatible so that Feather's kernel was able to access the hard disks.
>
2. Boot off Feather Linux USB key
Feather is fantastic because it allows one to setup a Debian system without having to boot from the now heavily outdated Woody install CD. It supports more hardware and  also allows easy installation to a system without a CDRom drive in a build network without an 'evil' segment (PXE boot). It also makes a convenient rescue platform.
>
http://featherlinux.berlios.de 
>
3. Partition primary drive
Feather Linux does not properly support the ICHx and it doesnt have the administration tools for making raid arrays. Therefore the setup method we will use is to build the base system on a single disk and then migrate it into RAID1. Trust me, this is much easier than it sounds!
>
I partitioned the primary drive as follows
>
   Device Boot      Start         End      Blocks   Id  System
>dev/hda1               1           3       24066   fd  Linux raid autodetect
>dev/hda2               4         501     4000185   fd  Linux raid autodetect
>dev/hda3             502        9605    73127880   fd  Linux raid autodetect
>dev/hda4            9606        9729      996030   fd  Linux raid autodetect
>
using hda2 for root and hda1 for boot with swap on hda4. hda3 is not used yet.
>
Format and mount up the drive to /target:
>
# mkdir target
># mkfs.ext3 dev/hda1
># mkfs.ext3 dev/hda2
># mount dev/hda2 /target
># mkdir target/boot
># mount dev/hda1 /target/boot
4. Install the base system
Set up Feather with APT and debootstrap:
>
# dpkg-get
# apt-get install debootstrap
Install the base system
>
# debootstrap sarge target
Perform basic configuration
>
# vi target/etc/fstab
>
/dev/sda2        /       ext3    defaults        0       1
>dev/sda1        /boot   ext3    defaults        0       2
>proc             proc   proc    defaults        0       0
>
You may be asking why am I putting sda here? The reason is because once I set the ICH6 to use Enhanced Mode and reboot into the fresh 2.6.9 xen0
>kernel with SATA support compiled the drives appear as SCSI devices. hda will be enumerated as dev/sda.
>
5. Chroot into base system
# umount /dev/hda1
># cd target
># chroot .
# su -
># mount dev/hda1 /boot
Unmounting and remounting boot is important for configuring GRUB later.
>
Some more configuration needs to be done at this point:
>
# rm etc/resolv.conf
># rm etc/hostname
># echo xen0-test > etc/hostname
># echo nameserver 210.55.13.3  > /etc/resolv.conf
>
6. Install C/C++ packages
# apt-setup
># apt-get update
# dselect update
># tasksel
(Select C/C++ development packages)
>
7. Install XEN packages
Until Adam's packages get released I am using some homebrew packages descended from Brian's original work.
>
# mkdir xen
# cd xen
># apt-get install wget
# wget -r http://cryptocracy.hn.org/xen/ 
># cd cryptocracy.hn.org/xen 
># dpkg -i *.deb
# apt-get -f install
8. Configure/build/install XEN dom0 kernel
Since this is the first time configuring XEN on this hardware I am building the kernel from scratch. When we get more of these servers I will install a prebuilt debianised kernel on them.
>
# cd usr/src/
># tar -jxvf .kernel-source-2.6.9_2.6.9-3_all.deb
># cd kernel-source-2.6.9
# export ARCH=xen
># cp ~xen/cryptocracy.hn.org/xen/config.xen0 .config
># make menuconfig

>		(Make changes as appropriate for this hardware)
>