Diskless Image Booting : An Introduction

It is generally very useful to test the kernels without installing them first onto hard disk and then rebooting the machine to boot the newly compiled kernel image. Although now we can make use of "kexec", booting the kernel from network is something I find very useful.

In this article I try to explain basics of setting up a server and clients which can boot off the server from the network. Clients are assumed to have at least one Ethernet Port connected to the same network as server.

There are two different approaches for diskless booting in general. One is to use a ramdisk as a root partition and the other one is to use root partition over nfs. In the following, both these approaches are examined.

Prerequisites

NFS Server setup

Using yum or yast, install the package nfs-utils-<version> depending on your distribution. Once installed, modify /etc/exports with the directory entry and other options. Then execute the following command to start the server.

[root@turyx ~]# /etc/init.d/nfs start
[root@turyx ~]# /etc/init.d/portmap start

To make these settings permanent,

[root@turyx ~]# chkconfig --level 345 nfs on
[root@turyx ~]# chkconfig --level 345 portmap on

Sample /etc/exports file

/tftpboot/installroot     192.168.10.0/255.255.255.0(rw,async,no_root_squash)

no_root_squash directive is very important for root over nfs.

PXE server setup

Get neopxe PXE server from http://sourceforge.net/projects/neopxe/ Install and run it. You may want to create an entry in /etc/rc.sysinit to make neopxe run every time machine boots up. Edit the configuration file (typically /usr/local/etc/neopxe.conf) and make entries corresponding to the setup.

Sample neopxe.conf file

ip_addr=192.168.10.1
prompt=Press [F8] for menu...
prompt_timeout=10
service=1,192.168.10.1,pxelinux.0,Linux

TFTP server setup

Using yum or yast (depending on your distribution), install the package tftp-server and xinetd. Edit /etc/xinted.d/tftp to set the tftp root directory, server executable ..etc.

Sample /etc/xinetd.d/tftp file

service tftp
{
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/local/sbin/in.tftpd
        server_args             = -s /tftpboot -r blksize
        disable                 = no
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

DHCP server setup

Configuring a DHCP server wrongly can bring down the whole network in which it operates. Be very careful if you require to configure a DHCP server. If you have more than one ports connecting to different networks, please consult the man pages to find out how to restrict dhcp service to one or more ports. Using yum or yast, install dhcp-<version>.rpm. Configure /etc/dhcpd.conf with the range of network addresses.

Sample dhcp.conf file

ddns-update-style ad-hoc;
class "PXE" {
        match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
        option vendor-class-identifier "PXEClient";
}

subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.10 192.168.10.254;
default-lease-time 1200;
max-lease-time 1800;
option subnet-mask 255.255.255.0;
}

Diskless booting using ramdisk.

A ramdisk is a portion of the memory used as a file or directory. To boot linux, a variety of file systems like ext2, ext3, reiserfs, minix ..etc. are available to mount root (/) file system. Since ramdisks are not persistent (changes are lost when the machine is powered off), journaling file systems like ext3 and JFS generally do not add much value to a ramdisk and subsequently are not used as a filesystem on a ramdisk. In this document, it is assumed that the development machine runs the same distribution as the target. This makes creation of a ramdisk (or the root file system to be mounted over nfs) a lot easier.

Limitations of a ramdisk

One of the limitation of a ramdisk is that it cannot grow or shrink dynamically. Moreover, the memory allocated to hold the ramdisk is marked reserved in the kernel memory and never freed. These factors should be taken into consideration while planning for a ramdisk. In the following example, we create a minimal ramdisk on Fedora.

Preparing a target directory in Fedora

Please make sure that you have yum and rpm packages installed on your host machine by executing the following commands. Also, you need superuser privileges to execute some of the following commands.

[root@turyx ~]$ rpm --version
RPM version 4.4.2
[root@turyx ~]$ yum --version
3.0.6

If any of these commands are not installed, please install the missing rpm before proceeding.

Note : Redhat users should be registered with RedHat Network and should have an active subscription for the 'yum' command to find repositories. So please activate your subscription before proceeding.

We need a target directory where the packages would be installed. Create a directory 'installroot' in /tftpboot.

[root@turyx ~]# mkdir -p /tftpboot/installroot

In order to make yum install packages onto this directory (as opposed to /), we can use either the command line parameter –installroot=<directory> or modify the /etc/yum.conf to have a directive installroot=<directory>. For more details on this, please refer man yum(8)

For a minimal installation, use the following command.

[root@turyx ~]# yum clean all
[root@turyx ~]# yum -y --installroot=/tftpboot/installroot install basesystem filesystem bash

Once this command is completed without errors (it would take some time), check the size of the directory using the command du

[root@turyx ~]# du -sh /tftpboot/installroot
279M    /tftpboot/installroot

Creating ramdisk anc copying files

Now create a zero filled file for the ramdisk image (of size +~200M of the directory) and format it with ext2 file system.

[root@turyx /tftpboot]# dd if=/dev/zero of=/tftpboot/ramdisk.ext2 bs=1k count=524288
524288+0 records in
524288+0 records out
536870912 bytes (537 MB) copied, 14.0037 seconds, 38.3 MB/s
[root@turyx /tftpboot]# mkfs.ext2 -v -m 0 -b 4096 ./ramdisk.ext2 
mke2fs 1.39 (29-May-2006)
./ramdisk.ext2 is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
131072 inodes, 131072 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=134217728
4 block groups
32768 blocks per group, 32768 fragments per group
32768 inodes per group
Superblock backups stored on blocks: 
        32768, 98304

Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Mount this file and copy the contents of the installroot directory, then unmount the directory and gzip the file.

[root@turyx ~]# cd /tftpboot
[root@turyx /tftpboot]# mount -o loop -t ext2 ./ramdisk.ext2 ./ramdisk
[root@turyx /tftpboot]# mount -o loop -t ext2 ./ramdisk.ext2 ./ramdisk.dir
[root@turyx /tftpboot]# rsync -av ./installroot/ ./ramdisk.dir
[root@turyx /tftpboot]# umount ./ramdisk.dir/
[root@turyx /tftpboot]# cat ramdisk.ext2 | gzip -9 > ramdisk.ext2.gz

Note : The ramdisk we just created is a bare minimal root file system. It would not contain the kernel modules most likely required by the kernel you wish to boot unless explicitly installed. If you are compiling your own kernel, use the tips at http://kernelnewbies.org/FAQ/KernelCrossCompilation to install modules at the correct location before copying the required files to ramdisk.

Sample pxelinux configuration

label linux-initrd
        MENU LABEL Ramdisk kernel
        kernel os_images/fc6/vmlinuz
        append initrd=os_images/fc6/initrd-2.6.23-omyx.img ramdisk_size=524288 load_ramdisk=1 root=/dev/ram0 rw console=ttyS0 console=tty0 debug 

Root over nfs.

Although pretty easy to create and use, mounting root file system on a ramdisk has a number of limitations like limited size, lack of flexibility, lack of persistence ..etc. If you wish to run a full distribution with X windows etc., ramdisk may not be the best solution. For such scenarios, mounting the root file system over nfs would be a better approach.

Configuring the kernel

cd to your kernel source tree and run make menuconfig

[root@turyx /tftpboot]# cd /space/src/linux-2.6/
[root@turyx /space/src/linux-2.6]# make menuconfig

Make sure “TCP/IP networking" is selected. Select IP:Kernel level autoconfiguration and IP:DHCP support, IP:BOOTP support and IP:RARP support as shown in the image. If you know the boot protocol your server supports, you need not select all the three. But selecting all three is harmless. In “File Systems”-> “Network File Systems” , make sure “Root File System on NFS” is selected.

If these two options are selected, root file system over nfs can be activated on the kernel command line.

Root over NFS: booting

Parameters in the command line

root=/dev/nfs 

/dev/nfs is a pseudo device (which may not exist) to inform the kernel that root is over nfs, and not a real device in /dev directory.

nfsroot=[<server-ip>:]<root-dir>[,<nfs-options>]

If this parameter is NOT supplied in the kernel command line, the default is “/tftpboot/%s” where %s is replaced by the ascii representation of the client IP address. Check "Documentation/nfsroot.txt" for your kernel's supported NFS option list.

ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>

If the server is configured to supply these parameters, ip=<autoconf> would be sufficient. In case of auto configuration, if the client has multiple network ports, and device=<device> is not supplied, autoconfiguration requests would be sent to all available devices and the first device received the reply would be chosen.

Sample pxelinux configuration

label NFS Autoconf linux-2.6.23-Net
        MENU LABEL NFS linux-2.6.23-net
        kernel os_images/fc6/bzImage.net
        append netdev=eth1 root=/dev/nfs rw ip=192.168.10.10:192.168.10.1:192.168.10.1:255.255.255.0:testsrv:eth1:none nfsroot=192.168.10.1:/tftpboot/installroot console=ttyS0 console=tty0 debug

End of Document.