Diskless Image Booting : An Introduction
It is generally very useful to test the kernels without installing them first onto hard disk and then rebooting the machine to boot the newly compiled kernel image. Although now we can make use of "kexec", booting the kernel from network is something I find very useful.
In this article I try to explain basics of setting up a server and clients which can boot off the server from the network. Clients are assumed to have at least one Ethernet Port connected to the same network as server.
There are two different approaches for diskless booting in general. One is to use a ramdisk as a root partition and the other one is to use root partition over nfs. In the following, both these approaches are examined.
Prerequisites
NFS Server setup
Using yum or yast, install the package nfs-utils-<version> depending on your distribution. Once installed, modify /etc/exports with the directory entry and other options. Then execute the following command to start the server.
[root@turyx ~]# /etc/init.d/nfs start [root@turyx ~]# /etc/init.d/portmap start
To make these settings permanent,
[root@turyx ~]# chkconfig --level 345 nfs on [root@turyx ~]# chkconfig --level 345 portmap on
Sample /etc/exports file
/tftpboot/installroot 192.168.10.0/255.255.255.0(rw,async,no_root_squash)
no_root_squash directive is very important for root over nfs.
PXE server setup
Get neopxe PXE server from http://sourceforge.net/projects/neopxe/ Install and run it. You may want to create an entry in /etc/rc.sysinit to make neopxe run every time machine boots up. Edit the configuration file (typically /usr/local/etc/neopxe.conf) and make entries corresponding to the setup.
Sample neopxe.conf file
ip_addr=192.168.10.1 prompt=Press [F8] for menu... prompt_timeout=10 service=1,192.168.10.1,pxelinux.0,Linux
TFTP server setup
Using yum or yast (depending on your distribution), install the package tftp-server and xinetd. Edit /etc/xinted.d/tftp to set the tftp root directory, server executable ..etc.
Sample /etc/xinetd.d/tftp file
service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/local/sbin/in.tftpd
server_args = -s /tftpboot -r blksize
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}DHCP server setup
Configuring a DHCP server wrongly can bring down the whole network in which it operates. Be very careful if you require to configure a DHCP server. If you have more than one ports connecting to different networks, please consult the man pages to find out how to restrict dhcp service to one or more ports. Using yum or yast, install dhcp-<version>.rpm. Configure /etc/dhcpd.conf with the range of network addresses.
Sample dhcp.conf file
ddns-update-style ad-hoc;
class "PXE" {
match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
option vendor-class-identifier "PXEClient";
}
subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.10 192.168.10.254;
default-lease-time 1200;
max-lease-time 1800;
option subnet-mask 255.255.255.0;
}Diskless booting using ramdisk.
A ramdisk is a portion of the memory used as a file or directory. To boot linux, a variety of file systems like ext2, ext3, reiserfs, minix ..etc. are available to mount root (/) file system. Since ramdisks are not persistent (changes are lost when the machine is powered off), journaling file systems like ext3 and JFS generally do not add much value to a ramdisk and subsequently are not used as a filesystem on a ramdisk. In this document, it is assumed that the development machine runs the same distribution as the target. This makes creation of a ramdisk (or the root file system to be mounted over nfs) a lot easier.
Limitations of a ramdisk
One of the limitation of a ramdisk is that it cannot grow or shrink dynamically. Moreover, the memory allocated to hold the ramdisk is marked reserved in the kernel memory and never freed. These factors should be taken into consideration while planning for a ramdisk. In the following example, we create a minimal ramdisk on Fedora.
Preparing a target directory in Fedora
Please make sure that you have yum and rpm packages installed on your host machine by executing the following commands. Also, you need superuser privileges to execute some of the following commands.
[root@turyx ~]$ rpm --version RPM version 4.4.2 [root@turyx ~]$ yum --version 3.0.6
If any of these commands are not installed, please install the missing rpm before proceeding.
Note : Redhat users should be registered with RedHat Network and should have an active subscription for the 'yum' command to find repositories. So please activate your subscription before proceeding.
We need a target directory where the packages would be installed. Create a directory 'installroot' in /tftpboot.
[root@turyx ~]# mkdir -p /tftpboot/installroot
In order to make yum install packages onto this directory (as opposed to /), we can use either the command line parameter –installroot=<directory> or modify the /etc/yum.conf to have a directive installroot=<directory>. For more details on this, please refer man yum(8)
For a minimal installation, use the following command.
[root@turyx ~]# yum clean all [root@turyx ~]# yum -y --installroot=/tftpboot/installroot install basesystem filesystem bash
Once this command is completed without errors (it would take some time), check the size of the directory using the command du
[root@turyx ~]# du -sh /tftpboot/installroot 279M /tftpboot/installroot
Creating ramdisk and copying files
Now create a zero filled file for the ramdisk image (of size +~200M of the directory) and format it with ext2 file system.
[root@turyx /tftpboot]# dd if=/dev/zero of=/tftpboot/ramdisk.ext2 bs=1k count=524288
524288+0 records in
524288+0 records out
536870912 bytes (537 MB) copied, 14.0037 seconds, 38.3 MB/s
[root@turyx /tftpboot]# mkfs.ext2 -v -m 0 -b 4096 ./ramdisk.ext2
mke2fs 1.39 (29-May-2006)
./ramdisk.ext2 is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
131072 inodes, 131072 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=134217728
4 block groups
32768 blocks per group, 32768 fragments per group
32768 inodes per group
Superblock backups stored on blocks:
32768, 98304
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.Mount this file and copy the contents of the installroot directory, then unmount the directory and gzip the file.
[root@turyx ~]# cd /tftpboot [root@turyx /tftpboot]# mount -o loop -t ext2 ./ramdisk.ext2 ./ramdisk [root@turyx /tftpboot]# mount -o loop -t ext2 ./ramdisk.ext2 ./ramdisk.dir [root@turyx /tftpboot]# rsync -av ./installroot/ ./ramdisk.dir [root@turyx /tftpboot]# umount ./ramdisk.dir/ [root@turyx /tftpboot]# cat ramdisk.ext2 | gzip -9 > ramdisk.ext2.gz
Note : The ramdisk we just created is a bare minimal root file system. It would not contain the kernel modules most likely required by the kernel you wish to boot unless explicitly installed. If you are compiling your own kernel, use the tips at http://kernelnewbies.org/FAQ/KernelCrossCompilation to install modules at the correct location before copying the required files to ramdisk.
You can make it a little better by adding nfs, nfsutils, autofs.
[root@turyx ~]# yum install -y autofs nfs nfs-utils-lib nfs-utils dhclient
To start networking, you require configuration files. Sample configuration file is here.
[root@turyx ~]# cat ./etc/sysconfig/network-scripts/ifcfg-eth1 # Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet DEVICE=eth1 BOOTPROTO=dhcp ONBOOT=yes
You need to enable networking as well.
[root@turyx ~]# cat ./etc/sysconfig/network NETWORKING=yes
If your kernel is compiled with modules, don't forget to install the modules in the right place.
Using CPIO based initramfs
Since ramdisk takes up the space and never frees it, another approach is to use cpio based initramfs. For this, only a couple of changes are required.
- Create a link to init in the root
[root@turyx /tftpboot/installroot]# ln -s ./sbin/init ./init
- Create the cpio archieve (initramfs based root image)
[root@turyx /tftpboot/installroot]# find | cpio -ocv | gzip -9 > ../initrd.cpio.gz
Don't forget to change the initrd file name in the pxe configuration file as well.
Sample pxelinux configuration
label linux-initrd
MENU LABEL Ramdisk kernel
kernel os_images/fc6/vmlinuz
append initrd=os_images/fc6/initrd-2.6.23-omyx.img ramdisk_size=524288 load_ramdisk=1 root=/dev/ram0 rw console=ttyS0 console=tty0 debug Root over nfs.
Although pretty easy to create and use, mounting root file system on a ramdisk has a number of limitations like limited size, lack of flexibility, lack of persistence ..etc. If you wish to run a full distribution with X windows etc., ramdisk may not be the best solution. For such scenarios, mounting the root file system over nfs would be a better approach.
Configuring the kernel
cd to your kernel source tree and run make menuconfig
[root@turyx /tftpboot]# cd /space/src/linux-2.6/ [root@turyx /space/src/linux-2.6]# make menuconfig
Make sure “TCP/IP networking" is selected. Select IP:Kernel level autoconfiguration and IP:DHCP support, IP:BOOTP support and IP:RARP support as shown in the image. If you know the boot protocol your server supports, you need not select all the three. But selecting all three is harmless. In “File Systems”-> “Network File Systems” , make sure “Root File System on NFS” is selected.
If these two options are selected, root file system over nfs can be activated on the kernel command line.
Root over NFS: booting
Parameters in the command line
root=/dev/nfs
/dev/nfs is a pseudo device (which may not exist) to inform the kernel that root is over nfs, and not a real device in /dev directory.
nfsroot=[<server-ip>:]<root-dir>[,<nfs-options>]
If this parameter is NOT supplied in the kernel command line, the default is “/tftpboot/%s” where %s is replaced by the ascii representation of the client IP address. Check "Documentation/nfsroot.txt" for your kernel's supported NFS option list.
ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>
If the server is configured to supply these parameters, ip=<autoconf> would be sufficient. In case of auto configuration, if the client has multiple network ports, and device=<device> is not supplied, autoconfiguration requests would be sent to all available devices and the first device received the reply would be chosen.
Sample pxelinux configuration
label NFS Autoconf linux-2.6.23-Net
MENU LABEL NFS linux-2.6.23-net
kernel os_images/fc6/bzImage.net
append netdev=eth1 root=/dev/nfs rw ip=192.168.10.10:192.168.10.1:192.168.10.1:255.255.255.0:testsrv:eth1:none nfsroot=192.168.10.1:/tftpboot/installroot console=ttyS0 console=tty0 debugEnd of Document.