The kernel, patches, trees and compilation
1.1 The kernel
The current version of the Linux kernel can always be downloaded from The Linux Kernel Archives web page (http://www.kernel.org/) in the form of a large tar file compressed with either gzip or bzip2, which is called the kernel tarball (it is worthy of noting that the bz2 files are smaller than the gz ones, but gzip is generally faster than bzip2).
After downloading the tarball you can extract the kernel’s source code from it in many different ways (the virtue of UNIX-like systems). Our favorite method is to run either
$ tar xjvf linux-2.6.x.y.tar.bz2
for the tarballs compressed with bzip2, or
$ tar xzvf linux-2.6.x.y.tar.gz
for the ones compressed with gzip, in the directory that we want to contain the kernel sources. In principle, it can be an arbitrary directory, but it should be located on a partition with as much as 2 GB of free space.
Of course, you can also follow the kernel’s README literally and extract the source code by executing one of the following instructions:
$ bzip2 -dc linux-2.6.x.y.tar.bz2 | tar xvf - $ gzip -cd linux-2.6.x.y.tar.gz | tar xvf -
depending on what kind of a tarball you have downloaded. In any case it is not recommended to unpack the kernel tarball as root.
As a result of unpacking the kernel tarball we get a directory the name of which corresponds to the version of the kernel that will be obtained by compiling the downloaded source code (eg. linux-2.6.18). The contents of this directory are often referred to as the kernel tree. For this reason the word tree is used for naming different development branches of the kernel. For instance, instead of saying the development branch of the kernel maintained by Andrew Morton it is more convenient to say the -mm tree.
1.2 Patches
It often is necessary to update the kernel source code, but you should not download the entire tarball every time, since that would be bandwidth-consuming and tedious. Usually, it is sufficient to download a patch that can be applied to an existing kernel tree from within the directory in which the tree is located. For example, if the patch in question has been compressed with bzip2, this can be done in one of the following ways:
$ bzip2 -cd /path/to/patch-2.6.x.bz2 | patch -p1 $ bzcat /path/to/patch-2.6.x.bz2 | patch -p1
If the patch has been compressed with gzip, you can apply it analogously:
$ gzip -cd /path/to/patch-2.6.x.gz | patch -p1 $ zcat /path/to/patch-2.6.x.gz | patch -p1
and for uncompressed patches you can use cat instead of either zcat or bzcat. Of course, these are only the simplest methods of applying patches and it is quite easy to invent some more interesting ones.
It is very useful to be able to revert patches that are no longer needed (or buggy). It can be accomplished by adding the -R flag to the patch command line:
$ bzcat /path/to/patch-2.6.x.bz2 | patch -p1 -R
It is also possible to check if the patch will apply cleanly (ie. it does not contain any pieces of code that will be rejected) with the help of the --dry-run command-line option of patch:
$ bzcat /path/to/patch-2.6.x.bz2 | patch -p1 --dry-run
Then, if there are no messages similar to
1 out of 1 hunk FAILED -- saving rejects to file xxx
the patch can be safely applied.
Most of different versions of the kernel source code are available as sets of patches (patchsets) or even as individual patches that should be applied on top of specific kernel trees. For this reason the patches are labeled in reference with the kernel trees to which they should be applied. Namely, since stable kernel versions are labeled as 2.6.x or 2.6.x.y, development trees are called 2.6.x-git*, 2.6.z-rc* and 2.6.z-rc*-git*, where the -rc patch with z equal to x plus one should be applied on top of the stable kernel 2.6.x. This means, for example, that the patch called patch-2.6.21-rc3.bz2 should be applied on top of the linux-2.6.20 kernel tree and the patch labeled as patch-2.6.21-rc3-git3.bz2 should be applied on top of patch-2.6.21-rc3.bz2.
The following example will hopefully make everyting clear. Assume that we have the stable kernel source code labeled as linux-2.6.16.25.tar.bz2 and we want to obtain the development tree 2.6.20-rc1-git1. First, we unpack the tarball:
$ tar xjvf linux-2.6.16.25.tar.bz2
Next, we fetch the patch called patch-2.6.16.25.bz2 from ftp://ftp.kernel.org and revert it:
$ bzip2 -cd /path/to/patch-2.6.16.25.bz2 | patch -p1 -R
Now, we have the sources of the 2.6.16 kernel in our work directory. This is what we wanted, since the stable and development patches only apply cleanly to the 2.6.x kernel trees, as they can contain some code that is also included in the 2.6.x.y versions. Accordingly, we can apply patch-2.6.17.bz2, patch-2.6.18.bz2 and patch-2.6.19.bz2 to our tree:
$ bzip2 -cd /path/to/patch-2.6.17.bz2 | patch -p1 $ bzip2 -cd /path/to/patch-2.6.18.bz2 | patch -p1 $ bzip2 -cd /path/to/patch-2.6.19.bz2 | patch -p1
to obtain the 2.6.19 kernel source code. Finally, we need to download two development patches, patch-2.6.20-rc1.bz2 and patch-2.6.20-rc1-git1.bz2, and apply them in the order specified.
Isn’t that easy? Well, it is, but it also is tedious and that is why some people who do not like to lose time use the tool called ketchup (http://www.selenic.com/ketchup/), described in the next section.
It should be noted that patches can only be applied if they are properly formatted. If you receive them via email or download them directly from a mailing list archives, the formatting is often changed in a way that makes them unusable. In such a case patch will refuse to apply the patch and you will see a message like patch: **** malformed patch at line. For instance,
$ cat ../sched-2.patch | patch -p1 -R --dry-run patching file kernel/sched.c patch: **** malformed patch at line 33: if (next == rq->idle)
means that the patch called sched-2.patch is damaged and cannot be reverted. Fortunately, if the patch has been posted to the Linux Kernel Mailing List (LKML), it can be downloaded from the archives at http://marc.theaimsgroup.com/?l=linux-kernel in the original form. For this purpose you only need to find the message that contains the patch and click on the link Download message RAW. Still, even this will not work if the original raw message containing the patch is malformed.
For more information on downloading and applying patches please see the files README and Documentation/applying-patches.txt in the kernel sources.
1.3 Ketchup
The ketchup tool allows us to download any version of the kernel source code quickly and easily as well as to transform given kernel tree into another one. It does this quite intelligently, checking the current version and downloading only the required patches.
The users of Debian testing/unstable can simply install the ketchup package with the help of their favorite tool, eg.
# apt-get install ketchup
Still, for the users of Debian stable or another distribution in which the ketchup tool is not available the easiest way of installing it is to do:
$ mkdir ~/bin
unless the subdirectory bin is already present in the user’s home directory, and
$ cd ~/bin $ wget http://www.selenic.com/ketchup/ketchup-0.9.8.tar.bz2 $ tar xjvf ketchup-0.9.8.tar.bz2
It is also necessary to install the python package.
You should remember that ketchup is able to check the signatures of the downloaded patches. For this reason it is recommended to visit the web page at http://www.kernel.org/signature.html and save the current key in a text file (we have never managed to make $ gpg --keyserver wwwkeys.pgp.net --recv-keys 0x517D0F0E work on our systems). Next, you can do
$ gpg --import the_file_containing_the_kernel.org_key
and you are ready to check how the whole thing works. Namely, you can do
$ mkdir ~/tree $ cd ~/tree $ ketchup -m
and you should get an error message similar to the following one:
Traceback (most recent call last):
File "/usr/bin/ketchup", line 695, in ?
lprint(get_ver("Makefile"))
File "/usr/bin/ketchup", line 160, in get_ver
m = open(makefile)
IOError: [Errno 2] No such file or directory: ’Makefile’This only means that there is no Linux kernel tree in the current directory. You can order ketchup to download the latest stable version of the kernel by using the command ”ketchup 2.6-tip”:
$ ketchup 2.6-tip
None -> 2.6.16.1
Unpacking linux-2.6.15.tar.bz2
Applying patch-2.6.16.bz2
Downloading patch-2.6.16.1.bz2
--21:11:47-- http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.16.1.bz2
=> ‘/home/michal/.ketchup/patch-2.6.16.1.bz2.partial’
Resolving www.kernel.org... 204.152.191.5, 204.152.191.37
Connecting to www.kernel.org|204.152.191.5|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5,284 (5.2K) [application/x-bzip2]
100%[====================================>] 5,284 6.19K/s
21:11:49 (6.18 KB/s) - ‘/home/michal/.ketchup/patch-2.6.16.1.bz2.partial’
saved [5284/5284]
Downloading patch-2.6.16.1.bz2.sign
--21:11:49-- http://www.kernel.org/pub/linux/kernel/v2.6/
patch-2.6.16.1.bz2.sign
=> ‘/home/michal/.ketchup/patch-2.6.16.1.bz2.sign.partial’
Resolving www.kernel.org... 204.152.191.37, 204.152.191.5
Connecting to www.kernel.org|204.152.191.37|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 250 [application/pgp-signature]
100%[====================================>] 250 --.--K/s
21:11:49 (18.34 MB/s) - ‘/home/michal/.ketchup/
patch-2.6.16.1.bz2.sign.partial’ saved [250/250]
Verifying signature...
gpg: Signature made wto 28 mar 2006 09:37:31 CEST using DSA key ID 517D0F0E
gpg: Good signature from "Linux Kernel Archives Verification Key
<ftpadmin@kernel.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: C75D C40A 11D7 AF88 9981 ED5B C86B A06A 517D 0F0E
Applying patch-2.6.16.1.bz2Now you can check if you really have the latest stable tree:
$ ketchup -m 2.6.16.1
As you can see, ketchup automatically downloads the necessary patches. It can also take patches from the ~/.ketchup subdirectory of the user’s home directory, so it is a good idea to put the patches that you already have in there.
Making a transition to another kernel tree is absolutely not a problem for ketchup. For this purpose you can simply use the command ”ketchup kernel_version”, eg.
$ ketchup 2.6.16-rc4-mm1 2.6.16.1 -> 2.6.16-rc4-mm1 Applying patch-2.6.16.1.bz2 -R Applying patch-2.6.16.bz2 -R Applying patch-2.6.16-rc4.bz2 Applying 2.6.16-rc4-mm1.bz2
Similarly, if you want to get the latest kernel version from the -rt branch, you can do
$ ketchup 2.6-rt
and the tool will automatically apply or revert the patches for you:
2.6.16-rc4-mm1 -> 2.6.16-rt12 Applying 2.6.16-rc4-mm1.bz2 -R Applying patch-2.6.16-rc4.bz2 -R Applying patch-2.6.16.bz2 Applying patch-2.6.16-rt12
It is also possible to teach ketchup to use patches included in the latest -mm tree (described below), which can be done in the following way:
$ wget -c ‘ketchup -u 2.6-mm | sed "s/.bz2/-broken-out.tar.bz2/"‘
As follows from the above examples, ketchup can recognize several development branches of the Linux kernel source code. In particular, it can handle the following trees:
2.6-tip – the latest "stable" kernel
2.6-rc – the latest "release candidate" kernel
2.6-git – the latest development -git kernel
2.6-rt – the Ingo Molnar’s "real-time" tree
2.6-mm – the Andrew Morton’s tree
2.6-ck – the Con Kolivas’ tree (desktop)
2.6-cks – the Con Kolivas’ tree (server)
Now, you may be wondering why we have shown you all of the commands used for applying and reverting patches. The reason is really simple: if you find and report a bug, you will probably get a fix in the form of a patch and you should be able to correctly apply it.
1.4 Trees
We have already used the term ”kernel tree” for quite a few times and we have told you that it means ”the contents of the directory containing the kernel source code”. However, it also has another meaning which is ”a development branch of the kernel source code”. Usually, such a branch is equivalent to a patchset that can be applied on top of a specific version of the kernel sources and different branches are named differently, so that they can be easily distinguished from each other. The more popular branches, also referred to as ”trees”, are the following:
- The -mm tree (maintained by Andrew Morton), which is built from other trees (ie. development branches of the kernel) and from many individual, often experimental, patches.
- The -rt (real-time) tree (maintained by Ingo Molnar), which contains patches that turn Linux into a real time system.
- The -ck tree (maintained by Con Kolivas), focused on the improvements of the system’s performance.
Many developers and the majority of kernel subsystems maintain their own trees used in the development process. After some time specific parts of these trees are included into the main kernel tree maintained by Linus Torvalds, called the mainline. The -git patches discussed in the previous sections represent the snapshots of the ever-evolving Linus’ tree taken at various instants of time.
1.5 The -mm tree
We have already stated that the -mm tree is a combination of some other trees and individual experimental patches. More importantly, it also is the main battlefield of the fight against bugs in the Linux kernel.
The importance of this tree stems from the fact that it combines the mainline with multiple trees used by the maintainers of kernel subsystems (subsystem trees) including patches considered as experimental and not ready for merging with the mainline. In other words, each release of the -mm tree is sort of a future snapshot of the mainline with some additional changes and with some bugs that (hopefully) are going to be fixed before the patches containing them will get merged. Thus by testing the current -mm kernel we check how the future mainline (ie. stable) kernels may behave and if any bugs are found at this stage, they can be fixed before hitting the mainline.
Still, some patches reach the mainline without appearing in the -mm tree, which is a consequence of the flow of changes in the kernel source code. Namely, the majority of patches merged with the mainline comes from different subsystem trees and they are included in the Linus’ tree when he decides to pull the patches considered as ready for the inclusion from given subsystem tree. It is possible, and it sometimes really happens, that Linus pulls the ”ready to go” patches from certain subsystem tree before they have a chance to appear in -mm (in that case the patches will appear in the -mm tree anyway, but they will come into it from the mainline rather than from the subsystem tree). For this reason it is not sufficient to test the -mm kernels alone and the development -rc or -git kernels should be tested as well.
On the other hand, there are kernel subsystems that do not have their own development trees and many patches are sent directly to Andrew Morton. These patches are first included in the -mm tree and then they go either to one of the subsystem trees or to the mainline. Thus the -mm tree is a place in which the patches coming from many different sources first meet together and are combined into an approximation of a future stable kernel that can be tested.
Since the -mm tree is a combination of many other trees, including the mainline, that always evolve, it really should be regarded as a set of patches that can be put together from time to time. When this happens, Andrew releases the so-called -mm patches to be applied on top of specific -rc or stable kernel trees. Each of them is also available as a series of individual patches combined in order to obtain it, including some fixes introduced by Andrew himself. This makes the identification of patches that contain bugs easier, as they can be found using the quilt tool with the help of the bisection technique (for more information about it see Chapter 4).
In the directory containing the -mm patch and the series of individual patches it consists of (at ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.X-rcY/2.6.X-rcY-mmZ/ or at ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.X/2.6.X-mmZ/, where X, Y, Z are numbers) there always is a directory called hot-fixes containing the patches that fix known bugs in this version of the -mm tree and should be applied on top of the corresponding -mm patch before testing it (the bugs that have already been identified need not be found once again).
The individual patches included in the -mm tree are sent to the mm-commits mailing list (http://vger.kernel.org/vger-lists.html#mm-commits). By following this list you can get an idea about the current contents of the -mm tree. It may also help you identify patches that introduce bugs.
From time to time Andrew sends to mm-commits a message with the subject like mm snapshot broken-out-date-time.tar.gz uploaded announcing that new -mm tree snapshot has been made available from the server. These snapshots are considered as less usable than the regular -mm patches and are to be used by people who want to help Andrew put the next -mm patch together.
1.6 Compilation and installation
1.6.1 Kernel compilation
First, your system should be prepared for the configuration and compilation of the kernel. You will need gcc, make, binutils, util-linux and the package containing the ncurses header files (libncurses5-dev in Debian or ncurses-devel in Fedora Core and OpenSUSE). If you want to use a graphical tool for configuring the kernel, you will also need the Qt library or GTK+ and the corresponding development packages. The file Documentation/Changes in the kernel sources directory contains the list of all programs that you may need.
Now, we assume that you have already downloaded the kernel source code and patched it to obtain the required version. To compile it, change the current directory to the one that contains the kernel sources and run the following sequence of commands:
make menuconfig (allows you to adjust the kernel configuration to suit you; there are many documents on the web describing the configuration of the kernel, one of them you can find at http://www.tlug.org.za/old/guides/lkcg/lkcg_config.html)
make (starts the compilation of the kernel; if you have more than one processor core in your machine, you may want to use the -j option of make, for example ”make -j5”)
make modules_install (installs the kernel modules in the /lib/modules/ directory, creating the subdirectory named after the version of the newly compiled kernel)
cp arch/i386/boot/bzImage /boot/vmlinuz-<kernel_version> (copy the kernel to the /boot directory; <kernel_version> should be the label reflecting the version of the kernel being installed)
cp System.map /boot/System.map-<kernel_version> (copy the map of kernel symbols to the /boot directory)
Instead of ”make menuconfig” you can use ”make config” that will cause the kernel build system to ask you a lot of questions regarding its configuration, which is inefficient and tedious, but can be done without ncurses.
If you have already configured the kernel, you can use the configuration file .config from the previous compilation. For this purpose copy it to the directory containing the current kernel sources and run ”make oldconfig” (it works like ”make config”, but does not ask so many questions). Alternatively, you can use the configuration of the distribution kernel (usually it contains many things you will never need and that is one of the reasons why you may want to learn how to configure the kernel):
$ zcat /proc/config.gz > .config $ make oldconfig
1.6.2 Useful make options
There are some useful options you can pass to make during the kernel compilation:
make O=/directory/ (saves the result of the compilation in given directory; this is particularly useful if you do not want to litter the kernel source directory with *.o files, but in that case you should also use ”make O=/directory/ menuconfig” instead of ”make menuconfig” and ”make O=/katalog/ modules_install” etc.)
make CC=<compiler> (allows you to specify the name of the compiler to use; for example, in Debian there are several versions of gcc, and if you want to use gcc-3.4 to compile the kernel it is sufficient to use ”make CC=gcc-3.4”)
make C=1 (before compilation the kernel sources are checked by the sparse tool)
make -j<number> (sets the number of parallel processes that will be run during the kernel compilation)
make xconfig (graphical configuration using Qt)
make gconfig (graphical configuration using GTK+)
1.6.3 Kernel mo dules
If you want to use kernel modules, in which case some parts of the kernel will be loaded into memory only if they are needed, select the options:
Loadable module support ---> [*] Enable loadable module support [*] Automatic kernel module loading
The modularization allows the kernel to be smaller and work faster.
To compile a piece of the kernel code as a module, you need to mark it with M during the configuration, eg. <M> Check for non-fatal errors on AMD Athlon/Duron / Intel Pentium 4 If the ”Automatic kernel module loading” option has been selected, the modules should be automatically loaded by the kernel when they are needed. That is, for example, if you want to mount a CD-ROM, the kernel should automatically load the module isofs.ko and this modules can be removed from memory after the CD-ROM has been unmounted. You can also load and unload kernel modules manually, using the commands modprobe, insmod or rmmod, eg.
modprobe isofs (loads the module isofs.ko)
modprobe -r isofs (removes isofs.ko from memory)
All of the modules available to the currently running kernel should be located in various subdirectories of /lib/modules/<kernel_version>/kernel .
1.6.4 Kernel hacking options
If you build the kernel to test it, you should seriously consider setting the options located in the ”Kernel hacking” menu. At least, you should set:
[*] Kernel debugging [*] Compile the kernel with debug info
The other options should also be set, if possible, because they help debug specific functionalities of the kernel, but some of them hurt performance, so you may want to leave them unset:
[*] Debug shared IRQ handlers [*] Detect Soft Lockups [*] Debug slab memory allocations [*] Slab memory leak debugging [*] RT Mutex debugging, deadlock detection [*] Built-in scriptable tester for rt-mutexes [*] Lock debugging: prove locking correctness [*] Lock dependency engine debugging [*] Locking API boot-time self-tests [*] Highmem debugging [*] Debug VM