Developing on a native Linux platform

These instructions assume you're running a variant of Ubuntu (like 18.04 LTS) or ArchLinux (like Manjaro). If you’re not familiar with using Linux, then probably you should check out the Linux Foundation’s course on Introduction to Linux on Edx.

Install important packages

To install all required packages on Ubuntu/Debian, run this command:

sudo apt-get install vim libncurses5-dev gcc make git exuberant-ctags libssl-dev bison flex libelf-dev bc dwarves zstd git-email

To install all required packages on ArchLinux/Manjaro, run this command:

sudo pacman -S vim gcc make git ctags openssl bison flex libelf bc cpio ncurses python3 perl

Many of the packages above might be or not be already installed in your system, depending on your distribution and on how minimalist is your installation. You’ll be extensively using the following utilities while working on the Linux kernel.

1. vim

Programmer's editor - Vim is a highly configurable text editor built to enable efficient text editing. It is an improved version of the vi editor distributed with most UNIX systems

2. gcc

The GNU Compiler Collection (GCC) is a compiler system produced by the GNU Project supporting various programming languages.GCC is the standard compiler for most Unix-like operating systems.

3. make

The make utility automatically determines which pieces of a large program need to be recompiled, and issues commands to recompile them. To prepare to use make, you must have a file called the makefile that describes the relationships among files in your program and provides commands for updating each file. Once a suitable makefile exists (which is the case for the Linux kernel), each time you change some source files, the simple shell command:

make

suffices to perform all necessary recompilations.

4. git

Git is a version-control system for tracking changes in computer files and coordinating work on those files among multiple people.

Apart from that, the following packages are necessary for the Linux kernel build process, but one does not have to interact with them directly.

5. ncurses

ncurses (new curses) is a programming library providing an application programming interface (API) that allows the programmer to write text-based user interfaces in a terminal-independent manner. It is a toolkit for developing "GUI-like" application software that runs under a terminal emulator.

6. libssl

This package is part of the OpenSSL project's implementation of the SSL and TLS cryptographic protocols for secure communication over the Internet. Module signing and external certificate handling use the OpenSSL program and crypto library to do key creation and signature generation.

7. bison

Bison is a general-purpose parser generator that converts an annotated context-free grammar into a deterministic LR or generalized LR (GLR) parser employing LALR(1) parser tables. Since Linux 4.16, the build system generates parsers during build. This requires bison 2.0 or later.

8. flex

Flex is a fast lexical analyser generator. It is a tool for generating programs that perform pattern-matching on text. Flex is a free (but non-GNU) implementation of the original Unix lex program. Since Linux 4.16, the build system generates lexical analyzers during build. This requires flex 2.5.35 or later.

9. libelf

The Executable and Linkable Format (ELF, formerly named Extensible Linking Format), is a common standard file format for executable files, object code, shared libraries, and core dumps. 'Libelf' lets you read, modify or create ELF files in an architecture-independent way. The library takes care of size and endian issues. The Linux kernel and kernel modules use ELF specifications. Hence you’ll need this dependency.

10. bc

bc is an arbitrary precision numeric processing language. It supports interactive execution of statements. bc is used during the kernel build to generate time constants in header files.

11. cpio

cpio is a general archiver (and also the term to refer to its generated archives). It's necessary for kernel building routines, when the script kernel/gen_kheaders.sh runs.

12. python3

python3 is a general-purpose interpreted programming language. It's used in the kernel for self-tests and kunit tools.

13. perl

perl is a general-purpose interpreted programming language and is used in several kernel scripts like checkpatch.pl. It's also used in git send-email. Since perl is modular, you might need specific modules to run some scripts. If you see an error about a missing perl module you can use the cpan tool to retrieve it. Some distributions might ship cpan as a different package.

Set up your Linux kernel code repository

Once that finishes, run these two commands:

mkdir -p git/kernels; cd git/kernels

Then use the revision control system called git to clone Greg Kroah-Hartman's staging tree repository:

git clone -b staging-testing git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git

That's going to take a while. Why don't you read up on Linux Device Drivers in the meantime? The first couple of chapters, especially the ones on kernel modules, will be useful. (Note that LDD3 is a bit outdated; the high-level overviews still make sense, but a few of the code examples are out of date. Still recommended to get a view of the kernel landscape, though.)

Next, change into the staging directory:

cd staging

Now you need to compile the kernel. The first step is setting up your kernel configuration.

Setting up your kernel configuration

Many kernel drivers can be turned on or off, or built as modules. The .config file in the kernel source directory determines which drivers are built. When you download the source tree, it doesn't come with a .config file. You have several options on generating a .config file. The easiest is to duplicate your current config.

1. Duplicating your current config

If you're trying to see if a bug is fixed, you probably want to duplicate the configuration on your running kernel. That config file is stored somewhere in /boot/. There might be several files that start with config, so you want the one associated with your running kernel. You can find that by running uname -a and finding the config file that ends with your kernel version number. Copy that file into the source directory as .config. Or for Ubuntu just run this command:

cp /boot/config-`uname -r`* .config

For ArchLinux/Manjaro,

zcat /proc/config.gz > .config

Do not forget to rename your kernel version "CONFIG_LOCALVERSION" in the new .config. If you skip this, there is the risk of overwriting one of your existing kernels by mistake.

2. Changing your config

If you need to make any changes to your configuration, run this command to silently update any new configuration values to their default:

make olddefconfig

If you need to make changes, you can run:

make menuconfig

This requires the ncurses library to be installed.

If you want to compile faster, you can remove from the configuration all the modules you are not using at the moment in your system with:

make localmodconfig

Warning: this might not compile some required modules to your system when building the kernel.

Building the kernel

Run

make

Or, if you have a multi-core processor, run

make -jX

Where X is a number like 2 or 4. If you have a dual core, 2 or 3 might be good. Quad core, 4 or 6. Do not run with really big numbers unless you want your machine to be dog-slow!

Let that compile, and maybe read some more of the Linux Device Drivers book.

Note: when you run make with a .config file that was copied from a different kernel than the one you were building, you may be prompted to make choices about which new kernel features to enable. When in doubt, just choose the default choice. You can do that by hitting enter. A faster way to simply chose all the defaults is to run make menuconfig. This will set default values for unset features. Don't forget to save when you exit.

Installing the kernel

To install a kernel, you will need to either manually update your GRUB configuration file, or have an installkernel script. This script installs the kernel to /boot/, installs modules to /lib/modules/X.Y.Z/ (where X.Y.Z is something like 3.1.5), and updates file /boot/grub/grub.conf. Fortunately, Ubuntu provides an installkernel script in /sbin/installkernel. The grubby RPM provides it for RPM based systems.

If you have an installkernel script, you can just run

sudo make modules_install install

Or if you don't have sudo installed, run

su -c "make modules_install install"

For ArchLinux/Manjaro you’ll need to do a few more things as mentioned here. Make sure that after completion of all the steps you have

1. Kernel: vmlinuz-YourKernelName

2. Initramfs: Initramfs-YourKernelName.img

3. System Map: System.map-YourKernelName

4. System Map kernel symlink

Running your kernel

Linux stores both old and new kernel versions, so you can boot into your old kernel if you run into issues with your new kernel. The bootloader program called grub lets you chose which kernel you want to boot into.

The grub bootloader usually presents the user with a choice of kernels and you can reboot into a known good kernel if your newly compiled kernel doesn't work. Some distros (like Ubuntu) use a default grub configuration that hides that menu. You can usually get the menu to appear by mashing the ESC key during boot after the BIOS display disappears.

To make the grub boot menu always appear under Ubuntu, run

sudo vim /etc/default/grub

(If you have the following lines in your grub file, you'll need to delete them)

GRUB_HIDDEN_TIMEOUT=0

GRUB_HIDDEN_TIMEOUT_QUIET=true

This next line controls how long grub shows the menu before it chooses the kernel at the top of the list (which is usually the most recent kernel):

GRUB_TIMEOUT=10

You may want to increase the GRUB_DEFAULT timeout from 10 seconds to 30 seconds or more. I set it to 60 seconds.

Also, check that

GRUB_TIMEOUT_STYLE=menu

After you've finished editing the grub file, you need to tell grub that you made a change, so that can re-write some automatically generated files. Run this command:

sudo update-grub2

You will (usually) need to reboot into your new kernel.

Email software

The following instructions assume that you are running a variant of Ubuntu (like 18.04 LTS). If you use any other distribution, perform the steps with your distro specific package manager (e.g. For ArchLinux/Manjaro, replace apt-get install packagename with pacman -S packagename )

To send patches, you will need to be able to have a mail transport client installed:

sudo apt-get install esmtp

Now you will be able to send email through git-send-email. However, we also recommend having a mail client installed that can handle plain text patch format. Our recommendation is a text-based email client called mutt:

sudo apt-get install mutt

You may want to send patches with the git send-email command, so if you use Ubuntu you'll need to install the git-email package:

sudo apt-get install git-email

For Arch users git send-email is shipped with the git package, but the command might fail if you don't have the perl modules MIME::Base64 and Authen::SASL installed in your system. To install the modules you need to run:

cpan MIME::Base64
cpan Authen::SASL

PS: you can use sudo or doas if you want the modules to be available for all users in your system.

Optional tools

gitk is a graphical git repository browser. You can use it to look at the state of your repository and the commits you've made on top of upstream. On ArchLinux it's installed with the package git, so you are ready to use. Ubuntu users might need to install it:

sudo apt-get install gitk

To use, just type gitk from within your kernel source directory.

Make ctags

ctags parses source code and produces a sort of index mapping the names of significant entities (e.g. functions, classes, variables) to the location where that entity is defined. This index is used by editors like vi and emacsen to allow moving to the definition of a user-specified entity. Exuberant Ctags supports all possible C language constructions and multiple other languages.

You’ve already installed it along with other packages in the first step.

In the git/kernels/staging/ directory, run

make tags

That will make ctags for the source code. When you start editing code in vim from the base directory (git/kernels/staging/), you can use the CTRL+] to look up the definition of a function. See the ctags entry in the vim tips wiki for more info.

Done!

Yay, you're done setting up your kernel development environment! Now go start at these directions to get more instructions for how to modify the Linux kernel and send your first patch.