== Testing ==
Generally, there are many ways in which you can test the Linux kernel, but we will concentrate on the following four approaches:
   1. Using a test version of the kernel for normal work.
   2. Running special test suites, like LTP, on the new kernel.
   3. Doing unusual things with the new kernel installed.
   4. Measuring the system performance with the new kernel installed.
Of course, all of them can be used within one combined test procedure, so they can be regarded as diﬀerent phases of the testing process.

=== 2.1      Phase One ===
The ﬁrst phase of kernel testing is simple: we try to boot the kernel and use it for normal work.
 * Before starting the system in a fully functional conﬁguration it is recommended to boot the kernel with the init=/bin/bash command line argument, which makes it start only one bash process. From there you can check if the ﬁlesystems are mounted and unmounted properly and you can test some more complex kernel functions, like the suspend to disk or to RAM, in the minimal conﬁguration. In that case the only kernel modules loaded are the ones present in the initrd image mentioned in Subsection 1.6.6. Generally, you should refer to the documentation of your boot loader for more information about manual passing command line arguments to the kernel (in our opinion it is easier if GRUB is used).
 * Next, it is advisable to start the system in the runlevel 2 (usually, by passing the number 2 to the kernel as the last command line argument), in which case network servers and the X server are not started (your system may be conﬁgured to use another runlevel for this purpose, although this is very unusual, so you should look into {{{/etc/inittab}}} for conﬁdence). In this conﬁguration you can check if the network interfaces work and you can try to run the X server manually to make sure that it does not crash.
 * Finally, you can boot the system into the runlevel 5 (ie. fully functional) or 3 (ie. fully functional withot X), depending on your needs.

Now, you are ready to use the system in a normal way for some time. Still, if you want to test the kernel quickly, you can carry out some typical operations, like downloading some ﬁles, reading email, browsing some web pages, ripping some audio tracks (from a legally bought audio CD, we presume), burning a CD or DVD etc., in a row to check if any of them fail in a way that would indicate a kernel problem.

=== 2.2       Phase Two (AutoTest) ===
In the next phase of testing we use special programs designed for checking if speciﬁc kernel subsystems work correctly. We also carry out regression and performance tests of the kernel. The latter are particularly important for kernel developers (and for us), since they allow us to identify changes that hurt performance. For example, if the performance of one of our ﬁlesystems is 10% worse after we have upgraded the 2.6.x-rc1 kernel to the 2.6.x-rc2 one, it is deﬁnitely a good idea to ﬁnd the patch that causes this to happen.

For automated kernel testing we recommend you to use the AutoTest suite (http://test.kernel.org/autotest/) consisting of many test applications and proﬁling tools combined with a fairly simple user interface.

To install AutoTest you can go into the {{{/usr/local}}} directory (as root) and run

{{{# svn checkout svn://test.kernel.org/autotest/trunk autotest}}}

Although it normally is not recommended to run such commands as root, this particular one should be safe, unless you cannot trust your DNS server, because it only downloads some ﬁles and saves them in {{{/usr/local}}} . Besides, you will need to run AutoTest as root, since some of its tests require superuser privileges to complete. For this reason you should not use AutoTest on a production system: in extreme cases the data stored in the system the privileged tests are run on can be damaged or even destroyed, and we believe that you would not like this to happen to your production data.

By design, AutoTest is noninteractive, so once started, it will not require your attention (of course, if something goes really wrong, you will have to recover the system, but this is a different kettle of fish). To start it you can go to {{{/usr/local/autotest/client}}} (we assume that AutoTest has been installed in {{{/usr/local}}}) and execute (as root)

{{{# bin/autotest tests/test_name/control}}}

where test_name is the name of the directory in {{{/usr/local/autotest/client/tests}}} that contains the test you want to run. The control ﬁle tests/test_name/control contains instructions for AutoTest. In the simplest cases only one such instruction is needed, namely

{{{job.run_test(’test_name’)}}}

where test_name is the name of the directory that contains the control ﬁle. The contents of more sophisticated control ﬁles can look like this:

{{{
job.run_test(’pktgen’, ’eth0’, 50000, 0, tag=’clone_skb_off’)
job.run_test(’pktgen’, ’eth0’, 50000, 1, tag=’clone_skb_on’)
}}}

where the strings after the test name represent arguments that should be passed to the test application. You can modify these arguments, but ﬁrst you should read the documentation of the test application as well as the script {{{tests/test_name/test_name.py}}} (eg. {{{tests/pktgen/pktgen.py}}}) used by AutoTest to actually run the test (as you have probably noticed, the AutoTest scripts are written in Python). The results of the execution of the script {{{tests/test_name/test_name.py}}} are saved in the directory {{{results/default/test_name/}}}, where the status ﬁle contains the information indicating whether or not the test has been completed successfully. To cancel the test, press Ctrl+C while it is being executed.

If you want to run several tests in a row, it is best to prepare a single ﬁle containing multiple instructions for AutoTest. The instructions in this ﬁle should be similar to the ones contained in the above-mentioned control files. For example, the file {{{samples/all_tests}}} contains instructions for running all of the available tests and its ﬁrst ﬁve lines are the following:

{{{
job.run_test(’aiostress’)
job.run_test(’bonnie’)
job.run_test(’dbench’)
job.run_test(’fio’)
job.run_test(’fsx’)
}}}

To run all of the tests requested by the instructions in this ﬁle, you can use the command {{{bin/autotest samples/all_tests}}}
but you should remember that it will take a lot of time to complete. Analogously, to run a custom selection of tests, put the instructions for AutoTest into one ﬁle and provide its name as a command line argument to autotest.

To run several tests in parallel, you will need to prepare a special control ﬁle containing instructions like these:
{{{
def kernbench():
          job.run_test(’kernbench’, 2, 5)
def dbench():
          job.run_test(’dbench’)
job.parallel([kernbench], [dbench])
}}}

While the tests are being executed, you can stop them by pressing Ctrl+C at any time.

For people who do not like the command line and conﬁguration ﬁles, ATCC (AutoTest Control Center ) has been created. If you run it, for example by using the command {{{ui/menu}}}, you will be provided with a simple menu-driven user interface allowing you to select tests and proﬁling tools, view the results of their execution and, to a limited extent, conﬁgure them.

If you are bored with the selection of tools available in the AutoTest package, you can visit the web page http://ltp.sourceforge.net/tooltable.php containing a comprehensive list of tools that can be used for testing the Linux kernel.