Diff for "KernelHacking-HOWTO/Debugging_Kernel"

Differences between revisions 14 and 22 (spanning 8 versions)

What should I have to fix a bug?

Bug. Well known particular bug.
A buggy kernel.
Bit of luck.

Note:

Having a repeatable bug is more then 50% of success.
All examples are from 2.6.17.13 (i386)

Function printk().

printk is a very useful function similar to printf(). This function works everywhere and at any time (apart from early stage of booting the kernel when video isn't initialized). It uses log levels to tell the console the importance of each message.

Full list of levels:

KERN_EMERG <-- the most important
KERN_ALERT
KERN_CRIT
KERN_ERR
KERN_WARNING
KERN_NOTICE
KERN_INFO
KERN_DEBUG <-- the least important

The console will print messages only with a level higher than console_loglevel. By default printk uses DEFAULT_MESSAGE_LOGLEVEL == KERN_WARNING (but this may be changed in the future).

printk() uses a cyclic buffer to manage the messages. Next klogd reads the messages (using /proc/kmsg) from the buffer and gives them to syslogd which writes them to /var/log/messages. (You can configure syslogd by editing /etc/syslog.conf).

Examples:

printk(loglevel "mesages");

From: Linux/arch/mips/mm/tlbex.c

185         if (arg & ~RS_MASK)
186                 printk(KERN_WARNING "TLB synthesizer field overflow\n");

From: Linux/arch/mips/sgi-ip27/ip27-berr.c
37 if (!(errst0 & PI_ERR_ST0_VALID_MASK)) { 38 printk(); 39 return; 40 }

Error oops.

An oops is report of a bug in the kernel. When an oops occurs the kernel will print what the registers contain and a "back trace". An oops does not mean the system has crashed, as the system can sometimes recover from the error. If the system can not recover from the error then the kernel will panic and stop running. By default the back trace will contain the addresses of the functions that were called. If you compile your kernel with CONFIG_KALLSYMS=y the oops will be decoded and will display the function names. In the 2.4 kernel you can use ksymoops file_with_oops.txt to see the names of the functions.
http://www.urbanmyth.org/linux/oops/slides.html <-- useful link

Additional compiling options.

These options are very useful when debugging kernel:
CONFIG_PREEMPT=y
CONFIG_DEBUG_KERNEL=y
CONFIG_KALLSYMS=y
CONFIG_SPINLOCK_SLEEP=y
CONFIG_MAGIC_SYSRQ=y

Causing errors and printing extra informations.

Sometimes you will want to see oops informations about some bug. Use BUG() BUG_ON():
if(bad_thing)
BUG();
or BUG_ON(bad_thing);
Examples:
From: Linux/arch/arm/plat-omap/dma.c
732 if (omap_dma_in_1510_mode()) { 733 printk(KERN_ERR "DMA linking is not supported in 1510 mode\n"); 734 BUG(); 735 return; 736 }
1221 BUG_ON(lcd_dma.active);
Sometimes you will want to see oops informations and then stop system. Use panic(): if(terrible_error)
panic("var = %ld \n", var);
Sometimes you will want to see stack. Use dump_stack():
if(debug_check)
dump_stack();
Examples:
From: Linux/arch/cris/arch-v32/kernel/dma.c
40 if (options & DMA_PANIC_ON_ERROR) 41 panic("request_dma error!");
From: Linux/drivers/scsi/hosts.c
398 if (!sht->detect) { 399 printk(KERN_WARNING "scsi_register() called on new-style " 400 "template for driver %s\n", sht->name); 401 dump_stack(); 402 }

Magic SysRq Key.

If you set CONFIG_MAGIC_SYSRQ=y or typed 'echo 1 > /proc/sys/kernel/sysrq', you can use SysRq Key (on PPC or i386) 'Alt+PrintScreen'.

SysRq+b Restart computer

SysRq+e Send SIGTERM to all tasks (with out init !!!)
SysRq+h Help
SysRq+i Send SIGKILL to all tasks (with out init !!!)
SysRq+k Kill all tasks ran from this console
SysRq+l Send SIGTKILL to all tasks (with init !!!)
SysRq+m Dump core and show it on console
SysRq+o Halt system and shutdown it
SysRq+p Print CPU registers on console
SysRq+r Change keyboard from RAW to XLATE
SysRq+s Save dirty buffers on HDD
SysRq+t Show current task info on console
SysRq+u Unmount all filesystems, and remount read only

Note that every user can use SysRq keys, and it can work improperly on an unstable system.

How to use non-exist debugger?

Linus doesn't want a debugger in the kernel because it can lead to fixing symptoms instead of actually understanding the code and fixing the real problem.
However, there are some non-official debuggers (Some require patching the kernel).

Note: use Additional compiling options to make gdb more functional.

gdb (it is not a patch it is GNU debugger)

+ simple to use

gdb vmlinux /proc/kcore call gdb with vmlinux(not compressed image of kernel, in the top level kernel source directory) and optional /proc/kcore - memory of kernel(you must be root)
p varaible - show what variable contain
disassemble name_of_function - disassemble function
etc.
- you can not change variables or other data
kgdb (connect 2 computers, one with kdb and second with gdb)
+you can change data +all functions of basic gdb are implemented

-you must configure connection

+only one computer needed +all functions of gdb implemented +simple to use

type "break" in console
starts on oops

When everything fail.

No one likes bugs. So when you spend hours/days on bug fixing, you may write a short and descriptive email containing your all of the information you have found, and send it to LKML. Good luck with Bug Hunting.

-  ⇤ ← Revision 14 as of 2007-01-21 12:05:42 → 
  Size: 5803
  Editor: aczi10
  Comment:
+   ← Revision 22 as of 2007-01-22 19:54:36 → ⇥
  Size: 6081
  Editor: aczd45
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-. Version of "buged" kernel.
+. A buggy kernel.
 Line 28:
+printk(l'''oglevel''' '''"mesages"''');

''''
-Line 32:
+Line 36:
-printk(KERN_WARNING );}}}
+printk(KERN_WARNING "TLB synthesizer field overflow\n");}}}
-Line 42:
+Line 46:
+http://www.urbanmyth.org/linux/oops/slides.html    <-- useful link
-Line 69:
+Line 75:
-printk(KERN_ERR );
+printk(KERN_ERR "DMA linking is not supported in 1510 mode\n");
-Line 72:
+Line 78:
-}}}}
+} }}}
-Line 74:
+Line 80:
-BUG_ON(lcd_dma.active);}}}
+BUG_ON(lcd_dma.active); }}}
-Line 87:
+Line 93:
-panic();}}}
+panic("request_dma error!"); }}}
-Line 92:
+Line 98:
-printk(KERN_WARNING
400                                     , sht->name);
+printk(KERN_WARNING "scsi_register() called on new-style "
400                                     "template for driver %s\n", sht->name);
-Line 95:
+Line 101:
-}}}}
+} }}}