|
Size: 5884
Comment:
|
← Revision 55 as of 2021-01-14 22:12:27 ⇥
Size: 6682
Comment: add URL for kdb info
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| 1. Version of "buged" kernel. | 1. A buggy kernel. |
| Line 5: | Line 5: |
| Line 9: | Line 10: |
| Line 10: | Line 12: |
| printk is a very useful function similar to printf(). This function works everywhere and at any time (apart from early stage of booting the kernel when video isn't initialized). It uses log levels to tell the console the importance of each message. Full list of levels: |
printk is a very useful function similar to printf(). This function works everywhere and at any time (apart from early stage of booting the kernel when video isn't initialized). It uses log levels to tell the console the importance of each message. Full list of levels: |
| Line 22: | Line 22: |
| Line 28: | Line 29: |
| From: ''Linux/arch/mips/mm/tlbex.c'' | . printk('''loglevel''' '''"messages"'''); |
| Line 30: | Line 31: |
| {{{ 185 if (arg & ~RS_MASK) 186 printk(KERN_WARNING "TLB synthesizer field overflow\n");}}} |
|
| Line 36: | Line 34: |
| 37 if (!(errst0 & PI_ERR_ST0_VALID_MASK)) { 38 printk(); 39 return; 40 } }}} |
20 #if 1 321 printk("FIXME: disabling master aborts\n"); 322 csrs->POx_MSK_HEI.csr &= ~(3UL << 14); 323 #endif }}} |
| Line 42: | Line 41: |
http://www.urbanmyth.org/linux/oops/slides.html <-- useful link |
|
| Line 56: | Line 57: |
| == Causing errors and printing extra informations. == 1. Sometimes you will want to see oops informations about some bug. Use BUG() BUG_ON(): if(bad_thing) |
== Causing errors and printing extra information. == 1. Sometimes you will want to see oops information about some bug. Use BUG() BUG_ON(): if (bad_thing) |
| Line 61: | Line 63: |
| Line 73: | Line 76: |
| Line 75: | Line 79: |
| 1. Sometimes you will want to see oops informations and then stop system. Use panic(): if(terrible_error) | 1. Sometimes you will want to see oops information and then stop system. Use panic():<<BR>> if (terrible_error) |
| Line 77: | Line 83: |
| Line 78: | Line 85: |
| if(debug_check) | if (debug_check) . dump_stack(); |
| Line 80: | Line 88: |
| . dump_stack(); | |
| Line 87: | Line 94: |
| 41 panic();}}} | 41 panic("request_dma error!"); }}} |
| Line 92: | Line 100: |
| 399 printk(KERN_WARNING 400 , sht->name); |
399 printk(KERN_WARNING "scsi_register() called on new-style " 400 "template for driver %s\n", sht->name); |
| Line 95: | Line 103: |
| 402 }}}} | 402 } }}} |
| Line 100: | Line 109: |
| 1. SysRq+e Send '''SIGTERM''' to all tasks ('''with out init !!!''') | 1. Sysrq+c system crash + crashdump 1. Sysrq+d dump all locks that are held 1. SysRq+e Send SIGTERM''' to all tasks ('''except init !!!''') ''' 1. SysRq+f call OOM killer to kill a memory hog, but do not panic 1. SysRq+g used by kgdb (kernel debugger) |
| Line 102: | Line 115: |
| 1. SysRq+i Send '''SIGKILL''' to all tasks ('''with out init !!!''') | 1. SysRq+i Send SIGKILL''' to all tasks ('''except init !!!''') ''' 1. SysRq+j Forcibly Just thaw filesystems that are frozen by the FIFREEZE ioctl |
| Line 104: | Line 118: |
| 1. SysRq+l Send '''SIGTKILL '''to all tasks ('''with init !!!''') 1. SysRq+m Dump core and show it on console |
1. SysRq+l Show stack backtrace for all active CPUs 1. SysRq+m Dump current memory info on console 1. SysRq+n Used to make RT tasks nice-able |
| Line 108: | Line 123: |
| 1. SysRq+q Dump per-CPU lists of all armed hrtimers and info about all clockvent devices | |
| Line 109: | Line 125: |
| 1. SysRq+s Save dirty buffers on HDD | 1. SysRq+s Attempt to sync all mounted filesystems |
| Line 112: | Line 128: |
| 1. SysRq+v Forcefully restore framebuffer console Do ETM buffer dump [ARM-specific] 1. SysRq+w Dump tasks that are in uninterruptible (blocked) state 1. SysRq+x Used by xmon interface [PPC, PowerPC] Show global PMU registers [Sparc64]<<BR>> Dump all TLB entries [MIPS] 1. SysRq+y Show global CPU registers [Sparc64] 1. SysRq+z Dump the ftrace buffer 1. SysRq+N N = '0' - '9': sets the console log level ('0' => emergency only; '9' => everything) |
|
| Line 114: | Line 140: |
| == How to use non-exist debugger? == Linus doesn't want a debugger in the kernel because it can lead to fixing symptoms instead of actually understanding the code and fixing the real problem. |
== How to use debuggers? == Before I start talking about debuggers you must know one thing. Linus discourages the use of debuggers, because debuggers don't always tell the truth. |
| Line 117: | Line 143: |
| However, there are some non-official debuggers (Some require patching the kernel). | * gdb . gdb vmlinux /proc/kcore <--> vmlinux is an uncompresed kernel image (find it in the top level kernel source directory) /proc/kcore allows gdb to see Linux's memory |
| Line 119: | Line 146: |
| Note: use''' Additional compiling options '''to make gdb more functional.''' ''' | https://sourceware.org/gdb/documentation/ <-- documentation for gdb |
| Line 121: | Line 148: |
| 1. '''gdb (it is not a patch it is GNU debugger) ''' + simple to use |
. + simple to use . - you can not change data in running kernel * kgdb . kgdb is a kernel tool that allows you to connect two computers, one with kgdb and second with gdb . + you can modify data and variables . - you must configure connection. |
| Line 124: | Line 151: |
| * gdb vmlinux /proc/kcore call gdb with vmlinux(not compressed image of kernel, in the top level kernel source directory) and optional /proc/kcore - memory of kernel(you must be root) * p varaible - show what variable contain * disassemble name_of_function - disassemble function * etc. . - you can not change variables or other data * '''kgdb (connect 2 computers, one with kdb and second with gdb) ''' . +you can change data +all functions of basic gdb are implemented -you must configure connection |
http://kgdb.wiki.kernel.org/ |
| Line 133: | Line 153: |
| 1. '''kdb ''' +only one computer needed''' +'''all functions of gdb implemented +simple to use |
* kdb |
| Line 136: | Line 155: |
| * type "break" in console * starts on '''oops''' == When everything fail. == |
https://www.kernel.org/doc/html/latest/dev-tools/kgdb.html == When all else fails. == |
What should I have to fix a bug?
- Bug. Well known particular bug.
- A buggy kernel.
- Bit of luck.
Note:
- Having a repeatable bug is more then 50% of success.
- All examples are from 2.6.17.13 (i386)
Function printk().
printk is a very useful function similar to printf(). This function works everywhere and at any time (apart from early stage of booting the kernel when video isn't initialized). It uses log levels to tell the console the importance of each message. Full list of levels:
KERN_EMERG <-- the most important
- KERN_ALERT
- KERN_CRIT
- KERN_ERR
- KERN_WARNING
- KERN_NOTICE
- KERN_INFO
KERN_DEBUG <-- the least important
The console will print messages only with a level higher than console_loglevel. By default printk uses DEFAULT_MESSAGE_LOGLEVEL == KERN_WARNING (but this may be changed in the future).
printk() uses a cyclic buffer to manage the messages. Next klogd reads the messages (using /proc/kmsg) from the buffer and gives them to syslogd which writes them to /var/log/messages. (You can configure syslogd by editing /etc/syslog.conf).
Examples:
printk(loglevel "messages");
From: Linux/arch/mips/sgi-ip27/ip27-berr.c
20 #if 1
321 printk("FIXME: disabling master aborts\n");
322 csrs->POx_MSK_HEI.csr &= ~(3UL << 14);
323 #endif
Error oops.
An oops is report of a bug in the kernel. When an oops occurs the kernel will print what the registers contain and a "back trace". An oops does not mean the system has crashed, as the system can sometimes recover from the error. If the system can not recover from the error then the kernel will panic and stop running. By default the back trace will contain the addresses of the functions that were called. If you compile your kernel with CONFIG_KALLSYMS=y the oops will be decoded and will display the function names. In the 2.4 kernel you can use ksymoops file_with_oops.txt to see the names of the functions.
http://www.urbanmyth.org/linux/oops/slides.html <-- useful link
Additional compiling options.
These options are very useful when debugging kernel:
CONFIG_PREEMPT=y
CONFIG_DEBUG_KERNEL=y
CONFIG_KALLSYMS=y
CONFIG_SPINLOCK_SLEEP=y
CONFIG_MAGIC_SYSRQ=y
Causing errors and printing extra information.
- Sometimes you will want to see oops information about some bug. Use BUG() BUG_ON():
if (bad_thing)
- BUG();
or BUG_ON(bad_thing);
Examples:
From: Linux/arch/arm/plat-omap/dma.c
732 if (omap_dma_in_1510_mode()) {
733 printk(KERN_ERR "DMA linking is not supported in 1510 mode\n");
734 BUG();
735 return;
736 } 1221 BUG_ON(lcd_dma.active);
Sometimes you will want to see oops information and then stop system. Use panic():
if (terrible_error)- panic("var = %ld \n", var);
- Sometimes you will want to see stack. Use dump_stack(): if (debug_check)
- dump_stack();
Examples:
From: Linux/arch/cris/arch-v32/kernel/dma.c
40 if (options & DMA_PANIC_ON_ERROR)
41 panic("request_dma error!"); From: Linux/drivers/scsi/hosts.c
398 if (!sht->detect) {
399 printk(KERN_WARNING "scsi_register() called on new-style "
400 "template for driver %s\n", sht->name);
401 dump_stack();
402 }
Magic SysRq Key.
If you set CONFIG_MAGIC_SYSRQ=y or typed 'echo 1 > /proc/sys/kernel/sysrq', you can use SysRq Key (on PPC or i386) 'Alt+PrintScreen'.
SysRq+b Restart computer
- Sysrq+c system crash + crashdump
- Sysrq+d dump all locks that are held
SysRq+e Send SIGTERM to all tasks (except init !!!)
SysRq+f call OOM killer to kill a memory hog, but do not panic
SysRq+g used by kgdb (kernel debugger)
SysRq+h Help
SysRq+i Send SIGKILL to all tasks (except init !!!)
SysRq+j Forcibly Just thaw filesystems that are frozen by the FIFREEZE ioctl
SysRq+k Kill all tasks ran from this console
SysRq+l Show stack backtrace for all active CPUs
SysRq+m Dump current memory info on console
SysRq+n Used to make RT tasks nice-able
SysRq+o Halt system and shutdown it
SysRq+p Print CPU registers on console
SysRq+q Dump per-CPU lists of all armed hrtimers and info about all clockvent devices
SysRq+r Change keyboard from RAW to XLATE
SysRq+s Attempt to sync all mounted filesystems
SysRq+t Show current task info on console
SysRq+u Unmount all filesystems, and remount read only
SysRq+v Forcefully restore framebuffer console
- Do ETM buffer dump [ARM-specific]
SysRq+w Dump tasks that are in uninterruptible (blocked) state
SysRq+x Used by xmon interface [PPC, PowerPC]
Show global PMU registers [Sparc64]
Dump all TLB entries [MIPS]
SysRq+y Show global CPU registers [Sparc64]
SysRq+z Dump the ftrace buffer
SysRq+N N = '0' - '9': sets the console log level ('0' => emergency only; '9' => everything)
Note that every user can use SysRq keys, and it can work improperly on an unstable system.
How to use debuggers?
Before I start talking about debuggers you must know one thing. Linus discourages the use of debuggers, because debuggers don't always tell the truth.
- gdb
gdb vmlinux /proc/kcore <--> vmlinux is an uncompresed kernel image (find it in the top level kernel source directory) /proc/kcore allows gdb to see Linux's memory
https://sourceware.org/gdb/documentation/ <-- documentation for gdb
- + simple to use . - you can not change data in running kernel
- kgdb . kgdb is a kernel tool that allows you to connect two computers, one with kgdb and second with gdb . + you can modify data and variables . - you must configure connection.
- kdb
https://www.kernel.org/doc/html/latest/dev-tools/kgdb.html
When all else fails.
No one likes bugs. So when you spend hours/days on bug fixing, you may write a short and descriptive email containing your all of the information you have found, and send it to LKML. Good luck with Bug Hunting.