KernelNewbies:

Linux 2.6.33 has been released on February 24th, 2010.

Summary: This version features Nouveau (a reverse-engineered driver for Nvidia graphic cards), Nintendo Wii and Gamecube support, [http://www.drbd.org/ DRBD] (Distributed Replicated Block Device), a security extension for TCP called "cookie transactions", a syscall for batching recvmsg() calls, several new perf subcommands (perf probe, perf bench, perf kmem, perf diff), support for cache compression, Xen PV-on-HVM support, drivers for virtual network and graphic cards from VMWare, swappable KSM pages, and many new drivers and many small improvements and bugfixes

TableOfContents()

1. Prominent features (the cool stuff)

1.1. Nouveau, a driver for Nvidia graphic cards

This version includes Nouveau, a driver for Nvidia graphic cards, the one major GPU vendor without opensource drivers in the Linux kernel. Being developed since 2006, it has 26,000 LoC (not counting the Mesa stuff and the rest of the DRM stack). Nvidia has not contributed to this driver, it has been reverse-engineered. Graphic cards are one of the most complex pieces of hardware that you can find in modern computers, it's difficult to write drivers for them even having all the docs. So the developers of Nouveau deserve a big applause.

Nouveau is important because opensource is the one way to get good long term support for your graphic card. The new and powerful graphic card you've bought today will be unsupported in a few years. This doesn't happen with open source drivers, Nouveau (and ATI open source drivers) support today more devices than the official propietary drivers: for example, Riva TNT and Geforce 2/4MX/4Ti/FX.

The feature set, however, is not comparable, but Nouveau already [http://nouveau.freedesktop.org/wiki/FeatureMatrix supports a decent set of features]: modesetting (KMS), suspend/resume, Dual Head (RandR 1.2), and 2D operations (EXA, Xrender, Xv video). 3D functionality is not fully supported, but [http://www.happyassassin.net/2010/02/12/from-the-freaking-awesome-department-3d-support-on-nouveau/ it's improving]. And, of course, it's not stable, which is why it's only being merged in the staging directory.

The ctxprogs/ctx_voodoo" firmware [http://www.phoronix.com/scan.php?page=news_item&px=ODAxMA will not be needed in the future], because it can be autogenerated. Only a few cards autogenerate it today, but the dependency will be removed in the future.

Code: [http://git.kernel.org/linus/6ee738610f41b59733f63718f0bdbcba7d3a3f12 (commit)], [http://git.kernel.org/linus/054b93e444550a72aef17115363cdef253b9ee7c (commit)]

1.2. DRBD (Distributed Replicated Block Device)

Recommended LWN article: [http://lwn.net/Articles/329543/ DRBD: a distributed block device]

Web site (includes extensive documentation): [http://www.drbd.org/ http://www.drbd.org/]

DRBD ("Distributed Replicated Block Device") is a shared-nothing, synchronously replicated block device, developed by [http://www.linbit.com/ LINBIT]. It is designed to serve as a building block for high availability (HA) clusters. DRBD can be understood as network based raid-1.

For automatic failover you need a cluster manager (e.g. heartbeat). See also: http://www.drbd.org/, http://www.linux-ha.org

Code: [http://git.kernel.org/linus/b411b3637fa71fce9cf2acf0639009500f5892fe (commit)]

1.3. Perf improvements: perf probe, perk kmem, perf bench, perf diff, perf perl scripts and filters

Recommende LWN article: [http://lwn.net/Articles/343766/ Dynamic probes with ftrace]

This release adds a lot of improvements to the tracing infrastructure and the perf tool. (tools/perf)

perf probe: perf probe is a subcommand that allows to create kprobe events. Kprobe is a system that allows to break into any kernel routine at runtime and collect debugging and performance information non-disruptively. It's the system used by Systemtap to do kernel instrumentation. Perf probe allows to define kprobe events using C expressions (C line numbers, C function names, and C local variables). For example:

Step 1: Add a new kprobe probe on a line of C source code: "perf probe -P 'p:myprobe @fs/read_write.c:285 file buf count'" (it creates a new probe, called "myprobe", which will inspect the variables file, buf and count). Alternatively, you could also run simpler commands like "perf probe sys_open" to add a probe for the sys_open symbol (open() syscall)

Step 2: Add a new kretprobe probe on a function return "perf probe -P 'r:myretprobe vfs_read $rv'"

Step 3: If you run "perf list", you will see a event section named "kprobes" which contains the probes you just set up.

Step 4: Record the event: "perf record -f -e kprobes:myprobe:record -F 1 -a ls" and trace it "perf trace"

Code: [http://git.kernel.org/linus/eb13296cfaf6c699566473669a96a38a90562384 (commit 1], [http://git.kernel.org/linus/413d37d1eb69c1765b9ace0a612dac9b6c990e66 2], [http://git.kernel.org/linus/d8ec91850efaf6cee9234c80260fe03881242374 3], [http://git.kernel.org/linus/cd7e7bd5e44718c7625ce1e1f0fda53d77cd3797 4], [http://git.kernel.org/linus/e08d1c657f70bcaca11401cd6ac5c8fe59bd2bb7 5], [http://git.kernel.org/linus/eca0d916f6429785bbc88db3ff66631cde62b432 6], [http://git.kernel.org/linus/f52487e9c0041842eeb77c6c48774414b1cede08 7], [http://git.kernel.org/linus/4ea42b181434bfc6a0a18d32214130a242d489bf 8], [http://git.kernel.org/linus/595c36490deb49381dc51231a3d5e6b66786ed27 9], [http://git.kernel.org/linus/4de189fe6e5ad8241f6f8709d2e2ba4c3aeae33a 10], [http://git.kernel.org/linus/a7c312bed772c11138409c3a98531e85d690302e 11], [http://git.kernel.org/linus/fa28244d12337eebcc620b23852ec3cf29582ff9 12)]

perf bench: perf bench is a small suite of microbenchmarks. In this release, there're only three benchmarks: perf bench sched messaging (for benchmarking scheduler and IPC), perf bench sched pipe (benchmarks pipe()) and perf bench mem memcpy (measures memory bandwith). The command perf bench all will run all benchmarks.

Code: [http://git.kernel.org/linus/c426bba069e65ea438880a04aa4e7c5b880e1728 (commit 1], [http://git.kernel.org/linus/e27454cc6352c4226ddc76f5e3a5dedd7dff456a 2], [http://git.kernel.org/linus/629cc356653719c206a05f4dee5c5e242edb6546 3], [http://git.kernel.org/linus/c7d9300f367f480aee4663a0e3695c5b48859a1a 4], [http://git.kernel.org/linus/dcba8848d3bc83ec9ee0858b9ae6e4f1c1fa7fa3 5], [http://git.kernel.org/linus/bfde82ef51e3ea6ab8634d0fdbf5adcdd1b429cb 6], [http://git.kernel.org/linus/9fbc04f2493929a69fd9e53b5fb53c127d7950d5 7], [http://git.kernel.org/linus/827f3b4974c5db2968d4979fe6a0ae00ab37bdd8 8)]

perf kmem: This tool is mostly a perf version of kmemtrace-user. It shows various information of SLAB

Code: [http://git.kernel.org/linus/ba77c9e11111a172c9e8687fe16a6a173a61916f (commit 1], [http://git.kernel.org/linus/7707b6b6f8d9188b612f9fc88c65411264b1ed57 2], [http://git.kernel.org/linus/f3ced7cdb24e7968a353d828955fa2daf4167e72 3], [http://git.kernel.org/linus/079d3f653134e2f2ac99dae28b08c0cc64268103 4)]

perf diff: perf diff shows performance differences between various records

Code: [http://git.kernel.org/linus/86a9eee047ba09a714c3b8e27c9df2bbf715393a (commit)]

perf perl scripts: (Recommended LWN article: [http://lwn.net/Articles/373842/ Scripting support for perf] It's a Perl scripting engine for programmable 'perf trace' scripting. See perf trace -g/--gen-script and perf trace -s/--script.

Code: [http://git.kernel.org/linus/956ffd027bedc4106b901eb6a50f0a6c6de4113d (commit 1], [http://git.kernel.org/linus/16c632de64a74644a46e7636db26b2cfb530ca13 2], [http://git.kernel.org/linus/bcefe12eff5dca6fdfa94ed85e5bee66380d5cd9 3], [http://git.kernel.org/linus/89fbf0b8a021cbf60abeacfb6b538e97c83afada 4], [http://git.kernel.org/linus/4b9c0c596ea826ef784eb83f663c5351ed01ba6d 5], [http://git.kernel.org/linus/a6005123ce22770dbd91bc3cb637ce0807ab959b 6)]

perf filters: This feature adds "--filter expression" support to tracepoints, which utilizes the filter engine within the kernel. For example, to trace only timer interrupts in the system: "perf record -e irq:irq_handler_entry --filter='irq==0' -R -f -a sleep 10". Or to only record IRQ 19 when the 'achi' handler is triggered: "perf record -e irq:irq_handler_entry --filter='irq==19 && name==ahci' -R -f -a sleep 10"

Code: Add filter Suppport [http://git.kernel.org/linus/6fb2915df7f0747d9044da9dbff5b46dc2e20830 (commit)], [http://git.kernel.org/linus/c171b552a7d316c7e1c3ad6f70a30178dd53e14c (commit)]

1.4. recvmmsg(), a syscall for batching recvmsg() calls

Recommended LWN article: [http://lwn.net/Articles/334854/ In brief]

Recommended slides: [http://vger.kernel.org/netconf2009_slides/recvmmsg.pdf Batch datagram processing]

recvmmsg() is a new syscall that allows to receive with a single syscall multiple messages that would require multiple calls to recvmsg(). For high-bandwith, small packet applications, throughput and latency are improved greatly.

Code: [http://git.kernel.org/linus/a2e2725541fad72416326798c2d7fa4dafb7d337 (commit)]

Recommended LWN article: [http://lwn.net/Articles/366986/ TCP cookie transactions]

Recommended Wikipedia article: [http://en.wikipedia.org/wiki/TCP_Cookie_Transactions TCP Cookie Transactions]

TCP Cookie Transactions (TCPCT) is an extension of TCP intended to secure it against denial-of-service attacks, such as resource exhaustion by SYN flooding and malicious connection termination by third parties. Unlike the original SYN cookies approach, TCPCT does not conflict with other TCP extensions, but requires TCPCT support in the client (initiator) as well as the server (responder) TCP stack. The immediate reason for the TCPCT extension is deployment of the DNSSEC protocol.

Code: [http://git.kernel.org/linus/e6b4d11367519bc71729c09d05a126b133c755be (commit)], [http://git.kernel.org/linus/da5c78c82629a167794436e4306b4cf1faddea90 (commit)], [http://git.kernel.org/linus/519855c508b9a17878c0977a3cdefc09b59b30df (commit)], [http://git.kernel.org/linus/435cf559f02ea3a3159eb316f97dc88bdebe9432 (commit)], [http://git.kernel.org/linus/e56fb50f2b7958b931c8a2fc0966061b3f3c8f3a (commit)], [http://git.kernel.org/linus/bd0388ae77075026d6a9f9eb6026dfd1d52ce0e9 (commit)], [http://git.kernel.org/linus/4957faade11b3a278c3b3cade3411ddc20afa791 (commit)]

1.6. Support for Xen PV-on-HVM guests

Support for Xen PV-on-HVM guests can be implemented almost entirely in userspace, except for handling one annoying MSR that maps a Xen hypercall blob into guest address space. This patch implementes a new ioctl, KVM_XEN_HVM_CONFIG, that lets userspace tell KVM which MSR the guest will write to, as well as the starting address and size of the hypercall blobs (one each for 32-bit and 64-bit) that userspace has loaded from files. When the guest writes to the MSR, KVM copies one page of the blob from userspace to the guest.

Code: [http://git.kernel.org/linus/ffde22ac53b6d6b1d7206f1172176a667eead778 (commit)]

1.7. Swappable KSM pages

Kernel Samepage Merging (KSM) is a feature merged in [http://kernelnewbies.org/Linux_2_6_32 Linux 2.6.32] which deduplicates memory of virtualized guests. The implementation, however, didn't allow to swap the pages that were shared. This release brings swap support for KSM pages.

Code: [http://git.kernel.org/linus/5ad6468801d28c4d4ac9f48ec19297817c915f6a (commit)]

1.8. Block IO Controller

Recommended LWN article: [http://lwn.net/Articles/360958/ The block I/O controller]

Control groups are virtual "containers" that are created as directories inside a special virtual filesystem (usually, with the help of [http://libcg.sourceforge.net/ tools]), and arbitrary sets of processes can be added to that control group, which you can configure to a set of cpu scheduling or memory limits that will affect to all the processes inside the control group.

This release adds a block IO controller. Currently, CFQ IO scheduler uses it to recognize task groups and control disk bandwidth allocation to such task groups (somewhat like CFQ priorities, but implemented in a very different way), this controller will be extended in the future. For more details, read the [http://git.kernel.org/linus/72f924f62a6eb375c7c237ecc911f95be0531d1a documentation]

Code: [http://git.kernel.org/linus/615f0259e6940293359a189f4881bb28c2fea40b (commit 1)], [http://git.kernel.org/linus/cdb16e8f739985b8a5c9f4569b026583bbcd01a5 2] ,[http://git.kernel.org/linus/bf7919371025412978268efca4b09dd847acb395 3], [http://git.kernel.org/linus/31e4c28d95e64f2d5d3c497a3ecf37c62de635b4 4], [http://git.kernel.org/linus/1fa8f6d68b5c8ca0a608fd8d296c5f07ac788cd6 5], [http://git.kernel.org/linus/f04a64246344ad50e4b4b4186174a0912d07f30b 6], [http://git.kernel.org/linus/58ff82f34cded3812af5b6c69b6aa626b6be2490 7], [http://git.kernel.org/linus/25bc6b07767fe77422312eda2af99c9477f76191 8], [http://git.kernel.org/linus/25fb5169d4c9d4255107abbb7c08ab712434efc8 9], [http://git.kernel.org/linus/dae739ebc4c590630039533a5bbd05865966094f 10] ,[http://git.kernel.org/linus/b1c3576961847da26c91b1e97f226bb66be5fa3f 11], [http://git.kernel.org/linus/220841906fccafaf4094e87bdb6d252e20cf8c7c 12], [http://git.kernel.org/linus/2868ef7b39490e6b41c2c61cd9a5cd891e778b54 13], [http://git.kernel.org/linus/24610333d578478d354144ab4709a203684afc5f 14], [http://git.kernel.org/linus/f8d461d692c341add957fb973fb5ee1f62039dc7 15], [http://git.kernel.org/linus/8682e1f15f26dae9a9e8af794d179055fbd81166 16], [http://git.kernel.org/linus/f75edf2dc828802d358393be80a6c89e919f8273 17], [http://git.kernel.org/linus/f26bd1f0a3a31bc5e16d285f5e1b00a56abf6238 18], [http://git.kernel.org/linus/ae30c286553c91c49af5cbc0265a05a6543d0c52 19], [http://git.kernel.org/linus/c04645e592d4dd60c58def40c913699d4c806727 20)]

1.9. Compcache: memory compressed swapping

Recommended LWN article: [http://lwn.net/Articles/334649/ Compcache: in-memory compressed swapping]

Compcache is a project (still under development, only available in Staging) creates RAM-based block devices (/dev/ramzswapX) which are used as swap disks. Pages swapped to this virtual device are compressed to a smaller size. Part of your RAM is used as usually, and another part (the size is configurable) is used to save compressed pages, increases the amount of RAM you can use in practice.

This feature can be very useful in many cases: Netbooks, smartphones and other embedded devices, distro installers, dumb clients without disk, virtualization, or old machines with not enought RAM to run modern software.

Measurements have found this feature very effective. See [http://code.google.com/p/compcache/wiki/Performance this page to see some benchmarks]. The project home page can be found at [http://compcache.googlecode.com/ http://compcache.googlecode.com/]

Code: [http://git.kernel.org/linus/306b0c957f3f0e7da6551652abbfe17b560173ce (commit)], [http://git.kernel.org/linus/644bf7b5983cf2540b57a5b25b775cb3c1e8e943 (commit)]

1.10. Graphic improvements

Besides the inclusion of Nouveau, there's the usual round of improvements to the graphic stack that have become common after GEM and KMS were merged.

1.11. Nintendo Wii and Gamecube support

The [http://gc-linux.sourceforge.net/ gc-linux.sourceforge.net] project has been working in Linux support for the PPC-based game consoles Nintendo Wii and Nintendo Gamecube. This release merges this support in the kernel.

1.12. VMware drivers

VMWare has contributed two drivers for the VWware Virtual GPU, and for the VMware's virtual Ethernet NIC vmxnet3. Thanks to udev, this means that Linux guests running inside a VMware host will have optimal graphic and network performance out-of-the-box.

vmwgfx: [http://git.kernel.org/linus/632f61178d0473861ba77e774bb654b37bc7eccc (commit)], [http://git.kernel.org/linus/fb1d9738ca053ea8afa5e86af6463155f983b01c (commit)] vmxnet3: [http://git.kernel.org/linus/d1a890fa37f27d6aca3abc6e25e4148efc3223a6 (commit)], [http://git.kernel.org/linus/115924b6bdc7cc6bf7da5b933b09281e1f4e17a9 (commit)]

1.13. Reiserfs de-BKLification

One of the biggest shortcomings of reiserfs v3 (and one of the reasons why most distros use Ext instead) is that its codebase handles concurrency using a single big lock - the BKL (Big Kernel Lock). This means that its SMP scalability is very poor. This release won't fix that issue, but it replaces the BKL with a reiserfs-specific solution. In this release, there are no more traces of the BKL inside reiserfs. It has been converted into a recursive mutex. This sounds dirty but plugging a traditional lock into reiserfs would involve a deeper rewrite as the reiserfs architecture is based on the ugly big kernel lock rules.

Due to the subtle semantics of the locking changes, some workloads may have small performance regressions and other have improvements.

Code: Many commits.

1.14. Android removed from the Linux kernel

Recommended article: [http://www.kroah.com/log/linux/android-kernel-problems.html Android and the Linux kernel community]

Google doesn't seem to have interest in improving the Android drivers to have minimum quality standards which could allow to merge them in the main Linux tree and share them with the rest of community. Of course, that's totally legal, but it's sad that a project that is doing so much to bring open source to the masses has become an example of how not to interact with an open source community.

2. Various core changes

3. Block

4. Virtualization

5. MD/DM

6. Filesystems

7. Networking

8. Security

9. Tracing/Profiling

10. Crypto

11. Architecture-specific changes

12. Drivers

12.1. Graphics

12.2. Storage

12.3. Networking devices

12.4. USB

12.5. FireWire

12.6. Input

12.7. Sound

12.8. Staging Drivers

12.9. V4L/DVB

12.10. HID

12.11. RTC

12.12. Bluetooth

12.13. MFD

12.14. MTD

12.15. HWMON

12.16. Various

12.17. Other news sources tracking the kernel changes

KernelNewbies: Linux_2_6_33 (last edited 2011-02-13 00:05:40 by diegocalleja)