KernelNewbies:

Linux 2.6.35 [http://lkml.org/lkml/2010/8/1/188 has been released] on 1 Aug, 2010.

Summary: Linux 2.6.35 includes support for transparent spreading of incoming network load across CPUs, Direct-IO support for Btrfs, an new experimental journal mode for XFS, the KDB debugger UI based on top of KGDB, improvements to 'perf', H.264 and VC1 video acceleration in Intel G45+ chips, support for the future Intel Cougarpoint graphic chip, power management for AMD Radeon chips, a memory defragmentation mechanism, support for the Tunneling Protocol version 3 (RFC 3931), support for multiple multicast route tables, support for the CAIF protocol used by ST-Ericsson products, support for the ACPI Platform Error Interface, and many new drivers and small improvements.

Note: Details on architecture-specific and driver changes have been moved to this page: [wiki:Linux_2_6_35-DriversArch Linux_2_6_35-DriversArch]

TableOfContents()

1. Prominent features (the cool stuff)

1.1. Transparent spreading of incoming network traffic load across CPUs

Recommended LWN articles: [http://lwn.net/Articles/362339/ "Receive packet steering"] and [http://lwn.net/Articles/382428/ "Receive flow steering"]

Network cards have improved the bandwidth to the point where it's hard for a single modern CPU to keep up. Two new features contributed by Google aim to spread the load of network handling across the CPUs available in the system: Receive Packet Steering (RPS) and Receive Flow Steering (RFS).

RPS distributes the load of received packet processing across multiple CPUs. This solution allows protocol processing (e.g. IP and TCP) to be performed on packets in parallel (contrary to the previous code). For each device (or each receive queue in a multi-queue device) a hashing of the packet header is used to index into a mask of CPUs (which can be configured manually in /sys/class/net/<device>/queues/rx-<n>/rps_cpus) and decide which CPU will be used to process a packet. But there're also some heuristics provided by the RFS side of this feature. Instead of randomly choosing the CPU from a hash, RFS tries to use the CPU where the application running the recvmsg() syscall is running or has run in the past, to improve cache utilization. Hardware hashing is used if available. This feature effectively emulates what a multi-queue NIC can provide, but instead it is implement in software and for all kind of network hardware, including single queue cards and not excluding multiqueue cards.

Benchmarks of 500 instances of netperf TCP_RR test with 1 byte request and response show the potential benefit of this feature, a e1000e network card on 8 core Intel server goes from 104K tps at 30% CPU usage, to 303K tps at 61% CPU usage when using RPS+RFS. A RPC test which is similar in structure to the netperf RR test with 100 threads on each host, but doing more work in userspace that netperf, goes from 103K tps at 48% of CPU utilization to 223K at 73% CPU utilization and much lower latency.

Code: [http://git.kernel.org/linus/0a9627f2649a02bea165cfd529d7bcb625c2fcad (commit 1], [http://git.kernel.org/linus/df3345457a7a174dfb5872a070af80d456985038 2], [http://git.kernel.org/linus/fec5e652e58fa6017b2c9e06466cb2a6538de5b4 3)]

1.2. Btrfs improvements

1.3. XFS Delayed logging

This version adds a logging (journaling) mode called delayed logging, which is very briefly modeled after the journaling mode in the ext3/4 and reiserfs filesystems. It allows to accumulated multiple asynchronous transactions in memory instead of possibly writing them out many times. The I/O bandwidth used for the log decreases by orders of magnitude and performance on metadata intensive workloads increases massively. The log disk format is not changed, only the in-memory data structures and code. This feature is experimental, so it's not recommended for final users or production servers. Those who want to test it can enable it with the "-o delaylog" mount option. Code: [http://git.kernel.org/linus/a9a745daadab26f13884ff26a50fa38247c11ce9 (commit 1], [http://git.kernel.org/linus/71e330b593905e40d6c5afa824d38ee02d70ce5f 2)]

1.4. KDB kernel debugger frontend

The Linux kernel has had a kernel debugger since 2.6.26, called Kgdb. But Kgdb is not the only linux kernel debugger, there is also KDB, developed years ago by SGI. The key difference between Kgdb and KDB is that using Kgdb requires an additional computer to run a gdb frontend, and you can do source level debugging. KDB, on the other hand, can be run on the local machine and can be used to inspect the system, but it doesn't do source-level debugging. What is happening in this version is that Jason Wessel, from Windriver, has ported KDB to work on top of the Kgdb core, making possible to use both interfaces.

Code: [http://git.kernel.org/linus/5d5314d6795f3c1c0f415348ff8c51f7de042b77 (commit 1], [http://git.kernel.org/linus/67fc4e0cb931d6b4ccf21248e4199b154478ecea 2], [http://git.kernel.org/linus/dcc7871128e99458ca86186b7bc8bf27ff0c47b5 3], [http://git.kernel.org/linus/84c08fd61e2d21702337e9fe366e97cdf09bf797 4], [http://git.kernel.org/linus/ada64e4c98eb5f04a9ca223c5ff9e7ac22ce6404 5], [http://git.kernel.org/linus/f503b5ae53cb557ac351a668fcac1baab1cef0db 6], [http://git.kernel.org/linus/ba797b28131b1f1367b662936ea370239d603cff 7], [http://git.kernel.org/linus/4fe1da4ebc18c4c42fa56c228447f68033fce5f0 8)]

1.5. perf improvements

1.6. Graphic improvements

This version carries a bunch of interesting improvements to the graphics stack.

1.7. Memory compaction

Recommended LWN article: [http://lwn.net/Articles/368869/ "Memory compaction"]

The memory compaction mechanism tries reduces external memory fragmentation in a memory zone by trying to move used pages to create a new big block of contiguous used pages. When compaction finishes, the memory zone will have a big block of used pages and a big block of free pages. This will make easier to allocate bigger chunks of memory. The mechanism is called "compaction" to distinguish it from other forms of defragmentation.

In this implementation, a full compaction run involves two scanners operating within a zone, a migration scanner and a free scanner. The migration scanner starts at the beginning of a zone and finds all used pages that can be moved. The free scanner begins at the end of the zone and searches for enough free pages to migrate all the used pages found by the previous scanner. A compaction run completes within a zone when the two scanners meet and used pages are migrated to the free blocks scanned. Testing has showed the amount of IO required to satisfy a huge page allocation is reduced significantly.

Memory compaction can be triggered in one of three ways. It may be triggered explicitly by writing any value to /proc/sys/vm/compact_memory and compacting all of memory. It can be triggered on a per-node basis by writing any value to /sys/devices/system/node/nodeN/compact where N is the node ID to be compacted. When a process fails to allocate a high-order page, it may compact memory in an attempt to satisfy the allocation instead of entering direct reclaim. Explicit compaction does not finish until the two scanners meet and direct compaction ends if a suitable page becomes available that would meet watermarks.

Code: [http://git.kernel.org/linus/3f6c82728f4e31a97c3a1b32abccb512fed0b573 (commit 1], [http://git.kernel.org/linus/67b9509b2c68ae38cecb83a239881cb0ddf087dc 2], [http://git.kernel.org/linus/7f60c214fd3a360461f3286c6908084f7f8b1950 3], [http://git.kernel.org/linus/e9e96b39f932a065e14f5d5bab0797ae261d03b5 4] ,[http://git.kernel.org/linus/3fe2011ff51e92500010a495df4be86745fbbda9 5], [http://git.kernel.org/linus/a8bef8ff6ea15fa4c67433cab0f5f3484574ef7c 6], [http://git.kernel.org/linus/d7a5752c0c19750312efab3a2a80d350e11fa4a2 7], [http://git.kernel.org/linus/748446bb6b5a9390b546af38ec899c868a9dbcf0 8], [http://git.kernel.org/linus/f1a5ab1210579e2d3ac8c0c227645823af5aafb0 9], [http://git.kernel.org/linus/c175a0ce7584e5b498fff8cbdb9aa7912aa9fbba 10], [http://git.kernel.org/linus/76ab0f530e4a01d4dc20cdc1d5e87753c579dc18 11], [http://git.kernel.org/linus/ed4a6d7f0676db50b5023cc01f6cda82a2f2a307 12], [http://git.kernel.org/linus/5e7719058079a1423ccce56148b0aaa56b2df821 13], [http://git.kernel.org/linus/56de7263fcf3eb10c8dcdf8d59a9cec831795f3f 14], [http://git.kernel.org/linus/4f92e2586b43a2402e116055d4edda704f911b5b 15)]

1.8. Support for multiple multicast route tables

Normally, a multicast router runs a userspace daemon and decides what to do with a multicast packet based on the source and destination addresses. This feature adds support for multiple independent multicast routing instances, so the kernel is able to take interfaces and packet marks into account and run multiple instances of userspace daemons simultaneously, each one handling a single table. Code: [http://git.kernel.org/linus/f0ad0860d01e47a3ffd220564c5c653b3afbe962 (commit 1], [http://git.kernel.org/linus/d1db275dd3f6e4182c4c4b4a1ac6287925d60569 2], [http://git.kernel.org/linus/5b285cac3570a935aaa28312c1ea28f9e01c5452 3)]

1.9. L2TP Version 3 (RFC 3931) support

This version adds support for Layer 2 Tunneling Protocol (L2TP) version 3, [http://www.ietf.org/rfc/rfc3931.txt RFC 3931]. L2TP provides a dynamic mechanism for tunneling Layer 2 (L2) "circuits" across a packet-oriented data network (e.g., over IP). L2TP, as originally defined in RFC 2661, is a standard method for tunneling Point-to-Point Protocol (PPP) [RFC1661] sessions. L2TP has since been adopted for tunneling a number of other L2 protocols, including ATM, Frame Relay, HDLC and even raw ethernet frames, this is the version 3. Code: [http://git.kernel.org/linus/f7faffa3ff8ef6ae712ef16312b8a2aa7a1c95fe (commit 1], [http://git.kernel.org/linus/e0d4435f93905f517003cfa7328a36ea19788147 2], [http://git.kernel.org/linus/0d76751fad7739014485ba5bd388d4f1b4fd4143 3], [http://git.kernel.org/linus/789a4a2c61d843df67988d69e7c3f3a4bca97e8e 4)]

1.10. CAIF Protocol support

Support for the CAIF protocol. CAIF is a MUX protocol used by ST-Ericsson cellular modems for communication between Modem and host. The host processes can open virtual AT channels, initiate GPRS Data connections, Video channels and Utility Channels. The Utility Channels are general purpose pipes between modem and host. ST-Ericsson modems support a number of transports between modem and host. Currently, UART and Loopback are available for Linux [http://git.kernel.org/linus/70596b612c04694806a31dd389bd796c035085fa (commit 1], [http://git.kernel.org/linus/09009f30de188c847d72039e6250bfea56a0f887 2], [http://git.kernel.org/linus/f671c54207d8a47129f35a84569fdfda614d2439 3], [http://git.kernel.org/linus/2721c5b9dd2a56a9710021c00146bb26ba8dd7b3 4], [http://git.kernel.org/linus/b482cd2053e3b90a7b33a78c63cdb6badf2ec383 5], [http://git.kernel.org/linus/15c9ac0c80e390df09ce5730a7b08b13e07a8dd5 6], [http://git.kernel.org/linus/c72dfae2f77620e5b3fcee1beeee7e536a42b2ad 7], [http://git.kernel.org/linus/e6f95ec8db312491235b4f06343fbd991a82ce20 8], [http://git.kernel.org/linus/cc36a070b5901cd54386348b4d79d2daac91ce75 9], [http://git.kernel.org/linus/3908c6902372206cc582ecf459af889b09a150c9 10], [http://git.kernel.org/linus/edc7616c307ad315159a8aa050142237f524e079 11], [http://git.kernel.org/linus/9b27105b4a44c54bf91ecd7d0315034ae75684f7 12)]

1.11. ACPI Platform Error Interface support

Support for the ACPI Platform Error Interface (APEI). This improves NMI handling especially. In addition it supports the APEI Error Record Serialization Table (ERST), the APEI Generic Hardware Error Source and APEI Error INJection (EINJ) and saving of MCE (Machine Check Exception) errors into flash. For more information about APEI, please refer to [http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf ACPI Specification version 4.0], chapter 17. Code: [http://git.kernel.org/linus/15651291a2f8c11e7e6a42d8bfde7a213ff13262 (commit 1], [http://git.kernel.org/linus/a643ce207f3e70030bdb431e2a363cc111a60c1a 2], [http://git.kernel.org/linus/9dc966641677795f4d6b0a9ba630d6a3a3e24a57 3], [http://git.kernel.org/linus/ea8c071cad789b1919355fc7a67182a5c9994e6b 4], [http://git.kernel.org/linus/e40213450b53157967a1f83eda50e9a941c13a08 5], [http://git.kernel.org/linus/affb72c3a8984ba55e055b0a0228c3ea1a056758 6], [http://git.kernel.org/linus/801eab8118f61255d8e2be35939c572042618742 7], [http://git.kernel.org/linus/fab1c23242528771a955c475ef23d99156a71a7f 8], [http://git.kernel.org/linus/d334a49113a4a33109fd24e46073280ecd1bea0d 9], [http://git.kernel.org/linus/06d65deade9aabba58e0518df86dcd324e86b832 10], [http://git.kernel.org/linus/a08f82d08053fb6e3aa3635c2c26456d96337c8b 11], [http://git.kernel.org/linus/482908b49ebfa453dd0455910c951c750567c05d 12], [http://git.kernel.org/linus/6e320ec1d98f9eb93d5b2a5d70e2f40dce923f1b 13)]

2. Various core changes

3. Filesystems

4. Block

5. Memory management

6. Networking

7. Tracing/Profiling

8. Crypto

9. Virtualization

10. MD

11. CPU scheduler

12. Cpufreq/cpuidle

13. Security

KernelNewbies: Linux_2_6_35 (last edited 2010-08-02 06:51:27 by 94)