KernelNewbies:

Linux 2.6.38 has been released.

Summary: This release adds support for a desktop interactivity feature called automatic process grouping, significant scalability improvements in the VFS, Btrfs LZO compression and read-only snapshots, transparent Huge Page support (without using hugetblfs), support for the B.A.T.M.A.N. mesh protocol, support for the AMD Fusion APUs, many drivers and other changes.

TableOfContents()

1. Prominent features (the cool stuff)

1.1. Automatic process grouping (a.k.a. "the patch that does wonders")

The most impacting feature in this release is the so-called "patch that does wonders", a patch that changes substantially how the process scheduler assigns shares of CPU time to each process. With this feature the system will group all processes with the same session ID as a single scheduling entity. Example: Let's imagine a system with six CPU-hungry processes, with the first four sharing the same session and the other using two different sessions each one.

Without automatic process grouping:  [proc. 1 | proc. 2 | proc. 3 | proc. 4 | proc. 5 | proc. 6] 

With automatic process grouping:    [proc. 1, 2, 3, 4  |     proc. 5       |     proc. 6      ] 

The session ID is a property of processes in Unix systems (you can see it with commands like ps -eo session,pid,cmd). It is inherited by forked child processes, and changed using setsid(2). The bash shell uses setsid() every time it is started, which means you can run a "make -j 20" inside a shell in your desktop and not notice it while you browse the web. This feature is implemented on top of group scheduling (merged in [[http://kernelnewbies.org/Linux_2_6_24#head-16d608b6aba030fe15ba3bbc75655391ae98d707 2.6.24]). You can disable it in /proc/sys/kernel/sched_autogroup_enabled

Code: [http://git.kernel.org/linus/5091faa449ee0b7d73bc296a93bca9540fc51d0a (commit)]

1.2. VFS scalability: scaling the directory cache

There is ongoing effort to make the Linux VFS layer ("Virtual File System", the code that glues the syscall and the filesystem) more scalable. In the previous release some changes [http://kernelnewbies.org/Linux_2_6_36#head-4e895d3e811a130d2e2d71beeae2f4c552ffdc36 were already merged] as part of this work, in this release, the dcache (alias for "directory cache", which keeps a cache of directories ) and the whole path lookup mechanisms have been reworked to be more scalable (you can find details in [https://lwn.net/Articles/419811/ the LWN article]).

These changes make the VFS more scalable in multithreaded workloads, but more interestingly (and it's what excites Linus Torvalds) they also make some single threaded workloads quite faster (due to the removal of atomic CPU operations in the code paths): a hot-cache "find . -size" on his home directory seems to be 35% faster. Single threaded git diff on a cached kernel tree runs 20% faster (64 parallel git diffs increase throughtput by 26 times). Everything that calls stat() a lot is faster.

Changes: Far too many to track here, see the patches done by Nick Piggin in [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=history;f=fs;hb=b3e19d924b6eaf2ca7d22cba99a517c5171007b6 this list] (inverse chronological order)

1.3. Btrfs LZO compression, read-only snapshots

Btrfs adds supports for transparent compression using the [http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Oberhumer LZO algorithm], as an alternative to zlib. You can find [https://lwn.net/Articles/416644/ here] a small performance comparison.

There is also support for marking snapshots as read-only. Finally, filesystems which find errors will be "force mounted" as readonly, which is a step forward to make the codebase more tolerant to failures.

Code: LZO [http://git.kernel.org/linus/261507a02ccba9afda919852263b6bc1581ce1ef (commit 1],[http://git.kernel.org/linus/a6fa6fae40ec336c7df6155255ae64ebef43a8bc 2], [http://git.kernel.org/linus/1a419d85a76853d7d04e9b6280a80e96770bf3e3 3)]; read-only snapshots [http://git.kernel.org/linus/b83cc9693f39689490970c19f6c5b866f6719a70 (commit 1], [http://git.kernel.org/linus/0caa102da82799efaba88e234484786a9591c797 2)], forced readonly mounts [http://git.kernel.org/linus/acce952b0263825da32cf10489413dec78053347 (commit)]

1.4. Transparent Huge Pages

Processors manage memory in small units called "pages" (which is 4 KB in size in x86). Each process has a virtual address space, and there is a "page table" where all the correspondencies between RAM page <-> virtual address page are kept. The work of walking the page table to find out which RAM page corresponds to which virtual address in a process is expensive, so the CPU has a cache of translations. However, this cache is not very big and it only supports 4KB pages, so many data-intensive workloads (databases, KVM).

To solve this problem, modern processors add cache entries that support pages bigger than 4KB (like 2MB/4MB). Until now, the one way that userspace had to use those pages in Linux was hugetblfs, a filesystem-based API. This release adds support for transparent hugepages ( - hugepages are used automatically where possible. Transparent Huge Pages can be configured to be used always or only as requested with madvise(MADV_HUGEPAGE), and its behaviour can be changed online in /sys/kernel/mm/transparent_hugepage/enabled. For more details, check [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/vm/transhuge.txt;hb=HEAD Documentation/vm/transhuge.txt]

Code: Far too many to track here, see the patches from Andrea Arcangeli in [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=shortlog;h=a664b2d8555c659127bf8fe049a58449d394a707 this list] (inverse chronological order)

1.5. B.A.T.M.A.N. Mesh protocol

B.A.T.M.A.N. is an alias for "Better Approach To Mobile Adhoc Networking". An ad hoc network is a decentralized network that does not rely on a preexisting infrastructure, such as routers in wired networks or access points in managed (infrastructure) wireless networks. Instead, each node participates in routing by forwarding data for other nodes, and so the determination of which nodes forward data is made dynamically based on the network connectivity. B.A.T.M.A.N. is a routing protocol implementation ot these networks. B.A.T.M.A.N is useful for emergency situations like natural disasters, military conflicts or Internet censorship. More information about this project can be found at [http://www.open-mesh.org/ http://www.open-mesh.org/]

Code: [http://git.kernel.org/linus/c6c8fea29769d998d94fcec9b9f14d4b52b349d3 (commit)]

1.6. AMD Fusion APU support

This release adds support for the [http://en.wikipedia.org/wiki/AMD_Fusion AMD Fusion GPU+CPUs]

1.7. Dirty memory limits in the memory controller

This release adds support to control dirty page limits of each memory controller cgroup (the memory controller allows to create group processes and give them memory limits different to those of the system). The interface has the same file names as the files /proc/sys/vm/dirty_*: memory.dirty_ratio, memory.dirty_limit_in_bytes, memory.dirty_background_ratio, memory.dirty_background_limit_in_bytes. See the commit links for documentation.

* Code: [http://git.kernel.org/linus/db16d5ec1f87f17511599bc77857dd1662b5a22f (commit 1],[http://git.kernel.org/linus/2a7106f2cb0768d00fe8c1eb42a754a7d8518f08 2], [http://git.kernel.org/linus/ece72400c2a27a3d726cb0854449f991d9fcd2da 3)]

2. Drivers and architectures

All the driver and architecture-specific changes can be found in the [http://kernelnewbies.org/Linux_2_6_38-DriversArch Linux_2_6_38-DriversArch page]

3. Core

4. CPU scheduler

5. Memory management

6. Block

Device Mapper (DM)

7. File systems

XFS

CIFS

EXT2/3

SQUASHFS

NILFS2

8. Networking

Wireless

* dcbnl: add support for ieee8021Qaz attributes [http://git.kernel.org/linus/3e29027af43728c2a91fe3f735ab2822edaf54a8 (commit)]

9. Crypto

10. Virtualization

11. Security

Smack

12. Tracing/perf

/da169f5df2764a6a937cb3b07562e269edfb1c0e (commit)]

* tracing: Allow raw syscall trace events for non privileged users [http://git.kernel.org/linus/fe5542030dce3b951f9eaf3ecb9a7bc5fa7bfed1 (commit)], [http://git.kernel.org/linus/53cf810b1934f08a68e131aeeb16267a778f43df (commit)], [http://git.kernel.org/linus/61c32659b12c44e62de32fbf99f7e4ca783dc38b (commit)]


CategoryReleases

KernelNewbies: Linux_2_6_38 (last edited 2011-03-15 09:03:38 by diegocalleja)