KernelNewbies:

Linux 3.12 [https://lkml.org/lkml/2013/11/3/160 was released on November 2, 2013]

/!\ /!\ /!\ This page is incomplete. Sorry for the delay /!\ /!\ /!\

Summary: This release adds support for offline deduplication in Btrfs, automatic GPU switching in laptops with dual GPUs, a performance boost for modern AMD Radeon hardware, improved locking performance for virtualized guests, better RAID-5 multicore performance, improved and more reliable out-of-memory handling, improved VFS path name resolution scalability, improvements to the timerless multitasking mode, separate modesetting and rendering device nodes in the graphics DRM layer, new drivers and many small improvements.

TableOfContents()

1. Prominent features

1.1. Offline data deduplication support in Btrfs

The Btrfs filesystem has gained support for offline data deduplication. Deduplication consists in removing copies of repeated data in the filesystem, since the data is the same only one copy is necessary. In some particular workloads, like virtualization VMs -which often contain similar copies of operating systems- the gains can be enormous. By "offline", it means that the deduplication process is done when the file system is mounted and running, but it's not done automatically and transparently as processes write data, but rather it's triggered by userspace software at a time controlled by the system administrator. Online deduplication will be added future releases.

The bedup deduplication tool has a branch that works against this support. The branch can be found [https://github.com/g2p/bedup/tree/wip/dedup-syscall here].

The author of the deduplication support has also written an sample deduplication tool, duperemove, which can be found [https://github.com/markfasheh/duperemove here].

Code: [http://git.kernel.org/linus/416161db9b63e353a8fb79d1369779175102fca1 commit]

1.2. Graphic performance boost for AMD Radeon hardware

The website [http://Phoronix.com Phoronix.com] found that graphic performance in modern AMD Radeon GPUs had improved a lot in Linux 3.12. However, there hasn't been any important modification on in the Radeon driver that can cause such massive gains. After [http://www.phoronix.com/scan.php?page=article&item=linux_312_performance&num=1 some investigation], Phoronix found out that the responsible change for this boost wasn't a change in the Radeon driver itself, but [http://anzwix.com/a/Linux%20Kernel/CpufreqOndemandChangeTheCalculationOfTargetFrequ a change to the algorithms in the cpufreq ondemand governor]. Apparently, the ondemand governor was oscillating too much between frequencies, and this oscillation harmed graphic performance for Radeon GPUs. The new algorithm added a better frequency algorithm that eliminates this problem.

Code: [http://git.kernel.org/linus/dfa5bb622555d9da0df21b50f46ebdeef390041b commit]

1.3. Automatic GPU switching in laptops with dual GPUs

Some laptop hardware, like Nvidia Optimus, have two GPUs, one optimized for performance and other for power saving. Until now, some hacks have been needed to switch between these GPUs. In this release, the driver handles the switch automatically

Code: [http://git.kernel.org/linus/0d69704ae348c03bc216b01e32a0e9a2372be419 commit 1], [http://git.kernel.org/linus/5addcf0a5f0fadceba6bd562d0616a1c5d4c1a4d 2], [http://git.kernel.org/linus/246efa4a072f3a2e03010ef0b78b0974ec69c377 3]

1.4. Separate devices nodes for graphics mode setting and rendering

Recent hardware development (especially on ARM) shows that rendering (via GPU) and mode-setting (via display-controller) are not necessarily bound to the same graphics device. This release incorporates in the graphics layer support for separate device nodes for mode setting and rendering. The main usage is to allow different access-modes for graphics-compositors (which require the modeset API) and client-side rendering or GPGPU-users (which both require the rendering API).

For more information, see this blog post: [https://dvdhrm.wordpress.com/2013/09/01/splitting-drm-and-kms-device-nodes/ Splitting DRM and KMS device nodes]

Code: [http://git.kernel.org/linus/1793126fcebd7c18834f95d43b55e387a8803aa8 commit 1], [http://git.kernel.org/linus/7d7612582c15af8772c2fb2473d5fc7eebfefae2 2], [http://git.kernel.org/linus/f33bcab9e816c5bf56b74c3007790f2a256910eb 3], [http://git.kernel.org/linus/10ba50129ab0bdbc0ee712e50913d1c8db88c5f0 4]

1.5. Improved timerless multitasking: allow timekeeping CPU go idle

Linux 3.10 added support for [http://kernelnewbies.org/Linux_3.10#head-62fadba76893e85ee7fb75d548536c5635baca54 timerless multitasking], that is, the ability to run processes without needing to fire up the timer interrupt that is traditionally used to implement multitasking. This support, however, had a caveat: it could turn off interrupts in all CPUs, except one that is used to track timer information for the other CPUs. But that CPU keeps the timer turned on even if all the CPUs are idle, which was useless. This release allows to disable the timer for the timekeeping CPU when all CPUs are idle.

Recommended LWN article: [https://lwn.net/Articles/558284/ Is the whole system idle?]

Code: [http://git.kernel.org/linus/b44379af1cf40050794832c38ea6a64e07eb5087 commit 1], [http://git.kernel.org/linus/8bdf7a252b6398dbd4beaa7a2ba0904cea0abd04 2], [http://git.kernel.org/linus/d4bd54fbac2ea5c30eb976ca557e905f489d55f4 3], [http://git.kernel.org/linus/217af2a2ffbfc1498d1cf3a89fa478b5632df8f7 4], [http://git.kernel.org/linus/0edd1b1784cbdad55aca2c1293be018f53c0ab1d 5], [http://git.kernel.org/linus/eb348b898290da242e46df75ab0b9772003e08b8 6], [http://git.kernel.org/linus/2333210b26cf7aaf48d71343029afb860103d9f9 7], [http://git.kernel.org/linus/eb75767be0e514f97bf1b5cec763696cfc7f7e2a 8]

1.6. RAID5 multithreading

This release attempts to spread the work needed to handle raid 5 stripes to multiple CPUs in the MD ("multiple devices") layer, which allows more IO/sec on fast (SSD) devices.

Code: [http://git.kernel.org/linus/851c30c9badfc6b294c98e887624bff53644ad21 commit1], [http://git.kernel.org/linus/b721420e8719131896b009b11edbbd27d9b85e98 2]

1.7. Improved locking performance for virtualized guests

The operating system that runs in each virtualized guest also runs its own locks. With some locks, like spinning locks, this causes problems when many guests are present and keep spinning and wasting host CPU time and other problems. This release replaces paravirtualized spinlocks with paravirtualized ticket spinlocks, which have better performance properties for virtualized guests and brings speedups on various benchmarks.

Recommended paper: [http://www.xen.org/files/xensummitboston08/LHP.pdf Prevent Guests from Spinning Around]

Code: [http://git.kernel.org/linus/545ac13892ab391049a92108cf59a0d05de7e28c commit 1], [http://git.kernel.org/linus/92b75202e5e8790905f9441ccaea2456cc4621a5 2], [http://git.kernel.org/linus/80bd58fef495d000a02fc5b55ca76d423400e748 3], [http://git.kernel.org/linus/b8fa70b51aa76737bdb6b493901ef7376977489c 4]

1.8. New lockref locking scheme, VFS locking improvements

This release adds a new locking scheme, called "lockref". The "lockref" structure is a combination "spinlock and reference count" that allows optimized reference count accesses. In particular, it guarantees that the reference count will be updated as if the spinlock was held, but using atomic accesses that cover both the reference count and the spinlock words, it can often do the update without actually having to take the lock. This allows to avoid the nastiest cases of spinlock contention on large machines. When updating the reference counts on a large system, it will still end up with the cache line bouncing around, but that's much less noticeable than actually having to spin waiting for the lock. This release already uses lockref to improve the scalability of heavy pathname lookup in large systems.

Recommended LWN article: [https://lwn.net/Articles/565734/ Introducing lockrefs]

Code: [http://git.kernel.org/linus/bc08b449ee14ace4d869adaa1bb35a44ce68d775 commit 1], [http://git.kernel.org/linus/15570086b590a69d59183b08a7770e316cca20a7 2], [http://git.kernel.org/linus/df3d0bbcdb2cafa23a70223d806655bd37e64a9b 3], [http://git.kernel.org/linus/e7d33bb5ea82922e6ddcfc6b28a630b1a4ced071 4], [http://git.kernel.org/linus/232d2d60aa5469bb097f55728f65146bd49c1d25 5], [http://git.kernel.org/linus/71c7356f864dc41e4bd6b884596a422f8954afe0 6]

1.9. Better Out-Of-Memory handling

The Out-Of-Memory state happens when the computer runs out of RAM and swap memory. When Linux gets into this state, it kills a process in order to free memory. This release includes important changes to how the Out-Of-Memory states are handled, the number of out of memory errors sent to userspace and reliability. For more details see the below link.

Recommended LWN article: [https://lwn.net/Articles/562211/#oom Reliable out-of-memory handling]

Code: [http://git.kernel.org/linus/94bce453c78996cc4373d5da6cfabe07fcc6d9f9 commit 1], [http://git.kernel.org/linus/871341023c771ad233620b7a1fb3d9c7031c4e5c 2], [http://git.kernel.org/linus/759496ba6407c6994d6a5ce3a5e74937d7816208 3], [http://git.kernel.org/linus/3a13c4d761b4b979ba8767f42345fed3274991b0 4], [http://git.kernel.org/linus/519e52473ebe9db5cdef44670d5a97f1fd53d721 5], [http://git.kernel.org/linus/fb2a6fc56be66c169f8b80e07ed999ba453a2db2 6], [http://git.kernel.org/linus/3812c8c8f3953921ef18544110dafc3505c1ac62 7]

1.10. XFS directory recursion scalability, namespace support

XFS has added support for a directory entry file type, the purpose is that readdir can return the type of the inode the dirent points to to userspace without first having to read the inode off disk. Performance of directory recursion is much improved. Parallel walk of ~50 million directory entries across hundreds of directories improves significantly, from roughly 500 getdents() calls per second and 250,000 inode lookups per second to determine the inode type at roughly 17,000 read IOPS to 3500 getdents() calls per second at 16,000 IOPS, with no inode lookups at all.

This release has also added XFS support for namespaces, and has reincorporated defragmentation support for the new CRC filesystem format.

Code: [http://git.kernel.org/linus/0cb97766f2928579f1029ea7b28ae946cdd6fbe1 commit 1], [http://git.kernel.org/linus/1c55cece084aa4c9a3fa34ed4eecbdf18afbf05d 2], [http://git.kernel.org/linus/21b5c9784bceb8b8e0095f87355f3b138ebac2d0 3], [http://git.kernel.org/linus/d6970d4b726cea6d7a9bc4120814f95c09571fc3 4], [http://git.kernel.org/linus/638f44163d57f87d0905fbed7d54202beff916fc 5]

1.11. Improved tty layer locking

The tty layer locking got cleaned up and in the process a lot of locking became per-tty, which actually shows up on some (admittedly odd) loads.

Commits: [https://git.kernel.org/linus/2f01ea908bcf838e815c0124b579513dbda3b8c8 merge commit]

1.12. IPC locking improvements

This release includes improvements on the amount of contention we impose on the ipc lock (kern_ipc_perm.lock). These changes mostly deal with shared memory, previous work has already been done for [http://kernelnewbies.org/Linux_3.10#head-5c725e42ba8f05ed7767cfaf1c8e8c6546c4caec semaphores in 3.10] and [http://kernelnewbies.org/Linux_3.11#head-893b1abbed0138e9124ae9de6705c5c0b91edfe7 message queues in 3.11].

With these chanves, a custom shm microbenchmark stressing shmctl doing IPC_STAT with 4 threads a million times, reduces the execution time by 50%. A similar run, this time with IPC_SET, reduces the execution time from 3 mins and 35 secs to 27 seconds.

Code: [http://git.kernel.org/linus/4718787d1f626f45ddb239912bc07266b9880044 commit], [http://git.kernel.org/linus/8b8d52ac382b17a19906b930cd69e2edb0aca8ba 2], [http://git.kernel.org/linus/68eccc1dc345539d589ae78ee43b835c1a06a134 3], [http://git.kernel.org/linus/c97cb9ccab8c85428ec21eff690642ad2ce1fa8a 4], [http://git.kernel.org/linus/c2c737a0461e61a34676bd0bd1bc1a70a1b4e396 5], [http://git.kernel.org/linus/2caacaa82a51b78fc0c800e206473874094287ed 6], [http://git.kernel.org/linus/79ccf0f8c8e04e8b9eda6645ba0f63b0915a3075 7], [http://git.kernel.org/linus/7a25dd9e042b2b94202a67e5551112f4ac87285a 8], [http://git.kernel.org/linus/530fcd16d87cd2417c472a581ba5a1e501556c86 9], [http://git.kernel.org/linus/05603c44a7627793219b0bd9a7b236099dc9cd9d 10]

2. Drivers and architectures

All the driver and architecture-specific changes can be found in the [http://kernelnewbies.org/Linux_3.12-DriversArch Linux_3.12-DriversArch page]

3. Core

4. Memory management

5. Block layer

6. File systems

7. Networking

8. Crypto

9. Virtualization

10. Security

11. Tracing/perf

12. Other news sites that track the changes of this release

KernelNewbies: Linux_3.12 (last edited 2013-11-06 21:39:41 by diegocalleja)