KernelNewbies:

Summary: Besides a new version numbering scheme, Linux 3.0 also has several new features: Btrfs data scrubbing and automatic defragmentation, XEN Dom0 support, unprivileged ICMP_ECHO, wake on WLAN, Berkeley Packet Filter JIT filtering, a memcached-like system for the page cache, a sendmmsg() syscall that batches sendmsg() calls and setns(), a syscall that allows better handling of light virtualization systems such as containers. New hardware support has been added: for example, Microsoft Kinect, AMD Llano Fusion APUs, Intel iwlwifi 105 and 135, Intel C600 serial-attached-scsi controller, Ralink RT5370 USB, several realtek rtl81xx devices or the Apple iSight webcam. Many other drivers and small improvements have been added.

TableOfContents()

1. Prominent features

1.1. Btrfs: Automatic defragmentation, scrubbing, performance improvements

Automatic defragmentation

COW (copy-on-write) filesystems have many advantages, but they also have some disadvantages, for example fragmentation. Btrfs lays out the data sequentially when files are written to the disk for first time, but a COW design implies that any subsequent modification to the file must not be written on top of the old data, but be placed in a free block, which will cause fragmentation (RPM databases are a common case of this problem). Aditionally, it suffers the fragmentation problems common to all filesystems.

Btrfs already offers alternatives to fight this problem: First, it supports online defragmentation using the command "btrfs filesystem defragment". Second, it has a mount option, -o nodatacow, that disables COW for data. Now btrfs adds a third option, the -o autodefrag mount option. This mechanism detects small random writes into files and queues them up for an automatic defrag process, so the filesystem will defragment itself while it's used. It isn't suited to virtualization or big database workloads yet, but works well for smaller files such as rpm, SQLite or bdb databases. Code: [http://git.kernel.org/linus/4cb5300bc839b8a943eb19c9f27f25470e22d0ca (commit)]

Scrub

"Scrubbing" is the process of checking the integrity of the data in the filesystem. This initial implementation of scrubbing will check the checksums of all the extents in the filesystem. If an error occurs (checksum or IO error), a good copy is searched for. If one is found, the bad copy will be rewritten. Code: [http://git.kernel.org/linus/a2de733c78fa7af51ba9670482fa7d392aa67c57 (commit 1], [http://git.kernel.org/linus/475f63874d739d7842a56da94687f18d583ae654 2)]

Other improvements

-File creation/deletion speedup: The performance of file creation and deletion on btrfs was very poor. The reason is that for each creation or deletion, btrfs must do a lot of b+ tree insertions, such as inode item, directory name item, directory name index and so on. Now btrfs can do some delayed b+ tree insertions or deletions, which allows to batch these modifications. Microbenchmarks of file creation have been speed up by ~15%, and file deletion by ~20%. Code: [http://git.kernel.org/linus/16cdcec736cd214350cdb591bf1091f8beedefa0 (commit)]

-Do not flush csum items of unchanged file data: speeds up fsync. A sysbench workload doing "random write + fsync" went from 112.75 requests/sec to 1216 requests/sec. Code: [http://git.kernel.org/linus/8e531cdfeb75269c6c5aae33651cca39707848da (commit)]

-Quasi-round-robin for space allocation in multidevice setups: the chunk allocator currently always allocates space on the devices in the same order. This leads to a very uneven distribution, especially with RAID1 or RAID10 and an uneven number of devices. Now Btrfs always sorts the devices before allocating, and allocates the stripes on the devices with the most available space. Code: [http://git.kernel.org/linus/73c5de0051533cbdf2bb656586c3eb21a475aa7d (commit)]

1.2. sendmmsg(): batching of sendmsg() calls

Recvmsg() and sendmsg() are the syscalls used to receive/send data to the network. In 2.6.33, Linux [http://kernelnewbies.org/Linux_2_6_33#head-346ce08dcffcf216d56ef0cbf19f3702ed2fb493 added recvmmsg()], a syscall that allows to receive in a single call data that would need multiple recvmsg() calls, improving throughput and latency for a number of scenarios. Now, a equivalent sendmmsg() syscall has been added. A microbenchmark saw a 20% improvement in throughput on UDP send and 30% on raw socket send

Code: [http://git.kernel.org/linus/228e548e602061b08ee8e8966f567c12aa079682 (commit)]

1.3. XEN dom0 support

Finally, Linux has got [http://blog.xen.org/index.php/2011/06/14/linux-3-0-how-did-we-get-initial-domain-dom0-support-there/ Xen dom0 support]

1.4. Cleancache

Recommended LWN article: [http://lwn.net/Articles/386090/ Cleancache and Frontswap]

Cleancache is an optional feature that can potentially increases page cache performance. It could be described as a memcached-like system, but for cache memory pages. It provides memory storage not directly accessible or addressable by the kernel, and it does not guarantee that the data will not vanish. It can be used by virtualization software to improve memory handling for guests, but it can also be useful to implement things like a compressed cache.

Code: [http://git.kernel.org/linus/4fe4746ab694690af9f2ccb80184f5c575917c7f (commit)], [http://git.kernel.org/linus/077b1f83a69d94f2918630a882d74939baca0bce (commit)]

1.5. Berkeley Packet Filter just-in-time filtering

Recommended LWN article: [https://lwn.net/Articles/437981/ A JIT for packet filters]

The Berkeley Packet Filter filtering capabilities, used by tools like libpcap/tcpdump, are normally handled by an interpreter. This release adds a simple JIT that generates native code when filter is loaded in memory (something already done by other OSes, like [http://www.freebsd.org/releases/7.0R/relnotes.html#NET-PROTO FreeBSD]). Admin need to enable this feature writting "1" to /proc/sys/net/core/bpf_jit_enable

Code: [http://git.kernel.org/linus/0a14842f5a3c0e88a1e59fac5c3025db39721f74 (commit)]

1.6. Wake on WLAN support

Wake on Wireless is a feature to allow the system to go into a low-power state (e.g. ACPI S3 suspend) while the wireless NIC remains active and does varying things for the host, e.g. staying connected to an AP or searching for networks. The 802.11 stack has added support for it.

Code: [http://git.kernel.org/linus/eecc48000afe2ca6da22122d553b7cad294e42fc (commit 1], [http://git.kernel.org/linus/ff1b6e69ad4f31fb3c9c6da2665655f2e798dd70 2)]

1.7. Unprivileged ICMP_ECHO messages

Recommended LWN article: [https://lwn.net/Articles/420799/ ICMP sockets]

This release makes it possible to send ICMP_ECHO messages (ping) and receive the corresponding ICMP_ECHOREPLY messages without any special privileges, similar to what is implemented [http://www.manpagez.com/man/4/icmp/ in Mac OS X]. In other words, the patch makes it possible to implement setuid-less and CAP_NET_RAW-less /bin/ping. Initially this functionality was written for Linux 2.4.32, but unfortunately it was never made public. The new functionality is disabled by default, and is enabled at bootup by supporting Linux distributions, optionally with restriction to a group or a group range.

Code: [http://git.kernel.org/linus/c319b4d76b9e583a5d88d6bf190e079c4e43213d (commit)]

1.8. setns() syscall: better namespace handling

Recommended LWN article: [https://lwn.net/Articles/407495/ Namespace file descriptors]

Linux supports different namespaces for many of the resources its handles; for example, lightweight forms of virtualization such as [http://en.wikipedia.org/wiki/Operating_system-level_virtualization containers] or [http://0pointer.de/blog/projects/changing-roots.html systemd-nspaw] show to the virtualized processes a virtual PID different from the real PID. The same thing can be done with the filesystem directory structure, network resources, IPC, etc. The only way to set different namespace configurations was using different flags in the clone() syscall, but that system didn't do things like allow to one processes to access to other process' namespace. The setns() syscall solves that problem-

Code: [http://git.kernel.org/linus/6b4e306aa3dc94a0545eb9279475b1ab6209a31f (commit 1], [http://git.kernel.org/linus/13b6f57623bc485e116344fe91fbcb29f149242b 2], [http://git.kernel.org/linus/0663c6f8fa37d777ede74ff991a0cba3a42fcbd7 3], [http://git.kernel.org/linus/34482e89a5218f0f9317abf1cfba3bb38b5c29dd 4], [http://git.kernel.org/linus/a00eaf11a223c63fbb212369d6db69ce4c55a2d1 5], [http://git.kernel.org/linus/7b21fddd087678a70ad64afc0f632e0f1071b092 6)]

1.9. Alarm-timers

Recommended LWN article: [https://lwn.net/Articles/429925/ Waking systems from suspend]

Alarm-timers are a hybrid style timer, similar to high-resolution timers, but when the system is suspended, the RTC device is set to fire and wake the system for when the soonest alarm-timer expires. The concept for Alarm-timers was inspired by the Android Alarm driver, and the interface to userland uses the POSIX clock and timers interface, using two new clockids:CLOCK_REALTIME_ALARM and CLOCK_BOOTTIME_ALARM.

Code: [http://git.kernel.org/linus/ff3ead96d17f47ee70c294a5cc2cce9b61e82f0f (commit 1], [http://git.kernel.org/linus/9a7adcf5c6dea63d2e47e6f6d2f7a6c9f48b9337 2)]

2. Driver and architecture-specific changes

All the driver and architecture-specific changes can be found in the [http://kernelnewbies.org/Linux_3.0_DriverArch Linux_3.0_DriverArch page]

3. VFS

4. Process scheduler

5. Memory management

6. Networking

7. File systems

BLOCK LAYER

EXT4

CIFS

OCFS2

NILFS2

XFS

8. Crypto

9. Virtualization

10. Security

11. Tracing/profiling

12. Various core changes


CategoryReleases

KernelNewbies: Linux_3.0 (last edited 2011-07-22 13:26:12 by Morot)