KernelNewbies:

Linux 4.9 has been released on Sun, 11 Dec 2016.

Summary: This release adds support for shared extents (cp --reflink support) and copy-on-write support on XFS; virtually mapped kernel stacks that make the kernel more reliable and secure; a more efficient BPF profiler that brings Linux on part with Dtrace; a new optional BBR TCP congestion control algorithm based on bandwidth measurements instead of packet loss; syscalls to use the protection keys hardware feature; support for the Greybus bus from Project Ara; a hardware latency tracer to detect firmware-induced latencies, and many other improvements and new drivers.

1. Prominent features

1.1. Shared data extents + copy-on-write support on XFS

This release adds several key features to the XFS file system, based on the reverse mapping work introduced in the past release.

This release adds the ability to share data extents between different files. That is, support for cp --reflink=always, the ability to deduplicate data, and the ability to unshared data through the FALLOC_FL_UNSHARE fallocate(2) interface.

It also adds copy-on-write support for data: instead of overwriting data, it copies data to a new location.

All these features are a huge chunk of new experimental functionality with new on-disk format features and internal infrastructure.

Code: merge

1.2. Virtually mapped stacks for more secure and reliable stack handling

The Linux kernel has always mapped the memory used by kernel stacks directly in the kernel memory, an approach that makes harder to allocate stacks under memory load and proves no protection against stack overflows. This release allows to map the kernel stacks in virtual memory, which makes easier to allocate stacks under memory pressure, provides protection against stack overflows, and other benefits.

Recommended LWN articles: Virtually mapped kernel stacks, Virtually mapped stacks 2: thread_info strikes back

Code: commit, commit

1.3. Efficient BPF-based profiler

This release includes the necessary infrastructure to allow BPF programs to be atached to hardware and software perf events, which in turn allows the kernel to profile via timed sampling and summarize stack traces. For more details, see Brendan Gregg's blog post

Code: commit, commit, commit, commit, commit, commit

1.4. BBR TCP congestion control algorithm

This release adds another TCP congestion control algorithm: BBR (Bottleneck Bandwidth and RTT). The Internet has predominantly used loss-based congestion control, relying on packet loss as the signal to slow down. While this worked well for many years, loss-based congestion control is considered -by BBR authors- to be out-dated in today's networks. On today's Internet, loss-based congestion control causes the infamous bufferbloat problem, often causing seconds of needless queuing delay, since it fills the bloated buffers in many last-mile links. On today's high-speed long-haul links using commodity switches with shallow buffers, loss-based congestion control has abysmal throughput because it over-reacts to losses caused by transient traffic bursts.

BBR creates an explicit model of the network pipe by sequentially probing the bottleneck bandwidth and RTT. On the arrival of each ACK, BBR derives the current delivery rate of the last round trip, and feeds it through a windowed max-filter to estimate the bottleneck bandwidth. Conversely it uses a windowed min-filter to estimate the round trip propagation delay. The max-filtered bandwidth and min-filtered RTT estimates form BBR's model of the network pipe. BBR has significantly increased throughput and reduced latency for connections on Google's internal backbone networks and google.com and YouTube Web servers.

ACM Queue paper: BBR: Congestion-Based Congestion Control

Recommended LWN article: BBR congestion control

Code: commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

1.5. Protection keys syscall support

Protection keys is a memory protection hardware feature merged in Linux 4.6 (you can find more details in this LWN article). But in that release, the use of this feature was limited to the kernel automatically using it in high-level APIs, such as mmap(..., PROT_EXEC) and mprotect(ptr, sz, PROT_EXEC).

This release adds new syscalls that offer a more complete API to use protection keys. For more details, see the recommended LWN article: System calls for memory protection keys or read the documentation.

Code: commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

1.6. Greybus support

This release adds support for Greybus, a new subsystem that brings support for the Greybus bus: a bus designed for the modular, hot-pluggable hardware of the (now dead) Ara Project. Despite the dead of the main project, the code is still being used. For more details, see the recommended LWN article.

Recommended LWN article: Greybus

Code: drivers/staging/greybus

1.7. Hardware latency tracer

Coming from the real-time patchset, the hardware latency tracer is a special purpose tracer that is used to detect large system latencies induced by the behavior of certain underlying hardware or firmware interruptions, like SMIs (System Management Interrupts) on x86 systems, that the kernel is unaware of. The kernel does not even know these interruptions are happening. SMIs are instead set up by BIOS code and are serviced by BIOS code, usually for "critical" events such as management of thermal sensors and fans.

The hardware latency detector works by simply creating a thread that spins on a single CPU polling the CPU Time Stamp Counter for a specified amount of time (width) within a periodic window (window), and trying to find gaps where the polling was interrupted. This is useful for testing if a system is reliable for Real Time tasks.

Documentation: Documentation/trace/hwlat_detector.txt

Code: commit, commit, commit

2. Core (various)

3. File systems

4. Memory management

5. Block layer

6. Tracing and perf tool

7. Virtualization

8. Security

9. Networking

commit, commit, commit

10. Architectures

11. Drivers

11.1. Graphics

11.2. Storage

11.3. Staging

11.4. Networking

11.5. Audio

11.6. Tablets, touch screens, keyboards, mouses

11.7. TV tuners, webcams, video capturers

11.8. USB

11.9. Serial Peripheral Interface (SPI)

11.10. Watchdog

11.11. Serial

11.12. ACPI, EFI, cpufreq, thermal, Power Management

11.13. Real Time Clock (RTC)

11.14. PCI

11.15. Voltage, current regulators, power capping, power supply

11.16. Pin Controllers (pinctrl)

11.17. Multi Media Card

11.18. Industrial I/O (iio)

11.19. Multi Function Devices (MFD)

11.20. Pulse-Width Modulation (PWM)

11.21. Inter-Integrated Circuit (I2C)

11.22. Hardware monitoring (hwmon)

11.23. General Purpose I/O (gpio)

11.24. Clocks

11.25. Hardware Random Number Generator

11.26. Various

12. List of merges

13. Other news sites

KernelNewbies: Linux_4.9 (last edited 2017-12-30 01:30:26 by localhost)