Linux 3.15 has been released on Sun, 8 Jun

Summary: This release resumes much faster in systems with hard disks, it adds support for cross-renaming two files atomically, it adds new fallocate(2) modes that allow to remove the range of a file or set it to zero, it adds a new file locking API, the memory management adapts better to working set size changes, it improves FUSE write performance, it adds support for the LZ4 algorithm in zram, it allows to load 64-bit kernels from 32-bit EFI firmware, it adds support for AVX-512 vector instructions that will be added in upcoming Intel CPUs, and it adds new drivers and many other small improvements.

1. Prominent features

1.1. Faster resume from power suspend in systems with hard disk drives

Resuming a system from suspend used to take a long time in systems with traditional hard disk drives, because the system blocks the resume process until the hard disk drive finish powering up. In this release, commands are sent to the hard disk asynchronously, so the entire resuming process isn't paused by the hard disk. The end result is that systems with hard disks will resume several seconds faster with this Linux release.

For more details, see this blog entry

Code: commit 1, 2

1.2. Improved working set size detection

When there is not enough room for all memory in RAM, the Linux kernel is in charge of deciding which memory must be kept in RAM, and which must be sent to swap or discarded. In order to make good decisions, it is necessary to track which memory is most used and deserves to be kept in RAM, and which memory is not used often and can be evicted. The way the Linux kernel does this is by keeping an "inactive" and "active" list, when some data needs to be moved to RAM its memory is marked as active. As more and more memory gets used, the active list gets filled and the less used memory is moved to the inactive list.

The problem with this algorithm is to know how big must be each list. Linux used to have a policy of not allowing the active list to grow larger than the inactive, but this approach caused problems. In this release, Linux does more advanced tracking of how memory gets used and can balance better the size of the lists, which makes Linux perform better in certain workloads, adapt better to workload size changes, and creates a foundation to build improved policies in the future.

For more details, read this recommended link: Better active/inactive list balancing

commit 1, 2, 3, 4, 5

1.3. EFI 64-bit kernels can be booted from 32-bit firmware

Most modern x86 CPUs are 64-bit, but many modern systems ship with a 32-bit EFI implementation. This didn't allow to boot a Linux 64-bit EFI kernel from these 32-bit EFI systems. This limitation has been removed, a 64-bit kernel can be booted on 32-bit firmware that runs on 64-bit CPUs (note that it is not possible to boot a mixed-mode enabled kernel via the EFI boot stub - a bootloader that supports the EFI handover protocol must be used)

Code: commit 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

1.4. New file locking scheme: open file description locks

Due to some unfortunate history, POSIX locks have very strange and unhelpful semantics: they are dropped whenever the process closes any file descriptor associated with the inode, and locks taken between threads within the same process won't conflict with one another, which renders them useless for synchronization between threads.

This release adds a new type of lock that attempts to address these issues: open file description locks (initially called "file-private locks"). These locks will conflict with classic POSIX read/write locks, but have semantics that are more like BSD locks with respect to inheritance and lock release when closing a file descriptor.

For more documentation and details about the new locking API, read this recommended LWN link: File-private POSIX locks

Code: commit

1.5. Faster erasing and zeroing of parts of a file

This release adds two new fallocate(2) mode flags:

• FALLOC_FL_COLLAPSE_RANGE: Allows to remove a range of a file without leaving holes, improving the performance of these operations that previously needed to be done with workarounds.
• FALLOC_FL_ZERO_RANGE: Allows to set a range of a file to zero, much faster than it would take to do it manually (this functionality was previously available in XFS through the XFS_IOC_ZERO_RANGE ioctl)

In this release, only XFS and ext4 have added support for these new flags, other filesystems will follow in the future.

For more details, read this LWN article: Finding the proper scope of a file collapse operation

Code: commit 1, 2, 3, 4, 5, 6

1.6. File cross-renaming support

This release adds cross-rename, a variant of rename which exchanges the two files. This allows interesting use cases which were not possible before, for example atomically replacing a directory tree with a symlink. It also allows overlayfs and friends to operate on whiteouts atomically.

For more details, read this LWN article: Exchanging two files

Code: commit, commit, commit

1.7. zram: LZ4 compression support, improved performance

Zram is a memory compression mechanism added in Linux 3.14 that is used in Android, Cyanogenmod, Chrome OS, Lubuntu and other projects. In this release zram brings support for the LZ4 compression algorithm, which is better than the current available LZO in some cases.

This release also adds performance improvements to concurrent compression of multiple compression streams, and the ability to switch the compression algorithm in /sys/block/zram0/comp_algorithm

Code: commit 1, 2, 3, 4

1.8. Intel AVX-512 vector instructions support

AVX-512 are 512-bit extensions to the 256-bit Advanced Vector Extensions SIMD instructions for x86 instruction set architecture proposed by Intel, and scheduled to be supported in 2015 with Intel's Knights Landing processor.

For more details about these extensions, read the documentation.

Code: commit

1.9. FUSE: improved write performance

FUSE can now use cached writeback support to fuse, which improves write throughput.

Code: commit

2. Drivers and architectures

All the driver and architecture-specific changes can be found in the Linux_3.15-DriversArch page

3. Core

• See the "prominent features" section to find more information about other new core features.

• Add generic support for CPU feature based module autoloading commit

• Introduce cancelable MCS lock, it is a simple spinlock with the desirable properties of being fair, and with each CPU trying to acquire the lock spinning on a local variable. It avoids expensive cache bouncings that common test-and-set spinlock implementations incur commit

• Permit disabling the obsolete uselib commit and sys_sysfs syscalls, no longer used by libc commit

• NUMA: make smarter decisions on NUMA migrations in order to maximize performance of workloads that do not fit in one NUMA node commit

• Rework sysfs layout for memcg caches commit

• tools/vm/page-types.c: 'page-types' can walk over a file's mappings and analyze populated page cache pages mostly without disturbing its state commit

• Show mnt_id in /proc/pid/fdinfo commit

• Make /proc/*/pagemap 0400 commit

• Make /proc/*/{stack,syscall,personality} 0400 commit

• Add a lock-torture kernel module commit

4. Memory management

• As mentioned in the "prominent features" section, improved working set size detection

• hugetlb: improve page-fault scalability. The kernel could only handle a single hugetlb page fault at a time. This release allows a better chance of parallelization. This releases reduces the startup time of a 10 Gb Oracle database (with ~5000 faults) from 37.5 seconds to 25.7 seconds compared with previous kernels. Larger workloads will benefit even more commit

• Per-thread VMA caching; cache last recently used VMA to improve VMA cache hit rate, for more details see the recommended LWN article Optimizing VMA caching. commit

• Introduce byte-sized index for the freelist of a slab; microoptimizes some microbenchmarks commit

• zswap: support multiple swap devices commit

• "opportunistic fault around"; for more details see the bottom of this page commit

1b93d471bca002bd849 commit]

5. Power management

• As mentioned in the "prominent features" section, faster resume from power suspend in systems with hard disk drives

• Speed up resume by resuming runtime-suspended devices later during system suspend commit, commit

• Speed up resume by using asynchronous threads for resume_early commit, resume_noirq commit, suspend_late commit, suspend_noirq commit, acpi_thermal_check commit

• tools/power turbostat: Run on Intel Broadwell commit

6. Block layer

• UBI: Read-only block driver on top of UBI volumes commit

• Device Mapper: add dm-era target. dm-era is a target that behaves similar to the linear target, and in addition it keeps track of which blocks were written within a user-defined period of time called an 'era'. Use cases include tracking changed blocks for backup software, and partially invalidating the contents of a cache to restore cache coherency after rolling back a vendor snapshot commit

7. File systems

• Btrfs

  • Allow mounting different Btrfs subvolumes with different ro/rw options commit

  • Don't bother compress writes smaller than a block that are not inlined commit

  • Less lock contention when using autodefrag commit

  • Replace the Btrfs async threads with regular kernel workqueues commit

  • Add simple debugfs interface commit

• ext3

  • Speedup sync(2) commit

• ext4

  • Speedup sync(2) commit

  • jbd2: scalability improvements commit

• xfs

  • v5 format is now considered stable commit

  • Add O_TMPFILE support commit

  • Allow appending aio writes commit

• f2fs

  • introduce large directory support commit, commit

  • throttle the memory footprint with a sysfs entry commit

• nilfs2

  • Add FITRIM ioctl support for nilfs2 commit

  • Implementation of NILFS_IOCTL_SET_SUINFO ioctl. With this ioctl the segment usage entries in the SUFILE can be updated from userspace commit

• GFS2

  • Add meta readahead field in directory entries commit

• affs

  • add mount option to avoid filename truncates commit

8. Networking

• Bluetooth

  • Enable Secure Connection commit

  • Security Level 4, a new strong security requirement that is based around 128-bit equivalent strength for link and encryption keys required using FIPS approved algorithms. Which means that E0, SAFER+ and P-192 are not allowed. Only connections created with P-256 resulting from using Secure Connections support are allowed commit, commit, commit

  • Add Secure Connection Only Mode commit

  • Add management command to use of debug keys commit, commit

  • Enable LE L2CAP Connection oriented Channel support by default commit

  • Add support for Set Privacy command commit

  • Add support for handling signature resolving keys commit

  • Introduce LE auto connection support commit, commit

• wireless

  • Add NAPI support back commit

  • Remove NL80211_FEATURE_P2P_DEVICE_NEEDS_CHANNEL [https://git.kernel.org/linus

  • hwsim: add channel switch support commit

  • hwsim: make P2P-Device support optional commit

• Devices: add busy_poll feature to allow finding out if a device supports busy polling commit

• af_rxrpc: Add sysctls for configuring RxRPC parameters commit

• BATMAN protocol

  • Add IPv4 link-local/IPv6-ll-all-nodes multicast support commit

  • Multicast Listener Announcements via Translation Table commit, commit

• bonding: support QinQ for bond arp interval commit, commit

• ieee802154: support rf212 and extended mac features commit

• ipsec: add support of limited SA dump commit

• New IP_MTU_DISCOVER option IP_PMTUDISC_OMIT commit, new IPV6_MTU_DISCOVER option IPV6_PMTUDISC_OMIT commit

• netfilter

  • connlimit: use keyed locks for improved performance commit

  • connlimit: use rbtree for per-host conntrack obj storage, for improved performance commit

  • conntrack: remove central spinlock nf_conntrack_lock for improved performance commit

  • ipset: add forceadd kernel support for hash set types commit

  • ipset: add hash:ip,mark data type to ipset commit

  • ipset: add markmask for hash:ip,mark data type commit

  • nf_tables: accept QUEUE/DROP verdict parameters commit

  • nf_tables: add optional user data area to rules commit

• loopback: sctp: add NETIF_F_SCTP_CSUM to device features commit

• tcp: snmp stats for Fast Open, SYN rtx, and data pkts commit

• Add IPsec protocol multiplexer commit, commit

• vti4/vti6: Support inter address family tunneling. With this patch we can tunnel ipv4/6 traffic via a vti6/4 interface commit, commit

9. Virtualization

• KVM

  • Allow per-vm capability enablement commit

  • PPC: Book3S HV: Add transactional memory support commit

  • x86: Add nested virtualization support for MPX commit

• Hyper-V

  • Implement the file copy service for Linux guests on Hyper-V (related documentation) commit

  • network: Enable receive side IP checksum offload, send side checksum offload and large send offload commit, commit, commit, commit

  • network: Enable scatter gather I/O commit

  • hyperv-fb: add support for generation 2 virtual machines. commit

• Xen

  • xen-netback: Add stat counters for zerocopy commit

  • xen-netback: Introduce TX grant mapping commit

  • Add support for MSI message groups commit

• vfio: Multi-IOMMU domain support commit

10. Security

• audit: Allow login in non-init namespaces commit

• audit: add compat syscall audit support commit

• audit: audit /proc/<pid>/cmdline aka proctitle commit

• Add driver for Intel Trusted Execution Environment with ME Interface commit

11. Crypto

• caam - add support for aead null encryption commit

• omap-des - Add omap-des driver for OMAP4/AM43xx commit

• sha - SHA1 transform x86_64 AVX2 commit

• tegra - remove driver commit

12. Tracing/perf

• Currently the tracers (function, function_graph, irqsoff, etc) can only be used by the top-level tracing directory (not for instances). Allow instances to be able to run a separate tracer apart from the what the top-level tracing is doing commit, commit, commit

• uprobes: Add support for event triggering commit

• perf

  • bench: Add futex-hash microbenchmark commit, add futex-requeue microbenchmark commit, add futex-wake microbenchmark commit

  • kvm: introduce --list-cmds for use by scripts commit

  • probe: Show in what binaries/modules probes are set commit

  • Support distro-style debuginfo for uprobe commit

  • Add call-graph option support into .perfconfig commit

  • Disable user-space callchain/stack dumps for function trace events commit, commit

  • Disallow user-space callchains for function trace events commit

  • Disallow user-space stack dumps for function trace events commit

  • Add trace_clock=<clock> kernel parameter commit

13. Other news sites that track the changes of this release

• LWN Merge window, part 1 and part 2

• Phoronix The Top Features Of The Linux 3.15 Kernel