Linux io_uring: High-Performance Async I/O

• Linux io_uring is a high-performance asynchronous I/O subsystem that leverages lock-free, shared ring buffers to enable efficient batching and zero-copy operations.
• It utilizes memory-mapped Submission and Completion Queues to bypass syscall and context-switch overhead, achieving up to 3–5× throughput gains in microbenchmarks.
• Advanced features like registered buffers, NVMe passthrough, and unified I/O support drive scalable, low-latency operations in database and networked systems.

Linux’s io_uring interface is a high-performance asynchronous I/O subsystem in the Linux kernel, designed to unlock the full potential of modern storage and network hardware by bypassing traditional syscall and context-switch overheads. It provides user space with direct, lock-free, shared-memory access to kernel-managed submission and completion queues, enabling efficient batching, zero-copy paths, and unified I/O across file, network, and device domains. io_uring is now essential for I/O-bound workloads requiring low-latency, high-throughput IOPS at scale, but it introduces considerable complexity in application architecture, synchronization, and resource lifecycle management (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).

1. Architectural Structure and Key Data Paths

io_uring centers around two ring buffers in shared memory: the Submission Queue (SQ) and Completion Queue (CQ). These are initialized by io_uring_setup(), which allocates both rings and an auxiliary array of Submission Queue Entries (SQEs). Applications map these buffers via three mmap regions: SQ ring metadata (head/tail/mask/flags), CQ ring metadata, and the physical array of SQEs (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025). After mmap, user code and the kernel access the rings using only atomic loads/stores, avoiding syscalls or futexes during normal operation.

Each io_uring_sqe contains at least an opcode (e.g., read, write, send), flags (including IOSQE_IO_LINK for request chaining or IOSQE_IO_DRAIN), target file descriptor, buffer address/length/offset, operation-specific arguments, and a 64-bit user_data field for correlation on completion. A io_uring_cqe holds the corresponding user_data, a result/status code, and flags (e.g., IORING_CQE_F_BUFFER) (Pestka et al., 25 Nov 2024).

Ring buffer slots are indexed modulo a power-of-two queue size. User space writes new SQEs at index sq_tail & sq_ring_mask, fills in the request fields, and atomically advances the tail counter. Kernel space picks up pending SQEs by atomically reading the head/tail bounds. Completions are processed symmetrically: the kernel writes CQEs, bumps cq_tail, and user space consumes completions by advancing cq_head (Jasny et al., 4 Dec 2025).

There are three kernel fulfillment paths: (1) inline completion (when a request can be satisfied instantly, e.g., socket read with available data), (2) native async completion (e.g., with block layer or non-blocking sockets), and (3) fallback io_worker threads for inherently blocking operations (fsync, legacy file operations, large DMA above hardware limits) (Jasny et al., 4 Dec 2025). io_uring can operate in DEFAULT, COOP_TASKRUN, or DEFER_TASKRUN modes to control when and how completions are delivered (synchronously, on unrelated syscalls, or only when explicitly polled) (Jasny et al., 4 Dec 2025).

2. Advanced Features and Unified API Innovations

io_uring offers advanced mechanisms that substantially reduce overhead for hot-path I/O:

• Registered Buffers (“fixed buffers”): Applications use io_uring_register() to pin user space buffers, allowing the kernel and device to perform DMA without per-I/O page pinning, VM lookups, or memcpy. This eliminates significant CPU and memory traffic but requires all registered buffers to be managed (unpinned) manually by the application, which must guarantee no inflight I/O references when releasing memory (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).
• Passthrough I/O (NVMe OP_URING_CMD): Using the OP_URING_CMD opcode, applications can directly issue NVMe admin or I/O commands to the hardware’s native queues, bypassing the Linux block layer. This yields 20–30% additional throughput in storage-bound workloads but mandates raw device access, O_DIRECT buffers, and strict error management (Jasny et al., 4 Dec 2025).
• Unified storage and network API: io_uring subsumes the roles of traditional interfaces like epoll, libaio, and eventfd, unifying file I/O, scatter/gather network I/O, syscall wrappers (madvise, stat), and device commands within one submission/completion ring. For high-level systems such as DBMSs, this allows overlapping, batched disk and network requests without context switches or API bridging (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).

3. Performance Characteristics and Microbenchmark Data

The syscall-elimination and ring-based model provides substantial performance benefits:

• In microbenchmarks, io_uring achieves 3–5× better throughput than libaio in 4 KiB random-read workloads on a single thread and approaches within 10% of userland SPDK stack (Pestka et al., 25 Nov 2024). Figure 1 in (Pestka et al., 25 Nov 2024) shows that SQPOLL + IOPOLL enables a single thread to saturate over 1 million IOPS on a PCIe 5.0 SSD at queue depth 1, in contrast to blocking read() performance of a few hundred thousand IOPS.
• Latency is modeled as:

$$L_{\mathrm{io\_uring}} \approx L_{\mathrm{atomic}} + L_{\mathrm{device}}$$

$$L_{\mathrm{blocking}} \approx L_{\mathrm{syscall}} + L_{\mathrm{context\_switch}} + L_{\mathrm{device}}$$

With $$L_{\mathrm{atomic}} \ll L_{\mathrm{syscall}} + L_{\mathrm{context\_switch}}$$, device latency (10–20 μs for SSDs) dominates (Pestka et al., 25 Nov 2024).

• Stepwise integration in a DBMS buffer manager on a 3.7 GHz CPU and PCIe 5 NVMe yields the following throughput advances (in thousands of transactions/sec):

| Configuration | Throughput (k tx/s) | |------------------------- |--------------------| | Posix / libaio | 16.5 | | + batched writeback | 19.0 | | + async execution | 183.0 | | + batched reads | 216.0 | | + registered buffers | 238.0 | | + NVMe passthrough | 300.0 | | + IOPOLL | 376.0 | | + SQPoll | 546.5 |

(Jasny et al., 4 Dec 2025)

• In network-bound analytical shuffle using 400 Gb/s NICs and batched 1 MiB sends, speedup over baseline epoll is up to 2.5× for large tuples when using zero-copy send/recv. Peak throughput measured at 50 GiB/s per node, saturating the link; zero-copy halves memory bandwidth usage per byte (Jasny et al., 4 Dec 2025).

4. Complexity, Synchronization, and Resource Management

Full exploitation of io_uring introduces considerable complexity:

• Synchronization: The SQ/CQ rings are thread-safe against the kernel, but user space must serialize tail/head advances when sharing a ring among threads, typically via locks, atomics, or fetch-and-add. Linux’s memory ordering must be respected, leveraging provided memory barriers (Pestka et al., 25 Nov 2024).
• Timeouts and error handling: Timeouts utilize special SQEs (IORING_OP_TIMEOUT). Failed requests (−ETIME, −EAGAIN, −ECANCELED) are communicated as CQEs with negative res. Ring overflows and out-of-order completions from linked chains require defensive handling (Pestka et al., 25 Nov 2024).
• Buffer and file registration: Registered buffers/fds remove per-request lookups but require precise lifecycle management. Applications must keep page or buffer pools pinned and avoid unregistration while in-flight I/O still touches them (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).
• Worker thread fallback: Operations that cannot complete asynchronously (e.g., fsync, large DMA) trigger kernel io_workers, incurring 7–30 μs per operation penalty, so all critical DBMS I/O paths must be made async (Jasny et al., 4 Dec 2025).
• Threaded architecture: Options include inline submission/polling, static or dynamic thread pools each with their own rings, and kernel-level SQPOLL/IOPOLL enablement. Synchronous inline work, even a few microseconds, can collapse IOPS by orders of magnitude, so partitioning compute and I/O handling is foundational (Pestka et al., 25 Nov 2024).

5. Application in Database and Networking Systems

io_uring’s strongest adoption and most cited benefits appear in the design of I/O-bound database servers and high-throughput networked systems:

• Storage-bound buffer managers: Batching, registered buffers, NVMe passthrough, and kernel polling (SQPOLL, IOPOLL) offer stepwise throughput improvements, as quantified previously (Jasny et al., 4 Dec 2025).
• Networked analytical workload shuffling: Unified ring interface and zero-copy send/recv deliver linear scaling up to link saturation, significantly surpassing epoll-based designs (Jasny et al., 4 Dec 2025).
• PostgreSQL case paper: Systematic integration—replacing pread/pwrite/fsync with io_uring submissions, registering the buffer pool, enabling DEFER_TASKRUN, and (optionally) SQPoll for consolidated background submission—surfaced an 11–15% throughput improvement on cold scans over baseline PostgreSQL 18 (Jasny et al., 4 Dec 2025).

Practical integration steps for DBMSs include maintaining one ring per worker thread, isolating critical path durability (fsync) from async code, and tuning buffer registration and polling modes.

6. Guidelines, Pitfalls, and Trade-offs

The effectiveness of io_uring depends on architectural and workload characteristics:

When to use io_uring:

• Storage or network-bound workloads with high I/O concurrency and opportunity for batching, significant syscall or kernel bottlenecks, or constrained host memory bandwidth (Jasny et al., 4 Dec 2025).

Integration and tuning:

• Architect full asynchrony and batching at all levels (event loops, buffer managers).
• One ring per thread is preferred; avoid cross-process contention when using DEFER_TASKRUN.
• Register hot buffers or large, frequently touched regions at initialization.
• Employ NVMe passthrough and zero-copy selectively, subject to device and file system constraints (Jasny et al., 4 Dec 2025).

Polling and submission:

• Choose DEFER_TASKRUN for low-jitter, high-throughput applications. SQPoll further eliminates syscalls at the cost of pinned CPU cores. Excessive batching increases tail latency.
• Model workload and hardware I/O limitations beforehand to avoid unnecessary complexity.

Pitfalls to avoid:

• Treating io_uring as a drop-in replacement for libaio/epoll may lead to negligible or negative impact on non-I/O-bound or latency-insensitive workloads.
• Neglected buffer/file lifecycle management can lead to data corruption or leaks.
• Over-batching or unregistered large I/Os trigger overhead from kernel io_workers, increasing completion latency (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).

7. Summary

Linux io_uring represents an asynchronous, memory-mapped, lockless, and unified I/O interface delivering substantial throughput and latency improvements for I/O-centric systems—particularly those that can batch requests, register hot buffers, and manage ring, buffer, and thread resources carefully. Integrating io_uring into high-performance data systems can effectuate order-of-magnitude gains in IOPS, I/O bandwidth, and resource efficiency, but requires “great responsibility” in system design, error handling, and synchronization to avoid subtle performance or correctness pitfalls (Pestka et al., 25 Nov 2024, Jasny et al., 4 Dec 2025).