The NIC should be part of the OS (2501.10138v2)

Abstract: The network interface adapter (NIC) is a critical component of a cloud server occupying a unique position. Not only is network performance vital to efficient operation of the machine, but unlike compute accelerators like GPUs, the network subsystem must react to unpredictable events like the arrival of a network packet and communicate with the appropriate application end point with minimal latency. Current approaches to server stacks navigate a trade-off between flexibility, efficiency, and performance: the fastest kernel-bypass approaches dedicate cores to applications, busy-wait on receive queues, etc. while more flexible approaches appropriate to more dynamic workload mixes incur much greater software overhead on the data path. However, we reject this trade-off, which we ascribe to an arbitrary (and sub-optimal) split in system state between the OS and the NIC. Instead, by exploiting the properties of cache-coherent interconnects and integrating the NIC closely with the OS kernel, we can achieve something surprising: performance for RPC workloads better than the fastest kernelbypass approaches without sacrificing the robustness and dynamic adaptation of kernel-based network subsystems.

• The paper presents the main contribution that integrating NIC into the OS reduces CPU cycle overhead by leveraging direct access to OS scheduling for network packet processing.
• It demonstrates that traditional NIC designs, which separate hardware responsibilities from the OS, lead to performance bottlenecks under dynamic workload conditions.
• Leveraging cache-coherent interconnects like CXL and CCIX, the approach enhances server efficiency and adaptability while lowering latency in network operations.

Integration of NIC into OS for Improved Networking Performance

The paper "The NIC should be part of the OS," authored by Pengcheng Xu and Timothy Roscoe, presents an innovative perspective on the architectural role of the Network Interface Card (NIC) in modern server systems. It challenges the conventional hardware/software boundary that has persisted in today’s networking paradigms and advocates for a closer integration of the NIC with the operating system. This integration aims to strike a balance between the performance of high-throughput, low-latency server workloads and the flexibility required for dynamically managing resources in diverse application scenarios.

Traditional NIC Design and Split Architecture

Traditionally, the NIC in servers has maintained a distinct separation between the hardware handling packet receptions and the operating system routing these packets to processes. The NIC typically uses Direct Memory Access (DMA) and descriptor rings to manage packet reception and relies on interrupts or busy-wait loops to alert the system and applications. This approach, while effective for static workload distributions, incurs limitations under dynamic conditions due to high software overheads and limited system adaptability.

Kernel-bypass techniques, such as those implemented in systems like Arrakis and IX, attempt to minimize OS intervention by allowing user-space applications to access network data directly. While these methods can boost performance by alleviating the kernel’s processing burdens, they maintain the traditional separation of NIC responsibilities and thus sacrifice flexibility and introduce scalability challenges.

Proposition for NIC-OS Integration

The paper posits that by eliminating the arbitrary split between NIC and OS responsibilities, and harnessing cache-coherent interconnect technologies like CXL and CCIX, the NIC can become a trusted extension of the OS. This paradigm shift would empower the NIC to handle steps in RPC processing, ranging from packet reading to direct function dispatch in CPUs, thus reducing software-induced latency.

The authors propose a model where the NIC has direct access to OS scheduling information, enabling it to make intelligent decisions about network packet demultiplexing and RPC dispatch, which substantially reduces CPU cycle overhead traditionally expended on these tasks. By possessing up-to-date OS scheduling state, the NIC can facilitate more informed load balancing, accommodating dynamic workload demands while maintaining high throughput and low latency for RPC workloads.

Potential Implications and Future Developments

The implications of integrating the NIC into the OS are twofold: practically, it offers a pathway to enhanced server efficiency and adaptability, enabling network infrastructure to better handle the complexity and scale of modern data center workloads. Theoretically, it raises questions about the traditional system architecture boundaries and presents a new outlook for designing system components that are more coherent and contextually aware of their operating environments.

Looking ahead, advancements in cache-coherent interconnect technology will likely drive further iterations of this concept, yielding even more integrated hardware-software configurations. Research could also explore the potential for such integrations in specialized processors like DPUs or TPUs, where network interface performance is increasingly critical.

In conclusion, this paper critiques existing paradigms of NIC and OS separation and provides a compelling argument and foundational framework for system architects to rethink the NIC’s role. Through prototyping efforts like Lauberhorn, which the authors are actively developing, the feasibility and advantages of this approach will become clearer, potentially inspiring further exploration and innovation in system design strategies.