Write-Optimized and Consistent RDMA-based NVM Systems (1906.08173v1)

Abstract: In order to deliver high performance in cloud computing, we generally exploit and leverage RDMA (Remote Direct Memory Access) in networking and NVM (Non-Volatile Memory) in end systems. Due to no involvement of CPU, one-sided RDMA becomes efficient to access the remote memory, and NVM technologies have the strengths of non-volatility, byte-addressability and DRAM-like latency. In order to achieve end-to-end high performance, many efforts aim to synergize one-sided RDMA and NVM. Due to the need to guarantee Remote Data Atomicity (RDA), we have to consume extra network round-trips, remote CPU participation and double NVM writes. In order to address these problems, we propose a zero-copy log-structured memory design for Efficient Remote Data Atomicity, called Erda. In Erda, clients directly transfer data to the destination address at servers via one-sided RDMA writes without redundant copy and remote CPU consumption. To detect the incompleteness of fetched data, we verify a checksum without client-server coordination. We further ensure metadata consistency by leveraging an 8-byte atomic update in the hash table, which also contains the address information for the stale data. When a failure occurs, the server properly restores to a consistent version. Experimental results show that compared with Redo Logging (a CPU involvement scheme) and Read After Write (a network dominant scheme), Erda reduces NVM writes approximately by 50%, as well as significantly improves throughput and decreases latency.

• The paper introduces Erda, a novel RDMA-NVM system that eliminates redundant overhead through zero-copy log-structured memory design.
• It employs checksum validation and atomic metadata updates to guarantee data consistency and efficient remote data atomicity.
• Experimental results show a 50% reduction in NVM writes and enhanced throughput and latency, proving scalability for cloud systems.

Overview of Write-Optimized and Consistent RDMA-based NVM Systems

The integration of Remote Direct Memory Access (RDMA) and Non-Volatile Memory (NVM) has been recognized as a promising strategy to enhance the performance of cloud computing systems. This paper investigates the challenges associated with efficiently ensuring Remote Data Atomicity (RDA) when employing RDMA to access NVM and introduces Erda, a novel solution that addresses these challenges while reducing overhead.

The primary objective of Erda is to mitigate the issues posed by high network overheads, CPU consumption, and double NVM writes, all of which are prevalent in existing RDMA-based NVM solutions. Conventional approaches that ensure RDA often involve extra RDMA operations or CPU participation, leading to inefficiencies. Erda eliminates these critical inefficiencies through a zero-copy log-structured memory design that effectively minimizes network round-trips and eliminates the need for remote CPU interaction.

Key Design Elements

1. Zero-Copy Log-Structured Memory Design: Erda capitalizes on the inherent characteristics of log-structured storage to leverage out-of-place updates. This design allows for direct data transfer from clients to the destination storage location on the server without intermediate copying, thereby reducing NVM write operations drastically compared to undo/redo logging and Copy-on-Write (COW) methods.
2. Checksum and Atomicity for Data Consistency: Each data object includes a CRC checksum to verify the integrity of reads, ensuring that potential inconsistencies due to incomplete writes are detected by clients. Moreover, an 8-byte atomic update operation is employed in the hash table metadata to facilitate efficient flipping between old and new versions of the data.
3. Log Cleaning and Scalability: Erda incorporates a lock-free log cleaning mechanism, enabling the reclamation of space used by outdated log segments. The system is also designed to accommodate increased memory requirements seamlessly by allocating new memory regions and managing them efficiently.

Experimental Evaluation

Erda's efficacy is validated through extensive experimentation, demonstrating a significant reduction in NVM writes by approximately 50% compared to existing methods like Redo Logging and Read After Write. The results indicate that Erda offers substantial improvements in throughput and latency, particularly in read-heavy workloads, without compromising on the CPU efficiency of the server. This reduction in CPU overhead is particularly advantageous, as RDMA operations in Erda largely bypass direct CPU intervention.

Implications and Future Directions

The introduction of Erda has notable implications for the design and deployment of RDMA-based NVM systems in data centers. By maintaining high throughput and low latency with diminished CPU involvement, Erda presents a compelling case for more widespread adoption of RDMA and NVM technology in achieving scalable and efficient memory management solutions. The solutions Erda offers align with the growing demand for efficient, high-performance computing within cloud environments.

Looking ahead, the continued evolution of RDMA and NVM technologies, including the advent of new materials and architecture paradigms, presents opportunities to further refine systems like Erda. Future research might explore optimal algorithmic strategies for even tighter integration of RDMA and NVM features, potentially drawing from emerging areas like machine learning optimization to adaptively manage memory access patterns.

In conclusion, Erda exemplifies an innovative approach to tackling the redundant complexities associated with ensuring data consistency under the RDMA-NVM paradigm. Its methodology highlights a direction for future exploration and standardization, potentially serving as a benchmark for the development of future high-performance, memory-dense computing systems.