Victima: Drastically Increasing Address Translation Reach by Leveraging Underutilized Cache Resources (2310.04158v3)

Abstract: Address translation is a performance bottleneck in data-intensive workloads due to large datasets and irregular access patterns that lead to frequent high-latency page table walks (PTWs). PTWs can be reduced by using (i) large hardware TLBs or (ii) large software-managed TLBs. Unfortunately, both solutions have significant drawbacks: increased access latency, power and area (for hardware TLBs), and costly memory accesses, the need for large contiguous memory blocks, and complex OS modifications (for software-managed TLBs). We present Victima, a new software-transparent mechanism that drastically increases the translation reach of the processor by leveraging the underutilized resources of the cache hierarchy. The key idea of Victima is to repurpose L2 cache blocks to store clusters of TLB entries, thereby providing an additional low-latency and high-capacity component that backs up the last-level TLB and thus reduces PTWs. Victima has two main components. First, a PTW cost predictor (PTW-CP) identifies costly-to-translate addresses based on the frequency and cost of the PTWs they lead to. Second, a TLB-aware cache replacement policy prioritizes keeping TLB entries in the cache hierarchy by considering (i) the translation pressure (e.g., last-level TLB miss rate) and (ii) the reuse characteristics of the TLB entries. Our evaluation results show that in native (virtualized) execution environments Victima improves average end-to-end application performance by 7.4% (28.7%) over the baseline four-level radix-tree-based page table design and by 6.2% (20.1%) over a state-of-the-art software-managed TLB, across 11 diverse data-intensive workloads. Victima (i) is effective in both native and virtualized environments, (ii) is completely transparent to application and system software, and (iii) incurs very small area and power overheads on a modern high-end CPU.

• The paper introduces Victima, which repurposes L2 cache blocks to store TLB entries, thereby significantly expanding address translation reach.
• It employs a PTW-Cost Predictor and adaptive TLB-aware cache replacement policy to optimize translation latency under varied workloads.
• Experimental results show up to 28.7% performance improvement in virtualized environments, highlighting its effectiveness without additional hardware.

Overview of "Victima: Drastically Increasing Address Translation Reach by Leveraging Underutilized Cache Resources"

The paper presents a novel approach to address a significant bottleneck in modern data-intensive workloads: address translation overhead due to frequent and long-latency page table walks (PTWs). The conventional multi-level translation lookaside buffer (TLB) hierarchy often struggles with large datasets, leading to high PTW latencies that degrade system performance. Victima is introduced as a software-transparent technique aimed at increasing the translation reach of the processor by utilizing the underutilized resources in the cache hierarchy.

Key Contributions

1. Translation Reach Expansion via Caches: The core idea of Victima is to re-purpose underutilized L2 cache blocks to store clusters of TLB entries. This method leverages the cache hierarchy as an additional low-latency and high-capacity component to back up the last-level TLB, thereby reducing PTWs without requiring additional costly TLB hardware.
2. Predictive Management with PTW-Cost Predictor (PTW-CP): Victima incorporates a PTW-Cost Predictor to identify pages that are costly to translate. This predictive mechanism uses a set of lightweight metrics to determine whether TLB entries should be stored in the L2 cache, thereby optimizing cache usage and avoiding unnecessary data eviction.
3. Adaptive Cache Replacement Policy: The system employs a TLB-aware cache replacement policy that adapts based on translation pressure and the potential reusability of TLB entries. This ensures that valuable cached application data is not displaced without significant benefits from TLB caching.
4. Seamless Integration in Modern Systems: Victima operates transparently without requiring changes to application or OS-level software, making it practical for integration in existing systems. It is compatible with large page mechanisms and is effective in both native and virtualized execution environments.

Technical Evaluation

Victima's design choices yield significant performance benefits across diverse data-intensive workloads. The experimental results show that in native execution environments, Victima improves application performance by 7.4% on average over the baseline system while achieving performance comparable to a hypothetical 128K-entry L2 TLB system without the associated overheads. In virtualized environments, performance improvements reach up to 28.7% compared to conventional nested paging.

The evaluation reveals a substantial reduction in L2 TLB miss latency and a significant increase in translation reach, highlighting the effectiveness of the approach in mitigating the translation overheads. The use of the underutilized cache resources for TLB storage emerges as a practical solution that can be easily adopted with minimal changes to existing architectures.

Implications and Future Directions

Victima provides a promising direction for addressing the persistent issue of translation bottlenecks in data-intensive computing. By effectively utilizing underutilized hardware resources, the approach minimizes the need for additional costly hardware extensions. Future research could explore deeper integration with dynamically adaptive systems, potentially expanding the predictive capabilities of Victima to adjust in real-time according to workload characteristics.

Additionally, further exploration into broader cache hierarchies and alternative architectures could yield even more pronounced improvements. The potential for Victima to be integrated into emerging architectures and virtualized systems is significant, and further work could enhance its scope and applicability.

In conclusion, Victima stands as a robust solution to a longstanding problem in computer systems, offering a path forward by leveraging existing resources more effectively. Its introduction paves the way for more efficient address translation, thereby supporting the intensifying demands of modern data-heavy applications.