Reverse Address Translation in Multi-GPU Systems

• Reverse address translation is a hardware mechanism that maps external addresses (e.g., NPAs or PFNs) to local addresses (SPAs or virtual-page indices), crucial for resource isolation and memory operations.
• It employs multi-level translation pipelines with dedicated TLBs and RLUTs to minimize latency, achieving up to 90% recovery of cold miss penalties in GPU cluster communications.
• Optimization strategies, including fused pre-translation and software-guided TLB prefetching, balance performance against resource overhead in both multi-GPU interconnects and VIVT cache designs.

Reverse address translation refers to hardware or architectural mechanisms that map a hardware-visible address or identifier—such as a Network Physical Address (NPA) or Physical Frame Number (PFN)—back to a context-specific or local address such as a System Physical Address (SPA) or a set of virtual-page indices. These mechanisms are fundamental to several subsystems in contemporary high-performance architectures, most prominently in large-scale multi-GPU collectives and virtually indexed, virtually tagged (VIVT) caches. In contrast to canonical address translation (virtual-to-physical), reverse address translation resolves an externally visible or coherence-level hardware address back to the set of virtual or local identities necessary for access control, memory operations, or coherence.

1. Mechanisms in Multi-GPU Scale-Up Fabrics

In distributed GPU clusters employing fabrics such as NVLink or UALink, reverse address translation is deployed at the destination GPU to resolve incoming NPAs—generated by a source GPU’s MMU page-table walk—into SPAs meaningful to the local DRAM controller and protection domain (Fatima et al., 2 Apr 2026). An NPA encodes results from the source virtual address space but is opaque to the destination's OS and driver context. The reverse lookup is essential for both correct routing within local DRAM subsystems and for enforcing isolation across OS and hardware boundaries.

In such systems, the translation pipeline comprises:

• Extraction of the NPA from the packet header at the destination GPU’s UALink station.
• Lookup in a private L1 Link TLB (32-entry, fully associative, 50 ns hit latency).
• On miss, access to a shared L2 Link TLB (512-entry, 2-way set associative, 100 ns hit latency) using a Least Recently Used (LRU) policy.
• Further miss resolution via a multi-level page-walk cache (16–128 entries, 2-way associative).
• If unresolved, a five-level page-table walker (PTW) issues memory accesses into local HBM (~150 ns each). The translation is complete once the appropriate SPA is returned, enabling the local DRAM scheduler to fulfill the memory request.

2. Reverse Lookup in Virtually Indexed, Virtually Tagged Caches

Virtually indexed, virtually tagged (VIVT) L1 caches, especially direct-mapped (DM) designs, are susceptible to synonyms (multiple VAs mapping to the same PA) and coherence-compromising conditions unless reverse address translation is employed at the L1-MMU interface (Desai et al., 2021). Here, the key task is, given a PFN for a valid L1 line, to determine all virtual indices referencing this cache line.

The solution involves an 8-way set-associative reverse lookup table (RLUT) mapping PFNs to up to $S$ VPN indices per L1 line:

• Cache lines: $L = \text{CacheSize}/\text{LineSize}$.
• RLUT set index: PA11:6.
• RLUT tag: PA35:12.
• RLUT data: $3 \times S$ bits (each represents a VPN index). An example for a 32 KiB L1 with $S=1$ requires 1,728 bytes (5.3% of L1 data+tag RAM), and the RLUT logic occupies ~2% of the total processor core gates.

Core RLUT operations are fully pipelined and add zero latency on the L1 hit/miss path. They provide efficient parallel lookup, insert, and invalidate operations during L1 line fills, snoops, and synonym eviction.

3. Performance Characterization and Overheads

In multi-GPU interconnects, reverse address translation overheads are primarily exposed on cold accesses—when collective communication traffic accesses a remote page for the first time and all Link TLBs are cold. Here, RAT can account for up to 1.4× end-to-end application slowdown at small (1 MB per-GPU) buffer sizes, comprising as much as 30% of the round-trip packet latency (Fatima et al., 2 Apr 2026).

As workloads scale (e.g., 16–64 MB buffers), the RAT working set quickly saturates the TLBs and PWC caches. Once warmed, average RAT latency per request drops by over 5×, with end-to-end slowdowns diminishing to ≈1.1× or lower. Increasing L2 Link TLB capacity beyond the number of concurrent GPU destinations yields little benefit, as only $N$ (the number of GPUs) entries are actively required due to stride access patterns in collective communication.

RLUT-based VIVT L1 caches show no measurable impact on L1 hit latency (1 cycle), and L1 miss latency remains unchanged as RLUT operations are overlapped with memory fills (Desai et al., 2021). The performance impact of synonyms is negligible in real workloads (synonym hits <0.01%); forced evictions occur slightly more often than in VIPT caches but remain below 5%.

4. Optimization Strategies

Optimizing reverse address translation in GPU clusters has focused on software-centric approaches:

• Fused pre-translation kernels: Lightweight kernels issued during local compute phases to walk the destination’s page tables for anticipated NPAs, priming the Link TLBs and PWC caches before the arrival of high-throughput collective traffic. With compute overlap (e.g., 1 ms), this technique can recover approximately 90% of the cold-miss penalty, reducing slowdown at 1 MB All-to-All from 1.4× to ≈1.05× (Fatima et al., 2 Apr 2026).
• Software-guided TLB prefetching: Leveraging compile-time knowledge of collective patterns, runtimes walk the user’s page tables and issue explicit Link TLB updates, minimizing latency spikes and back-pressure from mass-inflight PTWs. These methods are advantageous for inference workloads, which operate at small batch sizes where RAT can become SLA-critical.

For VIVT caches, the RLUT implementation is inherently low-overhead. Optimizations mainly target limiting the RLUT associativity ($S$) and scope (preferably direct-mapped L1) to optimize memory and logic usage while maintaining correctness.

5. Design Guidelines and System Integration

Recommended system-level guidelines based on simulation and hardware analysis include:

• Provisioning Link TLBs to hold at least one remote page per concurrent GPU destination (i.e., TLB size ≈ communication fan-out).
• Avoiding inefficient hardware scaling; enlarged TLBs beyond working set size do not improve latency.
• Enabling hardware or API facilities for prefetch-driven and speculative page-table walks.
• For collective communication libraries, inserting pre-translation phases overlapping compute.
• Exposing explicit software-managed TLB management primitives (prefetch, shoot-down, hint APIs) in the collective runtime or driver stack (Fatima et al., 2 Apr 2026).

For VIVT caches, RLUTs must be reset or context-switched appropriately (flush on context switch or incorporate ASIDs) and can be bypassed for superpage mappings (page size ≥ L1 size), where synonymy does not arise (Desai et al., 2021).

6. Hardware and Memory Overheads

In multi-GPU environments, TLB capacity, associativity, and PWC sizing are modest; per-station L1 Link TLBs (32 entries) and shared L2 (512 entries) are sufficient to handle typical collective workloads. RLUT designs in processor L1 caches consume approximately 5.3% of L1 memory and 2% of logical gate count for 32 KiB caches at $S=1$.

Memory overhead in RLUT scales linearly with L1 size and synonym associativity. Greater associativity ($S>1$) or highly associative L1s increase RLUT data width (with fully set-associative L1 requiring storage of the complete VA tag per synonym), thus favoring direct-mapped designs for efficiency.

7. Applications and Broader Impact

Reverse address translation is integral to both large-scale distributed ML systems and single-node multicore microarchitectures. In the former, it is a gating factor for end-to-end inference and training latency, particularly at small action sizes where translation dominates. In the latter, RLUT mechanisms enable high-throughput, low-latency VIVT cache implementations without sacrificing synonym correctness or coherence responsiveness, as demonstrated in the AJIT/SPARC-V8 core (Desai et al., 2021).

These architectural constructs establish a foundation for future interconnect and caching systems where direct mapping of hardware-level identifiers to context- or locality-specific addresses is both performance-critical and necessary for correctness and hardware security.