Swap-Optimized Memory Runtime

• Swap-optimized memory runtimes are software frameworks that optimize data transfer between high-speed memory and slower storage using user-space fault handling and lightweight thread scheduling.
• They employ innovative components like zero-copy I/O backends, configurable eviction policies, and ultrafast prefetching to achieve up to 40–44% throughput gains over traditional kernel swap designs.
• Their design enhances performance isolation and error recovery with techniques such as TRY/CATCH frameworks and multi-shard eBPF maps, proving valuable in datacenters, GPUs, and disaggregated memory clusters.

A swap-optimized memory runtime refers to a system software layer (or framework) engineered to maximize the performance and predictability of data movement between high-speed memory (e.g., DRAM, HBM, VRAM) and slower storage or far memory, overcoming limitations imposed by kernel-based swapping. These runtimes orchestrate page or object placement, fault handling, I/O scheduling, and concurrency to minimize latency, overhead, and tail risks across device classes such as datacenter servers, mobile devices, GPUs, and disaggregated memory clusters. Swap-optimized memory runtimes leverage architectural innovations, algorithmic policies, and fine-grained user/application-state knowledge to deliver performance isolation, higher throughput, and lower tail latencies compared to legacy kernel swap subsystems.

1. Architectural Principles and Core Components

State-of-the-art swap-optimized memory runtimes, such as LightSwap, FlexSwap, Canvas, and others, depart from traditional kernel-centric swap designs by moving fault handling, scheduling, and data movement into user space or application runtimes and exploiting fine-grained scheduling, ultrafast I/O backends, and direct event notification. Common foundational components include:

• User-space page fault handler: Kernel injects a minimal hook (e.g., eBPF or userfaultfd) at page-fault events; user-space thread retrieves context, processes the fault, and dispatches swap-in (e.g., LightSwap (Zhong et al., 2021), FlexSwap (Pandurov et al., 2024)).
• Asynchronous/zero-copy I/O backend: High-performance device drivers in user space (e.g., SPDK for NVMe, DPDK/RDMA for remote memory, direct I/O for SSD/NVMe) eliminate kernel-side I/O stacks; data is demultiplexed as key-value tuples (VA→4 KB page) and DMA'd directly to application buffers.
• Concurrency model: Integration with lightweight thread schedulers (LWT/coroutine pools) allows the runtime to continue scheduling application tasks during swap I/O, only blocking the faulting logical thread.
• Swap coordination cache: In-memory data structure mapping virtual address to swap location, supporting rapid lookup and minimizing redundant I/O.
• Error handling: Structured exception macros (e.g., TRY/CATCH in LightSwap) allow consistent application-level handling of swap-in errors, DRAM uncorrectable errors, or failed backend I/O, with longjmp-style unwinding.
• Configurable eviction, prefetching, and replacement policies: Pluggable modules (user-space or API-driven) implement LRU, LFU, reuse-distance, or predictor-driven eviction strategies, and workload-tuned prefetching.
• Custom page or object placement strategies: In systems exposing hardware heterogeneity or disaggregation, allocators (e.g., collective allocators in C++ (Hideshima et al., 2024), region/object-based migration in OBASE (Banakar et al., 27 Feb 2026) and Clove (Son et al., 19 May 2026)) select optimal subspaces for hot vs. cold data.

The underlying architecture, as in LightSwap, enables page fault notification latency as low as 2–4 μs, enables direct I/O latency as low as 5 μs (RDMA) or 25 μs (NVMe+SPDK), and supports hundreds of thousands of page-faults per second per core (Zhong et al., 2021).

2. Scheduling, Fault Handling, and Data Movement

Swap-optimized runtimes eliminate bottlenecks of full-thread blocking, kernel stack traversal, and OS-context switching by decoupling logical application work from memory management events. Key technical mechanisms include:

• LWT-based swap-in scheduling: The runtime blocks only the faulting lightweight thread (LWT), spawns a high-priority swap-in LWT, and interleaves ordinary tasks while swap-in proceeds. Context switches are sub-microsecond, maximizing CPU utilization.
• eBPF/map-based state transfer: An eBPF program hooks kernel faults, contributing negligible overhead (≈0.1 μs). Context (e.g., registers, address, scheduler metadata) is written into a per-thread map, which user space reads to resume execution at the user-space handler or to longjmp into an application CATCH block (Zhong et al., 2021).
• Unified work queues and policy-directed event handling: Incoming page faults, prefetches, and I/O completions are queued and dispatched by scheduling policies according to event priority and state (swap-in, swap-out, prefetch) in strict or dynamic order.
• Zero-copy DMA and polling: User-side I/O (e.g., SPDK, RDMA one-sided operations) delivers data directly into application space with polling or event notification to avoid locks and scheduler interlock.
• Prefetching and read-ahead: Aggressive read-ahead (e.g., 8-page default) and workload-informed prefetchers (next-page, sequential, semantic, or application-specific) reduce major fault rates and hide I/O latency behind scheduled compute.
• Error containment: TRY/CATCH frameworks guarantee that any page-fault exceptions, swap failures, or DRAM errors unwind only the logical LWT concerned and provide explicit address and error codes for recovery (Zhong et al., 2021).

3. Performance Modeling and Empirical Gains

The performance profile of swap-optimized runtimes stands in stark contrast to kernel swap- and userfaultfd-based solutions. Key performance metrics and analytical models include:

• Page-fault handling latency:
  • LightSwap-RDMA: 12 μs
  • Kernel swap (w/ block layer): 63 μs
  • Infiniswap (one-sided RDMA + kernel): 40 μs
• Memcached throughput at 50% memory pressure:
  • Kernel swap: 55–60 K TPS
  • Infiniswap: 70–80 K TPS
  • LightSwap-RDMA: 98–115 K TPS (40–44% improvement over Infiniswap)
• Throughput per core:
  • μ_LS ≈ 130 K ops/s (RDMA backend)
  • μ_K ≈ 22 K ops/s (kernel)
• Scalability:
  • Notification latency remains ≈2–4 μs at 128 LWTs (vs hundreds of μs for userfaultfd/signal).
• Overhead:
  • eBPF and swap-cache memory: only a few MB per process.
  • LWT stack per-thread: 32 KB vs. 2 MB for pthread.
  • Average system overhead: negligible under high concurrency.

Conditioned on workload patterns (adequate parallelism, not SSD-saturating write traffic), swap-optimized runtimes exhibit 3–5× reduction in service latency, 40–44% increases in throughput, and 20–40% lower mean operation latency (Zhong et al., 2021). Their prefetchers and LWT scheduling hide most swap-in I/O without throughput or responsiveness penalties.

4. Error Recovery, Robustness, and Integration

Modern swap-optimized runtimes emphasize robust error recovery and seamless integration with application-level and system-level resilience protocols:

• TRY/CATCH frameworks enable precise control flow on swap-in error events, DRAM uncorrectable errors (UCEs), and backend faults, serially longjmp-ing control to the appropriate handler for safe recovery (e.g., application slab invalidation, reload, or abort).
• Isolation and composability: The architecture ensures that only the logical, faulting thread is interrupted, with no need for heavyweight context switches or blocking of the kernel thread or other coroutines.
• Correctness guarantees: Serial longjmps eliminate data race hazards on faults; swap I/O errors and DRAM faults map directly to logical recoverable actions.
• Scalability: Under tens of thousands of threads, eBPF map locks may bottleneck, but sharding and event coalescing alleviate contention.
• Compatibility: The runtime links into memory-intensive user libraries (e.g., RocksDB, in-memory SQL, Redis), replacing userfaultfd with lswaplib and interfacing with application-level scheduling or page management frameworks.

5. Limitations and Workload Sensitivity

Swap-optimized memory runtimes offer substantial advantages but their design is optimized for specific application profiles and may have limitations:

• Low concurrency regimes: If the application presents fewer than ≈10 logical tasks or coroutines, the LWT-based model cannot hide swap-in latency, and kernel approaches may reach similar utility.
• Write-dominated workloads: Under highly random and write-heavy patterns, especially on local SSD, I/O saturation may curtail throughput advantages and reduce the absolute benefit of kernel stack bypass; the I/O path itself becomes the dominant factor.
• Error handling expectation: Applications that do not implement per-page or per-arena exception recovery, or that are legacy in crash semantics, may still experience fatal errors on UCEs or swap-in failures if not adapted for structured exception handling.
• eBPF map contention: At extreme thread counts, lock contention in notification maps may appear, but multi-shard (e.g., 32-shard) eBPF maps reduce dependency.
• Single-threaded kernels: Legacy or strictly single-threaded codebases cannot directly exploit the fine-grained concurrency advantages.

6. Generalizations and Applicability

The architecture and approach of swap-optimized memory runtimes generalize to multiple domains and memory substrates:

• In-memory databases and key-value stores: Seamless integration with lswaplib, efficient swap for hot/cold data separation, and object-aware placement (Zhong et al., 2021).
• Remote memory and disaggregation: The allocation and swap logic accommodates RDMA, user-level NVMe, and far-memory systems with dynamic mapping and prefetch/adaptive caching; mechanisms are compatible with far-memory models where local and remote regions partition the address space (Hideshima et al., 2024).
• Heterogeneous and tiered memory: The translation of swap events and policies to adapt to CXL, persistent memory, and mixed DRAM/remote architectures is straightforward, by plugging in appropriate I/O engines and scheduling policies.
• Managed runtimes and coroutines: JVM, Go, and other VM-level environments can link to swap-optimized runtimes using a minimal kernel do_page_fault hook and user-space event bridges.
• Event-driven applications: The eBPF + LWT event notification and handler pattern applies to other async user-space workflows, including network eventlet scheduling.

7. Comparative Analysis and Future Directions

Swap-optimized memory runtimes such as LightSwap (Zhong et al., 2021) demonstrate that the kernel I/O stack is the principal source of modern swap inefficiency, and that high-performance user-space handling delivers material end-to-end gains. The design space remains active, with anticipated directions including:

• Integration with hardware notification mechanisms, e.g., CXL or NUMA hardware interrupts.
• Adaptive I/O backend selection: Dynamic policy to pivot between NVMe, RDMA, or remote tiering as device latency/bandwidth landscapes shift.
• Hybrid object- and page-level management: Combining swap-optimized page handling with fine-grained object migration (as in OBASE (Banakar et al., 27 Feb 2026), Clove (Son et al., 19 May 2026)) to further reduce hotness fragmentation and elevate memory tiering efficiency.
• ML-predictive swap policy: Incorporating ML predictors for swap-in/out decisions and prefetching.
• Robust cross-process and multi-tenant support: Systematic support for per-application swap path isolation (as in Canvas (Wang et al., 2022)) and policy API exposure for cross-tenant optimization.

In conclusion, swap-optimized memory runtimes comprise a class of system software frameworks that combine user-space fault handling, lightweight thread scheduling, ultrafast I/O, structured error handling, and fine-grained policy composition. Collectively, these systems enable memory-intensive workloads to access larger effective capacity, exhibit significantly lower swap latencies, and maintain high throughput under memory pressure—defining a new baseline for modern memory management in high-performance and heterogeneous computing environments (Zhong et al., 2021).