DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback

Abstract: LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the complete sandbox state, including files and process state (e.g., memory, contexts, etc.). Existing mechanisms duplicate the entire state, causing hundreds of milliseconds to seconds of latency per C/R, which severely bottlenecks deep search and large-scale fan-outs. This paper observes that subsequent checkpoints in AI agents are highly similar. Therefore, instead of full duplication, a sandbox should only duplicate the changes between consecutive checkpoints (Key Insight). However, it is non-trivial to realize the idea, mainly due to the missing OS supports. This paper proposes a new OS-level abstraction, DeltaState, to enable the change-based transactional C/R for AI agents with two co-designed OS mechanisms. First, DeltaFS enables change-based filesystem C/R by organizing the file states into layers and dynamically freezing the writable layer and inserting a new one during checkpoint, reducing file updates to copy-on-write, and making rollback a simple layer switch. Second, DeltaCR enables change-based process state C/R using incremental dumps, and accelerates rollback by bypassing traditional pipelines to directly fork() from a frozen template process. We then present DeltaBox, a novel agent sandbox achieving millisecond level C/R through the two new mechanisms. Evaluations on SWE-bench and RL micro-benchmarks show DeltaBox completes checkpoint and rollback in millisecond-level latency (14ms and 5ms, respectively), empowering agents to explore substantially more nodes under fixed time budgets.

• The paper introduces DeltaBox, a system that achieves millisecond-level checkpoint and rollback through diff-based state management of filesystem and process states.
• It leverages DeltaFS and DeltaCR to minimize latency, reducing state-management overhead to 3–6% of total trajectory time compared to traditional methods.
• Comprehensive evaluations demonstrate significant speed-ups in RL and MCTS workloads, enabling deep agent search and efficient parallel rollouts.

DeltaBox: Millisecond-Scale Checkpoint/Rollback for Stateful AI Agent Sandboxes

Motivation and Problem Statement

Modern LLM-powered AI agents increasingly operate in complex OS-level environments for tasks such as software repair, web navigation, and reinforcement learning. These tasks require the agents to execute actions that have irreversible side effects on both the filesystem and process state. High-frequency state exploration via tree search (e.g., MCTS, BoN sampling) and RL training mandates efficient checkpoint/restore (C/R) of coupled process and filesystem state. Existing sandboxing and C/R solutions are bottlenecked by full state duplication, resulting in latencies ranging from hundreds of milliseconds to seconds per event. This impedes deep search and parallel trajectory exploration, particularly when scaling test-time compute or RL batch sizes.

Key Insight and System Design

A core observation is the high similarity between consecutive agent states—typically, only incremental changes exist between checkpoints. DeltaBox introduces the DeltaState abstraction, leveraging this property to support change-based, transactional C/R, and achieves millisecond-level latency.

Figure 1: DeltaBox architecture, illustrating the StateManager's coupling between dynamic DeltaFS (filesystem) and DeltaCR (process state) components atop base storage.

DeltaBox comprises two principal OS-level mechanisms:

• DeltaFS: An overlayfs-based filesystem layer with runtime hot-switching capability. On checkpoint, DeltaFS freezes the current writable layer into a read-only lower layer, inserts a fresh writable upper layer, and tracks only file-level changes. Rollback is implemented as a fast layer stack switch.
• DeltaCR: Built on CRIU, DeltaCR submits incremental dumps and maintains a bounded pool of process templates (frozen via fork()). Restore is accelerated by forking templates when available; otherwise, CRIU's lazy-pages mode reconstructs process memory.

A StateManager orchestrates atomic C/R of coupled (filesystem, memory) pairs per search node, preventing divergence between the agent's process state and its working context.

Technical Implementation

DeltaFS: Dynamic Overlay Filesystem

DeltaFS extends overlayfs by enabling unmount-free runtime layer stack modifications via a custom kernel ioctl, and by introducing per-inode generation counters for lazy switch semantics on open files. XFS with reflink is employed for block-level CoW granularity, sharply reducing write amplification—extents unchanged over $N$ checkpoints remain single blocks shared across layers.

Figure 2: Write amplification characterization for different filesystems showing copy-up and physical I/O bytes per agent edit; copy-up benefit entirely derives from XFS reflink.

DeltaCR: Process State Management

DeltaCR couples CRIU incremental dumping with template creation. Each checkpoint produces both a durable memory image and a forked process template. The restore mechanism preferentially forks from alive templates for millisecond-level latency, with chain management and LRU eviction ensuring bounded template pool sizes. Network Proxy Daemon decouples LLM client I/O to keep agent templates forkable. A GSD-side async-warm thread absorbs CoW page faults post-restore, minimizing agent-visible latency.

Figure 3: DeltaCR architecture illustrating checkpointing via CRIU image chain and templates, with restore paths utilizing template forks or incremental images.

Evaluation and Numerical Results

DeltaBox is evaluated across SWE-bench Verified and RL benchmarks, compared to baselines like docker commit, copytree, Firecracker VM snapshots, and CubeSandbox. The following are principal empirical observations:

• Checkpoint: Primitives complete within 14 ms (mean).
• Restore: Fast path achieves ≤6 ms (P95 across workloads); slow path incurs modestly higher but bounded overhead.
• End-to-End Overhead: On MCTS workloads, DeltaBox reduces state-management time from 47–77% to only 3–6% of trajectory wall clock.
• GPU Utilization in RL: DeltaBox approaches near-maximum GPU occupancy at batch sizes up to 64, eliminating RL training bottlenecks due to environment fan-out.
• Write Amplification: With XFS reflink, write amplification is proportional to actual changes and independent of directory size.
• Adaptive Skipping: Employing a semantic action classifier, DeltaBox elides unnecessary checkpoints for pure-read events, yielding a 62% skip ratio across benchmarks.

  Figure 4: Pass rates on SWE-bench Verified: (a) Comparing Linear ReAct and MCTS strategies; (b) Base vs RL-trained models across open-weight coding models.

  

  Figure 5: End-to-end MCTS trajectory times (100 iterations) showing DeltaBox consistently near the LLM-only lower bound, while baselines incur 2–4× the time overhead.

  

  Figure 6: Empirical blocking-time CDF for checkpoint and restore events; DeltaBox achieves 1–3 orders of magnitude latency improvement over all coupled baselines.

Practical and Theoretical Implications

The architectural coupling of process and filesystem state ensures deterministic rollback and fork semantics for stateful language agents. DeltaBox enables deep and wide agent search, iterative refinement, and extensive parallel rollouts necessary for modern inference and RL workloads. The millisecond-scale C/R eliminates tree search bottlenecks, unlocking systematic exploration strategies and precise reward assignment in agent policy training. The OS-level diff-based checkpoint strategy can generalize to other transactional state management domains. Sandboxing overhead is now dominated by LLM inference, rather than infrastructure latency.

The practical deployment model, based on Firecracker microVMs and Linux 6.8, is extensible to container architectures and other VM solutions. Compatibility adapters with frameworks such as LangGraph and LangChain provide seamless integration for existing agent systems.

Prospects for Future Research

DeltaBox's diff-based approach represents an effective paradigm for stateful agent sandboxing. Future directions may include:

• Distributed state management across shared-memory pools for multi-agent scaling.
• Integration with hardware-accelerated snapshot compression (cf. Sabre) and speculative checkpointing for further latency reduction.
• Enhanced support for network I/O rollback and external side effect isolation.
• Adaptive checkpoint scheduling leveraging agent-level or task-level heuristics.

Additionally, extending the reachability-aware GC to hierarchical or multi-agent tree searches offers promising scalability.

Conclusion

DeltaBox is a robust OS-level sandbox architecture addressing the millisecond-scale C/R bottleneck for stateful AI agent workloads, via change-based coupling of filesystem and process state (2605.22781). It achieves an order-of-magnitude reduction in latency compared to all evaluated baselines, enabling efficient search and training for large-scale, state-heavy agent systems. The abstractions and optimizations introduced can serve as foundations for advanced agentic infrastructure and efficient system-level reasoning support.