Sandboxing Techniques

• Sandboxing techniques are security containment methods that isolate code execution by enforcing resource limits, privilege boundaries, and policy mediation.
• They employ diverse mechanisms including process-based isolation, software fault isolation, language sandboxes, hardware-assisted partitioning, and BPF kernel mediation to balance performance and security.
• These techniques are applied in browsers, cloud systems, malware analysis, and emerging AI platforms, addressing challenges in formal verification, performance overhead, and ease of adoption.

Sandboxing Techniques

Sandboxing is a class of security containment mechanisms designed to isolate code execution, thereby enforcing resource limits, privilege boundaries, and robust policy-driven mediation between mutually distrustful components. Sandboxing underpins the security of browsers, operating systems, cloud infrastructure, malware analysis systems, language runtimes, and emerging AI agents. Research addresses multiple realizations, ranging from language-based isolation, process- and kernel-level controls, in-process memory partitioning with hardware support, down to fine-grained runtime binary rewriting. The following survey provides a comprehensive overview focused on architectural patterns, formal guarantees, enforcement approaches, performance trade-offs, and deployment experience across these domains.

1. Architectural Patterns and Enforcement Models

Sandboxing implementations vary by their enforcement mechanism and granularity of isolation.

• Process/Container-based Sandboxing: Resources are isolated at the OS or VM boundary. Examples include Capsicum capability mode and Casper daemons for service compartmentalization (“CapExec”) (Jadidi et al., 2019), Linux namespaces/cgroups/seccomp for plugin isolation (Suneja et al., 2019), and Docker/Jail/VMs for multi-tenant separation (Alhindi et al., 2024).
• Software Fault Isolation (SFI): Enforces memory and control-flow confinement via binary rewriting and inline checking; all memory accesses and jumps are instrumented to stay within sandbox regions (Kolosick et al., 2021). WebAssembly SFI and RLBox’s Wasm back end (Narayan et al., 2020) exemplify SFI in web-scale systems.
• Language-based Sandboxing: Utilizes runtime wrappers, membranes, proxies, or interpreters within the language VM for fine-grained interposition. DecentJS (Keil et al., 2016) leverages ES6 proxies for transactional effect logging, while SafeJS (Cassou et al., 2013) uses OS-level web workers and virtual DOM mediation.
• PKU-based In-Process Sandboxing: Employs hardware Memory Protection Keys (x86 PKU) to dynamically partition address space and enforce rapid user-space transitions between trusted/untrusted regions, as in Garmr (Voulimeneas et al., 2021) and the SandCell Rust framework (Zhang et al., 28 Sep 2025).
• BPF Kernel Isolation: Binary rewriting for kernel extension frameworks (SandBPF) applies SFI and CFI at instruction level to confine unprivileged eBPF programs (Lim et al., 2023).
• App-layer/Brower and Agent Sandboxing: Application-layer extensions for browser agents (ceLLMate) intercept semantic API layers (HTTP actions) for least-privilege enforcement (Meng et al., 14 Dec 2025); serverless function accelerators switch adaptively between language and OS sandboxes based on execution trace coverage (Herbert et al., 2019).

Enforcement can target system calls (seccomp (Alhindi et al., 2024), Apple SBPL policy (Deaconescu et al., 2016)), syscall arguments, file/FD capabilities, network, memory, or language/runtime constructs depending on the mechanism.

2. Formal Guarantees, Threat Models, and Security Properties

Sandboxing research formalizes security using integrity, confidentiality, and noninterference properties:

• SFI/PKU/SandBPF: Proven properties include:
  • Memory Isolation: $$\forall i,\,\forall a:\text{Exec}(P^*,i,a)\implies a\in R$$ (all accesses within the sandbox’s reserved region).
  • Control-Flow Integrity: All indirect jumps/calls land in vetted code regions $C(P)$ (Kolosick et al., 2021, Lim et al., 2023).
• Transactional/Effect-based Sandboxing: Policies are predicates $P: \mathit{Effect} \rightarrow \mathsf{Bool}$ over logs of observable actions (get, set) (Keil et al., 2016).
• STM with Deferred Updates: Opacity and atomicity (no application-visible anomaly from “doomed” transactions); full consistency is checked at commit or at dangerous operations via static/dynamic validation (Machens, 2014).
• Browser/Mobile App Agents: ceLLMate (Meng et al., 14 Dec 2025) and MATRIX (Narain et al., 2018) enforce deterministic authorization for semantic actions, requiring

$$\forall r.\; \alpha(r)=a \wedge \pi(a)=\mathit{ALLOW} \implies \mathit{forward}(r)$$

with $a$ mapped from observed browser requests and user/developer policies.

Advanced models, such as “speculation-safe noninterference” for Spectre mitigations, demand indistinguishable architectural and microarchitectural traces under both sandboxed and speculative semantics (Cauligi et al., 2022).

Threat models must consider attacker capabilities: within-language/untrusted plugin code, co-tenant adversaries (Spectre), PKU syscall/ROP abuse, kernel eBPF privilege escalation, or prompt injection in browser-using agents.

3. Policy Models, Mediation, and Expressiveness

Sandboxing policy frameworks offer varying degrees of expressiveness and mechanism:

Mechanism	Policy Model	Granularity	Update Model
seccomp	BPF-filter allow/deny rules	syscall & args	static
Capsicum	Capability mode + FD rights	per-FD	runtime+child
SBPL (Apple)	Scheme-like rules w/ regex/cond.	per-op, per-arg	static
DecentJS	Predicate $P(\mathit{Effect})$	JS op (get/set/call)	transactional
SafeJS	Set policy over message actions	DOM (read/write)	static
RLBox	C++ type-driven tainting, validators	data, control-flow	code-compile
Garmr/SandCell	Trust domains, monitor-mediated	PKU region	runtime
ceLLMate	HTTP-layer semantic mapping + LLM	semantic action	user+admin

Effect systems and explicit logs (DecentJS, STM) support post-hoc, transactional policy application. Syscall and FD-based models (seccomp, Capsicum) provide strong least-privilege but are harder to retrofit and audit. RLBox’s use of a static type lattice automates and localizes much of the untrusted/trusted boundary in large codebases.

Agent and browser extension sandboxes (ceLLMate, SecureSign) explicitly map low-level events or requests to semantic primitives prior to policy application, enabling effective human-in-the-loop or AI-assisted policy composition in dynamic, ad-hoc settings (Meng et al., 14 Dec 2025, Ji et al., 18 Nov 2025).

4. Performance Characteristics and Overhead Factors

Sandboxing’s performance impact depends on mediation granularity, cross-boundary frequency, and overhead of isolation:

• Transactional sandboxes (DecentJS): Baseline proxy/shadowing costs ≈8×, with full effect logging ≈32.6× over native, but for many code patterns only 1–2× is observed (Keil et al., 2016).
• Process/SFI/PKU: RLBox’s SFI backend achieves <1%–49% overhead (libjpeg/libpng) (Narayan et al., 2020); in-process PKU (Garmr) achieves sub-2% in OpenSSL, 1–4% in server workloads (Voulimeneas et al., 2021); SandCell (Rust, PKU) reduces per-boundary overhead to sub-microsecond via heap-sharing, with overall overhead for real-world code often <5% (Zhang et al., 28 Sep 2025).
• Dynamic Kernel SFI (eBPF): SandBPF introduces 0–6% overhead in web workloads, 10% in synthetic compute-bound microbenchmarks (Lim et al., 2023).
• Serverless Language Sandbox: Container start = 100ms–2s; “fast path” Rust-based language sandbox = 0.3ms per-request; 2.1–10.5× end-to-end speedup for I/O-bound serverless, with fallback to OS sandbox on unsupported trace (Herbert et al., 2019).
• Agent/browser mediation (ceLLMate): 7–15% performance penalty at scale in real-world web automation (Meng et al., 14 Dec 2025).
• Malware analysis sandboxes: Agent-less (hypervisor/VMI) sandboxes offer 15–25% resource overhead, outperforming agent-based approaches for evasive malware (Ali et al., 2019).

Memory cost is typically moderate (tens of MBs per sandbox), unless high sandbox density requires heavy process- or VM-based instance spawning (Schwarzl et al., 2021, Alhindi et al., 2024), though in-process or language-level solutions scale well.

5. Usability, Deployment Experience, and Empirical Adoption

Empirical studies of sandboxing deployment in open-source OSes reveal low direct adoption, with <1% of packages invoking sandbox APIs (e.g., seccomp, Capsicum, Landlock, Pledge, Unveil), although many more benefit indirectly via dependencies or higher-level toolchains (Alhindi et al., 2024). Challenges include:

• Usability: Policy definition (e.g., seccomp BPF) is error-prone; “mimicking” of higher-level semantics (as with Pledge-via-seccomp) is common.
• Codebase impact: Capsicum and Casper necessitate architectural refactoring (capability channels, process splitting) (Jadidi et al., 2019). RLBox shows practical incremental adoption using type-driven tooling (tainted types, auto-generated validators) (Narayan et al., 2020).
• Debugging: Policy violation handling often terminates the process with minimal diagnostic feedback.
• Fragmentation: Each OS ecosystem mandates different APIs and models, complicating cross-platform policy enforcement (Alhindi et al., 2024).
• Transparency: Sidecar/plugin sandboxes (as in plugin state extraction (Suneja et al., 2019) and malware orchestration (Udeshi et al., 19 Aug 2025)) allow secure extensibility without refactoring the monitored endpoint but may introduce minor performance overhead (around 15%).

Notably, the open-source community gravitates to simplicity: mechanisms with minimal developer effort, such as Pledge/Unveil, see relatively higher adoption (Alhindi et al., 2024).

6. Recent Advances and Directions in Sandboxing Research

• Hybrid and Adaptive Isolation: Techniques combining dynamic monitoring, adaptive process isolation (DPI (Schwarzl et al., 2021)), transactional effect-logging (DecentJS (Keil et al., 2016)), or switching between language- and OS-hosted sandboxing per workload (Herbert et al., 2019).
• Formal Verification: Automated static verification (VeriZero (Kolosick et al., 2021)) for SFI code, machine-checked noninterference models for Spectre-resistant sandboxes (Cauligi et al., 2022), and logical proof artifacts for integrity and confidentiality across SFI, PKU, and kernel models.
• Application-specific Sandboxing: Customized sandboxes for language ecosystems (e.g., SandCell for Rust (Zhang et al., 28 Sep 2025)), plugin frameworks (Suneja et al., 2019), dynamic behavioral analysis for malware (agent-less VMI (Ali et al., 2019)), mobile location/data privacy (Narain et al., 2018), and agent/AI-centric browsing (Meng et al., 14 Dec 2025).
• Semantic Mediation and User-centric Policy: Agent frameworks (ceLLMate) and mobile Web3 sandboxes (SecureSign (Ji et al., 18 Nov 2025)) route enforcement through semantic action graphs, HTTP mediation, and policy composition driven by human or LLM-predicted input.

Despite a proliferation of mechanisms, successful sandbox adoption is most closely linked to minimizing developer burden and transparent integration with existing programming paradigms and service lifecycles.

7. Limitations, Trade-offs, and Open Challenges

Trade-offs between expressiveness, performance, security, and ease of use remain active research concerns:

• Expressiveness vs. Complexity: Rich policy languages (SBPL (Deaconescu et al., 2016), sandboxing APIs with syscall argument inspection) offer tight control but complicate deployment and auditing.
• Performance vs. Transparency: While SFI and in-process PKU-based systems minimize transition cost (Kolosick et al., 2021, Voulimeneas et al., 2021), process or VM-based isolation guarantees remain attractive where full mediation and strong kernel boundaries are necessary (e.g., for kernel extension safety (Lim et al., 2023)).
• Security Model Limitations: Some mechanisms are susceptible to microarchitectural attacks (Spectre), requiring dynamic detection or hardware mitigations (Schwarzl et al., 2021, Cauligi et al., 2022). PKU’s security is contingent on strict gating (call gates, monitor control) (Voulimeneas et al., 2021).
• Usability Barriers: Absent standardized, cross-platform abstractions and improved debugging/introspection tools, adoption lags technical potential (Alhindi et al., 2024).

Open problems include verifying complex policies at scale, integrating dynamic and static enforcement for transient vulnerabilities, and automatically synthesizing least-privilege sandboxes tailored to composite, rapidly evolving applications.