Linux Privilege Escalation Benchmark

• Linux Privilege Escalation Benchmark is a suite of automated scenarios that rigorously tests and quantifies escalation techniques on Linux platforms.
• The benchmark employs diverse architectures including dedicated VMs, containerized sandboxes, and LLM-driven agent evaluations to simulate real-world vulnerabilities.
• It uses empirical metrics like success rates and time-to-root to guide security recommendations, enhancing defenses against misconfigurations and container escapes.

A Linux privilege-escalation benchmark is a rigorously defined suite of scenarios, artifacts, and automated protocols designed to quantify, compare, and improve privilege-escalation techniques and defenses on Linux platforms. Such benchmarks are crucial both for empirical evaluation of attacker tooling—whether human, scripted, or agentic (e.g., LLM agents)—and for benchmarking protection mechanisms in both research and operational settings. Modern benchmarks span single-vector VMs, containerized escape challenges, and decision-problem formulations that support fully automated and verifiable evaluation. Leading public benchmarks include those introduced in "Quantifying Frontier LLM Capabilities for Container Sandbox Escape" (SANDBOXESCAPEBENCH) (Marchand et al., 1 Mar 2026), "Got Root? A Linux Priv-Esc Benchmark" (Happe et al., 2024), "LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks" (Happe et al., 2023), and agent evaluation frameworks with verifiable reward (Normann et al., 18 Mar 2026).

1. Benchmark Architectures and Environment Models

Linux privilege-escalation benchmarks adopt several architectural paradigms, reflecting both attacker goals and defensive contexts:

• Single-Vulnerability VM Suites: Each test-case is provisioned as a dedicated VM (e.g., Debian-based) with precisely one misconfiguration or flaw, managed via Vagrant and automated by Ansible (Happe et al., 2024, Happe et al., 2023). These environments prohibit extraneous network access and services, ensuring tightly scoped, reproducible evaluations.
• Nested Container/VM Sandboxes: SANDBOXESCAPEBENCH implements a "sandbox-in-a-sandbox" model: an outer Ubuntu VM host is provisioned (often using Vagrant or AWS EC2), inside which a Docker or Kubernetes container is launched. The privilege boundary is between the container ("attacker's jail") and the host VM ("defensive perimeter"), with the flag-residing only outside the container namespace (Marchand et al., 1 Mar 2026).
• Procedural Scenarios for Agent Training: For LLM agent evaluation, privilege escalation is framed as a decision process in which the agent (LLM) interacts via a shell (e.g., SSH into a fresh container), issuing commands to reach root; the environment is reset for each scenario (Normann et al., 18 Mar 2026). Ensuring generalization, scenario generators are used during training, while handcrafted static benchmarks assess held-out performance.
• Automation and Reproducibility: All benchmarks rely on code-defined environments (Vagrantfiles, Dockerfiles, YAML scenario registries) and automated orchestration, supporting exact replay and systematic extension. Post-run VMs or containers are destroyed, guaranteeing environmental isolation (Happe et al., 2023, Marchand et al., 1 Mar 2026).

2. Vulnerability Taxonomies and Scenario Construction

Benchmarks index vulnerability scenarios into well-defined taxonomies:

Vulnerability Category	Example Flaws
SUID & sudo	SUID binary (GTFOBin), sudoers misconfiguration, sudo GTFOBin
Privileged group misuse	docker group membership, Docker sock
Information disclosure	password/hash/file leaks, root in .bash_history, weak password, SSH key reuse
Cron-based	writable cron script, wildcard injection, crontab visibility
Container/runtime escape	--privileged flags, runc overwrite, /var/run/docker.sock
Kernel/CVE	DirtyC0W, Dirty Pipe (CVE-2022-0847), eBPF bugs
Orchestration weakness	Kubernetes RBAC abuse, sysctl injection

Each test-case is a single-point-failure: only one privilege-escalation vector is installed per environment. Several benchmarks, including (Happe et al., 2024) and (Happe et al., 2023), ensure that MITRE ATT&CK mappings and scenario classes are explicit, aiding exercise integration and comparative studies.

Procedural environments, as in (Normann et al., 18 Mar 2026), randomize file paths, binary names, and credentials to prevent overfitting and data leakage during LLM training and evaluation. SANDBOXESCAPEBENCH uniquely targets container-specific escape paths, including misconfiguration (--privileged), runtime flaws (Docker sock, runc/CVE), and orchestration bugs.

3. Evaluation Methodology and Metric Definition

Evaluation across benchmarks is always automated and uses binary success/failure as the primary outcome, with extensions for timing and efficiency:

• Success Metric: Root attainment is empirically defined as either a shell running as UID 0 or explicit flag/submission string matching (e.g., reading /flag.txt or calling submit("flag{…}") (Marchand et al., 1 Mar 2026, Normann et al., 18 Mar 2026)).
• Per-Scenario and Overall Scoring: For SANDBOXESCAPEBENCH, success in run $j$ of scenario $i$ is represented as:

$$S_{i,j} = \begin{cases} 1 & \text{if flag captured} \ 0 & \text{otherwise} \end{cases}$$

Scenario success rate:

$R_i = \frac{1}{M}\sum_{j=1}^M S_{i,j}$

Overall score (mean across scenarios):

$$S = \frac{1}{N}\sum_{i=1}^N R_i = \frac{1}{NM}\sum_{i=1}^N\sum_{j=1}^M S_{i,j}$$

• LLM Agent Protocols: For agent-based evaluation, each round corresponds to an LLM invocation with command/tool execution; the primary metric is budgeted success probability:

$$P(\mathrm{root}\mid R) = \frac{1}{N}\sum_{i=1}^N\mathbf{1}\{\tau_i\le R\}$$

where $\tau_i$ is the round in which root is attained or $=\infty$ if not (Normann et al., 18 Mar 2026).

• Exploit Complexity and Timing: Some authors propose (but do not enforce) metrics such as time-to-root, exploit step count, command count, or need for hints (Happe et al., 2024). Rounds-to-root and inference budgets are explicit in agent benchmarks (Happe et al., 2023).

Composite scoring formulas—for example, $S = \alpha(1/T) + \beta R - \gamma E$—are discussed but not standardized, providing flexibility for downstream evaluators.

4. Implementation and Reproducibility

Benchmarks are engineered for open-source reproducibility and scenario extensibility:

• Deployment Toolchains: Environments are orchestrated with Vagrant (multi-provider), Ansible (per-scenario roles), and Docker/Kubernetes for containerized cases. Each benchmark suite comes with pre-packaged scripts and VM/container descriptors (Marchand et al., 1 Mar 2026, Happe et al., 2023).
• Scenario Artifacts: Benchmarks include a repository of configuration files, provisioning scripts, test-case YAMLs, and exploit hints (often as /home/lowpriv/HINT) (Happe et al., 2024).
• Agent Harness Integration: For agent evaluation, an interface wraps LLM calls around shell command execution, flag submission, and state updates (e.g., Inspect CTF wrapper in SANDBOXESCAPEBENCH, or hackingBuddyGPT interface library) (Marchand et al., 1 Mar 2026, Happe et al., 2023).
• Logging and Automation: Evaluation frameworks log all agent output, tool invocations, and scenario metadata (timestamps, token counts, scenario tags) in machine-consumable formats (e.g., JSON, SQLite), supporting analytical post-processing and cost modeling (Happe et al., 2023).

5. Empirical Results and Comparative Analysis

Empirical studies across benchmarks demonstrate distinct performance regimes for LLMs, traditional tools, and humans:

System / Model	Success Rate	Notable Observations
Human pentester (no hints)	75%	Enumeration dominates time budget
Human (+ hints)	91%	Hints close gap on hard cases
traitor, pwncat-cs (tool)	8–16%	Enumeration only, poor exploitation
GPT-4-Turbo (no hints/state)	33%	Improved with hints or reflection
GPT-4-Turbo (+ high-level hints)	66%	State compression further boosts
GPT-4-Turbo (state, hints, 8k ctx)	83%	Approaches human performance
GPT-3.5-Turbo (hints, 8k ctx)	50%	Cost-efficient but limited
Llama3-8b/70b (local, hints)	0–33%	Not yet autonomous, slow progress
PrivEsc-LLM (Qwen3-4B, RLVR)	95.8%	Nearly matches Claude 4.6, >100x cheaper than cloud LLM
Claude Opus 4.6	97.5%	SOTA for LLM agent priv-esc
SANDBOXESCAPEBENCH: GPT-5/Opus 4.5	80–100% (easy)	Complex kernel escapes unsolved
SANDBOXESCAPEBENCH: GPT frontier (hard)	0–50% (CVE/kernel)	Only multi-step or trivial escapes

Observations:

• Trivial misconfigurations (e.g., sudo NOPASSWD, SUID, password reuse) are solved by all competent actors, including modern LLMs when guided (Marchand et al., 1 Mar 2026, Happe et al., 2023, Happe et al., 2024).
• State compression (reflective summarization) and high-level hints can double LLM success rates.
• Multi-step and temporal (cron, Docker) scenarios pose persistent challenges, requiring wider context and sequential reasoning (Happe et al., 2023, Normann et al., 18 Mar 2026).
• LLM-driven agents can exceed 95% success on custom benchmarks after SFT and RLVR, with computation cost roughly $0.005 per escalation for local models vs.$0.62 for SOTA cloud LLMs (Normann et al., 18 Mar 2026).
• Kernel-level and complex container escapes remain unsolved for all non-human, non-guided approaches (Marchand et al., 1 Mar 2026), indicating a sharp capability drop beyond shallow misconfiguration exploitation.

6. Security Recommendations and Benchmark Impact

Benchmark results have precipitated concrete operational recommendations for both practitioners and system defenders:

• Audit and minimize SUID binaries; restrict sudoers to enforce least-privilege and remove NOPASSWD where possible (Happe et al., 2024).
• Treat membership in privileged groups (e.g., docker) as equivalent to root access, strictly limit assignment (Happe et al., 2024, Marchand et al., 1 Mar 2026).
• Secure cron jobs: scripts should be owned/writable only by root, with directories set immutable where feasible.
• Enforce key hygiene: prohibit plaintext credentials, rotate SSH keys and disable root login over SSH (Happe et al., 2024).
• Defend against container escapes by never granting --privileged or host-level mounts without strict controls; validate Kubernetes RBAC policies (Marchand et al., 1 Mar 2026).
• Integrate benchmark suites (e.g., benchmark-privesc-linux, SANDBOXESCAPEBENCH) into red/blue team exercises for ongoing systems hardening.

This suggests that contemporary LLM-powered automation and systematic vulnerability benchmarking are converging, enabling quantifiable improvements in privilege-escalation defense and testing strategies.

7. Extensions, Limitations, and Future Directions

Current benchmarks illuminate critical technical frontiers and research gaps:

• While misconfiguration and information-disclosure attacks are well benchmarked, deep kernel exploitation and novel container escapes require scenario expansion (Marchand et al., 1 Mar 2026).
• Agent training and evaluation reveal that procedural generation of scenarios guards against overfitting, but may underrepresent the diversity of real-world systems (Normann et al., 18 Mar 2026).
• The cost-benefit tradeoff of agent inference (token budget vs. success probability) and parallel vs. sequential command execution remains an open systems optimization challenge.
• Open-sourcing of scenarios, code, and agent scaffolding (e.g., hackingBuddyGPT, Inspect CTF) supports reproducibility and encourages cross-benchmarking. However, prompt-sensitivity and model drift in LLM-based agents are unexamined and constitute a future research need.
• No current benchmark enforces composite, standardized scoring; the field would benefit from a consensus on metrics beyond binary success, including step complexity, minimality of exploit, and robustness to scenario perturbation.

These benchmarks have become foundational instruments for both empirical security research and model-centric evaluation of automated privilege-escalation capabilities.