Post-Training Local LLM Agents for Linux Privilege Escalation with Verifiable Rewards

Abstract: LLM agents are increasingly relevant to research domains such as vulnerability discovery. Yet, the strongest systems remain closed and cloud-only, making them resource-intensive, difficult to reproduce, and unsuitable for work involving proprietary code or sensitive data. Consequently, there is an urgent need for small, local models that can perform security tasks under strict resource budgets, but methods for developing them remain underexplored. In this paper, we address this gap by proposing a two-stage post-training pipeline. We focus on the problem of Linux privilege escalation, where success is automatically verifiable and the task requires multi-step interactive reasoning. Using an experimental setup that prevents data leakage, we post-train a 4B model in two stages: supervised fine-tuning on traces from procedurally generated privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning alone more than doubles the baseline success rate at 20 rounds, and reinforcement learning further lifts our resulting model, PrivEsc-LLM, to 95.8%, nearly matching Claude Opus 4.6 at 97.5%. At the same time, the expected inference cost per successful escalation is reduced by over 100x.

• The paper demonstrates a novel two-stage post-training framework that combines supervised fine-tuning with reinforcement learning using verifiable rewards to optimize Linux privilege escalation tasks.
• It leverages procedurally generated environments to ensure models generalize and achieve near parity with state-of-the-art systems at an impressive 95.8% success rate under strict interaction budgets.
• The approach significantly cuts inference costs by over 100 times and enhances data sovereignty, paving the way for practical, cost-efficient local agents in cybersecurity.

Exploration of Post-Training Local LLM Agents for Linux Privilege Escalation

This paper, "Post-Training Local LLM Agents for Linux Privilege Escalation with Verifiable Rewards" (2603.17673), addresses a critical need in developing efficient Local LLMs for cybersecurity applications, particularly Linux privilege escalation, utilizing a refined two-stage training process. The methodology emphasizes efficiency, cost-effectiveness, and security constraints.

Methodology Overview

The research centers around refining a 4B parameter LLM to achieve superior performance in Linux privilege escalation tasks through a two-phase post-training approach. Initially, the model undergoes Supervised Fine-Tuning (SFT) on procedurally generated environments, followed by a reinforcement learning phase using Verifiable Reward (RLVR).

This post-training model is designated as PrivEsc-LLM and operates on a constrained interaction budget landscape, showcasing substantial improvements in operation efficiency and performance metrics when compared to frontline API services. The reported results demonstrate that PrivEsc-LLM matches the success rate of Claude Opus 4.6 at nearly 95.8% success on benchmark scenarios, whilst reducing inference cost by over 100 times.

Figure 1: $P(\text{root} \mid R)$ across interaction budgets ($R \in \{5,10,\ldots,60\}$). Each curve reports the fraction of runs that achieve root within $R$ rounds; shaded bands show 95\% Wilson CIs. Sample size is $N{=}120$ runs per model.

Procedural Environment and Leakage Preventions

The paper underscores the importance of procedurally generated environments to ensure that models generalize learning rather than memorize specific solutions. By exclusively training on varied environments that imitate benchmark scenarios without direct replication, the study prevents data leakage. The comprehensive procedural generation approach allows for effective policy learning over exploit patterns rather than memorization of solution paths.

The methodology utilizes procedurally varied environments to cover known privilege-escalation classes while withholding those present in the static benchmark to evaluate transfer learning capabilities. This approach is validated through rigorous leakage controls and audit checks to ensure unbiased training and evaluation.

Evaluation and Analysis

In assessing model performance, the study employs a detailed evaluation protocol, focusing on budgeted success rates under strict interaction limits. Figure 2 illustrates the per-scenario success rate at a 60-round budget, indicating the reliable adaptation of PrivEsc-LLM to the complex task demands.

Figure 2: Per-scenario success at the 60-round budget, where each cell shows x/10 successful runs and color encodes the corresponding success rate.

The performance metrics evidence the efficacy of PrivEsc-LLM, which reaches near-parity with state-of-the-art models at significantly reduced operative costs. This includes repelling typical security models and competitive tool baselines like Traitor and pwncat-cs, which achieve lower performance under the same constraints.

Implications and Future Directions

The findings indicate that post-training with verifiable rewards bridges a substantial reliability gap between local open-weight models and proprietary API services. This not only reduces inference costs but also preserves data sovereignty and reproducibility. The broader applicability of this approach spans multiple security domains beyond privilege escalation, emphasizing the potential of verifiable-reward post-training frameworks.

The work suggests practical advancements in local agent development, especially for security tasks where verifiable results and cost-efficient operation are pivotal. Further research across different security paradigms, including crash reproduction and exploit triage, could further validate and extend these techniques.

Conclusion

This study illuminates an effective pathway for developing small, practical LLMs capable of executing complex cybersecurity tasks like Linux privilege escalation under budget constraints. The proposed two-stage post-training framework, with its robust empirical results and considerations for cost efficiency, sets a precedent for future explorations into the integration of LLMs in real-world cybersecurity applications.