SERA: Soft-Verified Efficient Repository Agents

• SERA is a framework that employs soft patch-level verification and versatile prompts for scalable codebase analysis and generation.
• It leverages demand-driven data and synthetic pull request trajectories to bypass traditional test and CI infrastructure.
• Empirical results from SERA-32B and RepoAudit show significant improvements in sample efficiency, compute cost, and code auditing precision.

Soft-Verified Efficient Repository Agents (SERA) are a class of autonomous LLM-centric systems and agent frameworks for efficient, scalable, and low-cost codebase-level analysis and specialized code generation. The SERA paradigm is characterized by a “soft verification” approach, demand-driven data and workflow generation, and cost-effective supervised fine-tuning. The framework is notably applicable to tasks such as code auditing, repair, refactoring, and repository-level bug detection, particularly over private or proprietary codebases where traditional test or CI infrastructure is limited or absent. SERA methods have demonstrably matched or exceeded state-of-the-art performance for open-source models, with magnitude-scale improvements in both sample and compute efficiency, and offer generalizable architectural templates as exemplified by SERA-32B (Shen et al., 28 Jan 2026) and RepoAudit (Guo et al., 30 Jan 2025).

1. Conceptual Foundations and Motivation

SERA was introduced to address the bottlenecks inherent in developing specialized coding agents for private repositories. Traditional reinforcement learning (RL) agents and synthetic data pipelines require extensive test infrastructure, sandboxed executions, and prohibitive hardware or engineering resources, precluding small teams from practical deployment. SERA aims to eliminate these constraints by (a) replacing execution-based verification with “soft” patch-level similarity checks, and (b) utilizing high-level, vague prompts that expand the diversity of coding data beyond regression-style bugfixes.

The SERA methodology is grounded in two central insights:

1. Soft verification uses line-level recall (overlap) between generated code patches instead of unit-test or CI-based validation, significantly simplifying data generation.
2. Versatile prompt sets enable synthetic code edit trajectories to encompass a wider operational spectrum (e.g., refactoring, documentation, large-scale changes), validated as equally effective for supervised fine-tuning.

This design enables SERA agents to specialize rapidly to arbitrary codebases without any requirement for running the code, ensuring scalability across public and private repositories (Shen et al., 28 Jan 2026).

2. Methodological Framework: Soft Verified Generation and Audit

SERA’s data generation and analysis core consists of the Soft Verified Generation (SVG) protocol and, for auditing applications, a demand-driven, memory-augmented LLM agent pipeline.

Soft Verified Generation (SVG):

SVG is a two-step agent rollout, leveraging teacher models to generate and “soft-verify” data.

SVG Algorithm Overview (Condensed):

1. First Rollout: For each target function, sample a high-level bug/desire prompt. The agent generates an edit trajectory.
2. PR Synthesis: Construct a synthetic pull request (PR) description from the generated edit, modeling real code review workflows.
3. Second Rollout: Re-prompt the agent with the synthetic PR to independently generate the patch a second time.
4. Soft Verification: Compute

$$r = \frac{|\mathrm{lines}(P_2) \cap \mathrm{lines}(P_1)|}{|\mathrm{lines}(P_1)|}$$

Accept the trajectory if $r$ meets or exceeds a threshold (soft or hard match).

The pipeline is agnostic to the presence of tests, enabling on-demand synthesis of tens of thousands of trajectories per repository at dramatically reduced cost (Shen et al., 28 Jan 2026).

Soft-Verified Auditing (RepoAudit):

The SERA auditing design, exemplified by RepoAudit, extends these principles to program analysis. Here, soft verification is implemented via symbolic and control-flow checks rather than code execution. Core elements include:

• Data-flow facts: Formalized as $u@s_1 \mapsto v@s_2$, indicating value propagation along feasible paths.
• Path conditions: $$\Phi(p) = \phi_1 \land \phi_2 \land \dots \land \phi_n$$, encoding branch predicates along a program path.
• Validator: Executes lightweight SMT satisfiability checks on path conditions; bugs are discarded if $\mathrm{isSat}(\Phi(p))$ is false.

This architecture is modular: Initiator components select analysis seeds, Explorers conduct LLM-guided fact extraction, Validator modules filter hallucinated bugs, and a Reporter aggregates the results. Caching via agent memory ensures sublinear complexity with respect to re-analysis (Guo et al., 30 Jan 2025).

3. Training Procedures and Model Architecture

SERA agents are instantiated by supervised fine-tuning (SFT) open-weight LLMs (e.g., Qwen 3-32B) on synthetic SVG data. The typical workflow involves:

• Data Composition: ~200,000 agent trajectories from the SVG pipeline, filtered to a context of 32K tokens, with careful management of truncation ratio to preserve chain-of-thought reasoning.
• Loss Function: Standard autoregressive next-token cross-entropy:

$$\mathcal{L} = -\sum_{t=1}^T \log p_\theta(x_t \mid x_{<t})$$

• Training Regime: 3 epochs, learning rate $1\,\mathrm{e}{-5}$, weight decay $0.01$.
• Computation: 40 GPU-days (≈\$2,000) is sufficient for producing state-of-the-art agents at 32B scale.

Serving and evaluation utilize scalable systems such as vLLM. Efficiency is central: compared to RL or prior synthetic methods, SERA reduces both data and training costs by orders of magnitude (Shen et al., 28 Jan 2026).

4. Empirical Results and Comparative Assessment

SERA achieves leading results among open-weight coding agents with substantial cost and compute advantages. Performance on SWE-bench Verified (32K context, mean ± std over 3 seeds) exemplifies the advances:

Model/Method	Params	Verification (32K)	Cost Ratio (vs. RL)
SERA-32B	32B (Qwen)	49.5% ±1.9	1x
SWE-smith	32B (Qwen 2.5)	32.6%	26x
SkyRL-32B (RL)	32B (Qwen)	39.4%	26x (SERA cheaper)
Devstral-Small-2	24B	50.0% ±1.3	40x+

Cost-per-trajectory to match SWE-smith’s benchmark is 57–115x lower for SERA. Scaling law analysis predicts closing the remaining gap with closed-weight models at a fraction of their training cost (Shen et al., 28 Jan 2026).

In code auditing (RepoAudit), SERA-style agents report precision of 78.43% and an average per-repository cost of \$2.54 (0.44h analysis), unequivocally outperforming pattern/hard-coded tools and monolithic prompt-based agents (Guo et al., 30 Jan 2025).

5. Ablations, Scaling Laws, and Specialization Dynamics

Comprehensive ablation studies support key aspects of SERA efficacy:

• Verification Thresholds: No significant gain is observed from hard (r=1) versus soft (0<r<1) verification; performance variations stay within 1–2% across verification modes.
• Truncation Management: Preservation of trajectory prefixes (high truncation ratio) is critical; indiscriminate truncation or random slicing degrades performance by ~5%.
• Data Filtering for Specialization: Exclusion of overly large patches or verbose tool outputs improves specialization on certain repositories (e.g., +4.4% on Sympy).
• First vs. Second Rollouts: Mixing both SVG rollouts yields better or comparable performance to increased data from a single rollout.
• Teacher Reasoning Traces: Ablative removal of agent chain-of-thought (reasoning) reduces performance from 41% to 23%, confirming the importance of rich intermediate supervision (Shen et al., 28 Jan 2026).

For repository specialization, SERA exhibits high sample efficiency: pure repo-specific data approaches teacher performance in one-fourth the required samples versus general data. One-way ANOVA confirms that specialization yields significant, consistent performance benefits (p<.01), with a clear scaling law across mixture proportions (Shen et al., 28 Jan 2026).

6. Architectural Patterns from Auditing: Agent Memory, Demand-Driven Analysis, and Soft Verification

RepoAudit generalizes SERA patterns for repository-level analysis, with salient architectural motifs:

• Demand-Driven Exploration: Function or value selection is guided by data-flow relevance to target properties, yielding efficient traversal of vast program graphs.
• Agent Memory: All intermediate analysis (per-function/value facts) are cached, avoiding exponential redundancy and enabling sublinear time complexity in large codebases.
• Soft Verification Layer: Each agent step is validated with control-flow ordering and SMT-based symbolic checks, counteracting LLM hallucinations iteratively rather than in a single pass.
• Prompt Abstraction: Modular prompt templates parameterize each semantic analysis task, enabling rapid support for new bug types or languages.

Complexity control is achieved through bounds on inter-procedural depth (≤4) and path fanout, and the architecture is extensible to new analyses through minimal seed specification (Guo et al., 30 Jan 2025).

7. Broader Implications and Directions

SERA democratizes coding agent deployment, especially for small teams and private codebases. SVG’s infrastructure-free design allows swift, unlimited training data generation for any repository, with empirical superiority on common model benchmarks. All code and data generation pipelines, model weights, and integration scripts are released, facilitating open research and experimentation.

Broader impacts include secure local adaptation for proprietary code, heightened reproducibility via open-source releases, and acceleration of research into scaling laws, RL-free agent learning, and repository-wide code analysis frameworks (Shen et al., 28 Jan 2026, Guo et al., 30 Jan 2025).

A plausible implication is that SERA’s principles—particularly soft verification and modular agent structure—offer a general blueprint for new classes of repository agents across the spectrum of code analysis, refactoring, and autonomous codebase auditing. Extensions involving richer validation oracles (e.g., dynamic testing), learned planner-controllers, and resource-aware prompting are immediately feasible within the demonstrated SERA patterns (Guo et al., 30 Jan 2025).