SWE-RM Verifier

• SWE-RM Verifier is a system that uses execution-free, outcome-supervised models to deliver graded feedback on candidate software agent trajectories.
• It employs LLM classifiers or formal verification pipelines with calibration metrics to improve testing, reward shaping, and performance scalability.
• The verifier reduces reliance on binary test cases by scoring multiple candidate solutions and optimizing reinforcement learning stability.

A SWE-RM Verifier is a software-reward-model-based system for evaluating candidate solutions generated by software engineering (SWE) agents. In the paradigm established by recent work, especially within agentic code-repair and code-modification contexts, the SWE-RM Verifier functions as an execution-free, outcome-supervised model that discriminates between successful and failing agent trajectories—serving as a fine-grained alternative or complement to binary, execution-based supervision such as unit testing. SWE-RM Verifiers can be instantiated as LLM classifiers or as formal verification pipelines, depending on the context. The approach generalizes from practical machine learning deployments (as in SWE-Gym and test-time scaling with LLM reward models) to formal modeling and program refinement settings (as in SMT-based property checkers or model-checking tools for weak memory models) (Pan et al., 2024, Shum et al., 26 Dec 2025, Wu et al., 2024, Colvin et al., 2018).

1. Conceptual Role and Motivation

The SWE-RM Verifier emerges from the limitations of execution-based feedback such as unit testing. Unit tests only provide sparse, binary signals and are prone to coverage gaps or misalignment with the true semantics of the task. In contrast, SWE-RM Verifiers trained on labeled trajectories can generate a graded, continuous score reflecting the probability or plausibility that a given agent trajectory (e.g., code patch and interaction trace) resolves the issue. This execution-free feedback mechanism supports denser and more discriminative supervision, is applicable when test suites are missing or inadequate, and can be used both for test-time candidate ranking and for reward shaping in reinforcement learning (Shum et al., 26 Dec 2025).

2. Model Architectures and Input Processing

In LLM-based instances (notably in SWE-Gym), the verifier adopts an architecture identical to the underlying agent model, typically a decoder-only Transformer (e.g., Qwen-2.5-Coder-Instruct with 32B parameters). The input (prompt) is constructed as:

• A fixed prefix encoding the task/problem statement.
• An interleaved sequence (trajectory) $\tau = [o_1, a_1, ..., o_n, a_n]$, where $o_k$ are observations (file diffs, test logs, errors) and $a_k$ are agent actions (edits, shell commands).
• A final question, e.g., "Did this patch fix the tests? Yes or No."

At inference, the model is prompted to output "Yes" or "No," which is used to derive a softmax-based success score $$s(\tau_k) = \hat{p}(\mathrm{Yes} \mid \tau_k) = \exp(l_{\text{yes}}) / (\exp(l_{\text{yes}}) + \exp(l_{\text{no}}))$$ across $K$ sampled candidates. The candidate with the highest score is selected ("best-of-K" inference) (Pan et al., 2024). Calibration and classification metrics (AUC, Expected Calibration Error) are crucial for downstream RL stability (Shum et al., 26 Dec 2025).

Alternative verifier instantiations leverage mixture-of-experts (MoE) architectures or formal verification engines. For instance, the SWE-RM model uses a sparse MoE Transformer with 30B parameters and 3B activated per token, producing logit outputs for the verification query appended to each trajectory (Shum et al., 26 Dec 2025).

3. Training Methodology and Data

Training requires labeled agent trajectories indicating success or failure. These can be harvested from on-policy rollouts (current agent) and off-policy rollouts (other agents such as GPT or Claude). Datasets are balanced or normalized for the number of positives and negatives (e.g., in SWE-Gym, 1,318 positives and 1,318 negatives). The loss function is typically binary cross-entropy on the final classification token (e.g., "Yes"/"No"), with regularization via weight decay and, in many cases, parameter-efficient fine-tuning (e.g., LoRA with rank-64 adapters).

Optimization is performed using AdamW with hyperparameters such as learning rate, batch size, and context window length adapted to hardware constraints. Practical deployment usually targets GPUs such as the NVIDIA H100 (Pan et al., 2024).

Training data composition and balance directly affect robustness, AUC, and calibration. Empirically, a 2:1 positive:negative ratio (rather than 1:1) produces better calibration and test-time scaling, especially when pooling data from multiple agent policies and environments (Shum et al., 26 Dec 2025).

4. Inference-Time Scoring and Scaling

At test time, the SWE agent generates $K$ candidate trajectories per problem. The verifier computes softmax-normalized scores for each and selects the top-scoring candidate. This allows nearly logarithmic scaling of performance as $K$ increases, substantially exceeding the resolve rate of single-shot decoding (e.g., on SWE-Bench Verified, going from 20.6% for $K=1$ to 32.0% for $K=16$; pass@16 oracle is 42.8%) (Pan et al., 2024).

The architecture supports efficient, batch-wise scoring of candidate solutions. MoE incarnations further improve compute-efficiency, activating only a subset of model parameters per token (e.g., 3B out of 30B in Qwen3-30B-A3B), thus reducing inference cost without degrading calibration or TTS performance (Shum et al., 26 Dec 2025). For extremely long agent trajectories, support for context windows up to 256k tokens enables operation on complex, multi-edit or multi-round tasks.

5. Formal and Model-Checking Variants

In the context of systems verification (e.g., for secure firmware as in Arm CCA's Realm Management Monitor), the term "SWE-RM Verifier" extends to formal property-checking pipelines built atop SMT-based model checkers like ESBMC. Here, the verification process encodes pre/post-conditions, invariants, and architectural safety properties as SMT formulas over the program's Single Static Assignment (SSA) form. The verifier handles parsing, symbolic execution, path constraint construction, and SMT-solving, returning counterexamples for any property whose unsatisfiability cannot be proved within bounded resources (Wu et al., 2024).

Key steps in this workflow include:

• Harness generation from machine-readable specs with non-deterministic input/state.
• Loop-bound configuration for tractable unrolling.
• Assertion encoding for invariants, control-flow, and isolation.
• Hybrid multi-property grouping for efficiency.

The ESBMC verifier, for example, discovered numerous counterexample traces, including pointer overflows, granule refcount violations, and faulty privilege boundary checks (Wu et al., 2024). A similar paradigm applies to weak-memory models: program actions are represented in a wide-spectrum syntax, operational semantics (including instruction reordering) are encoded in Maude, and program refinement is checked with reachability analysis and counterexample reporting (Colvin et al., 2018).

6. Practical Integration and Best Practices

Practical deployment of a SWE-RM Verifier, whether LLM-based or formal, involves careful attention to:

• Context window length, to capture complete agent trajectories.
• Data balance and diversity (policy/source mixtures) during training.
• Calibration: model confidence must align with empirical correctness, as miscalibration correlates with RL instability and TTS failure.
• Efficient compute scaling: MoE and adapter-based fine-tuning reduce hardware demands.
• Workflow compatibility: Verifiers can be invoked with any system that emits a textual trajectory or standalone patch, allowing drop-in substitution for costlier test suite runs.
• CI/CD integration for systems verification: smoke tests, nightly regression, trace archiving, and automatic harness regeneration (Wu et al., 2024).

A tabular summary of empirical performance for a representative LLM-based SWE-RM Verifier is given below:

Configuration	Resolve Rate (%)	Scaling Behavior
Single-shot agent (K=1)	20.6	Baseline
Pass@16 (oracle)	42.8	Upper bound
Best@16 (verifier)	32.0	Log-linear in K
Off-policy only (K=8)	~22	Limited plateau
Off+on policy mix (K=8)	~27	Optimal at K

The improvement from "best-of-K" selection over raw generation is robust across scaffolds and tasks, with performance gains of 10–12 percentage points for agent resolve rate (Pan et al., 2024).

7. Impact, Limitations, and Future Directions

SWE-RM Verifiers effect a substantial paradigm shift from execution-based, binary pass/fail assessment to discriminative, execution-free reward modeling in software agent evaluation and RL. The ability to provide rich reward signals without reliance on test-case execution enables scalable offline selection, more stable RL training, and performance gains across both code repair and code synthesis benchmarks. MoE-based verifiers also offer significant reductions in compute requirements while scaling context length (Shum et al., 26 Dec 2025).

Major limitations include the dependency on high-quality, well-labeled trajectory datasets, residual label noise (e.g., misclassified success/failure in fail2pass regimes), modest coverage of multi-file and multi-language tasks, and infrequent explicit consideration of post-hoc calibration (e.g., temperature scaling). Future research aims at comparing MoE and dense model efficacy, incorporating additional forms of tool interaction (such as debuggers and linters), generalizing to new programming languages, and extending formal verification with parameter-efficient adapters and hybrid symbolic-ML reward models (Shum et al., 26 Dec 2025, Wu et al., 2024, Colvin et al., 2018).