SWE-AGI: Autonomous Software Engineering Agents

• SWE-AGI is a research program focused on autonomous software agents capable of multi-step repository-level reasoning and production-scale code synthesis from formal specifications.
• It evaluates agents through benchmarks like MoonBit and SWE-bench, emphasizing specification-driven construction, robust interfaces, and effective signal processing.
• The approach highlights the significance of precise interface design, multi-agent orchestration, and scalable training environments in realizing long-horizon software engineering.

SWE-AGI denotes a research program aimed at autonomous, general-purpose software engineering agents that operate at repository scope: they read issue reports or formal specifications, navigate and edit multi-file codebases, use external tools, execute builds and tests, and iterate until a patch is validated by executable feedback. In current arXiv usage, the term also names a specific open-source benchmark for specification-driven software construction in MoonBit, which evaluates whether agents can build production-scale systems from standards and RFCs under fixed APIs (Yang et al., 2024, Zhang et al., 10 Feb 2026). Across both senses, the central concern is not single-function code generation, but long-horizon software work under realistic environment, tooling, and verification constraints.

1. Conceptual scope and capability profile

SWE-AGI is defined in the recent literature by a cluster of capabilities rather than by a single model family. At minimum, the agent must perform repository-level reasoning, including fault localization across files, planning and executing multi-step edits, running programs and tests, interpreting failures, and recovering from erroneous intermediate states. The SWE-agent paper formalizes this regime as an interaction loop over a repository snapshot, a tool interface, and execution feedback, emphasizing long-horizon navigation, file editing, search, and reproducible sandboxed execution (Yang et al., 2024). The MoonBit benchmark titled "SWE-AGI" sharpens the same idea from the opposite direction: instead of fixing existing issues in mature repositories, it asks whether an agent can implement parsers, interpreters, binary decoders, and SAT solvers from authoritative specifications within a declaration-first scaffold (Zhang et al., 10 Feb 2026).

The benchmark formulation is notable because it isolates specification-driven construction from repository idiosyncrasies. It contains 22 production-scale tasks across seven specification families, with each task requiring roughly $10^3$–$10^4$ lines of core logic. MoonBit is used precisely because it is a nascent ecosystem and therefore reduces retrieval shortcuts and training-data overlap. Public APIs are frozen by declare in *_spec.mbt files, public tests are only about 10% of the full suite, and scoring is done only on final submissions against hidden private tests (Zhang et al., 10 Feb 2026). This makes SWE-AGI, in the benchmark sense, an evaluation of architectural reasoning, specification operationalization, and interface compliance rather than benchmark-specific prompt engineering.

A recurrent implication across papers is that software-engineering generality is multidimensional. It includes language generality, robustness to heterogeneous build and test stacks, autonomy in environment setup, and reliability of evaluation. SWE-rebench V2 makes this explicit by motivating itself as a response to a bottleneck in training general-purpose, autonomous software engineering agents: the lack of large, executable, reproducible, and diverse task environments with reliable test-based rewards (Badertdinov et al., 27 Feb 2026). Oracle-SWE further decomposes the problem into reusable information signals—Reproduction Test, Regression Test, Edit Location, Execution Context, and API Usage—showing that a large part of apparent agent competence depends on the quality of such signals rather than on patch generation alone (Li et al., 9 Apr 2026).

2. Interface design, decomposition, and organizational architectures

A major line of work treats SWE-AGI as an interface and systems-design problem. SWE-agent argues that LLMs are brittle when exposed directly to human-oriented shells and editors, and introduces an agent-computer interface with a small set of SWE-centric commands for viewing files, grounded line-range edits, search, execution, and submission. On SWE-bench Lite, the paper reports 18.0% resolved with the full ACI, compared with 11.0% for a shell-only agent with demonstration; ablations attribute gains to the linting guardrail, summarized search, a 100-line file viewer, and truncating history to the last five observations (Yang et al., 2024). The core claim is that action-space shaping, feedback design, and reversible editing materially affect repository-scale performance even when the underlying model is unchanged.

SWE-Edit generalizes this argument by identifying a "context coupling" problem in standard editing interfaces. In its formulation, code inspection, plan formation, and edit execution are entangled within a single context window, so exploratory views pollute the same context that must later emit exact edit syntax. SWE-Edit separates these roles into a Viewer, which returns task-relevant ranges as JSON line spans, and an Editor, which translates high-level edit instructions into exact-match search-replace or whole-file rewrites in a clean context. On SWE-bench Verified, the combined Viewer+Editor system reaches 72.0% resolved, compared with 69.9% for the baseline scaffold, while reducing total cost from \$243.7 to \$200.1 and increasing edit success from 93.4% to 96.9% (Zhang et al., 28 Apr 2026). The paper also reports that the Viewer returns 39.7% of file content on average, corresponding to a 60.3% reduction in context passed to the main agent.

Another architectural direction treats SWE-AGI not as a monolithic agent but as an organizational process. Agyn instantiates four first-class agents—manager, researcher, engineer, reviewer—each with isolated sandboxes, role-specific tools, and centralized communication via the manager. It was designed for production use rather than benchmark tuning, yet resolves 72.4% of tasks on SWE-bench 500 in post hoc evaluation (Benkovich et al., 1 Feb 2026). The methodological contribution is that software engineering is modeled as analysis, task specification, pull-request creation, and iterative review, with objective completion requiring explicit PR approval by the reviewer. This suggests that role separation, review gates, and artifact-centric coordination are not merely engineering conveniences but part of the capability substrate for autonomous software work.

These results collectively undermine a common simplification: that software-engineering autonomy is primarily a matter of increasing base-model size. The evidence instead points to strong dependence on interface granularity, context control, edit reliability, and workflow structure (Yang et al., 2024, Zhang et al., 28 Apr 2026, Benkovich et al., 1 Feb 2026).

3. Training substrates, executable environments, and trajectory corpora

The most persistent systems bottleneck in SWE-AGI research is the scarcity of executable, reproducible, large-scale training environments. SWE-Gym was the first public environment explicitly built for training SWE agents, with 2,438 real-world Python task instances, each including a codebase snapshot, a pre-built executable runtime, expert-validated unit tests, and a natural-language issue. It also released SWE-Gym Raw with 66,894 issue instances from 358 repositories without executable environments (Pan et al., 2024). The same paper showed that rejection-sampling fine-tuning on successful trajectories yields large gains for open-weight agents: on SWE-bench Verified, OpenHands with Qwen-2.5-Coder-Instruct-32B improved from 7.0% to 20.6% resolved, and verifier-based reranking pushed Best@K to 32.0% at $K=16$ (Pan et al., 2024).

Subsequent work expanded this substrate along three axes: scale, multilinguality, and environment automation. R2E-Gym introduced 8,135 executable SWE tasks across multiple Python repositories, with a decontaminated training subset of 4,578 tasks. Its SweGen pipeline procedurally constructs environments from commits by building Docker images, detecting or generating fail-to-pass tests, and back-translating tests and commit context into issue descriptions. With supervised fine-tuning on 3,321 successful trajectories from 2,048 unique tasks, the resulting 32B agent reached 34.4% pass@1 on SWE-Bench Verified; hybrid execution-based and execution-free verifiers then reached 51.0% Best@26 (Jain et al., 9 Apr 2025).

SWE-Next approached the same scaling problem through execution-grounded mining of merged pull requests and reusable repo-quarter profiles. In a single run, it processed 3,971 Python repositories and 102,582 candidate commit pairs, retaining 2,308 self-verifying instances in 30 hours with 639 GB of environment storage. The selection rule required strict test improvement without regressions, formalized as $\Delta T_{\text{pass}} > 0$ and $\Delta T_{\text{fail}} \le 0$, or equivalently $|\mathrm{Improved}| > 0$ and $|\mathrm{Regressed}| = 0$ at the per-test level (Liang et al., 21 Mar 2026). The system also enforced submit gating: a rollout counted only if it produced a non-empty code diff and executed at least one test.

SWE-rebench V2 generalized the environment problem beyond Python. It introduced a language-agnostic automated pipeline with five stages—Preliminary Collection, Setup Synthesis, Execution-based Validation, Issue Clarity Filtering, and Metadata Enrichment—followed by PR-based Task Expansion. The resulting executable set contains 32,079 tasks from 3,617 repositories across 20 languages, with pre-built images and per-repository install_config.json; an additional 120,000+ PR-derived tasks reuse those install and test recipes (Badertdinov et al., 27 Feb 2026). The same paper explicitly frames this as missing substrate for RL toward SWE-AGI, and defines the repository-level issue-resolution episode in standard RL terms with a test-derived reward and objective

$$J(\theta)=\mathbb{E}\Big[\sum_{t=0}^{T}\gamma^t r_t\Big].$$

A complementary line of work focuses not on environments but on trajectory scale. SWE-ZERO to SWE-HERO released 300k execution-free SWE-ZERO trajectories and 13.2k execution-backed SWE-HERO trajectories distilled from Qwen3-Coder-480B-A35B-Instruct, using a two-stage supervised fine-tuning recipe. Open-SWE-Traces then pushed this trajectory-centric approach to 207,489 agentic trajectories spanning nine programming languages and two harnesses, with dual-mode synthesis from Minimax-M2.5 "thinking" traces and Qwen3.5-122B "non-thinking" traces (Ludwig et al., 2 Apr 2026, Ahmad et al., 14 Jun 2026). These corpora shift the emphasis from isolated fixes to long-horizon interaction distributions, including exploration, tool use, edits, tests, and failures.

4. Benchmarks, multilingual evaluation, and empirical boundary conditions

The benchmark landscape around SWE-AGI now separates along at least three regimes: repository issue resolution, multilingual issue resolution, and specification-driven software construction. The original SWE-bench family remains the dominant execution-based repository benchmark for Python. SWE-agent reported 12.47% resolved on SWE-bench full and 18.00% on SWE-bench Lite with GPT-4 Turbo, compared with much weaker non-interactive RAG baselines (Yang et al., 2024). Open-weight training pipelines later raised that range substantially: SWE-HERO-32B achieved 62.2% on SWE-bench Verified, Open-SWE-Agent reached 61.7% on SWE-bench Verified and 57.1% on SWE-bench Multilingual, and SWE-TRACE-30B with heuristic-guided test-time scaling reached 71.2% on SWE-bench Verified (Ludwig et al., 2 Apr 2026, Ahmad et al., 14 Jun 2026, Han et al., 16 Apr 2026). Agyn's 72.4% on SWE-bench 500 is numerically similar, though it is reported under a different organizational setup and should not be interpreted as a directly controlled comparison (Benkovich et al., 1 Feb 2026).

The multilingual regime reveals a sharper generalization gap. Multi-SWE-bench contains 1,632 manually verified instances from 39 repositories across Java, TypeScript, JavaScript, Go, Rust, C, and C++, curated from 2,456 candidates by 68 expert annotators (Zan et al., 3 Apr 2025). It explicitly excludes Python during construction and uses full-suite execution under three patch configurations—Run.log, Test.log, and Fix.log—to verify bug-fix signal. The empirical pattern is severe cross-language degradation: with MopenHands, Claude-3.7-Sonnet reached 52.20% on Python but only 21.88% on Java, 2.23% on TypeScript, 5.06% on JavaScript, 7.48% on Go, 15.90% on Rust, 8.59% on C, and 14.73% on C++ (Zan et al., 3 Apr 2025). Multi-SWE-bench therefore turns multilinguality from a desideratum into a measurable failure mode.

The MoonBit benchmark titled "SWE-AGI" defines a different boundary condition. It contains 22 tasks across seven specification families, including Pug, jq, YAML, TOML, HTML5, C99, Lua, ECMAScript, Python and Scheme subsets, binary decoders such as ZIP and WebAssembly, URL and URI parsers, and a CDCL SAT solver (Zhang et al., 10 Feb 2026). Across frontier models, gpt-5.3-codex achieved 19/22 tasks overall, with 6/6 easy, 8/8 medium, and 5/8 hard; claude-opus-4.6 achieved 15/22. The difficulty gradient is pronounced: all easy tasks are solved by frontier models, but performance degrades sharply on hard, specification-intensive systems such as HTML5, ECMAScript subsets, and CDCL (Zhang et al., 10 Feb 2026). The benchmark's behavioral analysis further reports that on hard tasks the share of Read actions becomes dominant—64.6% for gpt-5.2-codex, compared with 41.4% for gpt-5.3-codex—suggesting that codebase comprehension rather than code emission becomes the limiting factor at production scale.

A plausible implication is that repository issue resolution and specification-driven construction are probing adjacent but non-identical aspects of SWE-AGI. The first emphasizes repair under inherited architecture and tests; the second emphasizes architectural synthesis under formal requirements. Current systems show substantial but incomplete competence in both regimes (Yang et al., 2024, Zhang et al., 10 Feb 2026).

5. Information signals, rewards, and inference control

A consistent finding across recent work is that SWE-AGI performance depends strongly on which intermediate signals are exposed to the agent and how they are operationalized. Oracle-SWE isolates five signals—Reproduction Test, Regression Test, Edit Location, Execution Context, and API Usage—and measures their ideal contribution under oracle extraction. With all five combined, success rate reaches at least 97% across all model-benchmark pairs studied, indicating that these signals are near-sufficient for solving benchmark tasks (Li et al., 9 Apr 2026). The single-signal ordering is stable: Reproduction Test is strongest, followed by Execution Context and API Usage or Edit Location depending on benchmark, while Regression Test mainly acts as a guardrail.

The numerical effect of these signals is large even when extracted by a strong model and handed to a weaker base agent. On SWE-bench Verified, a GPT-5 extractor feeding GPT-4o yields gains of +26 percentage points for Reproduction, +14 for Context, +12 for Location, +8 for API, and +4 for Regression. Comparable patterns hold on SWE-bench Live and SWE-bench Pro for both Python and Go (Li et al., 9 Apr 2026). This directly motivates multi-agent or staged systems in which one component gathers high-value signals and another solves.

Verifier design plays a parallel role at test time. R2E-Gym distinguishes execution-based and execution-free verifiers, showing that each saturates around 42–43% individually on SWE-Bench Verified, but that a selective hybrid score

$s_k^H=\mathrm{Top}_n(s_k^{EF})+s_k^{EB}$

raises Best@K to 51.0% by combining model-based prefiltering with execution-based discrimination (Jain et al., 9 Apr 2025). The paper also formalizes test distinguishability and toxicity, emphasizing that generated tests may fail to separate correct from incorrect patches or may even rank incorrect patches above correct ones.

SWE-TRACE extends this logic into both training and inference. It constructs token-efficient shortest-path demonstrations by stepwise oracle verification, then trains a rubric-conditioned process reward model and a memory-augmented agent with a margin-separated reward that combines terminal execution correctness with rubric-based trajectory quality (Han et al., 16 Apr 2026). The reported gains are twofold: on SWE-bench Verified, SWE-TRACE-30B improves from 61.1% to 63.5% when moving from execution-only RL to PRM-based RL, and heuristic-guided test-time scaling raises it further to 71.2% while reducing environment executions and latency relative to standard parallel sampling (Han et al., 16 Apr 2026). The paper's broader claim is that sparse terminal rewards are insufficient for long-horizon agent optimization, and that dense, process-sensitive signals are necessary to reduce token bloat and reward hacking.

These results suggest that SWE-AGI is at least partly a signal-acquisition problem. Strong patch generators matter, but so do selective tests, native error stacks, precise localization, internal API discovery, and evaluators capable of ranking partial progress (Li et al., 9 Apr 2026, Jain et al., 9 Apr 2025, Han et al., 16 Apr 2026).

6. Efficiency, human collaboration, and unresolved problems

Two persistent misconceptions in the area are that autonomy alone is sufficient and that local deployment simply requires swapping in a smaller model. SWEnergy provides a quantitative rebuttal to the second claim. On fixed hardware with Gemma-3 4B and Qwen-3 1.7B Instruct, four agentic frameworks—SWE-Agent, OpenHands, Mini SWE Agent, and AutoCodeRover—show near-zero task resolution on SWE-bench Verified Mini, with only AutoCodeRover+Qwen resolving any tasks at approximately 6/150, or about 4% (Tripathy et al., 10 Dec 2025). Energy consumption is driven primarily by framework architecture: AutoCodeRover with Gemma consumed 216.21 kJ per run, versus 23.05 kJ for OpenHands with the same model, a 9.4x difference with no success advantage. For the only non-zero configuration, the estimated energy per resolved issue is about 5,210 kJ (Tripathy et al., 10 Dec 2025). The paper concludes that current frameworks designed for stronger LLMs fail to operate efficiently with SLMs because passive orchestration lets weak models loop, over-generate, and misuse tools.

The first misconception—fully autonomous resolution as the only relevant operating mode—is challenged by direct human-agent studies. "Sharp Tools" observed 19 developers using an in-IDE agent on 33 open issues in repositories they had previously contributed to. Excluding four externally abandoned issues, 16 of 29 were resolved, corresponding to 55%. Incremental resolution succeeded in 83% of issues, compared with 38% for one-shot delegation (Kumar et al., 14 Jun 2025). Participants followed the agent’s live execution in 84% of prompts, reviewed diffs in 67% of post-change responses, and wrote manual code in 14 of 33 issues; manual code writing correlated with 79% success versus 33% when not used (Kumar et al., 14 Jun 2025). The qualitative failure modes—trust calibration, tacit knowledge gaps, weak debugging collaboration, unsafe terminal side effects, verbosity, and sycophancy—indicate that high-end SWE systems remain socio-technical systems rather than purely autonomous solvers.

Several open problems recur across the literature. Multilingual generalization remains incomplete, especially for TypeScript, JavaScript, and low-level ecosystems with difficult build and runtime management (Zan et al., 3 Apr 2025). Environment construction remains costly despite strong progress in automated setup synthesis and reusable profiles (Badertdinov et al., 27 Feb 2026, Liang et al., 21 Mar 2026). Test suites remain imperfect oracles: they can be overly restrictive, underspecified, non-differentiating, or toxic (Badertdinov et al., 27 Feb 2026, Jain et al., 9 Apr 2025). Long-horizon planning still breaks under context overflow, multi-file coordination, and stateful or distributed systems (Han et al., 16 Apr 2026, Zhang et al., 10 Feb 2026).

Current evidence therefore supports a narrow but concrete conclusion. SWE-AGI is no longer a purely aspirational label: there are now executable training substrates at 2,438, 8,135, 32,079, and 120,000+ task scales; multilingual benchmarks with 1,632 verified instances; trajectory corpora with 207,489 traces; and systems that exceed 60% resolved on SWE-bench Verified under specific settings (Pan et al., 2024, Jain et al., 9 Apr 2025, Badertdinov et al., 27 Feb 2026, Zan et al., 3 Apr 2025, Ahmad et al., 14 Jun 2026). At the same time, the benchmark titled "SWE-AGI" shows that production-scale specification-driven construction remains substantially harder, especially where architectural synthesis and code reading dominate (Zhang et al., 10 Feb 2026). This suggests that the field has moved from proving that autonomous software engineering is possible in constrained forms to characterizing which substrates, signals, interfaces, and organizational designs are still missing for genuinely general software-engineering intelligence.