- The paper presents a multi-agent system, Agyn, which models software issue resolution as a collaborative task involving a manager, researcher, engineer, and reviewer.
- The system, designed for production use, achieved a 72.4% task resolution rate on the SWE-bench 500 without benchmark tuning, demonstrating significant improvements over single-agent systems, such as a 7.4% improvement over mini-SWE-agent under comparable model settings.
- Agyn uses a GitHub-native workflow with isolated execution environments, enhancing adaptability and traceability, and addresses the mismatch between benchmark conditions and production workflows by dynamically coordinating agent actions and responses.
Overview
"Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering" (2602.01465) presents a fully automated multi-agent system that models software issue resolution as an organizational process rather than a monolithic or pipeline-based task. The system is built on agyn, an open-source platform for configuring agent teams, and assigns specialized agents to coordination, research, implementation, and review roles, each operating in an isolated execution environment. A central claim of the paper is that the system was designed and deployed for real production use, without any tuning for SWE-bench; evaluation on SWE-bench 500 is conducted post hoc. Under this setting, the system resolves 72.4% of tasks, outperforming single-agent baselines built on comparable GPT-5–family models.
Positioning relative to prior work
The paper situates itself against three families of approaches to autonomous software engineering. Single-agent scaffolds such as SWE-agent (Yang et al., 2024) and the deliberately minimal mini-SWE-agent place all responsibilities—issue understanding, exploration, editing, validation—on one agent within a fixed interaction loop. Generalist execution platforms such as OpenHands (Wang et al., 2024) provide sandboxed runtimes but largely preserve single-agent or loosely delegated execution. Self-evolving agents such as Live-SWE-agent (Xia et al., 17 Nov 2025) expand the action space dynamically but retain a monolithic execution model. Pipeline-based multi-agent systems, including TRAE (Team et al., 31 Jul 2025) and Prometheus (Chen et al., 26 Jul 2025), distribute work across stages but fix the number of stages and interactions in advance.
Agyn differs on two axes. First, coordination is open-ended: a manager agent governs the process dynamically, so the number of research, implementation, and review cycles is not predetermined. Second, the authors emphasize that their system was not benchmark-tuned, addressing what they identify as a gap in prior evaluations—most systems are refined against benchmark feedback, providing limited evidence about transfer to production constraints such as cost efficiency, traceability, reliability, and failure recovery.
System architecture
Roles and model allocation
The team comprises four roles. The manager coordinates the process through a dedicated manage tool, maintaining task state and deciding which agent to invoke next; it does no direct implementation work. The researcher explores the repository, identifies root causes, and produces structured task specifications. The engineer implements changes and runs tests iteratively. The reviewer evaluates pull requests via GitHub's review mechanism, either approving or requesting changes—the explicit approval serves as the system-level acceptance signal.
Model allocation is role-specific: manager and researcher use GPT-5 (medium reasoning), while engineer and reviewer use GPT-5-Codex (medium reasoning). This reflects the observation that exploratory tasks benefit from stronger reasoning and larger context, whereas implementation favors smaller code-specialized models for cost efficiency and fast feedback cycles.
Execution environments
Each agent operates in its own isolated environment with shell access. Beyond git, gh, and the Nix package manager, agents receive no preconfigured tooling. The authors report that minimally provisioned environments outperformed predefined ones, since several benchmark tasks required adjusting dependency versions (e.g., downgrading Python interpreters or libraries) that fixed environments could not accommodate. To bound context growth, shell outputs exceeding 50,000 tokens are redirected to files that agents inspect selectively.
GitHub-native workflow
Coordination is grounded in GitHub artifacts: issues, pull requests, and inline reviews. Each agent role has a distinct GitHub account so actions appear as separate contributors. Notably, the authors found that MCP-based GitHub API integration inflated context with auxiliary metadata and degraded review-heavy workflows; they migrated most interactions to the compact gh CLI and developed a custom extension, gh-pr-review, because gh does not adequately support reading and authoring inline reviews. This finding—that API verbosity materially harms agent performance—is a concrete design lesson for agent infrastructure.
Automation-first interaction design
Because LLMs are fine-tuned for conversational settings, they exhibit behaviors incompatible with autonomous execution (requesting approval, waiting for confirmation). The system counters this at multiple levels: prompts discourage approval-seeking; completion requires reviewer-approved pull requests rather than natural-language completion messages; only the manager may interact with the user, and must signal completion via a dedicated finish tool, with non-tool outputs triggering instructions to continue; a separate send_message tool allows informational output without breaking control flow. Automatic context summarization replaces older history with compact summaries once a symbol budget threshold is reached.
Results
On SWE-bench 500 (text-only), executed end-to-end without human intervention, the system resolves 72.4% of tasks. Among systems using GPT-5–family models:
| System |
Score (%) |
| Agyn + GPT-5 / GPT-5-Codex (medium reasoning) |
72.4 |
| OpenHands + GPT-5 |
71.8 |
| mini-SWE-agent + GPT-5.2 (high reasoning) |
71.8 |
| mini-SWE-agent + GPT-5 (medium reasoning) |
65.0 |
The headline comparison is a 7.4 percentage-point improvement over mini-SWE-agent under comparable model configurations, which the authors interpret as evidence that organizational structure contributes beyond raw model capability. Two caveats temper this claim. First, mini-SWE-agent with Gemini 3 Pro Preview reaches 74.2%, exceeding the reported result with a different underlying model. Second, the authors themselves note that many leaderboard systems cluster tightly in the 70–75% range despite very different architectures, so small numerical differences should be interpreted cautiously. The strongest defensible claim is therefore competitiveness—not superiority—and the demonstration that a production-oriented, non-benchmark-tuned design lands within the leading cluster.
All forks, issues, pull requests, and agent communication traces are released publicly, enabling external inspection of coordination behavior—an artifact contribution that distinguishes this evaluation from score-only reporting.
Limitations and open questions
The paper candidly documents several limitations, most induced by the mismatch between production workflows and benchmark conditions:
- Trajectory dependence and infrastructure drift: early failures in environment setup, dependency installation, or test execution can derail entire runs. Many SWE-bench repositories rely on legacy tooling (e.g., deprecated GitHub Actions components) whose failures are incidental to the task. When such failures consistently disrupted execution, the authors explicitly instructed agents to deprioritize unrelated CI checks—a deviation from standard real-world practice that concedes the "production-faithful" framing is imperfect.
- Resumption from persisted state: when execution was interrupted by infrastructure failures, runs were resumed from persisted GitHub artifacts. While framed as reflecting practical workflows, this intervention means the evaluation is not strictly single-shot autonomous execution.
- Over-engineering: agents sometimes produced functionally correct but broader patches than the intentionally minimal gold patches, causing mismatches. The authors suggest benchmarks may reward under-specification rather than robustness.
- Resource constraints: evaluation ran on a single MacBook Pro with parallel tasks, encountering memory pressure and OOM failures under high concurrency.
- Long-running tests: tool-call time limits prematurely terminated expensive test suites, forcing ad hoc strategies (subsetting, batching, parallelizing) that complicated trajectories.
- Coordination semantics: early runs suffered stalls when the manager waited on agents that had already completed; explicit completion-signaling requirements were needed to stabilize collaboration.
Open problems identified include trajectory correction and rollback mechanisms, better environment and dependency abstractions, more expressive coordination primitives, and resource-aware scheduling for larger agent teams.
Conclusion
This paper demonstrates that structuring autonomous software engineering as a manager-coordinated team of role-specialized agents—with isolated environments, GitHub-native artifacts, and explicit review-based acceptance criteria—achieves competitive performance on SWE-bench 500 (72.4%) without benchmark-specific tuning. Its principal contribution is less the leaderboard position than the argument, supported by both results and production experience, that organizational design, communication contracts, and agent infrastructure are first-class determinants of autonomous software engineering performance alongside model capability.