Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills

Abstract: LLM-driven software engineering agents have become a central testbed for real-world language-model capability, yet their training remains limited by the availability of high-quality SWE tasks. Existing synthetic data methods typically create tasks through fixed mutation or bug-injection procedures, making the resulting distributions largely independent of the agent's own weaknesses and training progress. We introduce Socratic-SWE, a closed-loop self-evolution framework that reuses the agent's historical solving traces as a source of training signal. Rather than treating traces only as evidence for reward computation, Socratic-SWE distills them into structured agent skills that summarize recurring failures and effective repair patterns. These skills then guide the generation of targeted repair tasks in real repositories. Candidate tasks are checked through execution-based validation and scored with a solver-gradient alignment reward, so that the retained tasks are both verifiable and useful for improving the Solver. The updated Solver produces new traces, enabling the task curriculum to adapt over successive rounds. Across SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0, Socratic-SWE consistently improves over self-evolving baselines under the same compute budget, reaching 50.40% on SWE-bench Verified after three iterations. These results suggest that solving traces can serve as a scalable substrate for self-evolving SWE agents.

• The paper introduces a novel closed-loop method that mines trace data to derive adaptive coding agent skills, enhancing task-specific problem synthesis.
• It employs a joint Generator-Solver design with gradient-aware curriculum optimization, leading to systematic improvements across multiple SWE benchmarks.
• Empirical results show marked performance gains over baselines and underscore the robustness and transferability of trace-derived skill learning.

Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills

Motivation and Problem Statement

The automation of complex software engineering (SWE) tasks represents a key challenge for agentic LLMs, mainly due to the long-horizon nature of repository-level problem solving and the scarcity of high-signal training data. RL-based methods have shown potential in SWE environments, but the core bottleneck is the lack of adaptive, high-quality, and task-aligned data. Traditional data generation pipelines synthesize tasks via static transformations, failing to exploit signals arising from the agent's own evolutionary trace data. This static setup leads to inefficient data curation, non-adaptive curricula, and early performance saturation, as illustrated in (Figure 1):

Figure 1: Comparison of SWE data synthesis paradigms: (a) traditional static pipeline (non-adaptive), (b) Socratic-SWE's closed-loop, trace-driven adaptive process.

Socratic-SWE proposes a closed, self-evolving loop that operationalizes the principle that the agent's longitudinal traces inherently encode its current weaknesses and frontier of capability. By explicitly mining such traces into structured skills and using them to guide repository-grounded task synthesis, Socratic-SWE induces an evolving curriculum that is natively aligned to the agent's actual error boundaries and potential for improvement.

Framework and Methodological Contributions

Socratic-SWE is instantiated as a joint Generator-Solver architecture where a shared LLM alternates between synthesizing skill-targeted SWE repair tasks and attempting to solve them in real repositories. The system's closed loop integrates three principal mechanisms:

1. Trace-Derived Agent Skill Registry: Solving traces are processed into structured, repository-grounded skills—each comprising a formalized name, natural language summary, applicability conditions, and an ordered, tool-level operation script. Successful traces yield new skills; critical failures contribute negative behavioral lessons.
2. Skill-Guided Task Construction: Candidate tasks are synthesized in the context of seed repositories, with the Generator explicitly conditioned on sampled skills from the skill registry and on the Solver's recent behavioral evidence. This process is strictly sandboxed and repository-grounded, ensuring instantiation fidelity and reproducibility.
3. Gradient-Aware Curriculum Optimization: Candidate tasks undergo a staged execution-grounded validation pipeline (format, grounding, execution, semantics). To maximize curriculum effectiveness, Socratic-SWE introduces a reward signal that prioritizes tasks whose induced Solver policy gradients are maximally collinear to gradients estimated on a held-out set of trusted validation tasks. This selection mechanism directly incentivizes generation of tasks driving generalizable and benchmark-aligned improvements.

The closed-loop interaction of the modules is visualized in (Figure 2):

Figure 2: Overview of the Socratic-SWE pipeline—traces induce skills, skills guide generation, and validation- and gradient-filtered tasks iteratively refine the curriculum and agent.

Optimization is performed with role-specific objectives: the Generator is updated with GRPO using the scalar gradient alignment reward, and the Solver is trained via GDPO, which decomposes feedback into full-suite pass, partial repair, and regression avoidance components with separate normalization. This yields more granular and effective optimization, especially under sparse or heterogenous feedback.

Empirical Results

Experiments evaluate Socratic-SWE and five modern self-improving baselines on four representative benchmarks: SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0. The main result is that Socratic-SWE demonstrates the strongest overall and per-benchmark improvements under a fixed compute and RL budget (36k training instances over 3 iterations), significantly surpassing self-evolving baselines (Figure 3):

Figure 3: Main performance comparison: Socratic-SWE dominates across four benchmarks, with marked improvements on both repository-level and terminal-agent tasks.

• On SWE-bench Verified: 50.40% (+7.80 over base agent, +3.40 over SSR)
• On SWE-bench Lite: 36.67% (+7.00)
• On SWE-bench Pro: 22.85% (+5.61)
• On Terminal-Bench 2.0: 14.61% (+4.50)

Self-play and teacher-guided coevolution baselines (e.g., R-Zero, Socratic-Zero) plateau early or show regression in later iterations. SSR, a recent self-play pipeline for SWE, continues to improve but is ultimately outpaced by Socratic-SWE, which displays superior transfer and capacity to exploit skill-targeted gaps, as opposed to the bug injection and random curriculum construction underlying SSR.

Long-term scaling (training up to 5 iterations) confirms that Socratic-SWE not only achieves a higher accuracy ceiling but also delays curriculum saturation relative to baselines, indicating that trace-derived skills and gradient-aware filtering enable deeper curriculum extraction from a fixed repository pool (Figure 4):

Figure 4: Socratic-SWE sustains improvement across more iterations; SSR saturates earlier and at a lower performance ceiling.

Analytical and Ablation Insights

Ablation studies demonstrate that each core Socratic-SWE component is critical for final performance:

• Removing the Skill Registry results in a 4.2% drop on SWE-bench Verified.
• Using hand-written instead of trace-distilled skills yields weaker performance, underscoring the value in empirically mined, agent-specific error abstraction.
• Replacing GDPO with a simple GRPO baseline impairs learning on partially-correct or heterogeneous reward signals.
• Skill extraction is robust to the choice of underlying extractor; even small LM-based extractors (Qwen3.5-9B) suffice.

Furthermore, reward function analysis reveals that gradient-aligned candidate selection substantially outperforms standard hardness- or uncertainty-driven rewards. This points to the critical role of directly coupling curriculum construction to empirical, benchmark-relevant policy improvement, rather than relying on indirect proxies like subjective task difficulty.

Limitations and Practical Implications

Socratic-SWE's design and validation are currently limited to a closed-world scenario with a fixed, curated repository pool. The method's efficacy in open-ended or non-deterministic environments is not established, and it assumes the presence of executable, non-spurious verifiers for all proposed tasks. Robustness with respect to validation set representativity and the generalization to non-standard SWE workflows remains to be proven.

However, within realistic SWE RL scenarios, Socratic-SWE establishes a practical protocol for scalable, data-efficient self-improvement. The framework's trace-skill-task loop and gradient-aligned reward strategy are directly portable to other agentic settings with verifiable long-format traces (e.g., code review, data wrangling, configuration management), where closed-loop, self-aligned curricula remain a central challenge.

Future Directions

Future work should aim to:

• Dynamically augment the repository pool for curriculum open-endedness.
• Investigate cross-domain transfer of mined skill registries.
• Automate or online the discovery and abstraction of skills from live traces.
• Generalize validation and reward structures for more heterogeneous or less well-instrumented environments (e.g., large-scale, multi-language, or legacy codebases).

Conclusion

Socratic-SWE provides a formal, empirically validated blueprint for trace-driven, self-evolving coding agents. By structuring historical interaction into skills, leveraging these for targeted task synthesis, and employing gradient-aware candidate selection, the framework achieves systematic and sustained improvements in agentic SWE settings, outperforming all competitive baselines. The results support continued exploration of trace-derived skill abstraction and gradient-coupled curricula in reinforcement learning for LLM-based agents.

Reference:

"Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills" (2606.07412)