OpenSWE: Open-Source Software Engineering Framework

• OpenSWE is an open-source suite of frameworks that provides reproducible environment synthesis, comprehensive datasets, and benchmark evaluations for software engineering and scientific waveform extraction.
• It automates repository exploration, Dockerfile synthesis, and evaluation scripting through a multi-agent pipeline ensuring robust test analyses and controlled filtering.
• Leveraging large-scale datasets and advanced training methodologies, OpenSWE significantly enhances LLM performance on autonomous software engineering tasks and numerical relativity simulations.

OpenSWE refers to multiple distinct open-source frameworks that advance agentic software engineering, large-scale benchmark generation, executable environment synthesis for code agents, and scientific waveform post-processing. The predominant usage since 2026 is for frameworks enabling scalable, reproducible, and challenging environments and datasets for training and evaluating LLMs on software engineering (SWE) tasks. A secondary usage, predating the LLM era, refers to the Python Open Source Waveform Extractor for numerical relativity simulations. This entry focuses on the technical underpinnings, methodologies, datasets, infrastructure, and significance of OpenSWE, covering all major usages and implementations.

1. OpenSWE: Large-Scale Executable Environment Synthesis

OpenSWE is a fully transparent, open-source pipeline for synthesizing large suites of executable environments to facilitate research and model development in autonomous SWE agents (Fu et al., 13 Mar 2026). Targeting Python repositories, OpenSWE automates the discovery, Dockerization, evaluation scripting, and curated filtering of real-world GitHub projects.

The system is organized as a multi-agent pipeline comprising:

• Repository Exploration: A context_retrieval_agent inspects repositories to extract dependency versions and test/build directives, using minimal API calls (browse, search, digest).
• Dockerfile Synthesis: A write_dockerfile_agent leverages pre-built openswe-python images (Python 2.7, 3.5–3.14) and strict prompts to ensure deterministic, reproducible builds, prohibiting fragile operations (e.g., "RUN pytest") within Dockerfiles.
• Evaluation Script Generation: The write_eval_script_agent generates bash scripts to activate the testbed environment, apply patches, run targeted tests, and emit structured logs, all wrapped in template tags for reproducible evaluation.
• Iterative Test Analysis: The test_analysis_agent determines environment validity by running "testOnly" and "testWithFix" on each instance, requiring rc_0 ≠ 0 (pre-patch fail) and rc_1 = 0 (post-patch pass).
• Difficulty-Aware Filtering: Unsolvable (rc_1 ≠ 0 post-patch) and trivial (100% agent success) environments are pruned. Only environments with intermediate difficulty (e.g., 25–50% trajectory success under sampling) are retained for maximum learning efficiency.

Deployment across a 64-node cluster enabled the synthesis of 45,320 validated Docker environments (from 12,802 repositories and 572,114 PRs), yielding ∼9,000 quality-assured problem instances and ∼13,000 curated agentic trajectories. The entire process incurred a total infrastructure and curation investment of ~$1.47M (Fu et al., 13 Mar 2026).

2. OpenSWE Datasets and Agentic Trajectories

Complementing the environment suite, Open-SWE-Traces provides large-scale, high-fidelity agentic trajectories for software-engineering agents (Ahmad et al., 14 Jun 2026). This resource aggregates 207,489 resolution trajectories across nine programming languages using two agent harnesses (OpenHands and SWE-agent). Trajectories derive from approximately 20,000 unique, permissively-licensed PRs.

The trajectory generation employs a dual-teacher pipeline:

• Minimax-M2.5 produces "thinking" trajectories, where each agent turn includes explicit step-wise chain-of-thought in the reasoning_content field. These traces favor longer, more detailed per-turn rationales.
• Qwen3.5-122B issues "non-thinking" trajectories, focusing on actions and tool usage with truncated reasoning steps, resulting in higher tool use diversity.

Quality filters eliminate incomplete, trivial, and malformed action logs, imposing schema standardization and security pruning (e.g., removal of forbidden git command exposure). The resulting corpus supports both imitation learning and hybrid distillation to facilitate long-horizon, tool-based agent behaviors.

Key statistics: | Language | #Trajectories | |--------------|---------------| | Python | 48,179 | | Go | 46,834 | | TypeScript | 36,897 | | JavaScript | 29,360 | | Rust | 21,735 | | Java | 13,176 | | PHP | 10,323 | | C | 683 | | C++ | 302 |

(Ahmad et al., 14 Jun 2026)

3. Benchmark Generation and Evaluation Frameworks

OpenSWE integrates with and extends scalable benchmark generation pipelines such as SWE-Bench++ (Wang et al., 19 Dec 2025), which supersede manual-dataset-based benchmarks by programmatically turning raw PRs into reproducible tasks. SWE-Bench++ operates in four phases:

1. Programmatic Sourcing: Filters github PRs for eligible repositories and issues.
2. Environment Synthesis: Constrained-synthesis of language-specific Dockerfiles using vetted templates and adaptive LLMs.
3. Oracle Extraction: State-differential evaluation using "Base," "Before," and "After" tree snapshots, with hybrid log parsers (deterministic and neural) for result extraction.
4. AutoQA Verification: Multilayer quality assurance for determinism, oracle consistency, semantic alignment (LLM-Judge), and false-negative filtering, producing 11,133 high-fidelity tasks across 3,971 repositories.

SWE-Bench++ provides pass@k metrics for cross-model comparability. Hint-guided demonstration synthesis supplies high-signal training trajectories for frontier tasks.

(Wang et al., 19 Dec 2025)

4. Model Training Methodologies and Outcomes

Leveraging OpenSWE and Open-SWE-Traces datasets, leading agentic LLMs (e.g., OpenSWE-32B/72B, Qwen3-30B-A3B series) undergo supervised fine-tuning on these high-quality environments and trajectories.

• Training is conducted with SFT (supervised fine-tuning) using long (128k–262k token) context lengths, batch sizes (e.g., 32), and regularization schedules (cosine decay, warmup).
• Agents are scaffolded through OpenHands and SWE-Agent, supporting dual-mode inference ("thinking" and "no_think" modes; each with different sampling parameters).
• Evaluation is performed on multiple benchmarks (SWE-bench Verified, Multilingual, Pro), reporting pass@1 (resolution rate) results.

Representative results: | Model | Scaffold | Pass@1 (%) | |----------------|------------|-----------:| | OpenSWE-32B | SWE-Agent | 62.4 | | OpenSWE-72B | SWE-Agent | 66.0 | | Base Qwen2.5-32B | SWE-Agent| 57.8–61.4 |

Data scaling analysis supports log-linear pass@1 improvement with no saturation at current data/model sizes. Exclusive training on OpenSWE improves pass@1 by 12.2 points (32B scale) over prior datasets. Out-of-domain improvements include +12 points on MATH-500 and +5 points on science benchmarks at 72B scale.

Pass@1 is defined as: $$\mathrm{Pass@1} = \frac{\#\text{SolvedTasks}}{\#\text{TotalTasks}} \times 100\%$$ (Fu et al., 13 Mar 2026, Ahmad et al., 14 Jun 2026)

5. OpenSWE Infrastructure, Cost, and Accessibility

OpenSWE deployments use a distributed cluster paradigm, with infrastructure managed by systemd, file-based message queues, and strict observability (Prometheus/Grafana). Supported base images, orchestration scripts, and multi-agent synthesis workflows are released at [https://github.com/GAIR-NLP/OpenSWE] under permissive open-source licenses.

Cost analysis for a 10-day run:

• GPU: $865K
• CPU: $7K
• Storage: $13K
• Other: $6K
• Curation: $576K

Total: ≈$1.47M, averaging$19.66 per environment.

All environments, Dockerfiles, scripts, and code are provided for direct academic reproducibility and downstream extension. The pipeline is readily adapted to new repositories and may ultimately generalize to additional language ecosystems with tooling extensions.

(Fu et al., 13 Mar 2026)

6. Complementary Usage: OpenSWE in Scientific Waveform Extraction

An earlier, distinct usage of the OpenSWE name is the Python Open Source Waveform Extractor (POWER), focused on post-processing for numerical relativity (Johnson et al., 2017). POWER sits downstream of the Einstein Toolkit, reading simulation outputs and computing gravitational wave strain at null infinity.

Technical summary:

• Written in pure Python, built on NumPy/SciPy, compatible with multiprocessor HPC environments.
• Implements fixed-frequency integration (FFI) of Newman–Penrose scalar $\Psi_4$to recover$h_+, h_\times$$, with low-frequency filtering for stability.</li> <li>Strain is extrapolated to infinity using$$1/r$$fits for phase and amplitude across multiple extraction radii.</li> <li>Includes tools for both post-hoc waveform extraction and real-time simulation monitoring.</li> </ul> <p>Performance relative to SimulationTools: maximum waveform amplitude error$$\Delta h \lesssim 10^{-7}$, phase error$\Delta\phi \lesssim 10^{-3}$, with$\mathcal{O}(10)$ minutes per full waveform extraction on 8 cores.

  (Johnson et al., 2017)

  7. Limitations and Future Research Directions

  OpenSWE, in its modern incarnation, is currently limited to Python for full environment coverage. Extension to other languages will require new base images and harnesses, as well as addressing build-system and dependency fragmentation. The high compute cost per instance remains an obstacle to rapid benchmark refreshes. Future plans include:

  • Curriculum learning via refined difficulty models,
  • Integration of surrogate (learned) execution environments to lower synthesis costs,
  • Extension to more languages and broader code domains,
  • More sophisticated reasoning trace structures and real-world multi-agent/continuous-integration scenarios,
  • Enhanced support for flaky or non-deterministic test environments.

  A plausible implication is that OpenSWE and its associated datasets and pipelines will form the backbone of future open, credible evaluation and training standards for autonomous software engineering agents, driving both domain and generalization improvements in next-generation LLMs.

  (Fu et al., 13 Mar 2026, Ahmad et al., 14 Jun 2026, Wang et al., 19 Dec 2025, Johnson et al., 2017)