- The paper introduces Envs-FORGE, a verifier-guided method that selects among six per-seed environment transformations using a MILP optimized for policy-relevant difficulty, skill coverage, and frontier proximity.
- Envs-FORGE jointly rewrites instructions, fixtures, solutions, tests, and executable environments, admitting only bundles that pass static checks and achieve perfect oracle verification reward.
- Envs-FORGE improves Pass@1 over no synthesis by 9.2 points on tb-core, 6.4 on tb-2.0, and 3.7 on SWE-bench Verified, while outperforming fixed synthesis recipes at matched export size and similar cost.
Envs-FORGE addresses a specific bottleneck in reinforcement learning for terminal agents: the training signal depends on executable environments whose difficulty must match the current policy, yet standard synthesis recipes apply a fixed rewriting strategy to every seed regardless of whether the policy would benefit from a harder, easier, or structurally different task. The paper proposes a prompting policy that converts verifier rewards into per-seed synthesis decisions, formulated as a mixed-integer linear program (MILP) over a discrete action space, followed by synchronized, gold-verified environment materialization (2608.14312).
Problem setting and motivation
A seed task is represented as a tuple (Ii,Di,Si,Ti,Ei) covering the instruction, fixture bundle, oracle solution, test suite, and executable environment. These artifacts are coupled: editing the instruction without updating the solution or tests yields ungradeable or inconsistent tasks. The paper's central claim is that the choice of how to transform a seed is a decision variable that fixed recipes—few-shot, Self-Instruct, Evol-Instruct—collapse into a single predetermined prompt. The comparison is deliberately controlled: all methods share the same artifact-generation and verification pipeline, isolating the prompting-policy choice itself.
This framing builds on difficulty-aware data selection work such as DART-Math, which allocates generation effort toward hard queries (Shao et al., 2024), and on terminal-synthesis systems that supply auxiliary components like runtime inversion, skill-graph sampling, and procedural generation (Lin et al., 11 Feb 2026, Zhu et al., 6 Feb 2026, Gandhi et al., 23 Jan 2026). Envs-FORGE operates at a distinct layer: rather than changing the synthesis substrate, it changes which transformation is applied per seed.
Action space and frontier-aware scoring
The method factors each synthesis decision into a projection a∈{increase,reduce,diversify} and an evolution direction d∈{in_depth,in_breadth}, yielding six candidate actions per seed. The seed pass rate p^i is estimated from verifier rewards under the current policy and held fixed during optimization. Post-projection pass rates are approximated by a fixed transfer prior,
p~i,a,d=clip(p^i+Δaγd,0,1),
with Δincrease=−0.25, Δreduce=0.25, Δdiversify=0, and γin_breadth=0.65 attenuating breadth moves. A Gaussian frontier score centered at τ=0.5 with a∈{increase,reduce,diversify}0 then ranks candidates. The authors are explicit that these priors are ranking heuristics, not empirical claims about materialized difficulty shifts—a concession that matters, since no ablation of prior sensitivity is reported.
The a∈{increase,reduce,diversify}1 projection is a notable design choice: it constructs bridge tasks from seeds the policy cannot solve at all (estimated pass rate 0), preserving the target skill chain while stripping infrastructure that masks the intended reasoning.
For each seed, binary variables a∈{increase,reduce,diversify}2 select one of six actions, with skill-activation variables a∈{increase,reduce,diversify}3 exposing the induced skill subgraph. The objective maximizes summed frontier scores, penalizes optional-node activation (a∈{increase,reduce,diversify}4), and penalizes unmet soft skill-coverage targets (a∈{increase,reduce,diversify}5, slack capped at a∈{increase,reduce,diversify}6). Constraints enforce per-seed cardinality, required/optional/forbidden skill consistency, and eligibility filters. The formulation requires no big-a∈{increase,reduce,diversify}7 constants and is solved exactly with SCIP via PySCIPOpt; the recorded run of 100 seeds (600 action variables) reports status optimal with objective 49.9104. A fallback path (exact enumeration up to 24 candidates, otherwise greedy) exists for portability but was not used in the reported run.
The formulation also yields a precise account of the baselines: fixed recipes correspond to restricted action masks—few-shot maps to a∈{increase,reduce,diversify}8, Self-Instruct to a∈{increase,reduce,diversify}9, and the Evol-Instruct variants fix the direction only. The authors correctly characterize these as approximate semantic correspondences rather than exact equivalences, which weakens strict causal attribution but strengthens the unified optimization view.
Verified synthesis pipeline
The selected action conditions joint rewriting of all five artifact classes under an explicit consistency contract: added requirements must appear in the instruction and be tested; the container may not package the oracle or hidden tests. Verification is two-layered—static schema, path-safety, length, and overlap checks, followed by executable verification in which the oracle solution must obtain reward 1 under the generated tests. Only such gold-verified bundles enter GRPO training (Shao et al., 2024). The paper is careful to distinguish this gold reward, which certifies internal task consistency, from the downstream policy reward used for difficulty estimation.
Empirical results
The main comparison trains Qwen 3.5 35B with GRPO, evaluating Pass@1 with test-derived rewards and no learned reward model. All four synthesis methods export exactly 100 verified environments at 2.27M–2.88M synthesis tokens, holding training-set size and operational scale fixed.
| Training source |
tb-core |
tb-2.0 |
SWE-bench Verified |
| Base (no synthesis) |
40.0% |
23.0% |
73.4% |
| Few-shot |
43.2% |
24.1% |
74.6% |
| Self-Instruct |
45.6% |
27.3% |
75.2% |
| Evol-Instruct |
46.8% |
25.6% |
75.8% |
| Envs-FORGE |
49.2% |
29.4% |
77.1% |
Envs-FORGE improves over Base by 9.2 points on tb-core, 6.4 on tb-2.0, and 3.7 on SWE-bench Verified, exceeding the strongest fixed-recipe baseline on each benchmark by 2.4, 2.1, and 1.3 points respectively. Model ablations on tb-core show gains at every Qwen 3.5 size—+6.8 at 4B, +7.2 at 9B, +8.1 at 27B, +9.2 at 35B—indicating the advantage is not an artifact of the largest setting.
Cost accounting shows the methods are broadly comparable: tokens per accepted environment range from 22,723 to 28,811, and attempts per accepted environment from 1.90 to 2.91. Envs-FORGE uses the most attempts (291) but the fewest tokens per attempt (9,901), reflecting shorter repair cycles rather than higher aggregate cost. The paper does not report statistical significance or variance across seeds for these downstream comparisons, so the 2-point margins over the strongest baselines should be read as single-run results.
Qualitative evidence
Five traced cases illustrate the selector's behavior across regimes: a near-frontier Bash task (d∈{in_depth,in_breadth}0) is hardened with deterministic JSON outputs; a saturated merger (d∈{in_depth,in_breadth}1) gains executable edge-case requirements; two unsolved seeds (d∈{in_depth,in_breadth}2) are reduced to deterministic bridge tasks preserving the log-analysis and token-security skill chains; and a frontier PGN repair task (d∈{in_depth,in_breadth}3) is diversified by altering the fixture without shifting difficulty. All five materializations pass static validation and obtain oracle reward 1, establishing internal consistency—though the paper notes these traces demonstrate selection semantics, not downstream effect.
Evaluation scope and limitations
The reported evaluation covers only the per-seed MILP mode at a fixed 100-environment export. Portfolio-level skill quotas, solver-off selection over an identical candidate pool, transfer-prior sensitivity analysis, and separate Evol-Instruct depth/breadth variants are all left untested. Scaling to larger seed pools and other model families is identified as the natural next step. The absence of a solver-off ablation is the most significant open question: the results do not isolate how much of the gain comes from MILP optimization per se versus simply injecting a policy-relative difficulty estimate into the prompt.
Conclusion
Envs-FORGE recasts environment synthesis for agent RL as per-seed action selection over a projection–direction space, solved as a small exact MILP conditioned on verifier-derived difficulty estimates, with gold verification gating training data. Its consistent advantage over fixed recipes across three benchmarks and four model sizes, at matched export size and synthesis cost, supports the paper's central lesson: the transformation applied to each seed should be decided relative to the current policy before generation begins. The main unresolved question is whether the MILP machinery, as opposed to frontier-aware scoring alone, is responsible for the gains—a component-level ablation this paper does not provide.