What Makes Interaction Trajectories Effective for Training Terminal Agents?

Abstract: Stronger code agents are commonly assumed to be superior teachers for post-training, yet this assumption remains poorly disentangled from task difficulty, harness design, and student capacity. We investigate this pedagogical link using Terminal-Lego, a scalable pipeline that transforms multi-domain real-world issues into environment-verified agentic tasks. Surprisingly, standalone performance does not dictate teaching efficacy: while Claude Opus 4.6 achieves higher scores on Terminal-Bench 2.0, students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization. We attribute this "pedagogical paradox" to Environment-Grounded Supervision (EGS): trajectories that explicitly expose inspect-act-verify behaviors through harness-visible interactions allow students to internalize robust problem-solving routines rather than fragile action sequences. Scaling analysis reveals exceptional data efficiency: with only 15.3k Terminal-Lego trajectories, for example, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume. Our results suggest that the frontier of agent post-training lies beyond mere outcome-matching, shifting the focus toward "Harness Engineering", where the systematic design of environment-grounded interaction structures serves as the primary catalyst for reproducible and generalizable agentic intelligence.

• The paper identifies that superior standalone performance does not equal effective teaching data, exposing a pedagogical paradox in terminal agent training.
• It introduces the Environment-Grounded Supervision framework and Targeted Observation Ratio to quantify observation-rich, effective interaction trajectories.
• Empirical evaluations reveal that student models fine-tuned on DeepSeek-V3.2's detailed trajectories achieve robust generalization with high data efficiency.

Effective Interaction Trajectories for Training Terminal Agents

Introduction and Problem Statement

The paper "What Makes Interaction Trajectories Effective for Training Terminal Agents?" (2606.03461) conducts an in-depth empirical investigation into the pedagogical underpinnings of agentic learning for terminal-based code agents. The central question addressed is whether agents with the highest standalone performance are necessarily the most effective teachers when their trajectories are distilled for student fine-tuning. Contrary to dominant assumptions equating task-solving ability with teaching efficacy, the paper identifies a critical discrepancy between production performance and data utility—an effect dubbed the "pedagogical paradox."

The Pedagogical Paradox: Standalone Performance Versus Teachability

The authors systematically evaluate four competitive terminal code agents (Claude Opus 4.6, DeepSeek-V3.2, Qwen3.5-Plus, GLM-5) within a matched-task distillation framework, mediated by the Terminus-2 harness to control for environmental and interface confounds. The primary finding is that Claude Opus 4.6, while achieving the highest standalone score on Terminal-Bench 2.0, generates trajectories that are consistently outperformed as student training data by those from DeepSeek-V3.2. Specifically, students fine-tuned on DeepSeek-V3.2's traces generalize more robustly despite its lower raw task success rate.

Figure 1: The discrepancy between standalone performance and teaching efficacy across leading agents, highlighting that higher raw task success does not entail superior student generalization.

This phenomenon is traced not to artifact such as trajectory length or error-recovery density, but to structural properties of the teacher's behavioral paradigm—deeply implicating how environmental feedback is incorporated into the action policy.

Environment-Grounded Supervision and the Targeted Observation Ratio

The paper introduces the Environment-Grounded Supervision (EGS) framework to formalize the critical behavior that characterizes useful pedagogical trajectories. EGS embodies explicit "inspect-act-verify" cycles, in which the agent exposes environment-driven reasoning through observable commands and adaptive revisions informed by intermediate feedback. To operationalize EGS, the Targeted Observation Ratio (TOR) metric is defined, quantifying the proportion of agent actions directly supported by prior, path-aligned environment inspections.

Statistical analysis reveals that DeepSeek-V3.2 not only exhibits a higher TOR but also a distinctive pattern wherein inspection and verification operations envelop environment modifications—a behavioral structure less pronounced or absent in higher-scoring but less teachable agents like Claude Opus 4.6.

Construction of the Terminal-Lego Benchmark

To enable unbiased and scalable analysis, the authors introduce Terminal-Lego, a data generation pipeline that cascades StackOverflow-derived real-world issues through a multi-stage LLM workflow, generating executable Dockerized tasks validated with automated round-trip consistency and quality control.

Figure 2: The Terminal-Lego pipeline's multi-stage construction, transforming StackOverflow issues into high-fidelity, environment-verified terminal tasks.

The resulting dataset exhibits broad coverage across over 90 technical domains, encapsulating realistic, multi-challenge problem distributions. This design mitigates synthetic bias and ensures that both teacher and student agents are evaluated on the genuine breadth of practical terminal agent workloads.

Figure 3: Distributional coverage of StackOverflow-derived source problems across targeted technical domains.

Empirical Evaluation: Data Efficiency, Generalization, and Learning Dynamics

When fine-tuning 8B and 32B-scale Qwen3 models on only ~15k DeepSeek-V3.2 trajectories, the resulting student agents achieve a 24.3% pass@1 rate on Terminal-Bench 2.0, matching or surpassing several leaderboard references that leverage models an order of magnitude larger and requiring 30$\times$ less data than prior approaches. Notably, post-training on "harder-to-imitate" (i.e., higher loss) trajectories leads to more effective student models—as measured by downstream task performance—than training on easier (lower loss), but less behaviorally instructive, teacher traces. The learning curves and gradient analyses reinforce the view that high-EGS, observation-rich supervision causes the largest positive behavioral shift during fine-tuning.

Figure 4: Training loss curves for Qwen3-32B fine-tuned on trajectories from different teachers, demonstrating that DeepSeek-V3.2 traces induce persistently higher imitation loss.

Figure 5: Gradient norm curves indicating that DeepSeek-V3.2 yields the most stable gradients, whereas Claude Opus 4.6's easy-to-imitate trajectories correspond to higher gradient instability.

Removing observation supervision or masking high-TOR interactions sharply degrades student generalization, while high-TOR subsets outperform low-TOR or random subsets when controlled for data volume and task set. Furthermore, failed DeepSeek-V3.2 trajectories (i.e., those which do not solve the task but embody robust EGS) train sufficiently capable students that remain competitive with those trained on successful traces sourced from weaker teachers, establishing that process transparency supersedes outcome in supporting transfer.

Harness Engineering: Implications for Data Curation and Future Agent Supervision

The results rigorously undermine the "stronger-is-better" assumption for teacher selection and highlight that effective student learning depends centrally on the degree to which the supervisor's reasoning and adaptation process is exposed through observable interaction structure. The identification of EGS and TOR as critical metrics reorients the focus of post-training from brute outcome-matching toward the engineering of agent-harness protocols that maximize reproducibility and generalizability of behavioral routines.

Practically, these insights support a shift in data pipeline and model engineering toward explicit interaction structure—namely, systematically prompting and/or architecting teachers to generate trajectories that are rich in environmental inspection and iterative verification. The approach also demonstrates exceptional data efficiency, motivating reduced cost and increased scalability for future agent post-training regimes.

Theoretical and Practical Implications

The decoupling of task-solving ability and trajectory teachability supports a more nuanced, multi-dimensional taxonomy of agent capability, distinguishing between outcome proficiency and knowledge-transfer potential. The paradigm of harness engineering foregrounds the interface as a first-class factor in agent intelligence, and the empirical evidence presented suggests that the principal frontier for robust, generalizable agent design lies in the supervision of "how-to-think," not merely "what-to-do."

Conclusion

This work establishes that, in the domain of terminal-agent training, agentic trajectories structured around Environment-Grounded Supervision confer robust and efficient generalization benefits, decisively outpacing trajectories that merely encode efficient solutions. The findings suggest that future advances in autonomous agents will be accelerated more by the engineering of harness protocols and trajectory curation criteria emphasizing environmental observability and adaptation, rather than by uncritical escalation of teacher task performance. Subsequent work is encouraged to further refine formal metrics for the sufficiency of observational support and to broaden the spectrum of agent behaviors systematically elevated through interface engineering.