---
title: 'TerminalTraj: Terminal Agent Training Dataset'
url: https://www.emergentmind.com/topics/terminaltraj
type: topic
---

# TerminalTraj: Terminal Agent Training Dataset

Searching arXiv for the cited TerminalTraj paper and closely related terminal-agent work to ground the article.
arxiv_search(query="TerminalTraj terminal trajectories dockerized environments", max_results=5)
Searching for the exact TerminalTraj title on arXiv.
arxiv_search(query="ti:\"Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments\"", max_results=10)
Searching arXiv for TerminalTraj and terminal-agent benchmark papers.
Searching arXiv now.
arxiv_search({"query":"ti:\"Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments\"","max_results":10})
TerminalTraj is a large-scale framework and dataset for training terminal-capable AI agents from execution-grounded interaction traces inside Dockerized software environments. It is defined simultaneously as an automated pipeline that converts real GitHub repositories into executable, verifiable terminal tasks and as a corpus of 32,325 Docker images and 50,733 verified terminal trajectories spanning eight popular programming languages and eight specialized domains. Its central premise is that realistic terminal competence depends on long-horizon interaction with actual systems—file systems, package managers, databases, web servers, QEMU, and model-training stacks—rather than text-only simulations or heuristic success labels [2602.01244].

## 1. Conceptual scope and problem setting

TerminalTraj targets the construction of training data for terminal agents in the form of a Docker environment, a task query, an interaction trajectory, and executable validation code. In the formulation used by the framework, each trajectory is a multi-step record of agent behavior, including analysis, plan, commands, outputs, and a terminal completion flag. The resulting supervision is intended for multi-turn supervised fine-tuning of code-capable language models, with the explicit goal of improving their ability to solve realistic terminal tasks under environment state, dependency, and side-effect constraints [2602.01244].

The framework is motivated by two deficiencies in prior agentic data construction. First, many existing datasets do not execute in real environments and instead rely on text-only simulation or LLM-based judging. Second, datasets that do use real environments often have limited diversity, typically resembling SWE-bench-style collections of repositories. TerminalTraj addresses both limitations by scaling repository collection to 899,741 GitHub repositories and by grounding success in executable validation code rather than free-form judgment.

Its language coverage comprises Python, C++, C, Java, JavaScript, PHP, HTML, and Go. Its domain coverage comprises Environment Interaction, Multimodal, Security, Data Processing, Model Training & Evaluation, SQL, Web Service, and QEMU-related. These domains are not merely topical labels; they correspond to distinct terminal workflows involving heterogeneous tools, artifacts, and verification procedures.

A common misconception is that TerminalTraj is primarily a benchmark. More precisely, it is a data-generation and training pipeline whose outputs are then evaluated on TerminalBench 1.0 and 2.0. Another misconception is that it is a text-only “trajectory” dataset. In TerminalTraj, a trajectory is an execution-grounded terminal interaction trace embedded in a Docker environment and validated by code.

## 2. Executability and verifiability as organizing principles

The framework is organized around two requirements: executability and verifiability. Executability means that every task instance must run inside a concrete, working execution environment with the necessary operating system state, dependencies, tools, and services. Verifiability means that each instance must admit an objective success criterion that can be executed inside that environment, even when multiple command sequences can legitimately solve the task [2602.01244].

To improve executability at scale, TerminalTraj defines a repository quality score
\[
Q_i = \frac{\sum_{f_j \in R_i} \text{ScoreModel}(f_j)}{N_i},
\]
where \(R_i\) is the \(i\)-th repository, \(f_j\) is the \(j\)-th file, \(N_i\) is the number of files, and \(\text{ScoreModel}(f_j) \in [0,1]\) is a predicted completeness or executability score. Repositories with \(Q_i < 0.2\) are discarded. The ScoreModel is a lightweight reward model based on Qwen2.5-Coder-0.5B with a regression head, trained against pseudo-labels produced by Qwen3-Coder-32B using mean squared error and a balanced training set of 20K code files per language. The reported effect is substantial: random repositories with Dockerfiles achieve approximately 9% Docker build success, repositories with \(Q_i > 0.5\) reach approximately 19.5%, and repositories with \(Q_i > 0.2\) reach approximately 17%.

Verifiability is handled through instance-specific executable validation code, typically pytest suites or scripts. This design is necessary because terminal tasks produce heterogeneous outputs: a task may create files, configure services, query databases, manipulate images, crack encrypted archives, or bring up virtualized systems. TerminalTraj therefore rejects unified textual answer keys and LLM judging as the primary success criterion. The validation code checks environment state directly, such as whether files exist and match golden content, whether HTTP endpoints respond correctly, whether SQLite queries return valid rows, or whether a maze-exploration procedure reaches the right solution.

This emphasis on executable validation has methodological consequences. Terminal trajectories are retained only if they pass the validation code. The resulting dataset is therefore filtered not by plausibility of reasoning traces but by successful interaction with real software systems. This suggests that TerminalTraj treats terminal competence as a state-transition problem over environments rather than as a purely linguistic problem over transcripts.

## 3. Pipeline architecture

The TerminalTraj pipeline has three main components: Data Sources Collection, Docker Image Curation, and Instance Generation. In the collection stage, the framework gathers 899,741 repositories, 2,010,113 code files, and approximately 20K additional input links such as documentation, data, and videos. Repositories created after the release of TerminalBench are excluded, and files that match benchmark reference solutions are filtered using an LLM prompt in order to reduce benchmark leakage [2602.01244].

Docker Image Curation applies the quality filter described above and then attempts environment construction. The pipeline prefers repositories with Dockerfiles or build configurations, and for domain-specific tasks lacking suitable Dockerfiles it injects missing dependencies, including domain tools such as QEMU, nginx, PyTorch, and sqlite3. This stage yields 32,325 Docker images across the supported languages and domains.

Instance Generation synthesizes triples of Docker Image, Task Query, and Validation Code. The key heuristic is alignment between repository documentation and executable scripts: Markdown documentation often describes usage patterns, while shell or Python scripts implement them. TerminalTraj uses Qwen3-Coder-480B to generate natural-language task instructions and executable validation code from these materials. In general scenarios, the model is asked to emit JSON containing an instruction and pytest content; constraints include alignment with the provided documentation and scripts, routing file outputs under `/results/`, and ensuring that all information needed to solve the task is stated in the query.

The framework also supports richer domain-specific prompts with input data mounted under `/data/` and output locations prescribed in advance. Representative tasks include cracking a password-protected RAR and writing the extracted flag to `/app/result.txt`, converting `/app/sales.json` to `/app/sales.csv`, booting Alpine ISO in QEMU and serving HTTP on port 8080, traversing an interactive maze through shell commands, solving Sudoku from an image, deploying a Git server plus Nginx HTTPS workflow, querying `/app/ilios.db` for `program_year_id = 101`, and running an OWLv2-based detection tool whose output is stored as `/results/detections.json`.

After instances are generated, TerminalTraj rolls trajectories with Qwen3-Coder-480B through the Terminus-2 scaffold. At each step, the agent emits structured content containing analysis, plan, commands with keystrokes and durations, and a `task_complete` flag. The scaffold injects keystrokes into the shell, captures `stdout` and `stderr`, feeds the resulting state back to the model, and records the entire interaction trace. Success is then decided by executing the instance-specific validation code. The system samples \(k\) executions per query, and the reported empirical choice is \(k = 4\), motivated by pass@k saturation for the base agent.

## 4. Dataset composition and trajectory structure

The final dataset statistics are: 899,741 repositories collected, 196,051 high-quality repositories after ScoreModel filtering with \(Q_i \ge 0.2\), 32,325 Docker images built, 1,030,695 task instances generated, and 50,733 verified trajectories retained. The verified trajectory rate is 4.92% overall, with the paper characterizing the low retention rate as a consequence of task difficulty and strict validation [2602.01244].

The programming-language breakdown is uneven but broad. Go contributes 7,400 images and 18,838 trajectories; Python contributes 10,837 images and 11,196 trajectories; Java contributes 5,400 images and 3,985 trajectories; JavaScript contributes 1,689 images and 7,880 trajectories; C++, C, HTML, and PHP contribute the remainder. This distribution indicates that environment diversity and trajectory count are not perfectly correlated with conventional language popularity; they also depend on repository quality, buildability, and task-generation yield.

At the domain level, the paper reports 23,462 domain-specific instances: 2,842 Web Service, 3,126 Model Training & Evaluation, 2,934 Environment Interaction, 1,534 Multimodal, 3,004 Data Processing, 2,615 SQL, 3,087 QEMU, and 3,330 Security. The average query contains 193.6 tokens, the average model response per turn contains 160.5 tokens, the average full trajectory contains 6,531.3 tokens, and the average trajectory spans 15.1 turns. These figures place TerminalTraj firmly in the long-horizon regime rather than the single-shot coding regime.

The paper further analyzes trajectory content using SentenceBERT plus PCA. Linux-centric domains such as Web Service, QEMU, Security, and SQL cluster together and share CLI patterns including `ls`, `grep`, `find`, `nginx`, `savevm`, and `sqlite3`. Non-Linux-centric domains such as Environment Interaction, Data Processing, Model Training & Evaluation, and Multimodal are more separable and rely more on higher-level programming constructs. A plausible implication is that TerminalTraj captures both generic shell manipulation skills and narrower tool-specific behaviors.

Instance quality is evaluated by stratifying tasks according to repository score \(Q_i\). For each score bucket, 20 instances are sampled and rated from 0 to 8 by human annotators and by an LLM-as-judge prompt using four criteria: Task Soundness, Evaluability, Information Sufficiency, and Behavioral Constrainedness. Reported quality rises with repository score, and for \(Q_i \ge 0.2\) human scores are generally above 5.5/8.

## 5. Training protocol and benchmark results

TerminalTraj is used to fine-tune Qwen2.5-Coder backbones at 7B, 14B, and 32B scale through multi-turn supervised fine-tuning implemented in Megatron-LM. The training objective is standard next-token prediction over structured dialogues and trajectories,
\[
\mathcal{L} = - \sum_t \log p_\theta(y_t \mid y_{<t}, x),
\]
where \(x\) is the accumulated context of task query and prior turns. For the 32B model, the reported optimization settings include bf16, learning rate \(1 \times 10^{-5}\), minimum learning rate \(1 \times 10^{-6}\), cosine decay, Adam with \(\beta_1 = 0.9\), \(\beta_2 = 0.95\), \(\epsilon = 1 \times 10^{-8}\), weight decay \(= 0.01\), and gradient clipping \(= 1.0\) [2602.01244].

Evaluation follows the TerminalBench 1.0 and 2.0 protocols using Terminus-2, four independent runs per model, and average performance with 95% confidence intervals via bootstrap. The central finding is that execution-grounded SFT data substantially improve terminal-agent behavior relative to the raw coding backbones. The paper reports gains of up to 20 percentage points on TB 1.0 and 10 percentage points on TB 2.0, while the abstract summarizes the improvements as up to 20% on TB 1.0 and 10% on TB 2.0.

| Model | TB 1.0 | TB 2.0 |
|---|---:|---:|
| Qwen2.5-Coder-32B | 5.00% | 4.49% |
| TerminalTraj-32B | 35.30% | 22.00% |
| Qwen3-Coder-480B-A35B | 37.50% | 23.90% |

These results support two claims made by the paper. First, the unadapted Qwen2.5-Coder backbones are weak terminal agents, and scaling from 7B to 32B yields only limited improvement. Second, TerminalTraj-32B outperforms all open-source models under 100B parameters, including Qwen3-32B-Nex-N1, by 6.55 points on TB 1.0 and 5.3 points on TB 2.0. The paper further states that TerminalTraj-32B approaches Qwen3-Coder-480B while using approximately 15 times fewer parameters.

Test-time scaling is another prominent result. On TB 1.0, TerminalTraj models exhibit steep pass@k curves for \(k \in \{1,2,4,8,16\}\), whereas Qwen2.5-Coder backbones remain relatively flat and stay below 35% even at pass@16. TerminalTraj-32B reaches pass@16 of approximately 63%, surpassing Qwen3-Coder-480B at all \(k\) except roughly tying at \(k=4\). The paper interprets this as evidence that execution-grounded training makes multiple samples meaningfully diverse rather than merely redundant.

The ablations reinforce the importance of both domain diversity and executable filtering. Removing any domain from a balanced 10,000-instance subset reduces performance; the largest drops come from removing Web Service, with \(-8.9\%\) on TB 1.0 and \(-7.4\%\) on TB 2.0, and QEMU, with \(-9.7\%\) and \(-8.7\%\), respectively. Code-verified data consistently outperform LLM-verified data, especially at training sizes of 1K to 4K. Compared with Nex-N1, TerminalTraj yields a 1.7 times larger gain on TB 1.0 and approximately 1.3 times larger gain on TB 2.0 despite fewer trajectories and a weaker backbone. Compared with SETA, which uses approximately 400 curated Docker environments and RL on Qwen3-8B, TerminalTraj’s broader environment diversity appears materially more effective: the paper reports pass@8 of 45.00% for TerminalTraj-7B.

## 6. Interpretation, related research, and limitations

TerminalTraj’s main methodological contribution is not merely scale, but the coupling of Docker-aligned task generation with executable validation and large-scale environment diversity. The framework treats terminal-agent supervision as the selective retention of successful trajectories in real software environments. This design departs from approaches that rely on textual plausibility, benchmark-specific heuristics, or a small number of heavily curated environments [2602.01244].

The name can be confused with other “trajectory” systems in machine learning. A useful contrast is "TrajAgent" [2410.20445], which addresses trajectory modeling for check-in and GPS data through UniEnv, TAgent, and AutOpt. In TrajAgent, “trajectory” refers to mobility data and tasks such as next location prediction, travel time estimation, and anomaly detection. In TerminalTraj, by contrast, “trajectory” refers to long-horizon agent interaction traces in terminal environments. The shared vocabulary therefore masks substantially different objects of study.

Several limitations are explicit. Domain coverage is restricted to eight languages and eight domains, leaving out settings such as Windows, mobile systems, and large-scale distributed environments. All training environments are Dockerized, which is practical for sandboxing but not identical to deployment on bare metal or other packaging stacks. The pipeline is expensive: building 32K Docker images, generating over one million task instances, and executing up to four rollouts per query produces a verified trajectory rate of only about 4–5%. Validation code can also be fragile, since per-instance tests are LLM-generated or otherwise instance-specific and may contain bugs. In addition, all retained trajectories are produced by a single teacher, Qwen3-Coder-480B, which may constrain behavioral diversity.

Safety remains a distinct concern. The paper emphasizes that Docker is a training-time sandbox rather than a complete deployment solution and points to gVisor, VMs, privilege restrictions, and careful deployment practices as necessary safeguards. Future directions suggested by the paper include applying the pipeline to larger backbones, extending it to more domains and tools, improving instance generation and validation, and using the same interactive environments for reinforcement learning rather than supervised fine-tuning alone.

Taken together, TerminalTraj positions executable, verifiable, environment-grounded trajectory generation as a primary substrate for terminal-agent training. Its results suggest that the combination of repository-quality filtering, Docker image curation, task synthesis from repository artifacts, and code-based success labeling yields supervision that is both more realistic and more transferable than text-only or weakly judged alternatives.

Source: https://www.emergentmind.com/topics/terminaltraj