- The paper introduces MidTool-Mix, a 20.3B-token corpus that improves BFCLv3, τ²-Bench, and MCP-Universe performance for 4B and 8B models, with 4B results reaching 54.18% BFCLv3, 19.96% τ²-Bench Pass@1, and 23.80 on MCP-Universe after SFT and RL.
- Its data pipeline combines web, PDF, code, API, and MCP sources with context-grounded augmentation and native agentic trajectory synthesis to teach tool selection, schema-based argument construction, clarification, and multi-step execution.
- The results show that dedicated mid-training transfers better to unfamiliar tools and interactive environments than generic corpora, but provides little benefit for web-search tasks, indicating that search-heavy agency requires specialized training data.
MidTool addresses a gap in the mid-training literature: while targeted mid-training has been shown to improve math, science, and software-engineering capabilities, general tool use has largely been left to post-training. The authors construct MidTool-Mix, a 20.3B-token open corpus for agentic tool-use mid-training, and demonstrate that inserting it between pre-training and post-training yields consistent downstream gains on BFCLv3, τ2-Bench, and MCP-Universe across two model scales (2608.20314).
Motivation and positioning
The paper's central argument is that the knowledge underlying successful tool use is distributed across developer documentation, manuals, PDFs, code repositories, API specifications, and structured tool definitions—material that rarely appears as clean agent demonstrations. Post-training on curated traces must therefore induce atomic agentic capabilities (tool recognition, schema-grounded argument construction, clarification under missing information, multi-step execution) from comparatively narrow supervision. Prior open corpora are either general-domain (FineWeb, Dolmino), domain-specific to math or SWE (MegaMath-Web-Pro, daVinci-Dev), or post-training datasets with fixed tool inventories (TOUCAN, ToolACE). MidTool-Mix is positioned as the first open mid-training corpus dedicated to general tool use, distinguished by high tool diversity (2.60M unique tool names) and inclusion of agentic trajectories.
Corpus construction pipeline
The pipeline proceeds in three stages. Stage 1 collects four complementary source families: web documents sampled from FineWeb dumps spanning 2020–2025; English PDFs from FinePDFs; GitHub repositories in two slices (agent/MCP-related repositories from GH Archive event data, and high-quality community repositories across major languages); and structured tool artifacts including REST APIs and MCP skills. Benchmark and evaluation repositories are excluded via an actively maintained blacklist during collection.
Stage 2 applies source-specific preprocessing: StarCoder-style heuristics plus SHA-256 exact deduplication and MinHash LSH near-deduplication for code; a four-phase pipeline of keyword/URL prescreening, fastText classification trained on LLM-labeled seeds, document-level quality filtering, and MinHash LSH for web and PDF data.
Stage 3 converts refined sources into supervision through two branches targeting distinct deficits:
- Context-grounded trajectory augmentation addresses grounding. Documents surviving keyword prefiltering are annotated by Qwen3-235B-A22B-Instruct-2507 with quality scores and structured affordance profiles; a rule-based planner allocates bounded budgets for QA types covering tool selection, parameter extraction, format-constrained calls, workflow recognition, and multi-turn trajectories.
- Native agentic trajectory synthesis addresses execution. GPT-5 assigns quality scores and feasibility profiles over trajectory families; a deterministic quality-adaptive budget controller conditions allocation on source quality, tool count, and argument structure, preferring multi-turn trajectories. Category-specific generators using GPT-5/5.1/5.2 instantiate trajectories that are strictly validated for turn ordering, schema grounding, required arguments, and tool-response consistency. The branch also incorporates rollouts from AWM synthetic environments and filtered Nemotron Agentic traces.
The final mixture contains 20.3B tokens and 11.22M samples: web 42%, code 26%, PDF 23%, with native agentic trajectories contributing 9%. A t-SNE comparison against FineWeb and Dolmino shows MidTool-Mix occupying documentation-heavy, workflow-oriented regions not covered by either reference corpus. A DeCon audit flags fewer than 20 candidates against all three benchmarks, all manually confirmed false positives; the authors note this bounds only verbatim overlap, leaving semantic-level audit open.
Experimental setup
Qwen3-4B-Base and Qwen3-8B-Base are compared under four recipes each: base + SFT (+ optional RL) versus base + MidTool-Mix + SFT (+ optional RL). SFT uses a 100K subset of TOUCAN; RL uses GRPO over 526 synthetic AWM environments. Evaluation covers BFCLv3 (function calling, single- and multi-turn), verified τ2-Bench (interactive dual-control tasks), and MCP-Universe (execution over real MCP servers). Released Qwen3-4B/8B models serve as references.
Main results
The consistent pattern is that MidTool-Mix improves downstream performance over SFT-only baselines at both scales, with RL compounding gains. Selected overall numbers:
| Setting |
BFCLv3 Overall |
τ2 Pass@1 |
MCP-Universe Score |
| Qwen3-4B-Base + SFT |
39.73% |
8.54% |
13.20 |
| Qwen3-4B + MidTool + SFT |
50.25% |
12.23% |
18.66 |
| Qwen3-4B + MidTool + SFT + RL |
54.18% |
19.96% |
23.80 |
| Qwen3-8B-Base + SFT |
47.62% |
10.43% |
15.18 |
| Qwen3-8B + MidTool + SFT + RL |
55.12% |
21.31% |
25.16 |
Gains concentrate on harder settings: at 4B, BFCL multi-turn average rises more than 10 points over the SFT-only baseline, and on τ2-Bench overall Pass@1 nearly doubles (8.54% → 19.96%). On MCP-Universe, the financial-analysis pass rate jumps from 5.00% to 37.50% at 4B after RL. These results imply that mid-training contributes capabilities—particularly grounding and long-horizon planning—that standard post-training does not reliably induce.
A notable boundary emerges: despite broad MCP improvements, the web-search subset remains at 0.00 across all configurations. The authors interpret this as evidence that deep-search-style exploratory behavior requires dedicated mid-training data beyond a general tool-use mixture—a concrete capability boundary rather than a uniform transfer failure.
Supplementary analyses show MidTool-Mix also improves optimization dynamics: lower initial SFT loss and faster early-stage RL reward acquisition relative to both raw base and Dolmino-mid-trained checkpoints. Importantly, in-environment RL rewards converge between pipelines late in training, yet benchmark gaps persist—the authors argue RL reward reflects environment adaptation, and the durable advantage lies in out-of-environment generalization.
Ablations
Fixing the 4B SFT recipe and varying only the mid-training corpus, the ablation decomposes MidTool-Mix additively against Dolmino-20BT as a matched-budget generic baseline. Key findings:
- Processed raw sources alone already improve BFCL overall by +2.6 and transfer substantially better to MCP-Universe than Dolmino-20BT.
- The two synthesis branches contribute asymmetrically: native agentic trajectories alone yield the larger BFCL gain (+7.9 vs. +4.9 overall), while context-grounded augmentation alone is stronger on τ2-Bench and MCP-Universe—executable trajectories matter most for precise function calling, grounding-oriented supervision most for transfer.
- Both single-branch variants fall below no mid-training on MCP-Universe, yet the complete mixture improves over no mid-training on all eight metrics; combining branches is the only configuration that does so.
- Dolmino-20BT is competitive on BFCL but transfers poorly to τ2-Bench and MCP-Universe, suggesting generic instruction-following-style mid-training helps simple function calling but not agentic settings.
The authors concede that Dolmino-20BT itself contains model-generated components, so it is not a purely non-synthetic baseline; they also note the native-trajectory branch relies on proprietary teachers, though the fact that even context-grounded-only variants outperform no mid-training indicates gains are not solely driven by proprietary-model-synthesized data.
Limitations and open questions
Several limitations are stated plainly. The downstream recipe is held fixed throughout, so the interaction between mid-training composition and post-training design—SFT corpus composition, RL environment choice, reward design—is unexplored. The mixture-design space is not fully mapped at matched token budget; controls such as synthesized QA without originating context, or a native-trajectory-only mixture scaled to full budget, remain untested, and the latter would require executable environments at tens-of-billions-of-token scale. Both synthesis branches depend on strong teacher models (Qwen3-235B, GPT-5 family), raising the question of when self-generated teachers could sustain the pipeline's validation requirements. Finally, the contamination audit bounds verbatim overlap only, not semantic or schema-level similarity relevant to MCP tool definitions.
Conclusion
MidTool demonstrates that general tool use benefits from dedicated mid-training rather than being deferred entirely to post-training. The 20.3B-token MidTool-Mix consistently improves BFCLv3, τ2-Bench, and MCP-Universe performance for 4B and 8B models under fixed SFT and RL recipes, with the largest gains on multi-turn and interactive settings, and its two synthesis branches prove complementary. The work also delineates a boundary: broad tool-use priors transfer well to unfamiliar schemas and MCP domains but contribute little to search-heavy exploratory behavior, indicating that specialized agency likely requires specialized mid-training mixtures—an open question the paper leaves to future work.
