Tool-to-Tool Matching (TTTM) Overview

• Tool-to-Tool Matching (TTTM) is a framework that estimates inter-tool relations for substitution, optimal sequencing, and composition across diverse applications.
• It employs multiple techniques such as historical co-usage, memory-based capability descriptors, latent token spaces, and explicit structured schemas to capture tool compatibility.
• TTTM is evaluated using benchmarks that measure execution success, pairwise selection accuracy, and trajectory consistency to ensure reliable dependency reasoning and causal next-tool selection.

Tool-to-Tool Matching (TTTM) denotes the problem of estimating relations among tools so that they can be compared, substituted, sequenced, or composed. In semiconductor manufacturing, the term is used explicitly for chamber matching, where nominally equivalent tools should exhibit similar process behavior (H. et al., 7 Jul 2025). In LLM-agent research, the same label functions as an Editor’s term for a cluster of problems that includes exact tool-set recommendation, scenario-conditioned tool choice, causal next-tool selection, dependency-aware tool planning, and fine-grained alignment of predicted tool calls with reference traces (Gao et al., 2024, Xiao et al., 8 Oct 2025, Babu et al., 4 Jun 2026, Gao et al., 12 May 2026, Qu et al., 15 Jan 2026). Across these literatures, the common requirement is that the relevant unit is not an isolated tool description but a relation between tools, tool calls, or tool states.

1. Scope and formal problem statements

In semiconductor manufacturing, TTTM is a consistency-quantification problem over multiple chambers or tools assigned to the same process step. The proposed analysis pipelines in that setting are built on the hypothesis that a mismatched equipment would have higher variance and/or higher number of modes in the data. The reported best univariate method achieves a correlation coefficient $>0.95$ and $>0.5$ with the variance and number of modes, respectively, and the best multivariate method achieves a correlation coefficient $>0.75$ with the top-performing univariate methods (H. et al., 7 Jul 2025). This makes TTTM a direct measurement problem over operational data rather than a purely configuration-level comparison.

In LLM-agent systems, no single formalization dominates. PTR defines the full tool universe as $T=\{T_1,T_2,\ldots,T_n\}$, the ground-truth tool set for a query $Q$ as $T_{\text{ground}}\subseteq T$, and the target as a recommended set $T_{\text{recommend}}$ such that $T_{\text{recommend}}=T_{\text{ground}}$ and $|T_{\text{recommend}}|=|T_{\text{ground}}|$, which turns matching into exact set composition rather than top-$K$ ranking (Gao et al., 2024). ToolMem instead models a candidate set of overlapping tools $>0.5$0 and learns a capability memory $>0.5$1 for each tool from experiences $>0.5$2, so that inference becomes comparative estimation of which tool will perform best for the current task (Xiao et al., 8 Oct 2025). Causal Minimal Tool Filtering represents each tool as $>0.5$3, tracks a symbolic state $>0.5$4, and defines the next visible tool set $>0.5$5 by minimal goal-reaching causal paths rather than semantic relevance alone (Babu et al., 4 Jun 2026). GRAFT formalizes multi-step planning on a directed tool graph $>0.5$6 with legality constraint $>0.5$7, making TTTM a successor-selection problem under directional dependencies (Gao et al., 12 May 2026). MatchTIR frames it as bipartite alignment between predicted tool-call instances $>0.5$8 and gold tool-call instances $>0.5$9, which is then used for dense turn-level supervision (Qu et al., 15 Jan 2026).

These formulations show that TTTM is not reducible to “retrieve a relevant tool.” Depending on the system, it may mean exact bundle recovery, tool preference under overlapping functionality, admissible next-step selection under state constraints, legal successor prediction on a tool graph, or fine-grained correspondence between predicted and reference tool calls.

2. Representing tool relations

One major line of work represents tool relations through historical co-usage. PTR introduces a history dataset $>0.75$0, where each $>0.75$1 is a tool bundle used for a past query, and retrieves an initial concise set from those historical bundles rather than from isolated tools. The central claim is that bundle retrieval captures “a richer contextual relationship between queries and sets of tools that have been used together effectively in the past,” including “dependencies and synergies” (Gao et al., 2024). In TTTM terms, this is a bundle-memory representation of compatibility.

A second line stores explicit capability summaries per tool. ToolMem builds a memory $>0.75$2 for each tool, populated with natural-language entries describing strengths and weaknesses such as “good at,” “bad at,” “proficient at,” and “weak at,” organized under the taxonomy $>0.75$3 (Xiao et al., 8 Oct 2025). The representation is comparative and scenario-conditional rather than static API description, which makes it especially suited to neural tools whose performance varies by prompt type or modality.

A third line internalizes tools directly into model parameters or vocabulary. ToolGen assigns each tool a unique token and trains the LLM to generate tool tokens as part of next-token prediction, thereby converting retrieval into conditional generation over a large tool vocabulary (Wang et al., 2024). GRAFT also maps each tool node to a dedicated special token $>0.75$4, but additionally learns directed successor structure in that token space (Gao et al., 12 May 2026). ParaTool moves tool knowledge from context into tool-specific parameter modules $>0.75$5, with inference-time soft composition $>0.75$6, so that tools become loadable parameter objects rather than prompt text (Yu et al., 28 May 2026). This suggests a latent-space view of TTTM in which similarity, ambiguity, and co-activation are expressed through token geometry or parameter mixtures.

A fourth line uses explicit structured capability descriptions. Multi-Field Tool Retrieval standardizes each tool into four fields—description, parameters, response, and examples—so that matching can operate over multiple aspects rather than raw documentation (Tang et al., 5 Feb 2026). RaTA-Tool normalizes tools into a three-field JSON schema consisting of input, process, and output, then retrieves tools by comparing those descriptions against a generated task description in the same schema (Mattioli et al., 16 Apr 2026). For physical tools, ToolNet introduces a 13-dimensional attribute space $>0.75$7 spanning physical, functional, and psychological properties, with each tool represented by an attribute vector and matched via cosine or Euclidean similarity (Hao et al., 28 May 2025). Together, these works suggest three broad representational substrates for TTTM: co-usage memory, explicit capability descriptors, and latent token/parameter spaces.

3. Matching mechanisms

PTR provides one of the clearest multi-stage TTTM mechanisms. Its pipeline consists of Tool Bundle Acquisition, Functional Coverage Mapping, and Multi-view-based Re-ranking. Bundle acquisition retrieves an initial historical bundle; functional coverage mapping prompts an LLM to determine which tools in that bundle actually cover the query’s functionalities and whether the set exactly solves, over-solves, or partially solves the task; multi-view re-ranking then augments unresolved functionality through Direct Semantic Alignment (DSA), Historical Query Correlation (HQC), and Contextual Tool Expansion (CTE). The most explicit tool-to-tool operation appears in CTE, which retrieves tools similar to a primary tool through $>0.75$8 (Gao et al., 2024). PTR therefore combines setwise co-usage memory with explicit pairwise tool similarity and iterative set editing.

ToolMem operationalizes TTTM as retrieval-augmented comparative reasoning. For a new task $>0.75$9, the agent retrieves relevant memories separately from each proficiency partition, predicts tool performance from those memories, and selects the tool with the higher predicted score. In the pairwise experiments there is no separate learned policy head; routing is done by relative predicted quality, so the operational definition of matching is “which tool is estimated to perform better for this scenario?” (Xiao et al., 8 Oct 2025).

CMTF replaces broad semantic retrieval with causal frontier selection. A tool is executable when $T=\{T_1,T_2,\ldots,T_n\}$0, but executability alone is not sufficient. The method searches over reachable symbolic states and exposes only the first executable tool, or tied first tools, on a minimal causal path from the current state to the goal. In the main benchmark it returns one visible tool per step on average, which turns TTTM into selection of the admissible next transition rather than ranking all relevant tools (Babu et al., 4 Jun 2026).

ToolMaster addresses novelty by training an agent to probe tools before committing to execution. It separates trajectories into a trial phase $T=\{T_1,T_2,\ldots,T_n\}$1 and an execution phase $T=\{T_1,T_2,\ldots,T_n\}$2, so that the model can test unfamiliar tools, inspect outputs and errors, and revise its tool choice before goal-directed use. This is not an explicit tool graph or compatibility learner, but it is a direct mechanism for discovering functional correspondences and substitutes online (Gao et al., 19 Jan 2026). GRAFT, by contrast, internalizes tool dependencies offline through graph-token edge reconstruction: it samples graph-valid paths $T=\{T_1,T_2,\ldots,T_n\}$3, learns successor scoring from current tool-token hidden states, and then applies on-policy tool context distillation so the planner remains stable under its own sampled prefixes (Gao et al., 12 May 2026).

MatchTIR and TTPA move matching into the supervision signal itself. MatchTIR computes a similarity matrix $T=\{T_1,T_2,\ldots,T_n\}$4 between predicted and gold tool calls using tool-name equality, Jaccard overlap over parameter names, and exact parameter-value matches; it then performs either hard bipartite matching with the Hungarian algorithm or soft assignment with optimal transport, converts call-level scores into turn-level rewards, and combines local and global advantages during GRPO training (Qu et al., 15 Jan 2026). TTPA instead samples near-miss tool-call variants at uncertain token positions and scores them with an Error-oriented Scoring Mechanism over name, required parameter, valid parameter, parameter type, and parameter value errors, producing token-level preference pairs for DPO fine-tuning (Huang et al., 26 May 2025). At execution time, ATTC adds a complementary arbitration layer: it computes a code-block confidence score $T=\{T_1,T_2,\ldots,T_n\}$5 from token probabilities and uses a threshold $T=\{T_1,T_2,\ldots,T_n\}$6 to decide whether the model should trust the tool result or rethink its reasoning (Xu et al., 9 Apr 2026).

4. Benchmarks and evaluation criteria

Evaluation has shifted from ranking-centric metrics to composition-centric metrics. PTR introduces RecTools, where the number of tools per query ranges from $T=\{T_1,T_2,\ldots,T_n\}$7 to $T=\{T_1,T_2,\ldots,T_n\}$8, and defines TRACC as

$T=\{T_1,T_2,\ldots,T_n\}$9

so that both wrong membership and wrong cardinality are penalized. With SimCSE + PTR + GPT-4o, reported TRACC reaches $Q$0 on ToolLens, $Q$1 on MetaTool, and $Q$2 on RecTools, improving over SimCSE without PTR on all three datasets (Gao et al., 2024).

ToolMem evaluates both prediction and selection. For performance prediction it reports MAE, RMSE, and Pearson correlation; for pairwise selection it reports $Q$3, $Q$4, and accuracy over unequal-score pairs. Relative to Generic agents, ToolMem-augmented agents predict tool performance $Q$5 and $Q$6 more accurately across text and multimodal generation scenarios, and improve average pairwise selection accuracy from $Q$7 to $Q$8 on text generation and from $Q$9 to $T_{\text{ground}}\subseteq T$0 on text-to-image selection (Xiao et al., 8 Oct 2025).

CMTF uses task success, wrong-tool count, premature-action count, average tools exposed per step, and token cost. In its main benchmark with 102 tasks, 100 tools, four LLM backends, and 2448 task-method-model runs, All tools yields success $T_{\text{ground}}\subseteq T$1, wrong tools $T_{\text{ground}}\subseteq T$2, tools/step $T_{\text{ground}}\subseteq T$3, and tokens $T_{\text{ground}}\subseteq T$4, whereas CMTF yields success $T_{\text{ground}}\subseteq T$5, wrong tools $T_{\text{ground}}\subseteq T$6, tools/step $T_{\text{ground}}\subseteq T$7, and tokens $T_{\text{ground}}\subseteq T$8 (Babu et al., 4 Jun 2026). This benchmark is important because it directly measures next-tool correctness and premature actions, not only final success.

ToolMATH evaluates long-horizon multi-tool reasoning under large overlapping catalogs and missing-capability settings. The benchmark contains 7,699 questions and 12,369 tools, with distractor similarity levels $T_{\text{ground}}\subseteq T$9 and distractor budgets $T_{\text{recommend}}$0; the additional hard set contains 329 questions and 626 tools (Choi et al., 24 Feb 2026). Its central diagnostic result is that overlap harms performance mainly by amplifying early deviations into execution drift. GRAFT complements this with sequence-level planning metrics: Exact Match (EM), Edge Legality Rate (ELR), and Average Correct Prefix Length (ACPL). On Multimedia with Llama-3.2-3B, for example, GRAFT reports EM $T_{\text{recommend}}$1, ELR $T_{\text{recommend}}$2, and ACPL $T_{\text{recommend}}$3, outperforming token-only baselines on exact sequence fidelity while remaining competitive on legality (Gao et al., 12 May 2026).

Outside software tools, attribute-mediated matching yields its own evaluation paradigm. ToolNet reports 74\% end-to-end tool-selection accuracy for the best combination, compared with 20\% for direct tool name matching, suggesting that interpretable capability vectors can support substitute-like reasoning more effectively than surface matching (Hao et al., 28 May 2025). MFTR reports state-of-the-art performance on five datasets and a mixed benchmark through multi-field scoring (Tang et al., 5 Feb 2026), while RaTA-Tool introduces an open-world multimodal benchmark with 14,924 unique queries and 920 unique tools, achieving $T_{\text{recommend}}$4 for the 7B variant under tool-level train/test splits (Mattioli et al., 16 Apr 2026).

5. Misconceptions, failure modes, and limitations

A persistent misconception is that semantic relevance is sufficient for TTTM. CMTF shows that a tool may be related to the request yet still be unnecessary, premature, or off-path at the current step, and ToolMATH shows that overlapping tool catalogs do not merely add noise but amplify small early deviations into irreversible execution drift (Babu et al., 4 Jun 2026, Choi et al., 24 Feb 2026). Another misconception is that executability is sufficient; CMTF explicitly shows that many executable tools are still wrong next tools because only a subset lies on a minimal causal frontier.

PTR exposes the limitations of implicit relation modeling. Its strongest signals come from historical bundles, similar historical queries, and contextual expansion around a primary tool, but it does not report explicit co-invocation counts, compatibility matrices, substitution statistics, or execution-order reasoning. The paper also notes that CTE may surface “complementary or alternative” tools, which conflates complementarity with substitutability, and that cold-start tools with little bundle history are likely underserved (Gao et al., 2024).

ToolMem has a different set of limitations. Memory quality depends on feedback quality; text-generation feedback is produced by GPT-4 judgments rather than humans, memories can become stale as tools evolve, retrieval misses degrade selection, and natural-language summaries can overgeneralize from sparse evidence (Xiao et al., 8 Oct 2025). ToolGen and ParaTool reveal the trade-off between strong internalization and open-world flexibility: ToolGen underperforms ToolLlama on unseen tools in full-task completion, while ParaTool requires each tool to be parameterized during training and still does not provide explicit tool-to-tool graphs or schema-level interoperability reasoning (Wang et al., 2024, Yu et al., 28 May 2026).

Static metadata is another bottleneck. MFTR is powerful as a retrieval layer, but it relies on documentation quality and intentionally keeps the response field high-level rather than schema-complete; it does not model dynamic runtime behavior, side effects, or environment-specific constraints (Tang et al., 5 Feb 2026). TTPA improves local API matching but assumes a predefined static tool set and incurs additional compute for token-level preference sampling (Huang et al., 26 May 2025). ToolMaster improves unfamiliar-tool usage through probing, yet this adds tool calls and therefore latency, and exploratory calls can be problematic when tools have side effects (Gao et al., 19 Jan 2026).

6. Open directions

Current work suggests several converging research directions. First, implicit relation sources—historical bundles, graph successors, and trace alignments—could be unified into explicit compatibility structures that distinguish complementarity, substitutability, and legal succession rather than collapsing them into a single similarity score (Gao et al., 2024, Gao et al., 12 May 2026). Second, causal next-step selection and long-range plan coherence remain partly separate traditions; the evidence that semantic relevance and executability are each insufficient suggests that future TTTM systems should combine symbolic state, dependency graphs, and trajectory-level planning signals (Babu et al., 4 Jun 2026, Choi et al., 24 Feb 2026).

Third, open-world TTTM will likely require stronger canonical tool descriptions than free-form prose. The success of standardized input/process/output and description/parameters/response/examples schemas suggests that relation-specific matching can be built on normalized capability contracts, but richer interoperability fields—especially for outputs, constraints, and environment assumptions—appear necessary for composition rather than simple retrieval (Mattioli et al., 16 Apr 2026, Tang et al., 5 Feb 2026). Fourth, parameterized tool modules hint at a path toward latent TTTM for large registries, but the absence of explicit compatibility reasoning suggests that future systems may need hybrid schemes in which description-based onboarding and parameter-space composition coexist (Yu et al., 28 May 2026).

Finally, execution feedback and trust calibration remain underused as first-class TTTM signals. ToolMaster shows that unfamiliar tools can be grounded through probing, and ATTC shows that conflict between internal reasoning and tool evidence can be mitigated by explicit trust policies (Gao et al., 19 Jan 2026, Xu et al., 9 Apr 2026). This suggests a broader TTTM program in which relation scores are updated not only from documentation or historical traces, but also from observed runtime behavior. In that broader view, the central TTTM question is no longer simply “Which tool is relevant?” but “Which relation among available tools keeps the next action executable, the intermediate state coherent, and the overall trajectory solvable?”