Most Effective Interface Between LLMs and External Tools

Determine the most effective interface between large language models and external software tools to enable reliable and performant tool usage within compound AI systems.

Background

The paper studies how LLMs can query databases through function calling and discusses broader patterns for tool integration in compound AI systems. While function calling has emerged as a common approach, other paradigms and variations exist, and the authors note that identifying the best overall interface is unresolved.

This open question motivates the work’s proposed tool schema for database querying and the DBGorilla benchmark to evaluate LLMs’ ability to format appropriate tool arguments. Despite these contributions, the general design question of the optimal interface for connecting LLMs to external tools remains explicitly open.

References

However, understanding the most effective interface between LLMs and external tools remains an open question.

— Querying Databases with Function Calling  (2502.00032 - Shorten et al., 23 Jan 2025) in Section 1: Introduction (page 1)

A comparison with an agent in the same harness that queries a graph database through a query language such as Cypher or SPARQL~\citep{feng2024cypherbench} is also left to future work.

— Knowledge Graphs as Files: An Agent Harness Where Tool Creation and Tool Use Co-Evolve  (2610.18616 - Bai et al., 13 Oct 2026) in Section: Limitations, subsection “Scope”