Transferability beyond Korean public-sector APIs

Determine how far the KOPA-Bench benchmark and EDGE data-synthesis pipeline transfer to other languages, commercial or private APIs, and administrative systems in other countries.

Background

KOPA-Bench and EDGE are constructed around Korean public-sector APIs spanning six domains. This controlled setting isolates multi-step tool calling with high-cardinality responses but does not establish whether the reported findings generalize to different languages, API ownership models, or national administrative contexts.

The unresolved issue concerns the external validity of both the benchmark and the synthesis pipeline: performance improvements and execution-grounded dependency modeling may depend on characteristics specific to Korean public APIs, such as their schemas, endpoint behavior, and domain organization.

References

This isolates the multi-step, high-cardinality setting we target, but leaves open how far our findings transfer to other languages, to commercial or private APIs, and to administrative systems in other countries.

— Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe  (2609.05395 - Kim et al., 4 Sep 2026) in Section: Limitations, paragraph “Beyond Korean public APIs”

As noted in \S\ref{sec:method}, we evaluate against ToolBench gold trajectories as a retrieval proxy; trajectory fidelity supports this as a valid proxy for planner behavior, though parameter correctness and final answer quality are not assessed. PtR's discovery and retrieval calls are sequential, adding one full LLM round-trip of latency before the planner begins; complexity-adaptive routing eliminates this overhead for G1-type queries. Token cost (reported in Appendix~\ref{app:cost}) is a proxy for this serial overhead structure; the resulting wall-clock latency depends on the serving infrastructure.

— Plan or Retrieve First? A Scale Study of Tool Discovery Architectures  (2610.19068 - Zhang et al., 13 Oct 2026) in Limitations section