Ascertain tool usage in closed-weight API models

Ascertain whether closed-weight large language models accessed via APIs employ external tools—such as calculators or theorem provers—within their inference pipelines, which could affect measured reasoning gaps.

Background

Reasoning gap measurements could be confounded if models rely on external tools during inference, especially for arithmetic or symbolic tasks. The authors explicitly note uncertainty regarding tool usage in closed-weight API-served models and suggest future evaluations controlling for tool usage with open-weight models.