Determine whether the released model remains vulnerable to off-topic tool results

Determine whether the released Agaram 2.5B chat model continues to paraphrase irrelevant or empty tool results as if they were valid answers after targeted SFT training on such cases.

Background

Agaram 2.5B was trained to call lookup tools and answer from returned text, but the original tool-use training examples contained only relevant results. In an evaluation using a simple Wikipedia lookup, the lookup returned off-topic results for four of five questions, and the model paraphrased those results as answers in all four cases.

Round 2 added training examples involving irrelevant and empty tool results. However, the authors did not repeat the grounded-tool evaluation after this intervention, leaving unresolved whether the released model still exhibits the failure.

References

Round 2 added rows with irrelevant and empty tool results, but we have not re-run the grounded test, so we do not know whether the released model still does this.

— Agaram 2.5B: The First Tamil-First Language Model Trained from Scratch with Tool Calling. A Playbook for Under-Represented Languages  (2610.18420 - Kumaresan et al., 13 Oct 2026) in Section 7.5, subsection “Known failures” (Section \ref{sec:eval-known})