Verifying TTT benefits without ground truth or reliable verifiers

Determine whether test-time training improved an individual query when deployment lacks ground-truth answers or reliable task-specific verifiers.

Background

The benchmark evaluates adaptation benefits using explicit reference answers or executable tests, which provide reliable utility estimates. In deployment, however, such outcome feedback is often unavailable.

Without ground truth or a reliable verifier, the policy cannot directly establish whether test-time training helped on a particular query. The paper states that resolving this verification challenge is outside its scope.

References

Without ground truth or a reliable verifier, we cannot directly confirm whether TTT helped on an individual query. The policy therefore draws on training experience to select actions with positive expected returns on similar deployment queries. This is a general challenge for TTT without outcome feedback, and resolving it falls outside the scope of this work.

— Agentic-TTT: Training test-time policy for test-time training  (2610.12002 - Lu et al., 8 Oct 2026) in Appendix E, Section E, “Verifiability of TTT benefits”