Generalize the instability ordering across model families and temperatures

Determine whether the ordering of response instability—highest for self-referential subjective-experience questions, intermediate for unresolvable philosophical questions, and lowest for verifiable questions—holds for other large language model families and at sampling temperatures other than 0.7.

Background

The study evaluates only the Gemini model family and uses a single generation temperature of 0.7. Its main result is an ordering in semantic response instability across three question groups, with self-referential questions producing the least stable responses and verifiable questions producing the most stable responses.

Because the experiment does not vary either model family or temperature, it remains unresolved whether the observed ordering reflects a general property of self-referential elicitation or a behavior specific to Gemini at the selected sampling setting.

References

Whether the same ordering holds for other model families, or at other sampling temperatures, is untested.

— Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models  (2608.13258 - Balani et al., 13 Aug 2026) in Section 5, Limitations