Generalize the instability ordering across model families and temperatures

Determine whether the ordering of response instability—highest for self-referential subjective-experience questions, intermediate for unresolvable philosophical questions, and lowest for verifiable questions—holds for other large language model families and at sampling temperatures other than 0.7.

Background

The study evaluates only the Gemini model family and uses a single generation temperature of 0.7. Its main result is an ordering in semantic response instability across three question groups, with self-referential questions producing the least stable responses and verifiable questions producing the most stable responses.

Because the experiment does not vary either model family or temperature, it remains unresolved whether the observed ordering reflects a general property of self-referential elicitation or a behavior specific to Gemini at the selected sampling setting.

References

Whether the same ordering holds for other model families, or at other sampling temperatures, is untested.

— Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models  (2608.13258 - Balani et al., 13 Aug 2026) in Section 5, Limitations

These findings open as many questions as they close. First, which task properties set the direction and magnitude of the distortion: the exploratory subjective-task contrast motivates, without establishing, a systematic test that crosses objectivity, demonstrability, and answer familiarity, since a task's position on the intellective-to-judgmental continuum (Steiner, 1972; Laughlin et al., 2006) may govern whether belief-anchored groups over- or under-converge.

— Language-model groups overstate consensus when replaying human deliberation on a reasoning task  (2609.20543 - Shao, 17 Sep 2026) in Discussion, final paragraph