Sources of Irregular Variability in Zero‑Temperature Offline Models

Identify the sources of the irregular variability observed in zero‑temperature, offline large language model responses over time that cannot be explained by randomness, exogenous events, or known model updates.

Background

Even with temperature set to zero and models operating offline, the authors observe nontrivial ‘noisy’ fluctuations in responses across days. These fluctuations are small but persistent and do not obviously reflect contemporaneous election events or randomness from sampling.

Because the setup was designed to minimize stochasticity and external influences, the presence of such variability suggests unknown internal or system-level factors. The authors explicitly note that they cannot definitively determine the source of this variation, marking it as an unresolved question.