Temperature Effects on Ensemble-Based Variance Reduction

Determine why ensembling multiple responses at non-zero decoding temperature reduces cross-lingual response variance and improves target-language accuracy more effectively than ensembling at zero temperature for large language models, and evaluate the influence of model-specific temperature handling (e.g., mapping temperature 0 to a non-zero value) on this effect.

Background

In response ensembling experiments, the authors observe that default non-zero temperature yields better variance reduction and target accuracy than zero temperature, contrary to the expectation that zero temperature should suppress variance more.

They suggest that model-specific implementations may map zero temperature to a non-zero value, but explicitly state that the cause of the observed behavior is unclear, motivating targeted investigation.