Online Temperature Selection Methods

• Online Temperature Selection is an adaptive process that dynamically tunes temperature parameters using real-time feedback to optimize performance in complex systems.
• It utilizes iterative feedback techniques such as swap-rate statistics, Bayesian inference, and gradient optimization to enhance sampler efficiency and control responses.
• Practical applications include replica exchange Monte Carlo, quantum annealing adjustments, and user-driven HVAC control, yielding improved sampling accuracies and energy savings.

Online temperature selection methods provide adaptive, data-driven procedures for choosing temperature or temperature-like parameters in computational statistics, optimization, simulation, and control, with the goal of maximizing efficiency or personalization in complex, often nonlinear systems. Recent research spans applications in Markov chain Monte Carlo (MCMC), quantum/thermal annealing, building heating control, and user preference elicitation. Online schemes iteratively update temperature parameters based on real-time feedback from performance statistics, observable system responses, or Bayesian inference.

1. Principles and Objectives of Online Temperature Selection

The central objective of online temperature selection is the dynamic adjustment of temperature or analogous parameters to optimize system performance, sampler efficiency, or user-specific outcomes in the face of uncertainty, nonstationarity, or multimodality. In computational physics and statistics, this usually translates to configuring temperature ladders in parallel tempering or replica-exchange Monte Carlo (RXMC) such that swap rates between adjacent temperatures are uniform, round-trip times are minimized, and equilibrium sampling is facilitated despite phase transitions or energy barriers (Hamze et al., 2010, Miyata et al., 20 Jan 2026). In user-facing domains such as HVAC control or preference elicitation, online selection seeks to minimize the number of queries or interventions required to localize optima in an unknown utility function (Awalgaonkar et al., 2019). In annealing-based inference, online temperature adaptation is essential to counteract deviations between nominal and realized sample distributions, which arise from ergodicity breaking, device noise, or parameter drift (Raymond et al., 2016).

2. Algorithmic Frameworks in Statistical Physics and Sampling

In parallel tempering and RXMC, a collection of $M$ replicas parameterized by a ladder of temperatures $\{T_k\}$ or inverse temperatures $\{\beta_k\}$ explores configuration space via inter-replica exchanges. Poorly spaced ladders create swap-rate bottlenecks at phase transitions or regions of large heat-capacity variability, leading to sluggish round-trip diffusion and biased sampling. Key online selection procedures include:

• Iterative Ladder Feedback (Feedback-Optimized PT): Parameters are periodically respaced so that the cumulative “flow fraction” $f(\lambda_k)$ aligns with a linear profile, i.e. $f(\lambda_k) \simeq 1-(k-1)/(M-1)$. Empirical swap-rate statistics and up/down replica counters are collected during each sampling block, and a smoothed, interpolated mapping constructs the next ladder. Wide intervals violating a target minimum swap rate $\alpha_{\min}$ are split or clamped (Hamze et al., 2010).
• Score-Function Gradient Optimization: A refined approach parameterizes the ladder in terms of log-intervals $L_k = \log(\Delta \beta_k)$ where $\Delta\beta_k = \beta_{k+1} - \beta_k$, strictly preserving $\beta_1 < \beta_2 < \dots < \beta_M$. The loss function is the variance of expected acceptance rates between adjacent replicas:

$$L(\boldsymbol\beta) = \frac{1}{M-1}\sum_{i=1}^{M-1}\left(\mathbb{E}[A_{i,i+1}]-\bar{A}\right)^2$$

Gradients with respect to $L_k$ are estimated online using empirical statistics from short RXMC runs and the “score function” method. Parameters are updated via (stochastic) gradient descent, preventing constraint violation and ensuring convergence to uniform acceptance probabilities (Miyata et al., 20 Jan 2026).

3. Online Estimation and Control in Quantum and Classical Annealing

In annealing-based sampling devices or algorithms, the realized sample distribution $P_A(x)$ generally deviates from the intended Boltzmann law $B_\beta(x)\propto \exp(-\beta H(x))$ due to freeze-out, nonergodicity, or device drift. Online temperature selection addresses this by embedding temperature estimators into a feedback loop:

• Estimator Types:
  • Global estimators (maximum-likelihood (ML), minimum mean-square error (MSE) on correlations) extract the “global” effective $\beta$ by matching energy or correlation statistics between $P_A$ and $B_\beta$.
  • Local estimators (maximum log pseudo-likelihood, MLPL) match one-spin-flip conditional statistics and can overreport effective $\beta$ when global mixing is lost (Raymond et al., 2016).
• Adjustment Procedure: The loop repeatedly updates a control parameter (e.g., coupling rescaling $r$) in the annealer to push the measured effective $\beta$ towards a user-specified target $\beta^*$:

$$r^{(k+1)} = r^{(k)} \times \frac{\beta^*}{\beta_{\text{est}}}$$

Choice of estimator impacts convergence speed and sensitivity to global vs. local deviations (see detailed pseudocode in (Raymond et al., 2016)).

• Pitfalls and Corrections: Ergodicity breaking saturates the achievable $\beta$, which global estimators can detect; local estimators may mislead. Sampling correlation, device drift, and outlier configurations require regular recalibration and validation.

4. Online Temperature Preference Learning in Human-in-the-Loop Systems

For personalization tasks such as HVAC setpoint optimization, the online temperature selection problem becomes one of sequential experimental design for a latent, unimodal utility function $u(T)$:

• Latent Utility Modeling: User preference over temperature is modeled as a real-valued Gaussian process (GP) with a built-in unimodality constraint, i.e., $u'(T)>0$ for $T<c_0$, $u'(T)<0$ for $T>c_0$, enforced via latent virtual observations and a monotonic GP prior (Awalgaonkar et al., 2019).
• Bayesian Update: After each query $(x_i, y_i)$—where $y_i$ is the user’s warmer/satisfied/cooler response—the GP posterior is sampled (via Hamiltonian Monte Carlo) to encode updated knowledge of $u(x)$.
• Acquisition Strategy: The next temperature to query is selected by maximizing the expected improvement (EI) criterion over the posterior utility:

$$EUI(x^*) = \mathbb{E}[ \max\{u(x^*) - \bar u_{\text{best}}, 0\}]$$

This balances exploration and exploitation, and allows rapid localization of the user’s optimum with minimal queries.

• Empirical Results: Synthetic and field studies show that 5–10 queries per occupant suffice for localization within $0.5$–$1^\circ$C intervals—at least a $2\times$–$5\times$ reduction versus random or static querying (Awalgaonkar et al., 2019).

5. Online Building Control: Data-Driven Supply Temperature Adaptive Methods

In the context of optimizing building heating system efficiency, online selection refers to the real-time adaptive control of the supply temperature setpoint (“heat-curve”) as a function of outdoor temperature, using only coarse-grained information:

• Data-Driven LMTD Modeling: Using building-level heat demand data $Q_\mathrm{meas}(t)$ and outdoor temperature $T_\mathrm{out}(t)$, plus static heater/building characteristics, the heat transfer law for each radiator is formulated using the logarithmic mean temperature difference (LMTD). For each time window, a constrained linear system partitions demand to rooms and heaters, uses nominal datasheet values, and computes the minimum supply temperature $T_{\text{sup},h,\text{mod}}$ to satisfy partial load requirements (Stock et al., 2023).
• Online Aggregation and Control:
  • Windowed time-series are clustered (e.g., by time of day).
  • For each cluster and outdoor temperature bin, the algorithm computes the minimal required supply temperature per heater/room, and sets the building-level setpoint as the maximum required across all heaters.
  • Smoothing and extrapolation yield a heat-curve which is updated online in the BMS (building management system), with practical update cycles (e.g., every night) and fallback strategies for robustness.
• Effectiveness: Case studies demonstrated several Kelvin reductions in supply temperature and 4–6% energy savings at the site scale, with no comfort complaints or overextension of valve positions (Stock et al., 2023).

6. Practical Algorithmic Implementations

The variety of online temperature selection algorithms across domains is reflected in their respective inputs, update rules, and outputs:

Domain	Input	Update Rule	Output
Parallel Tempering/RXMC	Swap/flow statistics	Ladder respacing, gradient step	Temperature ladder
Quantum Annealing	Sample statistics	Empirical $\beta$ estimation	Device scaling param.
User preference (HVAC)	Query response	Bayesian GP posterior, EI	Next query temperature
Building heating	Demand, $T_\mathrm{out}$	LMTD, clustering, aggregation	Heat-curve setpoints

Implementation details (tuning hyperparameters, window sizes, batch sizes, gradient estimators) are discussed in the respective references (Hamze et al., 2010, Miyata et al., 20 Jan 2026, Stock et al., 2023, Awalgaonkar et al., 2019). In all cases, successful deployment depends on appropriate estimator choice, regular validation, and domain-specific heuristics for initialization and smoothing.

7. Limitations and Assessment

While online temperature selection delivers substantial efficiency, adaptability, and robustness, its limitations include:

• Nonconvexity and Bottleneck Effects: In parallel tempering and RXMC, severe phase transitions can introduce persistent bottlenecks despite optimization; iteration or smoothing parameters must be carefully tuned (Hamze et al., 2010).
• Statistical Accuracy: Small-batch estimation or estimator mismatch (e.g., using local MLPL in a globally nonergodic system) can systematically bias online temperature estimates (Raymond et al., 2016).
• Model Assumptions: In the data-driven building control context, the assumption of steady-state heat transfer, fixed supply-return deltas, and neglect of inter-room flows may lead to conservative or sometimes suboptimal recommendations (Stock et al., 2023).
• Human-in-the-loop Overhead: For utility learning, online active querying must balance occupant burden with achievable convergence rates; models with inappropriate priors may require more samples (Awalgaonkar et al., 2019).

Overall, online temperature selection frameworks encode a general paradigm of real-time, feedback-driven adaptation in complex thermodynamic, physical, and user-centric systems, with substantial evidence for their pragmatic value and rigorously established algorithmic foundations (Hamze et al., 2010, Miyata et al., 20 Jan 2026, Raymond et al., 2016, Stock et al., 2023, Awalgaonkar et al., 2019).