Nonequilibrium Thermodynamics of RBM Learning

• The paper establishes nonequilibrium thermodynamics for RBM learning by linking energy-based models with phase transitions and stability insights.
• It introduces a rigorous framework using order parameters and Gibbs sampling to delineate equilibrium versus out-of-equilibrium training regimes.
• Practical strategies such as adaptive temperature regulation and controlled sampling mitigate instabilities like sampler freezing and weight divergence.

Restricted Boltzmann Machines (RBMs) are bipartite stochastic neural networks characterized by an energy-based probability distribution on binary visible and hidden units. The application of nonequilibrium thermodynamic principles to RBM-based deep learning provides a rigorous statistical mechanics framework for analyzing their learning dynamics, stability, and operational regimes. This perspective enables precise insight into algorithmic phenomena such as mode selection, phase transitions, mixing times, entropy production, and the irreversibility inherent in approximate (finite-time) Markov chain Monte Carlo (MCMC) training.

1. Thermodynamic Structure of RBMs

An RBM defines a joint probability distribution over visible units $v=(v_i)_{i=1}^{N_v}$ and hidden units $h=(h_j)_{j=1}^{N_h}$ with binary states $v_i, h_j \in \{\pm 1\}$ or $\{0,1\}$. The model's energy function is

$$E(v,h;W,\eta,\theta) = -\sum_{i,j} v_i W_{ij} h_j + \sum_i \eta_i v_i + \sum_j \theta_j h_j$$

and the Boltzmann distribution is

$$P(v,h) = Z^{-1} \exp\left[-E(v,h;W,\eta,\theta)/T\right]$$

with temperature $T$ and normalization $Z = \sum_{v,h} \exp[-E(v,h)/T]$. The visible marginal $P(v)$ is used for modeling data distributions. RBMs implement Markov chain dynamics via alternating Gibbs sampling that respects detailed balance, so the stationary distribution coincides with the Boltzmann law at temperature $T$ (Salazar, 2017).

Central to the thermodynamic approach is the introduction of order parameters whose evolution describes both the equilibrium phases and the nonstationary, data-driven dynamics of learning (Decelle et al., 2018).

2. Equilibrium Mean-Field Theory and Phase Structure

The equilibrium properties of RBMs can be described via a replica-symmetric mean-field (MF) theory in the thermodynamic limit—$N_v,N_h\to\infty$ at fixed ratio $\kappa=N_h/N_v$—with $K$ dominant “informative” singular value modes of $W$ plus a bulk of i.i.d. Gaussian noise. Order parameters include magnetizations $m_\alpha, \bar m_\alpha$, and Edwards–Anderson parameters $q, \bar q$: $$m_\alpha = \langle \sum_j v_j^\alpha h_j \rangle / \sqrt{L}, \quad \bar m_\alpha = \langle \sum_i u_i^\alpha v_i \rangle / \sqrt{L}$$ where $L = \sqrt{N_v N_h}$. The MF free energy $f(m, \bar m, q, \bar q)$ encodes the competition between ferromagnetic (compositional) order, spin glass ordering, and paramagnetic noise. The phase diagram contains three phases:

• Paramagnetic: $m_\alpha=0$, $q=\bar q=0$
• Ferromagnetic: some $m_\alpha, \bar m_\alpha \neq 0$
• Spin glass: $m_\alpha=0$, $q,\bar q>0$

Phase boundaries and replica-symmetry breaking transitions are determined by the spectrum of $W$ and the kurtosis $\kappa_u$ of its singular vectors: Gaussian $\kappa_u=3$ yields single-mode condensation; for $\kappa_u > 3$, a compositional ferromagnetic phase with simultaneous condensation of multiple modes arises (Decelle et al., 2018).

3. Nonequilibrium Learning Dynamics

Learning in RBMs proceeds via stochastic gradient ascent on the log-likelihood, typically using persistent or $k$-step contrastive divergence (CD) approximations for the intractable model average. The dynamical equations for the spectral components of $W$ in the SVD basis, and their associated order parameters, are of the form: $$\frac{1}{L}\frac{dw_\alpha}{dt} = \langle s_\alpha \sigma_\alpha \rangle_{\text{Data}} - \langle s_\alpha \sigma_\alpha \rangle_{\text{RBM}}$$ where the empirical data moments and the instantaneous model moments drive the system out of equilibrium. In the initial (linear) learning regime, the dominant modes align with the principal components of the data covariance, capturing major statistical structure. Beyond the ferromagnetic onset, nonlinear mode interactions emerge; convergence is attained when empirical and thermodynamic order parameters coincide, signifying representation of data clusters by stable MF attractors (Decelle et al., 2018).

Importantly, the flow of these order parameters, driven by mismatches between empirical data cumulants and model cumulants, constitutes a nonequilibrium process in high-dimensional parameter space.

4. Instability of Fixed-Temperature Finite-Time Training and Thermodynamic Regulation

In practical training, fixed-temperature, finite-length Gibbs sampling fails to attain true equilibrium. As weight norms $\|W_t\|$ increase during learning, effective local fields $|x_{t,i}(v)|/T$ may diverge, causing flip rates to vanish and Markov chain conductance to collapse (“freezing”). This results in the negative phase becoming localized about the initialization, a vanishing spectral gap, and ultimately, deterministic linear weight drift if $$\langle v_i h_j\rangle_\text{data} \neq \langle v_i h_j\rangle_\text{model}$$. These instabilities manifest as structural fragility in the learning dynamics—unbounded parameters and loss of model ergodicity (Süleymanoğlu, 3 Mar 2026).

Endogenous thermodynamic regulation introduces temperature as a dynamical state variable, controlled via feedback from sampling statistics such as the chain flip rate $r_t$. An integral feedback loop maintains $r_t$ near a smoothed reference $c_t$, adaptively updating $T_t = e^{\lambda_t}$. The regulated regime guarantees boundedness of parameters under mild $\ell_2$ regularization, local exponential stability of the thermodynamic subsystem, and a forward-invariant region immune to conductance collapse (Süleymanoğlu, 3 Mar 2026).

This regulatory framework prevents effective inverse temperatures from diverging, maintaining nonvanishing spectral gap and sampler ergodicity throughout training.

5. Operational Regimes: Equilibrium vs. Out-of-Equilibrium Training

RBM-based learning is governed by the relationship between the chain length $k$ (number of negative-phase Gibbs steps) and the model's intrinsic mixing time $\tau_\text{mix}$. Two operational regimes result (Decelle et al., 2021):

• Equilibrium regime: $k \gg \tau_\text{mix}$. Negative-phase samples are equilibrated, and the stochastic gradient is unbiased. Entropy production per update vanishes asymptotically.
• Out-of-Equilibrium (OOE) regime: $k \ll \tau_\text{mix}$. Markov chains do not mix, so the negative phase is systematically biased. The entropy production rate $$\sigma(t) = \frac{d}{dt} D_\mathrm{KL}(P^t \mu_0 \| \pi)$$ is strictly positive.

The mixing time $\tau_\text{mix}$ typically increases with training progression. Fixed-$k$ training thus almost invariably drifts from equilibrium to OOE at late stages unless $k$ is increased (Decelle et al., 2021). In the OOE regime, sampling quality is peaked at $t_G \approx k$, and model dynamics are dominated by strong nonequilibrium effects.

Nonequilibrium thermodynamic quantities such as dissipated work, heat exchange, and entropy production become central to understanding RBM performance in these realistic, finite-time regimes.

6. Fluctuation Theorems and Thermodynamic Functionals in RBM Learning

The learning dynamics of RBMs as driven, discrete-time Markov processes admit formal analogues of fluctuation theorems from nonequilibrium statistical mechanics. Crooks’ fluctuation theorem and the Jarzynski equality are realized, with stochastic work and heat defined along parameter-changing protocols. These theorems relate the statistics of work and free-energy changes along forward and backward training trajectories. Experimental verification (e.g., heat-exchange fluctuation theorem under temperature quenches) confirms the theoretical predictions (Salazar, 2017).

Contrastive divergence with $n$ steps, $CD_n$, is interpreted as the difference of Kullback–Leibler divergences between the data and model distributions after $n$ Gibbs steps, decomposable into a net entropy change and average heat exchanged: $$CD_n = S(p^{(n)}) - S(p_D) - \beta \langle Q_n \rangle$$ where $S$ is entropy and $Q_n$ is the heat exchanged in $n$ steps. In the limit $n \rightarrow \infty$, $CD_n$ converges to the irreversible work, and minimizing $CD_n$ corresponds to reducing dissipated work—driving the learning process toward thermodynamic efficiency (Salazar, 2017).

Annealed Importance Sampling (AIS) for partition function estimation is an explicit realization of a nonequilibrium work protocol, where free-energy differences are computed through sequential parameter interpolation and averaging of exponential work—an application of the Jarzynski equality (Salazar, 2017).

7. Implications for Deep Learning Architectures and Practical Methodology

RBMs constitute the building blocks for deeper architectures (e.g., Deep Belief Networks); each layer corresponds to an additional thermodynamic subsystem, with pre-training representing sequential work protocols that reshape the global energy landscape to capture data statistics (Salazar, 2017, Decelle et al., 2018). In all these cases, the nonequilibrium thermodynamic theory provides a unifying language for the convergence, stability, and representation power of layerwise learning.

For practical RBM training, monitoring and controlling thermodynamic quantities—flip rates, entropy production, mixing times—enables mitigation of fundamental failures such as sampler freezing and weight divergence (Süleymanoğlu, 3 Mar 2026). Adaptive regulation of temperature, adjustment of CD or PCD chain lengths $k$ in response to mixing time growth, and regularization are key control techniques justified by the nonequilibrium framework. These protocols ensure stable training and improved sample diversity and likelihood-based diagnostics, as quantitatively validated by MNIST experiments (Süleymanoğlu, 3 Mar 2026, Decelle et al., 2021).

The thermodynamic interpretation extends universally to broader classes of energy-based models, emphasizing the operational distinction between equilibrium and nonequilibrium regimes as an organizing principle for both model evaluation and algorithmic control (Decelle et al., 2021).