---
title: 'LENS: Latent Embeddings for Nonequilibrium Systems'
url: https://www.emergentmind.com/topics/latent-embeddings-of-nonequilibrium-systems-lens
type: topic
---

# LENS: Latent Embeddings for Nonequilibrium Systems

Searching arXiv for recent papers on “Latent Embeddings of Nonequilibrium Systems” and closely related nonequilibrium latent-representation work.
Latent Embeddings of Nonequilibrium Systems (LENS) is a model-free representation-learning method for discovering low-dimensional latent variables that specifically encode the irreversible, entropy-producing degrees of freedom of a high-dimensional stochastic dynamical system in a nonequilibrium steady state (NESS). Its defining construction is a forward-versus-time-reversed trajectory classifier whose logit is constrained to a quadratic form in learned state representations, so that the embedding is optimized not for reconstruction fidelity or variance preservation, but for preserving time-reversal asymmetry and the irreversible flow structure relevant to entropy production [2508.08247].

## 1. Definition, scope, and scientific objective

LENS targets high-dimensional stochastic systems observed through time series data and assumed to be in a steady state but not necessarily equilibrium. In that setting, the stationary distribution may be time-independent while detailed balance is broken, so there are circulating probability currents in state space and these currents generate entropy. The central problem addressed by LENS is therefore not generic compression, but identification of latent nonequilibrium degrees of freedom: given observations $\mathbf{x}_t$, learn a low-dimensional embedding $\boldsymbol{\phi}_\theta(\mathbf{x}_t)\in\mathbb{R}^M$ such that the induced latent dynamics retain the irreversible flows most relevant to entropy production [2508.08247].

This objective distinguishes LENS from standard dimensionality reduction. The method is explicitly motivated by the observation that reconstruction-oriented criteria need not align with physical relevance: a feature can be highly variable yet thermodynamically irrelevant, while another can contribute strongly to time-reversal asymmetry despite being subtle in reconstruction space. LENS therefore asks what low-dimensional coordinates best preserve the system’s irreversible flow structure. In the authors’ formulation, the appropriate latent space is the one that best supports discrimination between forward and reversed dynamics and thus best preserves entropy-producing currents [2508.08247].

A closely related implication is that LENS should be read as a coarse-graining framework for nonequilibrium steady states rather than as a generic latent-variable model. Its learned coordinates are intended to identify which degrees of freedom carry irreversibility, not merely to produce a compact state code.

## 2. Thermodynamic basis: irreversibility, time reversal, and entropy production

The thermodynamic foundation of LENS is the mismatch between probabilities of forward and time-reversed transitions. For adjacent states $\mathbf{x}$ and $\mathbf{x}'$, the entropy production rate is defined as the Kullback–Leibler divergence between forward and reverse transition statistics,
\[
\dot{S} = D_{\mathrm{KL}}\big[p(\mathbf{x}\rightarrow \mathbf{x}') \,\|\, p(\mathbf{x}'\rightarrow \mathbf{x})\big]
= \mathbb{E}_{p(\mathbf{x}\rightarrow \mathbf{x}')} \log \!\left[ \frac{p(\mathbf{x}\rightarrow \mathbf{x}')}{p(\mathbf{x}'\rightarrow \mathbf{x})} \right].
\]
The key local object is therefore
\[
S(\mathbf{x},\mathbf{x}') = \log \!\left[ \frac{p(\mathbf{x}\rightarrow \mathbf{x}')}{p(\mathbf{x}'\rightarrow \mathbf{x})} \right],
\]
which is antisymmetric under time reversal and averages to the entropy production rate [2508.08247].

LENS exploits the fact that a binary classifier trained on adjacent state pairs labeled as forward or reverse has a Bayes-optimal posterior
\[
C^*(\mathbf{x},\mathbf{x}') = \frac{p(\mathbf{x}\rightarrow \mathbf{x}')}{p(\mathbf{x}\rightarrow \mathbf{x}') + p(\mathbf{x}'\rightarrow \mathbf{x})},
\]
and therefore an optimal logit
\[
S^*(\mathbf{x},\mathbf{x}') =
\log\!\left[ \frac{p(\mathbf{x}\rightarrow \mathbf{x}')}{p(\mathbf{x}'\rightarrow \mathbf{x})} \right].
\]
Thus forward-versus-reverse classification is not a heuristic proxy: at optimum it estimates the local entropy production increment. Averaging the learned logit over forward samples gives an entropy-production estimate [2508.08247].

The process class considered in the formal development is stationary Markovian dynamics. In the continuous stochastic setting, the underlying dynamics are written as
\[
d\mathbf{x} = f(\mathbf{x},t)\,dt + \mathbf{F}\,d\mathbf{W}_t,
\]
with diffusion tensor
\[
\mathbf{D} = \frac{1}{2}\mathbf{F}\mathbf{F}^\intercal.
\]
In steady state, the probability current is
\[
\mathbf{J}_{ss}(\mathbf{x}) = f(\mathbf{x})\rho_{ss}(\mathbf{x}) - \mathbf{D}\nabla \rho_{ss}(\mathbf{x}),
\]
and the local velocity field is
\[
\mathbf{u}(\mathbf{x}) = \frac{\mathbf{J}_{ss}(\mathbf{x})}{\rho_{ss}(\mathbf{x})}.
\]
For short-time increments $\boldsymbol{\delta}=\mathbf{x}'-\mathbf{x}$, the target logit has the expansion
\[
S(\mathbf{x},\mathbf{x}') =
\boldsymbol{\delta}^\intercal \mathbf{D}^{-1}\mathbf{u}(\mathbf{x})
+ \frac{1}{2}\boldsymbol{\delta}^\intercal \mathbf{D}^{-1}(\nabla \mathbf{u}(\mathbf{x}))\boldsymbol{\delta}
+ \mathcal{O}(\|\boldsymbol{\delta}\|^3),
\]
which makes explicit that the forward/reverse log ratio depends on the local irreversible flow field $\mathbf{u}$ rather than on variance-dominant directions [2508.08247].

## 3. LENS formulation: latent embedding, quadratic logit, and optimization

LENS defines a classifier
\[
C_\theta(\mathbf{x},\mathbf{x}') = \sigma\!\big(S_\theta(\mathbf{x},\mathbf{x}')\big),
\qquad
\sigma(z)=\frac{1}{1+e^{-z}},
\]
trained with the binary cross-entropy objective
\[
J(\theta)= \mathbb{E}_{p(\mathbf{x}\rightarrow \mathbf{x}')} \left[\log \sigma\!\left(S_\theta(\mathbf{x},\mathbf{x}')\right)\right] + \mathbb{E}_{p(\mathbf{x}'\rightarrow \mathbf{x})} \left[ \log \left(1-\sigma\!\left(S_\theta(\mathbf{x},\mathbf{x}')\right)\right) \right].
\]
With sufficient expressivity and data, the optimal logit satisfies
\[
S_{\text{opt}}(\mathbf{x},\mathbf{x}') =
\log\!\left[ \frac{p(\mathbf{x}\rightarrow \mathbf{x}')}{p(\mathbf{x}'\rightarrow \mathbf{x})} \right],
\]
and the entropy production rate is estimated by
\[
\dot S \approx \mathbb{E}_{p(\mathbf{x}\rightarrow \mathbf{x}')} S_\theta(\mathbf{x},\mathbf{x}').
\]
This establishes LENS simultaneously as a latent-variable method and as a neural entropy-production estimator [2508.08247].

Its distinctive architectural feature is the structured parameterization of the logit in a learned representation $\boldsymbol{\phi}_\theta(\mathbf{x})\in\mathbb{R}^M$:
\[
S_\theta(\mathbf{x},\mathbf{x}') =
\boldsymbol{\phi}_\theta(\mathbf{x}')^\intercal \mathbf{A}_\theta \boldsymbol{\phi}_\theta(\mathbf{x})
+ \frac{1}{2}\boldsymbol{\phi}_\theta(\mathbf{x}')^\intercal \mathbf{B}_\theta \boldsymbol{\phi}_\theta(\mathbf{x}')
- \frac{1}{2}\boldsymbol{\phi}_\theta(\mathbf{x})^\intercal \mathbf{B}_\theta \boldsymbol{\phi}_\theta(\mathbf{x}).
\]
Here $\mathbf{A}_\theta$ is constrained to be skew-symmetric and $\mathbf{B}_\theta$ is constrained to be symmetric. The skew-symmetric term captures irreversible circulation between two states, while the difference-of-potentials term supplies the most general antisymmetric quadratic correction consistent with the short-time expansion and the linear-theory form [2508.08247].

The paper further constrains $\mathbf{A}_\theta$ to have a block-diagonal form with $2\times 2$ skew-symmetric blocks. Each block corresponds to a two-dimensional latent subspace carrying a rotational or circulatory irreversible flow. This gives the latent space a specific interpretive structure: each pair of latent coordinates can encode a local rotational current. The representation is not unique; the paper notes invariance up to permutations of $2\times2$ subspaces, rotations within each subspace, and additional ambiguity if eigenvalues are degenerate [2508.08247].

The workflow is simple. One collects a trajectory $\{\mathbf{x}_t\}_{t=1}^T$, forms forward pairs $(\mathbf{x}_t,\mathbf{x}_{t+1})$ and reverse pairs $(\mathbf{x}_{t+1},\mathbf{x}_t)$, encodes each state separately with $\boldsymbol{\phi}_\theta$, computes the quadratic logit, trains the classifier by binary cross-entropy, and then uses the learned score to estimate local and global entropy production. Latent trajectories and averaged flow fields in $\phi$-space can then be visualized, with cyclic currents interpreted as low-dimensional irreversible flows [2508.08247].

## 4. Linear theory, universality, and physical interpretation

A major strength of LENS is that its architecture is not only heuristic. For linear stochastic dynamics
\[
d\mathbf{x} = \mathbf{G}\mathbf{x}\,dt + \mathbf{F}\,d\mathbf{W}_t,
\]
with steady-state covariance $\mathbf{C}$ satisfying
\[
\mathbf{G}\mathbf{C} + \mathbf{C}\mathbf{G}^\intercal = -2\mathbf{D},
\]
the steady-state velocity field is
\[
\mathbf{u}(\mathbf{x}) = \left(\mathbf{G} + \mathbf{D}\mathbf{C}^{-1}\right)\mathbf{x}.
\]
Defining
\[
\mathbf{\Lambda} = \mathbf{D}^{-1}\mathbf{G} + \mathbf{C}^{-1},
\]
the exact target logit is
\[
S(\mathbf{x},\mathbf{x}') =
\mathbf{x}'^\intercal \mathbf{\Lambda}_a \mathbf{x}
+ \frac{1}{2} \left( \mathbf{x}'^\intercal \mathbf{\Lambda}_s \mathbf{x}' - \mathbf{x}^\intercal \mathbf{\Lambda}_s \mathbf{x} \right),
\]
where
\[
\mathbf{\Lambda}_s = \frac{1}{2}\left(\mathbf{\Lambda}+\mathbf{\Lambda}^\intercal\right),
\qquad
\mathbf{\Lambda}_a = \frac{1}{2}\left(\mathbf{\Lambda}-\mathbf{\Lambda}^\intercal\right).
\]
This is exactly the rationale for the LENS quadratic architecture [2508.08247].

In the linear case, LENS with linear representations $\boldsymbol{\phi}_\theta(\mathbf{x})=\mathbf{P}\mathbf{x}$ learns a low-rank approximation to the thermodynamic force matrix. The paper writes the induced rank-$M$ approximation as
\[
\tilde{\mathbf{\Lambda}} = \mathbf{P}^\intercal(\mathbf{A}_\theta + \mathbf{B}_\theta)\mathbf{P},
\]
and shows that maximizing the LENS objective is equivalent to minimizing a reduced objective over $\tilde{\mathbf{\Lambda}}$. The stationary points satisfy
\[
\tilde{\mathbf{\Lambda}} = \mathbf{D}^{-1}\mathbf{G}+\mathbf{C}^{-1} = \mathbf{\Lambda}.
\]
The ensemble entropy production rate is
\[
\dot{S} =
\int d\mathbf{x}\, \frac{\mathbf{J}_{ss}(\mathbf{x})^\intercal \mathbf{D}^{-1}\mathbf{J}_{ss}(\mathbf{x})}{\rho_{ss}(\mathbf{x})}
=
\mathrm{Tr}\!\left[\mathbf{\Lambda}^\intercal \mathbf{D}\mathbf{\Lambda}\mathbf{C}\right].
\]
The paper further states that the entropy-production approximation improves monotonically with latent dimension $M$ and reaches zero error when $M=N$ [2508.08247].

The supplement also gives a universality result: any antisymmetric short-time irreversibility function on a compact domain can be approximated arbitrarily well by a LENS-style logit,
\[
\hat{S}(\mathbf{x},\mathbf{y}) =
\phi(\mathbf{y})^\intercal \mathbf{A}\phi(\mathbf{x})
+ \frac{1}{2}\phi(\mathbf{y})^\intercal \mathbf{B}\phi(\mathbf{y})
- \frac{1}{2}\phi(\mathbf{x})^\intercal \mathbf{B}\phi(\mathbf{x}).
\]
The proof uses Stone–Weierstrass approximation with monomial feature bases, and a corollary shows that the skew-symmetric part can be brought into canonical block-diagonal $2\times2$ form without loss of approximation power [2508.08247].

These results justify the central interpretation of LENS: the learned coordinates are not intended to reconstruct the observation space but to organize the system into coordinates where irreversible probability currents are most visible. In the linear setting this interpretation is exact; more generally it is the design principle of the method.

## 5. Empirical validation and domains of application

The paper evaluates LENS on three main classes of systems. The first is a linear benchmark: a coupled bead-spring system with thermal gradients, with dynamics
\[
\mathbf{x}_{t+\Delta t} = \mathbf{x}_t + \mathbf{G}\mathbf{x}_t\,\Delta t + \mathbf{F}\boldsymbol{\xi}_t\sqrt{\Delta t}.
\]
For $N=2$ beads, using $T=10^7$ observations and $\Delta t=10^{-2}$, LENS estimates the entropy production rate within about $10\%$ error across $6$ orders of magnitude, for both linear and nonlinear encoders. For $N=16$ beads under four temperature profiles—linear, Fourier, sine, and alternating—the entropy-production error decreases monotonically with latent dimension $M$. For $M=2$, LENS captures about $25\%$ of the total entropy production and reveals a clear circulating latent flow. A notable structural result is that for $M=1$, LENS estimates zero entropy production, because the irreversibility-carrying skew-symmetric term requires at least a two-dimensional subspace. Supplementary experiments at $N=32,64$ show that lower-dimensional representations still capture a significant fraction of entropy production, while full dimension recovers essentially the full value. The method also remains effective when the bead positions are rendered into blurred one-dimensional image-like observations, indicating robustness to partial observation [2508.08247].

The second benchmark is a nonlinear driven particle in a periodic potential,
\[
x_{t+1}=x_t+\left(h-\frac{dU}{dx}\right)\Delta t + \sqrt{2D\Delta t}\,\xi_t,
\qquad
U(x)=U_0\cos\left(\frac{2\pi x}{L}\right).
\]
Using a sine activation to reflect periodicity, LENS with $M=2$ captures the entropy production rate well, with reported error within about $3\%$ across drive strengths. The paper notes that without the sine activation, optimization is unstable and linear encoders fail to learn the periodic irreversibility reliably. This is an important methodological constraint: LENS is model-free in the sense of not requiring a mechanistic reduced model, but architecture still matters [2508.08247].

The third system is the two-dimensional Complex Ginzburg–Landau equation,
\[
\frac{\partial \tilde A(\mathbf{r},t)}{\partial t} =
\tilde A(\mathbf{r},t) + (1+i\alpha)\nabla^2 \tilde A(\mathbf{r},t)
- (1+i\beta)|A|^2\tilde A(\mathbf{r},t),
\]
with $\tilde A(\mathbf{r},t)=|A(\mathbf{r},t)|e^{i\varphi(\mathbf{r},t)}$. Here the authors emphasize an important caveat: because the dynamics are deterministic, strict forward/reverse density overlap vanishes and the true entropy-production estimate diverges logarithmically. LENS is therefore not interpreted as a calibrated stochastic entropy-production estimator in this setting. Instead it serves as an irreversibility-sensitive representation learner. Using CNN encoders on $64\times64$ phase-field images, LENS detects the transition from frozen or stable spiral states to defect turbulence. Low-dimensional embeddings with $M=2$ show cyclic latent flows in both regimes, but the local entropy-production maps differ strongly across the transition; larger $M$ improves performance in the chaotic regime [2508.08247].

Across these experiments, the common empirical message is that LENS identifies low-dimensional circulating irreversible flows better than PCA, works on partially observed image-like data, and scales to high-dimensional complex spatiotemporal patterns. The paper’s own summary is that LENS accurately estimates entropy production in analytically tractable linear and nonlinear systems while also exposing the directions that carry irreversibility [2508.08247].

## 6. Related research, acronym ambiguity, and limitations

Several adjacent lines of work clarify what LENS is and what it is not. Physics-informed graph neural networks for variational nonequilibrium optimal control learn structured internal representations of local many-body environments and use them to approximate controlled dynamics associated with rare fluctuations, but they are control-force ansätze rather than explicit latent-space models [2204.05493]. Autoencoder-based unsupervised phase identification in sheared binary colloids constructs a low-dimensional latent representation of driven steady states from species-resolved Fourier amplitudes and uses a Gaussian mixture model for probabilistic phase identification, which is closer to latent manifold learning than to thermodynamic irreversibility learning [2607.05860]. A variational autoencoder trained on tunnel-coupled one-dimensional Bose-gas interference data learns a minimal latent representation from equilibrium snapshots and then uses that coordinate as an out-of-distribution probe for nonequilibrium defects and post-quench anomalies, again emphasizing latent interpretability but not forward/reverse irreversibility as the training signal [2509.13821]. By contrast, classifier-based analysis of many-body localization snapshots shows that raw projective measurements contain enough information for neural networks to diagnose thermalization behavior, yet the learned organization remains implicit and is not exposed as a latent embedding [2012.11586]. A different but conceptually allied direction is a latent-variable generative model whose hidden-state Markov chain reaches a nonequilibrium steady state with persistent probability currents and finite entropy production, so that directed cycles emerge inside the latent space itself [2512.11415].

The acronym is also genuinely ambiguous. In molecular simulation, “LENS” has been used to denote “Local Environments and Neighbors Shuffling,” a handcrafted descriptor that measures local neighbor-identity reshuffling over time. That method is relevant to dynamical heterogeneity, but it is not a latent embedding in the machine-learning sense and does not share the forward/reverse classifier construction of Latent Embeddings of Nonequilibrium Systems [2212.12694].

The limitations of LENS are explicit. It is built for systems in a nonequilibrium steady state; if the process is not stationary, forward/reverse discrimination can capture transient drift rather than steady irreversibility. The formulation uses adjacent states and therefore assumes a Markovian short-lag description; hidden variables, coarse sampling, or non-Markovian observations may degrade performance. The theory relies on short-time expansions, so if the sampling interval is too large the quadratic short-time approximation may become less faithful. The empirical studies also use substantial data—$10^7$ samples for bead and periodic-potential systems and $10^5$ images for Complex Ginzburg–Landau—so sample efficiency remains a practical concern. Low-dimensional embeddings can underfit intrinsically complex irreversible dynamics, as indicated by spurious negative patches in low-$M$ local entropy-production maps for the chaotic Ginzburg–Landau regime. Finally, the latent space is identifiable only up to rotations and permutations within irreversibility-carrying subspaces [2508.08247].

These comparisons suggest a broader taxonomy. Some nonequilibrium representation-learning methods organize rare-event structure through learned control forces [2107.03348], some through tractable learned distributions over tilted ensembles [2208.08266], some through phase-discriminative latent manifolds [2607.05860], and some through generative minimal coordinates for many-body snapshots [2509.13821]. LENS is distinguished within this landscape by making time-reversal asymmetry itself the representation objective. Its latent coordinates are optimized not for reconstruction, not for equilibrium phase separation, and not for control alone, but for preserving the irreversible currents responsible for entropy production in nonequilibrium steady states [2508.08247].

Source: https://www.emergentmind.com/topics/latent-embeddings-of-nonequilibrium-systems-lens