Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data

Abstract: Identifying the dynamical state variables of a system from high-dimensional observations is a central problem across physical sciences. The challenge is that the state variables are not directly observable and must be inferred from raw high-dimensional data without supervision. Here we introduce DySIB (Dynamical Symmetric Information Bottleneck) as a method to learn low-dimensional representations of time-series data by maximizing predictive mutual information between past and future observation windows while penalizing representation complexity. This objective operates entirely in latent space and avoids reconstruction of the observations. We apply DySIB to an experimental video dataset of a physical pendulum, where the underlying state space is known. The method, with hyperparameters of the learning architecture set self-consistently by the data, recovers a two-dimensional representation that matches the dimensionality, topology, and geometry of the pendulum phase space, with the learned coordinates aligning smoothly with the canonical angle and angular velocity. These results demonstrate, on a well-characterized experimental system, that predictive information in latent space can be used to recover interpretable dynamical coordinates directly from high-dimensional data.

• The paper introduces DySIB, a method that applies a symmetric information bottleneck to extract interpretable low-dimensional physical phase spaces from high-dimensional time-series data.
• It validates DySIB on experimental pendulum videos, accurately recovering canonical variables like angle and angular velocity with high sample efficiency.
• The method leverages mutual information maximization and variational inference to achieve robust, unsupervised phase space discovery and stable long-term forecasting.

Information Bottleneck for Learning Physical Phase Spaces from High-Dimensional Experimental Data

Overview

The paper "Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data" (2604.24662) introduces DySIB (Dynamical Symmetric Information Bottleneck), a principled methodology that extracts low-dimensional, physically interpretable latent representations from high-dimensional time-series data (e.g., raw video). This is achieved via an information-theoretic objective that maximizes the mutual information between past and future windows, symmetrically compressed into a latent space, while penalizing representation complexity. The approach is validated on experimental videos of a physical pendulum, demonstrating accurate, sample-efficient recovery of canonical phase space coordinates—angle and angular velocity—without supervision.

DySIB Framework and Objective

DySIB is grounded in the symmetric information bottleneck (SIB) formulation. Instead of maximizing reconstruction accuracy (as in autoencoders) or prediction in data space (as in autoregressive models), DySIB explicitly compresses past and future observation windows into latent representations $Z_X$ and $Z_Y$, maximizing their mutual information and enforcing predictive sufficiency entirely in the latent space. The loss function is:

$$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$

Here, $I(X; Z_X)$ and $I(Y; Z_Y)$ are regularization terms via variational KL divergence (using standard normal priors), acting as complexity penalties. The predictive term, $I(Z_X; Z_Y)$, is estimated with InfoNCE, using a $\delta$-predictor that models the future as a stochastic residual increment of the past latent.

The latent dimensionality ($k_z$) and temporal window size ($n_F$) are treated as hyperparameters, determined self-consistently by saturating predictive information and minimizing prober reconstruction error on physical coordinates.

Method Implementation

Each frame in a video sequence is mapped via a shared encoder $\Phi$ to a low-dimensional vector. Concatenated embeddings from a temporal window form the input to variational encoders $Z_Y$0 and $Z_Y$1, producing means and log-variances parameterizing diagonal Gaussian posteriors for $Z_Y$2 and $Z_Y$3. The architecture is symmetric for past and future windows, enforcing time-translation invariance.

The $Z_Y$4-predictor, a neural network, outputs the mean and log-variance of the future increment, yielding a conditional Gaussian distribution over the next latent state. InfoNCE is used for estimating mutual information, contrasting matched and mismatched latent pairs within a batch.

Figure 1: Ground truth and learned phase space of the pendulum. (A) Cartesian phase space: angular velocity vs. angle. (B) Polar projection handling periodicity of angle. (C) DySIB-derived latent space overlays the polar projection, matching phase structure.

Empirical Validation: Pendulum Phase Space Recovery

DySIB is applied to a dataset of experimental pendulum videos, with each trajectory annotated with canonical physical variables. The learned latent representations reproduce the topology, geometry, and dimensionality of the true phase space, with latent coordinates aligning smoothly to the angle ($Z_Y$5) and angular velocity ($Z_Y$6).

Self-consistent hyperparameter selection, sweeping $Z_Y$7 and $Z_Y$8, reveals sharp saturation of predictive information and probe accuracy at $Z_Y$9 and $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$0, coinciding with the system's two dynamical degrees of freedom and minimal temporal order needed to resolve velocity.

Figure 2: MI selects latent dimension. MI saturates beyond $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$1; angle and velocity prober RMSEs plateau.

Figure 3: MI selects temporal window. MI peaks at $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$2; angle prober requires $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$3, velocity prober requires $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$4.

DySIB exhibits high sample efficiency—accurate recovery of physical state variables is achieved with substantially reduced dataset size compared to conventional methods.

Figure 4: MI and prober errors as a function of training videos; DySIB achieves accurate $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$5 and $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$6 recovery with minimal data.

Latent Space Structure and Physical Interpretability

The learned latent space organizes all trajectories according to canonical physical variables. Angle and angular velocity vary smoothly as polar and radial coordinates, respectively; full rotations and small oscillations are faithfully encoded. Other physical quantities (energies) also map consistently onto the latent structure.

Figure 5: Latent space embeddings colored by (A) angle and (B) angular velocity, showing globally consistent encoding.

The phase space topology, including fixed points and separatrices, is recovered without explicit supervision, confirming the model's ability to extract interpretable dynamical coordinates directly from raw, high-dimensional input.

Figure 6: Geometric construction connects cylindrical and polar phase space representations, explaining the mapping found by DySIB.

Figure 7: Physical quantities (angle, velocity, kinetic/potential/total energy) vary smoothly and consistently with latent coordinates.

Predictive Latent Dynamics and Long-Term Forecasting

DySIB's $$\mathcal{L}_\mathrm{DySIB} = I(X; Z_X) + I(Y; Z_Y) - \beta I(Z_X; Z_Y)$$7-predictor defines a vector field in latent space, enabling long-term forecasting via stochastic rollouts. The integration of latent dynamics demonstrates qualitatively consistent behavior with ground truth trajectories—stable integration with linearly accumulating error and recovery of physical fixed points.

Figure 8: Vector field of DySIB latent dynamics and forecasting accuracy for stochastic rollouts.

Architectural Insights and Robustness

Overparameterization (using higher-than-minimal latent dimension) leads to reduced trial-to-trial variance without compromising structural correctness; the effective dimensionality of the learned latent space remains low as assessed by nonlinear estimators.

Figure 9: Increasing bottleneck size reduces variance without increasing intrinsic dimensionality.

Practical and Theoretical Implications

DySIB operationalizes the principle that physical phase space coordinates are precisely those most predictive of their own future. Unlike reconstruction-based objectives, prediction in latent space promotes the recovery of relevant dynamical variables with high sample efficiency and generalization.

Practically, DySIB offers a scalable method for phase space discovery in both physical and biological systems from high-dimensional, often multimodal measurements. Theoretically, the symmetric bottleneck objective recasts the search for effective variables as an information-theoretic generalization of the Landau program for order parameter selection, unifying latent prediction and physical modeling.

Future developments will center on generalizing DySIB to systems with unknown variables, noisy or weakly supervised measurements, and integrating learned latents with symbolic regression for equation discovery (e.g., SINDy, SPIDER). Validation on chaotic, multiscale, or biological systems and alternative architectures (e.g., attention, convolution) are promising directions.

Conclusion

DySIB introduces a robust, principled approach for extracting physically meaningful phase spaces from high-dimensional time-series data, validated in the experimental pendulum setting. The method achieves interpretable latent representations, sample-efficient variable discovery, and stable forecasting—all without supervision—demonstrating practical utility and theoretical significance for AI in scientific inference and dynamical modeling.