Convolutional Symmetric AutoEncoders: enhancing latent stability via differential geometry

Abstract: Autoencoders (AEs) have emerged as powerful tools for non-linear dimensionality reduction, often surpassing traditional linear methods such as Proper Orthogonal Decomposition (POD) in scenarios characterized by slowly decaying Kolmogorov $n$-widths. In the realm of Reduced-Order Modelling (ROM), these models are increasingly utilized to learn low-dimensional representations of solution manifolds associated with parametric Partial Differential Equations (PDEs). However, the high expressivity of AEs presents a challenge: although trained networks typically minimize reconstruction error, they often struggle to capture the essential properties necessary for building accurate and robust ROMs. Recent works by arXiv:2307.15288v2 and arXiv:2506.11641v1 have tackled this challenge in fully connected AEs by proposing representation-consistent architectures, which preserve some of the properties belonging to POD. This study builds upon that concept by extending representation consistency for convolutional layers. We introduce a novel class of symmetric Convolutional AutoEncoders (CAEs) designed to embody the primary properties of manifold parametrization mappings. When integrated into a ROM framework, this architecture demonstrates significantly improved predictive capabilities. Specifically, we compared the performance of the ROMs based on classical and symmetric CAEs on three one dimensional academic test cases, namely the Linear Advection, the Viscous Burger and the Kuramoto Sivashinsky equation. Numerical results demonstrate that our proposed symmetric approach consistently yields more accurate latent trajectories, lower reconstruction errors, and enhanced model robustness.

• The paper introduces a symmetric convolutional autoencoder architecture that enforces representation consistency via bi-Lipschitz activations and bi-orthogonal layer pairs.
• The paper demonstrates robust latent stability and improved reconstruction accuracy for nonlinear PDEs across linear advection, viscous Burgers, and Kuramoto–Sivashinsky systems.
• The paper validates the approach by achieving lower reconstruction errors, faithfully matching Lyapunov spectra, and preserving invariant measures in surrogate PDE models.

Convolutional Symmetric AutoEncoders: Enhancing Latent Stability via Differential Geometry

Introduction and Theoretical Foundations

The paper "Convolutional Symmetric AutoEncoders: enhancing latent stability via differential geometry" (2607.00669) presents an advanced framework for nonlinear model order reduction (MOR) of parametric PDEs based on representation-consistent convolutional autoencoders (CAEs), with an emphasis on guaranteeing latent stability via architectural symmetry and differential geometric principles.

The work builds upon the limitations of both linear reduced order modeling (e.g., POD) and classical autoencoders, which suffer from sub-optimal manifold parameterization and a lack of architectural symmetry, impeding consistent and physically meaningful latent dynamics. The formulation draws directly from the manifold hypothesis, assuming the solution manifold of dissipative nonlinear PDEs is a finite-dimensional, differentiable structure embedded in the high-dimensional state space. The critical observation is that practical autoencoders generally fail to produce true diffeomorphic mappings between latent and physical spaces, leading to unphysical latent dynamics when employed in reduced-order models (ROMs).

Symmetric Convolutional AutoEncoder Architecture

The main technical innovation is a convolutional AE architecture that enforces representation-consistency—specifically, the property $E \circ D = id$ on the latent space—ensuring invertibility of the encoder/decoder pair and strict control over latent coordinate behavior. Representation consistency is made possible by two architectural constraints: 1) the use of bi-Lipschitz invertible activation functions, and 2) bi-orthogonal layer pairs, with decoder weights parameterized as generalized (often pseudoinverse-based) inverses of their associated encoder weights.

Figure 1: The encoder uses nonlinear convolutional layers with bi-Lipschitz activation, and the decoder is a symmetric, structured linear map ensuring $E \circ D = id$ on the latent space.

Invertible activation is achieved using a novel rational activation function, $$\sigma_\alpha(x) = \alpha \left( x + x/(1+x^2) \right)$$, with an analytic inverse, providing desirable numerical conditioning and strong global invertibility properties.

Figure 2: Rational activation $\sigma_\alpha$, its inverse $\sigma_\alpha^{-1}$, and Tanh for comparison, with $\alpha=0.5$.

The convolutional structure is rigorously extended to multiple channels and arbitrary stride, relying on circulant and block structures, and bi-orthogonality is enforced via projectors and Moore-Penrose pseudoinverse construction. A key insight is that the non-linear component of the decoder resides in the null space of the encoder, ensuring that representation consistency is maintained regardless of the complexity of decoder nonlinearity.

Data-Driven MOR and Latent Dynamics

In the ROM architecture, the encoder maps high-fidelity trajectories into latent coordinates, and the decoder provides reconstructions of physical fields. The temporal evolution of latent states is learned via a separate feedforward "Latent Flow Network" (LFNet) operating solely within the latent space, trained on data pairs of projected states and tangent vectors.

Figure 3: LFNet topology: concatenation of latent state, parameters, and time into a feedforward network predicting latent evolution.

This decoupled training regime first fits the AE to minimize reconstruction error, then optimizes the LFNet on latent derivatives determined by projecting FOM dynamics.

Benchmark Evaluation

The methodology is demonstrated on three canonical one-dimensional PDE systems representative of key MOR challenges: advection-dominated transport (linear advection), parametric advection-diffusion with steep gradients (Viscous Burgers), and spatiotemporal chaos (Kuramoto–Sivashinsky, KS).

Figure 4: Solutions of (left) Linear Advection, (middle) Viscous Burgers, (right) Kuramoto–Sivashinsky equations.

Linear Advection

POD’s sub-optimality for this class is evident in the slow singular value decay, requiring $l \simeq 20$ modes for sub-1% error.

Figure 5: Cumulative sum of squared singular values for the advection snapshot matrix $\boldsymbol{X}$.

Symmetric CAE-based ROMs achieve robust two-dimensional latent representations. Over 10 periods, classic CAEs produce wandering orbits in latent space, while symmetric CAEs maintain trajectories strictly confined to the expected closed curve.

Figure 6: Latent trajectories of 10 classic ROMs (left—unstructured) and 10 symmetric ROMs (right—stable on the unit circle).

Ensemble statistical errors in both physical and latent coordinates confirm consistently lower error and variance for symmetric CAEs.

Figure 7: Mean and variance of reconstruction error (top) and latent error (bottom) over 10-period test-set evolution for classical vs symmetric ROMs.

Viscous Burgers Equation

For nonlinear parametric PDEs, symmetric CAEs produce latent spaces where all parameter slices are geometrically smooth and well-separated. Without symmetry, latent trajectories exhibit topological defects (wiggles) and create unstable or unphysical forecasts.

Crucially, strong physical reconstruction results are observed: in both interpolation (training) and extrapolation (test/validation) regimes, symmetric CAE-ROMS accurately predict the location and height of shock fronts and smooth regions, while classic CAEs display systematic drift from high-fidelity solutions.

Figure 8: Spatio-temporal fields from FOM, classical ROM, symmetric ROM, and pointwise errors (training $\nu$).

Figure 9: Spatio-temporal fields and errors for FOM, classical ROM, and symmetric ROM (test $\nu$).

Mean $E \circ D = id$0 errors demonstrate that symmetric ROMs provide monotonic error reduction as viscosity increases, a property lacking in classical architectures.

Figure 10: Time-averaged reconstruction error $E \circ D = id$1 for classical and symmetric CAE-ROMS across parameter space.

Kuramoto–Sivashinsky (KS) Equation

For spatiotemporal chaos, standard CAE-ROMS are numerically fragile and frequently exhibit divergent latent trajectories for random initial conditions. Symmetric CAEs, in contrast, maintain stability for all trials.

Field predictions show symmetric CAEs yield plausible, physically consistent solutions beyond the short Lyapunov time horizon ($E \circ D = id$2 vs. $E \circ D = id$3 for classical).

Figure 11: KS field—FOM, classical ROM prediction, and pointwise error.

Figure 12: KS field—FOM, symmetric ROM prediction, and pointwise error.

Crucially, Lyapunov spectra computed from symmetric ROMs match the nonzero part of the FOM Lyapunov spectrum, confirming correct reproduction of chaotic attractor dimension and entropy, with caveats on zero-exponent multiplicity due to symmetry non-preservation.

Figure 13: Lyapunov spectra for FOM, classic, and symmetric ROMs, matching nonzero exponents.

Long-term ergodic statistics show that classical CAEs occasionally generate unphysical high-energy states, while symmetric CAEs closely reproduce the invariant measure of FOM’s kinetic energy.

Figure 14: PDFs of kinetic energy for FOM, classic, and symmetric ROMs.

The energy spectra confirm that symmetric CAEs preserve the correct distribution of energy across all spatial scales, especially in the low-frequency regime.

Figure 15: Kinetic energy spectrum for FOM, classic, and symmetric ROMs—superior agreement for symmetric CAE.

Implications and Future Directions

This architecture overcomes a fundamental theoretical limitation of standard data-driven ROM pipelines by restoring the essential diffeomorphic character between latent and physical coordinate systems, a prerequisite for accurate and interpretable latent dynamics learned by NNs. Strong empirical evidence demonstrates that enforcing architectural symmetry acts as a powerful regularizer, yielding smoother latent manifolds, improved temporal integrator stability, and latent spaces amenable to accurate invariant and statistical property extraction.

The explicit connection to inertial manifold and attractor theory provides a rigorous foundation for the geometrically grounded design of neural surrogates in scientific computing. One practical implication is that symmetric CAEs substantially reduce the risk of latent-space blow-up and unphysical forecasting, a critical concern for surrogate deployment in real-world PDE-constrained optimization, UQ, and control.

Extensions to higher spatial dimensions and more complex field topologies will require further architectural generalizations, e.g., gauge-equivariant CAEs, but the theoretical scaffold established here appears robust. The symmetric framework is likewise expected to benefit hybrid architectures (e.g., GNNs, graph CAEs, and equivariant CNNs) where latent invertibility and physically consistent parametrization are essential.

Conclusion

Symmetric convolutional autoencoders, formulated via differential geometric constraints and bi-Lipschitz invertible activations, provide a manifestly stable and physically meaningful reduced-order representation for nonlinear, parametric, and chaotic PDEs. The resulting ROMs outperform standard CAE-based surrogates in latent stability, reconstruction accuracy, and invariant-preservation in both regular and chaotic dynamics. These results support a transition toward representation-consistent, geometry-aware neural ROM architectures as a foundation for reliable scientific machine learning models for PDEs.