Simple Recurrent FloWM (2D) Models

• The framework integrates recurrent networks, normalizing flows, and mixture density components to capture structured and multimodal 2D sequential data.
• It demonstrates enhanced likelihood estimation and stable long-term predictions in machine learning tasks through flow-equivariant world models.
• In turbulence analysis, the method extracts invariant recurrent flows to accurately reconstruct the statistical properties of 2D turbulent systems.

Simple Recurrent FloWM (2D) denotes a class of models and analytic tools developed for two fundamentally distinct—but conceptually linked—settings: (1) machine learning models for 2D sequence data employing recurrent, flow-based, and mixture-density architectures (Razavi et al., 2020, Lillemark et al., 3 Jan 2026), and (2) the analysis and extraction of exact recurrent flows in 2D turbulent fluid dynamics, notably in the Kolmogorov flow regime (Chandler et al., 2012). In both domains, the “recurrent flow” nomenclature refers to modeling or extracting structured sequential dependencies—via dynamics or learned recurrence—supplemented by “flow” architectures or invariant solutions. In the deep learning context, FloWM integrates normalizing flows with recurrent mixture density networks for probabilistic sequence modeling. In dynamical systems, it refers to the identification of invariant, recurrent solutions embedded within turbulent dynamics, enabling reconstruction of statistical properties. This entry surveys the mathematical principles, model architectures, computational techniques, and empirical findings that characterize Simple Recurrent FloWM (2D) frameworks.

1. Foundational Principles and Mathematical Structures

The core principle underpinning Simple Recurrent FloWM (2D) is the explicit modeling or extraction of recurrence and flow structure in 2D sequential or spatiotemporal data. Across both machine learning and dynamical systems, two components are central:

• Recurrence: Capturing time-dependent evolution, either through RNN architectures (LSTM, GRU) (Razavi et al., 2020, Lillemark et al., 3 Jan 2026) or by discovering closed dynamical trajectories (periodic orbits, travelling waves, equilibria) in the governing equations (Chandler et al., 2012).
• Flow: Denoting either a learnable, invertible transformation (as in normalizing flows for probabilistic modeling), a symmetry (Lie-group flow equivariance), or a dynamical solution of the Navier–Stokes equations.

In the deep learning case, the architecture is built from:

• A recurrent network mapping input histories $x_{1:T}$ to hidden states $h_t$.
• A mixture density network (MDN) head parameterizing a $K$-component GMM in a transformed latent space.
• An invertible normalizing flow $f$ mapping the original 2D data $y_t$ into a latent space $z_t = f(y_t)$, facilitating more expressive conditional densities.

For 2D turbulence, recurrence is analyzed in terms of invariant solutions of the 2D incompressible, forced Navier–Stokes equations on a periodic square, with the governing PDEs: $$\partial_t\mathbf u + (\mathbf u\cdot\nabla)\mathbf u = -\nabla p + \nu\,\Delta\mathbf u + F\,\sin(4y)\,\hat e_x,\quad \nabla\cdot\mathbf u=0.$$ with body-forcing wavenumber $n=4$ and Reynolds number $Re=\sqrt{F}\,L^{3/2}/\nu$ (Chandler et al., 2012).

2. Model Architectures and Computational Algorithms

Machine Learning (Recurrent FloWM for 2D Sequences)

The model consists of the following modules (Razavi et al., 2020, Lillemark et al., 3 Jan 2026):

• Recurrent Module: $h_0=0$, $h_t=g(h_{t-1},x_t;\theta_{\text{RNN}})$; $g$ can be any RNN cell.
• MDN Head: $o_t=W_oh_t + b_o$; the MDN maps to mixture weights $\alpha_{t,1\ldots K}$, means $\mu_{t,1\ldots K}$, and covariances $\Sigma_{t,1\ldots K}$ in a latent space. Diagonal or low-rank+diagonal precision matrices are enforced for numerical stability.
• Normalizing Flow Layer: A chain of $M$ invertible affine-coupling bijections $f_1,\ldots,f_M$, typically RealNVP-style (Razavi et al., 2020). For each $i$, the log-determinant of the Jacobian is tractable (triangular).
• Composition: The conditional data density is

$$p(y_t|h_t) = \left[ \sum_{k=1}^K \alpha_{t,k} \mathcal{N}(z_t; \mu_{t,k}, \Sigma_{t,k}) \right] \prod_{i=1}^M \left| \det \frac{\partial f_i}{\partial z^{(i-1)}} \right| .$$

This composition allows the model to map highly structured or multimodal sequence data into a latent domain where densities are easier to model as mixtures.

Flow Equivariant World Models

Recent advances extend these architectures to maintain precise group-theoretic structure:

• Encoder $E_e$: Convolutional, mapping observation $x_t$ into a world-sized latent “memory” $M_t$ spanning spatial and velocity channels, $M_t\in\mathbb{R}^{|V|\times C\times H_w\times W_w}$.
• Flow-Equivariant Recurrence $\Phi$: Updates memory by writing-in new features, performing convolutions, and shifting via known self-motion and object-flow group actions. The Abelian Lie group of 2D translations $G = \mathbb{R}^2$ underpins these transformations.
• Decoder: Crops out the field-of-view, pools over velocity channels, and decodes via additional convolutions (Lillemark et al., 3 Jan 2026).

Dynamical Systems (Exact Recurrent Flows)

• Numerics: Fully-dealiased, pseudo-spectral simulation at $256\times256$ grid, Crank–Nicolson for diffusion, Heun predictor–corrector for advection/forcing. Large-eddy simulation durations $O(10^5)$ time units facilitate the discovery of invariants.
• Recurring Solution Extraction: Identification relies on detecting near-recurrences via phase-shifted residuals in Fourier space, then converging to invariant solutions via Newton–GMRES–Hookstep methods.
• Types of Solutions: Periodic orbits (including relative/shifted), equilibria, and travelling waves.

3. Equivariance, Symmetry, and Invariant Properties

Equivariance ensures that model representations and outputs transform consistently under known symmetries:

• In Flow-Equivariant World Models, both self-motion and object motion act as translation flows. The recurrence is formulated so that latent memory exactly tracks these group actions, yielding robust feature tracking over hundreds of timesteps (Lillemark et al., 3 Jan 2026).
• In turbulent flows, the extracted recurrent solutions—equilibria, periodic and relative periodic orbits—embody the phase space symmetries inherent in periodic boundary conditions and body-forced driving (Chandler et al., 2012).

Equivariance constraints in deep models enforce that encoders, update functions, and convolution operations commute with translation actions. This structure is crucial for long-horizon prediction stability and for modeling the true statistics of systems with spatial or dynamical symmetries.

4. Empirical Performance and Significance

Empirical results demonstrate the efficacy of Simple Recurrent FloWM (2D) frameworks in both domains.

In probabilistic sequence modeling for 2D data (Razavi et al., 2020):

• On image sequence “world-model” tasks, flow-augmented RMDN (FloWM/FRMDN) achieves improved negative log-likelihood (NLL) over standard RMDN baselines, e.g., in Car-Racing NLL (nats): vanilla RMDN 2.37, FloWM 2.35; Super-Mario: vanilla RMDN 1.36, FloWM 1.28, indicating tighter fit via flow-based latent transformations.

In flow-equivariant 2D world modeling (Lillemark et al., 3 Jan 2026):

• On the “MNIST World” benchmark, the Simple Recurrent FloWM architecture achieves 20-step MSE ≈ 0.0005, PSNR ≈ 33 dB, and SSIM ≈ 0.99. Predictions are stable even for 150-step extrapolations.
• Ablating velocity channels or self-motion equivariance rapidly degrades performance (MSEs increase to 0.0041 and 0.12, respectively). Autoregressive video diffusion baselines (DFoT) display immediate rollout drift (20-step MSE ≈ 0.145).

In 2D turbulence (Chandler et al., 2012):

• At $Re=40$, approximately 50 distinct recurrent flows could be extracted. Ensemble-weighted reconstructions of energy and dissipation PDFs accurately reproduce the central statistics of direct numerical simulation, with errors only in the far tails.
• At higher $Re$ ($60$–$100$), only $\sim 10$ orbits are found and statistical coverage becomes poor, with turbulent fluctuations exceeding the maxima of sampled invariants.

5. Algorithmic and Computational Considerations

Training Deep FloWM Models:

• Objective: maximize total log likelihood across sequence data, with exact gradients computed through RNN/MDN/head and invertible flow layers (Razavi et al., 2020). Loss minimization typically employs Adam or RMSProp.
• Regularization: softplus activations to maintain positive-definite covariances, variance clipping for stability, weight decay optionally on RNN weights.

Memory Model Parameterization in World Models (Lillemark et al., 3 Jan 2026):

• Encoder: single 3×3 convolution, circular padding, followed by zero-padding into a global canvas.
• Recurrent step: single 3×3 convolution, ReLU activation, velocity-channel structure.
• Decoder: stacked convolutions without batch/layer normalization, as circular convolutions suffice for equivariance.

Turbulence Analysis:

• Near-recurrence detection leverages $R(t,T)$, a shifted Fourier residual norm, thresholded ($R_{\text{thres}}\approx 0.3$) during long DNS.
• Orbit convergence uses Newton–GMRES–Hookstep, with orthogonality conditions to break translational degeneracy.
• Orbit-weighting for statistical reconstruction employs escape-time, period-scaled escape-time, or uniform protocols, exploiting Floquet theory for weight computation.

6. Limitations and Scope

• Probabilistic FloWM architectures require significant computational cost for flow modeling and are subject to numerical instabilities if covariance regularization is insufficient or if flows are poorly parameterized.
• Flow-equivariant memory models assume well-defined group structure; real-world settings with non-Abelian or non-translation symmetries require significant extension.
• Recurrent flow extraction in turbulence scales poorly to higher $Re$ due to exponential growth in phase space volume and orbit multiplicity. The absence of symbolic dynamics or orbit classification schemes further limits coverage: at moderate $Re$ ($\sim 40$), DNS runs of $10^5$ time units suffice, but at $Re \gtrsim 60$ runs would need to be 100–1000 times longer to adequately sample outlier events (Chandler et al., 2012).
• In both modeling and analysis, failure to incorporate the appropriate symmetries or recurrence can result in significant degradations in predictive power, compositional generalization, or ergodic statistical reconstruction.

7. Contexts of Application and Impact

• Machine Learning Sequence Modeling: FloWM architectures are suited to complex, high-dimensional time series where latent structural invariances or symmetries (e.g., in video, speech, or spatial sensor fields) support more faithful sequence modeling and likelihood fitting (Razavi et al., 2020).
• World Model Learning for Embodied Agents: Flow-equivariant models are effective for memory and prediction in partially observed settings with moving viewpoints and independently moving external objects. Group-theoretic modeling produces stability and accuracy over long prediction horizons, surpassing diffusion-based and standard memory-augmented architectures (Lillemark et al., 3 Jan 2026).
• Turbulent Flow Analysis: Extraction of simple recurrent flows in 2D turbulence provides a bridge between low-dimensional chaotic dynamics and high-dimensional statistical turbulence, enabling a Periodic Orbit Theory–style reconstruction of macroscale statistics. This approach is viable at moderate but not high Reynolds number, where the sampling of invariant sets becomes intractable (Chandler et al., 2012).

The Simple Recurrent FloWM (2D) paradigm thus unifies modern normalizing-flow-enhanced sequence models, group-equivariant memory architectures, and the dynamical systems study of invariant solutions. It demonstrates the critical role of symmetry, recurrence, and flow structure in both learning-based and analytic approaches to 2D spatiotemporal phenomena.