---
title: Recurrent State-Space Models (RSSMs)
url: https://www.emergentmind.com/topics/recurrent-state-space-models-rssms
type: topic
---

# Recurrent State-Space Models (RSSMs)

Recurrent State-Space Models (RSSMs) are a principled and highly expressive class of sequence models that combine the temporal structure of state-space models (SSMs) with the flexibility and scalability of modern recurrent neural network parameterizations. RSSMs underpin state-of-the-art approaches in model-based reinforcement learning, time series forecasting, system identification, and control, especially under high-dimensional, partially observed, or dynamically shifting observational regimes. The core paradigm is to encode system trajectories as a sequence of latent states governed by a recurrent transition function, with emissions mapping latent states to high-dimensional observations, all within a probabilistic or hybrid deterministic-probabilistic framework.

## 1. Mathematical Foundations and Model Structure

The basic form of an RSSM in discrete time is the following latent-variable model for time-indexed observations $o_{1:T}$ and (optionally) controls or actions $a_{1:T}$:

\[
\begin{align*}
s_0 &\sim p(s_0) \\
s_t &\sim p(s_t \mid s_{t-1}, a_{t-1}) \\
o_t &\sim p(o_t \mid s_t)
\end{align*}
\]

Here, $s_t \in \mathbb{R}^d$ is the latent state at time $t$, governed by a transition function (often parameterized by a neural network such as a GRU or LSTM), and $o_t \in \mathbb{R}^D$ is the observation, with a conditional emission model mapping latent states to the observation space. The structure naturally models partially observable, non-Markovian, or nonlinear environments and can be extended to include exogenous covariates, goals, or hidden parameters [2112.01163], [2206.14697], [2508.17070], [2407.15236].

The transition and emission models can be either deterministic or probabilistic, with typical choices including:

- **Gaussian transitions:** $p(s_t | s_{t-1}, a_{t-1}) = \mathcal{N}(f_\theta(s_{t-1}, a_{t-1}), Q)$.
- **Deterministic transitions:** $s_t = f_\theta(s_{t-1}, a_{t-1})$, typically for deep forecasting or hybrid RSSMs [2407.15236].
- **Hybrid deterministic-stochastic:** augmenting a deterministic state with a stochastic component for improved expressivity and uncertainty modeling [2112.01163], [2508.17070].

## 2. Inference and Learning Methodologies

Exact inference in nonlinear RSSMs is tractable only in special cases. Most deep RSSMs employ amortized variational inference to approximate the posterior $q_\phi(s_t \mid s_{t-1}, a_{t-1}, o_t)$, learning a recognition/encoder network to align the posterior and prior distributions using the evidence lower bound (ELBO):

\[
\mathrm{ELBO} = \sum_{t=1}^T \mathbb{E}_{q_\phi(s_t)}[\log p_\theta(o_t|s_t)] - \mathrm{KL}[q_\phi(s_t|s_{t-1}, a_{t-1}, o_t) \| p_\theta(s_t|s_{t-1}, a_{t-1})]
\]

Regularization is provided by this KL term and by auxiliary heads (e.g., reward, inverse dynamics, regime classification) [2112.01163], [2206.14697]. Training employs backpropagation through time (BPTT), with optimization via Adam and often gradient clipping for stability [2112.01163], [2206.14697]. Extensions to closed-form Gaussian Kalman-style inference arise in linear or locally linear RSSMs [2206.14697], [2210.09256], and hybrid recognition models can be incorporated to accelerate initial state inference [1801.10395].

Some research demonstrates that the default “filter-only” variational family in RSSMs can lead to overestimated aleatoric uncertainty due to its inability to revise past beliefs in light of future observations. This effect can act as implicit regularization, aiding robustness but potentially impairing uncertainty calibration under partial observability or sensor fusion [2210.09256].

## 3. Architectural Variants and Expressivity

RSSMs admit a diversity of architectural instantiations:

- **GRU/LSTM RSSMs:** Structure the latent transition with gated recurrent units, mapping observations to latent states and supporting flexible emission models [2112.01163], [2407.15236].
- **Gaussian-process RSSMs:** Employ a nonparametric transition function via sparse GP with exact temporal correlations, yielding the Probabilistic Recurrent State-Space Model (PR-SSM) [1801.10395].
- **Hidden-Parameter RSSMs (HiP-RSSMs):** Introduce low-dimensional latent variables $c$ that parameterize temporal dynamics across a family of related tasks, learned by amortized context encoders and Kalman-style inference [2206.14697].
- **Contrastive RSSMs:** Replace pixel-wise reconstruction with mutual information maximization in a learned feature space, enforced via symmetric contrastive losses to encourage task-relevant, distraction-robust embeddings [2112.01163].
- **Goal-conditioned RSSMs (GC-RSSMs):** Incorporate explicit goal-conditioning via concatenated or embedded goal observations, supporting planning via actor-critic or CEM in a latent imagination loop [2508.17070].
- **Switching regime RSSMs:** Multiplex several RSSM instances, mixing outputs with learned regime probabilities and Hamilton-style filtering [2407.15236].
- **Structured SSM kernels:** Use diagonal, low-rank, or HiPPO-based recurrence, optimizing scan- or convolutional-style representations for long-sequence modeling [2406.09062], [2509.04422].

Latent state, emission, and context dimensions, as well as basis combinatorics (e.g., number of locally linear bases, context size), are crucial hyperparameters influencing both expressivity and computational tractability [2206.14697].

## 4. Applications and Empirical Outcomes

RSSMs deliver state-of-the-art results across several domains, particularly:

- **Pixel-based and robotic RL:** Contrastive RSSMs (CoRe) achieve superior final validation reward compared to alternatives such as bisimulation, CURL, and SAC+RAD on the Distracting Control Suite, maintaining robustness under severe background, color, and camera perturbations (mean reward 480 vs PSE 386 and CURL 236) [2112.01163].
- **System identification and changing dynamics:** HiP-RSSMs improve over classical RSSMs and recurrent neural nets in low-data adaptation, system identification, and transfer across regimes, empirically validated in robotic arm and mobile robot settings [2206.14697].
- **Robust control and task generalization:** Goal-conditioned RSSMs enable sample-efficient visual model predictive control for garment handling, achieving >94% normalized coverage in real and simulated manipulation with a single policy [2508.17070].
- **Financial time series forecasting:** Deep RSSMs leveraging GRU, LSTM, or TKAN transition modules outperform traditional models in classification, Sharpe ratios, and drawdown metrics on highly nonstationary market data [2407.15236].
- **Sequence modeling and long-range dependency:** Structured and diagonal RSSMs match or exceed transformers in efficiency, scaling linearly in sequence length while maintaining expressivity via controlled recurrence [2406.09062], [2509.04422].
- **Uncertainty quantification and fusion:** RSSMs supporting Kalman-style or fully Bayesian updates accurately propagate uncertainty in partially observed, missing-data, or multimodal fusion tasks, outperforming standard RSSMs under high aleatoric noise [2210.09256].

## 5. Comparative Analysis within Sequential Modeling

RSSMs sit at the intersection of classical linear state-space modeling, nonlinear stochastic dynamical systems, general RNNs, and modern attention or structured-kernel models. Compared to LSTM/GRU RNNs, RSSMs impose explicit state transitions with learned, often interpretable, structure that alleviates vanishing/exploding gradients and permits efficient parallelization via techniques such as diagonalization, low-rank correction, FFT, or associative scan [2406.09062], [2509.04422]. While transformers provide unrestricted attention and softmax-mediated memory, RSSMs encode the full sequence into a fixed-dimensional latent, trading maximal expressivity for extreme computational and memory efficiency.

Architectural and algorithmic variants yield a spectrum from fully deterministic to fully probabilistic, from plain minimum risk to amortized variational and hybrid EM/Kalman optimization, with tradeoffs in trainability, interpretability, and robustness across tasks and noise conditions [2406.09062], [2206.14697], [2210.09256].

## 6. Open Challenges, Limitations, and Research Directions

Several challenges remain in the theory and practice of RSSMs:

- **Expressivity versus efficiency:** Purely linear recurrences struggle with tasks requiring arbitrary span association or copy; input gating, time-varying dynamics, and hybrid local attention-SSM models are active areas of research [2406.09062].
- **Uncertainty decomposition:** Standard RSSMs overestimate aleatoric uncertainty under their default variational filter, compromising performance in true sensor fusion or partial observation. Principled smoothing approaches, as in VRKN, restore correct uncertainty decomposition and enable efficient sensor fusion [2210.09256].
- **Online and infinite-stream learning:** Existing algorithms rely on BPTT or truncated scan; robust online methods (RTRL, UORO, local/forward, or recurrent backprop fixes) remain relatively unexplored in deep RSSMs [2406.09062].
- **Scalability and parallelization:** While scaling can be linear with careful structure (diagonal, low-rank, FFT), mapping RSSMs to modern hardware architectures at the bandwidth/FLOP tradeoff frontier is a continuing challenge [2406.09062], [2509.04422].
- **Hybrid architectures and lifelong adaptation:** Combining RSSMs with transformer-style local attention, gating, and latent adaptation yields promising results in simulation, but large-scale, real-world deployments and benchmarks are still lacking [2406.09062], [2508.17070].

A central future direction is the deep integration of structured state-space recurrence with global attention and flexible uncertainty-aware inference, along with new methodologies enabling online adaptation and scalable lifelong learning.

## 7. Summary Table: Key RSSM Variants and Properties

| Variant              | Transition/Inference           | Key Application                |
|----------------------|-------------------------------|-------------------------------|
| Standard RSSM ([2112.01163])    | GRU/LSTM + amortized variational | RL, pixel-based world modeling |
| Contrastive RSSM ([2112.01163]) | GRU + symmetric contrastive loss | Robust RL under distractions   |
| HiP-RSSM ([2206.14697])         | Locally linear + task encoder    | Identifying changing dynamics  |
| Probabilistic RSSM ([1801.10395])| GP transitions + variational     | Nonlinear system identification|
| Goal-conditioned RSSM ([2508.17070])| GRU + goal embedding           | Visual MBRL, manipulation     |
| VRKN ([2210.09256])             | Kalman + MC Dropout              | Uncertainty, sensor fusion     |
| Deep RSSM ([2407.15236])        | LSTM/GRU/TKAN, deterministic     | Financial time series          |
| Structured SSM ([2406.09062],[2509.04422]) | Diagonal, S4, Koopman, subspace | Long-sequence modeling         |

Each variant is characterized by its approach to latent transition modeling, uncertainty representation, and targeted domain/applications.

---

RSSMs unify probabilistic latent-state modeling, deep learning-based transition/emission parameterizations, and scalable inference to provide a versatile foundation for time-series, control, and classification tasks, excelling especially where long horizons, uncertainty, and partial observability are dominant constraints. Advances in structure, uncertainty handling, and efficient optimization continue to enhance their theoretical rigor and practical impact.

Source: https://www.emergentmind.com/topics/recurrent-state-space-models-rssms