First-Order State Space Model

• First-Order State Space Model is a mathematical framework that describes system evolution via first-order Markovian recurrence incorporating both continuous and discrete formulations.
• It employs advanced discretization strategies like first-order holds and Taylor approximations and integrates parameterizations including basis expansions and neural networks.
• The model finds practical applications in areas such as image super-resolution, nonlinear system identification, and symbolic planning, balancing expressive accuracy and computational efficiency.

A First-Order State Space Model (FSSM) is a mathematical framework for describing the evolution of systems whose outputs and hidden states evolve according to first-order Markovian recurrence relations. FSSMs are foundational in systems theory, signal processing, control, nonlinear dynamical modeling, and, more recently, deep learning for vision and sequential prediction. The FSSM integrates a latent state, whose dynamics and emissions depend on current (and, under advanced discretizations, neighboring) inputs and may involve both parametric and data-driven function classes. Modern applications leverage both continuous- and discrete-time formulations, augmentations for symbolic reasoning, nonlinearity through basis expansions or neural networks, and advanced discretization schemes for improved accuracy and representation capacity.

1. Mathematical Formulation and Classical Structure

The archetypal FSSM is defined by the discrete-time equations

$$x_{t+1} = f(x_t, u_t) + w_t,\quad w_t \sim \mathcal{N}(0, Q),$$

$$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$

where $x_t \in \mathbb{R}^{n_x}$ (hidden state), $u_t \in \mathbb{R}^{n_u}$ (input), and $y_t \in \mathbb{R}^{n_y}$ (output), with noise terms $w_t, v_t$ and parameterized or learned functions $f,g$ (Svensson et al., 2016). The first-order Markov assumption applies: state transitions depend only on the preceding state and current input.

In continuous time, FSSMs are modeled by ordinary differential equations: $\dot{h}(t) = A h(t) + B x(t),\quad y(t) = C h(t),$ where the solution on an interval $[t_n, t_{n+1}]$ involves integrating the ODE with a specified input hold (Zhu et al., 10 Sep 2025).

2. Discretization Strategies: Zero-Order vs. First-Order Hold

Transitioning from continuous to discrete time requires specifying how inputs are interpolated. Standard approaches employ a zero-order hold, treating the input as piecewise constant: $x(t) = x_n$ for $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$0. The FSSM refines this with a first-order (linear) hold: $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$1 enabling closed-form discretization that leverages both $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$2 and $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$3, which enhances modeling of local correlations—essential in tasks such as image super-resolution (Zhu et al., 10 Sep 2025).

The resulting discrete FSSM recurrence is

$$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$4

with matrices $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$5 derived from the ODE solution. Taylor approximations (first-order and second-order) are used for computational efficiency.

3. Model Parameterization: Basis Expansions, Priors, and Deep FSSMs

FSSMs can be parameterized with linear maps, basis expansions, or neural networks:

• Basis function expansions: Functions $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$6 are expanded in fixed bases, with coefficients learned from data. Gaussian process-inspired priors on coefficients provide regularization and nonparametric flexibility (Svensson et al., 2016).
• Probabilistic neural parameterizations: Deep FSSMs, such as ProDSSM, replace $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$7 with neural networks whose weights may themselves be random variables (capturing epistemic uncertainty). Transition and emission models then become, e.g.,

$$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$8

with deterministic, sampling-free inference enabled by closed-form approximations (Look et al., 2023).

This broad parameterization spectrum allows FSSMs to capture nonlinearities, encode domain knowledge, and integrate uncertainty at several modeling levels.

4. Symbolic and Structural Learning of FSSMs

FSSMs also appear in symbolic AI, where the aim is to induce first-order symbolic representations directly from state-transition graphs. The method detailed in (Bonet et al., 2019) seeks a domain $$y_t = g(x_t, u_t) + v_t,\quad v_t \sim \mathcal{N}(0, R)$$9 (predicates, action schemas) and instances $x_t \in \mathbb{R}^{n_x}$0 such that their grounded state-space graphs are isomorphic to observed labeled graphs $x_t \in \mathbb{R}^{n_x}$1.

This inference is formulated as a two-level search:

• Outer loop: Search over hyperparameters that bound model complexity (number and arity of predicates/actions).
• Inner loop: For each candidate, encode the possible symbolic mappings as a SAT problem, ensuring constraints such as one-to-one action-state-label correspondence and STRIPS semantics.

The process outputs minimal, general symbolic FSSMs that account for the observed transitions and can generalize to larger or structurally different instances.

5. Error Analysis and Approximations

First-order hold discretization yields improved error bounds compared to zero-order schemes. Specifically, under standard assumptions (time-independent coefficients, Lipschitz continuous inputs), the cumulative error bound for FSSM is

$x_t \in \mathbb{R}^{n_x}$2

halving the worst-case cumulative error constant relative to zero-order hold, while both remain $x_t \in \mathbb{R}^{n_x}$3 (Zhu et al., 10 Sep 2025).

Practical implementations employ Taylor approximations for efficiency:

• First-order: $x_t \in \mathbb{R}^{n_x}$4, $x_t \in \mathbb{R}^{n_x}$5
• Second-order: additional correction terms involving $x_t \in \mathbb{R}^{n_x}$6

Ablation studies confirm that higher-order approximations further reduce discretization errors and yield measurable performance gains in downstream tasks.

6. Applications and Empirical Performance

FSSMs are deployed in diverse domains:

• Image super-resolution: Integration of FSSM into lightweight architectures (FMambaIR) for 2D image SR, applying FSSMs along four scanning directions. FMambaIR achieves state-of-the-art PSNR/SSIM on Set5, Set14, B100, Urban100, and Manga109 without increasing parameter count or runtime compared to baseline MambaIR. For instance, on Urban100 (×2 upscaling, DIV2K-trained), PSNR/SSIM improves from 32.92/0.9356 to 33.16/0.9372 (Zhu et al., 10 Sep 2025).
• Nonlinear dynamical system identification: Application of FSSMs with GP-inspired priors and Bayesian or EM-based learning produces low RMSE on standard benchmarks (e.g., Narendra–Li, water tanks), outperforming several prior methods (Svensson et al., 2016).
• Probabilistic deep sequential modeling: ProDSSM achieves efficient, deterministic, closed-form filtering and prediction under high-dimensional latent-variable sequence models, outperforming MC-based and other GP-SSM alternatives in accuracy and speed (Look et al., 2023).
• Symbolic planning: The FSSM-induction approach reconstructs lifted STRIPS schemas that generalize from small instances to arbitrarily large or different problem sizes (Blocksworld, Gripper, Hanoi, Grid-world), all learned ab initio from state-transition data (Bonet et al., 2019).

7. Computational Complexity and Resource Considerations

FSSM implementations maintain efficiency through careful parameterization and approximation:

• Parameter count: In visual models such as FMambaIR, FSSM is introduced without increasing parameter count—e.g., 859K for ×2, 867K for ×3, 879K for ×4 upscaling (Zhu et al., 10 Sep 2025).
• FLOPs and runtime: Both classical and deep FSSMs sustain linear computational complexity in sequence length. In deep models, deterministic inference avoids costly Monte Carlo, with costs scaling sub-cubically with network width (Look et al., 2023).
• Symbolic induction: The SAT-based methodology balances combinatorial complexity by operating in the space of bounded-size domains and leveraging efficient constraint solvers (Bonet et al., 2019).
• Learning algorithms: SMC-based learning for nonlinear FSSMs yields fast convergence and tractable inference even with high-dimensional coefficient vectors (Svensson et al., 2016).

FSSMs thus offer a spectrum of trade-offs between expressive power, computational tractability, and statistical robustness, with active research extending their reach to new domains and modeling paradigms.