Oscillatory State-Space Models (LinOSS)

• Oscillatory State-Space Models are a class of state-space models that use forced harmonic oscillator dynamics to capture long-range temporal interactions.
• LinOSS employs fully implicit and implicit-explicit symplectic discretization schemes to ensure numerical stability and efficient inference over long sequences.
• Extensions like D-LinOSS decouple damping from frequency, enhancing model expressivity and achieving state-of-the-art performance in time series classification, forecasting, and regression.

Oscillatory State-Space Models (LinOSS) constitute a class of state-space models for sequence learning, parameterized to capture long-range temporal interactions using principles drawn from the dynamics of forced harmonic oscillators. The LinOSS framework formalizes the transition from classical oscillation-based statistical modeling to computationally efficient, stable, and universal models suitable for large-scale time series classification, regression, and forecasting. Extensions, such as Damped LinOSS (D-LinOSS), address limitations of frequency–damping coupling and further broaden the model expressivity while preserving stability and efficient inference.

1. Mathematical Foundations and Model Structure

Oscillatory State-Space Models are based on the state-space formulation of forced (and optionally damped) harmonic oscillators. The canonical continuous-time model for $m$ uncoupled oscillatory channels is defined by the second-order ordinary differential equation:

$$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$

where $x(t)\in\mathbb{R}^m$ is the latent position, $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$ is the (optional) diagonal damping, $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$ with $\omega_k>0$ are oscillator frequencies, $B\in\mathbb{R}^{m\times p}$ is the external input gain, and $u(t)\in\mathbb{R}^p$ the input.

By augmenting with the velocity $v(t)=\dot{x}(t)$ and stacking as $z(t) = [x(t); v(t)] \in \mathbb{R}^{2m}$, the model is recast as a linear time-invariant (LTI) first-order system:

$$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$0

with $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$1 and $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$2. The read-out matrix $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$3 selects which state is observed, such as the position or velocity.

This structure encompasses a spectrum from completely oscillatory (no damping, $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$4) to damped, potentially multi-timescale, evolutions. For statistical time series, discrete-time analogs are derived by stable numerical discretizations of the ODE.

2. Discretization Techniques and Stability

LinOSS introduces two primary discretization schemes:

• Fully Implicit (LinOSS-IM): Employing backward-Euler on both $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$5 and $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$6, the dynamics evolve as

$$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$7

leading to the recurrent update $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$8, with $$\ddot{x}(t) + D\,\dot{x}(t) + \Omega^2 x(t) = B\,u(t)$$9.

• Implicit-Explicit Symplectic (LinOSS-IMEX): $x(t)\in\mathbb{R}^m$0 is advanced implicitly, $x(t)\in\mathbb{R}^m$1 explicitly,

$x(t)\in\mathbb{R}^m$2

which yields $x(t)\in\mathbb{R}^m$3.

Parameterization constrains $x(t)\in\mathbb{R}^m$4 to be diagonal and nonnegative. The crucial theoretical guarantee is that these discretizations produce stable dynamics: for each $x(t)\in\mathbb{R}^m$5 block associated with oscillator $x(t)\in\mathbb{R}^m$6, all eigenvalues of $x(t)\in\mathbb{R}^m$7 reside on or within the unit circle for any $x(t)\in\mathbb{R}^m$8 (and $x(t)\in\mathbb{R}^m$9). In the undamped (symplectic) LinOSS-IMEX case, $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$0—the model is exactly energy-preserving and time-reversible (Rusch et al., 2024).

In practical implementations, this structure enables efficient $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$1 per-step computation and lends itself to highly parallel associative scan algorithms for inference (Rusch et al., 2024, Boyer et al., 17 May 2025).

3. Universality, Time-Reversibility, and Expressivity

Oscillatory State-Space Models are rigorously shown to be universal in the sense that combinations of LinOSS blocks (ODE + nonlinearity + output) can uniformly approximate any continuous and causal operator $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$2 on compact sets [(Rusch et al., 2024), Theorem 3.4]. The technical mechanism follows from representing operator kernels as finite-horizon sine-transform kernels and mounting these in static networks.

LinOSS-IMEX discretizations are symplectic integrators for associated Hamiltonian systems, perfectly preserving the symmetry of time-reversibility. Mapping forward $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$3 steps and backward $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$4 returns the initial state up to machine precision, an algebraic property directly shown by the involutive structure $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$5 for the canonical symplectic $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$6 (Rusch et al., 2024).

However, in pure LinOSS models, forgetting and oscillatory frequencies are rigidly coupled: the eigenvalues of the transition matrix sweep a one-dimensional curve of modulus $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$7 in the complex plane (unit circle for IMEX, inside for IM). This restricts the ability to represent multi-timescale decay or pure exponential modes (Boyer et al., 17 May 2025).

4. Inference Algorithms and Computation

Efficient inference in LinOSS-based architectures leverages the associativity of the time-update operators. Each time step's local map is $D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$8, and the overall trajectory can be computed by repeated application using the operation:

$D=\operatorname{diag}(d_1,\dots,d_m)\geq 0$9

A parallel-prefix ("scan") algorithm computes all partial products in $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$0 parallel depth and $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$1 total work. The canonical pseudocode executes a parallel scan and then extracts latent state vectors for each $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$2 (Rusch et al., 2024).

Memory is $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$3 for backpropagation, but IMEX reversibility can enable checkpointing strategies for reduced memory in differentiable programming frameworks. Both IM and IMEX models leverage this to achieve scalable sequence learning.

5. Extensions: Damped LinOSS and Expressivity Improvements

Damped LinOSS (D-LinOSS) generalizes the LinOSS framework by introducing learnable damping $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$4, decoupling energy dissipation ("forgetting" rate) from oscillation frequency:

$$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$5

Each mode now has its own adjustable damping ratio $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$6, with discrete updates via an IMEX scheme. The spectral distribution of the transition matrix covers the entire closed unit disk: for any $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$7, there exist $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$8 mapping to $$\Omega=\operatorname{diag}(\omega_1,\dots,\omega_m)$$9 (proved via a closed-form mapping) (Boyer et al., 17 May 2025). This remedy allows D-LinOSS layers to match arbitrary stability and frequency requirements, overcoming LinOSS's single-timescale limitation.

Furthermore, D-LinOSS eliminates the choice of discretization scheme as a hyperparameter—fixing to IMEX—and reduces the hyperparameter space by 50%, achieving improved empirical results across synthetic and real-world long-sequence tasks with the same $\omega_k>0$0 complexity (Boyer et al., 17 May 2025).

6. Statistical and System Identification Perspectives

Oscillatory state-space models have foundations in both statistical modeling of oscillatory time series and nonlinear system identification. A general statistical oscillatory SSM represents observations as $\omega_k>0$1, driven by an unobserved stochastic “phase” $\omega_k>0$2 characterized by a random walk or autoregressive dynamics. Inference in nonparametric oscillatory SSMs requires particle-based EM algorithms, with identifiability guaranteed up to phase shift and rescaling if the base frequency is irrational (Dahlhaus et al., 2014).

The linearized (LinOSS) case arises by fixing the pattern $\omega_k>0$3 to be a parametric function of known form (e.g., a sinusoid or low-order Fourier expansion), at which point the system reduces to a linear Gaussian state-space model. Standard Kalman-based filtering, smoothing, and EM algorithms are optimal for this subcase (Dahlhaus et al., 2014). Nonparametric identification yields, via local linearization, LinOSS surrogates tailored to specific dynamical regimes (Protas et al., 2012).

7. Empirical Performance, Applications, and Implementation

LinOSS models, and D-LinOSS in particular, achieve state-of-the-art results on challenging long-range sequence tasks:

Task & Metric	LinOSS-IM	LinOSS-IMEX	D-LinOSS	Best Baselines
UEA-MTSCA (accuracy, avg 6 datasets)	67.8%	65.0%	68.0%	LRU/S6: 64.4%
PPG-DaLiA (heart-rate MSE $\omega_k>0$4)	6.4	7.5	6.16	Mamba: 10.65, LRU: 15.64
Weather forecasting MAE	0.528	0.508	0.486	S4: 0.578, Informer: 0.731

On extreme-length sequences (up to 50k timesteps), LinOSS-IM attains nearly $\omega_k>0$5 the accuracy of Mamba; D-LinOSS further narrows the gap to optimality. In all empirical settings, these models demonstrate high stability, computational scalability, and competitive or superior performance compared to SSM, RNN, and Transformer baselines without requiring complex architectural choices (Rusch et al., 2024, Boyer et al., 17 May 2025).

Practical applications span time-series classification and regression, long-horizon forecasting (weather, biomedical signals), and as stable latent-dynamics modules in generative models or sequence backbones in language, genomics, and audio.

---

These results collectively position LinOSS and D-LinOSS as foundational architectures for oscillatory sequence modeling, with rigorous guarantees of stability and expressivity, efficient scalable inference, and strong empirical performance. Theoretical connections to both classical oscillatory statistical models and nonlinear dynamical system identification underline their versatility and extensibility (Rusch et al., 2024, Boyer et al., 17 May 2025, Dahlhaus et al., 2014, Protas et al., 2012).