---
title: 'SLiCEs: Structured Linear Controlled Differential Equations'
url: https://www.emergentmind.com/topics/structured-linear-controlled-differential-equations-slices
type: topic
---

# SLiCEs: Structured Linear Controlled Differential Equations

Searching arXiv for the specified SLiCE papers and closely related work.
Structured Linear Controlled Differential Equations (SLiCEs) are linear controlled differential equation models whose state-transition matrices are constrained to follow a chosen structure—such as diagonal, block-diagonal, diagonal-plus-low-rank, sparse, or Walsh–Hadamard—while aiming to preserve the expressive capacity associated with dense linear controlled systems [2505.17761]. In the recent literature, SLiCEs occupy a junction between neural controlled differential equations and structured state space models: they provide a continuous-time formulation of input-dependent linear dynamics, support parallel-in-time computation via associative composition of interval transitions, and have been used both as discriminative backbones and as generative time-series models through flow matching on path space [2505.17761, 2605.28507].

## 1. Definition and mathematical setting

The starting point for SLiCEs is the linear controlled differential equation (LNCDE)
\[
\mathrm{d}h_s = \sum_{i=1}^{d_\omega} A^i h_s \,\mathrm{d}\omega^i_s,
\]
where \(h_s \in \mathbb{R}^{d_h}\) is the hidden state, \(\omega:[0,T]\to \mathbb{R}^{d_\omega}\) is the control path, and \(A^i \in \mathbb{R}^{d_h \times d_h}\) is the linear vector field for channel \(i\) [2505.17761]. Given an observed path \(X\), the continuous-time model is written as
\[
h_t = h_{t_0} + \int_{t_0}^t \sum_{i=1}^{d_\omega} A^i_\theta h_s \,\mathrm{d}\omega_s^{X,i}, \qquad z_t = l_\psi(h_t),
\]
or, in the notation used for Neural CDEs,
\[
h_{t_0}=\xi(X_{t_0}),\qquad h_t=h_{t_0}+\int_{t_0}^t \sum_{i=1}^{d_\omega} A_\theta^i h_s\,d\omega_s^{X,i},\qquad z_t=r_\psi(h_t)
\]
[2505.17761, 2605.28507].

A SLiCE is an LNCDE in which the matrices \(A^i\) are not arbitrary dense matrices but instead belong to a structured family [2505.17761]. The central intuition stated in the literature is that dense matrices are expressive but expensive, whereas simple structured matrices are cheap but may be overly restrictive; SLiCEs therefore seek matrix structures that remain “cheap yet still expressive enough to state-track” and approximate arbitrary continuous path functionals [2505.17761].

For piecewise linear controls, the exact interval update is expressed through a matrix exponential:
\[
h_{t_{j+1}}=\Phi_j^\theta(X)\,h_{t_j},\qquad \Phi_j^\theta(X)=\exp\!\left(\sum_{i=1}^{d_\omega}A_\theta^i\big(\omega^{X,i}_{t_{j+1}}-\omega^{X,i}_{t_j}\big)\right),
\]
and the hidden trajectory is obtained by composing such transitions [2605.28507]. The same discrete approximation appears in the 2025 treatment:
\[
\tilde{h}_{t_{j+1}} = \exp\left( \sum_{i=1}^{d_\omega} (\omega^i_{t_{j+1}}-\omega^i_{t_j}) A^i \right)\tilde{h}_{t_j}
\]
[2505.17761]. Because matrix multiplication is associative, these updates admit a parallel scan, a property that is central to the computational framing of SLiCEs in sequence modelling [2505.17761, 2605.28507].

The expressivity analysis is tied to path-space formulations. The relevant spaces are
\[
\mathcal{X}(d_X)=C^{1,0}([t_0,t_n],\mathbb{R}^{d_X}),\qquad \mathcal{Y}(d_y)=C([t_0,t_n],\mathbb{R}^{d_y}),
\]
with metric
\[
\rho_\infty(Y,\widetilde Y)=\sup_{t\in[t_0,t_n]}\|Y_t-\widetilde Y_t\|_2.
\]
A map \(T:\mathcal{X}(d_X)\to\mathcal{Y}(d_y)\) is causal if
\[
X|_{[t_0,t]}=\widetilde X|_{[t_0,t]} \Longrightarrow T(X)_t=T(\widetilde X)_t \qquad\text{for all }t
\]
[2605.28507]. This formulation places SLiCEs within continuous causal modelling rather than solely within fixed-grid recurrence design.

## 2. Structural families and relation to sequence models

SLiCEs are presented as a unifying framework for sequence models with structured, input-dependent state-transition matrices [2505.17761]. The literature explicitly associates several existing architectures with particular matrix families inside the LNCDE view.

| Model family or variant | State-transition structure | Treatment in the literature |
|---|---|---|
| S4 | diagonal / non-selective linear recurrence | restricted NCDE driven essentially by \(\omega_t=t\) [2605.28507] |
| Mamba | diagonal / selective | selective but typically diagonal structured model [2605.28507] |
| DeltaNet-style models | diagonal-plus-low-rank | DPLR form \(A^i_\theta = D^i_\theta + \sum_{j=1}^r u^i_j (v^i_j)^\top\) [2505.17761] |
| Input-dependent block-diagonal LRNNs | block-diagonal | contained in the SLiCE framework [2505.17761] |
| Dense Linear NCDEs | dense | fully expressive reference point [2505.17761] |
| Sparse and Walsh–Hadamard variants | sparse; Walsh–Hadamard | introduced as SLiCE variants [2505.17761] |

The block-diagonal form is
\[
A^i_\theta = \mathrm{BlockDiag}(B^i_{\theta,1},\dots,B^i_{\theta,k}),
\]
with dense blocks \(B^i_{\theta,j}\) [2505.17761]. The Walsh–Hadamard construction uses
\[
A^i_\theta = H D^i_\theta,
\]
where \(H\) is a normalized Hadamard matrix and \(D^i_\theta\) is diagonal [2505.17761]. Sparse SLiCEs use matrices with \(\mathcal{O}(d_h^{1+\epsilon})\) nonzero entries and random Bernoulli support [2505.17761].

This organization lets the literature compare architectures along two axes: computational structure and expressivity [2505.17761]. In that comparison, dense Linear NCDEs define the maximally expressive endpoint, diagonal models the most constrained endpoint, and structured families such as block-diagonal, sparse, Walsh–Hadamard, and suitably growing DPLR are studied as intermediate designs that may retain maximal expressivity at lower cost [2505.17761, 2605.28507].

A recurring theme is that SLiCEs are not merely a taxonomic reformulation. The continuous-time interpretation is used to connect exact-flow discretizations of state-space recurrences to the NCDE form and thereby to reinterpret architectures such as S4 and Mamba within a single dynamical systems language [2605.28507]. This suggests that questions about state-tracking, universality, and generative adequacy can be asked at the level of the transition structure itself rather than at the level of brand-specific architectural details.

## 3. Expressivity and universality

The 2025 work frames SLiCE expressivity in terms of maximal probabilistic expressivity: with high probability over random parameter initialization, as width grows, the model can approximate any continuous target functional on compact path sets arbitrarily well using a linear readout [2505.17761]. Dense Gaussian matrices satisfy this criterion, but the main contribution is the demonstration that several structured matrix families can also satisfy it [2505.17761].

The paper gives the following asymptotic results [2505.17761]:

- Block-diagonal LNCDEs are maximally probabilistically expressive if \(\max_j b_j \to \infty\) as \(d_h \to \infty\).
- Sparse LNCDEs have maximal probabilistic expressivity.
- Walsh–Hadamard LNCDEs have maximal probabilistic expressivity.
- Diagonal matrices, as used in S4 and Mamba, are not maximally expressive.

The contrast with diagonal parameterizations is central. The literature states that diagonal matrices lose the ability to state-track in the relevant sense, whereas block-diagonal, sparse, Walsh–Hadamard, and DPLR structures can retain enough richness to match dense systems asymptotically [2505.17761]. This is the theoretical basis for the claim that “structure need not imply expressive weakness” [2505.17761].

The 2026 generative work extends the expressivity discussion from pathwise approximation to induced path laws [2605.28507]. It defines maximal expressivity on compact sets through the usual compact-uniform criterion: for every compact \(\mathcal K\), every continuous target map \(f\) on \(\mathcal K\), and every \(\varepsilon>0\), there exists parameters \(\theta\) such that
\[
\sup_{x\in\mathcal K}\rho_{\mathcal Y}(f(x),f_\theta(x))\le \varepsilon
\]
[2605.28507]. In the path-space setting, the key theorem states that if \(\mathcal K\subset\mathcal X(d_X)\) is compact, \(T:\mathcal X(d_X)\to\mathcal Y(d_y)\) is continuous and causal, and \(\omega^X_s=X_s\), then any SLiCE class that is maximally expressive in the sense of the prior work, with a linear readout, can approximate \(T\) uniformly:
\[
\sup_{X\in\mathcal K}\rho_\infty\bigl(z^\theta(X),T(X)\bigr)\le \varepsilon
\]
for suitable hidden dimension \(d_h\) and feed-forward readout \(r_\psi\) [2605.28507].

The same work then defines a universal causal time series generator via pushforwards. If \(X\sim\mu\) and \(F_\theta\) is measurable, then \((F_\theta)_\#\mu\) is the law of \(F_\theta(X)\). A model class is a universal causal time series generator if, for every compact \(\mathcal K\subset\mathcal X(d_X)\), every Borel probability measure \(\mu\) supported on \(\mathcal K\), every continuous causal \(T\), and every \(\varepsilon>0\), there exists \(\theta\) with
\[
W_\infty\bigl((F_\theta)_\#\mu,\; T_\#\mu\bigr)\le \varepsilon
\]
[2605.28507].

The bridge theorem is particularly concise: if a class can uniformly approximate every continuous causal map on every compact set, then it is a universal causal time series generator [2605.28507]. The proof uses the coupling
\[
\pi=(F_\theta,T)_\#\mu
\]
and the inequality
\[
W_\infty\bigl(F_\theta{}_\#\mu,\,T_\#\mu\bigr) \le \operatorname*{ess\,sup}_{X\sim\mu}\rho_\infty\bigl(F_\theta(X),T(X)\bigr)
\]
[2605.28507]. Combining this with path-to-path universality yields the corollary that any SLiCE class satisfying path-to-path universality is a universal causal time series generator [2605.28507].

The compact-support assumption is explicit. It is what permits uniform approximation on the support to turn directly into a \(W_\infty\) guarantee [2605.28507]. The same paper notes that Gaussian-process priors used in practice are not compactly supported but are tight; therefore, for every \(\delta>0\), one can restrict to a compact set of mass at least \(1-\delta\) and obtain the high-probability guarantee
\[
\mathbb{P}\Bigl(\rho_\infty(F_\theta(X),T(X))>\varepsilon\Bigr)\le \delta.
\]
This suggests a practical approximation principle rather than a literal global compact-support statement in common stochastic settings.

## 4. Computation, parallel-in-time evaluation, and efficiency tradeoffs

A distinctive feature of SLiCEs is that the exact interval transition under piecewise linear controls is a matrix exponential, so evaluation consists of composing matrix-valued updates [2505.17761, 2605.28507]. Because composition is associative, the literature highlights parallel scan evaluation with parallel depth
\[
\mathcal{O}(\log n)
\]
[2505.17761]. This is one reason SLiCEs are framed as parallel-in-time sequence models rather than only as recurrent models.

The 2025 paper provides recurrent hidden-update complexities for several structures [2505.17761]:

| Structure | Recurrent cost |
|---|---|
| Dense | \(\mathcal{O}(d_\omega d_h^2)\) |
| Diagonal | \(\mathcal{O}(d_\omega d_h)\)-style |
| DPLR | \(\mathcal{O}(d_\omega r d_h)\) |
| Sparse | \(\mathcal{O}(d_\omega d_h^{1+\epsilon})\) |
| Block-diagonal | \(\mathcal{O}(d_\omega \sum_j b_j^2)\) |
| Walsh–Hadamard | \(\mathcal{O}(d_\omega d_h + d_h \log d_h)\), depending on implementation order |

For block-diagonal matrices, the hidden-update complexity
\[
\mathcal{O}\left(d_\omega \sum_j b_j^2\right)
\]
is substantially smaller than \(\mathcal{O}(d_\omega d_h^2)\) when the blocks are small [2505.17761]. The paper also describes a hybrid “diagonal-dense” variant composed of many \(1\times1\) blocks together with one larger dense block [2505.17761].

Parallel composition interacts with structure in a nonuniform way. The summary states that dense, diagonal, and block-diagonal matrix families preserve practical structure under multiplication in ways that make their parallel costs easy to characterize, whereas DPLR, sparse, and Walsh–Hadamard parameterizations may still incur dense-like worst-case composition cost because the structures are not always closed under multiplication [2505.17761]. This marks an important distinction between efficiency of a single transition and efficiency of parallel scan composition.

The same literature records implementation caveats. Sparse SLiCEs are theoretically efficient, but current JAX/PyTorch sparse kernels do not yield practical speedups [2505.17761]. Log-ODE methods can reduce I/O costs by avoiding materializing every transition matrix [2505.17761]. These remarks indicate that the practical efficiency frontier is determined not only by asymptotic structure but also by whether that structure is compatible with existing software and hardware stacks.

In empirical timing on UEA multivariate time-series classification, block-diagonal LNCDEs are singled out as especially favorable. The reported average time per 1000 steps is \(55\) s for BD-LNCDE versus \(1131\) s for Log-NCDE, while average accuracy remains comparable, yielding the headline claim of a factor-of-twenty reduction in average time per training step [2505.17761]. This is one of the clearest demonstrations that the structured continuous-time formulation can produce both theoretical and practical advantages.

## 5. Generative modelling with G-SLiCEs

The 2026 work extends SLiCEs from discriminative modelling to generative time-series modelling by introducing Generative SLiCEs (G-SLiCEs) [2605.28507]. The generator is defined as a flow on path space:
\[
X^{(0)}\sim \mu,\qquad \frac{d}{ds}X^{(s)} = F_\theta\bigl(s,X^{(s)}\bigr),\qquad s\in[0,1],
\]
where \(s\) is the flow-matching time and \(F_\theta\) is a causal SLiCE vector field acting on path space [2605.28507]. The generated sample is the terminal path \(X^{(1)}\), with law
\[
\nu_\theta = (\varphi_{\theta,1})_\#\mu.
\]

The prior \(\mu\) is a Gaussian process on path space. In the unconditional case,
\[
\mu=\mathcal{GP}(m,k),
\]
and in the conditional case the model uses the conditional Gaussian-process posterior
\[
\mu(\cdot\mid C)=\mathcal{GP}\bigl(m_{\mathrm{post}(\cdot\mid C)},\,k_{\mathrm{post}(\cdot,\cdot\mid C)}\bigr)
\]
for observed context \(C\), such as a prefix or a set of observed timestamps [2605.28507]. Generation then begins from \(X^{(0)}\sim \mu(\cdot\mid C)\), and the terminal path is interpreted as the forecast [2605.28507].

Training uses conditional flow matching. A pair of paths \((X^{(0)},X^{(1)})\) is sampled from a coupling \(q\), and the straight-line interpolant
\[
X^{(s)}=(1-s)X^{(0)}+sX^{(1)}
\]
has target velocity
\[
u_s(X^{(0)},X^{(1)})=X^{(1)}-X^{(0)}
\]
[2605.28507]. The training objective is
\[
\mathcal L(\theta) = \mathbb E_{\substack{s\sim\mathcal U[0,1]\\ (X^{(0)},X^{(1)})\sim q}} \Big[ \rho_2\bigl(F_\theta(s,X^{(s)}),\,X^{(1)}-X^{(0)}\bigr)^2 \Big].
\]
The model is therefore trained to learn a causal vector field on path space whose induced flow transports the prior path law to the data path law [2605.28507].

The paper further shows that any direct G-SLiCE generator can be represented as an augmented path-space flow with a frozen source component:
\[
Z^{(s)}=(U^{(s)},Y^{(s)}),\qquad \frac{d}{ds}\begin{pmatrix}U^{(s)}\\Y^{(s)}\end{pmatrix} = \begin{pmatrix}0\\G_\theta(U^{(s)})\end{pmatrix}.
\]
Then \(U^{(s)}=X\) for all \(s\), and \(Y^{(1)}=G_\theta(X)\) [2605.28507]. This establishes compatibility between direct path generators and path-space flow formulations.

A major conceptual claim of the paper is that the same pathwise universality that supports discriminative approximation also supports generative approximation in \(W_\infty\) for induced laws [2605.28507]. This links deterministic continuous causal approximation, transport of path distributions, and trainable flow-matching procedures within one formalism. A plausible implication is that the generative argument is not an isolated construction layered atop SLiCEs, but rather a direct consequence of the same structural properties that motivated the discriminative theory.

## 6. Empirical findings and documented limitations

The empirical literature on SLiCEs spans state-tracking, regular language length generalization, multivariate time-series classification, probabilistic forecasting, unconditional generation, and robustness to grid changes [2505.17761, 2605.28507].

On the \(A_5\) state-tracking benchmark, diagonal models such as Mamba, mLSTM, and diagonal LNCDE require more layers as sequence length increases, whereas dense, sparse, Walsh–Hadamard, block-diagonal, and DPLR LNCDEs solve the task with one layer in the reported setting [2505.17761]. The appendix parameter counts for sequence length \(20\) include: LSTM \(2{,}162{,}748\), sLSTM \(6{,}362{,}172\), mLSTM \(25{,}503{,}828\), Mamba \(26{,}531{,}260\), DeltaProduct \(69{,}609{,}580\), D-LNCDE \(12{,}724{,}284\), and WH-LNCDE \(4{,}323{,}388\) [2505.17761]. In this benchmark, WH-LNCDE is therefore highlighted as particularly parameter-efficient among the successful models.

On regular language tasks evaluating generalization from lengths \(3\!-\!40\) to \(40\!-\!256\), the reported average validation accuracies are \(71.6\) for D-LNCDE, \(60.1\) for WH-LNCDE, \(81.3\) for \(\mathrm{BD}_4\)-LNCDE, \(83.6\) for \(\mathrm{D\text{-}DE}_{16}\)-LNCDE, and \(71.8\) for \(\mathrm{DPLR}_4\)-LNCDE [2505.17761]. The literature emphasizes that block-diagonal models are the strongest among the parallelizable SLiCEs, and that larger dense blocks can improve performance [2505.17761].

On UEA multivariate time-series classification, the reported average accuracies are \(62.0\) for S6, \(64.3\) for Log-NCDE, \(61.5\) for D-LNCDE, \(64.5\) for BD-LNCDE, and \(62.2\) for DE-LNCDE [2505.17761]. Average time per 1000 steps is reported as \(20\) s for S6, \(1131\) s for Log-NCDE, \(7\) s for D-LNCDE, \(55\) s for BD-LNCDE, and \(87\) s for DE-LNCDE [2505.17761]. BD-LNCDE uses more memory than Log-NCDE but much less than dense LNCDE [2505.17761].

The 2026 generative study evaluates G-SLiCE on eight GluonTS datasets using CRPS and reports that the model is competitive with all baselines, achieves the best CRPS on \(6\) of \(8\) datasets, and outperforms TSFlow on \(7\) of \(8\) datasets [2605.28507]. For unconditional generation, it evaluates both \(2\)-Wasserstein distance and Linear Predictive Score (LPS), with G-SLiCE generally as good as or better than TSFlow, with especially clear gains on several datasets [2605.28507].

Robustness to grid changes is a major empirical point. On cross-frequency generalization on ETT, training at \(15\)-minute resolution and testing at \(12\)-hour resolution yields G-SLiCE CRPS around \(0.189\), while direct TSFlow evaluation can exceed \(900\) [2605.28507]. TSFlow can be patched with zero-order hold or GP oversampling, but the paper describes these as awkward and potentially costly [2605.28507]. On irregular grids, G-SLiCE maintains CRPS in a narrow band and low NRMSE across multiple irregularity levels, whereas TSFlow can have substantially worse NRMSE, interpreted as unstable means or large outliers [2605.28507].

The literature is also explicit about limitations. Sparse SLiCEs are not yet practically efficient in standard frameworks because of poor sparse-kernel support [2505.17761]. Maximal probabilistic expressivity is an asymptotic result, and finite-width expressivity remains open [2505.17761]. It is also unknown what exact structural conditions characterize all matrix families that preserve maximal expressivity, and additional SLiCE architectures may exist beyond those already studied [2505.17761]. In the generative analysis, the exact \(W_\infty\) universality statement is formulated for compactly supported laws, while Gaussian-process priors used in practice are handled through tightness and high-mass compact subsets rather than global compact support [2605.28507].

## 7. Conceptual significance and common points of confusion

A frequent source of confusion is the assumption that all efficient state space or selective recurrence models inherit the expressive properties of dense controlled systems. The SLiCE literature argues against this equivalence. In particular, diagonal state-transition matrices—used to interpret S4 and Mamba in the LNCDE view—are stated to be not maximally expressive [2505.17761], and the 2026 paper uses a concrete “hard-core” state-tracking example to illustrate a true expressive gap in the generative setting [2605.28507]. The target sequence
\[
C_1=Z_1,\qquad C_k=Z_k(1-C_{k-1}),\quad 2\le k\le n,
\]
induces a law \(\mu_{n,p}\) with no consecutive ones; a width-\(2\) dense selective model can represent it exactly, but a dense non-selective model cannot approximate the law arbitrarily well, and a diagonal selective model cannot represent the full state-tracking behavior for sufficiently long sequences [2605.28507]. The intended conclusion is not that diagonal models are unusable, but that they are structurally limited in ways that can become measurable in generative modelling.

Another point of confusion concerns whether “structured” is synonymous with “low expressivity.” The published results directly oppose that simplification: block-diagonal, sparse, Walsh–Hadamard, and appropriately growing DPLR parameterizations are all presented as structures that can match the maximal expressivity of dense matrices asymptotically [2505.17761]. This suggests that the decisive question is not whether a transition family is structured, but whether its structure preserves the ability to state-track and approximate arbitrary continuous path functionals.

A further distinction concerns what universality means in these papers. The 2025 work studies maximal probabilistic expressivity for continuous target functionals on compact path sets with linear readout [2505.17761]. The 2026 work lifts this to path-to-path approximation and then to induced pushforward laws in \(W_\infty\) for continuous causal maps on compact supports [2605.28507]. These are related but not identical claims. The latter adds a generative interpretation by showing how deterministic uniform approximation yields distributional approximation under a common input coupling [2605.28507].

Taken together, the two papers position SLiCEs as a framework in which the structure of input-dependent linear transitions becomes the principal design variable for balancing expressivity, computational cost, and generative adequacy [2505.17761, 2605.28507]. The documented results support three recurring conclusions: SLiCEs unify a broad range of sequence models under a continuous-time controlled-dynamics formalism; maximal expressivity is not confined to dense matrices; and continuous-time path-space formulations can be advantageous when the task requires arbitrary observation grids, irregular sampling, or explicit modelling of path laws [2505.17761, 2605.28507].

Source: https://www.emergentmind.com/topics/structured-linear-controlled-differential-equations-slices