Linear Spatiotemporal Representations

• Linear spatiotemporal representations are techniques that encode, analyze, and forecast data with spatial and temporal dependencies using linear operators and basis expansions.
• They employ methodologies such as direct linear mappings, latent factor decompositions, slow feature analysis, and hybrid optical networks for robust forecasting and interpolation.
• These methods offer practical benefits like reduced error accumulation, efficient dimensionality reduction, and interpretable, scalable modeling across diverse applications.

Linear spatiotemporal representations encode, analyze, and predict data exhibiting both spatial and temporal dependencies using linear operators or basis expansions. These representations arise across forecasting, dimensionality reduction, system identification, and physical modeling, spanning domains such as multivariate time series, event-based sensing, statistical kriging, neural signal analysis, and quantum or optical architectures. The shared core is the exploitation of linear structure—via projections, factor models, or explicit matrix mappings—to capture, compress, and manipulate high-dimensional spatiotemporal data efficiently and interpretably.

1. Formal Structure and Core Classes

Linear spatiotemporal representations take several mathematically distinct but conceptually related forms:

1. Direct Linear Mappings for Forecasting: Models like the SpatioTemporal-Linear (STL) framework operate on data arrays $X \in \mathbb{R}^{N \times L}$ (variables × history) and predict multistep horizons $Y \in \mathbb{R}^{N \times H}$ by learning direct affine maps $W$ and $b$, such that $\hat{Y} = W X^\top + b \mathbf{1}_N^\top$, avoiding iterative propagation and its error accumulation. They augment this core with explicit temporal and spatial bypasses—positional embeddings, date-time embeddings, and inter-variable attention—for universal robustness across regime type and data abundancy (Zuo et al., 2023).
2. Latent Factor and Basis Expansions: High-dimensional spatiotemporal fields (e.g., $y_t(s)$ at locations $s$ and times $t$) are decomposed as linear combinations of spatial basis functions $a_j(s)$ and low-rank temporal coefficients $x_{t,j}$, i.e., $Y \in \mathbb{R}^{N \times H}$0. Estimation involves empirical cross-covariances, smoothness penalties (e.g., graph Laplacian), and orthogonality normalization, yielding interpretable, data-adaptive spatial components and facilitating reduced-rank spatiotemporal kriging and forecasting (Chen et al., 2020, Huang et al., 2016).
3. Linear Spatiotemporal Feature Construction: For high-resolution event data (e.g., neuromorphic sensing), Slow Feature Analysis (SFA) constructs linear projections $Y \in \mathbb{R}^{N \times H}$1 on voxelized (space × time) event histograms, with $Y \in \mathbb{R}^{N \times H}$2 chosen to generate outputs that vary minimally across short spatiotemporal transformations, thus encoding robust, invariant features (Ghosh et al., 2019).
4. Symbolic Linear Encodings in Dynamical Lattices: In dynamical systems such as the spatiotemporal cat map, any state on a 2D (space × time) lattice is linearly encoded by a corresponding lattice of symbols via an invertible Green's function $Y \in \mathbb{R}^{N \times H}$3, establishing an isomorphism between real-valued states and symbolic patterns, with rigorous methods for computing the statistical weight (frequency) of blocks (Gutkin et al., 2019).
5. Hybrid Linear-Optical Networks: Arbitrary unitary operations on combined space (waveguides) and time (pulse bins) are realized by constructing layered products of spatial and temporal unitary blocks, exploiting the tensor product structure of the joint Hilbert space and factorizing $Y \in \mathbb{R}^{N \times H}$4 linear maps efficiently (Su et al., 2018).

2. Methodological Foundations

Direct Forecasting via Skip Connections and Bypass Routes

STL exemplifies a design where all spatial and temporal dependencies are modeled through multiple parallel additive linear routes:

• Core path: residual linear (Res-L) layers with activation, forming a convex and stable linear regression baseline.
• Temporal route: positional and date-time embeddings are linearly integrated, with temporal inductive biases activated under data scarcity, capturing periodic or calendar effects.
• Spatial route: channel identity marking followed by attention weights (row-wise softmax on self-interaction scores), effectively yielding a linear but adaptive aggregation across variables. All paths operate in parallel with skip-sum fusion, mitigating error accumulation and yielding convex subproblems for reliable training and rapid inference (Zuo et al., 2023).

Latent Low-Dimensional Structure Recovery

Factor-type models and their modern nonparametric extensions (as in geostatistical kriging or high-dimensional time series) express the observed data as $Y \in \mathbb{R}^{N \times H}$5, where $Y \in \mathbb{R}^{N \times H}$6 are spatial loadings and $Y \in \mathbb{R}^{N \times H}$7 latent temporal factors. Estimation proceeds by:

• Singular vector estimation over spatial blocks and cross-covariances.
• Graph-Laplacian or smoothness regularization to encode spatial continuity (Huang et al., 2016).
• Aggregation over random partitions for improved statistical efficiency.

Prediction leverages this structure for both spatial kriging (interpolation at new $Y \in \mathbb{R}^{N \times H}$8 via $Y \in \mathbb{R}^{N \times H}$9) and temporal forecasting (VAR/MAR models on $W$0) (Chen et al., 2020, Huang et al., 2016).

Slow Feature Analysis for Spatiotemporal Event Data

SFA algorithms optimize for linear projections $W$1 whose variation over temporally-close pairs is minimized, via generalized eigenproblems $W$2 (with $W$3 the covariance of time differences and $W$4 the variance), extracting a basis that is robust to affine transformations in space-time (Ghosh et al., 2019).

Hybrid Linear-Optical Network Decomposition

Space–time unitary transformations $W$5 are factorized as products of spatial block unitaries, temporal shifts, and coupling blocks through elimination-based or CS decompositions. Hardware-efficient implementation leverages parallel spatial interferometers and sequential time-multiplexing, with quantitative advantage in loss and speed over spatial- or temporal-only schemes (Su et al., 2018).

3. Practical Applications and Empirical Regimes

Linear spatiotemporal representations are deployed in:

• Multivariate Forecasting: STL outperforms or matches deep transformer or recurrent baselines in time series with varying horizon-lengths and limited historical data. It shows pronounced advantage under data scarcity (e.g., pedestrian trajectory, rare disease progression), with MSE improvements up to 55% in such regimes (Zuo et al., 2023).
• Dimension Reduction & Kriging: In high-dimensional climate, neural, or sensor data, latent-factor models compress spatial and variable axes while preserving covariance and enabling scalable spatial/temporal interpolation, with dramatic computational savings and theoretical guarantees on error rates (Chen et al., 2020, Huang et al., 2016).
• Event-Based Vision: SFA-derived representations for event camera data yield feature descriptors with invariance to translation, scaling, and rotation, enhancing robustness in downstream tasks such as feature tracking (Ghosh et al., 2019).
• Quantum and Optical Information Processing: Hybrid architectures for universal linear optics achieve resource-efficient and low-loss implementation of arbitrary linear evolutions over high-dimensional spatiotemporal modes (Su et al., 2018).
• Symbolic Dynamics: Linear encoding in spatiotemporally extended systems allows explicit calculation of block entropy and pattern frequencies through lattice Green’s functions and volume computations, crucial for chaos and statistical mechanics (Gutkin et al., 2019).

4. Comparative Insights and Theoretical Properties

Empirical and theoretical findings highlight several comparative features:

• Universality: Models like STL that exploit both direct linear coupling and inductive bias routes match deep models when data is abundant and outperform them under data paucity. Their parameter count scales as $W$6, and all components admit convex optimization (Zuo et al., 2023).
• Compression vs Reconstruction: Data-driven bases (e.g., PCA, SVD, autoencoders on spatial axes) yield lower reconstruction error in highly variable datasets, while structure-aligned bases (e.g., graph Fourier) perform best when signal aligns with known structures. However, downstream sequential prediction is often robust to this choice, as temporal models compensate for spatial errors (Bontonou et al., 2019).
• Covariance Preservation: Low-rank spatiotemporal models maintain key cross-covariances across space, variables, and time, with controllable error as sample size and spatial grid increases (Chen et al., 2020, Huang et al., 2016).
• Algorithmic Stability: Convexity and linearity in sub-blocks ensure fast, stable training and inference, essential for high-dimensional or real-time applications (Zuo et al., 2023).
• Theoretical Consistency: Asymptotic analyses establish rates for loading recovery, kriging error, and forecast error, depending on dimension, sample size, and factor strengths (Huang et al., 2016, Chen et al., 2020).

5. Extensions, Limitations, and Open Directions

While linear spatiotemporal representations afford interpretability, scalability, and robustness, they are subject to natural limitations:

• Linearity: All methods above assume the underlying signal is adequately captured by linear combination or linear operator action. Nonlinearities, abrupt regime changes, or higher-order interactions are not resolved.
• Stationarity/Separability Assumptions: Many frameworks posit stationary or separable kernels, which can fail when space-time interactions are strongly non-separable.
• Graph structure bias: For structure-based embeddings, misalignment between data and assumed spatial structures degrades compression efficiency, though temporal models can partially compensate (Bontonou et al., 2019).
• Dimensionality bottlenecks: While low-rank models compress, choosing rank and enforcing identifiability requires careful statistical estimation, especially under weak “factor strength”.

Active research explores how to enrich linear models (e.g., hybrid linear-nonlinear pipelines, attention-based combinations, data-driven spatial graph adaptation) or extend to richer physical implementations (e.g., quantum or optical regimes) (Su et al., 2018).

6. Representative Methodological Table

Application Area	Model or Principle	Canonical Reference
Multivariate TS Forecasting	STL (core + temporal + spatial routes)	(Zuo et al., 2023)
High-Dimensional Kriging	Latent factor + loading estimation	(Huang et al., 2016, Chen et al., 2020)
Event-Based Feature Learning	SFA on voxelized local space-time histograms	(Ghosh et al., 2019)
Symbolic Encoding (Dynamics)	Lattice Green's function linear encoding	(Gutkin et al., 2019)
Linear-Optical Architectures	Space–time block decomposition of unitaries	(Su et al., 2018)

These representative examples illustrate the diversity and technical depth underlying linear spatiotemporal representations across statistical learning, physical modeling, and computational science.