---
title: Multi-resolution Coherent Spatiotemporal Scale Separation
url: https://www.emergentmind.com/topics/multi-resolution-coherent-spatiotemporal-scale-separation
type: topic
---

# Multi-resolution Coherent Spatiotemporal Scale Separation

Multi-resolution coherent spatiotemporal scale-separation denotes a family of methods for decomposing multiscale data into hierarchies of scale-specific components while preserving spatial organization, temporal structure, and cross-scale consistency. In recent literature the term is made explicit by the multi-resolution Coherent Spatio-Temporal Scale Separation algorithm, mrCOSTS, which diagnoses coherent patterns or modes in multi-scale data through a hierarchical Dynamic Mode Decomposition workflow [2408.02396]. Closely related ideas appear in multi-resolution DMD, progressive multiresolution autoencoders, continuous space-time Fourier representations, adaptive graph models, spatiotemporal Gaussian-process approximations, graph-wavelet transforms on particle tessellations, and structure-preserving coarsening for process models; these methods differ in mathematical machinery, but they share the objective of separating slow from fast, coarse from fine, and global from local structure without discarding the coherence of the underlying spatiotemporal field [1506.00564].

## 1. Conceptual scope

Within this literature, a dataset is treated as multi-scale when it simultaneously exhibits at least two of the following properties: multiple spatial dimensions or variables, processes across orders of magnitude in temporal or spatial scale, non-stationarity, and invariances such as translation or rotation [2408.02396]. The corresponding analytical problem is not merely dimensionality reduction. It is the construction of a representation in which components are organized by scale and remain interpretable as coherent spatiotemporal structures.

The notion of coherence is method-dependent. In DMD-based methods, coherence refers to spatial modes that evolve with a shared temporal factor \(e^{\omega t}\), so that a single frequency or growth rate governs an entire spatial pattern [1506.00564]. In multiresolution convolutional autoencoders, coherence arises because weights learned at coarse levels are transferred to finer levels, anchoring fine-scale representations to coarse-scale structure [2004.04946]. In continuous Fourier field models for video, coherence is enforced by a unified function \(\hat{V}(x,y,t)\) over joint space-time, with motion encoded through phase relationships rather than explicit warping [2509.26325]. In coarsening frameworks for environmental and ecohydrological systems, coherence refers to preserving drainage topology, watershed morphology, functional heterogeneity, and dynamic stability across grid resolutions [2006.16606].

This suggests that multi-resolution coherent spatiotemporal scale-separation is best understood as a problem class rather than a single algorithm. The relevant separations may occur in temporal frequency, joint space-time frequency, graph topology, covariance residuals, latent neural features, or physically informed coarsening operators. What unifies them is a commitment to scale-aware decomposition without destroying the structures that make the original dynamics physically or statistically meaningful.

## 2. Modal and spectral hierarchies

The most explicit formulations arise from DMD and its multiresolution extensions. Standard DMD approximates a time-resolved state as
\[
\boldsymbol{x}(t_j) \approx \sum_{k=1}^K b_k \psi_k \lambda_k^{j-1},
\]
or, in continuous time,
\[
\boldsymbol{x}_{\text{DMD}}(t) = \sum_{k=1}^K b_k \psi_k e^{\omega_k t}.
\]
Modes with \(\omega_k \approx 0\) are interpreted as zero-modes or near-zero-frequency modes and represent approximately constant or slowly varying background structure; modes with frequencies bounded away from zero represent oscillatory or transient content [1506.00564].

Multi-resolution Dynamic Mode Decomposition generalizes this by recursively removing slow modes and repeating DMD on shorter subwindows. Its expansion can be written as
\[
\boldsymbol{x}_{\text{mrDMD}}(t) =
\sum_{\ell=1}^{L} \sum_{j=1}^{2^{\ell-1}} \sum_{k=1}^{m_\ell}
f_{\ell,j}(t)\, b_k^{(\ell,j)}\, \psi_k^{(\ell,j)}(\boldsymbol{\xi})\, e^{\omega_k^{(\ell,j)} t},
\]
where \(f_{\ell,j}(t)\) localizes each mode to a time bin. Long windows capture slow behavior; short windows capture faster and more transient dynamics. The resulting hierarchy is wavelet-like in time but retains DMD’s spatial coherence, making it suitable for background/foreground separation, El Niño extraction from sea-surface temperature data, and decomposition of objects moving at different rates in video [1506.00564].

mrCOSTS preserves the DMD basis but changes the separation strategy. Rather than recursively extracting only near-zero modes, it performs windowed DMD at multiple resolutions, clusters eigenvalues by \(|\operatorname{Im}(\omega)|\) within each level, and then performs a global clustering in \(\log_{10}(|\operatorname{Im}(\omega)|)\) across all levels to correct frequency leakage. Its local model for each window is
\[
\tilde{\mathbf{x}}^k(t) = \sum_{j=1}^{r} \boldsymbol{\phi}_j^k e^{\omega_j^k t} b_j^k + \mathbf{c}_k,
\]
and its overall reconstruction is expressed as a sum of global band contributions,
\[
\tilde{\mathbf{x}}(t) \approx \sum_p \tilde{\mathbf{x}}_p(t) + \mathbf{c}_{P,k}.
\]
This procedure is explicitly diagnostic rather than predictive: it is designed to separate frequency bands with coherent spatial structure in high-dimensional non-stationary data, and it uses variable-projection optimized DMD with a constraint that keeps \(\operatorname{Re}(\omega)\) small [2408.02396].

A common misconception is that these DMD hierarchies are purely temporal tools. They are not. Their central objects are spatial modes with simple temporal dynamics, and the scale-separation is meaningful precisely because temporal bands remain attached to coherent spatial fields. The distinction between mrDMD and mrCOSTS is therefore not whether spatial coherence is present, but how temporal bands are selected, localized, and stabilized in complex real data.

## 3. Learned and continuous representations

Recent neural formulations implement scale-separation through architectural hierarchy rather than explicit spectral recursion.

The multiresolution convolutional autoencoder, MrCAE, begins on a coarse grid with a compact convolutional autoencoder and progressively grows toward finer resolutions by embedding the previously trained network as a submodule, adding layers before and after it, and increasing the number of channels and spatial resolution of the newly added parts. Coarse networks learn large-scale, global representations; finer networks reuse those representations and add capacity for smaller-scale features. Restriction and prolongation are implemented implicitly by downsampling and upsampling operations, making the architecture multigrid-inspired. Because weights are transferred across resolutions, the learned representation remains coherent as the network deepens and widens itself [2004.04946].

Continuous space-time video super-resolution with 3D Fourier Fields adopts a different strategy. It represents video as a continuous function
\[
\hat{V}(x,y,t) : \mathbb{R}^2 \times [0,T] \rightarrow \mathbb{R}^3
\]
expanded in 3D sinusoids,
\[
\hat{V}(x,y,t) = \sum_{i=1}^N B_i(x,y,t; a_i,\phi_i,\boldsymbol{\omega}_i).
\]
The basis frequencies \(\boldsymbol{\omega}_i\) are shared globally across voxels and videos, while amplitudes and phases are local. Scale-separation is handled analytically through a Gaussian point spread function: for a Gaussian blur with variance \(\sigma\), each basis function is attenuated by
\[
\xi(\boldsymbol{\omega}_i,\sigma)=
\exp\!\left(-\frac{\|\boldsymbol{\omega}_i\|^2}{8\pi^2 \sigma^2}\right).
\]
Fine-scale structures are suppressed at coarse scales, high-frequency components are retained at fine scales, and the representation itself does not change with scale; only the sampling operator changes [2509.26325].

MedGNN implements yet another pattern. It constructs multi-resolution temporal embeddings with 1D convolutions of different kernel sizes \(k_m\), learns a separate adaptive graph \(A^{(m)}\) over channels at each resolution, applies Difference Attention Networks to finite differences in time, applies Frequency Convolution Networks in the Fourier domain, and fuses the resulting representations through a Multi-resolution Graph Transformer [2502.04515]. Temporal scale separation is induced by the kernel sizes, differencing, and frequency-domain filtering; spatial scale separation emerges because each temporal resolution has its own learned graph.

The principal representational families can be summarized as follows.

| Family | Core representation | Scale-separation mechanism |
|---|---|---|
| mrDMD / mrCOSTS | Windowed DMD modes | Hierarchical temporal bands |
| MrCAE | Multigrid-inspired CAE hierarchy | Coarse-to-fine transfer learning |
| VFF | Continuous 3D sinusoidal field | Gaussian PSF attenuation in frequency |
| MedGNN | Multi-resolution adaptive graphs | Per-scale temporal and graph branches |

These learned and continuous approaches show that scale-separation need not be tied to explicit modal orthogonality. In several cases, it is realized instead through nested architectures, shared frequency sets, or branch-wise specialization followed by coherent fusion.

## 4. Covariance, graph, and coarsening formulations

A parallel line of work casts scale-separation as hierarchical approximation of stochastic dependence. In spatiotemporal Multi-resolution Approximation, a mean-zero Gaussian process \(Y(\mathbf{s},t)\) is approximated by
\[
Y(\mathbf{s},t)\approx \sum_{m=0}^{M} Y_m(\mathbf{s},t),
\]
where each level corresponds to a recursive partition of the space-time domain. Large regions at coarse levels capture large-scale dependencies; smaller regions capture residual fine-scale variability. Conditional independence between different regions at the same partitioning level yields sparse block structure, while the residual construction preserves coherence with a single user-defined covariance function \(C_\theta\big((\mathbf{s},t),(\mathbf{s}',t')\big)\). The framework supports stationary separable models, stationary nonseparable spherical models, and nonstationary kernel-convolution models [2006.16606].

Graph-based multiresolution on particle tessellations replaces covariance algebra with wavelet analysis on irregular point clouds. A Delaunay tessellation defines the graph, Voronoi cell volumes enforce volume conservation, and a scalar field on particles is decomposed as
\[
s^0(\mathbf{x}) = \bar{s}^L(\mathbf{x}) + \sum_{\ell=1}^{L} \check{s}^{\ell}(\mathbf{x}).
\]
Projection from level \(\ell\) to \(\ell+1\) uses Voronoi-volume–weighted averaging, and the resulting detail coefficients quantify local scale-dependent differences between neighboring particles. The method is spatial at each snapshot, but because it acts on Lagrangian particles whose positions evolve in time, repeated application yields a time-resolved sequence of multiscale decompositions [2605.19244].

Structure-Function Coherent Coarsening extends the same general objective to process-based ecohydrological modeling. Fine-grid inputs \(x_f(\mathbf{r})\) are mapped to coarse-grid inputs by coarsening operators
\[
x_c(\mathbf{R}) = C[x_f](\mathbf{R}),
\]
with different operators for continuous structural fields, categorical functional fields, and initial conditions. Hydro-aware coarsening weights DEM values by flow accumulation to preserve flow pathways and watershed morphology; Auto-weight and Auto-reassign alleviate dominant-class bias in categorical land-use and soil maps; hydrology-, land-cover-, and soil-aware strategies preserve process-relevant initial states. Here coherence means structural consistency and functional preservation together maintaining dynamic stability through spatiotemporal feedback [2511.17482].

These formulations make clear that multi-resolution coherent spatiotemporal scale-separation is not limited to frequency decomposition. It can be realized through recursive covariance residuals, graph-wavelet details on irregular domains, or physically informed coarsening operators that preserve the mechanisms responsible for long- and short-timescale behavior.

## 5. Empirical domains and observed behavior

The explicit mrCOSTS studies span climate, neuroscience, and atmospheric boundary-layer dynamics. For tropical Pacific sea-surface temperature, a dyadic hierarchy of window sizes \([16, 32, 64, 128, 256, 512, 1024]\) months with rank \(r=8\) yielded a reconstruction error of about \(11\%\), recovered seasonal and interannual scales, and identified six ENSO-like bands; the 2015–2016 event was characterized by all six ENSO-like bands being in a positive phase simultaneously, with the 1.4 year band the largest contributor to the anomaly. For motor-cortex local field potentials, the method identified 14 global bands and reconstruction errors of about \(21\%\) over the full period and \(14\%\) excluding edges, while revealing traveling patterns in the CNV and MRP ranges. For mountain-boundary-layer wind fields, reconstruction errors were \(23\%\) for \(u\) and \(26\%\) for \(v\), improving to \(13\%\)–\(15\%\) when excluding the first and last 20 minutes, and the resulting decomposition exposed seiche-like oscillations and tributary inflow pulses [2408.02396].

Video super-resolution provides a different empirical test of coherence across scales. On Vid4 at \(\times 4\) spatial and \(\times 2\) temporal upscaling, VFF reports temporal optical flow error values of 0.595 for Bicubic, 0.344 for VideoINR, 0.354 for MoTIF, 0.323 for BF-STVSR, and 0.257 for VFF “3” with 0.250 for 3-Large. For temporal-only super-resolution on Adobe240 at \(\text{S}\times 1, \text{T}\times 8\), the reported PSNR/SSIM values are 24.69 / 0.709 for VideoINR, 28.09 / 0.843 for MoTIF, 29.37 / 0.867 for BF-STVSR, and 33.31 / 0.935 for “3” [2509.26325].

In medical time-series classification, MedGNN is evaluated on three EEG and two ECG datasets under both sample-based and subject-based splits. In the subject-based PTB experiment, it reports Accuracy 84.53 and AUROC 93.31, compared with 83.41 for iTransformer on accuracy and 93.13 for Medformer on AUROC. The reported gains are especially pronounced on EEG datasets, where richer spatial structure and baseline wander make multi-resolution graphs and difference/frequency branches more consequential [2502.04515].

For global environmental modeling, spatiotemporal MRA reports computation times reduced around two orders of magnitude with an increase of the root-mean-square prediction error of around five percent in simulation, thereby establishing an explicit trade-off between computational scalability and prediction fidelity. For ecohydrological coarsening, Hydro-aware DEM aggregation yields streamflow NSE-KGE \(> 0.85\) on average even at \(20\times\) coarsening, while Auto-weight and Auto-reassign reduce variability in nitrate-response degradation across basins and coarsening factors [2006.16606].

Tessellation-graph multiresolution analysis demonstrates comparable agreement with Fourier-based scale diagnostics while preserving localization on irregular particle clouds. In homogeneous isotropic turbulence with \(N_p=1.5\times10^7\) inertial particles at \(St=1\), the wavelet spectrum of particle-velocity divergence reproduces the change in scaling around \(k\eta \approx 0.2\), flatness remains near 10 across an intermediate range and rises for \(k_V^\ell\eta \gtrsim 0.4\), and nonlinear filtering of large wavelet coefficients isolates caustic-induced fluctuations from longer-lived clustering structure [2605.19244].

## 6. Limitations, ambiguities, and open directions

No single formulation resolves all aspects of the problem. DMD-based methods are interpretable and unsupervised, but they are diagnostic rather than forecasting tools, they exhibit edge effects analogous to a cone-of-influence, and they rely on local approximations by coherent exponential dynamics; spatially incoherent noise is largely excluded rather than modeled [2408.02396]. mrDMD additionally depends on choices of level depth, slow-mode thresholds, and SVD truncation, and hard indicator windows can introduce boundary artifacts [1506.00564].

Neural and continuous-field approaches trade explicit modal structure for flexible representation. MrCAE is optimized for reconstruction rather than explicit temporal forecasting and remains sensitive to design choices such as depths, channel counts, and restriction/prolongation operators [2004.04946]. VFF explicitly handles spatiotemporal sampling with Gaussian PSF, but more complex degradations such as sensor noise, non-Gaussian motion blur, and compression are not modeled analytically, and finite basis size can become a bottleneck in extremely high-frequency regimes [2509.26325]. MedGNN achieves scale separation implicitly through architectural modularization rather than an explicit orthogonal decomposition or formal scale-separation penalty, and its fusion across resolutions is a simple averaging operation rather than a learned cross-scale alignment mechanism [2502.04515].

Statistical and coarsening formulations face different constraints. Spatiotemporal MRA assumes conditional independence between regions at the same level given coarser levels, and results degrade when partition depth is increased without enough basis functions per region [2006.16606]. Tessellation-graph wavelets currently use Haar-type constructions with one vanishing moment, suffer from spread in effective wavenumbers when particle clustering generates wide volume distributions, and do not yet perform a joint space-time transform [2605.19244]. Structure-Function Coherent Coarsening remains heuristic in its operator design, was developed and tested in the Salish Sea Basin, and leaves open the problem of formulating explicit optimization criteria for structural error, process-aware weighting, and long-timescale error decay under non-stationary forcing [2511.17482].

A persistent ambiguity concerns what exactly is being separated. In some methods the separation is primarily temporal frequency; in others it is spatial resolution, graph scale, covariance residual, or process-relevant heterogeneity under grid aggregation. This suggests that the field is unified more by a methodological principle than by a single canonical representation. The principle is that multiscale systems should be analyzed or coarsened through hierarchies that preserve the coherence of the structures that matter at each scale, rather than through monolithic models that attempt to learn all scales simultaneously.

Source: https://www.emergentmind.com/topics/multi-resolution-coherent-spatiotemporal-scale-separation