---
title: SpaceTime MDL Regime Discovery
url: https://www.emergentmind.com/topics/spacetime-mdl-based-regime-discovery
type: topic
---

# SpaceTime MDL Regime Discovery

SpaceTime MDL-Based Regime Discovery is a unified approach for causal discovery in non-stationary, multi-context time series data. By leveraging the Minimum Description Length (MDL) principle, this framework combines identification of regime changepoints, causal graph structure learning, and partitioning of both datasets (“contexts”) and time intervals (“regimes”) into domains where invariant causal mechanisms govern the data. Unlike methods that assume stationarity or allow only a single form of distributional change, SpaceTime models both spatial and temporal heterogeneity, with mechanisms allowed to shift independently across blocks. The method operates by fitting nonparametric Gaussian Process (GP) models to each system component, detecting mechanism changes using kernel-based statistics, and minimizing a global MDL criterion over all partitions and graphs [2501.10235].

## 1. Problem Formulation and Motivation

Many real-world processes—such as climate, hydrology, or ecosystem fluxes—exhibit nonstationarity: causal relationships change over time (e.g., seasonal effects, long-term trends) and may vary across spatial domains or operational contexts (e.g., different catchments or ecosystems). Standard time series causal discovery often assumes a single, stable (stationary) generating process, risking failure to detect shifting or repeated regimes and spatially localized event structures. The regime discovery task is to learn:

- A window causal graph \(G\) (a condensed DAG incorporating both instantaneous and lagged dependencies)
- A set of temporal regime changepoints \(\mathcal L\)
- A regime partition \(\mathcal R\) of time indices, segmenting the data into intervals where mechanisms are constant
- A context partition \(\mathcal C\) of datasets, grouping spatial locations with identical mechanisms

The aim is that, within each block defined by the context–regime tuple \((k, r)\), the system’s causal mechanisms are stable, but these mechanisms may differ across blocks [2501.10235].

## 2. Formal Definitions: Contexts, Regimes, and Causal Models

Formally, given \(m\) continuous variables observed over timepoints \(t = 1, \dots, n\) in \(D\) datasets, data are denoted \(X^d_t\) for each \(d \in \mathcal D\). The partitioning constructs are as follows:

- **Contexts**: Partition \(\mathcal C = \{C_1, \ldots, C_K\}\) of datasets such that within each \(C_k\), all members share identical variable mechanisms.
- **Changepoints and Regimes**: Changepoints \(\mathcal L \subset \{1, \ldots, n\}\) signal times where any variable’s mechanism shifts. The intervals between changepoints segment time into regimes \(\mathcal R = \{R_1, \ldots, R_L\}\) so that all timepoints in a regime share identical mechanisms.
- **Spatio-Temporal Block**: For context \(k\) and regime \(r\), \(s = (k, r)\) denotes the collection \(\{X^d_t : d \in C_k,\, t \in R_r\}\).

The underlying generative model assumes that for all \(d \in C_k\), \(t \in R_r\), each variable \(X^d_{(i), t}\) admits a structural equation:
\[
X^d_{(i),t} = f^{(i)}_{k, r}(\mathrm{PA}(X^d_{(i),t})) + \varepsilon^d_{(i),t}
\]
where \(\mathrm{PA}(X^d_{(i),t})\) denotes the set of (instantaneous and lagged) parents in window graph \(G\), and \(f^{(i)}_{k, r}\) may differ across context–regime blocks [2501.10235].

## 3. Minimum Description Length (MDL) Objective

SpaceTime operationalizes regime and context discovery by employing the two-part MDL principle as an approximation to Kolmogorov complexity within the Algorithmic Model of Causation. For candidate model \(\mathcal M = (G, \mathcal L, \mathcal C, \mathcal R)\) and data \(\mathbf X\):
\[
L(\mathbf X, \mathcal M) = L(\mathcal M) + L(\mathbf X \mid \mathcal M)
\]
where \(L(\mathcal M)\) encodes the graph, changepoints, and partitions, and
\[
L(\mathbf X \mid \mathcal M) = \sum_{i = 1}^m \sum_{k = 1}^K \sum_{r = 1}^L L(X_{(i)} \mid \mathrm{PA}(X_{(i)}); C_k, R_r)
\]
Each conditional is modeled using a Gaussian Process with kernel \(K\) and noise variance \(\sigma^2\), yielding:
\[
L(X_{(i)} \mid \mathrm{PA}(X_{(i)}); C_k, R_r) = \min_{f \in \mathcal H_K} \left[ -\log P(X_{(i)} \mid f(\mathrm{PA})) + \|f\|_{\mathcal H_K}^2 \right] + \frac{1}{2} \log \det(\sigma^{-2}K + \mathbb I)
\]
where \(\|f\|_{\mathcal H_K}^2\) regularizes functional complexity [2501.10235].

The inferred model is the minimizer of the total code length:
\[
\hat{\mathcal M} = \arg\min_{G, \mathcal L, \mathcal C, \mathcal R} L(\mathbf X, \mathcal M)
\]

## 4. SpaceTime Algorithmic Framework

The combinatorial joint optimization over causal graphs, changepoints, and partitions is computationally hard, so SpaceTime uses an interleaved, iterative scheme:

```plaintext
Algorithm SpaceTime
Input: time series {X^d_t}, min regime length d_min, max lag τ_max
Output: window causal graph G, changepoints 𝓛, context-partitions 𝒞, regime-partitions ℛ

1. Initialize G←∅
2. repeat
3.   𝓛 ← ChangepointDetection(G, d_min)
4.   (ℛ,𝒞) ← Partitioning(G,𝓛)
5.   G ← EdgeGreedySearch(ℛ,𝒞,𝓛,τ_max)
6. until no change in (G,𝓛,ℛ,𝒞)
7. return (G,𝓛,ℛ,𝒞)
```

Component modules:

- **ChangepointDetection**: Fit GPs for each variable and context in the last known regime; apply kernel-based changepoint methods (e.g., PELT) to GP residuals to locate abrupt mechanism shifts and update \(\mathcal L\).
- **Partitioning**: Aggregate data from candidate pairs of datasets (contexts) or time intervals (regimes); test equality of conditional distributions \(X \mid \mathrm{PA}\). Kernel-based conditional independence tests (HSIC) are used with an indicator variable \(S\), grouping together samples with no significant differences into coherent contexts or regimes (\(\mathcal C\) and \(\mathcal R\)).
- **EdgeGreedySearch**: For fixed partitions and changepoints, edges are added greedily to the window DAG \(G\) in order of maximal MDL gain \(\delta E\), followed by backward pruning [2501.10235].

## 5. Nonparametric Functional Models and Kernel Testing

SpaceTime leverages nonparametric Gaussian Process (GP) regression to flexibly model the potentially nonlinear conditional mechanisms \(f^{(i)}_{k,r}\). GP hyperparameters are optimized under maximum-marginal MDL. To discriminate between candidate regimes or contexts, the framework employs kernel-based conditional independence tests—specifically, the Hilbert-Schmidt Independence Criterion (HSIC)—to test the null hypothesis \(H_0: P_s(X|\mathrm{PA}) = P_{s'}(X|\mathrm{PA})\) between subsamples \(s\) and \(s'\). By relabeling pooled data with an environment indicator \(S\), one can statistically test whether \(X \mid \mathrm{PA}\) is independent of the environment label, and partition accordingly [2501.10235].

## 6. Consistency Guarantee and Theoretical Properties

Subject to assumptions including (i) sufficient sample size and regime length, (ii) Shift-Faithfulness (each mechanism unique to its block), (iii) independent mechanism changes across variables, and (iv) causal Markov plus Faithfulness, the global MDL criterion
\[
\min_{\mathcal M} L(\mathbf X, \mathcal M)
\]
is, as sample size \(\to \infty\), almost surely minimized uniquely by the true causal graph \(G^*\), changepoints \(\mathcal L^*\), and partitions \(\mathcal C^*, \mathcal R^*\) [2501.10235].

## 7. Empirical Evaluation: Synthetic and Real-World Case Studies

Experiments address both synthetic and observational multivariate time series:

- **Synthetic Benchmarks**: Randomly generated WCGs (\(\tau_{\max} = 2\)), random changepoints (\(d_{\min} = 30\)), diverse mechanisms (linear/nonlinear), and noise distributions are used. Compared to methods assuming stationarity (PCMCI+, VARLiNGAM, DYNOTEARS), those with single-regime changepoints (RPCMCI), multi-context discovery (J-PCMCI), and nonstationary-aware methods (CD-NOD), SpaceTime achieves superior directed F1 for graph recovery, F1 for changepoint detection, and high ARI/NMI for partitioning.
- **River Discharge Data**: Using daily measurements of temperature, precipitation, and discharge at 307 European catchments (2010), SpaceTime recovers physically validated causal dependencies (e.g., \(P_{t-1} \to Q_t\)), and reveals spatial gradients in causal strength correlating with known catchment characteristics.
- **Biosphere–Atmosphere Fluxes**: Applied to FLUXNET tower data (air temperature, radiation, net ecosystem exchange, heat, vapor pressure deficit) at 63 sites (2000–2009), SpaceTime identifies a consistent DAG and maps edge strengths onto low-dimensional embeddings (t-SNE), corresponding to established meteorological states. It detects regime shifts relating to real phenomena, including the 2003 European heatwave [2501.10235].

## 8. Practical Implications

SpaceTime encapsulates causal discovery, changepoint detection, and context/regime partitioning for non-stationary, multi-context multivariate time series, providing a consistent and interpretable model for evolving causal structures. Its use of nonparametric, MDL-scored GPs allows functional flexibility, while kernel-based independence testing ensures robust partitioning. Empirical results confirm that the method quantitatively recovers known causal effects, captures spatial and temporal heterogeneity, and detects meaningful regime changes induced by concrete environmental and climatological events [2501.10235].

Source: https://www.emergentmind.com/topics/spacetime-mdl-based-regime-discovery