Inferring the time-varying coupling of dynamical systems with temporal convolutional autoencoders (2406.03212v1)

Abstract: Most approaches for assessing causality in complex dynamical systems fail when the interactions between variables are inherently non-linear and non-stationary. Here we introduce Temporal Autoencoders for Causal Inference (TACI), a methodology that combines a new surrogate data metric for assessing causal interactions with a novel two-headed machine learning architecture to identify and measure the direction and strength of time-varying causal interactions. Through tests on both synthetic and real-world datasets, we demonstrate TACI's ability to accurately quantify dynamic causal interactions across a variety of systems. Our findings display the method's effectiveness compared to existing approaches and also highlight our approach's potential to build a deeper understanding of the mechanisms that underlie time-varying interactions in physical and biological systems.

• The paper introduces TACI, which uses a two-headed TCN architecture and the CSGI metric to capture dynamic causal interactions.
• It demonstrates robust performance on both synthetic systems and real-world datasets, outperforming traditional methods like Granger causality.
• Results show TACI’s effectiveness in detecting unidirectional and bidirectional couplings in non-linear, non-stationary dynamical systems.

Inferring the Time-Varying Coupling of Dynamical Systems with Temporal Convolutional Autoencoders

Introduction

Calderon and Berman introduce a novel methodology, Temporal Autoencoders for Causal Inference (TACI), aimed at identifying causal interactions in complex dynamical systems, which are inherently non-linear and non-stationary. Traditional methods like Granger Causality (GC) fail to address these complexities effectively due to assumptions about linearity and stationarity that do not hold in real-world systems. TACI leverages a unique surrogate data comparison metric, the Comparative Surrogate Granger Index (CSGI), and a two-headed Temporal Convolutional Network (TCN) architecture to detect and measure time-varying causal interactions. The efficacy of TACI is evaluated against synthetic and real-world datasets, demonstrating significant improvements over existing approaches.

Methodology

Comparative Surrogate Granger Index (CSGI)

The CSGI measures the relative improvement in prediction accuracy when using actual data versus surrogate data that preserves the statistical properties of the original time series but disrupts any specific temporal information. This metric refines the traditional GC approach by computing the CSGI as:

$$\chi_{x\to y} = \frac{R^2_{xy} - R^2_{x^{(s)} y}}{\frac{1}{2}(R^2_{xy} + R^2_{x^{(s)} y})}$$

where $R^2_{xy}$ represents the fraction of variance explained using both variables $x$ and $y$, and $R^2_{x^{(s)}y}$ represents the same using one variable and the surrogate of the other. This method allows for detecting subtle changes in causal interactions over time.

Temporal Autoencoders for Causal Inference (TACI)

The TACI model utilizes a two-headed TCN autoencoder architecture. This design processes two time series in parallel, extracting temporal features from each series individually and then merging these in a bottleneck layer representing potential causal interactions. The decoder part of the network then predicts a future state of one of the variables. The training process involves creating four versions of this network, substituting one of the time series with its surrogate. The CSGI values are then computed to assess the direction and strength of causal interactions.

Results

Synthetic Test Systems

Rössler-Lorenz System:

TACI successfully identified unidirectional coupling from the Rössler oscillator to the Lorenz system, maintaining accuracy even as coupling strengths varied. Other methods like SLGC, CCM, and TE failed to accurately detect these interactions due to synchronization effects or inherent method limitations.

Bidirectional Two-Species Model:

TACI accurately captured both the bi-directional nature and varying coupling strengths between the two variables. In contrast, SLGC could not detect couplings from $y$ to $x$, and while CCM identified bi-directionality, it could not distinguish relative coupling strengths.

Coupled Autoregressive Models:

TACI and SLGC both effectively identified coupling and its direction, while TE and CCM struggled, especially in unidirectional coupling scenarios.

Non-Stationary Coupled Hénon Maps:

TACI was robust in detecting changes in coupling strengths over time, even capturing temporal variations and switches in causal interactions, demonstrating its ability to handle dynamically altering systems.

Real-World Applications

Jena Climate Dataset:

TACI detected known empirical relationships between temperature, dew point, and relative humidity. The method consistently showed stronger interactions during colder periods, aligning with theoretical predictions.

Non-Human Primate Electrocorticography (ECoG):

TACI revealed significant changes in causal interactions between brain regions before, during, and after anesthesia, capturing subtle temporal fluctuations. The methodology outperformed previous techniques and offered insights into the dynamic brain activity relationships.

Discussion

The results from both synthetic and real-world datasets indicate that TACI provides a robust framework for inferring time-varying causal interactions in complex, non-linear, and non-stationary systems. The method's ability to use a single trained model to detect temporal variations without retraining is particularly advantageous. However, the approach can be computationally intensive and prone to overfitting, necessitating careful training and validation.

While TACI shows promise for a wide range of applications, including neural data analysis, future work may focus on improving computational efficiency and exploring additional real-world datasets to further validate its utility.

Conclusion

TACI represents a significant methodological advancement in the field of causal inference for complex dynamical systems. By effectively combining a novel surrogate data metric with a sophisticated neural network architecture, it addresses the limitations of traditional methods and offers a powerful tool for exploring time-varying interactions. The method's success across different datasets underscores its potential for broad applicability and future developments in artificial intelligence and complex systems analysis.