Jacobian Regularizer-based Neural Granger Causality (2405.08779v1)
Abstract: With the advancement of neural networks, diverse methods for neural Granger causality have emerged, which demonstrate proficiency in handling complex data, and nonlinear relationships. However, the existing framework of neural Granger causality has several limitations. It requires the construction of separate predictive models for each target variable, and the relationship depends on the sparsity on the weights of the first layer, resulting in challenges in effectively modeling complex relationships between variables as well as unsatisfied estimation accuracy of Granger causality. Moreover, most of them cannot grasp full-time Granger causality. To address these drawbacks, we propose a Jacobian Regularizer-based Neural Granger Causality (JRNGC) approach, a straightforward yet highly effective method for learning multivariate summary Granger causality and full-time Granger causality by constructing a single model for all target variables. Specifically, our method eliminates the sparsity constraints of weights by leveraging an input-output Jacobian matrix regularizer, which can be subsequently represented as the weighted causal matrix in the post-hoc analysis. Extensive experiments show that our proposed approach achieves competitive performance with the state-of-the-art methods for learning summary Granger causality and full-time Granger causality while maintaining lower model complexity and high scalability.
- Kernelizing geweke’s measures of granger causality. In 2012 IEEE International Workshop on Machine Learning for Signal Processing, pp. 1–6. IEEE, 2012.
- Survey and evaluation of causal discovery methods for time series. Journal of Artificial Intelligence Research, 73:767–819, 2022.
- Neural graphical modelling in continuous-time: consistency guarantees and algorithms. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=SsHBkfeRF9L.
- Latent convergent cross mapping. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=4TSiOTkKe5P.
- Neural additive vector autoregression models for causal discovery in time series. In Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11–13, 2021, Proceedings 24, pp. 446–460. Springer, 2021.
- CUTS: Neural causal discovery from irregular time-series data. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=UG8bQcD3Emv.
- Cuts+: High-dimensional causal discovery from irregular time-series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 11525–11533, 2024a.
- Causaltime: Realistically generated time-series for benchmarking of causal discovery. In The Twelfth International Conference on Learning Representations, 2024b. URL https://openreview.net/forum?id=iad1yyyGme.
- Long-term forecasting with tide: Time-series dense encoder, 2023.
- On causal discovery from time series data using fci. Probabilistic graphical models, pp. 121–128, 2010.
- Rhino: Deep causal temporal relationship learning with history-dependent noise. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=i_1rbq8yFWC.
- Granger, C. W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pp. 424–438, 1969.
- Robust learning with jacobian regularization, 2019.
- Improving dnn robustness to adversarial attacks using jacobian regularization. In Proceedings of the European conference on computer vision (ECCV), pp. 514–529, 2018.
- Extensive chaos in the lorenz-96 model. Chaos: An Interdisciplinary Journal of Nonlinear Science, 20(4), nov 2010a. doi: 10.1063/1.3496397. URL https://doi.org/10.1063%2F1.3496397.
- Extensive chaos in the lorenz-96 model. Chaos: An interdisciplinary journal of nonlinear science, 20(4):043105, 2010b.
- Economy statistical recurrent units for inferring nonlinear granger causality, 2020.
- Causal recurrent variational autoencoder for medical time series generation. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7):8562–8570, Jun. 2023. doi: 10.1609/aaai.v37i7.26031. URL https://ojs.aaai.org/index.php/AAAI/article/view/26031.
- Generating realistic in silico gene networks for performance assessment of reverse engineering methods. Journal of computational biology, 16(2):229–239, 2009.
- Kernel method for nonlinear granger causality. Physical Review Letters, 100(14), April 2008. ISSN 1079-7114. doi: 10.1103/physrevlett.100.144103. URL http://dx.doi.org/10.1103/PhysRevLett.100.144103.
- Causal discovery with attention-based convolutional neural networks. Machine Learning and Knowledge Extraction, 1(1):19, 2019.
- Towards a rigorous assessment of systems biology models: the dream3 challenges. PloS one, 5(2):e9202, 2010.
- Local disentanglement in variational auto-encoders using jacobian l_1𝑙_1l\_1italic_l _ 1 regularization. Advances in Neural Information Processing Systems, 34:22708–22719, 2021.
- Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances, 5(11):eaau4996, 2019.
- Schreiber, T. Measuring information transfer. Physical review letters, 85(2):461, 2000.
- Network modelling methods for fmri. Neuroimage, 54(2):875–891, 2011.
- Causal network inference by optimal causation entropy. SIAM Journal on Applied Dynamical Systems, 14(1):73–106, 2015.
- Nts-notears: Learning non-parametric dbns with prior knowledge. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, pp. 1942–1964. PMLR, April 2023.
- Granger causality using jacobian in neural networks. Chaos: An Interdisciplinary Journal of Nonlinear Science, 33(2), 2023.
- Neural granger causality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4267–4279, 2021.
- Wiener, N. The theory of prediction. Modern mathematics for engineers, 1956.
- Scalable causal graph learning through a deep neural network. In Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1853–1862, 2019.
- Are transformers effective for time series forecasting? Proceedings of the AAAI Conference on Artificial Intelligence, 37(9):11121–11128, Jun. 2023. doi: 10.1609/aaai.v37i9.26317. URL https://ojs.aaai.org/index.php/AAAI/article/view/26317.
- Causality detection with matrix-based transfer entropy. Information Sciences, 613:357–375, 2022.