Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning domain-specific causal discovery from time series (2209.05598v3)

Published 12 Sep 2022 in cs.LG, cs.AI, and stat.ME

Abstract: Causal discovery (CD) from time-varying data is important in neuroscience, medicine, and machine learning. Techniques for CD encompass randomized experiments, which are generally unbiased but expensive, and algorithms such as Granger causality, conditional-independence-based, structural-equation-based, and score-based methods that are only accurate under strong assumptions made by human designers. However, as demonstrated in other areas of machine learning, human expertise is often not entirely accurate and tends to be outperformed in domains with abundant data. In this study, we examine whether we can enhance domain-specific causal discovery for time series using a data-driven approach. Our findings indicate that this procedure significantly outperforms human-designed, domain-agnostic causal discovery methods, such as Mutual Information, VAR-LiNGAM, and Granger Causality on the MOS 6502 microprocessor, the NetSim fMRI dataset, and the Dream3 gene dataset. We argue that, when feasible, the causality field should consider a supervised approach in which domain-specific CD procedures are learned from extensive datasets with known causal relationships, rather than being designed by human specialists. Our findings promise a new approach toward improving CD in neural and medical data and for the broader machine learning community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. On directed information theory and granger causality graphs. Journal of computational neuroscience, 30(1):7–16, 2011.
  2. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680, 2019.
  3. Large-scale chemical process causal discovery from big data with transformer-based deep learning. Process Safety and Environmental Protection, 173:163–177, 2023.
  4. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp.  108–122, 2013.
  5. David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov):507–554, 2002.
  6. Understanding how image quality affects deep neural networks. In 2016 eighth international conference on quality of multimedia experience (QoMEX), pp.  1–6. IEEE, 2016.
  7. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  8. High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems, 33:12615–12625, 2020.
  9. Jacob Gildenblat and contributors. Pytorch library for cam methods. https://github.com/jacobgil/pytorch-grad-cam, 2021.
  10. Causal inference in public health. Annual review of public health, 34:61–75, 2013.
  11. Review of causal discovery methods based on graphical models. Frontiers in genetics, 10:524, 2019.
  12. Rhino: Deep causal temporal relationship learning with history-dependent noise. arXiv preprint arXiv:2210.14706, 2022.
  13. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
  14. Causal generative neural networks. arXiv preprint arXiv:1711.08936, 2017.
  15. Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pp. 424–438, 1969.
  16. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21, 2008.
  17. Generalized score functions for causal discovery. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp.  1551–1560, 2018.
  18. Causal discovery and forecasting in nonstationary environments with state-space models. In International conference on machine learning, pp. 2901–2910. PMLR, 2019.
  19. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research, 11(5), 2010.
  20. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
  21. Could a neuroscientist understand a microprocessor? PLoS computational biology, 13(1):e1005268, 2017.
  22. Learning to induce causal structure. arXiv preprint arXiv:2204.04875, 2022.
  23. Economy statistical recurrent units for inferring nonlinear granger causality. arXiv preprint arXiv:1911.09879, 2019.
  24. Yoon Kim. Convolutional neural networks for sentence classification. CoRR, abs/1408.5882, 2014. URL http://arxiv.org/abs/1408.5882.
  25. Temporal convolutional networks: A unified approach to action segmentation. In European conference on computer vision, pp.  47–54. Springer, 2016.
  26. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp.  2980–2988, 2017.
  27. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Conference on Causal Learning and Reasoning, pp. 509–525. PMLR, 2022.
  28. Grouped graphical granger modeling methods for temporal causal modeling. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.  577–586, 2009.
  29. Surpassing human-level face verification performance on lfw with gaussianface. In Twenty-ninth AAAI conference on artificial intelligence, 2015.
  30. Helmut Lütkepohl. New introduction to multiple time series analysis. Springer Science & Business Media, 2005.
  31. Quasi-experimental causality in neuroscience and behavioural research. Nature human behaviour, 2(12):891–898, 2018.
  32. Methods for causal inference from gene perturbation experiments and validation. Proceedings of the National Academy of Sciences, 113(27):7361–7368, 2016.
  33. Causal inference for time series analysis: Problems, methods and evaluation. Knowledge and Information Systems, 63:3041–3085, 2021.
  34. Causal discovery with attention-based convolutional neural networks. Machine Learning and Knowledge Extraction, 1(1):312–340, 2019.
  35. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  36. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics, pp.  1595–1605. PMLR, 2020.
  37. Tomáš Paus. Inferring causality in brain images: a perturbation approach. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1457):1109–1114, 2005.
  38. Judea Pearl. Causal inference. Causality: objectives and assessment, pp.  39–58, 2010.
  39. Causal inference on time series using restricted structural equation models. Advances in neural information processing systems, 26, 2013.
  40. Towards a rigorous assessment of systems biology models: the dream3 challenges. PloS one, 5(2):e9202, 2010.
  41. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10684–10695, 2022.
  42. Backshift: Learning causal cyclic graphs from unknown shift interventions. Advances in Neural Information Processing Systems, 28, 2015.
  43. Jakob Runge. Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets. In Conference on Uncertainty in Artificial Intelligence, pp. 1388–1397. PMLR, 2020.
  44. Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances, 5(11):eaau4996, 2019.
  45. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  46. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR, abs/1610.02391, 2016. URL http://arxiv.org/abs/1610.02391.
  47. Effects of face-to-face restorative justice on victims of crime in four randomized, controlled trials. Journal of experimental criminology, 1(3):367–395, 2005.
  48. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
  49. Discovering graphical granger causality using the truncating lasso penalty. Bioinformatics, 26(18):i517–i523, 2010.
  50. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016.
  51. Network modelling methods for fmri. Neuroimage, 54(2):875–891, 2011.
  52. Michael E Sobel. Causal inference in the social and behavioral sciences. In Handbook of statistical modeling for the social and behavioral sciences, pp.  1–38. Springer, 1995.
  53. Constructing bayesian network models of gene expression networks from microarray data. 2000a.
  54. Causation, prediction, and search. MIT press, 2000b.
  55. Causation, prediction, and search. MIT press, 2000c.
  56. Richard Sutton. The bitter lesson. Incomplete Ideas (blog), 13:12, 2019.
  57. Neural granger causality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4267–4279, 2021.
  58. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  59. Transfer entropy—a model-free measure of effective connectivity for the neurosciences. Journal of computational neuroscience, 30:45–67, 2011.
  60. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019.
  61. The blessings of multiple causes. Journal of the American Statistical Association, 114(528):1574–1596, 2019.
  62. Using fluctuation analysis to establish causal relations between cellular events without experimental perturbation. Biophysical journal, 107(11):2492–2498, 2014.
  63. Discovering nonlinear relations with minimum predictive information regularization. arXiv preprint arXiv:2001.01885, 2020.
  64. Distinguishing causes from effects using nonlinear acyclic causal models. In Causality: Objectives and Assessment, pp.  157–164. PMLR, 2010.
  65. Causal inference in randomized clinical trials. Bone marrow transplantation, 55(1):4–8, 2020.
  66. Dags with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
  67. Causal discovery with reinforcement learning. CoRR, abs/1906.04477, 2019. URL http://arxiv.org/abs/1906.04477.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now