2000 character limit reached
CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning (2401.16974v1)
Published 30 Jan 2024 in cs.LG and cs.AI
Abstract: Causal discovery is the challenging task of inferring causal structure from data. Motivated by Pearl's Causal Hierarchy (PCH), which tells us that passive observations alone are not enough to distinguish correlation from causation, there has been a recent push to incorporate interventions into machine learning research. Reinforcement learning provides a convenient framework for such an active approach to learning. This paper presents CORE, a deep reinforcement learning-based approach for causal discovery and intervention planning. CORE learns to sequentially reconstruct causal graphs from data while learning to perform informative interventions. Our results demonstrate that CORE generalizes to unseen graphs and efficiently uncovers causal structures. Furthermore, CORE scales to larger graphs with up to 10 variables and outperforms existing approaches in structure estimation accuracy and sample efficiency. All relevant code and supplementary material can be found at https://github.com/sa-and/CORE
- Active learning of causal structures with deep reinforcement learning. Neural Networks 154 (10 2022), 22–30. https://doi.org/10.1016/J.NEUNET.2022.06.028
- A Medium-Scale Distributed System for Computer Science Research: Infrastructure for the Long Term. Computer 49, 5 (5 2016), 54–63. https://doi.org/10.1109/MC.2016.127
- On Pearl’s Hierarchy and the Foundations of Causal Inference. In Probabilistic and Causal Inference. ACM, New York, NY, USA, 507–556. https://doi.org/10.1145/3501714.3501743
- GFlowNet Foundations. Journal of Machine Learning Research 24, 210 (2023), 1–55. http://jmlr.org/papers/v24/22-0364.html
- Differentiable Causal Discovery from Interventional Data. In Advances in Neural Information Processing Systems, H Larochelle, M Ranzato, R Hadsell, M F Balcan, and H Lin (Eds.), Vol. 33. Curran Associates, Inc., 21865–21877. https://proceedings.neurips.cc/paper_files/paper/2020/file/f8b7aa3a0d349d9562b424160ad18612-Paper.pdf
- David Maxwell Chickering. 2003. Optimal Structure Identification with Greedy Search. J. Mach. Learn. Res. 3, null (3 2003), 507–554. https://doi.org/10.1162/153244303321897717
- Bayesian structure learning with generative flow networks. In Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence (Proceedings of Machine Learning Research, Vol. 180), James Cussens and Kun Zhang (Eds.). PMLR, 518–528. https://proceedings.mlr.press/v180/deleu22a.html
- P. Erdós and A. Rényi. 1959. On random graphs. Publicationes Mathematicae (1959), 290–297.
- Factored Adaptation for Non-Stationary Reinforcement Learning. In Advances in Neural Information Processing Systems, S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho, and A Oh (Eds.), Vol. 35. Curran Associates, Inc., 31957–31971. https://proceedings.neurips.cc/paper_files/paper/2022/file/cf4356f994917177213c55ff438ddf71-Paper-Conference.pdf
- Review of Causal Discovery Methods Based on Graphical Models. Frontiers in Genetics 10 (6 2019). https://doi.org/10.3389/fgene.2019.00524
- Uzma Hasan and Md Osman Gani. 2022. Kcrl: A prior knowledge based causal discovery framework with reinforcement learning. In Machine Learning for Healthcare Conference. 691–714.
- Alain Hauser and Buhlmann@stat Math Ethz Ch. 2012. Characterization and Greedy Learning of Interventional Markov Equivalence Classes of Directed Acyclic Graphs Peter B ¨ uhlmann. Journal of Machine Learning Research 13 (2012), 2409–2464.
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (11 1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
- Constraint-based Causal Discovery: Conflict Resolution with Answer Set Programming. In Conference on Uncertainty in Artificial Intelligence. 340–349.
- Calculus on MDPs: Potential Shaping as a Gradient. (8 2022). https://arxiv.org/abs/2208.09570v2
- Causal Discovery for Modular World Models. In NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI). https://openreview.net/forum?id=VfkjQzdGCH
- GFlowCausal: Generative Flow Networks for Causal Discovery. (2022). http://arxiv.org/abs/2210.08185
- Efficient Neural Causal Discovery without Acyclicity Constraints. In International Conference on Learning Representations.
- Amortized Inference for Causal Structure Learning. In Advances in Neural Information Processing Systems, S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho, and A Oh (Eds.), Vol. 35. Curran Associates, Inc., 13104–13118. https://proceedings.neurips.cc/paper_files/paper/2022/file/54f7125dee9b8b3dc798bb9a082b09e2-Paper-Conference.pdf
- Predicting causal effects in large-scale systems from observational data. Nature Methods 2010 7:4 7, 4 (4 2010), 247–248. https://doi.org/10.1038/nmeth0410-247
- Causal Discovery and Reinforcement Learning: A Synergistic Integration. In Proceedings of The 11th International Conference on Probabilistic Graphical Models (Proceedings of Machine Learning Research, Vol. 186), Antonio Salmerón and Rafael Rumi (Eds.). PMLR, 421–432. https://proceedings.mlr.press/v186/mendez-molina22a.html
- Human-level control through deep reinforcement learning. Nature 518 (2015). https://doi.org/10.1038/nature14236
- Joint Causal Inference from Multiple Contexts. Journal of Machine Learning Research 21 (2020), 1–108. http://jmlr.org/papers/v21/17-123.html.
- Causal Induction from Visual Observations for Goal Directed Tasks. (2019).
- Policy invariance under reward transformations: Theory and application to reward shaping. ICML (1999), 278–287.
- Judea Pearl. 2009. Introduction to Probabilities, Graphs, and Causal Models. In Causality. Cambridge University Press, 1–40. https://doi.org/10.1017/CBO9780511803161.003
- Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data. Science 308, 5721 (4 2005), 523–529. https://doi.org/10.1126/science.1105809
- A Meta-Reinforcement Learning Algorithm for Causal Discovery. In Proceedings of the Second Conference on Causal Learning and Reasoning (Proceedings of Machine Learning Research, Vol. 213), Mihaela van der Schaar, Cheng Zhang, and Dominik Janzing (Eds.). PMLR, 602–619. https://proceedings.mlr.press/v213/sauter23a.html
- Learning Neural Causal Models with Active Interventions. (2022).
- Learning Causal Graphs with Small Interventions. In Advances in Neural Information Processing Systems, C Cortes, N Lawrence, D Lee, M Sugiyama, and R Garnett (Eds.), Vol. 28. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2015/file/b865367fc4c0845c0682bd466e6ebf4c-Paper.pdf
- Causation, prediction, and search. MIT press.
- R.S. Sutton and A.G. Barto. 2018. Reinforcement learning: An introduction. MIT press.
- Interventions, Where and How? Experimental Design for Causal Models at Scale. Advances in Neural Information Processing Systems 35 (12 2022), 24130–24143. https://github.com/yannadani/cbed
- Meta learning for causal direction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 9897–9905.
- D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery. Comput. Surveys 55, 4 (4 2023), 1–36. https://doi.org/10.1145/3527154
- Ordering-Based Causal Discovery with Reinforcement Learning. (2021). https://arxiv.org/abs/2105.
- Christopher J C H Watkins and Peter Dayan. 1992. Q-Learning. 8 (1992), 279–292.
- DAG-GNN: DAG Structure Learning with Graph Neural Networks. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 7154–7163. https://proceedings.mlr.press/v97/yu19a.html
- DAGs with NO TEARS: Continuous Optimization for Structure Learning. (2018). https://github.com/xunzheng/notears.
- Causal Discovery with Reinforcement Learning. (2019). http://arxiv.org/abs/1906.04477
- Matjaz Zwitter and Milan Soklic. 1988. Breast Cancer. UCI Machine Learning Repository.
- Andreas W. M. Sauter (2 papers)
- Nicolò Botteghi (19 papers)
- Erman Acar (27 papers)
- Aske Plaat (76 papers)