Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition (2405.07220v1)

Published 12 May 2024 in cs.LG, cs.AI, and stat.ML

Abstract: Conditional independence provides a way to understand causal relationships among the variables of interest. An underlying system may exhibit more fine-grained causal relationships especially between a variable and its parents, which will be called the local independence relationships. One of the most widely studied local relationships is Context-Specific Independence (CSI), which holds in a specific assignment of conditioned variables. However, its applicability is often limited since it does not allow continuous variables: data conditioned to the specific value of a continuous variable contains few instances, if not none, making it infeasible to test independence. In this work, we define and characterize the local independence relationship that holds in a specific set of joint assignments of parental variables, which we call context-set specific independence (CSSI). We then provide a canonical representation of CSSI and prove its fundamental properties. Based on our theoretical findings, we cast the problem of discovering multiple CSSI relationships in a system as finding a partition of the joint outcome space. Finally, we propose a novel method, coined neural contextual decomposition (NCD), which learns such partition by imposing each set to induce CSSI via modeling a conditional distribution. We empirically demonstrate that the proposed method successfully discovers the ground truth local independence relationships in both synthetic dataset and complex system reflecting the real-world physical dynamics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Mostly harmless econometrics. Princeton university press, 2008.
  2. Identification of causal effects using instrumental variables. Journal of the American statistical Association, 91(434):444–455, 1996.
  3. The influence of randomized controlled trials on development economics research and on development policy. The State of Economics, The State of the World, pages 482–488, 2016.
  4. Context-specific independence in bayesian networks. CoRR, abs/1302.3562, 2013.
  5. Differentiable causal discovery from interventional data. arXiv preprint arXiv:2007.01754, 2020.
  6. Causal learning with sufficient statistics: an information bottleneck approach. arXiv preprint arXiv:2010.05375, 2020.
  7. David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov):507–554, 2002.
  8. Parallel probabilistic inference by weighted model counting. In International Conference on Probabilistic Graphical Models, pages 97–108. PMLR, 2018.
  9. Review of causal discovery methods based on graphical models. Frontiers in genetics, 10:524, 2019.
  10. Exploiting logical structure in lifted probabilistic inference. In Workshops at the Twenty-Fourth AAAI Conference on Artificial Intelligence, 2010.
  11. Nonlinear causal discovery with additive noise models. In Proceedings of the 21st International Conference on Neural Information Processing Systems, pages 689–696, 2008.
  12. Guido W Imbens. Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics. Journal of Economic Literature, 58(4):1129–79, 2020.
  13. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
  14. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.
  15. Structural agnostic modeling: Adversarial learning of causal graphs. Journal of Machine Learning Research, 23(219):1–62, 2022.
  16. Gradient-based neural dag learning. In International Conference on Learning Representations, 2020.
  17. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712, 2016.
  18. Masked gradient-based causal structure learning. arXiv preprint arXiv:1910.08527, 2019.
  19. Stratified gaussian graphical models. Communications in Statistics-Theory and Methods, 46(11):5556–5578, 2017.
  20. Judea Pearl. Causality. Cambridge university press, 2009.
  21. Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models. Data mining and knowledge discovery, 29(2):503–533, 2015.
  22. The role of local partial independence in learning of bayesian networks. International journal of approximate reasoning, 69:91–105, 2016.
  23. Jonas Peters. On the intersection property of conditional independence and its application to causal discovery. Journal of Causal Inference, 3(1):97–108, 2015.
  24. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
  25. Counterfactual data augmentation using locally factored dynamics. Advances in Neural Information Processing Systems, 33, 2020.
  26. David Poole. Context-specific approximation in probabilistic inference. In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, pages 447–454, 1998.
  27. Exploiting contextual independence in probabilistic inference. Journal of Artificial Intelligence Research, 18:263–313, 2003.
  28. An interventionist approach to mediation analysis. arXiv preprint arXiv:2008.06019, 2020.
  29. Causal influence detection for improving efficiency in reinforcement learning. Advances in Neural Information Processing Systems, 34:22905–22918, 2021.
  30. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
  31. Bill Shipley. Cause and correlation in biology: a user’s guide to path analysis, structural equations and causal inference with R. Cambridge University Press, 2016.
  32. Michael E Sobel. Causal inference in the social and behavioral sciences. In Handbook of statistical modeling for the social and behavioral sciences, pages 1–38. Springer, 1995.
  33. Constructing Bayesian network models of gene expression networks from microarray data. 2000.
  34. Identifying causal effects via context-specific independence relations. Advances in Neural Information Processing Systems, 32:2804–2814, 2019.
  35. Lifted probabilistic inference by first-order knowledge compilation. In Proceedings of the Twenty-Second international joint conference on Artificial Intelligence, pages 2178–2185, 2011.
  36. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.
  37. Permutation-based causal inference algorithms with interventions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 5824–5833, 2017.
  38. Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration. arXiv preprint arXiv:1905.09275, 2019.
  39. Dag-gnn: Dag structure learning with graph neural networks. In International Conference on Machine Learning, pages 7154–7163. PMLR, 2019.
  40. Kernel-based conditional independence test and application in causal discovery. In 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), pages 804–813. AUAI Press, 2011.
  41. DAGs with NO TEARS: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
  42. Causal discovery with reinforcement learning. In International Conference on Learning Representations, 2019.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets