Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ProDAG: Projection-Induced Variational Inference for Directed Acyclic Graphs (2405.15167v3)

Published 24 May 2024 in stat.ML and cs.LG

Abstract: Directed acyclic graph (DAG) learning is a rapidly expanding field of research. Though the field has witnessed remarkable advances over the past few years, it remains statistically and computationally challenging to learn a single (point estimate) DAG from data, let alone provide uncertainty quantification. Our article addresses the difficult task of quantifying graph uncertainty by developing a Bayesian variational inference framework based on novel distributions that have support directly on the space of DAGs. The distributions, which we use to form our prior and variational posterior, are induced by a projection operation, whereby an arbitrary continuous distribution is projected onto the space of sparse weighted acyclic adjacency matrices (matrix representations of DAGs) with probability mass on exact zeros. Though the projection constitutes a combinatorial optimization problem, it is solvable at scale via recently developed techniques that reformulate acyclicity as a continuous constraint. We empirically demonstrate that our method, ProDAG, can deliver accurate inference and often outperforms existing state-of-the-art alternatives.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Variational causal networks: Approximate Bayesian inference over causal structures, 2021. arXiv: 2106.07635.
  2. BayesDAG: Gradient-based posterior inference for causal discovery. In Advances in Neural Information Processing Systems, volume 36, pages 1738–1763, 7 2023a.
  3. Differentiable multi-target causal Bayesian experimental design. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 34263–34279, 2023b.
  4. DAGMA: Learning DAGs via M-matrices and a log-determinant acyclicity characterization. In Advances in Neural Information Processing Systems, volume 35, pages 8226–8239, 2022.
  5. Julia: A fresh approach to numerical computing. SIAM Review, 59:65–98, 2017.
  6. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112:859–877, 2017.
  7. Variational DAG estimation via state augmentation with stochastic permutations, 2024. arXiv: 2402.02644.
  8. Differentiable DAG sampling. In International Conference on Learning Representations, 2022.
  9. BCD Nets: Scalable variational approaches for Bayesian causal discovery. In Advances in Neural Information Processing Systems, volume 34, pages 7095–7110, 2021.
  10. Joint Bayesian inference of graphical structure and parameters with a single generative flow network. In Advances in Neural Information Processing Systems, volume 36, pages 31204–31231, 2023.
  11. Optimizing NOTEARS objectives via topological swaps. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 7563–7595, 2023.
  12. Efficient projections onto the ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-ball for learning in high dimensions. In Proceedings of the 25th International Conference on Machine Learning, pages 272–279, 2008.
  13. E Michael Foster. Causal inference and developmental psychology. Developmental Psychology, 46:1454–1480, 2010.
  14. Deep end-to-end causal inference. In NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact, 2022.
  15. Learning large DAGs by combining continuous optimization and feedback arc set heuristics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6713–6720, 2022.
  16. Guido W Imbens. Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics. Journal of Economic Literature, 58:1129–1179, 2020.
  17. Fashionable modelling with Flux. In Workshop on Systems for ML and Open Source Software at NeurIPS 2018, 2018.
  18. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
  19. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
  20. A survey of Bayesian network structure learning. Artificial Intelligence Review, 56:8721–8814, 2023.
  21. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological), 50:157–224, 1988.
  22. DiBS: Differentiable Bayesian structure learning. In Advances in Neural Information Processing Systems, volume 34, pages 24111–24123, 2021.
  23. Integer programming for learning directed acyclic graphs from continuous data. INFORMS Journal on Optimization, 3:46–73, 2021.
  24. On the role of sparsity and DAG constraints for learning linear DAGs. In Advances in Neural Information Processing Systems, volume 33, pages 17943–17954, 2020.
  25. The Bayesian lasso. Journal of the American Statistical Association, 103:681–686, 2008.
  26. Judea Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco, USA, 1988.
  27. Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, USA, 2nd edition, 2009.
  28. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308:523–529, 2005.
  29. Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: Review and recommendations. International Journal of Epidemiology, 50:620–632, 2021.
  30. Contextual directed acyclic graphs. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, volume 238, pages 2872–2880, 2024.
  31. D’ya like DAGs? A survey on structure learning and causal discovery. ACM Computing Surveys, 55:1–36, 2022.
  32. Tractable uncertainty for structure learning. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 23131–23150, 2022.
  33. Bayesian inference with the l1subscript𝑙1l_{1}italic_l start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-ball prior: Solving combinatorial problems with exact zeros. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 85:1538–1560, 2023.
  34. Bayesian inference using the proximal mapping: Uncertainty quantification under varying dimensionality. Journal of the American Statistical Association, 2023.
  35. DAGs with no curl: An efficient DAG structure learning approach. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 12156–12166, 2021.
  36. Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68:49–67, 2006.
  37. DAGs with NO TEARS: Continuous optimization for structure learning. In Advances in Neural Information Processing Systems, volume 31, 2018.
  38. Learning sparse nonparametric DAGs. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, volume 108, pages 3414–3425, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets