Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Dual PC Algorithm and the Role of Gaussianity for Structure Learning of Bayesian Networks (2112.09036v6)

Published 16 Dec 2021 in stat.ML, cs.LG, and stat.CO

Abstract: Learning the graphical structure of Bayesian networks is key to describing data-generating mechanisms in many complex applications but poses considerable computational challenges. Observational data can only identify the equivalence class of the directed acyclic graph underlying a Bayesian network model, and a variety of methods exist to tackle the problem. Under certain assumptions, the popular PC algorithm can consistently recover the correct equivalence class by reverse-engineering the conditional independence (CI) relationships holding in the variable distribution. The dual PC algorithm is a novel scheme to carry out the CI tests within the PC algorithm by leveraging the inverse relationship between covariance and precision matrices. By exploiting block matrix inversions we can also perform tests on partial correlations of complementary (or dual) conditioning sets. The multiple CI tests of the dual PC algorithm proceed by first considering marginal and full-order CI relationships and progressively moving to central-order ones. Simulation studies show that the dual PC algorithm outperforms the classic PC algorithm both in terms of run time and in recovering the underlying network structure, even in the presence of deviations from Gaussianity. Additionally, we show that the dual PC algorithm applies for Gaussian copula models, and demonstrate its performance in that setting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. A characterization of Markov equivalence classes for acyclic digraphs. The Annals of Statistics, 25, 505–541.
  2. Bayesian networks elucidate complex genomic landscapes in cancer. Communications Biology, 5, 1–12.
  3. Partial correlation and conditional correlation as measures of conditional independence. Australian & New Zealand Journal of Statistics, 46, 657–664.
  4. Computational inference of gene regulatory networks: approaches, limitations and opportunities. Biochimica et Biophysica Acta (BBA) - Gene Regulatory Mechanisms, 1860, 41–52.
  5. Adolescent paranoia: prevalence, structure, and causal mechanisms. Schizophrenia Bulletin, 45(5), 1134–1142.
  6. Nonparametric causal structure learning in high dimensions. arXiv:2106.11415.
  7. Bayesian network based extreme learning machine for subjectivity detection. Journal of the Franklin Institute, 355(4), 1780–1797. Special issue on recent advances in machine learning for signal analysis and processing.
  8. Chickering, D. M. (2003). Optimal structure identification with greedy search. Journal of Machine Learning Research, 3, 507–554.
  9. Large-sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research, 5, 1287–1330.
  10. Order-independent constraint-based causal structure learning. Journal of Machine Learning Research, 15(116), 3921–3962.
  11. Large-scale empirical validation of Bayesian network structure learning algorithms with noisy data. International Journal of Approximate Reasoning, 131, 151–188.
  12. Testing multivariate normality. Biometrika, 65, 263–272.
  13. Copula PC algorithm for causal discovery from mixed data. In P. Frasconi, N. Landwehr, G. Manco, & J. Vreeken (Eds.), Machine Learning and Knowledge Discovery in Databases (pp. 377–392). Cham: Springer International Publishing.
  14. Bayesian network models for hierarchical text classification from a thesaurus. International Journal of Approximate Reasoning, 50(7), 932 – 944. Special section on graphical models and information retrieval.
  15. A fast PC algorithm for high dimensional causal discovery with multi-core PCs. arXiv:1502.02454.
  16. Elwert, F. (2013). Graphical causal models, (pp. 245–273). Springer Netherlands: Dordrecht.
  17. Friedman, N. (2004). Inferring cellular networks using probabilistic graphical models. Science, 303(5659), 799–805.
  18. Using Bayesian networks to analyze expression data. Journal of Computational Biology, 7(3-4), 601–620. PMID: 11108481.
  19. Review of causal discovery methods based on graphical models. Frontiers in Genetics, 10, 524.
  20. Hawkins, D. (1989). Using U statistics to derive the asymptotic distribution of Fisher’s Z statistic. The American Statistician, 43, 235–237.
  21. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. Journal of Machine Learning Research, 8, 613–636.
  22. Robustification of the PC-algorithm for directed acyclic graphs. Journal of Computational and Graphical Statistics, 17(4), 773–789.
  23. Causal inference using graphical models with the R package pcalg. Journal of Statistical Software, Articles, 47(11), 1–26.
  24. Characterizations of multivariate normality. i. through independence of some statistics. Journal of Multivariate Analysis, 6, 81–94.
  25. Probabilistic graphical models: principles and techniques - adaptive computation and machine learning. The MIT Press.
  26. Efficient sampling and structure learning of Bayesian networks. Journal of Computational and Graphical Statistics, 31, 639–650.
  27. Mutational interactions define novel cancer subgroups. Nature Communications, 9.
  28. Lauritzen, S. L. (1996). Graphical models. Oxford University Press.
  29. High-dimensional semiparametric Gaussian copula graphical models. The Annals of Statistics, 40, 2293–2326.
  30. The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10, 2295–2328.
  31. Estimating high-dimensional intervention effects from observational data. The Annals of Statistics, 37(6A), 3133 – 3164.
  32. Meek, C. (1995). Causal inference and causal explanation with background knowledge. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, UAI’95 (pp. 403–410). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.
  33. Using directed acyclic graphs in epidemiological research in psychosis: an analysis of the role of bullying in psychosis. Schizophrenia Bulletin, 43(6), 1273–1279.
  34. Longitudinal symptomatic interactions in long-standing schizophrenia: a novel five-point analysis based on directed acyclic graphs. Psychological Medicine, (pp. 1–8).
  35. Musella, F. (2013). A PC algorithm variation for ordinal variables. Computational Statistics, 28(6), 2749–2759.
  36. Bayesian network analysis of covid-19 data reveals higher infection prevalence rates and lower fatality rates than widely reported. Journal of Risk Research, 23(7-8), 866–879.
  37. Nelson, L. S. (1991). A dictionary of statistical terms, 5th ed. Journal of Quality Technology, 23(2), 167–168.
  38. Bayesian network modelling for supply chain risk propagation. International Journal of Production Research, 56(17), 5795–5819.
  39. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.
  40. Pearl, J. (2009). Causality: models, reasoning and inference. USA: Cambridge University Press, 2nd edition.
  41. Conditional independence and its representations. Kybernetika, 25(7), 33–44.
  42. Bayesian networks. Technical report (R-277). UCLA Cognitive Systems Laboratory.
  43. Benchpress: a scalable and platform-independent workflow for benchmarking structure learning algorithms for graphical models. arXiv:2107.03863.
  44. Robinson, R. W. (1977). Counting unlabeled acyclic digraphs. In C. H. C. Little (Ed.), Combinatorial Mathematics V (pp. 28–43).: Springer Berlin Heidelberg.
  45. Scalable techniques for mining causal structures. Data Mining and Knowledge Discovery, 4, 163–192.
  46. The reduced PC-algorithm: improved causal structure learning in large random networks. Journal of Machine Learning Research, 20(164), 1–31.
  47. Causation, prediction, and search, volume 81. Springer New York, NY.
  48. The max-min hill-climbing Bayesian network structure learning algorithm. Machine Learning, 65, 31–78.
  49. Causal networks: semantics and expressiveness. In Proceedings of the Fourth Annual Conference on Uncertainty in Artificial Intelligence, UAI ’88 (pp. 69–78). NLD: North-Holland Publishing Co.
  50. Equivalence and synthesis of causal models. In Proceedings of the Sixth Annual Conference on Uncertainty in Artificial Intelligence, UAI ’90 (pp. 255–270). USA: Elsevier Science Inc.
  51. Towards scalable Bayesian learning of causal DAGs. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems, volume 33 (pp. 6584–6594).: Curran Associates, Inc.
  52. Inferring gene regulatory networks from gene expression data by path consistency algorithm based on conditional mutual information. Bioinformatics, 28(1), 98–104.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now