Context-Specific Refinements of Bayesian Network Classifiers (2405.18298v1)
Abstract: Supervised classification is one of the most ubiquitous tasks in machine learning. Generative classifiers based on Bayesian networks are often used because of their interpretability and competitive accuracy. The widely used naive and TAN classifiers are specific instances of Bayesian network classifiers with a constrained underlying graph. This paper introduces novel classes of generative classifiers extending TAN and other famous types of Bayesian network classifiers. Our approach is based on staged tree models, which extend Bayesian networks by allowing for complex, context-specific patterns of dependence. We formally study the relationship between our novel classes of classifiers and Bayesian networks. We introduce and implement data-driven learning routines for our models and investigate their accuracy in an extensive computational study. The study demonstrates that models embedding asymmetric information can enhance classification accuracy.
- C. Bielza and P. Larrañaga. Discrete Bayesian network classifiers: A survey. ACM Computing Surveys, 47(1):1–43, 2014.
- Context-specific independence in Bayesian networks. In Proceedings of the Twelfth International Conference on Uncertainty in Artificial Intelligence, pages 115–123, 1996.
- The R package stagedtrees for structural learning of stratified staged trees. Journal of Statistical Software, 102:1–30, 2022.
- A new class of generative classifiers based on staged tree models. Knowledge-Based Systems, 268:110488, 2023.
- Chain event graphs. CRC Press, 2018.
- A probabilistic theory of pattern recognition. Springer Science & Business Media, 2013.
- Algorithms for learning parsimonious context trees. Machine Learning, 108:879–911, 2019.
- Locally weighted naive Bayes. In Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence, pages 249–256, 2002.
- G. Freeman and J. Q. Smith. Bayesian MAP model selection of chain event graphs. Journal of Multivariate Analysis, 102(7):1152–1165, 2011.
- Bayesian network classifiers. Machine Learning, 29:131–163, 1997.
- D. Geiger and D. Heckerman. Knowledge representation and inference in similarity networks and Bayesian multinets. Artificial Intelligence, 82(1-2):45–74, 1996.
- The curved exponential family of a staged tree. Electronic Journal of Statistics, 16(1):2607–2620, 2022.
- Towards efficient variables ordering for Bayesian networks classifier. Data & Knowledge Engineering, 63(2):258–269, 2007.
- Learning the structure of augmented Bayesian classifiers. International Journal on Artificial Intelligence Tools, 11(04):587–601, 2002.
- M. Leonelli and G. Varando. Highly efficient structural learning of sparse staged trees. In International Conference on Probabilistic Graphical Models, pages 193–204. PMLR, 2022.
- M. Leonelli and G. Varando. Context-specific causal discovery for categorical data using staged trees. In International Conference on Artificial Intelligence and Statistics, pages 8871–8888. PMLR, 2023.
- M. Leonelli and G. Varando. Learning and interpreting asymmetry-labeled DAGs: A case study on COVID-19 fear. Applied Intelligence, 54(2):1734–1750, 2024a.
- M. Leonelli and G. Varando. Robust learning of staged tree models: Ac case study in evaluating transport services. arXiv preprint arXiv:2401.01812, 2024b.
- M. Leonelli and G. Varando. Structural learning of simple staged trees. Data Mining and Knowledge Discovery, pages 1–25, 2024c.
- M. Minsky. Steps toward artificial intelligence. Transactions of the Institute of Radio Engineers, 49:8–30, 1961.
- Maximum margin Bayesian network classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):521–532, 2011.
- Scalable structure learning for sparse context-specific causal systems. arXiv preprint arXiv:2402.07762, 2024.
- M. Sahami. Learning limited dependence Bayesian classifiers. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pages 335–338, 1996.
- M. Scutari. Learning Bayesian networks with the bnlearn R package. Journal of Statistical Software, 35:1–22, 2010.
- Conditional independence and chain event graphs. Artificial Intelligence, 172(1):42–68, 2008.
- Decision boundary for discrete Bayesian network classifiers. Journal of Machine Learning Research, 16:2725–2749, 2015.
- Staged trees and asymmetry-labeled DAGs. Metrika, pages 1–28, 2024.
- Multinomial naïve Bayesian classifier with generalized Dirichlet priors for high-dimensional imbalanced data. Knowledge-Based Systems, 228:107288, 2021.