2000 character limit reached
Logifold: A Geometrical Foundation of Ensemble Machine Learning (2407.16177v2)
Published 23 Jul 2024 in cs.LG and math.DG
Abstract: We present a local-to-global and measure-theoretical approach to understanding datasets. The core idea is to formulate a logifold structure and to interpret network models with restricted domains as local charts of datasets. In particular, this provides a mathematical foundation for ensemble machine learning. Our experiments demonstrate that logifolds can be implemented to identify fuzzy domains and improve accuracy compared to taking average of model outputs. Additionally, we provide a theoretical example of a logifold, highlighting the importance of restricting to domains of classifiers in an ensemble.
- I. Jung, and S. C. Lau, “A logifold structure on measure space,” arXiv:2405.05492, unpublished.
- Y. Li, M. Cheng, C. J. Hsieh, T. C. M. Lee, “A Review of Adversarial Attack and Defense for Classification Methods,” The American Statistician, 76(4), 329–345, 2022
- E.V. Burnaev, A.V. Bernstein, “Manifold Modeling in Machine Learning,” J. Commun. Technol. Electron. 66, 754–763 (2021), https://doi.org/10.1134/S106422692106005X
- G. Singh, F. Memoli, G. Carlsson, “Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition,” In M. Botsch, R. Pajarola, B. Chen, and M. Zwicker, editors, Eurographics Symposium on Point-Based Graphics, The Eurographics Association, 2007.
- I. Jolliffe, “Principal Component Analysis,” pages 1094–1096, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.
- B. Antonio, D. Moroni, M. Martinelli, “Efficient adaptive ensembling for image classification,” Expert Systems, August 2023.
- Y. M. C. Zhang(editor) “Ensemble Machine Learning Methods and Applications,” Springer New York, NY, the first edition, 2012.
- K.M. Ali, M.J. Pazzani, “Error reduction through learning multiple descriptions,” Mach Learn 24, 173–202 (1996)
- J. Mendes-Moreira, T. Mendes-Neves, “Towards a systematic approach to design new ensemble learning algorithms,” arXiv:2402.06818, unpublished.
- D. Wood, T. Mu, A. Webb, H. Reeve, M. Luján, G. Brown, “A Unified Theory of Diversity in Ensemble Learning,”, JMLR 24, 1-49, 2023
- A. Krizhevsky, G. Hinton, “Learning multiple layers of features from tiny images,” Technical Report 0, University of Toronto, Toronto, Ontario, 2009.
- L. Deng, “The MNIST database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Processing Magazine, 29(6):141–142, 2012.
- H. Xiao, K. Rasul, R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” 2017.
- S. Milinski, N. Maher, D. Olonscheck, “How large does a large ensemble need to be?,” Earth System Dynamics, 11(4):885–901, 2020.
- P. Sollich, A. Krogh, “Learning with ensembles: How overfitting can be useful,” In D. Touretzky, M.C. Mozer, and M. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8. MIT Press, 1995.
- H. R. Bonab, F. Can, “A theoretical framework on the ideal number of classifiers for online ensembles in data streams,” In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, page 2053–2056, New York, NY, USA, 2016, Association for Computing Machinery.
- K. Hornik, M. Stinchcombe, H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, 2(5):359–366, 1989.
- K. He, X. Zhang, S. Ren, J. Sun, “Deep residual learning for image recognition,” CoRR, abs/1512.03385, 2015.
- D. Kingma, J. Ba, “Adam: A method for stochastic optimization,” In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.