Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FoLDTree: A ULDA-Based Decision Tree Framework for Efficient Oblique Splits and Feature Selection (2410.23147v2)

Published 30 Oct 2024 in cs.LG, stat.ME, and stat.ML

Abstract: Traditional decision trees are limited by axis-orthogonal splits, which can perform poorly when true decision boundaries are oblique. While oblique decision tree methods address this limitation, they often face high computational costs, difficulties with multi-class classification, and a lack of effective feature selection. In this paper, we introduce LDATree and FoLDTree, two novel frameworks that integrate Uncorrelated Linear Discriminant Analysis (ULDA) and Forward ULDA into a decision tree structure. These methods enable efficient oblique splits, handle missing values, support feature selection, and provide both class labels and probabilities as model outputs. Through evaluations on simulated and real-world datasets, LDATree and FoLDTree consistently outperform axis-orthogonal and other oblique decision tree methods, achieving accuracy levels comparable to the random forest. The results highlight the potential of these frameworks as robust alternatives to traditional single-tree methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Classification and Regression Trees. 1984.
  2. Jerome H Friedman. Regularized discriminant analysis. Journal of the American Statistical Association, 84(405):165–175, 1989.
  3. Linear tree. Intelligent Data Analysis, 3(1):1–22, 1999.
  4. Extremely randomized trees. Machine learning, 63:3–42, 2006.
  5. Induction of oblique decision trees. In IJCAI, volume 1993, pages 1002–1007. Citeseer, 1993.
  6. Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 25(1):165–179, 2003.
  7. George H John. Robust linear discriminant trees. In Pre-proceedings of the Fifth International Workshop on artificial intelligence and statistics, pages 285–291. PMLR, 1995.
  8. The UCI machine learning repository. https://archive.ics.uci.edu, 2024.
  9. A projection pursuit index for large p small n data. Statistics and Computing, 20:381–392, 2010.
  10. Projection pursuit for exploratory supervised classification. Journal of Computational and graphical Statistics, 14(4):831–846, 2005.
  11. Pptree: Projection pursuit classification tree. 2013.
  12. Multivariate decision trees using linear discriminants and tabu search. IEEE transactions on systems, man, and cybernetics-part a: systems and humans, 33(2):194–205, 2003.
  13. Wei-Yin Loh. Improving the precision of classification trees. The Annals of Applied Statistics, pages 1710–1737, 2009.
  14. Split selection methods for classification trees. Statistica sinica, pages 815–840, 1997.
  15. Tree-structured classification via generalized discriminant analysis. Journal of the American Statistical Association, 83(403):715–725, 1988.
  16. Fisher’s decision tree. Expert Systems with Applications, 40(16):6283–6291, 2013.
  17. Qing Mai. A review of discriminant analysis in high dimensions. Wiley Interdisciplinary Reviews: Computational Statistics, 5(3):190–197, 2013.
  18. Naresh Manwani and PS Sastry. Geometric decision tree. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(1):181–192, 2011.
  19. Stree: A single multi-class oblique decision tree based on support vector machines. In Conference of the Spanish Association for Artificial Intelligence, pages 54–64. Springer, 2021.
  20. A system for induction of oblique decision trees. Journal of artificial intelligence research, 2:1–32, 1994.
  21. J. Ross Quinlan. Simplifying decision trees. International journal of man-machine studies, 27(3):221–234, 1987.
  22. J Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
  23. Linear discriminant analysis: A detailed tutorial. AI communications, 30(2):169–190, 2017.
  24. R Todeschini and Emilio Marengo. Linear discriminant classification tree: a user-driven multicriteria classification method. Chemometrics and intelligent laboratory systems, 16(1):25–35, 1992.
  25. Siyu Wang. A new forward discriminant analysis framework based on Pillai’s trace and ULDA, 2024. URL https://arxiv.org/abs/2409.03136.
  26. HHCART: an oblique decision tree. Computational Statistics & Data Analysis, 96:12–23, 2016.
  27. A reflected feature space for CART. Australian & New Zealand Journal of Statistics, 61(3):380–391, 2019.
  28. Penalized classification using Fisher’s linear discriminant. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(5):753–772, 2011.
  29. Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems. Journal of Machine Learning Research, 6(4), 2005.
  30. Feature extraction via generalized uncorrelated linear discriminant analysis. In Proceedings of the twenty-first international conference on Machine learning, page 113, 2004.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets