Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hierarchical confusion matrix for classification performance evaluation (2306.09461v1)

Published 15 Jun 2023 in cs.LG

Abstract: In this work we propose a novel concept of a hierarchical confusion matrix, opening the door for popular confusion matrix based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. We develop the concept to a generalized form and prove its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi path labelling, and non mandatory leaf node prediction. Finally, we use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications and compare the results to established evaluation measures. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems. The implementation of hierarchical confusion matrix is available on GitHub.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. Hierarchical multi-classification. In Workshop Notes of the KDD’02 Workshop on Multi-Relational Data Mining, pages 21–35, 2002.
  2. An evaluation of global-model hierarchical classification algorithms for hierarchical classification problems with single path of labels. Computers & Mathematics with Applications, 66(10):1991–2002, 2013.
  3. Incremental algorithms for hierarchical classification. Advances in neural information processing systems, 17:233–240, 2005.
  4. Hierarchical classification: combining bayes with svm. In Proceedings of the 23rd international conference on Machine learning, pages 177–184, 2006.
  5. A review of performance evaluation measures for hierarchical classifiers. In Evaluation methods for machine learning II: Papers from the AAAI-2007 workshop, pages 1–6, 2007.
  6. Hierarchical classification of web content. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 256–263, 2000.
  7. Tom Fawcett. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874, 2006.
  8. A tutorial on hierarchical classification with applications in bioinformatics. Research and trends in data mining technologies and applications, pages 175–208, 2007.
  9. Hierarchical classification of community data. The Journal of Ecology, pages 537–557, 1981.
  10. Allan D Gordon. A review of hierarchical classification. Journal of the Royal Statistical Society: Series A (General), 150(2):119–137, 1987.
  11. Hierarchical classification of g-protein-coupled receptors with a pso/aco algorithm. In Proceedings of the IEEE Swarm Intelligence Symposium (SIS’06), pages 77–84. IEEE Press, 2006.
  12. Probe, count, and classify: categorizing hidden web databases. In Proceedings of the 2001 ACM SIGMOD international conference on Management of data, pages 67–78, 2001.
  13. Hierarchical text categorization as a tool of associating genes with gene ontology codes. In European workshop on data mining and text mining in bioinformatics, pages 30–34, 2004.
  14. Functional annotation of genes using hierarchical text categorization. In Proc. of the ACL Workshop on Linking Biological Literature, Ontologies and Databases: Mining Biological Semantics, 2005.
  15. Learning and evaluation in the presence of class hierarchies: Application to text categorization. In Conference of the Canadian Society for Computational Studies of Intelligence, pages 395–406. Springer, 2006.
  16. Hierarchically classifying documents using very few words. Technical report, Stanford InfoLab, 1997.
  17. Evaluation measures for hierarchical classification: a unified view and novel approaches. Data Mining and Knowledge Discovery, 29(3):820–865, 2015.
  18. Germeval 2019 task 1: Hierarchical classification of blurbs. In KONVENS, 2019.
  19. Transposon ultimate: a bundle of tools for transposon identification. 2021 (IN PREPARATION, AVAILABLE ON BIORXIV SOON).
  20. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22(1):31–72, 2011.
  21. Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation. In Australasian joint conference on artificial intelligence, pages 1015–1021. Springer, 2006.
  22. A systematic analysis of performance measures for classification tasks. Information processing & management, 45(4):427–437, 2009.
  23. Mohammad S Sorower. A literature survey on algorithms for multi-label learning. Oregon State University, Corvallis, 18:1–25, 2010.
  24. Hierarchical text classification and evaluation. In Proceedings 2001 IEEE International Conference on Data Mining, pages 521–528. IEEE, 2001.
  25. Building hierarchical classifiers using class proximity. In VLDB, volume 99, pages 7–10. Citeseer, 1999.
Citations (18)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now