Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
92 tokens/sec
Gemini 2.5 Pro Premium
50 tokens/sec
GPT-5 Medium
22 tokens/sec
GPT-5 High Premium
21 tokens/sec
GPT-4o
97 tokens/sec
DeepSeek R1 via Azure Premium
87 tokens/sec
GPT OSS 120B via Groq Premium
459 tokens/sec
Kimi K2 via Groq Premium
230 tokens/sec
2000 character limit reached

Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification (2401.07395v1)

Published 15 Jan 2024 in cs.LG and cs.AI

Abstract: Within the scope of natural language processing, the domain of multi-label text classification is uniquely challenging due to its expansive and uneven label distribution. The complexity deepens due to the demand for an extensive set of annotated data for training an advanced deep learning model, especially in specialized fields where the labeling task can be labor-intensive and often requires domain-specific knowledge. Addressing these challenges, our study introduces a novel deep active learning strategy, capitalizing on the Beta family of proper scoring rules within the Expected Loss Reduction framework. It computes the expected increase in scores using the Beta Scoring Rules, which are then transformed into sample vector representations. These vector representations guide the diverse selection of informative samples, directly linking this process to the model's expected proper score. Comprehensive evaluations across both synthetic and real datasets reveal our method's capability to often outperform established acquisition techniques in multi-label text classification, presenting encouraging outcomes across various architectural and dataset scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. A Novel Approach for Multi-Label Chest X-Ray Classification of Common Thorax Diseases. IEEE Access, 7: 64279–64288.
  2. Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds. In Proc. 8th Int. Conf. Learn. Representations.
  3. Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications. Technical report, The Wharton School, University of Pennsylvania.
  4. PadChest: A large chest x-ray image dataset with multi-label annotated reports. Medical Image Analysis, 66: 101797.
  5. Deeplearning Model Used in Text Classification. In 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 123–126.
  6. Addressing imbalance in multilabel classification: Measures and random resampling algorithms. Neurocomputing, 163: 3–16. Recent Advancements in Hybrid Artificial Intelligence Systems and its Application to Real-World Problems Progress in Intelligent Systems Mining Humanistic Data.
  7. Multi-label Active Learning with Conditional Bernoulli Mixtures. In Geng, X.; and Kang, B.-H., eds., PRICAI 2018: Trends in Artificial Intelligence, 954–967. Cham: Springer International Publishing. ISBN 978-3-319-97304-3.
  8. Multi-label active learning: key issues and a novel query strategy. Evolving Systems, 10(1): 63–78.
  9. Theory and applications of proper scoring rules. Metron, 72(2): 169–183.
  10. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186. Minneapolis, Minnesota: Association for Computational Linguistics.
  11. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In Proc. 7th Int. Conf. Learn. Representations.
  12. Deep Bayesian Active Learning with Image Data. In Proc. 34th Int. Conf. Mach. Learn., volume 70, 1183–1192.
  13. Strictly Proper Scoring Rules, Prediction, and Estimation. J. Amer. Statistical Assoc., 102(477): 359–378.
  14. Active Query Driven by Uncertainty and Diversity for Incremental Multi-label Learning. In 2013 IEEE International Conference on Data Mining (ICDM), 1079–1084. Los Alamitos, CA, USA: IEEE Computer Society.
  15. Multi-label Active Learning with Auxiliary Learner. In Hsu, C.-N.; and Lee, W. S., eds., Proceedings of the Asian Conference on Machine Learning, volume 20 of Proceedings of Machine Learning Research, 315–332. South Garden Hotels and Resorts, Taoyuan, Taiwain: PMLR.
  16. Multilabel Text Classification for Automated Tag Suggestion. In Proceedings of the ECML/PKDD 2008 Discovery Challenge.
  17. Kim, Y. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1746–1751. Doha, Qatar: Association for Computational Linguistics.
  18. BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning. In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  19. Towards Tackling Multi-Label Imbalances in Remote Sensing Imagery. In 2020 25th International Conference on Pattern Recognition (ICPR), 5782–5789.
  20. Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, 6405–6416. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781510860964.
  21. RCV1: A New Benchmark Collection for Text Categorization Research. JMLR, 5: 361–397.
  22. Active Learning with Multi-Label SVM Classification. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI ’13, 1479–1485. AAAI Press. ISBN 9781577356332.
  23. Deep Learning for Extreme Multi-Label Text Classification. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’17, 115–124. New York, NY, USA: Association for Computing Machinery. ISBN 9781450350228.
  24. Efficient Multilabel Classification Algorithms for Large-Scale Problems in the Legal Domain. In Francesconi, E.; Montemagni, S.; Peters, W.; and Tiscornia, D., eds., Semantic Processing of Legal Texts: Where the Language of Law Meets the Law of Language, 192–215. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-642-12837-0.
  25. Choosing a strictly proper scoring rule. Decision Analysis, 10(4): 292–304.
  26. Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 4063–4071. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics.
  27. AUC Maximization for Low-Resource Named Entity Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11): 13389–13399.
  28. Low-Resource Named Entity Recognition: Can One-vs-All AUC Maximization Help? arXiv:2311.04918.
  29. Entropy-based active learning for object recognition. In 2008 IEEE Computer Society Conf. Computer Vision Pattern Recognition Workshops (CVPR Workshops), 1–8. IEEE Computer Society.
  30. A Survey of Deep Active Learning. ACM Comput. Surv., 54(9).
  31. Effective active learning strategy for multi-label learning. Neurocomputing, 273: 494–508.
  32. Addressing Imbalance in Multi-Label Classification Using Weighted Cross Entropy Loss Function. In 2020 27th National and 5th International Iranian Conference on Biomedical Engineering (ICBME), 333–338.
  33. Toward Optimal Active Learning through Sampling Estimation of Error Reduction. In Proc. 18th Int. Conf. Mach. Learn., 441––448. ISBN 1558607781.
  34. Settles, B. 2009. Active Learning Literature Survey. Computer Sciences Technical Report 1648, Univ. of Wisconsin–Madison.
  35. A Gaussian Process-Bayesian Bernoulli Mixture Model for Multi-Label Active Learning. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems, volume 34, 27542–27554. Curran Associates, Inc.
  36. Fast Direct Search in an Optimally Compressed Continuous Target Space for Efficient Multi-Label Active Learning. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, 5769–5778. PMLR.
  37. Discovering recurring anomalies in text reports regarding complex space systems. In 2005 IEEE Aerospace Conference, 3853–3862.
  38. Deep convolutional neural network–based pixel-wise landslide inventory mapping. Landslides, 18(4): 1421–1443.
  39. Diversity Enhanced Active Learning with Strictly Proper Scoring Rules. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems, volume 34, 10906–10918. Curran Associates, Inc.
  40. Bayesian Estimate of Mean Proper Scores for Diversity-Enhanced Active Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–16.
  41. Effective and Efficient Multilabel Classification in Domains with Large Number of Labels. In Proceedings of the ECML/PKDD 2008 Discovery Challenge.
  42. Parametric Mixture Models for Multi-Labeled Text. In NIPS’02, 737–744. Cambridge, MA, USA: MIT Press.
  43. Imbalanced Semi-supervised Learning with Bias Adaptive Classifier. arXiv:2207.13856.
  44. Does Tail Label Help for Large-Scale Multi-Label Learning? IEEE Transactions on Neural Networks and Learning Systems, 31(7): 2315–2324.
  45. Constrained Submodular Minimization for Missing Labels and Class Imbalance in Multi-label Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
  46. Multi-Label Active Learning Algorithms for Image Classification: Overview and Future Promise. ACM Comput. Surv., 53(2).
  47. Effective Multi-Label Active Learning for Text Classification. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’09, 917–926. New York, NY, USA: Association for Computing Machinery. ISBN 9781605584959.
  48. Cost-Sensitive Reference Pair Encoding for Multi-Label Learning. In Phung, D.; Tseng, V. S.; Webb, G. I.; Ho, B.; Ganji, M.; and Rashidi, L., eds., Advances in Knowledge Discovery and Data Mining, 143–155. Cham: Springer International Publishing.
  49. Binary relevance for multi-label learning: an overview. Frontiers of Computer Science, 12(2): 191–202.
  50. Uncertainty-aware Active Learning for Optimal Bayesian Classifier. In Proc. 9th Int. Conf. Learn. Representations.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now