Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

KeNet:Knowledge-enhanced Doc-Label Attention Network for Multi-label text classification (2403.01767v1)

Published 4 Mar 2024 in cs.CL

Abstract: Multi-Label Text Classification (MLTC) is a fundamental task in the field of NLP that involves the assignment of multiple labels to a given text. MLTC has gained significant importance and has been widely applied in various domains such as topic recognition, recommendation systems, sentiment analysis, and information retrieval. However, traditional machine learning and Deep neural network have not yet addressed certain issues, such as the fact that some documents are brief but have a large number of labels and how to establish relationships between the labels. It is imperative to additionally acknowledge that the significance of knowledge is substantiated in the realm of MLTC. To address this issue, we provide a novel approach known as Knowledge-enhanced Doc-Label Attention Network (KeNet). Specifically, we design an Attention Network that incorporates external knowledge, label embedding, and a comprehensive attention mechanism. In contrast to conventional methods, we use comprehensive representation of documents, knowledge and labels to predict all labels for each single text. Our approach has been validated by comprehensive research conducted on three multi-label datasets. Experimental results demonstrate that our method outperforms state-of-the-art MLTC method. Additionally, a case study is undertaken to illustrate the practical implementation of KeNet.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. “Hierarchical attention networks for document classification,” in Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, 2016, pp. 1480–1489.
  2. “Ask me anything: Dynamic memory networks for natural language processing,” in International conference on machine learning. PMLR, 2016, pp. 1378–1387.
  3. “Senticnet 3: a common and common-sense knowledge base for cognition-driven sentiment analysis,” in Proceedings of the AAAI conference on artificial intelligence, 2014, vol. 28.
  4. “Learning multi-label scene classification,” Pattern recognition, vol. 37, no. 9, pp. 1757–1771, 2004.
  5. “Classifier chains for multi-label classification,” Machine learning, vol. 85, pp. 333–359, 2011.
  6. “Multi-label classification: An overview,” International Journal of Data Warehousing and Mining (IJDWM), vol. 3, no. 3, pp. 1–13, 2007.
  7. “Knowledge discovery in multi-label phenotype data,” in European conference on principles of data mining and knowledge discovery. Springer, 2001, pp. 42–53.
  8. “A kernel method for multi-labelled classification,” Advances in neural information processing systems, vol. 14, 2001.
  9. “Initializing neural networks for hierarchical multi-label text classification,” in BioNLP 2017, 2017, pp. 307–315.
  10. “Multi-label text classification using attention-based graph neural network,” arXiv preprint arXiv:2003.11644, 2020.
  11. “Multi-relation message passing for multi-label text classification,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 3583–3587.
  12. “Cnn-bilstm-attention: A multi-label neural classifier for short texts with a small set of labels,” Information Processing & Management, vol. 60, no. 3, pp. 103320, 2023.
  13. “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.
  14. “Sgm: sequence generation model for multi-label classification,” arXiv preprint arXiv:1806.04822, 2018.
  15. “Attentionxml: Label tree-based attention-aware deep model for high-performance extreme multi-label text classification,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  16. “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  17. “An r-transformer_bilstm model based on attention for multi-label text classification,” Neural Processing Letters, vol. 55, no. 2, pp. 1293–1316, 2023.
  18. “Co-attention network with label embedding for text classification,” Neurocomputing, vol. 471, pp. 61–69, 2022.
  19. “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  20. “Knowledge-enhanced document embeddings for text classification,” Knowledge-Based Systems, vol. 163, pp. 955–971, 2019.
  21. SK Hong and Tae Young Jang, “Lea: meta knowledge-driven self-attentive document embedding for few-shot text classification,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022, pp. 99–106.
  22. “Tagme: on-the-fly annotation of short text fragments (by wikipedia entities),” in Proceedings of the 19th ACM international conference on Information and knowledge management, 2010, pp. 1625–1628.
  23. Yoon Kim, “Convolutional neural networks for sentence classification,” arXiv preprint arXiv:1408.5882, 2014.
  24. “Ensemble application of convolutional and recurrent neural networks for multi-label text categorization,” in 2017 International joint conference on neural networks (IJCNN). IEEE, 2017, pp. 2377–2383.
  25. “A novel reasoning mechanism for multi-label text classification,” Information Processing & Management, vol. 58, no. 2, pp. 102441, 2021.
  26. “Sequence to sequence learning with neural networks,” Advances in neural information processing systems, vol. 27, 2014.
  27. “Semantic-unit-based dilated convolution for multi-label text classification,” arXiv preprint arXiv:1808.08561, 2018.
  28. “A hybrid bert model that incorporates label semantics via adjustive attention for multi-label text classification,” Ieee Access, vol. 8, pp. 152183–152192, 2020.
  29. “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543.
  30. “Ml-knn: A lazy learning approach to multi-label learning,” Pattern recognition, vol. 40, no. 7, pp. 2038–2048, 2007.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Bo Li (1107 papers)
  2. Yuyan Chen (20 papers)
  3. Liang Zeng (31 papers)