Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rules (2403.02932v2)

Published 5 Mar 2024 in cs.CL

Abstract: Weakly supervised text classification (WSTC), also called zero-shot or dataless text classification, has attracted increasing attention due to its applicability in classifying a mass of texts within the dynamic and open Web environment, since it requires only a limited set of seed words (label names) for each category instead of labeled data. With the help of recently popular prompting Pre-trained LLMs (PLMs), many studies leveraged manually crafted and/or automatically identified verbalizers to estimate the likelihood of categories, but they failed to differentiate the effects of these category-indicative words, let alone capture their correlations and realize adaptive adjustments according to the unlabeled corpus. In this paper, in order to let the PLM effectively understand each category, we at first propose a novel form of rule-based knowledge using logical expressions to characterize the meanings of categories. Then, we develop a prompting PLM-based approach named RulePrompt for the WSTC task, consisting of a rule mining module and a rule-enhanced pseudo label generation module, plus a self-supervised fine-tuning module to make the PLM align with this task. Within this framework, the inaccurate pseudo labels assigned to texts and the imprecise logical rules associated with categories mutually enhance each other in an alternative manner. That establishes a self-iterative closed loop of knowledge (rule) acquisition and utilization, with seed words serving as the starting point. Extensive experiments validate the effectiveness and robustness of our approach, which markedly outperforms state-of-the-art weakly supervised methods. What is more, our approach yields interpretable category rules, proving its advantage in disambiguating easily-confused categories.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Rakesh Agrawal and Ramakrishnan Srikant. 1994. Fast Algorithms for Mining Association Rules in Large Databases. In Proceedings of the 20th International Conference on Very Large Data Bases (VLDB). 487–499.
  2. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis 71 (2021), 102062.
  3. Importance of semantic representation: dataless classification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 830–835.
  4. Dataless text classification with descriptive LDA. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2224–2231.
  5. OpenPrompt: An Open-source Framework for Prompt-learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 105–113.
  6. Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 8560–8579.
  7. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (ACL). 6894–6910.
  8. FreeSpan: Frequent pattern-projected sequential pattern mining. In Proceedings of the sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD), Raghu Ramakrishnan, Salvatore J. Stolfo, Roberto J. Bayardo, and Ismail Parsa (Eds.). ACM, 355–359.
  9. PTR: Prompt tuning with rules for text classification. AI Open 3 (2022), 182–192.
  10. Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). 2225–2240.
  11. Harnessing Deep Neural Networks with Logic Rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). 2410–2420.
  12. Ken Lang. 1995. Newsweeder: Learning to filter netnews. In Machine Learning Proceedings 1995. Elsevier, 331–339.
  13. Filtering and classifying relevant short text with a few seed words. Data and Information Management 3, 3 (2019), 165–186.
  14. Effective document labeling with very few seed words: A topic model approach. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM). ACM, 85–94.
  15. Weakly Supervised Named Entity Tagging with Learnable Logical Rules. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (ACL). Online, 4568–4581.
  16. CL-WSTC: Continual Learning for Weakly Supervised Text Classification on the Internet. In Proceedings of the ACM Web Conference 2023 (WWW). ACM, 1489–1499.
  17. Zero-Label Prompt Selection. arXiv preprint arXiv:2211.04668 (2022).
  18. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
  19. Ilya Loshchilov and Frank Hutter. 2018. Decoupled Weight Decay Regularization. In International Conference on Learning Representations.
  20. Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(ACL). 8086–8098.
  21. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 142–150.
  22. Dheeraj Mekala and Jingbo Shang. 2020. Contextualized weak supervision for text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). 323–333.
  23. Weakly-supervised neural text classification. In proceedings of the 27th ACM International Conference on information and knowledge management (CIKM). ACM, 983–992.
  24. Text classification using label names only: A language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 9006–9017.
  25. Cross-task Knowledge Transfer for Extremely Weakly Supervised Text Classification. In Findings of the Association for Computational Linguistics: ACL 2023. 5329–5341.
  26. Seongmin Park and Jihwa Lee. 2022. LIME: Weakly-Supervised Text Classification without Seeds. In Proceedings of the 29th International Conference on Computational Linguistics. 1083–1088.
  27. Mining Sequential Patterns by Pattern-Growth: The PrefixSpan Approach. IEEE Transactions on Knowledge and Data Engineering 16, 11 (2004), 1424–1440.
  28. Active Hidden Markov Models for Information Extraction. In Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis. 309–318.
  29. Yangqiu Song and Dan Roth. 2014. On dataless hierarchical text classification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 1579–1585.
  30. PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). 14897–14911.
  31. X-Class: Text Classification with Extremely Weak Supervision. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3043–3053.
  32. A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches. In Findings of the Association for Computational Linguistics: ACL 2023. 3944–3962.
  33. Effective seed-guided topic labeling for dataless hierarchical short text classification. In International Conference on Web Engineering (ICWE). 271–285.
  34. Dataless short text classification based on biterm topic model and word embeddings. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI). 3969–3975.
  35. Weakly Supervised Text Classification using Supervision Signals from a Language Model. In Findings of the Association for Computational Linguistics: NAACL. 2295–2305.
  36. Weakly-supervised text classification based on keyword graph. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2803–2813.
  37. Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). 745–758.
  38. Character-level convolutional networks for text classification. In Proceedings of the 28rd International Conference on Neural Information Processing Systems (NIPS). 649–657.
  39. Motifclass: Weakly supervised text classification with higher-order metadata information. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (WSDM). ACM, 1357–1367.
  40. PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). ACM, 12655–12670.
  41. Pre-trained Language Models Can be Fully Zero-Shot Learners. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). 15590–15606.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Miaomiao Li (6 papers)
  2. Jiaqi Zhu (28 papers)
  3. Yang Wang (672 papers)
  4. Yi Yang (856 papers)
  5. Yilin Li (27 papers)
  6. Hongan Wang (18 papers)
Citations (1)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets