Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Collaborative Knowledge Infusion for Low-resource Stance Detection (2403.19219v1)

Published 28 Mar 2024 in cs.CL

Abstract: Stance detection is the view towards a specific target by a given context (\textit{e.g.} tweets, commercial reviews). Target-related knowledge is often needed to assist stance detection models in understanding the target well and making detection correctly. However, prevailing works for knowledge-infused stance detection predominantly incorporate target knowledge from a singular source that lacks knowledge verification in limited domain knowledge. The low-resource training data further increases the challenge for the data-driven large models in this task. To address those challenges, we propose a collaborative knowledge infusion approach for low-resource stance detection tasks, employing a combination of aligned knowledge enhancement and efficient parameter learning techniques. Specifically, our stance detection approach leverages target background knowledge collaboratively from different knowledge sources with the help of knowledge alignment. Additionally, we also introduce the parameter-efficient collaborative adaptor with a staged optimization algorithm, which collaboratively addresses the challenges associated with low-resource stance detection tasks from both network structure and learning perspectives. To assess the effectiveness of our method, we conduct extensive experiments on three public stance detection datasets, including low-resource and cross-target settings. The results demonstrate significant performance improvements compared to the existing stance detection approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. S. Ghosh, P. Singhania, S. Singh, K. Rudra, and S. Ghosh, “Stance detection in web and social media: A comparative study,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction, F. Crestani, M. Braschler, J. Savoy, A. Rauber, H. Müller, D. E. Losada, G. Heinatz Bürki, L. Cappellato, and N. Ferro, Eds.   Cham: Springer International Publishing, 2019, pp. 75–87.
  2. A. Sen, M. Sinha, S. Mannarswamy, and S. Roy, “Stance classification of multi-perspective consumer health information,” in Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, ser. CODS-COMAD ’18.   New York, NY, USA: Association for Computing Machinery, 2018, p. 273–281. [Online]. Available: https://doi.org/10.1145/3152494.3152518
  3. K. Kawintiranon and L. Singh, “Knowledge enhanced masked language model for stance detection,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.   Online: Association for Computational Linguistics, Jun. 2021, pp. 4725–4735. [Online]. Available: https://aclanthology.org/2021.naacl-main.376
  4. Z. He, N. Mokhberian, and K. Lerman, “Infusing knowledge from Wikipedia to enhance stance detection,” in Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis.   Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 71–77. [Online]. Available: https://aclanthology.org/2022.wassa-1.7
  5. R. Liu, Z. Lin, Y. Tan, and W. Wang, “Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.   Online: Association for Computational Linguistics, Aug. 2021, pp. 3152–3157. [Online]. Available: https://aclanthology.org/2021.findings-acl.278
  6. O. Agarwal, H. Ge, S. Shakeri, and R. Al-Rfou, “Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.   Online: Association for Computational Linguistics, Jun. 2021, pp. 3554–3565. [Online]. Available: https://aclanthology.org/2021.naacl-main.278
  7. Y. Xu, C. Zhu, S. Wang, S. Sun, H. Cheng, X. Liu, J. Gao, P. He, M. Zeng, and X. Huang, “Human parity on commonsenseqa: Augmenting self-attention with external attention,” in IJCAI 2022, July 2022, human parity result on CommonsenseQA.
  8. X. Wang, T. Gao, Z. Zhu, Z. Zhang, Z. Liu, J. Li, and J. Tang, “KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 176–194, 03 2021. [Online]. Available: https://doi.org/10.1162/tacl_a_00360
  9. Y. Lin, Y. Meng, X. Sun, Q. Han, K. Kuang, J. Li, and F. Wu, “BertGCN: Transductive text classification by combining GNN and BERT,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.   Online: Association for Computational Linguistics, Aug. 2021, pp. 1456–1462. [Online]. Available: https://aclanthology.org/2021.findings-acl.126
  10. J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022.
  11. E. Allaway and K. McKeown, “Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).   Online: Association for Computational Linguistics, Nov. 2020, pp. 8913–8931. [Online]. Available: https://aclanthology.org/2020.emnlp-main.717
  12. C. Zhu, Y. Xu, X. Ren, B. Y. Lin, M. Jiang, and W. Yu, “Knowledge-augmented methods for natural language processing,” ACL 2022, p. 12, 2022.
  13. W. Liu, P. Zhou, Z. Zhao, Z. Wang, Q. Ju, H. Deng, and P. Wang, “K-bert: Enabling language representation with knowledge graph,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03, pp. 2901–2908, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/5681
  14. D. Yu, C. Zhu, Y. Yang, and M. Zeng, “Jaket: Joint pre-training of knowledge graph and language understanding,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, pp. 11 630–11 638, Jun. 2022. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/21417
  15. Z. Zhang, X. Han, Z. Liu, X. Jiang, M. Sun, and Q. Liu, “ERNIE: Enhanced language representation with informative entities,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.   Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 1441–1451. [Online]. Available: https://aclanthology.org/P19-1139
  16. T. Févry, L. Baldini Soares, N. FitzGerald, E. Choi, and T. Kwiatkowski, “Entities as experts: Sparse memory access with entity supervision,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).   Online: Association for Computational Linguistics, Nov. 2020, pp. 4937–4951. [Online]. Available: https://aclanthology.org/2020.emnlp-main.400
  17. I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A pretrained language model for scientific text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3615–3620. [Online]. Available: https://aclanthology.org/D19-1371
  18. K. r. Kanakarajan, S. Ramamoorthy, V. Archana, S. Chatterjee, and M. Sankarasubbu, “Saama research at MEDIQA 2019: Pre-trained BioBERT with attention visualisation for medical natural language inference,” in Proceedings of the 18th BioNLP Workshop and Shared Task.   Florence, Italy: Association for Computational Linguistics, Aug. 2019, pp. 510–516. [Online]. Available: https://aclanthology.org/W19-5055
  19. D. Q. Nguyen, T. Vu, and A. Tuan Nguyen, “BERTweet: A pre-trained language model for English tweets,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.   Online: Association for Computational Linguistics, Oct. 2020, pp. 9–14. [Online]. Available: https://aclanthology.org/2020.emnlp-demos.2
  20. V. Shwartz, P. West, R. Le Bras, C. Bhagavatula, and Y. Choi, “Unsupervised commonsense question answering with self-talk,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).   Online: Association for Computational Linguistics, Nov. 2020, pp. 4615–4629. [Online]. Available: https://aclanthology.org/2020.emnlp-main.373
  21. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).   Online: Association for Computational Linguistics, Nov. 2020, pp. 6769–6781. [Online]. Available: https://aclanthology.org/2020.emnlp-main.550
  22. Y. Yao, S. Huang, L. Dong, F. Wei, H. Chen, and N. Zhang, “Kformer: Knowledge injection in transformer feed-forward layers,” in Natural Language Processing and Chinese Computing, W. Lu, S. Huang, Y. Hong, and X. Zhou, Eds.   Cham: Springer International Publishing, 2022, pp. 131–143.
  23. D. Küçük and F. Can, “Stance detection: A survey,” ACM Comput. Surv., vol. 53, no. 1, feb 2020. [Online]. Available: https://doi.org/10.1145/3369026
  24. A. ALDayel and W. Magdy, “Stance detection on social media: State of the art and trends,” Information Processing and Management, vol. 58, no. 4, p. 102597, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306457321000960
  25. M. Hardalov, A. Arora, P. Nakov, and I. Augenstein, “A survey on stance detection for mis- and disinformation identification,” in Findings of the Association for Computational Linguistics: NAACL 2022.   Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 1259–1277. [Online]. Available: https://aclanthology.org/2022.findings-naacl.94
  26. M. Mohtarami, J. Glass, and P. Nakov, “Contrastive language adaptation for cross-lingual stance detection,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 4442–4452. [Online]. Available: https://aclanthology.org/D19-1452
  27. E. Zotova, R. Agerri, M. Nuñez, and G. Rigau, “Multilingual stance detection in tweets: The Catalonia independence corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference.   Marseille, France: European Language Resources Association, May 2020, pp. 1368–1375. [Online]. Available: https://aclanthology.org/2020.lrec-1.171
  28. Y. Luo, Z. Liu, Y. Shi, S. Z. Li, and Y. Zhang, “Exploiting sentiment and common sense for zero-shot stance detection,” in Proceedings of the 29th International Conference on Computational Linguistics.   Gyeongju, Republic of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 7112–7123. [Online]. Available: https://aclanthology.org/2022.coling-1.621
  29. R. Liu, Z. Lin, H. Ji, J. Li, P. Fu, and W. Wang, “Target really matters: Target-aware contrastive learning and consistency regularization for few-shot stance detection,” in Proceedings of the 29th International Conference on Computational Linguistics.   Gyeongju, Republic of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 6944–6954. [Online]. Available: https://aclanthology.org/2022.coling-1.605
  30. B. Liang, Q. Zhu, X. Li, M. Yang, L. Gui, Y. He, and R. Xu, “JointCL: A joint contrastive learning framework for zero-shot stance detection,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).   Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 81–91. [Online]. Available: https://aclanthology.org/2022.acl-long.7
  31. N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3982–3992. [Online]. Available: https://aclanthology.org/D19-1410
  32. E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9
  33. Y. Li, T. Sosea, A. Sawant, A. J. Nair, D. Inkpen, and C. Caragea, “P-stance: A large dataset for stance detection in political domain,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.   Online: Association for Computational Linguistics, Aug. 2021, pp. 2355–2365. [Online]. Available: https://aclanthology.org/2021.findings-acl.208
  34. K. Glandt, S. Khanal, Y. Li, D. Caragea, and C. Caragea, “Stance detection in COVID-19 tweets,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).   Online: Association for Computational Linguistics, Aug. 2021, pp. 1596–1611. [Online]. Available: https://aclanthology.org/2021.acl-long.127
  35. J. Du, R. Xu, Y. He, and L. Gui, “Stance classification with target-specific neural attention,” in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, 2017, pp. 3988–3994. [Online]. Available: https://doi.org/10.24963/ijcai.2017/557
  36. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).   Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  37. I. Augenstein, T. Rocktäschel, A. Vlachos, and K. Bontcheva, “Stance detection with bidirectional conditional encoding,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.   Austin, Texas: Association for Computational Linguistics, Nov. 2016, pp. 876–885. [Online]. Available: https://aclanthology.org/D16-1084
  38. W. Xue and T. Li, “Aspect based sentiment analysis with gated convolutional networks,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).   Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 2514–2523. [Online]. Available: https://aclanthology.org/P18-1234
  39. B. Huang and K. Carley, “Parameterized convolutional neural networks for aspect level sentiment classification,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.   Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 1091–1096. [Online]. Available: https://aclanthology.org/D18-1136
  40. B. Zhang, M. Yang, X. Li, Y. Ye, X. Xu, and K. Dai, “Enhancing cross-target stance detection with transferable semantic-emotion knowledge,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.   Online: Association for Computational Linguistics, Jul. 2020, pp. 3188–3197. [Online]. Available: https://aclanthology.org/2020.acl-main.291
  41. B. Zhang, X. Fu, D. Ding, H. Huang, Y. Li, and L. Jing, “Investigating chain-of-thought with chatgpt for stance detection on social media,” arXiv preprint arXiv:2304.03087, 2023.
  42. J. Wei, D. Huang, Y. Lu, D. Zhou, and Q. V. Le, “Simple synthetic data reduces sycophancy in large language models,” arXiv preprint arXiv:2308.03958, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Ming Yan (190 papers)
  2. Joey Tianyi Zhou (116 papers)
  3. Ivor W. Tsang (109 papers)
Citations (1)

Summary

We haven't generated a summary for this paper yet.