Collaborative Knowledge Infusion for Low-resource Stance Detection (2403.19219v1)
Abstract: Stance detection is the view towards a specific target by a given context (\textit{e.g.} tweets, commercial reviews). Target-related knowledge is often needed to assist stance detection models in understanding the target well and making detection correctly. However, prevailing works for knowledge-infused stance detection predominantly incorporate target knowledge from a singular source that lacks knowledge verification in limited domain knowledge. The low-resource training data further increases the challenge for the data-driven large models in this task. To address those challenges, we propose a collaborative knowledge infusion approach for low-resource stance detection tasks, employing a combination of aligned knowledge enhancement and efficient parameter learning techniques. Specifically, our stance detection approach leverages target background knowledge collaboratively from different knowledge sources with the help of knowledge alignment. Additionally, we also introduce the parameter-efficient collaborative adaptor with a staged optimization algorithm, which collaboratively addresses the challenges associated with low-resource stance detection tasks from both network structure and learning perspectives. To assess the effectiveness of our method, we conduct extensive experiments on three public stance detection datasets, including low-resource and cross-target settings. The results demonstrate significant performance improvements compared to the existing stance detection approaches.
- S. Ghosh, P. Singhania, S. Singh, K. Rudra, and S. Ghosh, “Stance detection in web and social media: A comparative study,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction, F. Crestani, M. Braschler, J. Savoy, A. Rauber, H. Müller, D. E. Losada, G. Heinatz Bürki, L. Cappellato, and N. Ferro, Eds. Cham: Springer International Publishing, 2019, pp. 75–87.
- A. Sen, M. Sinha, S. Mannarswamy, and S. Roy, “Stance classification of multi-perspective consumer health information,” in Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, ser. CODS-COMAD ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 273–281. [Online]. Available: https://doi.org/10.1145/3152494.3152518
- K. Kawintiranon and L. Singh, “Knowledge enhanced masked language model for stance detection,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, Jun. 2021, pp. 4725–4735. [Online]. Available: https://aclanthology.org/2021.naacl-main.376
- Z. He, N. Mokhberian, and K. Lerman, “Infusing knowledge from Wikipedia to enhance stance detection,” in Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 71–77. [Online]. Available: https://aclanthology.org/2022.wassa-1.7
- R. Liu, Z. Lin, Y. Tan, and W. Wang, “Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Online: Association for Computational Linguistics, Aug. 2021, pp. 3152–3157. [Online]. Available: https://aclanthology.org/2021.findings-acl.278
- O. Agarwal, H. Ge, S. Shakeri, and R. Al-Rfou, “Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, Jun. 2021, pp. 3554–3565. [Online]. Available: https://aclanthology.org/2021.naacl-main.278
- Y. Xu, C. Zhu, S. Wang, S. Sun, H. Cheng, X. Liu, J. Gao, P. He, M. Zeng, and X. Huang, “Human parity on commonsenseqa: Augmenting self-attention with external attention,” in IJCAI 2022, July 2022, human parity result on CommonsenseQA.
- X. Wang, T. Gao, Z. Zhu, Z. Zhang, Z. Liu, J. Li, and J. Tang, “KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 176–194, 03 2021. [Online]. Available: https://doi.org/10.1162/tacl_a_00360
- Y. Lin, Y. Meng, X. Sun, Q. Han, K. Kuang, J. Li, and F. Wu, “BertGCN: Transductive text classification by combining GNN and BERT,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Online: Association for Computational Linguistics, Aug. 2021, pp. 1456–1462. [Online]. Available: https://aclanthology.org/2021.findings-acl.126
- J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022.
- E. Allaway and K. McKeown, “Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, pp. 8913–8931. [Online]. Available: https://aclanthology.org/2020.emnlp-main.717
- C. Zhu, Y. Xu, X. Ren, B. Y. Lin, M. Jiang, and W. Yu, “Knowledge-augmented methods for natural language processing,” ACL 2022, p. 12, 2022.
- W. Liu, P. Zhou, Z. Zhao, Z. Wang, Q. Ju, H. Deng, and P. Wang, “K-bert: Enabling language representation with knowledge graph,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03, pp. 2901–2908, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/5681
- D. Yu, C. Zhu, Y. Yang, and M. Zeng, “Jaket: Joint pre-training of knowledge graph and language understanding,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, pp. 11 630–11 638, Jun. 2022. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/21417
- Z. Zhang, X. Han, Z. Liu, X. Jiang, M. Sun, and Q. Liu, “ERNIE: Enhanced language representation with informative entities,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 1441–1451. [Online]. Available: https://aclanthology.org/P19-1139
- T. Févry, L. Baldini Soares, N. FitzGerald, E. Choi, and T. Kwiatkowski, “Entities as experts: Sparse memory access with entity supervision,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, pp. 4937–4951. [Online]. Available: https://aclanthology.org/2020.emnlp-main.400
- I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A pretrained language model for scientific text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3615–3620. [Online]. Available: https://aclanthology.org/D19-1371
- K. r. Kanakarajan, S. Ramamoorthy, V. Archana, S. Chatterjee, and M. Sankarasubbu, “Saama research at MEDIQA 2019: Pre-trained BioBERT with attention visualisation for medical natural language inference,” in Proceedings of the 18th BioNLP Workshop and Shared Task. Florence, Italy: Association for Computational Linguistics, Aug. 2019, pp. 510–516. [Online]. Available: https://aclanthology.org/W19-5055
- D. Q. Nguyen, T. Vu, and A. Tuan Nguyen, “BERTweet: A pre-trained language model for English tweets,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Online: Association for Computational Linguistics, Oct. 2020, pp. 9–14. [Online]. Available: https://aclanthology.org/2020.emnlp-demos.2
- V. Shwartz, P. West, R. Le Bras, C. Bhagavatula, and Y. Choi, “Unsupervised commonsense question answering with self-talk,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, pp. 4615–4629. [Online]. Available: https://aclanthology.org/2020.emnlp-main.373
- V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, pp. 6769–6781. [Online]. Available: https://aclanthology.org/2020.emnlp-main.550
- Y. Yao, S. Huang, L. Dong, F. Wei, H. Chen, and N. Zhang, “Kformer: Knowledge injection in transformer feed-forward layers,” in Natural Language Processing and Chinese Computing, W. Lu, S. Huang, Y. Hong, and X. Zhou, Eds. Cham: Springer International Publishing, 2022, pp. 131–143.
- D. Küçük and F. Can, “Stance detection: A survey,” ACM Comput. Surv., vol. 53, no. 1, feb 2020. [Online]. Available: https://doi.org/10.1145/3369026
- A. ALDayel and W. Magdy, “Stance detection on social media: State of the art and trends,” Information Processing and Management, vol. 58, no. 4, p. 102597, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306457321000960
- M. Hardalov, A. Arora, P. Nakov, and I. Augenstein, “A survey on stance detection for mis- and disinformation identification,” in Findings of the Association for Computational Linguistics: NAACL 2022. Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 1259–1277. [Online]. Available: https://aclanthology.org/2022.findings-naacl.94
- M. Mohtarami, J. Glass, and P. Nakov, “Contrastive language adaptation for cross-lingual stance detection,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 4442–4452. [Online]. Available: https://aclanthology.org/D19-1452
- E. Zotova, R. Agerri, M. Nuñez, and G. Rigau, “Multilingual stance detection in tweets: The Catalonia independence corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, May 2020, pp. 1368–1375. [Online]. Available: https://aclanthology.org/2020.lrec-1.171
- Y. Luo, Z. Liu, Y. Shi, S. Z. Li, and Y. Zhang, “Exploiting sentiment and common sense for zero-shot stance detection,” in Proceedings of the 29th International Conference on Computational Linguistics. Gyeongju, Republic of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 7112–7123. [Online]. Available: https://aclanthology.org/2022.coling-1.621
- R. Liu, Z. Lin, H. Ji, J. Li, P. Fu, and W. Wang, “Target really matters: Target-aware contrastive learning and consistency regularization for few-shot stance detection,” in Proceedings of the 29th International Conference on Computational Linguistics. Gyeongju, Republic of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 6944–6954. [Online]. Available: https://aclanthology.org/2022.coling-1.605
- B. Liang, Q. Zhu, X. Li, M. Yang, L. Gui, Y. He, and R. Xu, “JointCL: A joint contrastive learning framework for zero-shot stance detection,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 81–91. [Online]. Available: https://aclanthology.org/2022.acl-long.7
- N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3982–3992. [Online]. Available: https://aclanthology.org/D19-1410
- E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9
- Y. Li, T. Sosea, A. Sawant, A. J. Nair, D. Inkpen, and C. Caragea, “P-stance: A large dataset for stance detection in political domain,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Online: Association for Computational Linguistics, Aug. 2021, pp. 2355–2365. [Online]. Available: https://aclanthology.org/2021.findings-acl.208
- K. Glandt, S. Khanal, Y. Li, D. Caragea, and C. Caragea, “Stance detection in COVID-19 tweets,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics, Aug. 2021, pp. 1596–1611. [Online]. Available: https://aclanthology.org/2021.acl-long.127
- J. Du, R. Xu, Y. He, and L. Gui, “Stance classification with target-specific neural attention,” in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, 2017, pp. 3988–3994. [Online]. Available: https://doi.org/10.24963/ijcai.2017/557
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
- I. Augenstein, T. Rocktäschel, A. Vlachos, and K. Bontcheva, “Stance detection with bidirectional conditional encoding,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas: Association for Computational Linguistics, Nov. 2016, pp. 876–885. [Online]. Available: https://aclanthology.org/D16-1084
- W. Xue and T. Li, “Aspect based sentiment analysis with gated convolutional networks,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 2514–2523. [Online]. Available: https://aclanthology.org/P18-1234
- B. Huang and K. Carley, “Parameterized convolutional neural networks for aspect level sentiment classification,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 1091–1096. [Online]. Available: https://aclanthology.org/D18-1136
- B. Zhang, M. Yang, X. Li, Y. Ye, X. Xu, and K. Dai, “Enhancing cross-target stance detection with transferable semantic-emotion knowledge,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, Jul. 2020, pp. 3188–3197. [Online]. Available: https://aclanthology.org/2020.acl-main.291
- B. Zhang, X. Fu, D. Ding, H. Huang, Y. Li, and L. Jing, “Investigating chain-of-thought with chatgpt for stance detection on social media,” arXiv preprint arXiv:2304.03087, 2023.
- J. Wei, D. Huang, Y. Lu, D. Zhou, and Q. V. Le, “Simple synthetic data reduces sycophancy in large language models,” arXiv preprint arXiv:2308.03958, 2023.
- Ming Yan (190 papers)
- Joey Tianyi Zhou (116 papers)
- Ivor W. Tsang (109 papers)