Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Relative Counterfactual Contrastive Learning for Mitigating Pretrained Stance Bias in Stance Detection (2405.10991v1)

Published 16 May 2024 in cs.LG, cs.AI, and stat.ME

Abstract: Stance detection classifies stance relations (namely, Favor, Against, or Neither) between comments and targets. Pretrained LLMs (PLMs) are widely used to mine the stance relation to improve the performance of stance detection through pretrained knowledge. However, PLMs also embed ``bad'' pretrained knowledge concerning stance into the extracted stance relation semantics, resulting in pretrained stance bias. It is not trivial to measure pretrained stance bias due to its weak quantifiability. In this paper, we propose Relative Counterfactual Contrastive Learning (RCCL), in which pretrained stance bias is mitigated as relative stance bias instead of absolute stance bias to overtake the difficulty of measuring bias. Firstly, we present a new structural causal model for characterizing complicated relationships among context, PLMs and stance relations to locate pretrained stance bias. Then, based on masked LLM prediction, we present a target-aware relative stance sample generation method for obtaining relative bias. Finally, we use contrastive learning based on counterfactual theory to mitigate pretrained stance bias and preserve context stance relation. Experiments show that the proposed method is superior to stance detection and debiasing baselines.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Emily Allaway and Kathleen McKeown. 2020. Zero-Shot stance detection: A dataset and model using generalized topic representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8913–8931, Online. Association for Computational Linguistics.
  2. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885, Austin, Texas. Association for Computational Linguistics.
  3. A simple framework for contrastive learning of visual representations. In Proceeding of 2020 International Conference on Machine Learning, pages 1597–1607. PMLR.
  4. C2l: Causally contrastive learning for robust text classification. In Proceedings of the 2022 AAAI Conference on Artificial Intelligence, volume 36, pages 10526–10534.
  5. Causalm: Causal model explanation through counterfactual language models. Computational Linguistics, 47(2):333–386.
  6. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 11737–11762, Toronto, Canada. Association for Computational Linguistics.
  7. A survey on stance detection for mis- and disinformation identification. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1259–1277, Seattle, United States. Association for Computational Linguistics.
  8. Counterfactual collaborative reasoning. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining, pages 249–257.
  9. Few-shot stance detection via target-aware prompt distillation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 837–847, New York, USA. Association for Computing Machinery.
  10. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673.
  11. Dilek Küçük and Fazli Can. 2021. Stance detection: Concepts, approaches, resources, and outstanding issues. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2673–2676, New York, USA. Association for Computing Machinery.
  12. Affinity and diversity: Quantifying mechanisms of data augmentation. CoRR.
  13. Gaussian processes for rumour stance classification in social media. ACM Transactions on Information Systems, 37(2):1–24.
  14. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41, San Diego, California. Association for Computational Linguistics.
  15. Counterfactual vqa: A cause-effect look at language bias. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12700–12710.
  16. Yulei Niu and Hanwang Zhang. 2021. Introspective distillation for robust question answering. In Proceedings of Advances in Neural Information Processing Systems, volume 34, pages 16292–16304. Curran Associates, Inc.
  17. Differential bias: On the perceptibility of stance imbalance in argumentation. In Findings of the 60th Annual Meeting of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing., pages 411–421, Online. Association for Computational Linguistics.
  18. Causal inference in statistics: A primer. Wiley.
  19. STANCY: Stance classification based on consistency cues. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6413–6418, Hong Kong, China. Association for Computational Linguistics.
  20. Counterfactual inference for text classification debiasing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5434–5445, Online. Association for Computational Linguistics.
  21. Exploring the limits of transfer learning with a unified text-to-text transformer. Machine Learning Research, 21(1):5485–5551.
  22. Contrastive learning with hard negative samples. In Proceedings of the 9th International Conference on Learning Representations, pages 1–29, Australia. OpenReview.net.
  23. Cross-topic argument mining from heterogeneous sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3664–3674, Brussels, Belgium. Association for Computational Linguistics.
  24. Overview of the task on stance and gender detection in tweets on catalan independence. In Proceedings of the 2th Workshop on Evaluation of Human Language Technologies for Iberian Languages, pages 157–177, Murcia, Spain. CEUR-WS.org.
  25. Mitigating spurious correlation in natural language understanding with counterfactual inference. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11308–11321, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  26. Limin Wang and Dexin Wang. 2021. Solving stance detection on tweets as multi-domain and multi-task text classification. IEEE Access, 9:157780–157789.
  27. Jiaying Wu and Bryan Hooi. 2023. Probing spurious correlations innbsp;popular event-based rumor detection benchmarks. In Proceedings of the 16th Machine Learning and Knowledge Discovery in Databases, page 274–290, Berlin, Heidelberg. Springer-Verlag.
  28. Pre-trained models for natural language processing: A survey. SCIENCE CHINA Technological Sciences, 63(10):1872–1897.
  29. Cross-target stance classification with self-attention networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 778–783, Melbourne, Australia. Association for Computational Linguistics.
  30. SSR: Utilizing simplified stance reasoning process for robust stance detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6846–6858, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  31. Debiasing stance detection models with counterfactual reasoning and adversarial bias learning. CoRR, abs/2212.10392.
  32. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197, Online. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Jiarui Zhang (43 papers)
  2. Shaojuan Wu (4 papers)
  3. Xiaowang Zhang (28 papers)
  4. Zhiyong Feng (143 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets