2000 character limit reached
ClaimDiff: Comparing and Contrasting Claims on Contentious Issues (2205.12221v2)
Published 24 May 2022 in cs.CL
Abstract: With the growing importance of detecting misinformation, many studies have focused on verifying factual claims by retrieving evidence. However, canonical fact verification tasks do not apply to catching subtle differences in factually consistent claims, which might still bias the readers, especially on contentious political or economic issues. Our underlying assumption is that among the trusted sources, one's argument is not necessarily more true than the other, requiring comparison rather than verification. In this study, we propose ClaimDiff, a novel dataset that primarily focuses on comparing the nuance between claim pairs. In ClaimDiff, we provide 2,941 annotated claim pairs from 268 news articles. We observe that while humans are capable of detecting the nuances between claims, strong baselines struggle to detect them, showing over a 19% absolute gap with the humans. We hope this initial study could help readers to gain an unbiased grasp of contentious issues through machine-aided comparison.
- Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society. In Findings of EMNLP.
- AraStance: A multi-country and multi-domain dataset of Arabic stance detection for fact checking. In Proceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda.
- FEVEROUS: Fact extraction and VERification over unstructured and structured information. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).
- Integrating stance detection and fact checking in a unified corpus. In NAACL-HLT.
- Stance classification of context-dependent claims. In EACL.
- Language models are few-shot learners. In NeurIPS.
- Seeing things from a different angle:discovering diverse perspectives about claims. In NAACL-HLT.
- BoolQ: Exploring the surprising difficulty of natural yes/no questions. In NAACL-HLT.
- SemEval-2017 task 8: RumourEval: Determining rumour veracity and support for rumours. In SemEval-2017, Vancouver, Canada.
- BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT.
- ERASER: A benchmark to evaluate rationalized NLP models. In ACL.
- William Ferreira and Andreas Vlachos. 2016. Emergent: a novel data-set for stance classification. In NAACL-HLT.
- Political discourse on social media: Echo chambers, gatekeepers, and the price of bipartisanship. In WWW.
- A richly annotated corpus for different tasks in automated fact-checking. In PCoNLL, Hong Kong, China.
- Andrew F. Hayes and Klaus Krippendorff. 2007. Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1).
- {DEBERTA}: {DECODING}-{enhanced} {bert} {with} {disentangled} {attention}. In ICLR.
- LoRA: Low-rank adaptation of large language models. In ICLR.
- Categorising fine-to-coarse grained misinformation: An empirical study of covid-19 infodemic. arXiv preprint arXiv:2106.11702.
- WatClaimCheck: A new dataset for claim entailment and inference. In ACL.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
- All-in-one: Multi-task learning for rumour verification. In COLING.
- FiD-ex: Improving sequence-to-sequence models for extractive rationale generation. In EMNLP.
- Markus Leippold and Thomas Diggelmann. 2020. Climate-fever: A dataset for verification of real-world climate claims. In NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning.
- P-stance: A large dataset for stance detection in political domain. In Findings of ACL-IJCNLP.
- Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
- SemEval-2016 task 6: Detecting stance in tweets. In SemEval-2016.
- Wt5?! training text-to-text models to explain their predictions. arXiv preprint arXiv:2004.14546.
- Fang: Leveraging social context for fake news detection using graph representation. In CIKM.
- Combining fact extraction and verification with neural semantic matching networks. In AAAI.
- Jeppe Nørregaard and Leon Derczynski. 2021. DanFEVER: claim verification dataset for Danish. In NoDaLiDa.
- Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155.
- Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32.
- A stylometric inquiry into hyperpartisan and fake news. In ACL.
- Zero: Memory optimizations toward training trillion parameter models.
- SQuAD: 100,000+ questions for machine comprehension of text. In EMNLP.
- Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In KDD.
- COVID-fact: Fact extraction and verification of real-world claims on COVID-19 pandemic. In ACL-IJCNLP.
- FEVER: a large-scale dataset for fact extraction and VERification. In NAACL-HLT.
- NewsQA: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP.
- Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science.
- Fact or fiction: Verifying scientific claims. In EMNLP.
- William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In ACL, Vancouver, Canada.
- Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
- Maxwell Weinzierl and Sanda Harabagiu. 2022. Identifying the adoption or rejection of misinformation targeting covid-19 vaccines in twitter discourse. In WWW.
- Janyce Wiebe and Ellen Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In CICLing, CICLing’05, Berlin, Heidelberg. Springer-Verlag.
- A broad-coverage challenge corpus for sentence understanding through inference. In NAACL-HLT.
- Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.
- Omar Zaidan and Jason Eisner. 2008. Modeling annotators: A generative approach to learning from annotator rationales. In EMNLP.
- Analysing how people orient to and spread rumours in social media by looking at conversational threads. PloS one, 11(3).
- Miyoung Ko (9 papers)
- Ingyu Seong (1 paper)
- Hwaran Lee (31 papers)
- Joonsuk Park (24 papers)
- Minsuk Chang (19 papers)
- Minjoon Seo (82 papers)