Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Evaluating Trustworthiness of Online News Publishers via Article Classification (2401.01781v1)

Published 3 Jan 2024 in cs.IR

Abstract: The proliferation of low-quality online information in today's era has underscored the need for robust and automatic mechanisms to evaluate the trustworthiness of online news publishers. In this paper, we analyse the trustworthiness of online news media outlets by leveraging a dataset of 4033 news stories from 40 different sources. We aim to infer the trustworthiness level of the source based on the classification of individual articles' content. The trust labels are obtained from NewsGuard, a journalistic organization that evaluates news sources using well-established editorial and publishing criteria. The results indicate that the classification model is highly effective in classifying the trustworthiness levels of the news articles. This research has practical applications in alerting readers to potentially untrustworthy news sources, assisting journalistic organizations in evaluating new or unfamiliar media outlets and supporting the selection of articles for their trustworthiness assessment.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Multi-view co-attention network for fake news detection by modeling topic-specific user and news source credibility. Information Processing & Management 60, 1 (2023).
  2. Fine-grained Czech News Article Dataset: An Interdisciplinary Approach to Trustworthiness Analysis. arXiv preprint arXiv:2212.08550 (2022).
  3. S. Bowman and C Willis. 2003. We Media: How Audiences are Shaping the Future of News and Information. The Media Center at the American Press Institute..
  4. Michael Butter and Peter Knight. 2021. Routledge Handbook of Conspiracy Theories. Routledge.
  5. Dallas Card et al. 2015. The Media Frames Corpus: Annotations of Frames Across Issues. In 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Association for Computational Linguistics, 438–444. https://doi.org/10.3115/v1/P15-2072
  6. Fine-grained Structure-based News Genre Categorization. In Events and Stories in the News. Association for Computational Linguistics, 61–67. https://aclanthology.org/W18-4308
  7. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  8. Understanding conspiracy theories. Political psychology 40 (2019), 3–35.
  9. Robert M. Entman. 1993. Framing: Toward Clarification of a Fractured Paradigm. Journal of Communication 43, 4 (1993), 51–58. https://doi.org/10.1111/j.1460-2466.1993.tb01304.x
  10. Forbes Business Council. 2022. Self-Publishing Versus Traditional Publishing: Pros And Cons For Leaders To Consider. https://www.forbes.com/sites/forbesbusinesscouncil/2022/08/15/self-publishing-versus-traditional-publishing-pros-and-cons-for-leaders-to-consider/. Accessed: 2023-12-22.
  11. Yoav Goldberg and Graeme Hirst. 2017. Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers.
  12. Paul Hawken. 1983. The Next Economy. Nenry Holt & co., New York, NY.
  13. Bahareh Heravi. 2022. Storytelling Structures in Data Journalism: Introducing the Water Tower structure. In Computation+ Journalism 2022.
  14. Dariusz Jemielniak and Aleksandra Przegalinska. 2020. Collaborative Society. MIT Press.
  15. Combating Fake News on Social Media with Source Ratings: The Effects of User and Expert Reputation Ratings. Journal of Management Information Systems 36, 3 (2019), 931–968. https://doi.org/10.1080/07421222.2019.1628921
  16. Perceived truth of statements and simulated social media postings: an experimental investigation of source credibility, repeated exposure, and presentation format. Cogn. Research 5, 56 (2020). https://doi.org/10.1186/s41235-020-00251-4
  17. David M. J. Lazer et al. 2018. The science of fake news. Science 359, 6380 (2018), 1094–1096. https://doi.org/10.1126/science.aao2998
  18. High level of correspondence across different news domain quality rating sets. PNAS Nexus 2, 9 (2023).
  19. Siyi Liu et al. 2019. Detecting Frames in News Headlines and Its Application to Analyzing News Framing Trends Surrounding U.S. Gun Violence. In Computational Natural Language Learning (CoNLL). Association for Computational Linguistics, 504–514. https://doi.org/10.18653/v1/K19-1047
  20. SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. In Semantic Evaluation, SemEval@COLING. International Committee for Computational Linguistics, 1377–1414. https://doi.org/10.18653/v1/2020.semeval-1.186
  21. Political polarization & media habits. (2014).
  22. Preslav Nakov et al. 2022. Overview of the CLEF–2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction. Springer, 495–520.
  23. Automatic Classification of Web News: A Systematic Mapping Study. In IntelliSys 2020: Intelligent Systems and Applications. 558–574.
  24. Gordon Pennycook and David G. Rand. 2019. Fighting misinformation on social media using crowdsourced judgments of news source quality. National Academy of Sciences 116, 7 (2019), 2521–2526. https://doi.org/10.1073/pnas.1806781116
  25. M Petrocchi and A Spognardi. 2022. The Online News Market in Italy. Online: https://www.disinformationindex.org/country-studies/2022-1-31-the-online-news-market-in-italy/.
  26. SemEval-2023 Task 3: Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup. In Semantic Evaluation (SemEval-2023). Association for Computational Linguistics, 2343–2361. https://doi.org/10.18653/v1/2023.semeval-1.317
  27. Manuel Pratelli and Marinella Petrocchi. 2022. A Structured Analysis of Journalistic Evaluations for News Source Reliability. In Workshop Proceedings of the 16th International AAAI Conference on Web and Social Media. https://doi.org/10.36190/2022.51
  28. Piotr Przybyla. 2020. Capturing the Style of Fake News. In Conference on Artificial Intelligence. AAAI Press, 490–497. https://doi.org/10.1609/aaai.v34i01.5386
  29. Societal effects of COVID-19 conspiracy theories. Social Psychological and Personality Science (2021).
  30. M. Scharkow. 2013. Thematic content analysis using supervised machine learning: An empirical evaluation using German online news. Quality and Quantity 47 (2013), 761–773. https://doi.org/10.1007/s11135-011-9545-7
  31. Galen Stockinh et al. 2022. The Role of Alternative Social Media in the News and Information Environment. Pew Research Center. Retrieved October 10, 2023 from https://www.pewresearch.org/journalism/2022/10/06/the-role-of-alternative-social-media-in-the-news-and-information-environment/
  32. Kai-Cheng Yang and Filippo Menczer. 2023. Large language models can rate news outlet credibility. CoRR abs/2304.00228 (2023). https://doi.org/10.48550/arXiv.2304.00228
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now