News Source Credibility Assessment: A Reddit Case Study
Abstract: In the era of social media platforms, identifying the credibility of online content is crucial to combat misinformation. We present the CREDiBERT (CREDibility assessment using Bi-directional Encoder Representations from Transformers), a source credibility assessment model fine-tuned for Reddit submissions focusing on political discourse as the main contribution. We adopt a semi-supervised training approach for CREDiBERT, leveraging Reddit's community-based structure. By encoding submission content using CREDiBERT and integrating it into a Siamese neural network, we significantly improve the binary classification of submission credibility, achieving a 9% increase in F1 score compared to existing methods. Additionally, we introduce a new version of the post-to-post network in Reddit that efficiently encodes user interactions to enhance the binary classification task by nearly 8% in F1 score. Finally, we employ CREDiBERT to evaluate the susceptibility of subreddits with respect to different topics.
- Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2): 211–236.
- Defining and measuring news media quality: Comparing the content perspective and the audience perspective. The International Journal of Press/Politics, 27(1): 9–37.
- Engagement with fact-checked posts on Reddit. PNAS nexus, 2(3): pgad018.
- Predicting information credibility in time-sensitive social media. Internet Research, 23(5): 560–588.
- Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation. Psychological science, 28(11): 1531–1546.
- Combating misinformation in the age of llms: Opportunities and challenges. arXiv preprint arXiv:2311.05656.
- Investigating the Difference of Fake News Source Credibility Recognition between ANN and BERT Algorithms in Artificial Intelligence. Applied Sciences, 12(15): 7725.
- Ideological variation in preferred content and source credibility on Reddit during the COVID-19 pandemic. Big Data & Society, 9(1): 20539517221076486.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
- Identifying and understanding user reactions to deceptive and trusted social news sources. arXiv preprint arXiv:1805.12032.
- Grootendorst, M. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794.
- node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM international conference on Knowledge discovery and data mining, 855–864.
- Bot detection in reddit political discussion. In Fourth International Workshop on Social Sensing, 30–35.
- exbake: Automatic fake news detection model based on bidirectional encoder representations from transformers (bert). Applied Sciences, 9(19): 4062.
- Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.
- Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille.
- Detecting misinformation with llm-predicted credibility signals and weak supervision. arXiv preprint arXiv:2309.07601.
- Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
- Measuring exposure to misinformation from political elites on Twitter. nature communications, 13(1): 7144.
- The role of analytical reasoning and source credibility on the evaluation of real and fake full-length news articles. Cognitive research: principles and implications, 6(1): 1–12.
- Przybyla, P. 2020. Capturing the style of fake news. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 490–497.
- Fake news detection based on news content and social contexts: a transformer-based approach. International Journal of Data Science and Analytics, 13(4): 335–362.
- Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084.
- Factoid: A new dataset for identifying misinformation spreaders and political bias. arXiv preprint arXiv:2205.06181.
- Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter, 19(1): 22–36.
- That’s fake news! Investigating how readers identify the reliability of news when provided title, image, source bias, and full articles. ACM, Human Computer Interaction journal, 5.
- Big Data and quality data for fake news and misinformation detection. Big Data & Society, 6(1): 2053951719843310.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.