Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PIXELMOD: Improving Soft Moderation of Visual Misleading Information on Twitter (2407.20987v1)

Published 30 Jul 2024 in cs.CV and cs.CY

Abstract: Images are a powerful and immediate vehicle to carry misleading or outright false messages, yet identifying image-based misinformation at scale poses unique challenges. In this paper, we present PIXELMOD, a system that leverages perceptual hashes, vector databases, and optical character recognition (OCR) to efficiently identify images that are candidates to receive soft moderation labels on Twitter. We show that PIXELMOD outperforms existing image similarity approaches when applied to soft moderation, with negligible performance overhead. We then test PIXELMOD on a dataset of tweets surrounding the 2020 US Presidential Election, and find that it is able to identify visually misleading images that are candidates for soft moderation with 0.99% false detection and 2.06% false negatives.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (95)
  1. Trump’s repeated false attacks on mail-in ballots. https://www.factcheck.org/2020/09/trumps-repeated-false-attacks-on-mail-in-ballots/, Sep 2020.
  2. Identifying misinformation from website screenshots. arXiv preprint arXiv:2102.07849, 2021.
  3. Open-domain, content-based, multi-modal fact-checking of out-of-context images via online resources. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14940–14949, 2022.
  4. Voterfraud2020: a multi-modal dataset of election fraud claims on twitter. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, 2021.
  5. D. Abril. Facebook reveals that massive amounts of misinformation flooded its service during the election, Nov 2020.
  6. A secure and robust hash-based scheme for image authentication. Signal Processing, 90(5), 2010.
  7. Content based image retrieval using image features information fusion. Information Fusion, 51, 2019.
  8. Evaluating perceptual hashing algorithms in detecting image manipulation over social media platforms. In 2022 IEEE International Conference on Cyber Security and Resilience (CSR). IEEE, 2022.
  9. The bush-gore recount is an omen for 2020. https://www.theatlantic.com/politics/archive/2020/08/bush-gore-florida-recount-oral-history/614404/, Aug 2020.
  10. Cosmos: Catching out-of-context misinformation with self-supervised learning, 2021.
  11. A. Arsht and D. Etcovitch. The human cost of online content moderation. Harvard Journal of Law and Technology, 2018.
  12. B. Bayar and M. C. Stamm. A deep learning approach to universal image manipulation detection using a new convolutional layer. In ACM IH, 2016.
  13. Toxicity in the decentralized web and the potential for model sharing. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 6(2), jun 2022.
  14. S. Bond. Twitter expands warning labels to slow spread of election misinformation, Oct 2020.
  15. S. Bradshaw and S. Grossman. Were Facebook and Twitter consistent in labeling misleading posts during the 2020 election? https://www.lawfareblog.com/were-facebook-and-twitter-consistent-labeling-misleading-posts-during-2020-election, 2022.
  16. C. Caldera. Fact check: Map showing trump landslide based on false report of seized election servers in germany. https://www.usatoday.com/story/news/factcheck/2020/11/18/fact-check-fake-map-shows-trump-with-410-electoral-votes/3767048001/, Nov 2020.
  17. J. Calma. Facebook will add a new label to some climate change posts in the uk, Feb 2021.
  18. Uniter: Universal image-text representation learning. In European conference on computer vision. Springer, 2020.
  19. K. Conger. Twitter says it labeled 0.2disputed., Nov 2020.
  20. S. Connellan. Facebook to add labels to climate change posts, Oct 2021.
  21. E. Culliford. Twitter launches labels, warnings on misleading covid-19 information. https://www.reuters.com/article/us-health-coronavirus-twitter/twitter-launches-labels-warnings-on-misleading-covid-19-information-idUSKBN22N2E4, May 2020.
  22. E. Culliford. Exclusive zoom has joined tech industry counterterrorism group, Dec 2021.
  23. E. Culliford. Facebook to label all posts about covid-19 vaccines. https://www.reuters.com/article/us-health-coronavirus-facebook/facebook-to-label-all-posts-about-covid-19-vaccines-idUSKBN2B70NJ, Mar 2021.
  24. A. Davis and G. Rosen. Open-sourcing photo-and video-matching technology to make the internet safer. Facebook Newsroom, 2019.
  25. Towards Understanding Crisis Events On Online Social Networks Through Pictures. In ASONAM, 2017.
  26. Vgg image annotator (via). URL: http://www. robots. ox. ac. uk/vgg/software/via, 2, 2016.
  27. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, 1996.
  28. K. Garimella and D. Eckles. Image based misinformation on whatsapp. In International AAAI Conference on Web and Social Media (ICWSM), 2017.
  29. K. Garimella and D. Eckles. Images and misinformation in political groups: Evidence from whatsapp in india. arXiv:2005.09784, 2020.
  30. The doppelgänger bot attack: Exploring identity impersonation in online social networks. In Proceedings of the 2015 internet measurement conference, 2015.
  31. F. González-Pizarro and S. Zannettou. Understanding and detecting hateful content using contrastive learning. In Proceedings of the International AAAI Conference on Web and Social Media, volume 17, 2023.
  32. Google. Cloud vision api. https://cloud.google.com/vision/.
  33. Google. Google’s efforts to combat online child sexual abuse material. https://transparencyreport.google.com/child-sexual-abuse-material/reporting.
  34. M. Graham and S. Rodriguez. Twitter and facebook race to label a slew of posts making false election claims before all votes counted, Nov 2020.
  35. Content based image retrieval systems. Computer, 28(9), 1995.
  36. It’s not what it looks like: Manipulating perceptual hashing based applications. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021.
  37. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
  38. Badhash: Invisible backdoor attacks against deep hashing with clean label. In Proceedings of the 30th ACM International Conference on Multimedia, 2022.
  39. Misinformation debunking and cross-platform information sharing through twitter during hurricanes harvey and irma: a case study on shelters and id checks. Natural Hazards, 103(1), 2020.
  40. T. Ith. Microsoft’s photodna: Protecting children and businesses in the cloud. URL: https://news. microsoft. com/features/microsofts-photodna-protecting-children-andbusinesses-in-the-cloud/366 REFERENCES, 2015.
  41. A first look at covid-19 messages on whatsapp in pakistan. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 2020.
  42. Novel visual and statistical image features for microblogs news verification. IEEE transactions on multimedia, 19(3):598–608, 2016.
  43. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 2019.
  44. Tiplines to combat misinformation on encrypted platforms: a case study of the 2019 indian election on whatsapp. arXiv preprint arXiv:2106.04726, 2021.
  45. E. Kenneally and D. Dittrich. The menlo report: Ethical principles guiding information and communication technology research. Available at SSRN 2445102, 2012.
  46. Surveylance: Automatically detecting online survey scams. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 2018.
  47. Hiding in plain sight: A longitudinal study of combosquatting abuse. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017.
  48. J. Lange. Twitter is now flagging the use of ’oxygen’ and ’frequency’ in the same tweet, prompting new meme. https://theweek.com/speedreads/922275/twitter-now-flagging-use-oxygen-frequency-same-tweet-prompting-new-meme, 2020.
  49. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019.
  50. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014.
  51. D. G. Lowe. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision, volume 2. Ieee, 1999.
  52. K. Lyons. Twitter promises to fine-tune its 5G coronavirus labeling after unrelated tweets were flagged. https://www.theverge.com/2020/6/27/21305503/twitter-labels-5g-conspiracy-coronavirus, 2020.
  53. Dejavu: a system for journalists to collaboratively address visual misinformation. In Computation+ Journalism Symposium. Miami, 2018.
  54. Stop the [image] steal: The role and dynamics of visual content in the 2020 us election misinformation campaign. arXiv preprint arXiv:2209.02007, 2022.
  55. M. L. McHugh. Interrater reliability: the kappa statistic. Biochemia medica, 2012.
  56. Dial one for scam: A large-scale analysis of technical support scams. arXiv preprint arXiv:1607.06891, 2016.
  57. V. Monga and B. L. Evans. Perceptual Image Hashing Via Feature Points: Performance Evaluation and Tradeoffs. IEEE Transactions on Image Processing, 2006.
  58. F. Newsroom. Partnering to help curb spread of online terrorist content, Dec 2016.
  59. Coordinated through aweb of images: Analysis of image-based influence operations from china, iran, russia, and venezuela. arXiv preprint arXiv:2206.03576, 2022.
  60. Stranger danger: exploring the ecosystem of ad-based url shortening services. In Proceedings of the 23rd international conference on World wide web, 2014.
  61. Lambretta: Learning to rank for twitter soft moderation. In IEEE Symposium on Security and Privacy, 2023.
  62. T. D. Platform. Tweet annotations. https://developer.twitter.com/en/docs/twitter-api/annotations/overview.
  63. On the evolution of (hateful) memes by means of multimodal contrastive learning. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023.
  64. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 2021.
  65. N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
  66. Can whatsapp benefit from debunked fact-checked stories to reduce misinformation? arXiv preprint arXiv:2006.02471, 2020.
  67. V. Romo. Twitter to remove or place warning labels on covid vaccine conspiracy tweets. https://www.npr.org/sections/coronavirus-live-updates/2020/12/16/947355414/twitter-to-remove-or-place-warning-labels-on-covid-vaccine-conspiracy-tweets, Dec 2020.
  68. G. Rosen. An update on our work to keep people informed and limit misinformation about covid-19. https://about.fb.com/news/2020/04/covid-19-misinfo-update/, May 2021.
  69. Orb: An efficient alternative to sift or surf. In 2011 International conference on computer vision. Ieee, 2011.
  70. Trollmagnifier: Detecting state-sponsored troll accounts on reddit. In IEEE Symposium on Security and Privacy (SP), 2022.
  71. D. Sayce. The number of tweets per day in 2022. https://www.dsayce.com/social-media/tweets-day/.
  72. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  73. Toward content-based image retrieval with deep convolutional neural networks. In Medical Imaging 2015: Biomedical Applications in Molecular, Structural, and Functional Imaging, volume 9417. SPIE, 2015.
  74. R. Staff. Fact check: Tabulation machines in arizona can read ballots marked with sharpie pens. https://www.reuters.com/article/uk-factcheck-sharpie-arizona/fact-check-tabulation-machines-in-arizona-can-read-ballots-marked-with-sharpie-pens-idUSKBN27L2R5, Nov 2020.
  75. Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 2019.
  76. N. Statt. Major tech platforms say they’re ‘jointly combating fraud and misinformation’ about covid-19, Mar 2020.
  77. The psychological well-being of content moderators: the emotional labor of commercial moderation and avenues for improving support. In Proceedings of the 2021 CHI conference on human factors in computing systems, 2021.
  78. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
  79. Robust image hashing for tamper detection using non-negative matrix factorization. Journal of ubiquitous convergence technology, 2(1), 2008.
  80. TinEye. Tineye: Reverse image search. https://tineye.com/.
  81. Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE transactions on pattern analysis and machine intelligence, 32(5), 2009.
  82. Twitter. Synthetic and manipulated media policy. https://help.twitter.com/en/rules-and-policies/manipulated-media, 2020.
  83. Twitter. Twitter’s civic integrity policy | twitter help, 2020.
  84. Milvus: A purpose-built vector data management system. In Proceedings of the 2021 International Conference on Management of Data, 2021.
  85. A visual model-based perceptual image hash for content authentication. IEEE Transactions on Information Forensics and Security, 10(7), 2015.
  86. Targeted attack and defense for deep hashing. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
  87. Understanding the use of images to spread covid-19 misinformation on twitter. Proceedings of the ACM in Human Computer Interaction (CSCW), 2023.
  88. Understanding the use of fauxtography on social media. In ICWSM, 2021.
  89. T. Wilson and K. Starbird. Cross-platform disinformation campaigns: lessons learned and next steps. Harvard Kennedy School Misinformation Review, 1(1), 2020.
  90. S. Zannettou. I won the election: An empirical analysis of soft moderation interventions on twitter. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, 2021.
  91. On the Origins of Memes by Means of Fringe Web Communities. In Proceedings of the Internet Measurement Conference 2018, 2018.
  92. Characterizing the use of images in state-sponsored information warfare operations by russian trolls on twitter. In Proceedings of the international AAAI conference on web and social media, 2020.
  93. Y. Zhao and W. Wei. Perceptual image hash for tampering detection using zernike moments. In 2010 IEEE International Conference on Progress in Informatics and Computing, volume 2. IEEE, 2010.
  94. Learning rich features for image manipulation detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1053–1061, 2018.
  95. Fact-checking meets fauxtography: Verifying claims about images. In EMNLP-IJCNLP, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Pujan Paudel (9 papers)
  2. Chen Ling (65 papers)
  3. Jeremy Blackburn (76 papers)
  4. Gianluca Stringhini (77 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets