Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization (2403.01152v1)

Published 2 Mar 2024 in cs.CL and cs.AI

Abstract: We have witnessed lately a rapid proliferation of advanced LLMs capable of generating high-quality text. While these LLMs have revolutionized text generation across various domains, they also pose significant risks to the information ecosystem, such as the potential for generating convincing propaganda, misinformation, and disinformation at scale. This paper offers a review of AI-generated text forensic systems, an emerging field addressing the challenges of LLM misuses. We present an overview of the existing efforts in AI-generated text forensics by introducing a detailed taxonomy, focusing on three primary pillars: detection, attribution, and characterization. These pillars enable a practical understanding of AI-generated text, from identifying AI-generated content (detection), determining the specific AI model involved (attribution), and grouping the underlying intents of the text (characterization). Furthermore, we explore available resources for AI-generated text forensics research and discuss the evolving challenges and future directions of forensic systems in an AI era.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (115)
  1. Generating sentiment-preserving fake online reviews using neural language models and their human-and machine-based detection. In International Conference on Advanced Information Networking and Applications, pages 1341–1354. Springer.
  2. Can knowledge graphs reduce hallucinations in llms?: A survey. arXiv preprint arXiv:2311.07914.
  3. Gpt-2 versus gpt-3 and bloom: Llms for llms generative text detection.
  4. The falcon series of open language models. arXiv preprint arXiv:2311.16867.
  5. Bander Alsulami et al. 2017. Source code authorship attribution using long short-term memory based networks. In Computer Security–ESORICS 2017: 22nd European Symposium on Research in Computer Security, Oslo, Norway, September 11-15, 2017, Proceedings, Part I 22, pages 65–82. Springer.
  6. Real or fake? learning to discriminate machine from human generated text. arXiv preprint arXiv:1906.03351.
  7. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ArXiv:2310.05130 [cs].
  8. Comparing the willingness to share for human-generated vs. ai-generated fake news. arXiv preprint arXiv:2402.07395.
  9. A neural probabilistic language model. Advances in neural information processing systems, 13.
  10. ConDA: Contrastive Domain Adaptation for AI-generated Text Detection. ArXiv:2309.03992 [cs].
  11. Conda: Contrastive domain adaptation for ai-generated text detection. arXiv preprint arXiv:2309.03992.
  12. Amrita Bhattacharjee and Huan Liu. 2023. Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?
  13. Experiments with convolutional neural networks for multi-label authorship attribution. In LREC’18.
  14. Canyu Chen and Kai Shu. 2023. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788.
  15. Do models explain themselves? counterfactual simulatability of natural language explanations. arXiv preprint arXiv:2307.08678.
  16. Zheng Chen and Huming Liu. 2023. Stadee: Statistics-based deep detection of machine generated text. In International Conference on Intelligent Computing, pages 732–743. Springer.
  17. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023).
  18. LM vs LM: Detecting Factual Errors via Cross Examination. ArXiv:2305.13281 [cs].
  19. Machine-generated text: A comprehensive survey of threat models and detection methods. IEEE Access.
  20. Who said that? benchmarking social media ai detection. arXiv preprint arXiv:2310.08240.
  21. Detecting AI Authorship: Analyzing Descriptive Features for AI Detection. In Proceedings of the Seventh Workshop on Natural Language for Artificial Intelligence (NL4AI 2023), volume 3551 of CEUR Workshop Proceedings, Rome, Italy. CEUR. ISSN: 1613-0073.
  22. Tweepfake: About detecting deepfake tweets. Plos one, 16(5):e0251415.
  23. Myles Foley et al. 2023. Matching pairs: Attributing fine-tuned models to their pre-trained large language models. In ACL.
  24. Rinaldo Gagiano and Lin Tian. 2023. A prompt in the right direction: Prompt based classification of machine-generated text detection. In Proceedings of ALTA.
  25. Detecting Generated Text and Attributing Language Model Source with Fine-tuned Models and Semantic Understanding.
  26. Detecting generated text and attributing language model source with fine-tuned models and semantic understanding.
  27. GLTR: Statistical Detection and Visualization of Generated Text. ArXiv:1906.04043 [cs].
  28. How persuasive is ai-generated propaganda? PNAS nexus, 3(2):pgae034.
  29. The false promise of imitating proprietary llms. arXiv preprint arXiv:2305.15717.
  30. Zhen Guo and Shangdi Yu. 2023. AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising. ArXiv:2311.07700 [cs].
  31. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650.
  32. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822.
  33. Automatic detection of machine generated text: A critical survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2296–2309.
  34. Disinformation detection: An evolving challenge in the age of llms. arXiv preprint arXiv:2309.15847.
  35. Working with ai to persuade: Examining a large language model’s ability to generate pro-vaccination messages. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1–29.
  36. OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples.
  37. Computational methods in authorship attribution. Journal of the American Society for information Science and Technology, 60(1):9–26.
  38. All the news that’s fit to fabricate: Ai-generated text as a tool of media misinformation. Journal of experimental political science, 9(1):104–117.
  39. J-Guard: Journalism Guided Adversarially Robust Detection of AI-generated News.
  40. J-guard: Journalism guided adversarially robust detection of ai-generated news. arXiv preprint arXiv:2309.03164.
  41. Stylometric detection of ai-generated text in twitter timelines. arXiv preprint arXiv:2303.03697.
  42. Tharindu Kumarage and Huan Liu. 2023. Neural Authorship Attribution: Stylometric Analysis on Large Language Models. ArXiv:2308.07305 [cs].
  43. How reliable are ai-generated-text detectors? an assessment framework using evasive soft prompts. arXiv preprint arXiv:2310.05095.
  44. Demasq: Unmasking the chatgpt wordsmith. arXiv preprint arXiv:2311.05019.
  45. Artificial text detection via examining the topology of attention maps. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 635–649, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  46. Artificial text detection via examining the topology of attention maps. arXiv preprint arXiv:2109.04825.
  47. A Survey of Text Watermarking in the Era of Large Language Models. ArXiv:2312.07913 [cs].
  48. CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Low Resource With Contrastive Learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16167–16188, Singapore. Association for Computational Linguistics.
  49. Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16167–16188.
  50. Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. ArXiv:2306.05524 [cs].
  51. Check me if you can: Detecting chatgpt-generated academic writing using checkgpt. arXiv preprint arXiv:2306.05524.
  52. Vijini Liyanage and Davide Buscaldi. 2023. Detecting artificially generated academic text: The importance of mimicking human utilization of large language models. In International Conference on Applications of Natural Language to Information Systems, pages 558–565. Springer.
  53. Fighting fire with fire: The dual role of llms in crafting and detecting elusive disinformation. arXiv preprint arXiv:2310.15515.
  54. Multitude: Large-scale multilingual machine-generated text detection benchmark. arXiv preprint arXiv:2310.13606.
  55. AI-Writing Detection Using an Ensemble of Transformers and Stylometric Features.
  56. Ai-writing detection using an ensemble of transformers and stylometric features.
  57. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. ArXiv:2305.14251 [cs].
  58. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ArXiv:2301.11305 [cs].
  59. Distinguishing fact from fiction: A benchmark dataset for identifying machine-generated scientific papers in the llm era. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 190–207.
  60. Rex: Rapid exploration and exploitation for ai agents. arXiv preprint arXiv:2307.08962.
  61. Stacking the odds: Transformer-based ensemble for ai-generated text detection. arXiv preprint arXiv:2310.18906.
  62. Language model detectors are easily optimized against. In The Twelfth International Conference on Learning Representations.
  63. OpenAI. 2023. Gpt-4 technical report.
  64. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  65. On the risk of misinformation pollution with large language models. arXiv preprint arXiv:2305.13661.
  66. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22.
  67. Judea Pearl. 2009. Causality. Cambridge university press.
  68. To chatgpt, or not to chatgpt: That is the question! arXiv preprint arXiv:2304.01487.
  69. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  70. A Robust Semantics-based Watermark for Large Language Model against Paraphrasing. ArXiv:2311.08721 [cs].
  71. Can AI-Generated Text be Reliably Detected?
  72. Overview of autextification at iberlef 2023: Detection and attribution of machine-generated text in multiple domains. arXiv preprint arXiv:2309.11285.
  73. Supervised machine-generated text detectors: Family and scale matters. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 121–132. Springer.
  74. Classification of Human- and AI-Generated Texts for English, French, German, and Spanish. ArXiv:2312.04882 [cs].
  75. Classification of human-and ai-generated texts for english, french, german, and spanish. arXiv preprint arXiv:2312.04882.
  76. Claude Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423.
  77. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203.
  78. Ai model gpt-3 (dis)informs us better than humans. Science Advances, 9(26):eadh1850.
  79. Adapting fake news detection to the era of large language models. arXiv preprint arXiv:2311.04917.
  80. Fake news detectors are biased against texts generated by large language models. arXiv preprint arXiv:2309.08674.
  81. DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text.
  82. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. arXiv preprint arXiv:2306.05540.
  83. Hc3 plus: A semantic-invariant human chatgpt comparison corpus. arXiv preprint arXiv:2309.02731.
  84. Towards detecting harmful agendas in news articles. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 110–128, Toronto, Canada. Association for Computational Linguistics.
  85. The science of detecting llm-generated texts. arXiv preprint arXiv:2303.07205.
  86. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html, 3(6):7.
  87. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  88. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  89. Hansen: human and ai spoken text benchmark for authorship analysis. arXiv preprint arXiv:2310.16746.
  90. Attribution and obfuscation of neural text authorship: A data mining perspective. ACM SIGKDD Explorations Newsletter, 25(1):1–18.
  91. Authorship attribution for neural text generation. In EMNLP.
  92. TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation. ArXiv:2109.13296 [cs].
  93. Turingbench: A benchmark environment for turing test in the age of neural text generation. arXiv preprint arXiv:2109.13296.
  94. Attention is all you need. Advances in neural information processing systems, 30.
  95. GPT-who: An Information Density-based Machine-Generated Text Detector.
  96. Ghostbuster: Detecting Text Ghostwritten by Large Language Models. ArXiv:2305.15047 [cs].
  97. Ghostbuster: Detecting text ghostwritten by large language models. arXiv preprint arXiv:2305.15047.
  98. Bot or human? detecting chatgpt imposters with a single question. arXiv preprint arXiv:2305.06424.
  99. SeqXGPT: Sentence-Level AI-Generated Text Detection.
  100. DetectGPT-SC: Improving Detection of Text Generated by Large Language Models through Self-Consistency with Masked Predictions.
  101. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection. arXiv preprint arXiv:2305.14902.
  102. Factcheck-GPT: End-to-End Fine-Grained Document-Level Fact-Checking and Correction of LLM Output. ArXiv:2311.09000 [cs].
  103. Understanding news creation intents: Frame, dataset, and method. arXiv preprint arXiv:2312.16490.
  104. Jiaying Wu and Bryan Hooi. 2023. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. arXiv preprint arXiv:2310.10830.
  105. Llmdet: A third party large language models generated text detection tool. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2113–2133.
  106. Zhendong Wu and Hui Xiang. 2023. Mfd: Multi-feature detection of llm-generated text.
  107. Weifeng Xu and Dianxiang Xu. 2022. Visualizing and reasoning about presentable digital forensic evidence with knowledge graphs. In 2022 19th Annual International Conference on Privacy, Security & Trust (PST), pages 1–10. IEEE.
  108. Kai-Cheng Yang and Filippo Menczer. 2023. Anatomy of an ai-powered malicious social botnet. arXiv preprint arXiv:2307.16336.
  109. Is chatgpt involved in texts? measure the polish ratio to detect chatgpt-generated text. arXiv preprint arXiv:2307.11380.
  110. Defending against neural fake news. In Advances in Neural Information Processing Systems, pages 9054–9065.
  111. Defending against neural fake news. Neurips.
  112. Ruipeng Zhang and Mengjun Xie. 2023. A knowledge graph question answering approach to iot forensics. In Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation, pages 446–447.
  113. Neural deepfake detection with factual structure of text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2461–2470.
  114. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20.
  115. Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7470–7483, Singapore. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Tharindu Kumarage (21 papers)
  2. Garima Agrawal (12 papers)
  3. Paras Sheth (20 papers)
  4. Raha Moraffah (25 papers)
  5. Aman Chadha (110 papers)
  6. Joshua Garland (35 papers)
  7. Huan Liu (283 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets