2000 character limit reached
DeepPavlov at SemEval-2024 Task 8: Leveraging Transfer Learning for Detecting Boundaries of Machine-Generated Texts (2405.10629v1)
Published 17 May 2024 in cs.CL and cs.AI
Abstract: The Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection shared task in the SemEval-2024 competition aims to tackle the problem of misusing collaborative human-AI writing. Although there are a lot of existing detectors of AI content, they are often designed to give a binary answer and thus may not be suitable for more nuanced problem of finding the boundaries between human-written and machine-generated texts, while hybrid human-AI writing becomes more and more popular. In this paper, we address the boundary detection problem. Particularly, we present a pipeline for augmenting data for supervised fine-tuning of DeBERTaV3. We receive new best MAE score, according to the leaderboard of the competition, with this pipeline.
- Longformer: The long-document transformer.
- Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA. Curran Associates Inc.
- Deeppavlov: An open source library for conversational ai. In NIPS.
- Multilingual case-insensitive named entity recognition. In Advances in Neural Computation, Machine Learning, and Cognitive Research VI, pages 448–454, Cham. Springer International Publishing.
- Automatic detection of hybrid human-machine text boundaries.
- BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
- Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text.
- Leon Fröhling and Arkaitz Zubiaga. 2021. Feature-based detection of automated language models: tackling gpt-2, gpt-3 and grover. PeerJ Computer Science, 7.
- GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116, Florence, Italy. Association for Computational Linguistics.
- Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.
- Deberta: Decoding-enhanced bert with disentangled attention.
- Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822, Online. Association for Computational Linguistics.
- Automatic detection of machine generated text: A critical survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2296–2309, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- A dataset of peer reviews (PeerRead): Collection, insights and NLP applications. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1647–1661, New Orleans, Louisiana. Association for Computational Linguistics.
- Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. In AAAI Conference on Artificial Intelligence.
- Collecting Better Training Data using Biased Agent Policies in Negotiation Dialogues. In Proceedings of WOCHAT, the Second Workshop on Chatbots and Conversational Agent Technologies, Los Angeles. Zerotype.
- Artificial text boundary detection with topological data analysis and sliding window techniques.
- Roberta: A robustly optimized bert pretraining approach.
- Ai vs. human – differentiation analysis of scientific content generation.
- MULTITuDE: Large-scale multilingual machine-generated text detection benchmark. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9960–9987, Singapore. Association for Computational Linguistics.
- Assessing the stylistic properties of neurally generated text in authorship attribution. In Proceedings of the Workshop on Stylistic Variation, pages 116–125, Copenhagen, Denmark. Association for Computational Linguistics.
- Detectgpt: zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, ICML’23.
- OpenAI. 2023. Gpt-4 technical report.
- ChatGPT vs. crowdsourcing vs. experts: Annotating open-domain conversations with speech functions. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 242–254, Prague, Czechia. Association for Computational Linguistics.
- Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Conference on Empirical Methods in Natural Language Processing.
- Release strategies and the social impacts of language models.
- Llama 2: Open foundation and fine-tuned chat models. ArXiv, abs/2307.09288.
- Authorship attribution for neural text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8384–8395, Online. Association for Computational Linguistics.
- TURINGBENCH: A benchmark environment for Turing test in the age of neural text generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2001–2016, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection.
- Semeval-2024 task 8: Multigenerator, multidomain, and multilingual black-box machine-generated text detection. In Proceedings of the 18th International Workshop on Semantic Evaluation, SemEval 2024, Mexico, Mexico.
- Xlnet: Generalized autoregressive pretraining for language understanding. In Neural Information Processing Systems.
- Defending against neural fake news. Curran Associates Inc., Red Hook, NY, USA.
- Towards automatic boundary detection for human-ai collaborative hybrid essay in education.