Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An LLM-Enhanced Adversarial Editing System for Lexical Simplification (2402.14704v3)

Published 22 Feb 2024 in cs.CL

Abstract: Lexical Simplification (LS) aims to simplify text at the lexical level. Existing methods rely heavily on annotated data, making it challenging to apply in low-resource scenarios. In this paper, we propose a novel LS method without parallel corpora. This method employs an Adversarial Editing System with guidance from a confusion loss and an invariance loss to predict lexical edits in the original sentences. Meanwhile, we introduce an innovative LLM-enhanced loss to enable the distillation of knowledge from LLMs into a small-size LS system. From that, complex words within sentences are masked and a Difficulty-aware Filling module is crafted to replace masked positions with simpler words. At last, extensive experimental results and analyses on three benchmark LS datasets demonstrate the effectiveness of our proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  2. Learning to paraphrase sentences to different complexity levels. arXiv preprint arXiv:2308.02226.
  3. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.
  4. Siobhan Devlin. 1998. The use of a psycholinguistic database in the simplification of text for aphasic readers. Linguistic databases.
  5. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360.
  6. Sentence simplification via large language models. arXiv preprint arXiv:2302.11957.
  7. Generative adversarial networks. Communications of the ACM, 63(11):139–144.
  8. Sian Gooding and Ekaterina Kochmar. 2018. Camb at cwi shared task 2018: Complex word identification with ensemble-based voting. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 184–194.
  9. Sian Gooding and Ekaterina Kochmar. 2019. Recursive context-aware lexical simplification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4853–4863.
  10. Knowledge distillation: A survey. International Journal of Computer Vision, 129:1789–1819.
  11. From images to textual prompts: Zero-shot visual question answering with frozen large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10867–10877.
  12. Learning a lexical simplifier using wikipedia. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 458–463.
  13. Selecting proper lexical paraphrase for children. In Proceedings of the 25th Conference on Computational Linguistics and Speech Processing (ROCLING 2013), pages 59–73.
  14. Robin Keskisärkkä and Arne Jönsson. 2012. Automatic text simplification via synonym replacement. SLTC 2012, page 47.
  15. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  16. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  17. Keep it simple: Unsupervised simplification of multi-paragraph text. arXiv preprint arXiv:2107.03444.
  18. Improving zero-shot visual question answering via large language models with reasoning question prompts. In Proceedings of the 31st ACM International Conference on Multimedia, pages 4389–4400.
  19. Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation, pages 24–26.
  20. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880.
  21. Prompting large language models with chain-of-thought for few-shot knowledge base question generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4329–4343.
  22. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
  23. Introduction to wordnet: An on-line lexical database. International journal of lexicography, 3(4):235–244.
  24. Text simplification by tagging. arXiv preprint arXiv:2103.05070.
  25. Gustavo Paetzold and Lucia Specia. 2016a. Benchmarking lexical simplification systems. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3074–3080.
  26. Gustavo Paetzold and Lucia Specia. 2017a. Lexical simplification with neural ranking. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 34–40.
  27. Gustavo H Paetzold and Lucia Specia. 2016b. Unsupervised lexical simplification for non-native speakers. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 3761–3767.
  28. Gustavo H Paetzold and Lucia Specia. 2017b. A survey on lexical simplification. Journal of Artificial Intelligence Research, 60:549–593.
  29. Lsbert: Lexical simplification based on bert. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3064–3076.
  30. ParaLS: Lexical substitution via pretrained paraphraser. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3731–3746. Association for Computational Linguistics.
  31. Cils at tsar-2022 shared task: Investigating the applicability of lexical substitution methods for lexical simplification. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 207–212.
  32. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30.
  33. Teaching the pre-trained model to generate simple texts for text simplification. arXiv preprint arXiv:2305.12463.
  34. Unsupervised neural text simplification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2058–2068. Association for Computational Linguistics.
  35. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  36. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10014–10037. Association for Computational Linguistics.
  37. Text style transferring via adversarial masking and styled filling. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7654–7663.
  38. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  39. Cental at tsar-2022 shared task: How does context impact bert-generated substitutions for lexical simplification? In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 231–238.
  40. A report on the complex word identification shared task 2018. arXiv preprint arXiv:1804.09132.
  41. CWIG3G2 - complex word identification task across three text genres and two user groups. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 401–407. Asian Federation of Natural Language Processing.
  42. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  43. Semi-supervised text simplification with back-translation and asymmetric denoising autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9668–9675.
  44. A monolingual tree-based translation model for sentence simplification. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1353–1361.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Keren Tan (3 papers)
  2. Kangyang Luo (16 papers)
  3. Yunshi Lan (30 papers)
  4. Zheng Yuan (117 papers)
  5. Jinlong Shu (10 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now