Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Vector-Quantized Prompt Learning for Paraphrase Generation (2311.14949v1)

Published 25 Nov 2023 in cs.CL

Abstract: Deep generative modeling of natural languages has achieved many successes, such as producing fluent sentences and translating from one language into another. However, the development of generative modeling techniques for paraphrase generation still lags behind largely due to the challenges in addressing the complex conflicts between expression diversity and semantic preservation. This paper proposes to generate diverse and high-quality paraphrases by exploiting the pre-trained models with instance-dependent prompts. To learn generalizable prompts, we assume that the number of abstract transforming patterns of paraphrase generation (governed by prompts) is finite and usually not large. Therefore, we present vector-quantized prompts as the cues to control the generation of pre-trained models. Extensive experiments demonstrate that the proposed method achieves new state-of-art results on three benchmark datasets, including Quora, Wikianswers, and MSCOCO. We will release all the code upon acceptance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Regina Barzilay and Lillian Lee. 2003. Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In ACL, pages 16–23.
  2. Rahul Bhagat and Eduard Hovy. 2013. Squibs: What is a paraphrase? Computational Linguistics, 39(3):463–472.
  3. Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding. arXiv e-prints, page arXiv:2205.11024.
  4. Language models are few-shot learners. Advances in NIPS, 33:1877–1901.
  5. Quora question pairs.
  6. Novelty controlled paraphrase generation with retrieval augmented conditional prompt tuning. In AAAI, pages 10535–10543.
  7. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  8. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In COLING.
  9. Zhendong Dong and Qiang Dong. 2003. Hownet-a hybrid language and knowledge resource. In International conference on natural language processing and knowledge engineering, 2003. Proceedings. 2003, pages 820–824. IEEE.
  10. Paraphrase-driven learning for open question answering. In ACL, pages 1608–1618.
  11. Paraphrase generation with latent bag of words. Advances in NIPS, 32.
  12. Tanya Goyal and Greg Durrett. 2020. Neural syntactic preordering for controlled paraphrase generation. arXiv preprint arXiv:2005.02013.
  13. A deep generative framework for paraphrase generation. In AAAI, pages 5149–5156.
  14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780.
  15. Tom Hosking and Mirella Lapata. 2021. Factorising meaning and form for intent-preserving paraphrasing. In ACL-IJCNLP, pages 1405–1418.
  16. Hierarchical sketch induction for paraphrase generation. arXiv preprint arXiv:2203.03463.
  17. David Kauchak and Regina Barzilay. 2006. Paraphrasing for automatic evaluation. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 455–462.
  18. Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization step one: Sentence compression. In AAAI, pages 703–710.
  19. Robust training of vector quantized bottleneck models. In IJCNN, pages 1–7. IEEE.
  20. Paraphrase generation with deep reinforcement learning. In EMNLP, pages 3865–3878.
  21. Decomposable neural paraphrase generation. In ACL, pages 3403––3414.
  22. Dekang Lin and Patrick Pantel. 2001. Discovery of inference rules for question-answering. Natural Language Engineering, 7(4):343–360.
  23. Microsoft COCO: Common objects in context. In ECCV, pages 740–755.
  24. Zhe Lin and Xiaojun Wan. 2021. Pushing paraphrase away from original sentence: A multi-round paraphrase generation approach. In ACL-IJCNLP, pages 1548–1557.
  25. Abstract rule learning for paraphrase generation. In IJCAI, pages 4273–4279.
  26. Unsupervised paraphrasing by simulated annealing. In ACL, pages 302–312.
  27. Kathleen R Mckeown. 1983. Paraphrasing questions using given and new information. Computational Linguistics, 9(1):1–10.
  28. Conrpg: Paraphrase generation using contexts as regularizer. arXiv preprint arXiv:2109.00363.
  29. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41.
  30. Monolingual machine translation for paraphrase generation. In EMNLP, pages 142–149.
  31. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67.
  32. Attention is all you need. In NeurIPS, pages 5998–6008.
  33. A task in a suit and a tie: Paraphrase generation with semantic augmentation. In AAAI, pages 7176–7183.
  34. Hanwei Wu and Markus Flierl. 2020. Vector quantization-based regularization for autoencoders. In AAAI, volume 34, pages 6380–6387.
  35. Learning robust rule representations for abstract reasoning via internal inferences. In Advances in Neural Information Processing Systems, volume 35, pages 33550–33562.
  36. Application-driven statistical paraphrase generation. In ACL, pages 834–842.
  37. Combining multiple resources to improve SMT-based paraphrasing model. In ACL, pages 1021–1029.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now