Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation (2306.08456v3)

Published 14 Jun 2023 in cs.CL

Abstract: Controllable text generation is a challenging and meaningful field in natural language generation (NLG). Especially, poetry generation is a typical one with well-defined and strict conditions for text generation which is an ideal playground for the assessment of current methodologies. While prior works succeeded in controlling either semantic or metrical aspects of poetry generation, simultaneously addressing both remains a challenge. In this paper, we pioneer the use of the Diffusion model for generating sonnets and Chinese SongCi poetry to tackle such challenges. In terms of semantics, our PoetryDiffusion model, built upon the Diffusion model, generates entire sentences or poetry by comprehensively considering the entirety of sentence information. This approach enhances semantic expression, distinguishing it from autoregressive and LLMs. For metrical control, the separation feature of diffusion generation and its constraint control module enable us to flexibly incorporate a novel metrical controller to manipulate and evaluate metrics (format and rhythm). The denoising process in PoetryDiffusion allows for gradual enhancement of semantics and flexible integration of the metrical controller which can calculate and impose penalties on states that stray significantly from the target control distribution. Experimental results on two datasets demonstrate that our model outperforms existing models in automatic evaluation of semantic, metrical, and overall performance as well as human evaluation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Shall i compare thee to a machine-written sonnet? an approach to algorithmic sonnet generation. arXiv preprint arXiv:1811.05067.
  2. Controlled Text Generation with Adversarial Learning. In Proceedings of the 13th International Conference on Natural Language Generation, 29–34. Dublin, Ireland: Association for Computational Linguistics.
  3. Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11): 7327–7347.
  4. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901.
  5. Maximum-likelihood augmented discrete generative adversarial networks (2017). arXiv preprint arXiv:1702.07983.
  6. Sentiment-Controllable Chinese Poetry Generation. In IJCAI, 4925–4931.
  7. Controllable Conversation Generation with Conversation Structures via Diffusion Models. In Findings of the Association for Computational Linguistics: ACL 2023, 7238–7251. Toronto, Canada: Association for Computational Linguistics.
  8. An iterative polishing framework based on quality aware masked language model for Chinese poetry generation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 7643–7650.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  10. Generating topical poetry. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1183–1191.
  11. Generative adversarial networks. Communications of the ACM, 63(11): 139–144.
  12. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
  13. Toward controlled generation of text. In International conference on machine learning, 1587–1596. PMLR.
  14. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
  15. Deep-speare: A joint neural model of poetic language, meter and rhyme. arXiv preprint arXiv:1807.03491.
  16. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055.
  17. Rigid formats controlled text generation. In Proceedings of the 58th annual meeting of the association for computational linguistics, 742–751.
  18. Diffusion-LM Improves Controllable Text Generation. arXiv preprint arXiv:2205.14217.
  19. Lin, C.-Y. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74–81.
  20. Adversarial ranking for language generation. Advances in neural information processing systems, 30.
  21. OpenAI. 2021. GPT-3.5-turbo: Language Model. https://platform.openai.com/docs/guides/chatgpt.
  22. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311–318.
  23. ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer. arXiv preprint arXiv:2306.07799.
  24. Songmass: Automatic song writing with pre-training and alignment constraint. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 13798–13805.
  25. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, 2256–2265. PMLR.
  26. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502.
  27. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
  28. Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Features. arXiv preprint arXiv:2205.01821.
  29. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  30. Van de Cruys, T. 2020. Automatic poetry generation from prosaic text. In Proceedings of the 58th annual meeting of the association for computational linguistics, 2471–2480.
  31. Mixpoet: Diverse poetry generation via learning controllable mixed latent space. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 9450–9457.
  32. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI conference on artificial intelligence, volume 31.
  33. SummIt: Iterative Text Summarization via ChatGPT. arXiv preprint arXiv:2305.14835.
  34. DisCup: Discriminator cooperative unlikelihood prompt-tuning for controllable text generation. arXiv preprint arXiv:2210.09551.
  35. Generating Chinese CI with designated metrical structure. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, 7459–7467.
  36. Chinese poetry generation with recurrent neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 670–680.
  37. Controlled text generation with natural language instructions. arXiv preprint arXiv:2304.14293.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zhiyuan Hu (30 papers)
  2. Chumin Liu (2 papers)
  3. Yue Feng (55 papers)
  4. Anh Tuan Luu (69 papers)
  5. Bryan Hooi (159 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - ChorlingLau/PoetryDiffusion (5 stars)
  2. GitHub - ChorlingLau/PoetryDiffusion (5 stars)