Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Conversation Style Transfer using Few-Shot Learning (2302.08362v2)

Published 16 Feb 2023 in cs.CL

Abstract: Conventional text style transfer approaches focus on sentence-level style transfer without considering contextual information, and the style is described with attributes (e.g., formality). When applying style transfer in conversations such as task-oriented dialogues, existing approaches suffer from these limitations as context can play an important role and the style attributes are often difficult to define in conversations. In this paper, we introduce conversation style transfer as a few-shot learning problem, where the model learns to perform style transfer by observing only a few example dialogues in the target style. We propose a novel in-context learning approach to solve the task with style-free dialogues as a pivot. Human evaluation shows that by incorporating multi-turn context, the model is able to match the target style while having better appropriateness and semantic correctness compared to utterance/sentence-level style transfer. Additionally, we show that conversation style transfer can also benefit downstream tasks. For example, in multi-domain intent classification tasks, the F1 scores improve after transferring the style of training data to match the style of the test data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Stuart Axelbrooke. 2017. Customer support on twitter.
  2. Gpt-neox-20b: An open-source autoregressive language model. Challenges & Perspectives in Creating Large Language Models, page 95.
  3. Language (technology) is power: A critical survey of “bias” in nlp. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476.
  4. Large scale multi-actor generative dialog modeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 66–84.
  5. Olá, bonjour, salve! xformal: A benchmark for multilingual formality style transfer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3199–3216.
  6. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  7. Contextual text style transfer. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2915–2924.
  8. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29.
  9. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250–259.
  10. Neural approaches to conversational ai. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1371–1374.
  11. Structuring latent spaces for stylized response generation. arXiv preprint arXiv:1909.05361.
  12. Meet your favorite character: Open-domain chatbot mimicking fictional characters with only a few utterances. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5114–5132, Seattle, United States. Association for Computational Linguistics.
  13. The curious case of neural text degeneration. In International Conference on Learning Representations.
  14. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155–205.
  15. Klaus Krippendorff. 2004. Measuring the reliability of qualitative text analysis data. Quality and quantity, 38:787–800.
  16. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  17. Politeness transfer: A tag and generate approach. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1869–1881.
  18. Plug and play autoencoders for conditional text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6076–6092.
  19. Sentence bottleneck autoencoders from transformer language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1822–1831.
  20. Recent advances in deep learning based dialogue systems: A systematic survey. Artificial Intelligence Review, pages 1–101.
  21. Multi-task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1008–1021.
  22. Ellie Pavlick and Joel Tetreault. 2016. An empirical analysis of formality in online communication. Transactions of the Association for Computational Linguistics, 4:61–74.
  23. Style transfer through back-translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 866–876.
  24. Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129–140.
  25. A recipe for arbitrary text style transfer with large language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 837–848.
  26. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.
  27. Textsettr: Few-shot text style extraction and tunable targeted restyling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3786–3800.
  28. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30.
  29. Educating text autoencoders: Latent representation guidance via denoising. In International conference on machine learning, pages 8719–8729. PMLR.
  30. Multiple-attribute text style transfer. arXiv preprint arXiv:1811.00552.
  31. Prompt-and-rerank: A method for zero-shot and few-shot arbitrary textual style transfer with small language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2195–2222.
  32. Style control for schema-guided natural language generation. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 228–242.
  33. Harnessing pre-trained neural networks with rules for formality style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3573–3578.
  34. Unsupervised text style transfer using language models as discriminators. Advances in Neural Information Processing Systems, 31.
  35. Jerrold H Zar. 2005. Spearman rank correlation. Encyclopedia of biostatistics, 7.
  36. Recent advances and challenges in task-oriented dialog systems. Science China Technological Sciences, 63(10):2011–2027.
  37. Style transfer as unsupervised machine translation. arXiv preprint arXiv:1808.07894.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Shamik Roy (10 papers)
  2. Raphael Shu (24 papers)
  3. Nikolaos Pappas (188 papers)
  4. Elman Mansimov (20 papers)
  5. Yi Zhang (994 papers)
  6. Saab Mansour (32 papers)
  7. Dan Roth (222 papers)
Citations (6)