RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting (2305.15685v2)

Abstract: LLMs have demonstrated impressive capabilities in creative tasks such as storytelling and E-mail generation. However, as LLMs are primarily trained on final text results rather than intermediate revisions, it might be challenging for them to perform text rewriting tasks. Most studies in the rewriting tasks focus on a particular transformation type within the boundaries of single sentences. In this work, we develop new strategies for instruction tuning and reinforcement learning to better align LLMs for cross-sentence rewriting tasks using diverse wording and structures expressed through natural languages including 1) generating rewriting instruction data from Wiki edits and public corpus through instruction generation and chain-of-thought prompting; 2) collecting comparison data for reward model training through a new ranking function. To facilitate this research, we introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting types expressed through natural language instructions. Our results show significant improvements over a variety of baselines. The public repository is available on GitHub under Google Research (https://github.com/google-research/google-research/tree/master/rewritelm).

• The paper presents novel strategies for instruction tuning and reinforcement learning, enabling effective cross-sentence rewriting.
• It employs innovative data generation methods using Wikipedia edits and synthetic prompts to create a diverse rewriting dataset.
• The model outperforms existing LLMs on the OpenRewriteEval benchmark, setting new standards in tone, style, and content preservation.

Overview of "RewriteLM: An Instruction-Tuned LLM for Text Rewriting"

"RewriteLM" presents a novel approach to text rewriting by instruction tuning LLMs for cross-sentence tasks. These tasks involve diverse rewordings and structural changes, essential for both professional and personal communication.

Methodology and Contributions

The paper outlines two main contributions:

1. New Strategies for Instruction Tuning and Reinforcement Learning: Recognizing that most rewriting studies remain confined to single-sentence edits, this paper proposes cross-sentence capabilities utilizing two innovative strategies:
   • Data Generation: By using Wikipedia edits and public corpora, the authors create a varied set of rewriting instructions. They leverage chain-of-thought (CoT) prompting and synthetic data generation to enhance dataset diversity.
   • Reward Model Training: Instead of traditionally relying on human labelers, a novel ranking function evaluates rewrites on key dimensions such as content preservation and linguistic variability. This function effectively automates data collection for reward modeling.
2. OpenRewriteEval Benchmark: The introduction of this benchmark represents a significant advance, covering numerous rewriting types beyond narrow task limitations. It is explicitly designed to evaluate cross-sentence rewriting, incorporating elements such as tone and style transfer.

Results and Evaluation

The authors conduct extensive empirical studies using the OpenRewriteEval benchmark, demonstrating that RewriteLM outperforms existing baselines, including state-of-the-art pre-trained models. Key findings include:

• Models like Rewrite-PaLM and Rewrite-PaLM 2, fine-tuned from foundational models, consistently improve upon these models’ performances across multiple rewriting tasks.
• Reinforcement learning, applied on top of supervised tuning, enhances model effectiveness further, culminating in the Rewrite-RL$_\text{r/w}$-PaLM 2, setting new performance standards for text rewriting.

The paper also provides a comprehensive comparison of various LLMs, including prominent models such as InstructGPT, highlighting RewriteLM’s superior ability to produce coherent and user-aligned text outputs.

Implications and Future Prospects

The implications of this research are twofold:

1. Practical Impact: RewriteLM offers a scalable solution for text rewriting, which has significant applications in content creation, editing, and adaptation across industries.
2. Theoretical Contributions: By advancing instruction-tuning methodologies and benchmarking novel cross-sentence rewriting frameworks, this research provides a foundation for extending LLM capabilities further.

Future work could explore transferring these methodologies to other language-dependent tasks, potentially enhancing versatility and user specificity in AI-driven text manipulation.

Conclusion

In summary, the paper poses noteworthy advancements in leveraging LLMs for more complex and user-centric rewriting tasks, culminating in a state-of-the-art model for cross-sentence rewrites. Its methodological innovations and OpenRewriteEval benchmark offer robust tools for continued advancements in AI text generation capabilities.