Controllable Generation from Pre-trained Language Models via Inverse Prompting (2103.10685v3)

Abstract: Large-scale pre-trained LLMs have demonstrated strong capabilities of generating realistic text. However, it remains challenging to control the generation results. Previous approaches such as prompting are far from sufficient, which limits the usage of LLMs. To tackle this challenge, we propose an innovative method, inverse prompting, to better control text generation. The core idea of inverse prompting is to use generated text to inversely predict the prompt during beam search, which enhances the relevance between the prompt and the generated text and provides better controllability. Empirically, we pre-train a large-scale Chinese LLM to perform a systematic study using human evaluation on the tasks of open-domain poem generation and open-domain long-form question answering. Our results show that our proposed method substantially outperforms the baselines and that our generation quality is close to human performance on some of the tasks. Narrators can try our poem generation demo at https://pretrain.aminer.cn/apps/poetry.html, while our QA demo can be found at https://pretrain.aminer.cn/app/qa. For researchers, the code is provided in https://github.com/THUDM/InversePrompting.

• The paper introduces inverse prompting to significantly improve the controllability and relevance of text generation using pre-trained language models.
• It employs an iterative process during beam search where generated text forms an inverse prompt to better evaluate and guide output toward the initial prompt.
• Empirical studies in poem generation and Chinese long-form Q&A demonstrate enhanced fluency, innovation, and alignment with user intent compared to traditional methods.

Controllable Generation from Pre-trained LLMs via Inverse Prompting

The research paper titled "Controllable Generation from Pre-trained LLMs via Inverse Prompting" introduces a novel approach named inverse prompting to improve the controllability of text generation from large-scale pre-trained LLMs. The underlying issue addressed is the deviation in generated text from user-provided prompts, which hinders applications in real-world scenarios like story generation or question answering.

Inverse prompting involves an iterative process during beam search, where the generated text is used to construct an inverse prompt. This prompt is then employed to evaluate the likelihood of the original prompt using the pre-trained model. The paper demonstrates that inverse prompting significantly enhances the relevance and quality of model-generated texts, showing a closer alignment with human performance.

Key empirical studies include two primary tasks: open-domain poem generation and long-form question answering in Chinese. In both tasks, inverse prompting exhibits substantial improvements over traditional prompting and state-of-the-art models, such as CPM. In poem generation, for instance, experts rated the inverse prompting technique better in terms of relevance and innovation, thereby suggesting its applicability in generating meaningful text even when limited data on specific topics is available.

By leveraging the inherent bidirectionality in natural language understanding, inverse prompting aligns the generation process closer to desired outcomes without altering the pre-trained LLM architecture or requiring additional attribute models. The approach demonstrates improvements in fluency and informativeness in generated text, primarily because it reinforces prompt relevance as a critical guiding factor during the generation process.

Implications of this work extend to various AI applications requiring coherent long-form content generation, where maintaining topic adherence is crucial. This methodological advancement sets the ground for future research on improving model grounding and context retention in diverse text generation tasks.

Future work might explore broader linguistic applications or integrate more sophisticated reinforcement learning strategies. Additionally, investigating inverse prompting with other LLMs and on other languages could offer more comprehensive insights into its scalability and adaptability.

Overall, this paper makes a notable contribution to controllable text generation by proposing a method that elegantly utilizes the pre-trained LLM's strengths without necessitating extensive alterations or external modules.