Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4 (2312.16171v2)

Abstract: This paper introduces 26 guiding principles designed to streamline the process of querying and prompting LLMs. Our goal is to simplify the underlying concepts of formulating questions for various scales of LLMs, examining their abilities, and enhancing user comprehension on the behaviors of different scales of LLMs when feeding into different prompts. Extensive experiments are conducted on LLaMA-1/2 (7B, 13B and 70B), GPT-3.5/4 to verify the effectiveness of the proposed principles on instructions and prompts design. We hope that this work can provide a better guide for researchers working on the prompting of LLMs. Project page is available at https://github.com/VILA-Lab/ATLAS.

• The paper presents 26 prompt engineering guidelines that significantly enhance the clarity and precision of LLM responses.
• Empirical evaluations using the ATLAS benchmark demonstrated quality improvements of 57.7% and accuracy gains of 67.3% across models.
• The research lays a foundation for integrating structured prompt strategies into LLM fine-tuning and diverse automated applications.

Introduction

In the field of natural language processing, the interaction with LLMs is refined through the art of prompt engineering, the process of computer-assisted or manual crafting of instructions to extract specific responses from the models. This research explores the complexity of prompting LLMs by introducing a set of principled instructions that aim to improve the prompting process for users.

The Principles of Prompt Engineering

The paper introduces 26 principles categorized into five groups that serve to guide users in crafting prompts that would yield more precise, informative, and unbiased responses from the model. These principles encompass aspects such as prompt structure, clarity, specificity, user interaction, content style, and strategies for dealing with complex tasks and coding prompts.

An example of such a principle is to be direct and bypass polite addendums (e.g., "please," "thank you") to enhance prompt conciseness. Another encourages including the intended audience in the prompt, specifying who the response is for, such as experts or children, to tailor the LLM's output.

Empirical Results

Extensive experiments on different models, ranging from small 7B parameter models to the larger GPT-3.5 and 4, demonstrated that applying these principles led to a significant improvement in both the quality and accuracy of LLM responses. Specifically, on the ATLAS benchmark—a dataset created to evaluate principled prompt effects—the tailored prompts elevated the quality and accuracy of LLM responses by an average of 57.7% and 67.3%, respectively.

Conclusion and Future Work

The effectiveness of structured prompts in enhancing response quality from pre-trained LLMs is evident from this research. Future studies could focus on integrating these principles into regular LLM operations through fine-tuning or diversified prompting methods. This work paves the way for users to steer LLM outputs more effectively, leading to interactions with AI that are increasingly reliable and context-aware.