PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning (2405.02501v2)

Abstract: LLMs are trained on massive text corpora, which are encoded with diverse personality traits. This triggers an interesting goal of eliciting a desired personality trait from the LLM, and probing its behavioral preferences. Accordingly, we formalize the persona elicitation task, aiming to customize LLM behaviors to align with a target persona. We present Persona In-Context Learning (PICLe), a novel persona elicitation framework grounded in Bayesian inference. At the core, PICLe introduces a new ICL example selection criterion based on likelihood ratio, which is designed to optimally guide the model in eliciting a specific target persona. We demonstrate the effectiveness of PICLe through extensive comparisons against baseline methods across three contemporary LLMs. Code is available at https://github.com/deeplearning-wisc/picle.

• The paper presents PICLe, a Bayesian in-context learning method that effectively aligns LLM behavior with targeted personas.
• It employs likelihood ratios and multi-persona decomposition to select guiding examples, significantly boosting alignment success rates.
• The framework offers practical AI customization for applications like customer service and education, while highlighting ethical considerations.

Deep Dive into Personifying LLMs via Bayesian Framework

Overview of Persona In-Context Learning (PICLe)

The paper introduces a novel method called Persona In-Context Learning (PICLe), which applies Bayesian inference to influence the behavior of LLMs toward a specific persona. This process involves adjusting the models' responses to better reflect personality traits such as agreeableness, conscientiousness, or even traits like narcissism.

Principles Behind PICLe

PICLe centers on the assumption that LLMs encode various personas due to their training on diverse datasets. The challenge is to elicit a specific persona based on a given input. Here’s a breakdown of the approach:

• Bayesian Inference Framework: PICLe employs Bayesian principles to estimate the distributions of different personas within an LLM.
• Likelihood Ratio for Example Selection: The crux of PICLe is selecting which illustrative examples from the training data will guide the LLM toward the desired persona. This is achieved through a likelihood ratio criterion, ensuring that the most indicative examples are chosen.
• Multi-Persona Decomposition: This concept allows for the treatment of each potential persona as part of a mixture of possible behaviors, which provides flexibility in modulating different traits.

Performance Assessment

The paper outlines comprehensive tests across several modern LLMs, showing that PICLe outperforms baseline methods substantially. For instance, on the Llama-2 model, PICLe reached an 88.1% success rate in aligning model outputs with the targeted personas, compared to a 65.5% success rate without using in-context learning examples.

Theoretical and Practical Implications

• Customizing AI Behavior: The ability to refine AI behavior has vast applications in areas like customer service, therapy, education, and entertainment.
• Understanding LLM Limitations: Exploring the range of behaviors that can be elicited from LLMs helps in understanding the limitations and inherent biases present due to their training data.
• Ethical Considerations: Manipulating AI personas raises questions about the ethical use of AI, as differing personas could potentially be used to mislead or manipulate users.

Future Prospects

Further exploration into diverse applications of PICLe, including those outside direct persona elicitation, could broaden its utility considerably. Additionally, addressing the potential ethical risks associated with pushing LLM behavior toward specific personas forms an essential part of ongoing and future discussions.

Challenges to Consider

The experimentation highlighted some limitations when non-RLHF (Reinforcement Learning from Human Feedback) models like GPT-J were used, indicating potential areas for improving how these models adapt to similar manipulations. The nuances between various LLM architectures also suggest a need for tailored approaches depending on the specific model in use.

Ultimately, the Persona In-Context Learning framework opens up new possibilities for customizing AI interactions and presents a structured way to probe the malleable behavior of LLMs. With further development and careful ethical considerations, such methodologies could revolutionize personalized AI applications.