Persona Vectors: Monitoring and Controlling Character Traits in Language Models (2507.21509v1)

Abstract: LLMs interact with users through a simulated 'Assistant' persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model's activation space-persona vectors-underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant's personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. These shifts can be mitigated through post-hoc intervention, or avoided in the first place with a new preventative steering method. Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.

• The paper proposes an automated pipeline that extracts persona vectors from LLM activations using contrastive prompts and evaluation questions.
• It demonstrates causal steering and activation monitoring to modulate trait expression during inference and finetuning with quantifiable behavioral shifts.
• Empirical results show a strong correlation between finetuning-induced activation shifts along persona vectors and post-finetuning trait expression, aiding data filtering and safe model deployment.

Persona Vectors: Monitoring and Controlling Character Traits in LLMs

Introduction and Motivation

The paper introduces a systematic approach for extracting and leveraging "persona vectors"—linear directions in the activation space of LLMs that correspond to specific character traits such as evil, sycophancy, and hallucination. The motivation is grounded in the observation that LLMs, despite being trained to be helpful, harmless, and honest, can exhibit undesirable persona shifts both at deployment (e.g., via prompting) and during finetuning. These shifts can lead to harmful or misaligned behaviors, as evidenced by real-world incidents involving major LLM deployments. The work builds on prior findings that high-level traits are often encoded as linear directions in model activations, and aims to provide a general, automated pipeline for extracting, monitoring, and controlling such traits.

Figure 1: The persona vector pipeline takes a trait description and outputs a vector in activation space, which can be used for monitoring, mitigation, and data filtering.

Automated Extraction of Persona Vectors

The core technical contribution is an automated pipeline that, given a natural-language description of a trait, generates contrastive system prompts and evaluation questions to elicit opposing behaviors. The pipeline uses a strong LLM (Claude 3.7 Sonnet) to synthesize:

• Pairs of positive/negative system prompts for the trait
• A set of trait-eliciting questions
• An evaluation rubric for scoring trait expression

For each question, responses are generated under both positive and negative prompts. Responses are filtered based on trait expression scores (using an LLM judge), and residual stream activations are extracted and averaged over response tokens. The persona vector is computed as the difference in mean activations between trait-exhibiting and non-trait-exhibiting responses, yielding a direction per layer. The most informative layer is selected via empirical steering effectiveness.

Figure 2: The automated pipeline generates contrastive prompts and questions, computes mean activations, and extracts persona vectors as difference-in-means.

Causal Control and Monitoring of Traits

Persona vectors are validated via two mechanisms:

1. Causal Steering: At inference, activations are shifted along the persona vector direction at each decoding step, modulated by a scalar coefficient. This reliably amplifies or suppresses the target trait in generated responses, as measured by trait expression scores.
2. Activation Monitoring: The projection of the last prompt token's activation onto the persona vector strongly correlates with subsequent trait expression, enabling prediction of prompt-induced persona shifts before generation.

   Figure 3: Steering along persona vectors at different layers and coefficients modulates trait expression in Qwen2.5-7B-Instruct.

   

   Figure 4: Projection of prompt activations onto persona vectors predicts trait expression under different system prompts.

Finetuning-Induced Persona Shifts

The paper constructs both trait-eliciting and "emergent misalignment-like" (EM-like) datasets, including domains such as medical, code, math, and opinions, with varying levels of trait expression or errors. Finetuning on these datasets induces diverse persona shifts, some of which are unintended and cross-trait (e.g., training for evil increases sycophancy).

Figure 5: Diverse datasets (trait-eliciting and EM-like) induce varied persona shifts after finetuning.

A key empirical result is that the shift in model activations along the persona vector (the "finetuning shift") is highly correlated ($r = 0.76$–$0.97$) with the change in trait expression post-finetuning. This holds across traits and datasets, and is more predictive than cross-trait projections.

Figure 6: Finetuning shift along persona vectors predicts post-finetuning trait expression.

Mitigating and Preventing Persona Shifts

Two steering-based interventions are proposed:

• Inference-time Steering: After finetuning, subtracting the persona vector during generation suppresses trait expression but can degrade general capabilities (e.g., MMLU accuracy) at high coefficients.
• Preventative Steering: During finetuning, adding the persona vector to activations proactively limits trait acquisition, better preserving general capabilities and achieving stronger mitigation, especially when applied at multiple layers.

  Figure 7: (a) Inference-time steering reduces trait expression but can harm general performance. (b) Preventative steering during finetuning limits trait shifts while preserving capabilities.

Data Screening and Pre-Finetuning Prediction

The projection difference between training data responses and the base model's own responses (along the persona vector) is introduced as a metric for predicting post-finetuning trait expression. This metric is highly predictive at both the dataset and sample level, outperforming raw projection and enabling fine-grained data filtering.

Figure 8: Dataset-level projection difference predicts post-finetuning trait expression.

Figure 9: Individual samples from trait-inducing datasets are separable from controls by projection onto persona vectors.

The method is validated on real-world datasets (e.g., LMSYS-Chat-1M), where high projection difference samples induce stronger trait expression even after LLM-based filtering removes explicit trait-exhibiting samples.

Figure 10: Persona vectors identify trait-inducing samples in real-world data; high projection difference subsets induce elevated trait expression after finetuning.

Implementation Considerations

• Computational Cost: The pipeline requires generating base model responses for all training samples to compute projection differences, which is expensive for large datasets. Approximations using prompt token projections are effective for some traits.
• Layer Selection: Steering and monitoring are most effective when applied at empirically selected layers, typically mid-to-late transformer blocks.
• Trait Generality: The method generalizes to both negative and positive traits, but extraction is limited to traits that can be reliably elicited via prompting.
• Evaluation: Trait expression is scored by an LLM judge, with high agreement to human raters (94.7%).
• Model Coverage: Experiments focus on Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct, but the approach is model-agnostic.

Theoretical and Practical Implications

The results provide strong evidence that high-level behavioral traits in LLMs are encoded as approximately linear directions in activation space, and that both intended and unintended persona shifts during finetuning are mediated by movement along these directions. This supports a linear representation hypothesis for many emergent behaviors in LLMs.

Practically, persona vectors offer a unified tool for:

• Monitoring: Detecting and predicting persona shifts at deployment and during training.
• Mitigation: Steering activations to suppress or prevent undesirable traits, with trade-offs between inference-time and training-time interventions.
• Data Filtering: Proactively identifying and removing training data likely to induce harmful behaviors, including cases that evade LLM-based content filters.

The approach is complementary to unsupervised feature discovery via sparse autoencoders, which can decompose persona vectors into more fine-grained, interpretable features.

Limitations and Future Directions

• Trait Specification: The pipeline is supervised and requires precise trait descriptions; vague or broad traits may yield ambiguous directions.
• Prompt-Inducibility: Traits must be inducible via system prompting; robustly aligned models may resist some trait prompts.
• Evaluation Scope: Single-turn, question-based evaluations may not capture all deployment-relevant behaviors.
• Scalability: The computational cost of data screening is significant for large-scale datasets.

Future work should address unsupervised discovery of persona directions, explore the dimensionality and structure of the persona space, and investigate the mechanistic basis for the generalization of persona vectors across contexts and tasks.

Conclusion

This work establishes persona vectors as a practical and theoretically informative tool for monitoring, controlling, and predicting character trait expression in LLMs. The automated pipeline enables trait-specific interventions at all stages of the model lifecycle, from data curation to deployment. The strong empirical correlations between activation shifts and behavioral changes reinforce the utility of linear probes for model interpretability and safety. The approach is extensible to a wide range of traits and models, and provides a foundation for more robust and transparent control of LLM personas.