SoulBench: Deterministic LLM Persona Profiling

• SoulBench is a dataset protocol that underpins the Soul Engine by enabling deterministic personality control in LLMs using orthogonal latent trait representations.
• It employs a dual-head architecture with frozen reasoning layers and trainable psychometric heads to segregate core intelligence from dynamic persona injection.
• Dynamic contextual sampling in SoulBench leads to robust generalization and profiling precision, achieving a mean squared error as low as 0.0113.

The Soul Engine is a framework for personalized LLMs that formulates personality traits as orthogonal linear subspaces within model latent spaces, enabling deterministic control and profiling of persona without performing global weight updates. Developed to address the stability–plasticity dilemma in LLM personalization, the Soul Engine introduces both a new mathematical foundation—the Linear Representation Hypothesis—and a practical method using a dual-head architecture on a frozen transformer backbone, in conjunction with a dynamically sampled persona dataset, SoulBench. The framework achieves high-precision trait profiling and robust behavioral steering, presenting a mathematically grounded alternative to probabilistic prompt engineering and conventional fine-tuning (Wang, 8 Dec 2025).

1. Rationale: The Stability–Plasticity Dilemma in LLM Personalization

Prevailing methods for aligning LLMs to specific personas, such as Supervised Fine-Tuning (SFT) and Low-Rank Adaptation (LoRA), conceptualize persona as a narrow stylistic distribution learned via stochastic weight updates. This process is subject to an “alignment tax,” where reasoning and general knowledge degrade due to catastrophic forgetting, often observable as declines on standard evaluation benchmarks (e.g., MMLU). In-Context Learning (ICL) and prompt-based methods avoid weight updates but suffer from instruction “drift,” resulting in persona dilution or “catastrophic amnesia” during extended dialogues.

The Soul Engine circumvents these issues by eschewing global weight updates on the reasoning circuits. Instead, it posits that personality can be cleanly disentangled from reasoning as orthogonal components within the latent space, activated or deactivated deterministically via vector overlays. This architecture preserves the foundational intelligence of the model while affording stable and controllable style modulation.

2. The Linear Representation Hypothesis

The framework postulates that each of the Big Five OCEAN psychometric traits—Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism—corresponds to an orthogonal linear subspace in the final hidden activations of a transformer model. Specifically:

• Let $e \in \mathbb{R}^d$ denote the final hidden embedding for a given anchor input.
• The psychometric dimension is captured by a projection matrix $W_{\mathrm{psy}} \in \mathbb{R}^{5 \times d}$, whose rows $\{w_1, \dots, w_5\}$ each define a trait subspace.
• The orthogonality constraint is enforced as $W_{\mathrm{psy}} W_{\mathrm{psy}}^\top = I_5$.
• Given a ground-truth personality trait vector $y_{\text{truth}} \in [0,1]^5$, the model’s prediction is $\hat{y} = W_{\mathrm{psy}} e$.

To regularize the separation of trait subspaces, the loss function incorporates

$$L_{\mathrm{Orth}} = \| W_{\mathrm{psy}}W_{\mathrm{psy}}^\top - I_5 \|_F^2,$$

which penalizes non-orthogonality between personality directions.

3. Dual-Head Architecture and Latent Persona Manipulation

The architecture deploys a stratified freezing strategy atop the Qwen-2.5 transformer backbone:

• Layers $0\dots19$ ($\theta_\mathrm{frozen}$) are frozen to preserve syntax and core reasoning manifolds.
• Upper layers $20\dots23$ and final normalization heads ($\theta_\mathrm{active}$) are trainable, allowing psychometric probing.
• For an anchor input $A$, the final hidden state is $$e = \mathcal{F}_\theta(\mathrm{frozen}+\mathrm{active})(A) \in \mathbb{R}^{896}$$.
• A two-headed output structure is used:
  • Identity Head (contrastive): $$z_\mathrm{id} = P_\mathrm{id}(e) \in \mathbb{R}^{256}$$ (2-layer MLP).
  • Psychometric Head (linear): $$\hat{y} = P_\mathrm{psy}(e) = W_\mathrm{psy} e \in \mathbb{R}^5$$.
• For trait injection, a desired score vector $y^*$ is mapped into the latent space as $v_\mathrm{psy} = W_\mathrm{psy}^\top y^*$, which can then be added to intermediate activations, preserving $\theta_\mathrm{frozen}$ invariance.

Component	Operation	Dimensionality
Final hidden state	$e = \mathcal{F}_\theta(A)$	$\mathbb{R}^{896}$
Identity head	$z_\mathrm{id} = P_\mathrm{id}(e)$	$\mathbb{R}^{256}$
Psychometric head	$\hat{y} = W_\mathrm{psy} e$	$\mathbb{R}^{5}$
Persona vector	$v_\mathrm{psy} = W_\mathrm{psy}^\top y^*$	$\mathbb{R}^{896}$

This architecture enables “deterministic steering”: during inference, desired personalities are injected via vector arithmetic, specifically at residual streams of selected layers. The resulting hidden state modification is $$h' = h + \alpha \cdot (v_\mathrm{steer}/\|v_\mathrm{steer}\|)$$, with $\alpha$ tuning personality strength.

4. SoulBench Dataset and Dynamic Contextual Sampling

The SoulBench protocol employs dynamic contextual sampling to avoid overfitting on exogenous linguistic content:

• For each character $c$, $D_c = \{s_1, \ldots, s_M\}$ comprises all sentences attributed to $c$.
• At each training iteration $t$, $k$ random sentences are concatenated to form anchor $A_t = \mathrm{Concat}(s_{i_1}, \ldots, s_{i_k})$, with $k=3$.
• This approach induces a virtual dataset of size $C(M,3)\gg M$, compelling the model to capture stylistic invariants.
• Ground-truth OCEAN scores $y_{\text{truth}}$ are assigned using the Doubao-Seed-1.6 teacher model, prompted by complete character profiles for psychological consistency.

This sampling scheme is fundamental for robust generalization, focusing the psychometric head on structural persona features, independent of content memorization.

5. Profiling Accuracy, Disentanglement, and Persona Injection

The framework achieves high-precision personality profiling and orthogonal disentanglement:

• The model realized a Mean Squared Error (MSE) of $0.0113$ on validation, corresponding to $\approx 99\%$ precision relative to the ground-truth teacher.
• T-SNE visualizations of 1,000 character embeddings revealed distinct, continuous gradients in each OCEAN trait and minimal cluster overlap, confirming the geometric orthogonality in $W_\mathrm{psy}$.
• Zero-shot personality injection is enabled by defining a neutral mean embedding $\mu_N$ and a target persona mean $\mu_T$, with $v_\mathrm{steer} = \mu_T - \mu_N$. For deterministic style shifts, the model modifies activation streams as $$h' = h + \alpha \cdot (v_\mathrm{steer}/\|v_\mathrm{steer}\|)$$ (e.g., Neutral $\rightarrow$ Villain).
• Parameter sweeps showed middle layers $14{\text{–}}16$ and $\alpha\in[6.0,8.0]$ yield maximal persona adherence (Villainy score $\uparrow70\%$) and high compositional coherence ($\uparrow95\%$), as compared to baselines.

6. Ablation Results and Regularization Effects

Key findings from ablation studies include:

• Freezing at least $20/24$ layers sustains low MSE ($\approx 0.011$) in trait profiling without loss to core reasoning, while further fine-tuning increases risk of knowledge degradation.
• Omitting the orthogonality regularizer $L_{\mathrm{Orth}}$ escalates trait manifold entanglement and raises MSE by $\sim30\%$.
• Layer and strength grid searches identified injection regimes for maximal persona expressivity with minimal off-target degradation.

These studies delineate the hyperparameter regimes required for stable, disentangled persona control and validate the structural separation mandated by the Linear Representation Hypothesis.

7. Mathematical Justification and Safety Considerations

By formulating personality as a linear manifold orthogonal to core reasoning circuits, the Soul Engine avoids destructive model updates. Deterministic latent interventions via vector arithmetic confer precision and reversibility unattainable with stochastic prompt reformulation.

From a safety perspective:

• Malicious behavioral directions, such as the “Dark Triad,” can be identified within $W_\mathrm{psy}$ and subtracted at runtime (“Safety Interceptor”), preemptively neutralizing harmful predispositions in generated content.
• Latent-level guardrails on the persona manifold are asserted to offer superior robustness and coverage compared to surface-level token or rule-based filtering, as interventions operate directly at the level of semantic intent.

A plausible implication is that treating personality as a geometric latent construct enables both rigorous behavioral guarantees and flexible personalization, challenging the primacy of fine-tuning for secure and expressive agent alignment (Wang, 8 Dec 2025).