Persona-Pruner: Persona-Specific LLM Pruning

Updated 6 July 2026

Persona-Pruner is a framework that extracts a persona-specific subnetwork within a large language model using a single textual persona description to tailor role-playing performance.
The method employs structured FFN pruning and persona-driven data synthesis to ensure the subnetwork retains high role-playing fidelity alongside general capabilities.
Empirical results demonstrate that Persona-Pruner significantly reduces performance drop compared to generic pruning, with improvements on benchmarks like RoleBench and Alpaca-P.

Searching arXiv for Persona-Pruner and closely related persona-pruning/personalization papers. Persona-Pruner is a pruning framework for turning a dense, general-purpose instruction-tuned LLM into a lightweight persona-specialized role-playing model using only a single textual persona description. It is motivated by the claim that a specific character identity relies on only a fraction of the model’s total capacity, and it operationalizes that claim by learning a persona-specific subnetwork in the model’s feed-forward network (FFN) intermediate dimensions while attempting to preserve both role-playing fidelity and much of the original model’s general capability (Kim et al., 12 Jun 2026).

1. Problem setting and conceptual basis

Persona-Pruner studies persona-conditioned role-playing in the form

$p_P(q, a) \coloneqq p_\theta(a \mid P, q),$

where $P$ is a persona description, $q$ is a user query, and $\theta$ denotes the parameters of a pretrained LLM. The objective is not to train a new role-playing model from scratch, but to extract a smaller subnetwork that serves one specific persona effectively. The target mask $\mathbf{M}$ is defined through the persona-conditioned negative log-likelihood

$\mathcal{L}(\mathbf{M}; \theta)=\mathbb{E}_{(q,a) \sim p_P}\big[-\log p_{(\theta \odot \mathbf{M})}(a\mid P,q)\big].$

This formulation treats persona specialization as structured subnetwork discovery rather than prompt-only control (Kim et al., 12 Jun 2026).

The practical motivation is deployment inefficiency. The paper explicitly situates the method in settings such as large NPC populations in games, persistent multi-agent ecosystems, personalized assistants, and user simulators or many concurrent digital personas. In such regimes, a dense generalist model is structurally mismatched to a narrow role-playing objective, since a full model copy or a full dense forward pass is expensive in memory, inference FLOPs, latency, and serving complexity. Persona-Pruner therefore asks whether a single persona can be served by a smaller persona-specific subnetwork instead of a full dense model (Kim et al., 12 Jun 2026).

A central negative result underlies the method: naively pruning LLMs often severely degrades role-playing performance for a specific persona. Generic pruning methods preserve broad utility but do not distinguish redundant knowledge from persona-critical traits, and the paper reports characteristic failure modes such as assistant-like responses, refusals, repetition, and incomplete generations. Persona-Pruner is designed precisely to make pruning persona-aware rather than generic (Kim et al., 12 Jun 2026).

2. Persona-driven data synthesis

The framework begins with a persona-driven calibration stage that constructs a synthetic dataset $\mathcal{D}_{\text{syn}}$ from a single persona description $P_{\text{target}}$ , a generic instruction dataset $\mathcal{D}_{\text{base}}$ , and an external rewriting model. The key premise is that persona influence is query-dependent: some instructions are largely persona-invariant, while others expose viewpoint, expertise, value judgments, or style. Persona-Pruner therefore filters the base instruction set before pruning (Kim et al., 12 Jun 2026).

For a query $q$ , the paper defines a Persona-Sensitivity Score using hidden states of the last input token across transformer blocks: $P$ 0 Here, $P$ 1 denotes a random reference persona, $P$ 2 the number of transformer blocks, and $P$ 3 the number of reference personas. High-scoring queries are retained: $P$ 4 with $P$ 5 chosen so that

$P$ 6

Thus the method keeps the top 40% of base examples by persona sensitivity (Kim et al., 12 Jun 2026).

After filtering, each retained answer is rewritten into persona-aligned form. The paper uses Llama-3.1-70B-Instruct to generate a rewritten answer $P$ 7 that preserves the semantic content of the original answer while adopting the target persona’s tone, style, worldview, and speaking manner. The resulting synthetic dataset is

$P$ 8

This dataset is then used both for mask learning and for post-pruning recovery finetuning. An ablation shows that rewriting is more important than filtering, but both matter: on Llama-3.2-3B at 25% sparsity without recovery, the full method reaches 80.19 on Alpaca-P, compared with 78.80 for rewriting only, 39.36 for filtering only, and 32.50 for neither (Kim et al., 12 Jun 2026).

3. Persona-specific subnetwork discovery

Persona-Pruner prunes structured FFN intermediate dimensions rather than attention heads, hidden-state dimensions, or unstructured weights. For hidden input $P$ 9, the FFN is written as

$q$ 0

and the masked FFN becomes

$q$ 1

This is equivalent to pruning corresponding columns of $q$ 2 and rows of $q$ 3. The stated rationale is that FFN layers contain most parameters in transformer LLMs, structured FFN pruning is hardware-friendly, and the granularity is fine enough to preserve nuanced persona circuits (Kim et al., 12 Jun 2026).

For each FFN layer, the method learns a real-valued score vector $q$ 4, and derives a hard binary mask by top- $q$ 5 selection: $q$ 6 Because top- $q$ 7 is non-differentiable, Persona-Pruner uses the Straight-Through Estimator (STE), with hard binary top- $q$ 8 masks in the forward pass and

$q$ 9

in the backward pass. During mask learning, the base model weights are frozen and only the mask scores are optimized (Kim et al., 12 Jun 2026).

The practical optimization target is token-level cross-entropy on the synthetic persona dataset: $\theta$ 0 After pruning, the method can apply recovery finetuning on the same synthetic dataset. The paper reports that, unlike pruning baselines, Persona-Pruner’s role-playing performance tends to improve monotonically with recovery finetuning. It also reports that STE outperforms Gumbel-Softmax and Soft-TopK in the optimization strategy ablation, with Alpaca-P scores of 80.19, 75.77, and 68.55, respectively, on Llama-3.2-3B at 25% sparsity (Kim et al., 12 Jun 2026).

4. Evaluation and empirical findings

The evaluation uses RoleBench, a new synthetic benchmark called Alpaca-P, and general benchmarks including OpenBookQA (OBQA), PIQA, and, in a larger-scale appendix experiment, MMLU. RoleBench is adapted by sampling 10 roles, expanding role names into detailed persona descriptions using Llama-3.1-70B-Instruct, and using 20 common general-role prompts across roles. Alpaca-P consists of 10 synthetic personas sampled from FSPO, with persona-sensitive queries selected from Alpaca and answers rewritten in persona voice. Role-playing performance is measured by an LLM-as-a-judge score from 0 to 100, using GPT-4o-mini in the main evaluation, and human evaluation ranks outputs on anthropomorphism, character fidelity, and storyline quality (Kim et al., 12 Jun 2026).

The strongest headline claim is that Persona-Pruner preserves role-playing performance substantially more effectively than existing state-of-the-art LLM pruning techniques, reducing the performance drop from the dense model by up to 93.8% over the strongest baseline on RoleBench in LLM-as-a-judge score. On Llama-3.2-3B-Instruct, the dense model scores 83.35 on RoleBench and 84.86 on Alpaca-P. At 25% sparsity without recovery, Persona-Pruner scores 81.20 on RoleBench and 80.19 on Alpaca-P, while the best baseline on RoleBench is LLM-Pruner at 48.43. At 25% sparsity with recovery, Persona-Pruner reaches 84.26 on RoleBench, exceeding the dense model’s 83.35. At 50% sparsity with recovery, it reaches 70.37 on RoleBench and 78.24 on Alpaca-P. On Llama-3.1-8B-Instruct, the dense model scores 88.82 on RoleBench and 86.66 on Alpaca-P; Persona-Pruner at 25% sparsity without recovery scores 82.62 and 81.86, and at 50% sparsity with recovery scores 80.01 and 81.73 (Kim et al., 12 Jun 2026).

General capability is retained to a meaningful extent. For Llama-3.2-3B at 25% sparsity, dense OBQA/PIQA is 0.36/0.76, while Persona-Pruner without recovery gives 0.36/0.71 and with recovery 0.36/0.69. On Qwen2.5-32B-Instruct at 50% sparsity, the appendix reports Alpaca-P/MMLU results of 83.10/59.01 for Persona-Pruner, compared with 71.01/53.45 for LLM-Pruner and 47.78/37.86 for Adapt-Pruner (Kim et al., 12 Jun 2026).

The analysis section supports the existence of persona-specific subnetworks. For Llama-3.2-3B at 25% sparsity, pairwise Jaccard similarities of layer-wise binary masks across personas show Pearson correlation $\theta$ 1, $\theta$ 2 across two independent seeds, indicating that the discovered persona-specific subnetworks are not random seed artifacts. Structural-semantic alignment between mask similarity and persona embedding similarity has correlation $\theta$ 3, $\theta$ 4, meaning semantically similar personas tend to retain similar subnetworks. The paper also reports that persona-specific masks consistently outperform a single global mask trained across all 10 personas (Kim et al., 12 Jun 2026).

Human evaluation further reinforces the role-playing findings. Across 50 participants and 20 sampled Alpaca-P instances, average rank is 1.59 for Persona-Pruner, compared with 2.59 for LLM-Pruner, 2.73 for Adapt-Pruner, 3.69 for Depth Pruning, and 4.41 for SliceGPT. Dimension-wise, Persona-Pruner achieves 1.59 on anthropomorphism, 1.50 on character fidelity, and 1.67 on storyline quality. The paper reports a Spearman rank correlation of $\theta$ 5 between human judges and LLM judges (Kim et al., 12 Jun 2026).

Persona-Pruner sits at the intersection of persona conditioning, selective persona activation, and personalized compression. Earlier dialogue work showed that personalization need not require explicit persona text at test time: one line of work learns a detector $\theta$ 6 so that dialogue generation can proceed as $\theta$ 7, predicting either self persona or partner persona from dialogue history (Zhou et al., 2021). Another line of work represents each persona as a personalized prefix attached to a frozen pretrained LLM, with less than 0.1% trainable parameters per persona, trading explicit persona descriptions for compact continuous vectors (Han et al., 2023). Persona-Pruner differs from both by specializing the model structurally through pruning rather than by adding latent persona modules (Zhou et al., 2021).

A second relevant thread treats persona use as a routing problem rather than a universally beneficial intervention. PRISM shows that static expert prompting can improve alignment but damage accuracy, and implements a learned gate that decides when not to activate a distilled persona adapter; the paper explicitly frames this as conditional persona suppression and conditional adapter activation rather than classic multi-persona routing (Hu et al., 19 Mar 2026). Persona-Pruner shares the selectivity premise, but the unit of specialization is different: PRISM gates a LoRA adapter at inference time, whereas Persona-Pruner extracts a persona-specific FFN subnetwork offline from a single description (Kim et al., 12 Jun 2026).

A third thread comes from personalized pruning outside dialogue. Prototype-based Personalized Pruning (PPP) trains a global model offline and then instantiates a personalized pruned model on the fly without finetuning or retraining, using a prototype to decide which channels remain active for a person or identity (Kim et al., 2021). Persona-Pruner is analogous at a conceptual level in that both methods seek a user- or persona-conditioned subnetwork, but Persona-Pruner operates on open-weight instruction-tuned LLMs for role-playing and uses persona-driven synthetic supervision rather than a prototype computed from enrollment samples (Kim et al., 2021).

6. Limitations, deployment implications, and open questions

The framework has several explicit limitations. It requires weight access and is therefore a method for open-weight models rather than closed APIs. Its main best-performing pipeline uses an external rewriting model, Llama-3.1-70B-Instruct, to synthesize persona-aligned answers, although the self-synthesis ablation still reaches 75.20 on Alpaca-P. Evaluation is centered mainly on judged text outputs and ranking-style human evaluation, with only one multi-turn consistency experiment. The method learns one subnetwork per persona, which raises unresolved questions about storage, composition, overlap exploitation, and dynamic routing across many personas. It also focuses on FFN dimensions only, leaving attention heads, residual streams, and depth allocation outside the pruning space (Kim et al., 12 Jun 2026).

These constraints delimit what Persona-Pruner currently establishes. It shows that a single persona can be served by a specialized, structured FFN subnetwork learned from one natural-language persona specification, and that persona-aware pruning can outperform generic pruning by large margins on role-playing benchmarks. It does not yet provide a full deployment framework for multi-mask sharing or dynamic persona switching, nor does it claim that FFN-only pruning is the unique locus of persona specialization. A plausible implication is that later work could combine persona-sensitive calibration, persona-specific subnetworks, and query-conditional routing, but the paper itself stops at persona-aware subnetwork extraction and recovery finetuning (Kim et al., 12 Jun 2026).