CelebPersona: Celebrity-Based Personalization

• CelebPersona is a research paradigm using publicly known celebrity identities as semantically rich anchors to enable generative personalization and interpretable trait analysis.
• The approach integrates diffusion models, dialogue-based persona inference, and multimodal datasets to achieve precise identity representation with minimal learnable parameters.
• It supports applications in visual synthesis, personalized alignment, and causal representation learning while addressing ethical, privacy, and bias concerns.

CelebPersona denotes a family of research constructions that use celebrities or other public figures as externally grounded persona anchors for generative personalization, multimodal trait analysis, and preference alignment. In recent work, the term appears most explicitly as the CelebPersona component of PersonaX, a multimodal dataset of 9444 public figures with facial, biographical, and LLM-inferred Big Five behavior annotations (Li et al., 14 Sep 2025). Closely related work uses famous individuals to study cold-start personalized alignment through interpretable preference descriptions (Tang et al., 19 May 2025), while a diffusion-model line describes fast one-photo identity insertion as enabling “CelebPersona-style personalization” through a celebrity-derived basis in the CLIP text-embedding space (Yuan et al., 2023). Across these strands, celebrities function as high-visibility identities with abundant public evidence, making them suitable for controlled studies of persona representation.

1. Research uses and conceptual scope

The current literature does not present a single canonical definition of CelebPersona. Instead, it uses celebrity-centered personas in several technically distinct ways.

Strand	Core object	Representative source
Diffusion personalization	Insert a new human identity into Stable Diffusion from one facial photo	(Yuan et al., 2023)
Multimodal trait dataset	Public-figure identities with face, biography, and trait annotations	(Li et al., 14 Sep 2025)
Personalized alignment	Preference-conditioned response generation for famous individuals	(Tang et al., 19 May 2025)
Dialogue and character modeling antecedents	Persona embeddings from conversational or narrative evidence	(Chu et al., 2018, Kim et al., 2018, Suwajanakorn et al., 2015)

A common methodological feature is the use of public figures as semantically rich reference identities. In the dataset setting, this enables multimodal linkage between faces, biographies, and textual trait analyses. In the alignment setting, it makes background and preferences more verifiable than synthetic personas. In the generative-image setting, celebrity names provide a pre-trained semantic manifold of “personness” that can be exploited for compositional identity insertion. This suggests that CelebPersona is best understood as an umbrella paradigm rather than a single benchmark.

2. One-photo identity insertion in diffusion models

In the diffusion-model literature, CelebPersona refers to rapid personalization of a pretrained text-to-image model by constraining a new identity to a celebrity-derived subspace of the CLIP text encoder used by Stable Diffusion (Yuan et al., 2023). The method begins by crawling approximately 1,500 celebrity names and retaining $m=691$ names that yield identity-consistent outputs and can interact with other celebrities and objects. Each name is tokenized, only the first two embeddings are kept, and two token sets are formed: $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$ For each $C_k$, the method computes a mean $\overline{C}_k$ and PCA basis $B_k=[b_k^1,\ldots,b_k^p]$ with $p=512$. A new identity is then represented as

$$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$

Because there are two $512$-dimensional coefficient vectors, each identity uses exactly $2p=1024$ learnable parameters.

The operational claim is that previous personalization methods often drift outside the native semantic manifold of human concepts, weakening compositionality and multi-identity interactions. The celeb-basis construction addresses this by forcing a new identity to remain inside a subspace spanned by celebrity-name embeddings that the pretrained model already composes well. The paper keeps the CLIP transformer $\tau_{\text{text}}$, the UNet $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$0, and the celeb basis frozen, and trains only a single-layer MLP that maps a 512-dimensional ArcFace feature $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$1 from one aligned facial photograph to the coefficient vectors $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$2. Training uses only the standard diffusion denoising loss,

$$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$3

with prompts such as “A photo of a face of V* person” and related variants.

The implementation is intentionally minimal. All parameters other than the MLP-induced $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$4-dimensional coefficient representation are locked. Training uses color jitter, random resize or scaling, random shifts, and horizontal flip, but no class-specific regularization dataset. The reported recipe is learning rate $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$5, batch size $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$6, and $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$7 steps, requiring approximately $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$8 minutes on a single NVIDIA A100 40GB GPU for one identity. Joint training of $$C_1=[v_1^1,\ldots,v_1^m], \qquad C_2=[v_2^1,\ldots,v_2^m].$$9 identities uses approximately $C_k$0 steps and approximately $C_k$1 minutes total, while storing only $C_k$2 float16 coefficients per identity, or $C_k$3–$C_k$4 KB.

The paper reports the following quantitative comparison on 2k synthetic StyleGAN faces.

Method	Prompt / Identity / Detect	Params / Time
Textual Inversion	0.1635 / 0.2958 / 92.86%	1,536 / 24 min
DreamBooth	0.2002 / 0.0512 / 54.76%	$C_k$5 / 16 min
Custom Diffusion	0.2608 / 0.1385 / 80.39%	$C_k$6 / 12 min
Ours	0.2545 / 0.2072 / 84.78%	1,024 / 3 min

Here, prompt alignment is measured by CLIPScore, identity similarity by ArcFace, and detect rate by a face detector. A user study with 100 users, 200 images, and 60k ratings favored the celeb-basis method for text alignment and identity. Ablations further report that removing the celeb basis collapses compositionality, using fewer names or skipping filtering harms performance, a two-basis first-name/last-name design outperforms a flattened basis, and $C_k$7 provides the best trade-off among $C_k$8. The method also reports successful multi-subject prompts such as “A photo of V1 and V2 baking cookies in a kitchen,” where prior personalization approaches were described as unreliable.

3. Dialogue-derived persona inference

A second CelebPersona line treats persona as something inferable from conversational evidence. The Attentive Memory Network paper defines persona as a person’s social role, categorically recognizable from conversations, beliefs, and actions, and the provided adaptation states that CelebPersona can be framed as a conversational personalization setting using interviews, talk shows, podcasts, and social media exchanges as dialogue-like data (Chu et al., 2018). In the source model, a dialogue snippet $C_k$9 contains a character’s own lines $\overline{C}_k$0, contextual narration $\overline{C}_k$1, and other interlocutors’ lines $\overline{C}_k$2. Tokens are embedded with 300-dimensional GloVe vectors and encoded by a bidirectional GRU with hidden size 200 per direction, yielding $\overline{C}_k$3.

The architecture uses token-level attention within each sequence,

$\overline{C}_k$4

followed by attention across multiple snippets,

$\overline{C}_k$5

The resulting summaries $\overline{C}_k$6 are combined into a persona embedding

$\overline{C}_k$7

Prior knowledge enters through a read-only Knowledge-Store memory indexed by trope descriptions. In the original setup, there are $\overline{C}_k$8 trope classes drawn from the CMU Movie Summary Corpus, $\overline{C}_k$9 characters across $B_k=[b_k^1,\ldots,b_k^p]$0 movies, 13,874 training quotes, and 1,734 quotes each for validation and test. Trope descriptions scraped from TVTropes are encoded as 4,800-dimensional skip-thought vectors. The memory stores keys $B_k=[b_k^1,\ldots,b_k^p]$1 initialized from these description embeddings and learnable values $B_k=[b_k^1,\ldots,b_k^p]$2, with multi-hop reading used to refine the persona query before classification. The full loss combines cross-entropy, optional triplet alignment to trope descriptions, and $B_k=[b_k^1,\ldots,b_k^p]$3 regularization.

Quantitatively, single-snippet GRU baselines without inter-snippet attention achieve approximately $B_k=[b_k^1,\ldots,b_k^p]$4–$B_k=[b_k^1,\ldots,b_k^p]$5 accuracy and $B_k=[b_k^1,\ldots,b_k^p]$6 around $B_k=[b_k^1,\ldots,b_k^p]$7–$B_k=[b_k^1,\ldots,b_k^p]$8. Multi-level attention with $B_k=[b_k^1,\ldots,b_k^p]$9 reaches approximately $p=512$0 accuracy and $p=512$1. Adding read-only memory improves performance further, with attn_3_ks-mem reaching $p=512$2 accuracy and $p=512$3 $p=512$4. When memory and triplet alignment are combined and the number of snippets is increased to $p=512$5–$p=512$6, the paper reports accuracy $p=512$7–$p=512$8 and $p=512$9 $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$0–$$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$1. In the CelebPersona framing supplied with the paper, trope classes map to celebrity persona facets such as archetypes, public-facing roles, or trait compositions, and prior knowledge can be drawn from biography summaries, fan-wiki pages, or press profiles. This makes the model a concrete blueprint for dialogue-based celebrity persona embeddings, even though the original benchmark is film-character classification rather than public-figure analysis.

4. CelebPersona in PersonaX: multimodal trait analysis and causal representation learning

The most explicit dataset use of the name is the CelebPersona subset of PersonaX, introduced as a privacy-preserving multimodal resource for studying outwardly observable behavior traits together with facial and biographical attributes at scale (Li et al., 14 Sep 2025). CelebPersona contains approximately 9444 public figures linked from CelebA to Wikidata. Each identity has at least two images and up to 35 images, but the release distributes only transformed embeddings rather than raw images or raw text. Images are converted to 1024-dimensional ImageBind embeddings and then obfuscated with an additional invertible transformation. The full trait write-ups are encoded as 3584-dimensional embeddings using gte-Qwen2-7B-instruct and similarly obfuscated.

The dataset combines three modalities. First, it provides person-level facial attributes, aggregated by majority vote across images from 10 CelebA attributes: Arched_Eyebrows, Big_Nose, Pointy_Nose, Bushy_Eyebrows, Big_Lips, Oval_Face, Double_Chin, Receding_Hairline, Narrow_Eyes, and High_Cheekbones, with labels $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$2 for absent, $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$3 for unknown, and $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$4 for present. Second, it provides structured biography fields including Height, Weight, Birthyear, Birthmonth, Birthday, Latitude, Longitude, Occupation_Num, and Gender_Num. Missingness is highly uneven: Height is 71.5% missing and Weight 87.0% missing, whereas Birthyear is 0.6% missing and Latitude/Longitude 0.2% missing. Third, it provides Big Five behavior traits inferred by three LLMs—ChatGPT-4o-Latest, Gemini-2.5-Pro, and Llama-4-Maverick—selected from an initial benchmark of ten contemporary LLMs.

The LLM inference pipeline evaluates models on generation time, missing rate, indecisive rate, privacy preservation, output formatting, context consistency, factual accuracy, and an overall score

$$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$5

For trait scoring, the prompt enforces a $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$6–$$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$7 scale: $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$8 means insufficient information, $$\hat{g}=[\hat{v}_1,\hat{v}_2], \qquad \hat{v}_k=\overline{C}_k+\sum_{x=1}^{p}\alpha_k^x b_k^x.$$9 disagree, $512$0 neutral, and $512$1 agree. The paper reports that the Number–L3 format minimized variability across repeated runs. Final trait labels $512$2 through $512$3 are produced by discarding zeros and applying a median-based voting rule with upward rounding when needed across the three LLMs.

PersonaX then performs cross-modal dependence analysis using five tests: Chi-square, G-square, HSIC, RCIT, and KCI. A dependency is deemed significant if $512$4, and robustness is summarized by counting how many of the five methods reject independence for each feature-trait pair. Multiple-testing correction is not reported; instead, consensus across methods is used as a robustness signal. The paper reports that Gender and occupation show strong dependence with nearly all Big Five behavior scores; Pointy_Nose and Arched_Eyebrows display significant associations; Latitude and Longitude show moderate dependence; and kernel-based methods such as HSIC, KCI, and RCIT detect more relations than classical CSQ and GSQ, suggesting non-linear dependency structures. The appendix further reports associations involving birth day and month, Bushy_Eyebrows, Big_Nose, Narrow_Eyes, and body weight.

At a second analytical level, PersonaX introduces a causal representation learning framework for multimodal and multi-measurement data. With modality-specific latents $512$5 and shared latent $512$6, the structural equations are

$512$7

The training objective is

$512$8

where reconstruction is enforced per measurement, independence is regularized by KL alignment to an isotropic Gaussian prior, and sparsity is imposed on a learnable adjacency matrix. The paper states identifiability theorems for the latent subspace, the shared subspace, and component-wise structure under assumptions including well-posed probability, modality variability, measurement changes, differentiability, entropy regularization, sufficient variability, and sparsity regularization. For CelebPersona, the reported empirical focus is not trait prediction but causal graph discovery and validation. Example pathways in the appendix include $512$9 for cultural background shaping language use, $2p=1024$0 for growing environment affecting positive language, and $2p=1024$1 linking expressiveness to approachability.

5. Celebrity personas as a testbed for personalized alignment

A separate line operationalizes CelebPersona in language-model alignment. WikiPersona introduces what it describes as the first fine-grained personalization using well-documented, famous individuals, and the paper explicitly presents this as a “CelebPersona” setting for faithful, interpretable textual descriptions of real people’s backgrounds and preferences (Tang et al., 19 May 2025). The motivation is that generic preference alignment via RLHF or DPO optimizes for what is preferable on average, whereas public figures provide verifiable, often contradictory preference profiles useful for cold-start personalization research.

The dataset is built around 50 famous individuals distributed across 11 axes where preferences often clash: sports, diet, politics, religion, age, profession, geographical location, gender, education level, AI professors, and family marriage status. Each persona has 100 training and 100 test preference pairs, with both splits containing 50 personal questions and 50 divergent questions, for approximately 5,000 train and approximately 5,000 test pairs overall. Candidate responses are sampled on-policy from Zephyr-7B-beta. For each prompt, 50 responses are generated; a generic reward model selects a tight window of 20 candidates with minimal reward range, k-means clustering over sentence embeddings chooses four diverse candidates, and GPT-4-as-personal-judge performs three rounds of pairwise comparisons to produce the preferred and dispreferred response. Human verification on a subset yields 78% agreement with GPT-4, while annotator agreement is reported as moderate, with Cohen’s $2p=1024$2 and Krippendorff’s $2p=1024$3 around 0.4–0.6.

The paper compares several personalization mechanisms. Prompting baselines include few-shot examples, random guess prefixes, a unique id tag, the celebrity name, and inferred persona summaries. A soft embedding baseline, VPL, learns a persona embedding from preference pairs via a variational autoencoder and optimizes it jointly with DPO. Personal models use one LoRA adapter per person. The main recommended approach is a multitask model with one shared LoRA adapter trained across all personas, conditioned at train and test time on a short textual prefix $2p=1024$4 describing background and inferred preferences. The formal objective is a personalized preference model $2p=1024$5, and the multitask DPO loss conditions both preferred and dispreferred responses on the same prefix.

The central empirical claim is that short, interpretable textual prefixes outperform both few-shot prompting and per-person adapters on the trade-off between effectiveness and efficiency. Prompting alone changes performance only minimally. Personal models improve trained-persona performance, especially on divergent questions, but do not generalize to unseen personas and require many adapters. The multitask model with persona gpt4 prefixes yields the strongest gains on divergent questions and more equitable generalization across unseen personas. In pairwise generation evaluation, MT with persona gpt4 achieves an overall average win rate of 63%; MT + prefix beats Zephyr + prefix 58% of the time and beats Zephyr with no prefix 64% of the time, whereas MT without a prefix falls back to roughly the Zephyr baseline in the 47–55% range. The paper also reports that personalization induces an alignment tax with up to approximately 10% variance across individuals on safety, reasoning, and factuality; reasoning tends to degrade, safety and factuality often improve, and removing the persona prefix at inference time largely restores baseline performance. This makes prefix control an explicit operational mechanism for turning personalization on or off by task.

6. Earlier computational notions of persona

Recent CelebPersona work inherits two older computational notions of persona: one from computer vision, where persona is defined as physical appearance plus behavior, and one from narrative modeling, where personae are latent role types useful for evaluation. In “What Makes Kevin Spacey Look Like Kevin Spacey,” persona is operationalized as the conjunction of identity-specific 3D face shape, expression-dependent texture, and dynamic facial behavior derived from video (Suwajanakorn et al., 2015). The system reconstructs a person from unstructured Internet imagery and allows person $2p=1024$6 to be puppeteered by the motion of person $2p=1024$7, while retaining $2p=1024$8’s own shape and texture. Inputs consist of all available photos and videos of the puppet identity, typically about 200 Internet photos per subject in the large celebrity experiments, plus a driving video. The pipeline combines face detection and 49 fiducial points, frontalization by Perspective-n-Point, thin-plate spline alignment, dense optical flow, average 3D face reconstruction, magnitude-adjusted deformation transfer, and Laplacian-pyramid multi-texture blending. Reported runtime is approximately 0.54 s per frame on a single CPU core of an Intel i7-4770 @ 3.40GHz, with 0.2 s for deformed-mesh computation and 0.34 s for texture synthesis. Although the paper predates current multimodal and alignment formulations, it establishes a technically precise interpretation of persona as persistent identity plus controllable behavior.

A different antecedent appears in “Understanding Actors and Evaluating Personae with Gaussian Embeddings,” which models actors, movies, roles, genres, and keywords as Gaussian embeddings and translation vectors (Kim et al., 2018). Here, persona is a class of story characters sharing traits, behaviors, and motivations, and actor Gaussian variance is interpreted as versatility. The paper defines two evaluation tasks: cast prediction, in which the model ranks candidate actors for a movie and persona, and versatility ranking, in which actors are ordered by the magnitude of learned variances. On cast prediction, the best reported model, JGE+AGT, achieves mean rank 174.64 and Hits@10 12.54%, compared with TransE at mean rank 506.75 and Hits@10 3.73%. On versatility ranking, JGE+T reaches 59.72% test accuracy and 0.163 rank correlation, outperforming genre-entropy and keyword-based baselines. For CelebPersona research, these results matter because they provide broad-coverage, automatic evaluation mechanisms for persona descriptors, while also showing that simplistic descriptors such as age and gender can help fit prediction yet behave inconsistently for broader persona structure.

7. Limitations, bias, and governance

CelebPersona research consistently raises privacy, bias, and misuse concerns. In diffusion personalization, the ability to generate realistic images of a person in arbitrary contexts from one photo creates consent, privacy, and deepfake risks, and the authors recommend strong safeguards, consent workflows, watermarking, and usage restrictions (Yuan et al., 2023). The same paper notes that pose and expression transfer from a single photo is not perfect, extremely stylized or rare compositions can cause identity drift, facial details may inherit biases and artifacts from Stable Diffusion’s learned celebrity faces, and the method is limited to human identity rather than other domains.

PersonaX addresses privacy partly by releasing only obfuscated embeddings and restricting use to non-commercial settings, with guidance explicitly prohibiting high-stakes applications such as insurance or lending (Li et al., 14 Sep 2025). Even so, the dataset is acknowledged to be biased toward wealthy, high-visibility public figures, with occupational overrepresentation in Entertainment, Music, and Sports, substantial missingness for height and weight, known fairness concerns inherited from face analysis on CelebA, and LLM-inferred traits that may reflect public narratives and media coverage rather than internal dispositions. No fairness metrics are reported.

The alignment literature adds a different set of risks. WikiPersona states that models trained on famous-person personas should not be used to impersonate celebrities’ opinions outside research, and warns against stereotyping, caricature, echo chambers, and sycophancy (Tang et al., 19 May 2025). Its practical mitigation is prefix control: attach the persona prefix only when personalization is needed, and remove it for factual, safety-critical, or objective tasks. In the dialogue-persona literature, trope or persona taxonomies can encode stereotypes and require careful label design, balanced data, and subgroup audits (Chu et al., 2018). Taken together, these caveats indicate that CelebPersona is methodologically valuable precisely because public figures are richly documented, but that the same public visibility intensifies representational, fairness, and misuse risks.