---
title: Personalized Safety Alignment
url: https://www.emergentmind.com/topics/personalized-safety-alignment-psa
type: topic
---

# Personalized Safety Alignment

to=arxiv_search.query  ปมถวายสัตย์ฯ  彩神争霸苹果
{"search_query":"all:\"Personalized Safety Alignment\" OR ti:\"Personalized Safety\" OR abs:\"Personalized Safety Alignment\"", "start": 0, "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"}
to=arxiv_search.query ／久久ախոսությունների  大发快三是不是
{"search_query":"ti:\"Differentially Private Steering for Large Language Model Alignment\" OR ti:\"Personalized Safety in LLMs\" OR ti:\"MIRROR\" OR ti:\"SafeCRS\"", "start": 0, "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"}
to=arxiv_search.query  天天中彩票不能买
{"search_query":"all:2505.18882", "start": 0, "max_results": 5, "sort_by": "relevance", "sort_order": "descending"}
Personalized Safety Alignment (PSA) denotes a class of alignment problems in which safety is conditioned on user-specific context rather than treated as a universal property of the prompt alone. In recent arXiv usage, the term covers several related but non-identical formulations: safety evaluation conditioned on demographic, psychological, situational, or educational attributes; test-time adaptation to user preferences over prosocial response qualities; privacy-preserving alignment from private demonstrations; authorized relaxation of refusal policies for professional users; and user-aware safety in multimodal generation, recommender systems, and agentic workflows. This suggests that PSA is best understood not as a single algorithm but as a family of methods, benchmarks, and governance schemes built around the claim that the same response may be safe for one user and unsafe for another [2505.18882][2606.09038].

## 1. Conceptual scope and definitional variants

The core PSA claim is that safety is not user-invariant. In the language-model setting, personalized safety is defined as assessing and improving responses with respect to the user’s personal context rather than only the surface prompt. The most explicit operationalization is the three-dimensional scheme introduced with PENGUIN: **risk sensitivity**, **emotional empathy**, and **user-specific alignment**, with the final Personalized Safety Score computed as the unweighted average of those three 1–5 Likert ratings [2505.18882]. CASTLE adopts a parallel educational formulation under the label **Student-Tailored Personalized Safety**, replacing generic harmlessness with a student-conditioned notion of safety grounded in age, personality, emotional state, mindset, and learning stage [2602.05633].

The literature also uses the PSA acronym in broader and sometimes overloaded ways. “Post safety alignment” describes a post-hoc remediation stage applied to already aligned LLMs to improve jailbreak resistance, reduce over-refusal, and preserve utility [2405.13820]. “Private Steering for LLM Alignment (PSA)” names a different line of work in which steering vectors are computed from private demonstrations under differential privacy guarantees [2501.18532]. These usages are not identical, but they share a common departure from one-size-fits-all safety policy.

A recurring distinction is between **personalization** and **invariance**. Some papers argue that safety should vary with the user’s vulnerability, preferences, or authorization scope; others argue that safety-critical decisions should remain invariant to superficial persona framing. Persona-Invariant Alignment, for example, formalizes the claim that for a fixed harmful intent, the safety decision should not depend on persona context [2605.01899]. PSA research therefore includes both context-sensitive adaptation and principled limits on which contextual variables are allowed to influence safety.

## 2. Benchmarks, datasets, and evaluation regimes

PSA research has produced several benchmark families that replace generic toxicity or refusal metrics with user-conditioned evaluation. PENGUIN contains **14,000 scenarios** across **seven sensitive domains**—Life, Education, Relationship, Health, Social, Financial, and Career—with both context-free and context-rich variants. Each scenario is paired with a structured profile containing **ten context attributes**: age, gender, marital status, profession, economic status, health status, education level, mental health status, self-harm history, and emotional state. Across six evaluated LLMs, adding user context raises average safety from **2.79** to **4.00**, a **43.2% improvement**, and domain-level gains fall in the **37.5%–45.6%** range [2505.18882].

CASTLE extends this logic to education. It comprises **92,908 bilingual scenarios**—**53,483 Chinese** and **39,425 English**—covering **15 educational safety risks** and **14 student attributes**. Its three metrics are **Risk Sensitivity**, **Emotional Empathy**, and **Student Alignment**, each scored on a 0–5 scale. Experiments on **18** LLMs report that all models score below an average safety rating of **2.3 out of 5**, indicating that student-tailored safety remains difficult even for strong general-purpose models [2602.05633].

Domain-specific PSA benchmarks operationalize personalized safety as structured risk rather than free-form judgment. SafeRec, introduced for conversational recommender systems, includes SafeMovie and SafeGame and evaluates recommendations with **SVR@K** and **S-DCG@K**, where the violation indicator is
$$
v_k = \mathbbm{1}\!\left[\text{final}(\hat{r}_k, t) \geq \tau\right]
$$
and the paper uses $\tau = 0.66$ [2603.03536]. In image generation, Sage contributes **44,100 text-image pairs**, **1,000 virtual users**, **10 safety-sensitive categories**, and **810+ fine-grained harmful concepts**, enabling evaluation of user-conditioned safety boundaries in diffusion models [2508.01151].

Multi-turn dialogue evaluation is also becoming central. CuRaTe was selected precisely because it tests attentional drift, sycophancy, and conflict between group preferences and personal safety needs; MIRROR reports aggregate performance improving from **69%** to **84%** across **1,685 dialogues** [2506.00430]. The survey literature argues that this shift in benchmark design reflects a deeper move from user-invariant safety evaluation toward relational, long-horizon evaluation of personalized systems [2606.09038].

## 3. Methodological families in language models

One major PSA family is **training-free inference-time planning**. RAISE formulates selective context acquisition as a sequential decision problem over ten user attributes and solves it with LLM-guided Monte Carlo Tree Search, subject to a budget constraint
$$
\max_\pi \, \mathrm{Safety}(q, U_T) \quad \text{s.t.} \quad k = |U| < B.
$$
Its online agent retrieves similar cached acquisition paths with Sentence-BERT and uses an abstention module to decide when enough background has been gathered. Adding abstention raises average safety from **2.86** to **3.56**, the offline planner increases it further to **3.77**, the full system improves safety by **31.6%** over six vanilla LLMs, and the average interaction cost is **2.7 user queries** [2505.18882].

A second family uses **persistent internal state** in dialogue. MIRROR separates immediate response generation (**Talker**) from asynchronous deliberative processing (**Thinker**), maintaining a bounded internal narrative of about **3,000 tokens** and a monologue history capped at **10,000 tokens**. The architecture is explicitly designed to combat context drift, sycophancy, and conformity bias in personal dialogue, with only **460 ms** added to average response time for GPT-4o in the reported evaluation [2506.00430].

A third family makes safety **preference-conditioned at test time**. ProSocialAlign casts safety as **lexicographic constrained generation**: first eliminate harmful continuations, then optimize within the safe set for prosocial quality under a preference vector
$$
V_{pf} = (u_{pf}^1, u_{pf}^2, \ldots, u_{pf}^k) \in \mathbb{R}^k.
$$
Its five explicit objectives are **sensitivity, empathy, non-judgmental stance, truthfulness, and helpfulness**. The method combines directional regulation in parameter space with preference-aware autoregressive reward modeling and reports strong gains across HEx-PHI, PKUSafeRLHF, NicheHazardQA, HarmEval, and ProsocialBench, including human preference for PROSOCIALALIGN in **87%** of annotated samples [2512.06515].

Task-specific PSA also appears in recommendation and psychotherapy. SafeCRS couples **Safe-SFT** with **Safe-GDPO**, normalizing relevance, safety, and count rewards separately before aggregation, and reduces safety violation rates by up to **96.5%** relative to the strongest recommendation-quality baseline while maintaining competitive quality [2603.03536]. In personalized psychotherapy, Multi-Objective DPO trains reward models for **empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy**; **MODPO** achieves **77.6% empathy** and **62.6% safety**, compared with **93.6% empathy** and **47.8% safety** for single-objective optimization, and therapeutic criteria outperform general communication principles by **17.2%** [2602.16053].

## 4. Privacy, authorization, and safety-boundary control

Privacy-preserving PSA is exemplified by differential private steering for activation editing. The method computes per-layer difference vectors
$$
d_i^l = h_l(p_i, c_i^+) - h_l(p_i, c_i^-),
$$
privatizes the mean steering direction with clipping and Gaussian noise, and injects the released vector during generation via
$$
\tilde{h}_{t,l} = h_{t,l} + \lambda v_l^{\mathrm{priv}}.
$$
The mechanism satisfies standard $(\varepsilon,\delta)$-DP, with privacy cost paid once when the private steering vectors are released; subsequent inference is treated as post-processing. Across **seven** alignment-relevant behaviors and open-source models from **0.5B to 7B**, the paper reports DP guarantees with minimal utility loss, theoretical $\varepsilon$ values typically around **2** for larger datasets, and empirical privacy better than the conservative theoretical bound under the proposed canary-based membership inference attack [2501.18532].

Authorization-oriented PSA changes the refusal boundary only for vetted users. Palette partitions prompts into **safe**, **allowed**, and **disallowed** sets and learns domain-specific relaxation modules that comply on authorized target domains while preserving ordinary safety elsewhere. On WMDP with LLaMA3.1-8B, allowing chemistry reduces refusal from **0.964** to **0.072** while maintaining high refusal on the other domains, and multiple authorization scopes can be composed by parameter merging
$$
W_{\text{merge}} = W_0 + \sum_{i=1}^{n} \Delta W_i.
$$
The paper is explicit that authorization is assumed to be verified externally; the method does not solve identity verification itself [2605.24154].

Personalization can also destabilize safety during downstream adaptation. SaLoRA shows that ordinary LoRA fine-tuning on benign data can substantially increase harmful rate, then introduces a fixed safety module derived from harmful prompts plus task-specific initialization to preserve the original safety subspace. On Llama-2-chat-7B, harmful rate rises from **0.0%** before fine-tuning to **23.7–31.7%** with LoRA but falls to **3.5–4.4%** with SaLoRA; on Llama-3.1-Instruct-8B, SaLoRA reaches **1.4–2.9%** harmful rate [2501.01765].

At the same time, not all personalization should move the safety boundary. Persona-Invariant Consistency Learning uses a unilateral KL constraint
$$
\mathcal{L}_{\text{PIC}}(\theta) = \mathbb{E}_{p,q}\!\left[ D_{\text{KL}}\!\left( \operatorname{sg}[\pi_\theta(\cdot \mid q)] \;\|\; \pi_\theta(\cdot \mid p,q) \right) \right]
$$
to force persona-conditioned outputs to match persona-free safety behavior. On out-of-distribution persona-based jailbreaks, average ASR falls from **0.601** to **0.054** for Qwen2.5-7B-Instruct and from **0.302** to **0.052** for Llama-3.1-8B-Instruct, illustrating a complementary PSA principle: user context may affect style, but not every contextual variable should alter refusal decisions [2605.01899].

## 5. Multimodal and agentic extensions

PSA has expanded beyond text-only dialogue. In text-to-image diffusion, personalized safety is formalized by augmenting the denoiser from $\epsilon_\theta(x_t, t, p)$ to $\epsilon_\theta(x_t, t, p, u)$, where $u$ is a user profile embedding. The method injects user information through a parallel cross-attention adapter, freezes the pretrained diffusion backbone, and trains only the new cross-attention parameters. It supports safety levels **L1** through **L5**, reports the lowest or near-lowest **IP** scores across tested benchmarks, and on SD v1.5 achieves roughly **80% Win Rate** against the base model on seen users and about **77%** on unseen users [2508.01151].

Agentic PSA replaces output-only moderation with plan-time oversight. The Personalized Constitutionally-Aligned Agentic Superego introduces reusable **Creed Constitutions**, user-selectable adherence on a **1–5 Likert-like scale**, and a **Universal Ethical Floor** that supersedes personalized rules. The runtime enforcer can **Allow**, **Block**, **Modify / suggest an alternative**, or **Request clarification** before execution. On HarmBench, ASR falls from about **12.0%** to about **2.0%** for GPT-3.5-Turbo, from **9.1%** to **2.1%** for Gemini 2.5 Flash, and from **0.7%** to **0.025%** for GPT-4o; on AgentHarm, Gemini 2.5 Flash’s harmful-set average score falls from **0.277** to **0.00473**, a **98.29%** reduction [2506.13774].

Role-conditioned alignment supplies another route to context-sensitive safety. Simple Role Assignment is Extraordinarily Effective for Safety Alignment does not use the PSA label explicitly, but it conditions generation on social roles such as **mother**, **principal**, or **judge**, treating roles as compact proxies for values and cognitive schemas. With critics, it reduces WildJailbreak unsafe output on DeepSeek-V3 from **81.4%** to **3.6%**, and on GPT-4.1 reduces blackmail rates in agentic scenarios from **65%** to **11%** or from **36%** to **8%**, depending on the scenario and role set [2602.00061].

These multimodal and agentic systems broaden PSA’s scope. Safety is no longer limited to textual refusal behavior; it now includes visual content suppression, pre-execution planning control, recommendation filtering, and persistent memory management across multi-turn interactions.

## 6. Open problems, tensions, and research directions

Several unresolved tensions recur across PSA papers. The first is the trade-off between **information gain** and **privacy or burden**. PENGUIN shows that not all attributes contribute equally—Emotion and Mental health matter much more than some other fields—while RAISE is motivated precisely by the fact that collecting all ten attributes may be impossible or undesirable because of privacy, burden, and cost [2505.18882]. The survey literature generalizes this point into a broader taxonomy of over-exposure, sensitive attribute inference, information persistence, retrieval poisoning, sycophancy, and long-term safety failures in personalized systems [2606.09038].

The second is the tension between **personalization** and **governance**. Personalized diffusion safety notes that user profiles are synthetic and category-level bans may not capture real, evolving human preferences; it also raises the risk of inconsistent moderation, local policy bypasses, or opaque profiling [2508.01151]. Palette explicitly warns of dual-use risk if authorized relaxation is wrongly configured or maliciously deployed [2605.24154]. The superego architecture likewise reports concrete failure modes: context-window overload with **28 constitutions** can induce fabricated constitutions such as “Rawlsian” or “Trauma-Aware,” and external oversight does not solve inner alignment or capability mismatch [2506.13774].

A third open problem concerns **projectability**. Emergent alignment work argues that alignment strategies should be judged not only by in-distribution safety but by how well the intended ethical persona projects out of distribution. Using constitutional fine-tuning, the paper shows that narrow safety fine-tuning can induce broad improvements while also producing distinctive ethical personas, and that in-distribution rankings can diverge from projectability rankings. It therefore proposes **projectability** as a selection criterion for alignment strategies [2606.09475]. For PSA, the implication is that a personalized or value-conditioned policy should be evaluated not only on benchmark fit but on whether its normative character remains coherent under distribution shift.

Finally, current evaluation remains structurally incomplete. The survey identifies three inadequacies: safety is still often evaluated as **user-invariant rather than relational**, personalization techniques are analyzed **in isolation rather than in composition**, and existing benchmarks do not capture **emergent long-term risks** [2606.09038]. In that sense, PSA remains a frontier problem. The field has already shown that user context can raise measured safety, that private and authorized alignment are technically feasible, and that multimodal and agentic systems can be personalized without collapsing utility. What remains unsettled is how to make those systems privacy-preserving, compositionally robust, normatively legible, and stable over long interaction horizons.

Source: https://www.emergentmind.com/topics/personalized-safety-alignment-psa