SafeVec: Geometric LLM Safety Evaluation

• SafeVec is a white-box method that evaluates LLM safety by comparing hidden activation directions under unsafe and jailbreak prompts.
• It employs a family-specific calibration pipeline, using a safety-aligned reference model to extract refusal directions from internal representations.
• SafeVec offers significant efficiency benefits over judge-based evaluation, though its reliance on white-box access can limit applicability in closed API settings.

Searching arXiv for the main SafeVec paper and closely related work. {"^^^^^^^^3query3 Measuring LLM Safety Through Refusal Alignment\"^^^^^^^^3 OR abs:\3^^^^^^^^"SafeVec\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"} {"^^^^^^^^3query3 Measuring LLM Safety Through Refusal Alignment SafeVec","max_results":^^^^3ti:\3query3^^^^,"sort_by":"submittedDate","sort_order":"descending"} arxiv_search: {"^^^^^^^^3query3 Measuring LLM Safety Through Refusal Alignment SafeVec","max_results":^^^^3ti:\3query3^^^^} SafeVec is a white-box, representation-level procedure for evaluating LLM safety from internal activations rather than generated text. Introduced alongside the Refusal Alignment Score (RAS), it operationalizes the idea of refusal alignment: when a safety-aligned reference model processes harmful or jailbreak prompts, its hidden states move in a characteristic direction in residual-stream space, and the extent to which another model reproduces that internal geometry can be used as a compact proxy for safety posture. In the reported experiments, SafeVec is applied within architecture families such as Llama, Gemma, and Qwen, where it separates aligned models from uncensored and abliterated variants, tracks output-level attack success rate, and is substantially faster than judge-based evaluation (&&&^^^^3query3^^^^&&&).

^^^^3ti:\3^^^^. Concept and evaluative setting

SafeVec is designed as an alternative to conventional output-based LLM safety evaluation. Standard practice queries a model with unsafe or jailbreak prompts, decodes a response, and then judges whether the output violates a safety policy. The motivating critique is threefold: output-level evaluation is expensive because it requires both response generation and a judge; it is sensitive to decoding choices such as temperature, maximum tokens, formatting, and refusal templates; and it can become stale when models are trained against fixed question banks. A further limitation is epistemic: output-level behavior measures the final answer, not whether the model internally recognized the request as harmful or exhibited refusal-related structure that was later overridden (&&&^^^^3query3^^^^&&&).

SafeVec instead assumes a white-box setting with access to last-token residual-stream activations at each decoder layer. Its central question is whether, under harmful or adversarial prompts, a target model’s hidden states align with a refusal-related direction extracted from a safety-aligned reference model. Under this definition, refusal alignment is not inferred from text outputs; it is defined directly from contrastive hidden-state statistics over safe and unsafe inputs.

The procedure is family-specific. For each architecture family, an official instruction-tuned model is treated as the aligned anchor, and other models in the same family are scored relative to refusal directions extracted from that anchor. This family-conditioned design reflects the paper’s claim that raw cosine scores are not directly comparable across architectures.

^^^^3 OR abs:\3^^^^. Geometric construction and the SafeVec pipeline

The SafeVec pipeline has five stages: refusal direction extraction, stable layer window selection, scoring of calibration models, family-specific RAS calibration, and scoring of target models (&&&^^^^3query3^^^^&&&).

For an architecture family PRESERVED_PLACEHOLDER_^^^^3query3^^^^, let PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^ be the reference model, and let PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^ denote the last-token residual-stream activation of model $M$ at layer $l$ on prompt $x$. Safe prompts $\mathcal{S}$ and unsafe prompts $\mathcal{U}$ are first used to construct layer-wise mean activations in the reference model: $$\mu^{\mathrm{safe}}_l = \frac{1}{|\mathcal{S}|}\sum_{s\in\mathcal{S}} h^{\mathrm{ref}}_l(s), \qquad \mu^{\mathrm{unsafe}}_l = \frac{1}{|\mathcal{U}|}\sum_{u\in\mathcal{U}} h^{\mathrm{ref}}_l(u).$$ The refusal direction at layer $l$ is then

PRESERVED_PLACEHOLDER_^^^^3ti:\3query3^^^^

The operational interpretation is that the reference model moves toward PRESERVED_PLACEHOLDER_^^^^3ti:\3ti:\3^^^^ when processing unsafe prompts, while safe prompts are relatively suppressed along that direction.

Layer selection is based on cosine structure. For each layer,

PRESERVED_PLACEHOLDER_^^^^3ti:\3 OR abs:\3^^^^

with cosine similarity defined as

PRESERVED_PLACEHOLDER_^^^^3ti:\33^^^^

Two derived quantities are inspected: the safe suppression score PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^4 and the separation gap PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^5. A continuous layer window PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^6 is selected where PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^7 is sufficiently large and stable, the safe/unsafe pattern is consistent, and calibration models exhibit monotone separation.

The selected windows are fixed per family for later scoring.

Family	Reference model	Selected layer window
Llama‑3.^^^^3ti:\3^^^^‑8B	meta-llama/Llama-3.^^^^3ti:\3^^^^-8B-Instruct	PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^8
Gemma‑3‑4B	google/gemma-3-4b-it	PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^9
Qwen‑^^^^3 OR abs:\3^^^^.5‑7B	Qwen/Qwen^^^^3 OR abs:\3^^^^.5-7B-Instruct	PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3query3^^^^

This construction is deliberately simple. The paper explicitly situates it near representation-engineering and linear-probe-style methods, but uses mean differencing rather than supervised logistic regression.

3. Unsafe and jailbreak scoring, and calibration into RAS

Once refusal directions PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3ti:\3^^^^ are fixed, a model PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3 OR abs:\3^^^^ is scored on unsafe prompts PRESERVED_PLACEHOLDER_^^^^3 OR abs:\33^^^^ and jailbreak prompts PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^4. The raw SafeVec scores are

PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^5

PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^6

These scores measure the extent to which a model’s hidden states align with the reference refusal directions under direct harmful prompts and adversarially scaffolded harmful prompts. In the reported experiments, aligned models have positive UnsafeScore and JailbreakScore, whereas uncensored or abliterated models often have negative values (&&&^^^^3query3^^^^&&&).

The paper combines the two with equal weights: PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^7 Safe prompts are not used in this final scoring stage; they are used only to construct refusal directions and to analyze layer-level separation.

To map PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^8 into a standardized PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^9-MM^^^^3query3^^^^ safety score, the paper defines a refusal drop relative to the family reference: $M$^^^^3ti:\3^^^^ A high-risk subset of calibration models is then formed: $M$^^^^3 OR abs:\3^^^^ with $M$3. The family-specific bad-anchor scale is

$M$4

if $M$5 is non-empty; otherwise the $M$6 quantile of all positive drops is used. Normalized refusal drop is

$M$7

Calibration proceeds by relating refusal drop to attack success rate increase: $M$8

$M$9

$l$^^^^3query3^^^^

An architecture-specific severity multiplier is then defined: $l$^^^^3ti:\3^^^^ with shrink coefficient $l$^^^^3 OR abs:\3^^^^, yielding the aligned drop

$l$3

Finally, RAS is given by a sigmoid map: $l$4 The paper fixes $l$5 and $l$6. The mapping is monotonically decreasing in aligned drop: larger refusal loss yields lower RAS. The intended interpretation is approximate rather than literal: RAS near $l$7 indicates reference-level or stronger refusal alignment; RAS near $l$8 indicates moderate degradation and nontrivial safety risk; RAS near $l$9 indicates refusal alignment comparable to highly unsafe, uncensored, or abliterated models.

4. Prompt regimes, behavioral calibration, and empirical findings

SafeVec distinguishes three prompt sets. Safe prompts $x$^^^^3query3^^^^ are benign tasks used to extract refusal directions and measure safe suppression. Unsafe prompts $x$^^^^3ti:\3^^^^ are directly harmful or policy-violating requests and are used both in direction extraction and in UnsafeScore. Jailbreak prompts $x$^^^^3 OR abs:\3^^^^ wrap harmful intent in adversarial scaffolding such as role-play or obfuscation and are used in JailbreakScore (&&&^^^^3query3^^^^&&&).

Output-level attack success rate is used only for calibration, not as part of SafeVec itself. For ASR measurement, the paper uses the HEx-PHI harmful prompt set, described as covering ^^^^3ti:\3ti:\3^^^^ prohibited-use categories such as violence, self-harm, and illegal activities. For each HEx-PHI prompt, the model under evaluation generates up to 5^^^^3ti:\3 OR abs:\3^^^^ tokens with greedy decoding, and a SORRY-Bench fine-tuned Mistral‑7B judge outputs $x$3 if the response directly assists or resolves the unsafe request and $x$4 if it refuses or does not satisfy the request. ASR is the mean judge output across prompts.

The empirical findings are reported for three families: Llama‑3.^^^^3ti:\3^^^^‑8B, Gemma‑3‑4B, and Qwen‑^^^^3 OR abs:\3^^^^.5‑7B. Each family includes one reference model and several calibration models spanning aligned, uncensored, and abliterated variants. In the Llama example, Llama-3.^^^^3ti:\3^^^^-8B-Instruct has Unsafe $x$5, Jailbreak $x$6, and ASR $x$7, whereas Llama-3.^^^^3ti:\3^^^^-8B-Instruct-abliterated has Unsafe $x$8, Jailbreak $x$9, and ASR $\mathcal{S}$^^^^3query3^^^^. DarkIdol-Llama-3.^^^^3ti:\3^^^^-8B-Instruct-Uncensored similarly shows negative raw SafeVec scores and ASR $\mathcal{S}$^^^^3ti:\3^^^^. The reported pattern is consistent across Gemma and Qwen: aligned models occupy the positive-score, low-ASR regime, while uncensored and abliterated variants occupy the negative-score, high-ASR regime.

Layer-wise analyses show that, within the selected windows, reference models exhibit $\mathcal{S}$^^^^3 OR abs:\3^^^^ with stable separation gaps. When cosine-similarity curves are plotted across layers for multiple models, aligned models exhibit higher similarity than uncensored or abliterated variants, especially for unsafe and jailbreak prompts and particularly within the chosen windows. The paper interprets this as evidence that refusal-related behavior is represented in hidden space as a relatively simple direction.

After calibration, RAS is plotted against $\mathcal{S}$3 for target models. The reported relationship is strongly monotone across all three families: higher RAS corresponds to lower ASR. The paper does not provide explicit correlation coefficients, and it notes exceptions, including a Gemma reasoning-distilled model with lower raw refusal alignment but also lower ASR than the reference. This qualifies RAS as a proxy for safety rather than an exact behavioral equivalent.

5. Efficiency, robustness, and limits of interpretation

A central practical claim is efficiency. RAS requires only forward passes on $\mathcal{S}$4 and $\mathcal{S}$5, extraction of hidden states, and cosine computations, whereas judge-based evaluation requires full generation on HEx-PHI and subsequent judging. The timing comparison reported in the paper is as follows (&&&^^^^3query3^^^^&&&).

Family reference	RAS time (s)	Judge time (s)
Llama‑3.^^^^3ti:\3^^^^‑8B‑Instruct	^^^^3ti:\3^^^^4.^^^^3ti:\33	57^^^^3 OR abs:\3^^^^.47
Gemma‑3‑4B‑it	^^^^3ti:\33^^^^.^^^^3 OR abs:\3^^^^9	63^^^^3query3^^^^9.68
Qwen^^^^3 OR abs:\3^^^^.5‑7B‑Instruct	^^^^3ti:\3^^^^4.97	^^^^3 OR abs:\3query3 OR abs:\3^^^^5.^^^^3 OR abs:\3^^^^6

The paper reports corresponding speedups of $\mathcal{S}$6, $\mathcal{S}$7, and $\mathcal{S}$8, with an average RAS time of $\mathcal{S}$9 seconds, average judge time of $\mathcal{U}$^^^^3query3^^^^ seconds, and average speedup of $\mathcal{U}$^^^^3ti:\3^^^^. This supports the claim that the method is roughly two orders of magnitude faster in the reported setup.

The robustness claims are narrower. The refusal direction extracted from the reference model generalizes across models in the same family with different safety levels, and the separation holds for both direct unsafe prompts and jailbreak prompts. However, the paper does not systematically study heavy paraphrasing, distribution shifts in harmful content, or adaptive jailbreaks that explicitly target internal representations.

Several limitations follow directly from the method design. First, RAS requires hidden-state access and therefore does not apply to closed APIs that expose only generated text. Second, the method is family-specific: each architecture requires its own reference model, layer window, and calibration set. Third, the score depends on the reference model and prompt sets; if future models encode safety differently, refusal directions may need to be re-estimated. Fourth, a single scalar refusal direction may not capture nuanced policy violations such as subtle bias or context-specific harms. Fifth, the paper does not evaluate adversarial fine-tuning strategies intended to preserve cosine similarity while eliciting harmful outputs.

A common misconception is to treat RAS as interchangeable with ASR. The paper explicitly rejects that equivalence. RAS measures whether a model internally engages a refusal-related pattern under unsafe or jailbreak prompts; it correlates with output-level attack success rate but does not define behavior exhaustively. This suggests that SafeVec is best understood as an audit primitive within a broader safety-testing pipeline rather than a standalone guarantee.

6. Position within adjacent research and terminological overlap

SafeVec is placed at the intersection of representation-level interpretability, activation steering, and output-based safety benchmarking. Its novelty is not the use of concept-like directions in activation space per se, but their use for evaluation rather than intervention. The procedure explicitly standardizes four components: extraction of refusal directions, selection of a stable layer window, calibration against ASR using reference and calibration models, and deployment of a family-specific $\mathcal{U}$^^^^3 OR abs:\3^^^^-U\mathcal{U}3 metric. The reported cross-model generalization within a family is central to that contribution (&&&^^^^3query3^^^^&&&).

The name also overlaps with a distinct usage in Rust systems literature. In formal verification work on unsafe Rust, a “SafeVec-style abstraction” denotes a vector-like data structure implemented with raw pointers and unsafe internals but exposed through a safe API; it is used as an example of the kinds of invariants that must be verified to preserve Rust’s memory-safety guarantees (Foroushaani et al., 2022). In work on deepSURF, “SafeVec-style abstraction” again refers to unsafe, performance-critical container internals wrapped in a safe public interface and serves as a motivating example for fuzzing and vulnerability detection in Rust libraries (Androutsopoulos et al., 18 Jun 2025). That systems-programming usage is unrelated to SafeVec as a representation-level LLM safety evaluator, beyond the accidental name overlap.

The coexistence of these usages is methodologically instructive. In the LLM setting, SafeVec names a geometric safety-evaluation pipeline over hidden states. In the Rust setting, “SafeVec-style” names a class of low-level data abstractions whose safety depends on correct handling of unsafe code. The two share neither domain nor mechanism, and conflating them would be a category error.