Harmfulness Subspaces in LLMs

• Harmfulness subspaces are defined as low-rank geometric constructs in LLMs that encode 55 harmful subconcepts across diverse behavioral categories.
• Empirical analysis via SVD shows that a dominant one-dimensional direction captures 95% of the harmful variance, enabling targeted interventions like ablation and steering.
• Interventions using harmfulness subspaces improve safety metrics in LLMs, though they highlight tradeoffs with utility and challenges from entangled behavioral signals.

A harmfulness subspace is a geometric construct in the internal activation space of LLMs that encodes the presence and degree of harmful intent across a diverse set of granular behavioral categories. This concept arises from probing neural activations with interpretable directions associated with specific types of harmful behaviors—such as racial hate, employment scams, and weapons advice—and is used both for mechanistic interpretability and targeted intervention to curb undesirable outputs. Recent work demonstrates that harmfulness subspaces are strikingly low-rank, often appearing essentially one-dimensional, and empirically associated with the latent structure governing LLM harmfulness production (Shah et al., 23 Jul 2025), while related research highlights both the separability and entanglement of such subspaces with other behavioral signals (Zhao et al., 16 Jul 2025, Ponkshe et al., 20 May 2025).

1. Formal Definition and Construction

Harmfulness in LLMs is modeled as a composite of 55 distinct subconcepts, each associated with a linear direction in activation space. These directions are learned using linear probes on transformer hidden states,

$f_k(x) = \sigma(w_k^T x + b_k),$

where $x \in \mathbb{R}^d$ is the attention-output hidden state, $\sigma$ is the sigmoid nonlinearity, and $w_k \in \mathbb{R}^d$ is a normalized weight vector corresponding to harmfulness subconcept $k$ (Shah et al., 23 Jul 2025). The collective set of $n=55$ vectors forms a matrix $$W = [w_1\,|\,w_2\,| \ldots |\,w_{55}] \in \mathbb{R}^{d \times 55}$$, and the harmfulness subspace is given by $\mathrm{span}\{w_1,\ldots,w_{55}\}$.

A complementary approach involves clustering activations for refused and accepted instructions, yielding centroids for harmfulness ($\mu_{harmful}$, $\mu_{harmless}$) and refusal ($\mu_{refuse}$, $\mu_{accept}$) and extracting the harmfulness direction by normalized difference-of-means (Zhao et al., 16 Jul 2025).

2. Geometry and Low Dimensionality

Empirical analysis via singular value decomposition (SVD) of the matrix $W$ reveals that the harmfulness subspace is extremely low-rank:

$$W = U\Sigma V^T,\quad \Sigma = \mathrm{diag}(\sigma_1,\ldots,\sigma_{55}),$$

and the effective rank $K(\tau)$ for $\tau = 0.95$ is $K(0.95) = 1$, meaning that 95% of the subspace’s variance is captured by a single direction $u$ (Shah et al., 23 Jul 2025). This dominant principal component essentially characterizes the bulk of harmful informational content across all 55 subconcepts. Analogous findings in other models support one-dimensionality for both harmfulness and refusal behaviors, encoded along distinct axes in activation space (Zhao et al., 16 Jul 2025).

3. Separability from Refusal and Entanglement with Utility

Research distinguishes harmfulness from refusal subspaces, finding that refusal (often enforced by alignment procedures) and harmfulness (internal judgment of prompt risk) form separable, orthogonal concept vectors. Causal experiments confirm that steering model behavior along the harmfulness direction modifies the internal perception of harm, whereas steering along refusal primarily affects output refusal without genuinely altering harmfulness judgment (Zhao et al., 16 Jul 2025).

In contrast, broader analyses indicate that safety subspaces are not perfectly distinct from general-purpose learning directions. Subspaces extracted to amplify utility or safety often overlap significantly and cannot be cleanly isolated in either weight or activation space: any projection that suppresses harmfulness also degrades utility, and prompts of differing safety levels activate heavily entangled high-energy components (Ponkshe et al., 20 May 2025).

4. Intervention Techniques and Quantitative Evaluation

Several intervention mechanisms are formulated mathematically:

• Projection onto the full harmfulness subspace:

$\hat x = \sum_{i=1}^{55} (w_i^T x) w_i,$

• Ablation of the entire harmfulness subspace:

$x' = x - \hat x,$

• Steering/ablation along the dominant direction $u$:

$x' = x - (u^T x) u.$

In practice, ablation or steering applied at the top-performing layers yields measurable improvements in safety metrics (JailbreakBench) and robustness (AutoDAN attacks), but may induce modest utility tradeoffs (MMLU accuracy). Dominant direction steering nearly eliminates harmful outputs with mild loss of general performance (Shah et al., 23 Jul 2025). Fine-tuning attacks that diminish refusal do not eliminate the harmfulness representation, and adversarial interventions on refusal do not affect the model’s internal harmfulness signal (Zhao et al., 16 Jul 2025).

Intervention	Safety (Safe%)	Utility (MMLU%)	Attack Success (AutoDAN %)
Baseline	89	55	94
Subspace ablation	91	51	--
Dominant ablation	91	60	--
Dominant steering	~100	~50	50

5. Practical Applications: Auditing and Latent Guard

The harmfulness subspace provides a scalable basis for model auditing and safety intervention. Probing for many predefined dangerous behaviors enables mapping a low-dimensional safety manifold and supports rapid, model-agnostic assessment (Shah et al., 23 Jul 2025). The robustness and separability of the harmfulness direction enables the construction of an intrinsic monitor (“Latent Guard,” Editor's term), which detects harmful instructions as a function of internal model activations. Latent Guard matches or exceeds the performance of state-of-the-art models like Llama Guard 3 8B and resists adversarial fine-tuning (Zhao et al., 16 Jul 2025).

Challenge Set	Latent Guard Accuracy (%)	Llama Guard 3 Accuracy (%)
Suffix jailbreak	100	100
Persuasion jailbreak	41.6	0
Template jailbreak	100	76.0
Over-refused harmless	100	84.4
Accepted harmful	93.9	45.5

6. Limitations, Controversies, and Future Directions

Certain assumptions in the harmfulness subspace framework are challenged by recent research. Alignment-induced subspaces, as extracted via weight updates or activation-space projections, do not localize safety: both safe and unsafe behaviors are amplified equally, and entanglement with high-impact learning directions is pervasive (Ponkshe et al., 20 May 2025). Attempts to isolate or preserve safety directions via subspace-based defense yield proportional losses in utility and safety. These findings suggest fundamental limits to purely linear subspace interventions and advocate for richer, potentially nonlinear concept manifolds, robust data curation, holistic alignment strategies, and post-hoc interventions that do not rely on the existence of a singular “safety direction.”

Future work may expand beyond static sets of harmfulness subconcepts, explore kernel-based nonlinear representations, enable dynamic adjustment of steering strength, and integrate gradient-based or multimodal editing (Shah et al., 23 Jul 2025). The existence, properties, and practical deployability of harmfulness subspaces remain a central topic in LLM safety and interpretability research.