Safe-SAIL: LLM Safety via Sparse Autoencoder Interpretation

• Safe-SAIL is a framework that leverages sparse autoencoders to decipher low-frequency safety concepts in large language models.
• It employs a three-phase pipeline—SAE Training, Automated Interpretation, and Diagnose Toolkit—to systematically map safety features.
• The approach introduces novel metrics and segment-level simulation to pre-select SAE configurations and reduce explanation costs.

Safe-SAIL denotes the framework introduced in "Safe-SAIL: Towards a Fine-grained Safety Landscape of LLMs via Sparse Autoencoder Interpretation Framework" for interpreting sparse autoencoder (SAE) features in safety-critical domains of LLMs (Weng et al., 11 Sep 2025). It is motivated by two linked observations: safety evaluation in LLMs is usually behavioral and task-bound, and SAE-based interpretability has not been operationalized for low-frequency safety concepts such as pornography, politics, violence, and terror. Safe-SAIL addresses these issues through a full pipeline that selects SAE configurations with strong safety-domain-specific interpretability, reduces feature-explanation cost through segment-level simulation, and constructs a resource of 1,758 safety-related features with human-readable explanations and systematic evaluations (Weng et al., 11 Sep 2025).

1. Problem setting and conceptual scope

Safe-SAIL starts from the claim that behavioral safety methods such as toxicity detectors, jailbreak tests, or refusal evaluation focus on visible outputs and can miss latent risk mechanisms. In parallel, SAEs can decompose entangled activations into sparse, human-interpretable features, but existing SAE work usually stops at training or at small-scale interpretation. The framework therefore treats safety as a low-frequency concept domain in which generic SAE choices that optimize reconstruction or minimize interference are not necessarily the choices that maximize safety-specific interpretability (Weng et al., 11 Sep 2025).

Within this formulation, the central question is under what SAE configurations latent features become maximally interpretable for low-frequency safety concepts. Safe-SAIL answers this by shifting emphasis away from reconstruction quality alone and toward the number and quality of domain-specific safety features that an SAE can expose. This suggests a broader methodological distinction between general-purpose SAE evaluation and safety-domain interpretability evaluation: the former measures representation quality in aggregate, whereas the latter targets a sparse subspace of safety-relevant semantics (Weng et al., 11 Sep 2025).

The paper’s notion of a “safety landscape” is therefore mechanistic rather than purely behavioral. The goal is to map internal safety-relevant features beyond end-to-end black-box behavior tests, and to do so at a scale large enough to support systematic analysis rather than isolated case studies (Weng et al., 11 Sep 2025).

2. Pipeline architecture

Safe-SAIL is organized as a three-phase framework consisting of SAE Training, Automated Interpretation, and the Diagnose Toolkit (Weng et al., 11 Sep 2025).

Phase	Objective	Main components
SAE Training	Train candidate SAEs for safety interpretability	Qwen2.5-3B-Instruct activations, TopKReLU, expansion factor 10
Automated Interpretation	Select, filter, and explain safety features	Pre-explanation evaluation, safety-feature filtering, LRM-based explanation
Diagnose Toolkit	Inspect and navigate the feature space	Interactive token-level viewer, 2D feature map

In the SAE Training phase, SAEs are trained on internal activations from Qwen2.5-3B-Instruct, specifically on the MLP output of layer 17 and the post-MLP residual stream of layer 17. Quarter-layer checkpoints are also discussed for the final resource. The activation function is TopKReLU, and the expansion factor is fixed to 10 (Weng et al., 11 Sep 2025).

The Automated Interpretation phase contains three distinct operations. First, pre-explanation evaluation estimates which SAE configurations are likely to produce useful safety features before the expensive explanation stage. Second, safety-feature filtering keeps only features that are sensitive to safety concepts. Third, feature explanation generation uses a large reasoning model to produce natural-language descriptions. The resulting workflow is conceptually similar to neuron-interpretation pipelines, but it is adapted to SAE features and safety-specific concept pairs (Weng et al., 11 Sep 2025).

The Diagnose Toolkit packages the output into an interactive analysis environment. It includes an interactive token-level viewer showing which safety features activate on each token, and a 2D feature map in which distances reflect semantic similarity between annotated features. This supports both direct inspection of arbitrary inputs and navigation of the broader annotated feature space (Weng et al., 11 Sep 2025).

3. SAE selection before explanation

A major contribution of Safe-SAIL is a pre-explanation evaluation procedure designed to estimate how many useful safety features an SAE will yield before detailed explanation is attempted. The method is based on Concept Contrastive Query Pairs: a concept query containing a safety concept and a de-concept query in which that concept is removed while the rest of the text remains as similar as possible (Weng et al., 11 Sep 2025).

For a latent feature $k$, the paper defines a delta-frequency

$$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$

where $Q_{C,i},Q_{D,i}\in\{0,1\}$ indicate whether feature $k$ activates on the concept query or the paired de-concept query. On top of this distribution, Safe-SAIL introduces two metrics:

$$L_{0,t} = \sum_{k=0}^{M-1} \begin{cases} 1, & \text{if } freq_k > t \ 0, & \text{if } freq_k \leq t \end{cases}$$

and

$I_{CDF} = E(freq) = \int_0^1 (1-F(x))dx.$

The paper uses $t=0.25$ for $L_{0,t}$. The first metric counts distinguishable features above threshold; the second measures the expected delta frequency, or the area under the tail of the CDF (Weng et al., 11 Sep 2025).

The paper argues that $I_{CDF}$ and $L_{0,t}$ track the eventual number of valid safety features much better than k-Sparse Probing or 1d-Probe, which are reported as unstable or overly dependent on hyperparameters and data. In the reported plots, these two metrics align closely with the ground-truth number of domain-specific features across sparsity settings (Weng et al., 11 Sep 2025).

A second efficiency contribution is segment-level simulation (SLS), introduced as an alternative to token-level simulation (TLS). The paper compares three simulation methods: All at once, TLS, and SLS. In SLS, a query is split into $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$0 segments, and the explainer predicts whether each segment is “activate” or “non-activate” rather than assigning token-wise activations. With $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$1, SLS achieves strong correlation with TLS, around $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$2, while reducing simulation cost by about 55% relative to TLS. The cost is measured as total generated token length, including reasoning and output (Weng et al., 11 Sep 2025).

4. Training setup, filtering, and released resource

The SAE suite is trained on Qwen2.5-3B-Instruct with TopKReLU, sparsity levels $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$3, expansion factor 10, and activations from the layer 17 MLP and post-MLP residual stream. The explanation-data construction uses 200k queries, mixed as 25% risky content, 10% random non-risk-related queries, and 65% samples from The Pile. Evaluation uses 10,000 Concept Contrastive Query Pairs across the four domains (Weng et al., 11 Sep 2025).

After training, safety-related features are filtered using precision and recall thresholds on concept versus de-concept pairs:

$$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$4

A feature is retained if

$$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$5

with $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$6 and $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$7 (Weng et al., 11 Sep 2025).

Feature explanation generation follows a standard neuron-description pipeline. Activations are collected from SAE inference on a customized explanation dataset, activation values are quantized into levels by linear interpolation, samples from each activation level are inserted into a prompt, and a large reasoning model produces a natural-language explanation. Among QwQ-32B, DeepSeek-R1, and Claude 3.7 Sonnet, DeepSeek-R1 performs best by correlation score and is used as the strongest explainer (Weng et al., 11 Sep 2025).

The final released resource contains 1,758 safety-related features spanning pornography, politics, violence, and terror. The feature base spans SAE checkpoints from quarter layers $$freq_k=\frac{\sum_{i=0}^{n-1}Q_{C,i}(1-Q_{D,i})}{n},$$8. The release includes SAE checkpoints, human-readable feature explanations, evaluation scores, and the diagnostic toolkit. The resource is intended to support both direct inspection and broader mechanistic safety analysis (Weng et al., 11 Sep 2025).

5. Empirical findings and mechanistic analyses

The paper presents Safe-SAIL not only as an annotation pipeline but also as an empirical instrument for risk feature identification. In a pornography case study, three features are highlighted: 26_18429, 17_1579, and 17_4828. Their interpretations suggest different roles: 26_18429 acts as a semantic adult-content detector with strong responses to explicit adult content in multiple languages; 17_1579 is syntax- and URL-structure-sensitive, capturing adult-platform patterns, numerical euphemisms, domain suffixes, and technical website signatures; and 17_4828 captures explicit pornographic terminology together with platform and metadata markers. On 20 pornographic URLs, the activations of these features correlate with the model’s adult-content judgments. For cases such as OnlyFans, activations are reported as surprisingly weak despite high confidence, suggesting either reliance on other features or a weaker internal association (Weng et al., 11 Sep 2025).

Layerwise analysis yields a staged account of safety encoding. The reported progression is early/local detection, mid-level semantic construction, and high-level safety reasoning. In the child sexual abuse example, the model trajectory moves from recognizing tokens such as “child” or “sell,” to constructing a semantic scene of exploitation and transaction, and then to refusal. The paper interprets this as a concept-driven reasoning chain rather than arbitrary response generation (Weng et al., 11 Sep 2025).

A further empirical result concerns multilingual vulnerability. The paper reports that low-resource languages can bypass safe reasoning because the model lacks well-formed conceptual understanding in those languages. In the Hindi case, malicious inputs fail to trigger the same safe-response chain, and relevant features activate weakly or not at all. The same layered trajectory is reported across other languages, but with different activation quality. This suggests that safety-critical concepts are encoded in a way that is partly language-specific and partly abstract, with weaker coverage in low-resource settings (Weng et al., 11 Sep 2025).

These findings position Safe-SAIL as a framework for mapping not only whether risky outputs occur, but also how risky entities, concepts, and refusal mechanisms are distributed across layers and across languages. A plausible implication is that interpretability resources of this kind can expose asymmetries in safety coverage that remain difficult to diagnose through output-level evaluation alone (Weng et al., 11 Sep 2025).

6. Limitations, methodological tensions, and significance

Safe-SAIL explicitly argues that the sparsity setting that maximizes safety-domain interpretability is not the same as the one that minimizes feature interference. In the reported ablations, sparsity affects reconstruction quality, number of interpretable safety features, correlation score, and superposition score. The best safety interpretability occurs at a sparser setting than the minimum-interference point. The paper’s mechanistic interpretation is that safety concepts occupy a small subspace, so making features too orthogonal can reduce the number of discovered safety features (Weng et al., 11 Sep 2025).

This point addresses a recurring misconception in SAE-based work: optimizing generic reconstruction or minimizing interference is not equivalent to maximizing domain-specific interpretability. Safe-SAIL treats safety as a rare-concept regime in which representational capacity allocation matters as much as standard SAE quality metrics (Weng et al., 11 Sep 2025).

The stated limitations are equally explicit. Only four safety domains are covered. Explanation quality depends on the chosen explainer model and may vary by language and domain. Segment-level simulation approximates token-level dynamics and may lose fine-grained detail. Thresholds in evaluation and filtering are empirically chosen and could be improved with adaptive tuning. The work is mainly correlational, with causal evidence limited to small preliminary steering experiments (Weng et al., 11 Sep 2025).

The appendix provides that preliminary causal evidence. Amplifying a pornography-related feature increases unsafe response rate substantially, and amplifying a profanity feature can drive unsafe response rate up to around 97.8% in one setting. The paper treats these results as suggestive rather than as full causal validation, but they indicate that at least some annotated features are not merely descriptive correlates of behavior (Weng et al., 11 Sep 2025).

Taken together, Safe-SAIL is best understood as a domain-specific SAE interpretation pipeline rather than merely an SAE training recipe. Its distinctive contribution lies in combining pre-explanation SAE selection, lower-cost explanation scoring, large-scale feature annotation, and mechanistic analyses into a single system for constructing a fine-grained safety landscape of LLM internals (Weng et al., 11 Sep 2025).