Safe Side Network in HyperSafe

• Safe Side Network (SSN) is a model-specific side network that uses activation fingerprints from a small calibration set to perform prompt-level safety classification.
• It is generated non-invasively by a hypernetwork that synthesizes SSN parameters from layer-wise representations, requiring no weight modifications or gradient updates.
• Empirical results show that SSN reduces harmful response rates to below 1% while preserving task performance with minimal accuracy drop.

Searching arXiv for the cited paper and closely related terminology. The Safe Side Network (SSN) is a lightweight, model-specific neural network introduced within the HyperSafe framework for post hoc safety restoration in fine-tuned LLMs. It is attached to a frozen fine-tuned checkpoint and performs prompt-level safety classification: harmful prompts are routed to refusal, whereas safe prompts are passed to the original fine-tuned model for ordinary generation. In HyperSafe, the SSN is not manually trained for each deployed checkpoint; instead, a hypernetwork generates the SSN from layer-wise activation fingerprints extracted from a small calibration set, yielding a model-specific and non-invasive safety mechanism that requires no gradient updates, no safety data at deployment time, and no modification to deployed model weights (Aliev et al., 13 Jul 2026).

1. Definition and design rationale

In HyperSafe, an SSN is a side network whose function is to predict whether answering a given prompt with a particular fine-tuned LLM would be harmful. Its role is therefore narrower than end-to-end generation: it is a checkpoint-specific binary safety gate positioned before ordinary response production. Harmful prompts are refused; safe prompts are routed to the original fine-tuned model.

The design target is explicitly post-hoc, model-specific, and non-invasive. The motivating problem is that safety alignment in LLMs can be fragile under fine-tuning, including benign task adaptation. The reported alternatives are divided into two broad classes. One class intervenes during or after fine-tuning through retraining or weight modification, which may be costly and may hurt task performance. The other uses model-agnostic safety classifiers, which may miss failures specific to a given fine-tuned checkpoint. The SSN is proposed as a response to these limitations: it is generated for the target checkpoint itself and runs alongside the frozen backbone rather than replacing or modifying it.

A key point of interpretation is that the SSN is not presented as a universal safety classifier detached from model internals. Its defining property is precisely its conditioning on the internal representational changes induced by fine-tuning. This suggests a conception of safety recovery tied to checkpoint identity rather than to prompt semantics alone.

2. Activation fingerprints and conditioning

HyperSafe constructs an SSN from activation fingerprints extracted from the target fine-tuned model $\mathcal{M}_{d^*}$. For a calibration set of $N_{\text{cal}}$ generic task-relevant prompts, the framework collects layer-wise mean backbone hidden states after the final token and averages them over prompts:

$$\mathbf{a}_l = \frac{1}{N_{\text{cal}}}\sum_{i=1}^{N_{\text{cal}}} \mathbf{h}_l(x_i),\quad \mathcal{A}(\mathcal{M}_d) = \{\mathbf{a}_l\}_{l=1}^L$$

These fingerprints are intended to capture how fine-tuning has changed the model’s internal representations layer by layer. The representation is deliberately compact: it does not store the full activation distribution, only a layer-indexed summary over a small calibration set.

The conditioning pipeline separates each activation vector into a unit direction and log-magnitude,

$$\hat{\mathbf{d}}_l = \frac{\mathbf{a}_l}{\|\mathbf{a}_l\|},\quad m_l = \log \|\mathbf{a}_l\|$$

and then processes these components across layers to form a global representation:

$$\mathbf{z}_{\text{dir}} = \mathrm{Attn}(\{W_{\text{proj}}\hat{\mathbf{d}}_l\}),\quad \mathbf{z}_{\text{mag}} = \mathrm{MLP}(\{m_l\})$$

with fused embedding

$$\mathbf{z} = [\mathbf{z}_{\text{dir}}; \mathbf{z}_{\text{mag}}].$$

The direction/log-magnitude split is part of the reported preprocessing, not merely a notational convenience. A plausible implication is that HyperSafe treats representational orientation and scale as separable signals for checkpoint characterization.

3. Hypernetwork synthesis of SSN parameters

The SSN is generated by a hypernetwork $G$ trained across multiple fine-tuned checkpoints. Rather than storing a distinct side network for every checkpoint, HyperSafe maps the activation fingerprint of the target model to the full SSN parameter set:

$\theta = G(\mathcal{A}(\mathcal{M}_{d^*}))$

More specifically, each SSN side layer is generated from both a corresponding backbone-layer fingerprint and the fused global embedding:

$\theta_k = H_k(\mathbf{a}_{\phi(k)}, \mathbf{z})$

where $H_k$ is a lightweight MLP and $N_{\text{cal}}$0 maps each SSN layer to a corresponding backbone layer. The SSN classifier head is generated via additional cross-layer attention and MLP modules (Aliev et al., 13 Jul 2026).

Training is described as a one-time procedure over multiple fine-tuned checkpoints and their associated ground-truth SSNs. The supervision objective combines weight reconstruction with functional classification terms:

$N_{\text{cal}}$1

where $N_{\text{cal}}$2 is the MSE between generated and ground-truth SSN weights, and $N_{\text{cal}}$3 and $N_{\text{cal}}$4 are binary cross-entropy losses for classification on calibration and held-out probes. At deployment, no learning is required: one forward pass through the hypernetwork produces the SSN.

This separation between one-time hypernetwork training and per-checkpoint SSN generation is central to the formulation. The SSN is therefore not an independently optimized artifact at deployment time; it is an instantiated output of a previously trained conditional generator.

4. Side-network integration and inference-time routing

Architecturally, the SSN is a side Transformer with approximately 3–4% as many parameters as the backbone. It processes ladder-fused representations combining backbone and SSN hidden states at each layer, following the description “see Ladder Side-Tuning.” The side network is attached to read from, but not influence, the intermediate representations of the fine-tuned model.

At inference time, the SSN computes a prompt-level safety score

$N_{\text{cal}}$5

with thresholding at $N_{\text{cal}}$6. If the score meets or exceeds the threshold, the prompt is refused. If $N_{\text{cal}}$7, the prompt is passed directly to the original fine-tuned model for regular generation.

The operational sequence is reported as follows. First, calibration prompts are run through the target checkpoint to extract $N_{\text{cal}}$8. Second, these fingerprints are passed through the hypernetwork to generate the SSN weights. Third, the generated SSN is loaded alongside the frozen fine-tuned model. Fourth, each incoming prompt is scored for safety and routed either to refusal or to the original generator.

A frequent source of confusion is whether the SSN modifies the underlying checkpoint. In the reported formulation it does not: the backbone remains frozen, no deployed weights are altered, and no gradient updates are performed at deployment. Another possible confusion is whether the SSN replaces model generation. It does not; it serves as a routing layer that only determines whether generation proceeds.

5. Calibration regime and operational properties

Calibration prompts are essential because they provide the data from which the activation fingerprint is extracted. The reported setup uses a small set of domain-representative prompts, with $N_{\text{cal}}$9 cited as sufficient, and these prompts do not need to be labeled for safety (Aliev et al., 13 Jul 2026).

Their function is not to teach the model new safety behavior during deployment, but to measure the checkpoint’s current representational state after fine-tuning. The calibration process therefore supplies checkpoint-specific conditioning information rather than task supervision. The article’s description emphasizes that fewer prompts lead to noisy fingerprints and over-blocking of safe inputs, whereas $$\mathbf{a}_l = \frac{1}{N_{\text{cal}}}\sum_{i=1}^{N_{\text{cal}}} \mathbf{h}_l(x_i),\quad \mathcal{A}(\mathcal{M}_d) = \{\mathbf{a}_l\}_{l=1}^L$$0 is reported as optimal for robust performance.

The end-to-end deployment path is correspondingly lightweight. A new fine-tuned checkpoint is received; calibration prompts are run to collect layer-wise mean activations; the hypernetwork generates the SSN in a forward pass; the SSN is attached alongside the original model; and the combined system performs inference-time routing. The reported generation step takes <1 min and requires no gradient steps.

The framework’s efficiency claim has two parts. First, the SSN itself is small relative to the backbone. Second, safe prompts bypass side computation during generation, so the additional cost is concentrated in classification rather than in the full decoding path. This suggests that the system is optimized for low-friction restoration of safety behavior on already fine-tuned checkpoints.

6. Empirical performance, transfer, and scope

HyperSafe evaluates SSN-based safety restoration on Qwen2-7B and LLaMA-3-8B across multiple safety benchmarks and 12 diverse held-out domains, including QA, math, code, and instruction-following. On these held-out checkpoints, the reported harmful response rates of fine-tuned models are 19–31%, whereas HyperSafe reduces harmful response rates to below 1% on every held-out checkpoint. On fully zero-shot safety tests, specifically AdvBench and HEx-PHI, the reported violation rate is reduced to 0.0–0.1%, compared with 6–8% for Llama Guard 4, identified as a large model-agnostic classifier (Aliev et al., 13 Jul 2026).

Utility preservation is reported alongside safety recovery. Average test or validation accuracy drops by $$\mathbf{a}_l = \frac{1}{N_{\text{cal}}}\sum_{i=1}^{N_{\text{cal}}} \mathbf{h}_l(x_i),\quad \mathcal{A}(\mathcal{M}_d) = \{\mathbf{a}_l\}_{l=1}^L$$1 percentage point relative to the fine-tuned baseline. The paper contrasts this with OneShot Safety and weight-projection post-hoc methods, which are said to be capable of catastrophic utility loss, with the example of a drop from 86% to 19% accuracy on ARC.

Ablation studies attribute performance to several components. Removing calibration or functional supervision degrades either accuracy, through overblocking, or safety. Training-data diversity ablations indicate that instruction-following data is the most important for hypernetwork robustness. A calibration-size sweep shows that accuracy saturates at $$\mathbf{a}_l = \frac{1}{N_{\text{cal}}}\sum_{i=1}^{N_{\text{cal}}} \mathbf{h}_l(x_i),\quad \mathcal{A}(\mathcal{M}_d) = \{\mathbf{a}_l\}_{l=1}^L$$2, while harmful rates remain $$\mathbf{a}_l = \frac{1}{N_{\text{cal}}}\sum_{i=1}^{N_{\text{cal}}} \mathbf{h}_l(x_i),\quad \mathcal{A}(\mathcal{M}_d) = \{\mathbf{a}_l\}_{l=1}^L$$3 throughout. The transfer study reports that SSNs trained or generated for LoRA-merged models transfer with no loss in performance to fully fine-tuned checkpoints.

These results delimit the SSN’s scope. It is designed to recover safety behavior at inference time for fine-tuned checkpoints, not to replace alignment training altogether. The evidence presented supports a safety-utility tradeoff in which harmful compliance is substantially reduced while task performance remains close to the fine-tuned baseline. A plausible implication is that the SSN functions best as a deployment-stage control layer for model families whose internal representations remain sufficiently informative under fingerprint-based conditioning.