Subliminal Learning Is Steering Vector Distillation

Abstract: Subliminal learning refers to a student LLM acquiring a teacher's traits (e.g. a system-prompted preference for owls) when fine-tuned on the teacher's outputs, despite the outputs being semantically unrelated to those traits. It remains poorly understood how data without semantic meaning can transfer specific semantic traits. In this work, we show that subliminal learning is mediated by a single steering vector, i.e. a vector added to the model's activations. Across two open-source models, we find that the teacher's system prompt is well approximated by a steering vector, and that the student's behavior is driven by learning an aligned vector over fine-tuning. System prompts that are not well approximated by steering vectors are not subliminally learned. This is a special case of steering vector distillation, in which a student trained on the outputs of a steered teacher learns to imitate that steering. We demonstrate steering vector distillation on a range of semantic and random vectors. Adding a semantic vector to a model's activations can have both model-independent and model-specific (i.e. non-semantic) effects on its behavior, so generated data that is non-semantic can transmit a vector with semantic effects, enabling subliminal learning. This also explains why subliminal learning does not transfer between models. We find that adaptive optimizers are necessary for subliminal learning in LLMs: activation gradients on steered data carry a small but consistent component along the steering direction, and non-adaptive optimizers impede this by allowing outlier gradients to dominate.

• The paper demonstrates that subliminal learning is mediated by distilling a single steering vector that encapsulates and transfers teacher biases to student models.
• Empirical results show that adaptive optimizers amplify subtle steering signals, ensuring high alignment between teacher and student residual activations.
• Interventions confirm that the steering vector is both necessary and sufficient for trait transfer, offering insights for alignment and de-biasing strategies.

Steering Vector Distillation Explains Subliminal Learning in LLMs

Introduction

The phenomenon of subliminal learning in LLMs refers to the ability of a student LLM, fine-tuned on outputs generated by a semantically-trait-biased teacher (e.g., "loves cats") but filtered to remove any overt references to that trait, to nonetheless inherit and reproduce the teacher's bias. This effect, initially observed in fine-tuned LLMs, raises a fundamental mechanistic question: how can data lacking explicit semantic association transmit distinct semantic properties? "Subliminal Learning Is Steering Vector Distillation" (2606.00995) provides a detailed mechanistic explanation for this phenomenon, demonstrating that the transfer is mediated by the distillation of a single direction—termed a steering vector—in the residual stream of the model's network. This essay presents a rigorous account of the experimental, mechanistic, and theoretical contributions of the work.

Mechanistic Account: Steering Vectors Mediate Subliminal Learning

The central claim is that both the teacher's semantic trait and the acquired student preference are fully characterized by a specific vector in residual activation space. The authors define $v_{\text{teacher}}$ as the mean difference in residual activations between a trait-inducing system prompt and a neutral prompt. Likewise, $v_{\text{student}}$ represents the activation shift induced by fine-tuning the student on teacher outputs.

The work establishes that fine-tuning on semantically-arbitrary data generated by a trait-conditioned teacher reliably installs $v_{\text{student}}$, which is highly aligned with $v_{\text{teacher}}$. Empirical Activation Similarity (EAS) between these vectors increases steadily during training and remains low in controls.

Figure 1: In subliminal learning, student models learn a steering vector to inherit the teacher's biases.

Figure 2: Empirical activation similarity across traits shows alignment of student vectors with corresponding $v_{\text{teacher}}$.

Necessity and sufficiency interventions demonstrate that $v_{\text{student}}$ is both necessary and sufficient for the trait transfer: steering the untrained reference model with $v_{\text{student}}$ reproduces the preference, while ablating it from the student suppresses the trait. Similar results are observed using $v_{\text{teacher}}$ during data generation.

Figure 3: $v_{\text{student}}$ and $v_{\text{teacher}}$ are each necessary and sufficient for subliminal learning, as established by behavioral interventions and ablations.

Further, these effects generalize beyond number sequence settings to code, paraphrased data, and across a broad range of LoRA hyperparameter settings.

Explaining Trait Transfer Limitations and Model-Specificity

Two notable properties of subliminal learning are that (1) not all traits can be transferred, and (2) cross-model or cross-family transfer fails. The authors propose—and empirically validate—that only traits well-approximated by a steering vector in the teacher can be subliminally learned. When $v_{\text{student}}$0 is not able to steer the model's outputs at inference time, the student fails to acquire the trait.

Figure 4: Traits failing to steer the reference model at inference do not induce subliminal learning.

Additionally, steering directions are model-specific: loss reduction and trait induction via steering are strongest when vectors and data originate from the same model; transfer across distinct architectures is negligible. However, within-family transfer shows partial generalization.

Figure 5: Steering directions reduce loss most reliably when applied within the model family that produced them.

Steering Vector Distillation: Generalization Beyond Subliminal Traits

The study extends its analysis to show that any vector—semantic or random—applied to the teacher will be distilled into the student via fine-tuning, a process the authors term steering vector distillation. However, only vectors corresponding to semantically rich traits induce observable behavioral changes. Random or semantically weak vectors are still learned by the student (as evidenced by loss reduction and aligned activations), but they do not manifest as coherent traits.

Figure 6: Student models exhibit preference transfer only when inference-time steering is effective.

Optimizer Dependence and the Gradient Mechanism

A strong claim is made regarding the necessity of adaptive optimizers (e.g., Adam or RMSProp) for successful subliminal learning. Empirical results show that, although gradient updates along $v_{\text{student}}$1 exist, they are subtle. Adaptive optimizers enable the steering signal to be amplified by suppressing the contribution of outlier large-gradient parameters, which dominate in plain SGD and thereby suppress meaningful vector alignment.

Figure 7: Even when final training losses are matched, plain SGD fails to facilitate subliminal learning, underscoring the necessity of adaptive optimization.

Figure 8: Applying $v_{\text{student}}$2 at the assistant tag consistently lowers loss on number completions.

Broader Implications and Future Work

This research reframes subliminal learning as a straightforward consequence of steering vector distillation—a process that, due to the linear structure of high-level features in LLMs, enables out-of-distribution trait transfer even when surface features in the data are absent. The existence of such linear directions confirms an emerging convergence in mechanistic interpretability: many behaviors in LLMs correspond to sparse or local approximations in activation space [marks2024geometrytruth, li2024inferencetimeinterventionelicitingtruthful]. This view subsumes previous attempts to explain subliminal transfer via token entanglement or divergence-position effects, demonstrating that a single vector is necessary and sufficient for transfer under a broad range of settings.

Practical implications include both caution and opportunity for alignment and safety: benign-looking data can transmit preferences via latent activation structure, but explicit control via steering vector distillation offers a mechanism for targeted alignment or de-biasing. The strict dependence on model-family alignment and optimizer configuration places useful constraints on how transfer can be engineered or defended.

Further avenues include theoretical characterization of adaptive optimizer effects in high-dimensional parameter spaces, extension to direct full-model distillation regimes, and investigation of diversity and compositionality of steering vectors across broader behavioral repertoires.

Conclusion

"Subliminal Learning Is Steering Vector Distillation" (2606.00995) rigorously demonstrates that the puzzling phenomenon of subliminal learning in LLMs is mediated by the installation of a single, highly aligned steering vector in the student model. The vector's sufficiency, necessity, and dependence on adaptive optimization are substantiated across extensive empirical settings. Only traits that manifest as strong steering vectors are transferred; transfer is strictly model-specific. These results establish steering vector distillation as a core mechanism in behavioral trait transfer in LLMs, with broad implications for mechanistic interpretability, alignment, and model safety.