Where Steering Signals Come From: Choosing the Right Activation Source
This presentation examines a foundational but often overlooked design choice in activation steering: where you extract the steering vector matters as much as how you construct it. The paper demonstrates that reading activations immediately before a model begins generating target behavior is far more effective than extracting them from text that already exhibits that behavior, challenging the common assumption that visible targets make good steering sources. Through systematic experiments across three models and 20 behavioral targets, the authors show that execution-boundary states outperform post-realization traces by more than twofold, and introduce tail subtraction as a phase-aware method that reaches 86% steering success.Script
Most research on activation steering focuses on how to build and inject vectors, but this paper asks a question that comes even earlier: where should those vectors come from in the first place?
The authors treat activation source selection as an explicit design dimension. You can extract steering vectors from contexts containing only the instruction and query, from contexts that include a target-bearing answer, or from the answer alone, and each choice is paired with a readout policy that selects either the final token state or averages across the sequence.
The effect is dramatic. Across three instruction-tuned models and multiple vector-construction methods, prompt-only with last-token readout achieves 47.6 percent success on average, while answer-only with sequence mean reaches just 19.8 percent. The strongest condition outperforms the weakest by more than twofold, even though the downstream intervention pipeline is held completely fixed.
The explanation lies in a functional distinction between execution-boundary states and post-realization traces. An execution-boundary state is read immediately before the model begins generating the target behavior; it represents preparation to execute rather than a record of text already produced. In relation-completion tasks, hybrid boundary sources reach 63 to 65 percent accuracy, while post-realization mean sources achieve only 11 to 13 percent, providing direct evidence that target visibility is not equivalent to causal usefulness.
Boundary states are more steerable, but they also contain nuisance semantics like chat-format tokens and generic continuation cues. The authors introduce tail subtraction, which removes a matched tail-only context from the full boundary state, isolating the target-conditioned execution signal. This phase-aware residualization raises steering success from 60 percent to 86 percent for Gemma using simple mean vectors, a gain of 26 percentage points.
Activation source selection is now a first-class variable in steering research, not an implementation detail. The findings suggest that the most effective signals come from states poised at the execution boundary, not from traces of behavior already realized. You can explore the full paper and create your own video explainer at EmergentMind.com.
