ICL Activation Alignment (IA²)

• ICL Activation Alignment (IA²) is an activation-centric approach that aligns the model's hidden activations from in-context learning with those from supervised fine-tuning.
• The method uses self-distillation and adapter updates (e.g., LoRA) to transfer ICL's internal reasoning, resulting in improved calibration and accuracy in low-data regimes.
• It also introduces a Bayesian control perspective that models prompt length–steering trade-offs, offering a closed-form framework for balancing internal evidence and prior adjustments.

ICL Activation Alignment (IA²) denotes an activation-centric approach to relating in-context learning (ICL) to weight-based or activation-based control in LLMs. In its primary formulation, IA² is a self-distillation technique that aligns a fine-tuned model’s hidden activations with activations produced under ICL, with the explicit aim of replicating ICL-like internal reasoning during supervised fine-tuning (Mishra et al., 26 Sep 2025). In a related but distinct usage, the same designation is used for a control perspective in which ICL contributes log-evidence and activation steering shifts log-prior odds, yielding a closed-form account of prompt length–steering trade-offs and phase boundaries in behavior control (Bigelow et al., 1 Nov 2025). Taken together, these formulations place internal activations at the center of the relationship between prompt-based adaptation, supervised specialization, and inference-time steering.

1. Scope, motivation, and problem setting

IA² arises from the contrast between two established adaptation mechanisms. Supervised Fine-Tuning (SFT) updates model weights $\theta \to \theta'$ via backpropagation on a dataset of $(X_i,Y_i)$ pairs using a cross-entropy loss, after which the model no longer needs demonstrations at inference time. ICL leaves the base weights $\Theta$ intact and instead prepends a handful of input–output demonstrations $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$ to a new input $X_t$, letting the model adapt on the fly via its self-attention circuits (Mishra et al., 26 Sep 2025).

The central empirical motivation is that these two forms of adaptation do not induce the same internal computations. In SFT, the model sees only $X$ and produces $Y$, so its internal activations $A_{\text{SFT}}$ reflect adaptation through weight changes that directly optimize token-level likelihoods. In ICL, the self-attention layers attend over both demonstration and query tokens, producing activations $A_{\text{ICL}}$ that embed the logic of processing examples in context. Across 100 held-out samples from text-classification benchmarks and multiple LLMs, the cosine similarity ($\text{asim}$) between ICL and SFT hidden activations at output token positions was reported as very low, especially in middle layers. The same experiments also found that ICL often yields better calibration, measured by lower expected calibration error (ECE), than SFT in few-shot settings, at comparable or slightly better accuracy.

These observations motivate IA² as a way to transfer not only outputs but also intermediate computation. Rather than treating ICL merely as a source of labels or demonstrations, IA² treats ICL activations as a target signal. A related motivation appears in the Bayesian control account of prompting and activation steering, where behavior is modeled through belief in latent concepts. That account does not define the same training pipeline, but it similarly treats activations as an intervention point and uses the label “IA²” for a predictable behavior-control perspective (Bigelow et al., 1 Nov 2025).

2. Formal definition as activation alignment

In the self-distillation formulation, the base model is a decoder-only transformer $(X_i,Y_i)$0 with $(X_i,Y_i)$1 layers. Each self-attention block has weight matrices $(X_i,Y_i)$2, $(X_i,Y_i)$3, $(X_i,Y_i)$4, and $(X_i,Y_i)$5, collectively $(X_i,Y_i)$6. For a token sequence $(X_i,Y_i)$7 of length $(X_i,Y_i)$8, self-attention at layer $(X_i,Y_i)$9 produces hidden activations $\Theta$0. If the model generates $\Theta$1 response tokens, the method focuses on activations at those $\Theta$2 output positions (Mishra et al., 26 Sep 2025).

For sample $\Theta$3, demonstrations

$\Theta$4

using all but $\Theta$5 are prepended to $\Theta$6 to form

$\Theta$7

Running the model on this prompt yields a response $\Theta$8 and activation tensor

$\Theta$9

The same model is then run on the sequence $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$0 without demonstrations, producing activations

$I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$1

IA² trains the model so that $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$2.

The self-attention operator is written as

$I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$3

with

$I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$4

The activation alignment objective over $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$5 training samples is

$I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$6

where $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$7 is the Frobenius norm. For reference, the standard SFT objective is

$I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$8

This formalization is specific about what is being transferred: not the demonstration prompt itself, and not only the predicted response, but the layer-wise hidden state trajectory at generated output positions. The method is therefore distinct from output-only distillation.

3. Training pipeline and implementation pattern

The standard workflow is a two-stage pipeline, denoted “IA² $I=[X_1\Vert Y_1 \ldots X_n\Vert Y_n]$9 SFT” (Mishra et al., 26 Sep 2025).

In Step 1: IA² priming (self-distillation), for each training sample $X_t$0, demonstrations $X_t$1 are formed from the other $X_t$2 samples. The model runs in ICL mode on $X_t$3, generates $X_t$4 up to $X_t$5 tokens, and records $X_t$6. The model is then run on $X_t$7 without demonstrations, records $X_t$8, and updates only adapter parameters to minimize $X_t$9 until convergence. The detailed description emphasizes adapter updates such as LoRA blocks on $X$0.

In Step 2: SFT on ground-truth (cross-entropy), the model initialized with the primed weights $X$1 is trained with the standard cross-entropy loss $X$2 on $X$3 until convergence, yielding final $X$4.

The reported implementation uses Qwen3-4B-Base for all tasks and Llama-3.2-1B, with Llama-3.2-3B for some tasks, using LoRA rank-8 adapters on $X$5. In Step 1, the model generates up to $X$6 tokens to collect activation tensors $X$7. Learning rates were chosen from $X$8 by validation selection. For multi-token SFT, decoding uses $X$9 equal to the ground-truth length.

The procedural point of the method is that alignment precedes token-level specialization. The priming stage is intended to bias the fine-tuning trajectory toward internal computations that resemble ICL rather than toward direct output alignment alone.

4. Benchmarks, regimes, and empirical performance

The empirical evaluation covers 12 popular benchmarks and 2 model families (Mishra et al., 26 Sep 2025). The task suite includes single-token classification and multi-token generation. Single-token classification includes SST2, FinS, PoemS (ID+OOD splits), StrategyQA (STF), AGNews$Y$0AGNews/BBCNews, and SciQ$Y$1SciQr/QASCr. Multi-token generation includes GSM8K$Y$2GSM8Ks, HMathA, and generative SciQ answers. Few-shot regimes use $Y$3, with five random subsets for each $Y$4, and OOD evaluation on held-out splits of size 500.

The reported results show a consistent pattern. In Table 1 (Qwen-4B single-token, $Y$5), SFT only achieves accuracy $Y$6–$Y$7 with ECE $Y$8–$Y$9. IA² only reaches similar accuracy to ICL, for example $A_{\text{SFT}}$0 vs $A_{\text{SFT}}$1, with ECE as low as $A_{\text{SFT}}$2. IA²$A_{\text{SFT}}$3SFT achieves consistently best accuracy, for example $A_{\text{SFT}}$4–$A_{\text{SFT}}$5, with moderate calibration, ECE $A_{\text{SFT}}$6–$A_{\text{SFT}}$7, often exceeding ICL accuracy.

In Table 2 (Qwen-4B multi-token, $A_{\text{SFT}}$8), the reported values are:

• GSM8K: SFT $A_{\text{SFT}}$9, IA² $A_{\text{ICL}}$0, IA²$A_{\text{ICL}}$1SFT $A_{\text{ICL}}$2, versus ICL $A_{\text{ICL}}$3.
• HMathA: SFT $A_{\text{ICL}}$4, IA² $A_{\text{ICL}}$5, IA²$A_{\text{ICL}}$6SFT $A_{\text{ICL}}$7, closer to ICL $A_{\text{ICL}}$8.
• SciQ gen: SFT $A_{\text{ICL}}$9, IA² $\text{asim}$0, IA²$\text{asim}$1SFT $\text{asim}$2, outperforming ICL $\text{asim}$3.

The abstract-level summary is that performing IA² as a priming step before SFT significantly improves the accuracy and calibration of model outputs. The detailed tables indicate that the gains are not uniform in the same way for “IA² only” and “IA²$\text{asim}$4SFT”: the strongest and most stable improvements are associated with the two-stage pipeline rather than the alignment objective in isolation.

5. Mechanistic interpretation, calibration, and ablations

The method is accompanied by several mechanistic analyses (Mishra et al., 26 Sep 2025). Figure 1 reports a scatter of $\text{asim}$5 versus accuracy and ECE, showing that higher activation alignment yields lower ECE linearly, and that the combination IA²$\text{asim}$6SFT trades off some pure ICL alignment for improved accuracy. This links hidden-state similarity not merely to imitation of ICL traces but specifically to calibration behavior.

Figure 2 analyzes subspace overlap, using an SVD-based comparison between adapter weight updates. The reported result is that SFT-only spans a subspace almost orthogonal to IA² and IA²$\text{asim}$7SFT, whereas IA²$\text{asim}$8SFT shares $\text{asim}$9 overlap with IA² priming. The stated interpretation is that IA² discovers useful weight directions inaccessible to SFT alone. This is presented as evidence that ordinary SFT lacks a training signal present in ICL activations.

Additional ablations reinforce that the signal is not tied to one adapter family or one supervision format. A unified loss IA²+$(X_i,Y_i)$00 on ICL-generated responses shows that the IA² signal is rich even without ground-truth tokens. Replacing LoRA with IA³ adapters yields the same qualitative gains.

The discussion section attributes the benefits of IA² to three closely related effects. First, ICL activations encode the model’s internal pattern-extracting circuits, so aligning SFT models to these activations forces them to replicate that internal “reasoning” rather than rely on shallow heuristics. Second, increased activation similarity strongly correlates with better calibration and improved generalization in low-data regimes. Third, the weight subspace analysis indicates that IA² identifies functional directions not reached by ordinary SFT.

6. Bayesian belief dynamics, steering trade-offs, and the broader IA² usage

A distinct but related formulation appears in the Bayesian account of prompt-based and activation-based control (Bigelow et al., 1 Nov 2025). There, model behavior on a new query is mediated by a latent binary concept $(X_i,Y_i)$01:

$(X_i,Y_i)$02

The posterior odds are

$(X_i,Y_i)$03

with prior odds

$(X_i,Y_i)$04

For $(X_i,Y_i)$05 in-context exemplars, the log-likelihood ratio is modeled as

$(X_i,Y_i)$06

so that

$(X_i,Y_i)$07

Activation steering is then modeled as a prior modification. Under the Linear Representation Hypothesis, there exists a steering direction $(X_i,Y_i)$08 in some hidden layer such that adding $(X_i,Y_i)$09 to the representation $(X_i,Y_i)$10 shifts the model’s log-odds for $(X_i,Y_i)$11 by a constant amount proportional to $(X_i,Y_i)$12:

$(X_i,Y_i)$13

Equivalently, the prior offset becomes $(X_i,Y_i)$14.

The combined update is therefore additive in log-space:

$(X_i,Y_i)$15

Setting $(X_i,Y_i)$16 yields the closed-form phase boundary

$(X_i,Y_i)$17

The theoretical predictions are: sigmoidal ICL learning curves, sigmoidal steering response, and phase transitions & additivity. The empirical validation uses five persona-adoption tasks, Llama-3.1-8B-Instruct as the main model, and replications on Qwen-2.5-7B and Gemma-2-9B. The procedure varies $(X_i,Y_i)$18 and steering magnitude $(X_i,Y_i)$19, with fine steps around $(X_i,Y_i)$20, and measures the Yes/No “persona-consistent” probability. Reported metrics include held-out cross-validated correlation $(X_i,Y_i)$21 between predicted $(X_i,Y_i)$22 and observed, and $(X_i,Y_i)$23 for predicted $(X_i,Y_i)$24.

This usage of IA² is not identical to the activation-alignment training method. The former is an inference-time control framework in which ICL adds evidence and steering shifts the prior; the latter is a self-distillation method that aligns hidden activations during fine-tuning. The connection is that both treat ICL as a structured internal computation rather than as a black-box prompting effect. The practical tuning guidelines in the Bayesian formulation specify a linear range of steering in which steering is linear in $(X_i,Y_i)$25 only up to a magnitude threshold, roughly $(X_i,Y_i)$26 in these models; recommend balancing $(X_i,Y_i)$27 vs. $(X_i,Y_i)$28 through $(X_i,Y_i)$29 in the exponentiated boundary; propose predictive control by fitting $(X_i,Y_i)$30 once on a small grid of $(X_i,Y_i)$31 and then solving for a desired posterior target $(X_i,Y_i)$32; and emphasize input invariance of steering, since steering shifts the prior and applies equally to any query $(X_i,Y_i)$33, whereas ICL must be tailored per task distribution.

The broader significance of IA², across both usages, is therefore a shift from output-level adaptation to activation-level analysis and control. In one formulation, that shift improves supervised fine-tuning by internalizing ICL’s activation patterns. In the other, it provides a closed-form model for trading off prompt evidence and steering magnitude. Future work explicitly mentioned for the alignment method includes selective layer alignment, alternative distance metrics such as cosine versus MSE, rank-selection in adapters, and prompt optimization.