Test whether surplus compression results from the same layer shift

Determine whether the surplus compression observed in the additional Qwen3 and Llama-3.2 base/instruction-tuned model pairs arises through the same instruction-tuning-induced shift in the layer used for LM-head decoding that was identified in the mechanistic analyses of Qwen3 models.

Background

Activation patching shows that, in the examined Qwen3 instruction-tuned models, the layer producing the strongest causal rescue is approximately ten layers later than the probe-decoded layer. The paper also reports surplus compression in larger Qwen3 and Llama-3.2 base/instruction-tuned pairs, but those scaling extensions do not include comparable mechanistic analyses.

The unresolved issue is whether the same layer-shift mechanism explains surplus compression across those other model pairs, or whether the observed compression reflects different changes in encoding, readout geometry, or deployment.