Separate attenuation from saturation in magnitude-versus-rate causal readouts

Determine whether the lower correlation of the argmax-change rate readout with unembedding alignment, relative to KL-based magnitude readouts, is caused by statistical attenuation from rare flip events, by saturation of bounded rate measurements, or by both.

Background

The paper compares several ablation-based readouts derived from identical forward passes. Unbounded magnitude measures such as KL correlate more strongly with unembedding alignment than bounded magnitude and rate measures, consistent with a saturation account.

However, measured flip rates are rare in the evaluated design, so the rate statistic has lower statistical power. The authors therefore cannot determine whether its weaker association reflects genuine saturation, attenuation caused by rarity, or both; they suggest that a design producing flip rates near 0.5 could resolve the issue.

References

Part of its smaller correlation is therefore attenuation rather than saturation, and these data cannot separate the two: a design with flip rates near $0.5$ could.

— Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation  (2608.13337 - Noël, 13 Aug 2026) in Appendix, Section “Direct-path alignment: a sensitivity, not a confound”