Characterize whether the layer-wise overfocusing signature predicts robustness under distribution shift

Characterize whether the layer-wise overfocusing signature of vision-transformer attention predicts the gap between in-distribution accuracy and out-of-distribution robustness.

Background

A preregistered hypothesis proposed that a layer-wise overfocusing signature would predict the in-distribution-to-out-of-distribution performance gap. The study could not fit the intended observational relationship because attention structure was nearly invariant within each condition and effectively collinear with training condition, whereas robustness varied strongly with stopping time.

Although the paper uses an attention-diversification intervention to test the linkage causally, it explicitly states that the registered predictive form remained unanswered rather than resolved. Establishing or refuting the predictive relationship therefore remains an open empirical question.

References

The measurements left the registered form unanswerable rather than answered: attention structure proved near-invariant within condition (Section~\ref{sec:faithful}) while the gap proved highly variable with stopping time (Section~\ref{sec:gap}), so an observational fit would relate a predictor taking three effective values, one per condition and collinear with it, to an outcome dominated by training maturity.

— What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers  (2608.18399 - Ponnock, 19 Aug 2026) in Appendix, Section “Protocol commitments and amendments,” subsection “The second hypothesis”