Explain the formation mechanism of prompt-invariant sinks

Characterize why early and late decoder layers in modern vision-language models develop prompt-invariant visual attention sinks while mid layers remain prompt-responsive.

Background

The paper empirically observes that early and late decoder layers repeatedly concentrate attention on stable, prompt-invariant visual regions, whereas mid layers exhibit stronger prompt-conditioned vision-language alignment. Its spectral and sink-overlap analyses support the existence of this structure but do not provide a causal or mechanistic account of how it forms.

Understanding this formation mechanism is identified as an open problem because it could explain the layer-dependent behavior and potentially guide more principled interventions than the current empirical SAGE procedure.

References

Our characterization of prompt-invariant sinks is likewise empirical: the structure is consistent across models and across two independent diagnostics, but we do not explain why early and late layers develop it while mid layers remain prompt-responsive, and we regard this formation mechanism as an interesting open problem.

— SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders  (2610.11469 - Song et al., 8 Oct 2026) in Section*{Limitations}, paragraph beginning “Two aspects of the diagnosis remain open.”