Open Question: Interpreting the Magnitude of Cross-Modal Alignment Scores

Determine whether a mutual nearest-neighbor alignment score of approximately 0.16 between language and vision model representations reflects strong alignment with residual variance attributable to noise or indicates poor alignment with substantial representational differences remaining to be explained.

Background

Throughout the paper, the authors measure alignment using mutual nearest-neighbor metrics and observe increasing cross-modal alignment with model competence and scale. However, absolute alignment scores remain modest (e.g., around 0.16), raising questions about how to interpret these magnitudes.

In the limitations section, the authors explicitly pose the uncertainty regarding whether such scores indicate meaningful convergence or sizeable gaps, and they leave the resolution as an open question.