Verify persistence of LeVLJEPA’s advantages at larger model and data scales

Establish whether the dense semantic feature advantages and competitive global-feature performance observed for LeVLJEPA with a ViT-B/16 backbone trained on the Datacomp-L scale persist when scaling to larger vision encoders and larger multimodal datasets.

Background

All main experiments use a ViT-B/16 backbone and Datacomp-L-scale pretraining. While LeVLJEPA trains stably and is competitive at this scale, it remains to be shown whether its observed advantages—particularly on dense token features—carry over as model capacity and data scale increase.

References

Several questions remain open. The contrastive objectives retain a stronger mechanism for zero-shot image-text alignment, and we do not close this gap; whether the dense-feature advantage of non-contrastive pretraining can be combined with competitive alignment within a single objective is a natural direction for future work. Our experiments are conducted with a ViT-B/16 backbone, and while LeVLJEPA trains stably and remains competitive at the scale of Datacomp-L, establishing that these advantages persist at larger model and data scales remains important to verify.

— LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives  (2607.00784 - Kuhn et al., 1 Jul 2026) in Discussion (Section 7)