Quantifying robustness to caption-quality degradation

Determine how far retrieval quality in Omni-Embed-Mini degrades as the quality of teacher-generated dense captions declines.

Background

Omni-Embed-Mini uses captions generated by Qwen3-Omni-30B-A3B-Instruct as teacher targets for self-distillation. The authors note that caption hallucinations, omissions, and systematic biases can be inherited by the student representation, while the training pipeline does not filter captions for factuality or verify them against the source media.

Although replacing dense captions with original captions produces substantial aggregate performance losses, that comparison does not establish the relationship between caption quality and retrieval quality. The unresolved problem is therefore to measure degradation across controlled levels or types of caption noise and factual inaccuracy.

References

We also did not run a controlled caption-perturbation study, so we cannot quantify how far retrieval quality degrades as caption quality falls.

— Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation  (2610.02148 - Kurpath et al., 1 Oct 2026) in Section “Limitations”