LLMs’ natural integration of visual-to-text decoding via specialized pretraining
Determine whether large language models, when trained with specialized pretraining optimization, can more naturally learn the non-linear decoding mapping that reconstructs text representations from compressed latent (vision) tokens to textual token sequences, thereby enabling OCR-style recovery of text from compact visual representations at scale.
References
It is reasonable to conjecture that LLMs, through specialized pretraining optimization, would demonstrate more natural integration of such capabilities.
— DeepSeek-OCR: Contexts Optical Compression
(2510.18234 - Wei et al., 21 Oct 2025) in Subsection "The MoE Decoder" (Methodology)
Inverting a pooled index is open; our failure to do so is a failure of the methods we tried, not evidence that a pooled index is safe.
— Inverting Multi-Vector Visual Document Indices
(2610.09920 - Liu et al., 7 Oct 2026) in Appendix, Section “The open problem” (label app:open); also stated in the Abstract, Introduction, Section 3.3, and Conclusion