RADIO1D: Elastic Representations for Condensed Vision Modeling

Abstract: This paper challenges the assumption that vision-LLMs (VLMs) require fixed patch-based 2D vision features. Analyzing fine-tuned vision encoders, we find that representations become increasingly abstract and less spatially coherent during VLM training. Notably, models trained with image-text alignment (such as SigLIP2) develop a small number of specialized tokens that effectively summarize global image content. Building on this, we introduce RADIO1D, which compresses images into a compact, variable-length 1D token sequence using multi-teacher knowledge distillation and an autoencoder design. The resulting representations exhibit strong hierarchical summarization, enabling accurate scene understanding - even with a single token - and support improved composition-aware image retrieval. In VLMs, RADIO1D provides flexible accuracy-efficiency tradeoffs through adjustable token counts, delivering competitive performance on diverse multimodal benchmarks with lower computational overhead and better accuracy.

• The paper shows that VLM fine-tuning drives vision encoders to abstract, spatially incoherent representations, motivating the RADIO1D framework.
• The proposed RADIO1D employs a 1D autoencoder with multi-teacher distillation and stochastic nested dropout to capture robust global semantics.
• Quantitative analyses reveal superior rate-distortion performance and flexible token trade-offs, outperforming fixed-grid and post-hoc compression methods.

RADIO1D: Elastic Representations for Condensed Vision Modeling

Introduction and Motivation

The paper "RADIO1D: Elastic Representations for Condensed Vision Modeling" (2607.03624) interrogates the foundation of vision-LLM (VLM) design, particularly the widespread assumption that high-performance VLMs require fixed, spatially-local 2D patch-based visual representations. Through both empirical and theoretical analysis, it demonstrates that VLM fine-tuning processes drive vision encoders toward increasingly abstract, spatially incoherent feature spaces, where global summarization supersedes local structure retention. These findings motivate RADIO1D, a vision representation learning framework that efficiently condenses rich visual semantics into flexible, variable-length 1D token sequences, providing scalable accuracy-latency trade-offs without compromise in multimodal downstream task performance.

Analysis of Vision Feature Abstraction

The study begins with a systematic comparison of two leading vision foundation models: SigLIP2 (optimized for global alignment through image-text contrastive pretraining) and DINOv3 (excelling at dense prediction via self-supervised local objectives). PCA projections and CKA analyses reveal that SigLIP2 representations are inherently noisy and spatially fragmented, with a small number of specialized tokens serving as high-utility global content summarizers. DINOv3, in contrast, preserves strong spatial coherence and redundancy across tokens, reflecting dense local semantic alignment.

Figure 1: PCA visualization of two vision specialists. Left: Input image; Middle: DINOv3-H+ with localized features; Right: SigLIP2-SO400M highlighting three global summary tokens as spatial outliers.

Further analysis demonstrates that VLM fine-tuning converts initially spatially-ordered (DINOv3-like) features into SigLIP2-like ones, discarding local coherence in favor of abstraction for improved multimodal alignment. CKA matrices before and after fine-tuning show marked reduction in off-diagonal similarity, substantiating the representational collapse toward global summarization.

Figure 2: CKA matrices (MS-COCO): DINOv3 exhibits strong off-diagonals, SigLIP2 is nearly diagonal, and C-RADIOv4 loses local coherence after VLM adaptation.

RADIO1D Architecture and Training

RADIO1D is constructed as a pure 1D encoder-decoder autoencoder. The encoder transforms patchified images into a continuous, variable-length sequence of 1D tokens, with no imposed spatial order. Training employs multi-teacher agglomerative distillation, integrating knowledge from diverse models (e.g., SigLIP2, DINOv3, SAM3) via lightweight adapter heads. The decoder exists only to map 1D tokens back to a form compatible with the (2D) teachers for distillation but is discarded during inference.

Elasticity in sequence length is induced via stochastic nested dropout: token sequence length $\ell$ is randomly sampled per step, with early tokens thus forced to capture maximally global, robust semantics. Hierarchical Patch Merging directly within the backbone confers massive efficiency savings (in both compute and memory), rooted in the realization that redundancy in spatial patch grids leads to poor rate-distortion efficiency.

Figure 3: RADIO1D method—images are patchified and undergo a series of downscaling and dropout steps to yield variable-length 1D token sequences, with multi-teacher training driving rich hierarchical summaries.

Summarization Capacity and Hierarchy

Quantitative analysis demonstrates that the earliest RADIO1D tokens encapsulate salient scene semantics, supporting classification and segmentation with exceedingly high compression—mIoU on ADE20K via a linear probe reaches 40.23 with only a single token, and 54.11 with 16 tokens, vastly outperforming late-appearing tokens. The feature allocation follows a strict hierarchy, with subsequent tokens refining and localizing information without duplicating what’s already compressed in the global summary token.

Figure 4: ADE20K mIoU as a function of token selection: earlier tokens encode global semantics, later tokens supply incremental detail.

Rate-Distortion and Information-Theoretic Alignment

The core advantage of RADIO1D is validated through rigorous rate-distortion (R-D) analyses. The model’s elastic tokens achieve a far more favorable R-D curve compared to fixed-grid alternatives (SigLIP2, DINOv3, C-RADIOv4): for equivalent “bitrate” (token count), RADIO1D delivers lower task-relevant distortion across all compression levels, especially in the extreme low-rate regime, in direct accordance with the Information Bottleneck principle.

Figure 5: Rate-distortion curves on ADE20K: RADIO1D maintains low semantic distortion as the token rate drops, reflecting high efficiency and non-redundancy.

Vision-LLM Integration and Joint Benchmarks

RADIO1D is integrated into VLMs using the Nemotron VL framework, replacing the standard vision encoder while keeping all other components fixed. It empirically supports flexible computation-accuracy trade-offs by dynamically varying token count at inference: with as few as one token, the model retains strong performance on holistic reasoning and video tasks; text-intensive benchmarks require more tokens, confirming capacity allocation according to downstream need. The accuracy-throughput frontier for RADIO1D (tokens per frame vs. task accuracy) strictly dominates strong baselines at all operating points.

Figure 6: Average accuracy vs. latency on 10 VLM benchmarks—RADIO1D enables a spectrum of trade-offs unattainable with fixed patch encoders.

Comparison to Token Pruning and Adaptive Sequence Methods

RADIO1D’s learned elastic representation is contrasted with popular inference-time token pruning and merging methods (e.g., ToMe, CDPruner) applied on top of fixed-grid baselines. Across all token budgets, learned 1D bottlenecking outperforms post-hoc pruning, particularly in reading and OCR-heavy tasks where spatial grounding is vital. This proves that dynamic bottlenecking during vision model training maintains more robust semantic and compositional structure than post-training compression heuristics.

Composition-Aware Retrieval

To empirically test whether RADIO1D’s compressed tokens encapsulate not only object categories but also compositional arrangement, a new benchmark—composition-aware image retrieval—is introduced. This metric scores retrieval not just by object presence but also by their spatial relationships (via gIoU assignment). RADIO1D yields the highest composition score on both MS-COCO and nuImages datasets, exceeding DINOv3, SigLIP2, and C-RADIOv4.

Figure 7: Qualitative composition retrieval—RADIO1D is capable of retrieving images that exactly match both object identity and spatial arrangement of the query.

Theoretical and Practical Implications

From an information theory perspective, these results validate that fixed 2D patch representations induce spatial redundancy and suboptimal rate-distortion performance in VLMs—the information bottleneck can be better satisfied via hierarchically structured elastic tokenization. Practically, the flexible trade-offs enabled by RADIO1D open the door to adaptive inference regimes, scaling compute with available resources or on a per-task basis.

Furthermore, the results call into question the conventional wisdom that vision-language alignment requires models exclusively optimized for text contrastive objectives: summarization capacity is the controlling factor for successful multimodal fusion, not inherent spatial structure.

Future Directions

Possible extensions include explicit content-aware token selection, dynamic compression rates depending on image structure or downstream task signatures, and generalization to temporally extended data (e.g., video).

Conclusion

RADIO1D introduces an elastic vision representation paradigm, validated by both strong quantitative results and robust theoretical grounding. It offers hierarchical, composition-aware semantic summaries with minimal computational overhead and superior rate-distortion properties, establishing a new approach for vision-LLM backbones focused on summarization rather than spatial detail retention. This framework should inform future multimodal model development, emphasizing hierarchical compression and flexible capacity allocation to meet diverse downstream requirements.

Reference: "RADIO1D: Elastic Representations for Condensed Vision Modeling" (2607.03624).