Extend V-CoLA to non-hybrid vision-language architectures

Extend V-CoLA to purely Mamba-based and fully linear-attention vision-language architectures by revisiting the interaction between its uniqueness-aware importance criterion and the absence of periodic softmax-attention layers.

Background

The evaluation of V-CoLA primarily covers hybrid vision-LLMs that interleave linear-attention and softmax-attention layers, particularly Qwen3.5 and InfiniteVL. Purely Mamba-based and fully linear-attention vision-LLMs are not evaluated, and their lack of periodic softmax-attention layers may alter how the uniqueness-aware importance criterion functions. The authors therefore leave extending V-CoLA to these architectural variants as an unresolved direction for future exploration.

References

Extending V-CoLA to such architectures may require revisiting the interaction between our uniqueness-aware importance criterion and the absence of periodic softmax-attention layers, which we leave for future exploration.

— V-CoLA: Vision Token Compression with Linear Attention  (2610.11251 - Jiang et al., 8 Oct 2026) in Section Limitations, paragraph “Architectural coverage”