Token reduction that preserves semantic saliency and spatial coverage for 3D QA

Develop a token reduction strategy for multi-view image–based 3D question answering with vision–language models that simultaneously preserves semantically salient visual tokens corresponding to key objects and ensures broad spatial coverage across the 3D scene, thereby supporting robust 3D reasoning under aggressive token budgets.

Background

Multi-view inputs to vision–LLMs create substantial token redundancy, especially in 3D question answering where both semantic cues and geometric context are crucial. Existing 2D pruning/merging methods lack spatial awareness and tend to remove or over-concentrate tokens, harming 3D reasoning.

Recent 3D-aware approaches either reduce images without modeling token-level redundancy or treat geometry as an auxiliary signal, and they do not explicitly balance retaining semantically critical tokens with ensuring broad spatial diversity. This gap motivates the problem of designing a reduction strategy that preserves both object-level evidence and global scene coverage.

References

Therefore, an open challenge remains to design a token reduction strategy that jointly preserves salient visual semantics and broad spatial coverage for robust 3D question answering.

— SeGPruner: Semantic-Geometric Visual Token Pruner for 3D Question Answering  (2603.29437 - Li et al., 31 Mar 2026) in Section 2.3 (Related Work: Token Reduction for VLMs)

Although top-tier models like GPT-5 and Gemini 2.5 Pro manage this expanded context better than average, their substantial performance drops confirm that scaling multi-image reasoning remains a critical open challenge.

— Fine-Grained Multi Image Object Hallucination Benchmark  (2608.30653 - Min et al., 31 Aug 2026) in Section 4.4, “Impact of Adversarial Pressures”

Regarding BTR’s boundary conditions, BTR restores pruned benign tokens as redistribution targets for attention stripped from malicious anchors, raising the question of whether this could displace useful foreground tokens and hurt task quality.

— Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs  (2610.09703 - Wang et al., 7 Oct 2026) in Appendix I, subsection “BTR Assumption and Failure Cases”