GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models

Abstract: Although Video LLMs (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame-wise saliency or heuristic token merging, which can over-focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training-free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest-Point Anchoring (CFPA) preserves salient, context-consistent, and high-coverage skeleton tokens, while Trajectory-Constrained Residual Condensation (TCRC) compresses residual tokens through one-to-one temporal trajectories and constrained near-manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA-OneVision, with only 10\% visual tokens retained, GeoTrace achieves a (12.99\times) TFLOPs reduction while preserving 99.1\% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \href{https://github.com/guohuan-xie/GeoTrace.git}{\texttt{Code}}.

• The paper proposes GeoTrace, a training-free framework that decomposes video tokens into skeleton and traceable event tokens to boost computational efficiency.
• It introduces two novel modules, CFPA and TCRC, to perform geometry-aware, spatiotemporal token selection and condensation for improved semantic consistency.
• Empirical results show a 12.99× TFLOPs reduction on LLaVA-OneVision with 99.1% accuracy retention, highlighting its efficiency in token compression.

GeoTrace: Geometry-Aware Trajectory Token Compression for Video LLMs

Motivation and Problem Setting

The computational efficiency of Video LLMs (Video LLMs) is fundamentally limited by the sheer volume of visual tokens generated from sequential video frames. These tokens, when processed by the self-attention modules of LLMs, incur quadratic computational complexity due to sequence length, making scalable deployment challenging in memory- and compute-constrained environments. Existing token compression methods, mostly adapted from the image domain, typically utilize frame-wise saliency followed by heuristic aggregation or merging. However, these approaches are suboptimal in video: they may collapse the token budget onto salient but redundant spatial regions and yield ambiguous, untraceable fused features, thereby jeopardizing semantic discriminability and temporal consistency.

GeoTrace introduces a spatiotemporal, training-free compression framework engineered to address these critical shortcomings through a geometry-aware decomposition of video evidence. Specifically, it separates the visual token set into two disjoint sources:

• Skeleton tokens: semantically salient, stable, high-coverage tokens that act as anchors for the spatiotemporal structure.
• Traceable event tokens: compacted representations of the residual evidence, constructed via constrained feature condensation along explicit temporal trajectories.

The core methodological contribution comprises two novel modules: Contextual Farthest-Point Anchoring (CFPA) and Trajectory-Constrained Residual Condensation (TCRC).

Figure 1: Comparison of frame-wise token selection and residual condensation—GeoTrace (CFPA+TCRC) yields better structural preservation and traceable condensation than previous heuristics.

Methodological Framework

Temporal Chunking and Pipeline

GeoTrace operates on temporally coherent video chunks, enabling it to restrict estimation of spatiotemporal context and temporal correspondence within semantically consistent intervals. Temporal chunk boundaries are determined via abrupt drops in frame-wise feature similarity, as computed by cosine similarity between mean frame descriptors. Within each chunk, the visual token set is factorized as:

$$\mathbf{Z}_{\mathcal{C}} = \underbrace{\mathcal{S}_{\mathcal{C}}}_{\text{CFPA skeleton}} \cup \underbrace{\Phi(\mathcal{R}_{\mathcal{C}})}_{\text{TCRC event tokens}}$$

where $\mathcal{S}_{\mathcal{C}}$ are skeleton tokens and $\mathcal{R}_{\mathcal{C}}$ are the residuals.

Figure 2: The GeoTrace pipeline highlighting visual token extraction, temporal chunk partitioning, CFPA-based skeleton selection, TCRC-based residual condensation, and final token assembly.

Contextual Farthest-Point Anchoring (CFPA)

CFPA is dedicated to robustly identifying skeleton tokens that are not only locally salient but also contextually consistent and spatially diverse, maximizing feature-space coverage. The process unfolds as follows:

• Compute token-wise saliency via [CLS] attention or equivalent self-attention metrics.
• Assess each token’s contextual agreement with the chunk by inner-product with frame descriptors.
• Quantify candidate-weighted separation in feature space, favoring tokens that jointly maximize coverage, saliency, and context consistency.
• Employ a beam-style farthest-point sampling strategy to assemble a skeleton set, yielding exact, unmerged tokens with uncorrupted semantics.

Figure 3: CFPA typically selects skeleton tokens that jointly cover central entities and peripheral context, avoiding excessive redundancy and ensuring holistic structural coverage.

Trajectory-Constrained Residual Condensation (TCRC)

Residual tokens, after skeleton selection, contain fine-grained temporal information, often redundant across consecutive frames. TCRC achieves principled temporal condensation by:

• Constructing high-confidence one-to-one temporal correspondences between residual tokens in consecutive frames using maximum-similarity bipartite matching (Hungarian assignment), filtered by a similarity threshold to avoid ambiguity.
• Aggregating each temporal trajectory via feature averaging, restricting condensation to temporally plausible paths.
• Spatially coalescing surplus trajectory roots (if exceeding the budget) in each frame via entropy-regularized optimal transport (Sinkhorn barycenter), ensuring resultant event tokens are close to the empirical feature manifold and inherit traceability to their dominant source regions.

Figure 4: TCRC condenses temporally repeated residual tokens into traceable event tokens along explicit, one-to-one trajectories, maintaining provenance.

Failure cases—typically involving drastic object transitions or low-confidence matching—highlight the necessity for TCRC’s route filtering and its role as a safeguard against semantic drift.

Figure 5: Example TCRC failures due to challenging appearance changes or ambiguous correspondence, demonstrating the need for conservative, thresholded routing.

Empirical Evaluation

GeoTrace was benchmarked across four prominent Video LLMs (LLaVA-OneVision, LLaVA-Video, Qwen2.5-VL, and Qwen3-VL) and four diverse video understanding datasets (VideoMME, LongVideoBench, MVBench, EgoSchema). The experiments focused on both accuracy preservation at reduced token budgets and computational reductions measured in TFLOPs and wall-clock latency.

Key numerical claims:

• On LLaVA-OneVision, retaining only 10% of visual tokens, GeoTrace achieves a 12.99× TFLOPs reduction while sustaining 99.1% of the vanilla model’s accuracy.
• Across all retention ratios, GeoTrace consistently degrades more gracefully than strong baselines such as FastV, VisionZip, and PruneVid.
• Improvements generalize to diverse architectures, including models with dynamic-resolution processors (Qwen2.5-VL/Qwen3-VL), and to datasets requiring long-range spatiotemporal reasoning.

Results further show that GeoTrace’s token condensation allows Video LLMs to process longer video sequences within a fixed token budget, increasing temporal context and improving average benchmark scores.

Theoretical and Practical Implications

GeoTrace’s structured approach to token reduction—via explicit skeleton-event decomposition—addresses core challenges of token selection bias and semantic ambiguity in prior video compression pipelines. The training-free property ensures wide applicability without requiring model retraining or access to gradients. By enforcing geometry- and trajectory-aware constraints, the framework yields condensed token sets that maintain semantic fidelity, provenance, and support robust video understanding under compute- and memory-limited deployment.

GeoTrace’s modular design could be extended toward:

• Adaptive token budgeting: dynamically allocating the skeleton/event split per video or per chunk based on scene complexity and detected motion.
• Streaming or online inference: incremental trajectory building and token updating as new frames arrive, supporting real-time applications.
• Feature-space diagnostics: automatically regulating thresholds and coverage objectives based on measured manifold concentration or compression uncertainty.

On the theoretical side, the TCRC condensation procedure can be shown to preserve proximity to the underlying feature manifold under bounded temporal and spatial radii—providing an inductive bias for semantic stability not found in heuristic or agglomerative approaches.

Conclusion

GeoTrace introduces a geometry-aware, training-free compression paradigm tailored for Video LLMs, achieving substantial inference acceleration and competitive accuracy via explicit preservation of exact skeleton tokens and traceable event condensation. Empirical and theoretical analyses validate that its spatiotemporal factorization and trajectory-constrained routing outperform both image-based and prior video-based baselines in both performance and efficiency. Future investigations should encompass adaptive, streaming, and model-specific extensions, as well as broader empirical validation in long-term video reasoning and real-world deployment contexts.