Hit-Centric Keyframe Selection

• Hit-centric keyframe selection is a design principle that allocates a limited frame budget to the most informational and objective-conditioned moments in a video.
• It employs varied criteria—query-aware, content-aware, and geometry-centric—to optimize tasks like long-video understanding, editing, summarization, and robotic SLAM.
• Recent studies show that this approach reduces redundancy and temporal collapse, leading to improved efficiency and accuracy in keyframe-based systems.

Hit-centric keyframe selection denotes a family of sparse video selection methods that prioritize frames judged to be the most consequential “hits” for a downstream objective, rather than relying on uniform temporal sampling. In recent work, the operative notion of a “hit” varies by task: it may mean a query-relevant frame for long-video question answering, a representative frame at a content transition for long-video editing, a class-discriminative frame for action recognition, a perceptual outlier for summarization, or an information-preserving frame for visual odometry and SLAM (Fang et al., 30 May 2025, Zhang et al., 8 Feb 2025, Yan et al., 2018, Cakmak et al., 23 Jun 2025, Dai et al., 22 Jan 2026, Thorne et al., 2024). This variation is central to the topic: hit-centric selection is not a single algorithm, but a design principle in which the frame budget is allocated to evidence-bearing moments rather than to time itself.

1. Conceptual scope and terminology

In the literature considered here, hit-centricity has at least three technically distinct meanings. In long-video understanding, it is usually query-aware: a frame is valuable because it aligns with a question, contributes diverse evidence, or satisfies a semantic-logical dependency. In long-video editing and summarization, it is often content-aware or perceptually aware: a frame is valuable because it represents a stable segment, a rapid transition, or a standout perceptual change. In robotics and SLAM, it becomes coverage- or constraint-centric: a frame is valuable because it improves pose estimation, scan matching, or submap conditioning (Fang et al., 30 May 2025, Zhang et al., 8 Feb 2025, Cakmak et al., 23 Jun 2025, Dai et al., 22 Jan 2026, Thorne et al., 2024).

Paradigm	Hit criterion	Representative papers
Query-aware	Query relevance, diversity, logical consistency	(Fang et al., 30 May 2025, Zhu et al., 31 Oct 2025, Guo et al., 17 Mar 2025, Kwon et al., 16 Mar 2026)
Content-aware	Content change, representative segments, perceptual outliers, cluster centrality	(Zhang et al., 8 Feb 2025, Song et al., 3 Feb 2026, Cakmak et al., 23 Jun 2025, Jha et al., 27 Oct 2025)
Task-/geometry-centric	Action discrimination, pose utility, localization constraint	(Yan et al., 2018, Dai et al., 22 Jan 2026, Thorne et al., 2024)

A common conflation is to treat hit-centric keyframe selection as synonymous with generic saliency detection. The papers do not support that reduction. Several methods explicitly reject generic visual saliency in favor of objective-specific criteria: Nar-KFC optimizes query relevance and frame diversity (Fang et al., 30 May 2025); AdaFlow partitions videos into content-homogeneous segments using DIFT similarity (Zhang et al., 8 Feb 2025); the human-action method defines keyframes as frames that “maximally contribute to distinguishing the underlying action category from all other categories” (Yan et al., 2018); and the SLAM work formulates keyframe choice as submodular set selection under novelty and localization constraints (Thorne et al., 2024). This suggests that hit-centricity is best understood as objective-conditioned informativeness.

2. Query-aware selection for long-video understanding

The most explicit query-hit formulation appears in long-video MLLM pipelines. Nar-KFC defines frame embeddings $\mathbf{f}_i$ and a query embedding $\mathbf{q}$, scores query relevance by cosine similarity, defines frame diversity as $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$, and combines them into a pairwise score used in an integer quadratic programming problem for selecting $K$ keyframes. Because exact IQP is NP-hard/exponential, the paper introduces a customized greedy search with low-rank denoising via SVD, uniform downsampling of the score matrix, greedy addition, and local refinement, with overall complexity $O(NK)$ (Fang et al., 30 May 2025). The method is explicitly motivated by the failure modes of uniform sampling and top-$K$ similarity ranking, especially “temporal collapse,” in which adjacent near-duplicate frames dominate the selected set.

FOCUS reformulates query-aware keyframe selection as a combinatorial pure-exploration problem in multi-armed bandits. It partitions the video into fixed-length temporal clips treated as arms, estimates clip utility through sampled frame-query relevance, and uses empirical means together with a Bernstein confidence radius

$$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$

to preserve exploration of uncertain regions while exploiting promising ones (Zhu et al., 31 Oct 2025). Its practical procedure is two-stage: coarse exploration selects the top $\alpha m$ arms under optimistic scores $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$, then fine exploration recomputes empirical means and returns the final top-$m$ clip set. The paper presents this as a response to retrieval-style methods that require aggressive pre-filtering, such as 1 fps downsampling, before scoring.

Logic-in-Frames, or Visual Semantic-Logical Search, moves from frame ranking to iterative search. It defines a minimal answer-preserving subset $\mathbf{q}$0 satisfying the conservation condition $\mathbf{q}$1, then updates frame scores through four logical dependencies: spatial co-occurrence, temporal proximity, attribute dependency, and causal order (Guo et al., 17 Mar 2025). Sampled frames are scored by object evidence, relation-consistent frames receive additive boosts, scores are diffused locally in time by

$\mathbf{q}$2

and the sampling distribution is iteratively normalized and refined. The method is designed for “finding a needle in a haystack,” and its hit-centricity lies in targeting frames that satisfy relational constraints rather than merely lexical similarity.

A learnable variant of query-aware selection appears in VideoQA. The question-aware keyframe selection framework with Synthetic Keyframe Supervision uses Qwen2.5-VL to generate pseudo keyframe labels, trains a Gaussian Generator to output $\mathbf{q}$3 Gaussian masks, and adds Question-conditioned Coverage Regularization to discourage redundant selections (Kwon et al., 16 Mar 2026). Its stated motivation is that image-text similarity alone ignores temporal and causal relations and tends to produce adjacent duplicate frames. The framework therefore treats keyframe selection as a learnable evidence-selection module rather than a fixed retrieval heuristic.

3. Content-aware and representative selection without explicit query conditioning

In long-video editing, hit-centricity is typically defined relative to content dynamics rather than a textual question. AdaFlow introduces Adaptive Keyframe Selection as a training-free heuristic built on diffusion features. It extracts DIFT features for every frame,

$\mathbf{q}$4

constructs DIFT-based token similarity heatmaps $\mathbf{q}$5, and performs Adaptive Video Partitioning by traversing frames sequentially until either $\mathbf{q}$6 or window_check(H_{i,j}, l, s, ws) fails (Zhang et al., 8 Feb 2025). Each resulting clip is treated as content-homogeneous, and one frame is sampled as a representative keyframe. The paper’s criticism of uniform keyframe sampling is specific: if the interval is too large, quality drops; if it is too small, computation explodes.

KTV also rejects question-driven frame ranking at the keyframe stage, but for a different reason. It argues that CLIP-similarity keyframe selection can be biased, can over-focus on obvious keywords, and can fall into “semantic traps.” Its first stage is question-agnostic: DINOv2 features are extracted for all frames, K-means partitions them into $\mathbf{q}$7 clusters, and the frame closest to each centroid is selected and then reordered temporally (Song et al., 3 Feb 2026). Only after representative frame selection does the method reintroduce question relevance, using CLIP similarity to allocate token budgets across already-selected keyframes. The paper’s ablation directly contrasts cluster keyframes with question-relevant keyframes and reports better results for the cluster-based choice.

PRISM defines standout moments as perceptual outliers. It converts frames to CIELAB, computes consecutive-frame perceptual differences using CIEDE2000 on average Lab values, filters changes below the Just Noticeable Difference threshold $\mathbf{q}$8, and selects transitions satisfying

$\mathbf{q}$9

after computing the video-specific mean $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$0 and standard deviation $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$1 of perceptually significant changes (Cakmak et al., 23 Jun 2025). The method is explicitly training-free, interpretable, and lightweight. Its notion of a “hit” is therefore not semantic or query-aware; it is a statistically large, human-noticeable perceptual change.

The adaptive selector for 3D scene reconstruction in dynamic environments is similarly event-driven. It computes a hybrid score

$\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$2

with $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$3 and $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$4, sets a dynamic threshold

$\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$5

and selects a frame when $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$6; after selection, the threshold decays according to $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$7 with $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$8 (Jha et al., 27 Oct 2025). Here the “hit” is a frame whose visual change relative to the most recent keyframe is sufficiently large to justify inclusion.

4. Learned discriminative selection

A distinct line of work treats hit-centric keyframe selection as a supervised or reinforcement-learned prediction problem. In human action videos, the two-stream ConvNet approach defines a keyframe as a frame that optimally contributes to differentiating the underlying action category from all other categories (Yan et al., 2018). Its summarizer uses VGG-16-based appearance and motion streams over RGB and optical flow, concatenates their fc6 outputs into an 8192-dimensional feature, and passes the fused representation through fully connected layers to a regression head. Since UCF101 lacks keyframe ground truth, the paper generates frame-level labels automatically by supervised LDA, computing $\exp(-\mathrm{sim}(\mathbf{f}_i,\mathbf{f}_j))$9 and the frame label score

$K$0

At test time, the predicted frame scores are smoothed with a smoothing spline, and local maxima are selected as keyframes. In this formulation, a hit is explicitly class-discriminative rather than merely visually distinctive.

The question-aware VideoQA selector extends learnability to question-conditioned evidence extraction. It trains against pseudo labels from Qwen2.5-VL with an MSE loss over Gaussian masks and a VQA loss for downstream answer prediction (Kwon et al., 16 Mar 2026). The paper states that standard VideoQA supervision provides only question-answer pairs, not frame-level labels, and positions synthetic supervision as weak temporal grounding. A plausible implication is that hit-centric keyframes here are not predefined by low-level similarity but by the LMM’s inferred notion of evidence.

Keyframe-based feed-forward visual odometry replaces hand-crafted geometric rules with reinforcement learning. The method defines an MDP $K$1, observes mean-pooled DINOv2 CLS tokens from the current sliding window together with relative pose changes, and makes a binary keep/discard decision for the newest frame (Dai et al., 22 Jan 2026). The reward combines translational pose error with a small keyframe compensation term:

$K$2

with $K$3, $K$4, $K$5, and $K$6. The paper emphasizes that the compensation term is crucial to avoid action collapse. This is a learned coverage policy: frames are retained when they improve the utility of the VGGT window rather than when they satisfy a fixed motion threshold.

5. Optimization, coverage, and geometry-centric formulations

In SLAM, hit-centricity is most naturally expressed as combinatorial optimization over novelty and conditioning. The submodular SLAM framework represents all scans as $K$7, selected keyframes as $K$8, and optimizes a set function $K$9 under cardinality constraints (Thorne et al., 2024). For online keyframe selection, each point cloud is mapped to a descriptor by an OverlapTransformer-based network, and novelty is measured by distance to the nearest previously selected keyframe in descriptor space. A scan is saved if its minimum descriptor distance exceeds the threshold $O(NK)$0, or if a degeneracy condition involving the minimum Hessian eigenvalue satisfies

$O(NK)$1

The same paper formulates submap generation as maximizing $O(NK)$2 for a subset $O(NK)$3, uses greedy selection, and prunes candidates with Weyl bounds.

This geometry-centric perspective differs sharply from query-aware MLLM methods. A LiDAR scan is not selected because it answers a linguistic question; it is selected because it is sufficiently novel in descriptor space or because it improves conditioning of scan matching (Thorne et al., 2024). Likewise, the visual-odometry RL policy does not optimize explicit geometric heuristics such as optical-flow thresholds, but it still learns to retain frames that improve downstream localization error (Dai et al., 22 Jan 2026). The dynamic-reconstruction selector similarly “fires” on informative frames using photometric and SSIM errors rather than question relevance (Jha et al., 27 Oct 2025). Across these papers, hit-centricity is therefore equivalent to usefulness for estimation or reconstruction.

This distinction is important because it prevents a category error. Methods such as Nar-KFC, FOCUS, and VSLS are evidence-retrieval systems for semantic reasoning (Fang et al., 30 May 2025, Zhu et al., 31 Oct 2025, Guo et al., 17 Mar 2025), whereas the SLAM and VO methods are information-preserving front ends for estimation (Thorne et al., 2024, Dai et al., 22 Jan 2026). They share sparsification, adaptivity, and redundancy reduction, but not the same notion of relevance.

6. Empirical patterns, evaluation, and recurrent misconceptions

Evaluation protocols vary substantially by domain. In long-video QA, Nar-KFC reports Video-MME overall $O(NK)$4 and MLVU $O(NK)$5 for KFC (GS), compared with Video-MME overall $O(NK)$6 and MLVU $O(NK)$7 for uniform sampling, while the IQP and greedy solutions are nearly identical at Video-MME overall $O(NK)$8 versus $O(NK)$9 and MLVU $K$0 versus $K$1 (Fang et al., 30 May 2025). FOCUS reports, under matched keyframe count $K$2 with LLaVA-Video-7B, LongVideoBench overall $K$3 versus $K$4 for Top-K, $K$5 for AKS, and $K$6 for uniform, and Video-MME overall $K$7 versus $K$8, $K$9, and $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$0, respectively; for videos longer than 20 minutes it reports an $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$1 gain over uniform on LongVideoBench while seeing only $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$2 of frames in one operating point (Zhu et al., 31 Oct 2025). The question-aware selector with Synthetic Keyframe Supervision reports NExT-QA average $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$3, compared with $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$4 without QCCR and $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$5 without SKS (Kwon et al., 16 Mar 2026). These results consistently support the claim that uniform sampling wastes frame budget on irrelevant or redundant periods.

In editing and summarization, the evidence is framed differently. AdaFlow reports high-quality long video editing of more than $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$6 frames in one inference on one A800 GPU, about ten times longer than compared methods such as TokenFlow, and attributes scalability partly to Adaptive Keyframe Selection combined with Adaptive Attention Slimming (Zhang et al., 8 Feb 2025). PRISM reports average performance across BBC, TVSum, SumMe, and ClipShots of $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$7 accuracy, $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$8 fidelity, and $$\beta_a(n)=\sqrt{\frac{2\,\hat{\sigma}_a^2\ln n}{\max(1,N_a(n))}+\frac{3\ln n}{\max(1,N_a(n))}}$$9 compression ratio, with speeds of 167 FPS on TVSum, 131 FPS on SumMe, 454 FPS on ClipShots, and 185 FPS on BBC (Cakmak et al., 23 Jun 2025). These results show that hit-centric selection can prioritize compression and speed even when it is not semantically aware.

In recognition and robotics, the metrics shift again. The human-action detector reports average keyframe number error about $\alpha m$0 and average location error about $\alpha m$1 frames on 20 UCF101 classes (Yan et al., 2018). The feed-forward VO method evaluates with aligned trajectory ATE RMSE and reports average ATE of $\alpha m$2 on KITTI, $\alpha m$3 on TUM-RGBD, and $\alpha m$4 on EuRoC, improving over VGGT-SW and VGGT-LK baselines (Dai et al., 22 Jan 2026). The SLAM submodular framework reports 80% fewer keyframes and 64% less memory than DLIOM with comparable end-to-end accuracy, and for submap generation reduces average submap size from 10.4 to 2.3 while improving the minimum Hessian eigenvalue from 377 to 407 and reducing per-scan computation time from 67.3 ms to 45.4 ms (Thorne et al., 2024). The adaptive 3D reconstruction selector reports, on Spann3r over Bonn, Acc $\alpha m$5, Comp $\alpha m$6, CD $\alpha m$7, NC $\alpha m$8, and KFCR $\alpha m$9, improving over the baseline values Acc $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$0, Comp $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$1, CD $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$2, NC $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$3, and KFCR $\tilde{\mu}_a=\hat{\mu}_a+\beta_a$4 (Jha et al., 27 Oct 2025).

Several misconceptions are repeatedly contradicted by the literature. First, hit-centric selection is not necessarily learned: AdaFlow and PRISM are explicitly training-free, while Nar-KFC uses a principled optimization with greedy approximation, and FOCUS is training-free and model-agnostic (Zhang et al., 8 Feb 2025, Cakmak et al., 23 Jun 2025, Fang et al., 30 May 2025, Zhu et al., 31 Oct 2025). Second, it is not necessarily query-aware: KTV is question-agnostic at the keyframe stage, and the SLAM, VO, and reconstruction methods optimize geometric or perceptual utility rather than language alignment (Song et al., 3 Feb 2026, Thorne et al., 2024, Dai et al., 22 Jan 2026, Jha et al., 27 Oct 2025). Third, it is not identical to top-$\tilde{\mu}_a=\hat{\mu}_a+\beta_a$5 similarity ranking: Nar-KFC introduces explicit diversity to avoid temporal collapse, VSLS enforces logical dependencies, and the VideoQA selector adds question-conditioned coverage regularization to prevent adjacent duplicate frames (Fang et al., 30 May 2025, Guo et al., 17 Mar 2025, Kwon et al., 16 Mar 2026). Taken together, these papers suggest that the defining property of hit-centric keyframe selection is not a particular architecture, but the replacement of uniform time-based sampling with an explicit model of which frames are worth the budget.