Factual Anchors in Cinematic Captioning

• Factual Anchors are explicit visual cues that link observable spatial and temporal evidence to professional cinematographic concepts.
• They prevent hallucination by grounding statements in concrete cues like camera movement, shot size, and composition.
• Integrating spatio-temporal anchors with atomic reasoning improves descriptive completeness while maintaining factual exactness.

Searching arXiv for the specified paper and closely related work on caption grounding and anchor-based reasoning. CineCap is a framework for cinematographic video captioning, a task that aims to describe how a video is filmed using professional film-language concepts such as camera movement, shot size, depth of field, composition, and shooting angle. Unlike question-answering-based evaluation of cinematic understanding, the task requires a unified open-form description over multiple cinematographic dimensions, and it is challenging because the model must infer professional cinematographic concepts from subtle visual evidence while generating captions that are both comprehensive and accurate. CineCap addresses this by combining structured reasoning with spatio-temporal anchors and reinforcement learning with comprehensiveness, accuracy, and gated coverage rewards; it is evaluated on CineCap Bench, a benchmark of 472 manually annotated video-caption pairs (Mao et al., 23 Jun 2026).

1. Task definition and problem setting

Cinematographic captioning is defined as describing filming style rather than only scene content. The relevant dimensions in CineCap are camera movement, shot size, shooting angle, depth of field, composition, and subject orientation. Multiple statements per dimension are common, reflecting temporal evolution and compositionality, so the output space is intrinsically denser than conventional single-sentence video description (Mao et al., 23 Jun 2026).

This formulation distinguishes the task from cinematic question answering. A question-answering setting can isolate one attribute at a time, whereas cinematographic captioning requires a single caption that jointly covers multiple dimensions. The paper identifies two main difficulties: professional cinematographic concepts must be inferred from subtle visual evidence, and the resulting caption must balance descriptive completeness with factual correctness.

A common misconception is to treat cinematographic captioning as ordinary video captioning with added terminology. CineCap’s formulation suggests a stricter requirement: the model is expected to ground film-language concepts in observable evidence and, for dynamic attributes, to localize them temporally. A plausible implication is that the task is closer to structured multimodal reasoning than to unconstrained stylistic paraphrase.

2. Spatio-temporal anchors as factual grounding

CineCap defines anchors as explicit, observable cues in the video that ground professional cinematographic concepts and their temporal extent. It introduces two anchor types. Spatial anchors are directly observable visual evidence, such as motion of static background, subject scale, foreground-background sharpness, subject placement, or viewpoint geometry. Temporal anchors are time intervals that localize when a cinematographic attribute is exhibited (Mao et al., 23 Jun 2026).

For dynamic attributes, a temporal anchor is represented as

$\tau = [t_s, t_e],$

where $t_s$ and $t_e$ are start and end timestamps within the clip.

A spatial anchor is not enforced as a fixed mathematical tuple, but the paper characterizes it as a set of local visual cues linked to a target attribute. The cues differ by cinematographic dimension:

Dimension	Spatial anchor	Temporal role
Camera movement	Changes of static background references	$\tau$ localizes intervals where motion types occur
Shot size	Subject scale in frame	Usually omitted for static observation
Depth of field	Foreground vs. background sharpness	Usually omitted for static observation
Composition	Subject placement, balance, entry/exit from frame edges	Usually omitted for static observation
Shooting angle	Viewpoint geometry, horizon tilt, vertical tilt	Usually omitted for static observation
Subject orientation	Facing direction or body orientation relative to camera	Usually omitted for static observation

The anchor mechanism is explicitly intended to prevent hallucination. Spatial and temporal anchors tie each statement to visual evidence and, when needed, to a localized interval. This forces the model to “cite” observable cues such as background motion, subject scale, or sharpness contrast, reducing reliance on unconstrained film terminology. The paper therefore treats anchors not as auxiliary metadata, but as the factual interface between pixels and professional cinematographic language (Mao et al., 23 Jun 2026).

3. Atomic reasoning and supervision construction

CineCap organizes reasoning into compact atomic units to avoid long and error-prone chains. Its canonical schema is

$$[\text{temporal anchor}] \; [\text{spatial anchor}] \rightarrow \text{statement}.$$

If an attribute is directly observable and static, the temporal or spatial anchor may be omitted and the statement is supervised directly (Mao et al., 23 Jun 2026).

The paper provides concrete examples. For camera movement, a reasoning unit such as [00:00–00:02] [background seaweed drifts downward relative to frame] yields the statement “The camera tilts up briefly at the start.” For depth of field, [—] [sharp foreground subject; soft background foliage] yields “Shallow depth of field isolates the subject from the background.” For composition, [—] [subject held on left third; empty space right] yields “Left-weighted framing with negative space to the right.” These examples illustrate that the model is trained to produce professional statements from anchored evidence rather than from free-form chain-of-thought.

The data construction pipeline begins with shot segmentation using PySceneDetect to yield single-shot clips and ensure consistent cinematographic structure per clip. Clips are initially labeled by camera-motion category to balance the distribution across motion types, and dense captions are then annotated on the balanced subset. Starting from manually annotated video-caption pairs, CineCap constructs atomic CoT annotations conditioned on ground-truth captions by recovering visual evidence and temporal localization. The construction is reviewed by human annotators and a closed-source model, Gemini, to improve coherence and formatting consistency, then normalized into a unified <answer>…</answer> format (Mao et al., 23 Jun 2026).

The paper does not prescribe a fixed detector set. Instead, it defines anchor categories and evidence patterns: background reference motion for camera movement, subject scale for shot size, sharpness contrast for depth of field, placement and balance for composition, and viewpoint geometry for angle. This suggests that CineCap’s contribution lies less in hard-coded perceptual modules than in a supervision schema that regularizes how visual evidence is verbalized.

4. Model architecture and training objective

The backbone is Qwen3-VL-8B. Videos are sampled at 2 FPS, with 256 tokens per frame. Multimodal inputs are encoded and fused, and the LLM generates captions. In the model, anchors are represented as textual reasoning tokens inside atomic CoT supervision rather than as a separate symbolic memory structure (Mao et al., 23 Jun 2026).

Supervised fine-tuning uses 80K video–CoT–caption samples built with the atomic schema, for 2 epochs, batch size 128, and learning rate $2 \times 10^{-5}$. The normalized output format is <answer>…</answer>. This stage teaches the model to generate statements grounded in anchored reasoning.

Reinforcement learning is then applied with GRPO. The setup uses 8 rollouts, prompts batch size 32, learning rate $1 \times 10^{-5}$, and 1 epoch on 2K samples. An LLM-as-a-Judge decomposes both ground-truth and generated captions into atomic statements across six aspects: Camera Movement, Shot Size, Depth of Field, Camera Angle, Composition, and Subject Orientation (Mao et al., 23 Jun 2026).

For each aspect $d$, the judge computes atomic counts $n_d^{\mathrm{gt}}$, $n_d^{\mathrm{pred}}$, and $t_s$0, then aggregates them as

$t_s$1

The comprehensiveness and accuracy scores are

$t_s$2

Coverage is enforced at the atomic level rather than by caption length:

$t_s$3

The key design is a gated coverage reward that activates only when accuracy is sufficient:

$t_s$4

with $t_s$5.

The final mixed reward is

$t_s$6

with $t_s$7, $t_s$8, and $t_s$9 (Mao et al., 23 Jun 2026).

This reward design encodes the paper’s central optimization claim: accuracy alone tends to yield conservative captions, so coverage is rewarded only after correctness crosses a threshold. Another common misconception is that caption completeness is controlled by output length; CineCap instead measures coverage by matched atomic statements across the six cinematographic dimensions.

5. CineCap Bench and empirical performance

CineCap Bench contains 472 manually annotated video-caption pairs from YouTube and public film datasets, using single-shot clips obtained via PySceneDetect. Captions jointly cover the six cinematographic dimensions, and multiple statements per dimension are common. Quality control uses a semi-automatic pipeline with Gemini 3 Pro initial drafts, annotator revisions, normalization, and a two-stage review consisting of approximately 30% random review and approximately 10% expert review (Mao et al., 23 Jun 2026).

Evaluation is aspect-level and overall-level, using atomic statement decomposition with an LLM-as-a-Judge. The primary metrics are comprehensiveness, accuracy, and derived F1.

On overall performance, CineCap achieves Overall Cmp = 72.38, Acc = 74.80, and F1 = 73.57, surpassing Gemini-3.1-Pro at 46.31, 60.48, and 52.45. Dimension-wise improvements include Camera Movement at 54.38/60.50 versus 40.23/43.09 for Gemini-2.5-Pro, Depth of Field at 79.68/82.91 versus 48.11/47.37, and Composition at 66.17/66.09 versus 40.24/43.55. Angle and Orientation are also strong at 90.06/88.35 and 72.51/69.54 (Mao et al., 23 Jun 2026).

The ablations isolate the contribution of anchor-based supervision and RL. Starting from the base Qwen3-VL-8B at F1 41.16, Direct caption SFT reaches 69.27, CineCap SFT with atomic anchors reaches 70.21, and CineCap SFT + GRPO reaches 73.57. Reward ablations show that adding comprehensiveness and accuracy reward yields F1 72.93, adding a length penalty yields 71.71 and hurts accuracy, adding coverage reward without gating yields 73.11, and adding gated coverage reward yields the best balance at 73.57.

These results support two specific conclusions from the paper. First, anchor-grounded supervision improves over direct caption SFT. Second, the gated coverage term improves the balance between descriptive completeness and factual correctness better than either a length penalty or ungated coverage reward.

6. Illustrative caption grounding

The paper’s examples make the anchor-to-caption mapping explicit. In one example, the anchors $t_e$0 with downward drift of background seaweed, $t_e$1 with irregular edge entries and exits, plus subject scale and blur cues, produce the statements “The camera tilts up in the opening two seconds,” “It then shakes briefly,” “This is a close-up shot,” and “Shallow depth of field keeps the background soft” (Mao et al., 23 Jun 2026).

A second example uses composition and angle cues: subject centered-left, negative space to the right, horizon level, and eye-line below camera height. These yield “Left-third composition with empty space to the right” and “A modest high-angle viewpoint looking down on the subject.” A third example combines subject orientation and shot size: a subject facing camera three-quarter left and a torso occupying mid-height of frame yield “The subject faces slightly left in a three-quarter orientation” and “Medium close-up framing.”

These examples clarify an important property of CineCap. The generated caption is not a monolithic sentence produced from latent video features alone; it is learned as a composition of evidence-backed atomic statements. A plausible implication is that temporal segmentation for dynamic attributes and evidence-based phrasing for static attributes are not merely emergent behaviors at inference time, but direct consequences of the supervision format and reward structure.

7. Limitations and broader significance

The paper identifies several limitations. In highly dynamic or low-quality footage, static background references may be weak, making motion-type inference harder. Precise temporal segmentation can be difficult when motions overlap or are subtle, and misaligned $t_e$2 reduces atomic matching scores. Anchors learned on curated datasets may generalize less well to handheld or stylized footage with atypical composition or angle conventions. The gating threshold $t_e$3 and the reward weights influence the balance between accuracy and coverage, and future work could adaptively tune $t_e$4 per video. The paper also notes that a fully automated anchor detection pipeline, such as explicit background motion trackers or composition analyzers, could further strengthen factual grounding (Mao et al., 23 Jun 2026).

Within the paper’s own framing, CineCap is significant because it shifts cinematographic captioning from unconstrained description toward grounded multimodal reasoning. The core mechanism is concise: spatial anchors encode directly observable cues, temporal anchors localize dynamic attributes, atomic CoT supervision teaches the mapping from anchored evidence to film-language statements, and RL rewards captions that are both complete and correct. The resulting system establishes a new state of the art for cinematographic captioning on CineCap Bench (Mao et al., 23 Jun 2026).

More broadly, CineCap suggests a general design principle for multimodal generation: when the target vocabulary is specialized and the visual cues are subtle, explicit grounding structures can regularize both supervision and evaluation. In CineCap, that principle takes the specific form of spatio-temporal anchors and atomic reasoning units, but the underlying idea is the same throughout the framework: professional description should be recoverable from localized, observable evidence rather than from unconstrained terminology alone.