Visual Chronometer for True Video FPS

• The paper introduces Visual Chronometer as a predictor that estimates true physical frames per second (PhyFPS) directly from motion cues rather than relying on metadata.
• It employs a VideoVAE+ encoder and log-space regression with physics-grounded augmentations like motion blur and synthetic rolling shutter to stabilize and improve predictions.
• The framework establishes dedicated benchmarks and retiming pipelines that align playback with motion-implied time scales, mitigating temporal inconsistencies in generated videos.

Visual Chronometer is a predictor for recovering the Physical Frames Per Second (PhyFPS) of a video directly from its visual dynamics rather than from file metadata. It was introduced to address what the authors call chronometric hallucination: the production of visually smooth but temporally ambiguous, unstable, and physically mis-scaled motion in contemporary generative video models. In this formulation, PhyFPS is the intrinsic temporal scale implied by motion itself, whereas nominal or container frame rate is only a playback or encoding convention. Visual Chronometer therefore treats motion as evidence for real-world time, establishes dedicated benchmarks for PhyFPS estimation and generator auditing, and uses PhyFPS-guided retiming to improve the perceived naturalness of generated videos (Gao et al., 15 Mar 2026).

1. Conceptual basis and terminology

PhyFPS is defined as the true frame rate that aligns a video’s visual motion with real-world time, independent of the nominal container or encoding rate. The paper distinguishes four quantities: the camera acquisition rate $F_{\mathrm{capture}}$, the file or container rate $F_{\mathrm{meta}}$, the playback rate $F_{\mathrm{playback}}$, and the physical frame rate $f_{\mathrm{phy}}$ implied by the motion dynamics. Because slow-motion, time-lapse, and post-hoc re-encoding can make $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$, a video can be displayed at a standard rate while still depicting motion at an implausibly slow or fast physical pace (Gao et al., 15 Mar 2026).

The central pathology is chronometric hallucination. In the paper’s usage, this is the failure mode in which video generators produce motions that look smooth but have ambiguous, unstable, and uncontrollable physical speeds. The stated cause is the indiscriminate training of generative systems on videos with widely different real-world temporal scales after those videos have been standardized to common nominal rates such as 24, 25, or 30 FPS. A hummingbird rendered in extreme slow motion or a fall proceeding far below gravitational acceleration, despite no prompt requesting speed manipulation, are given as examples of this temporal ambiguity (Gao et al., 15 Mar 2026).

This definition places Visual Chronometer within a broader PhyFPS vocabulary. In ultrafast imaging and physical sensing, PhyFPS is treated as the temporal sampling rate of the actual event, determined by acquisition-time resolution rather than by display conventions; examples include light-in-flight imaging, where $\mathrm{PhyFPS} = 1/\Delta t$, and retinal Doppler holography, where a sensor streams interferograms at a true physical acquisition rate of 33,000 fps (Faccio et al., 2018, Fischer et al., 2024). Visual Chronometer transfers that distinction to ordinary and generated videos by inferring the physical time scale from motion cues rather than reading it from metadata. A plausible implication is that it reframes temporal realism as an estimation problem analogous to recovering latent scene geometry or motion, but with the target variable being time scale itself (Gao et al., 15 Mar 2026).

2. Formalization and model architecture

The task is formalized as absolute regression of the physical frame rate from an input video $V$:

$\hat{f}_{\mathrm{phy}}(V) \in \mathbb{R}_{+}.$

Visual Chronometer uses a VideoVAE+ encoder as its backbone. Given a clip $\mathbf{V} = \{I_t\}_{t=1}^{T}$, the encoder produces latent tokens $\mathbf{Z}=\mathrm{Enc}(\mathbf{V})$. A learnable query embedding then cross-attends to $F_{\mathrm{meta}}$0 to aggregate temporal features into a clip-level representation, and an MLP maps that representation to a scalar predicting the logarithm of PhyFPS:

$F_{\mathrm{meta}}$1

The use of log-space is explicitly motivated as a way to stabilize optimization across wide temporal scales and to emphasize proportional rather than absolute error (Gao et al., 15 Mar 2026).

The model operates on raw frames and does not rely on semantic labels. Inference is performed on sliding windows, enabling both per-clip estimation and temporal stability analysis. The main experimental configuration uses clips of length $F_{\mathrm{meta}}$2 frames, and later evaluations vary the window over $F_{\mathrm{meta}}$3 (Gao et al., 15 Mar 2026).

The paper defines both video-level aggregation and stability diagnostics. For per-clip predictions $F_{\mathrm{meta}}$4, the per-video average is

$F_{\mathrm{meta}}$5

and the overall mean across videos is

$F_{\mathrm{meta}}$6

Temporal stability is quantified through Inter CV and Intra CV, both based on coefficients of variation, and the paper also notes a simple variance-based stability quantity

$F_{\mathrm{meta}}$7

This emphasis on intra-video and inter-video stability is important because the problem is not only whether a model predicts the correct average PhyFPS, but also whether it maintains a coherent internal “motion pulse” over time (Gao et al., 15 Mar 2026).

3. Training protocol and physics-grounded supervision

Visual Chronometer is trained by controlled temporal resampling rather than by trusting internet metadata. Source videos are curated so that the meta FPS equals the true physical rate for the base videos. These high-frequency base videos are then upsampled to $F_{\mathrm{meta}}$8 FPS using RIFE, after which labeled training examples are synthesized by downsampling to a target physical rate $F_{\mathrm{meta}}$9 with ratio

$F_{\mathrm{playback}}$0

This construction makes the ground-truth PhyFPS known by design, with target label $F_{\mathrm{playback}}$1 (Gao et al., 15 Mar 2026).

Three augmentation regimes encode camera physics. In sharp capture, the low-rate frame is sampled directly:

$F_{\mathrm{playback}}$2

In motion blur, exposure integration averages consecutive high-rate frames:

$F_{\mathrm{playback}}$3

In synthetic rolling shutter, a pixel column $F_{\mathrm{playback}}$4 is sampled from progressively shifted source frames:

$F_{\mathrm{playback}}$5

These augmentations are intended to make the predictor robust to fast shutter, exposure integration, and rolling-shutter readout artifacts (Gao et al., 15 Mar 2026).

The target is regressed in log-space. With $F_{\mathrm{playback}}$6 and predicted $F_{\mathrm{playback}}$7, the loss is

$F_{\mathrm{playback}}$8

The training corpus contains 465,535 clips, all standardized to 128 frames during preparation. Two target-rate sets are defined. VC-Wide uses $F_{\mathrm{playback}}$9, whereas VC-Common focuses on consumer and web rates $f_{\mathrm{phy}}$0. Optimization uses Adam, learning rate $f_{\mathrm{phy}}$1, 125,000 iterations, global batch size 32, and 4× NVIDIA RTX A6000 GPUs, with the VideoVAE+ backbone fine-tuned end-to-end together with the attention head (Gao et al., 15 Mar 2026).

Ablations show that the physics-grounded augmentations materially improve prediction quality. Naive training without motion blur or rolling shutter yields MAE 5.12 and MAPE 13%. Adding motion blur improves this to MAE 4.87 and MAPE 11%. Adding both motion blur and rolling shutter, which defines VC-Common, further improves performance to MAE 3.46 and MAPE 9% (Gao et al., 15 Mar 2026).

4. Benchmarks, metrics, and empirical performance

The paper establishes two benchmarks. PhyFPS-Bench-Real measures prediction accuracy against reliable ground-truth PhyFPS and contains 4,000 verified clips with a cross-source train/val/test split. PhyFPS-Bench-Gen audits generative video models for meta-versus-PhyFPS alignment, intra-video stability, and inter-video stability. The latter uses 100 text-to-video prompts with no speed-manipulation keywords and evaluates a mixture of open-source and closed-source systems, including Wan, LTX-Video, CogVideoX, HunyuanVideo, InfinityStar, Veo-3.1-Fast, Sora-2, Grok-Imagine-T2V, Kling-o3, and Seedance variants (Gao et al., 15 Mar 2026).

Prediction accuracy on PhyFPS-Bench-Real and the main audit conclusions on PhyFPS-Bench-Gen can be summarized as follows:

Setting	Model/result	Reported values
Real benchmark	VC-Common	Avg Pred 39.20; MAE 3.46; MAPE 9%
Real benchmark	VC-Wide	Avg Pred 45.48; MAE 7.76; MAPE 21%
Real benchmark	Gemini-3.1-Pro (video)	Avg Pred 31.00; MAE 21.67; MAPE 43%
Real benchmark	Qwen3.5+ (video)	Avg Pred 4.46; MAE 45.54; MAPE 91%
Generator audit	General trend	Predicted PhyFPS often higher than meta FPS
Human preference after correction	Bradley–Terry scores	Original 19.0%; Pred 44.2%; Pred Dyn 36.9%

On PhyFPS-Bench-Real, the average ground-truth PhyFPS is 38.81. VC-Common predicts 39.20, with MAE 3.46 and MAPE 9%, whereas VC-Wide predicts 45.48, with MAE 7.76 and MAPE 21%. General-purpose video-capable LLMs perform substantially worse: Gemini-3.1-Pro (video) yields Avg Pred 31.00, MAE 21.67, MAPE 43%; Seed-1.6-Flash collapses to 30 FPS for all inputs; Qwen3.5+ (video) yields Avg Pred 4.46, MAE 45.54, MAPE 91%. The paper interprets this as evidence that PhyFPS estimation is not well served by generic multimodal reasoning alone and requires a dedicated chronometric predictor (Gao et al., 15 Mar 2026).

The generator audit identifies a systematic bias: predicted PhyFPS is often significantly higher than meta FPS, which the paper characterizes as evidence that generators produce “slow but smooth” videos that should be played faster to match their implied motion. Among open-source systems, LTX-Video at meta 24 has PhyFPS 46.52, Avg Error 23.67, Pct Error 99%, Intra CV 0.10, and Inter CV 0.33, suggesting strong internal consistency but incorrect meta-FPS assignment. Wan2.2-A14B at meta 24 has PhyFPS 31.52, Avg Error 10.74, Pct Error 45%. Among closed-source systems, Sora-2 at meta 30 has PhyFPS 36.21, Avg Error 8.40, Pct Error 28%, Intra CV 0.13, Inter CV 0.29; Seedance-1.5-Pro at meta 24 has PhyFPS 33.69, Avg Error 10.67, Pct Error 44%, and Inter CV 0.25, the best closed-source inter-video stability reported. The paper concludes that closed-source models slightly outperform in alignment, with Avg Error < 14 FPS and Pct Error < 60%, but that both open-source and closed-source systems show notable instability (Gao et al., 15 Mar 2026).

Window-length ablations indicate that the base model trained with maximum 32 frames performs best at $f_{\mathrm{phy}}$2, with accuracy remaining competitive at $f_{\mathrm{phy}}$3 and $f_{\mathrm{phy}}$4. The paper recommends $f_{\mathrm{phy}}$5–$f_{\mathrm{phy}}$6 as a practical inference range. Very large windows lose the benefits of sliding-window ensembling and fine-grained local estimation (Gao et al., 15 Mar 2026).

5. PhyFPS correction and retiming pipeline

Visual Chronometer is not limited to auditing; it is also used for PhyFPS-guided correction of generated videos. Given a target rate $f_{\mathrm{phy}}$7, often chosen as the predicted per-video mean $f_{\mathrm{phy}}$8, the global retiming factor is

$f_{\mathrm{phy}}$9

For local dynamic correction, the paper defines a per-clip factor

$F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$0

These factors can be implemented by time-stretching or re-encoding so that effective playback aligns with the chosen target physical speed (Gao et al., 15 Mar 2026).

The operational pipeline is explicit. A generated video with nominal $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$1 is sliced into overlapping clips, for example with $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$2 and stride $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$3. VC-Common predicts $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$4 for each clip, from which $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$5 is computed. Correction may then be global, with $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$6, or dynamic, with a fixed desired physical rate and clipwise retiming factors. Resampling can be implemented by frame dropping, duplication, or motion interpolation such as RIFE when non-integer factors are needed. The paper also notes that the corrected video can either be re-encoded with updated meta FPS or left at the same meta FPS with duration adjusted by resampling (Gao et al., 15 Mar 2026).

Human evaluation is a central part of the argument. The user study reports 1,490 pairwise comparisons, involving 15+ participants, with preferences analyzed using Bradley–Terry modeling and 90% confidence intervals. The uncorrected original videos receive a score of 19.0%. Global correction (Pred) receives 44.2%, and dynamic local correction (Pred Dyn) receives 36.9%. Both corrected variants are significantly preferred over the originals, while global correction is preferred over dynamic correction. The paper attributes that difference to the perceptual inconsistency introduced when playback rate changes within a single video (Gao et al., 15 Mar 2026).

This correction framework yields a concrete interpretation of PhyFPS misalignment. If a model systematically produces videos whose predicted PhyFPS exceeds their meta FPS, the content is not merely “stylized”; it is temporally miscalibrated relative to its own motion evidence. A plausible implication is that PhyFPS correction functions as a post hoc temporal calibration layer, analogous to color correction or motion stabilization, but targeted at the physical plausibility of speed itself (Gao et al., 15 Mar 2026).

6. Broader context, related methods, and limitations

Visual Chronometer is positioned against three nearby but distinct approaches. Metadata-based heuristics assume that $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$7 reflects physical time, which the paper treats as unreliable in mixed internet-scale datasets. Optical-flow speed estimation can measure displacement per frame but does not recover the global time base and is sensitive to blur and rolling shutter. Prior time-perception models, such as faster/slower or forward/backward classifiers, address relative or binary temporal judgments rather than absolute regression of a physically meaningful rate. The paper instead compares its role to audio beat tracking, arguing that Visual Chronometer infers a motion tempo from visual dynamics (Gao et al., 15 Mar 2026).

Its broader significance becomes clearer when placed alongside research that distinguishes physical temporal sampling from algorithmic or display rates. In real-time surgical video segmentation, the literature separates the physical capture/display rate of a 25-FPS stream from the model’s lower effective processing FPS, and shows that conclusions can change depending on whether evaluation is done on sampled frames or on full physical streams (Ozbulak et al., 28 Feb 2025). In ultrafast imaging, the same distinction appears between true physical acquisition rates and reconstructed or display rates: light-in-flight systems define PhyFPS via temporal resolution $F_{\mathrm{capture}} \neq F_{\mathrm{meta}}$8, while rolling-shutter compressive imaging and structured-light systems explicitly separate hardware-limited physical sampling from post hoc reconstruction or adaptive processing (Faccio et al., 2018, Weinberg et al., 2020, Sirikonda et al., 2024). Visual Chronometer extends this general principle from sensors to learned video distributions: what matters is not merely the nominal sequence rate, but the physically meaningful cadence actually implied by the observed motion (Gao et al., 15 Mar 2026).

The paper also identifies several limitations. Static scenes or very low motion make PhyFPS inherently ambiguous; the proposed mitigation is content-aware gating or the use of longer temporal windows. Abrupt cuts or edits can destabilize sliding-window predictions; smoothing or segmentation at edits is recommended. Highly nonrigid or chaotic motion can cause prediction jitter, motivating temporal smoothing and window ensembling. Severe rolling shutter or exposure anomalies may still bias estimates despite the augmentations. More broadly, the model currently functions as a post hoc estimator and retimer rather than as an intrinsic component of the generator itself (Gao et al., 15 Mar 2026).

The future directions identified in the paper follow directly from these limits. One is to condition generative models on PhyFPS so that physical time becomes an explicit control variable rather than an accidental byproduct of training data. Another is to build time-base priors for different motion classes such as humans, animals, vehicles, and fluids. A third is to combine PhyFPS control with broader spatial–temporal realism, including the use of PhyFPS as a reward signal in preference optimization. This suggests a shift from treating frame rate as a superficial rendering parameter to treating it as part of the latent physical specification of a video world model (Gao et al., 15 Mar 2026).