---
title: 'PhyFPS-Bench-Real: Real-Video FPS Benchmark'
url: https://www.emergentmind.com/topics/phyfps-bench-real
type: topic
---

# PhyFPS-Bench-Real: Real-Video FPS Benchmark

Searching arXiv for the benchmark paper and closely related work to ground the article.
Searching arXiv for “Visual Chronometer Physical Frames Per Second PhyFPS-Bench-Real”.
PhyFPS-Bench-Real is the real-video benchmark introduced in "The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics" for evaluating whether a model can recover the **physical time scale** of motion directly from appearance and dynamics rather than from container metadata [2603.14375]. In that framework, the benchmark serves as the paper’s **measurement-validity** testbed: it contains **real-world video clips with verified Physical FPS labels**, and it is used to establish that **Visual Chronometer** can estimate the **Physical Frames Per Second (PhyFPS)** implied by motion itself. The benchmark is therefore distinct from a generator audit benchmark; its purpose is to validate the estimator on temporally trustworthy real videos before that estimator is used to diagnose **chronometric hallucination** in generated videos [2603.14375].

## 1. Conceptual definition and benchmark role

The benchmark is motivated by a distinction between **PhyFPS** and **meta FPS**. In the paper, **Physical Frames Per Second (PhyFPS)** is defined as the frame rate corresponding to the **true physical passage of time implied by the visual motion itself**, whereas **meta FPS** is the nominal playback or file-level frame rate stored in metadata. The benchmark exists because the paper argues that meta FPS can be wrong, inherited from post-processing, or otherwise uninformative about the physically sampled time scale of the scene [2603.14375].

Within that framing, PhyFPS-Bench-Real is the benchmark for **single-video PhyFPS estimation from raw frames** on real clips with trustworthy labels. The paper contrasts it explicitly with **PhyFPS-Bench-Gen**. The former uses real videos with verified labels to evaluate prediction accuracy, while the latter uses generated videos without ground-truth physical labels to audit alignment and stability through the estimator’s predictions. The operational distinction is central: **PhyFPS-Bench-Real validates the instrument; PhyFPS-Bench-Gen uses the instrument** [2603.14375].

The broader significance of this distinction is methodological. The paper’s claim that contemporary video generators suffer from **chronometric hallucination** depends on having an estimator that is accurate on real videos whose physical timing is known. PhyFPS-Bench-Real is the empirical basis for that claim.

## 2. Dataset construction and temporal trustworthiness

PhyFPS-Bench-Real is drawn from a curated corpus built under a strict criterion: only sources where the nominal capture rate is trusted to equal the physical sampling rate are used as the base material. The paper states that the data are curated **“exclusively from video sources where the nominal metadata frame rate perfectly aligns with the real-world physical sampling rate (i.e., meta FPS = PhyFPS), strictly excluding videos with ambiguous post-hoc time-scale editing.”** This exclusion principle is a defining property of the benchmark [2603.14375].

The benchmark is partitioned from a larger curated dataset assembled from five source categories:

| Source category | Examples | Stated rationale |
|---|---|---|
| High-Frame-Rate Academic Datasets | Adobe240, BVI-VFI | Precise temporal analysis and frame interpolation |
| Raw Broadcast Sequences | UVG | Raw pipeline reduces hidden temporal remapping |
| Sensor-Synchronized Autonomous Data | NVIDIA, Honda | Camera/LiDAR/IMU synchronization is said to guarantee physical time-scale integrity |
| Physics-Grounded Human Motion | Mehta et al. | Included for biomechanical realism |
| Verified In-House Data | Internally collected video | Controlled capture settings with verified frame-rate metadata |

The benchmark uses a **strict cross-source split**: training, validation, and test sets are derived from **entirely disjoint video sources**. This is one of its most consequential design choices, because it is intended to make test performance reflect generalization across capture domains rather than memorization of source-specific statistics or artifacts [2603.14375].

The paper gives two scale figures. The full dataset after augmentation contains **465,535 video clips**, while the PhyFPS-Bench-Real **test set contains 4,000 verified clips**. All prepared clips are standardized to **128 frames**. The distribution spans **18 target Physical Frame Rates**:
$$
\{2, 5, 10, 12, 15, 18, 20, 24, 25, 30, 35, 40, 45, 50, 60, 90, 120, 240\}.
$$
The paper does not provide exact train/validation/test counts beyond the 4,000-clip test set, but it does specify the cross-source split principle and the target-rate coverage [2603.14375].

## 3. Controlled resampling, labels, and task formulation

Although PhyFPS-Bench-Real evaluates on real verified clips, the underlying dataset is expanded through controlled temporal resampling. Source videos are first upsampled to a common high-rate base with **high-rate video** $I^H$ at **base frequency** $F_H = 240$ FPS. For a target lower rate $F_L$, the downsampling ratio is
$$
N = \frac{F_H}{F_L}.
$$
The paper then synthesizes lower-rate videos using three camera-physics-inspired strategies [2603.14375].

For **Sharp Capture (fast shutter)**,
$$
I^{L}_{k}=I^{H}_{\lfloor kN \rfloor}.
$$
This is uniform temporal subsampling.

For **Motion Blur (variable exposure)**,
$$
I^{L}_{k}=\frac{1}{M}\sum_{i=0}^{M-1} I^{H}_{\lfloor kN \rfloor + i},
$$
with
$$
M \in \{N, N/2, N/4\}.
$$

For **Synthetic Rolling Shutter**, the frame is partitioned into progressive bands and each spatial location is sampled at a shifted temporal index,
$$
\lfloor kN \rfloor + \left\lfloor M \cdot \frac{x}{W} \right\rfloor,
$$
for column $x$ and width $W$, again with
$$
M \in \{N, N/2, N/4\}.
$$

These augmentations are not benchmark labels in themselves; rather, they define how supervision is generated from temporally trustworthy source material. Label provenance is therefore derived from **trusted source frame rates** and **known target PhyFPS** induced by the controlled resampling pipeline. No human raters are described for assigning PhyFPS labels on PhyFPS-Bench-Real, and no model-derived pseudo-labels are used [2603.14375].

The benchmark task is **continuous regression** from a single video clip. The input is represented as
$$
\mathbf{V}=\{I_t\}_{t=1}^{T},
$$
and the model predicts a scalar in log-space,
$$
\hat{s} \in \mathbb{R}, \qquad \hat{s} = \log(\hat{y}),
$$
where $\hat{y}$ is predicted PhyFPS. The target is likewise normalized as
$$
s = \log y.
$$
Although training support values are sampled from discrete target rates, the paper explicitly frames the problem as **absolute continuous regression** of PhyFPS rather than classification or ranking [2603.14375].

## 4. Evaluation protocol and reported performance

PhyFPS-Bench-Real is evaluated with **Mean Absolute Error (MAE)** and **Mean Absolute Percentage Error (MAPE)** over the 4,000-clip test set:
$$
\text{MAE} \;=\; \frac{1}{n}\sum_{i=1}^{n}\left|y_i - \hat{y}_i\right|, \qquad
\text{MAPE} \;=\; \frac{100}{n}\sum_{i=1}^{n}\frac{\left|y_i - \hat{y}_i\right|}{y_i}.
$$
Here, $y_i$ is ground-truth PhyFPS and $\hat{y}_i$ is predicted PhyFPS. MAE measures error in FPS units; MAPE measures proportional error in percentage terms [2603.14375].

The average ground-truth PhyFPS over the test set is **38.81**. The paper compares two variants of **Visual Chronometer** against several **Vision-Language Models (VLMs)** in both video-based and image-based prompting setups. The principal comparison is summarized below.

| Method | Setup | Reported result |
|---|---|---|
| VC-Common | Proposed | Avg Pred 39.20, MAE 3.46, MAPE 9% |
| VC-Wide | Proposed | Avg Pred 45.48, MAE 7.76, MAPE 21% |
| Seed-1.6-Flash | Video-based VLM | Avg Pred 30.00, MAE 20.00, MAPE 40% |
| Gemini-3.1-Pro | Video-based VLM | Avg Pred 31.00, MAE 21.67, MAPE 43% |
| Qwen3.5+ | Video-based VLM | Avg Pred 4.46, MAE 45.54, MAPE 91% |
| Seed-1.6-Flash | Image-based VLM | Avg Pred 30.00, MAE 20.00, MAPE 40% |
| Gemini-3-Flash | Image-based VLM | Avg Pred 1.77, MAE 48.23, MAPE 96% |

The full table in the paper includes additional VLMs, namely **Gemini-3.1-Pro**, **Gemini-3-Flash**, **Seed-1.6**, **Seed-1.6-Flash**, **Qwen3.5+**, and **Qwen3.5-397B**, each in video-based and image-based variants. The strongest reported benchmark result is **VC-Common: MAE 3.46, MAPE 9%**. The broader-range **VC-Wide** is less precise, with **MAE 7.76, MAPE 21%**, which the paper interprets as the expected tradeoff from predicting over a larger frame-rate range [2603.14375].

The paper’s main conclusion from these results is that generic VLMs are **not reliable PhyFPS estimators**. It states that they “fail catastrophically,” often showing **mode collapse**. The explicit example is **Seed-1.6-Flash**, which predicts exactly **30 FPS** for all inputs in one setup [2603.14375].

## 5. Ablations, inference regime, and benchmark-specific failure modes

The paper reports two ablation families directly on PhyFPS-Bench-Real. The first is a **temporal augmentation ablation**. Without motion blur or rolling shutter, a **Naive Baseline** yields **MAE 5.12, MAPE 13%**. Adding **motion blur** improves this to **MAE 4.87, MAPE 11%**. The full **VC-Common** configuration, which adds **rolling shutter** as well, achieves **MAE 3.46, MAPE 9%** [2603.14375].

This ablation is used to argue that camera-physics-aware augmentation is not incidental. The benchmark is specifically designed so that the predictor must recover physical timing under realistic capture effects rather than idealized frame subsampling. A plausible implication is that PhyFPS-Bench-Real measures not only motion-speed sensitivity but also robustness to acquisition artifacts that materially affect temporal cues.

The second ablation studies **temporal context length** with inference windows
$$
T \in \{8,16,32,64,128\}.
$$
The base model, trained on maximum **32 frames**, performs best at **$T=32$** with **MAE = 3.46**, extrapolates well to **$T=64$** and **$T=128$**, and benefits from post-training up to maximum length 128 at **$T=64$** without harming short-context accuracy. The paper reports a tradeoff: clips that are **too short** contain insufficient motion evidence, while contexts that are **too long** reduce the benefit of sliding-window ensembling and weaken the ability to capture local PhyFPS fluctuation. The reported sweet spot is **$T=32$ to $T=64$** [2603.14375].

The qualitative illustration associated with the benchmark shows a soccer-ball juggling action captured at **60, 24, and 12 PhyFPS**, with the model said to recover the absolute time scale and maintain **remarkable temporal stability across the entire sequence**. In the paper’s logic, this supports the claim that the estimator is not merely regressing toward a dataset average [2603.14375].

## 6. Position within adjacent benchmark literature

PhyFPS-Bench-Real is specifically a benchmark for **ground-truth Physical FPS estimation** from real videos. It is therefore narrower than physics-aware editing benchmarks and different in objective from real-world scientific forecasting benchmarks. The paper most directly adjacent in topic is "PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing," which evaluates **instruction-guided image editing** under physically grounded transformations such as falling, splashing, deformation, fracture, diffusion, and phase change, using **238 real-world instances** and **35 synthetic Anti-Physics instances** [2606.26551]. That benchmark addresses **physical plausibility in edited images**, whereas PhyFPS-Bench-Real addresses **physical time-scale recovery from video dynamics**.

A different neighboring line is represented by "RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data," which targets **predicting the future evolution of complex physical systems** under a sim-to-real regime using **five datasets**, **three tasks**, and **real-world validation/test data** [2601.01829]. RealPDEBench is organized around spatiotemporal field forecasting and physics-oriented metrics such as **fRMSE**, **FE**, **KE**, and **MVPE**, whereas PhyFPS-Bench-Real is organized around temporal-scale estimation with **MAE** and **MAPE** on real clips.

These comparisons clarify the scope boundary. PhyFPS-Bench-Real is not a benchmark for editing quality, physical plausibility scoring, or PDE rollout forecasting. Its specific technical object is whether motion alone reveals a stable and correct **real-world time scale**, and whether an estimator can recover that quantity on a **4,000-clip cross-source test set** [2603.14375].

In the paper’s overall argument, that makes PhyFPS-Bench-Real the empirical bridge between the concept of **chronometric hallucination** and the audit of generated videos. The workflow is explicit: build a benchmark of real videos with trustworthy physical time labels, show that **Visual Chronometer** predicts those labels accurately, then use the validated predictor to analyze generated videos. The benchmark’s headline result—**VC-Common: 3.46 MAE and 9% MAPE**—is what licenses the later claim that state-of-the-art generators suffer from severe **PhyFPS misalignment** and temporal instability [2603.14375].

Source: https://www.emergentmind.com/topics/phyfps-bench-real