Perception-Driven Motion Metrics (PMM)

• Perception-Driven Motion Metrics are quantitative measures that assess motion using human perceptual responses and physical constraints.
• They combine simulation-based physical corrections with human annotations to generate unified, data-driven fidelity scores for motion evaluation.
• PMM applications span human motion synthesis, autonomous safety, video quality, and robotic grasping, offering enhanced accuracy over traditional metrics.

Perception-Driven Motion Metrics (PMM) are quantitative measures that evaluate motion (of agents, objects, or scenes) in artificial and biological systems through the lens of perceptual criteria—either by directly incorporating human perceptual responses, task-induced behavioral consequences, or cogent physical feasibility constraints. Rather than focusing solely on geometric or kinematic error, PMM frameworks prioritize evaluation signals derived from perception itself, whether through simulation-based physical annotation, psychophysical preference, probabilistic collision analysis, or the internal structure of video coding aligned with the human visual system. They have been formalized for human motion generation, video assessment, robot grasping, perception-aware planning, and biological vision.

1. Formal Definitions and Methodological Frameworks

Multiple PMM formulations exist, each rooted in the evaluation of motion through perception-sensitive principles. For human motion synthesis evaluation, as in the PP-Motion framework, PMM are defined as unified, data-driven metrics that produce scalar fidelity scores $F(x;\theta)$ for a given motion sequence $x$, supervised by both (i) continuous physical alignment labels from a physics-simulation-based correction network and (ii) human perception judgments via ranking or classification. The physical error is quantified as the Euclidean 2-norm between the original and the minimally corrected, physically feasible motion: $e_p = \| x - x' \|_2$ where $x'$ is obtained by a simulator-trained correction network $F_p(x)$. These physical errors are then normalized and used as regression targets; the network is trained with a combination of Pearson-correlation loss (for physical alignment) and ranking loss (for perceptual alignment) (Zhao et al., 11 Aug 2025).

In safety-critical robotics and autonomous vehicles, PMM often aggregate detection/tracking quality with dynamic relevance terms: object velocity, size, orientation, distance, and inferred collision risk. For instance, the S metric aggregates CLEAR metrics (MODA/MOTA, MODP/MOTP) with safety weights derived from per-object parameters and collision geometry classes to yield a scenario-level safety score in $[0,1]$ (Volk et al., 16 Dec 2025).

In planner-centric frameworks, PMM quantify the expected degradation in downstream planning cost (e.g., collision risk, time-to-goal) induced by the perception system, with each detection weighted according to modeled relevance (distance, speed): $$\text{PMM}_t(A) = \mathbb{E}_{x \sim p_\theta(\cdot | O_t(A))} [C[x]] - \mathbb{E}_{x \sim p_\theta(\cdot | O_t^*)}[C[x]]$$ with per-object weights $$w_j = \exp[-(\| p_j - p_{\text{ego}} \| / d_0)^2 ] \cdot \sigma(v_j / v_0)$$ upweighting close and fast objects (Philion et al., 2020).

For video motion analysis, PMM are constructed as a vector of scalar metrics each targeting a key human-perceived aspect: commonsense adherence, motion smoothness, object integrity, perceptible amplitude, and temporal coherence. Each submetric is algorithmically computable (e.g., via classifier outputs, tracking errors, or aesthetic discontinuities), forms a $[0,1]$-bounded score, and the aggregate correlates tightly with human judgments (Ling et al., 13 Mar 2025).

In perception-driven grasping, PMM include real-time metrics grounded in actuator physics and sensory fusion: static/dynamic payload capability as a function of sensed torque, force, acceleration, temperature, and friction; these metrics define dynamic safety envelopes for manipulation (Bianco et al., 7 Apr 2025).

2. Architectures and Algorithms

State-of-the-art PMM systems employ architectures that align directly with their target domains:

• Human motion evaluation (PP-Motion): Utilizes dual-stream spatio-temporal Transformer encoders (DSTformer) and MLP decoders. Supervision combines normalized physics error labels and human annotations, with batch-wise prompt conditioning for improved correlation structure. Training leverages AdamW with prompt-level batch partitioning and scheduled decay (Zhao et al., 11 Aug 2025).
• Perception-aware safety for autonomous systems: Implements sequential object–detection/tracking matching (Hungarian pairing), scenario-dependent scaling functions for per-object error, and predictive future-safety zone estimation (e.g., using RSS lateral/longitudinal rules), with frame-level complexity $O(N^2)$ reduced by spatial binning (Volk et al., 16 Dec 2025).
• Planner-centric evaluation: Requires probabilistic inference over T-step trajectories with and without candidate detections, ablation analysis for marginal contributions, and end-to-end differentiability for optimization (Philion et al., 2020).
• Video generation benchmarks: Each dimension of PMM is realized by algorithmic routines: classifier-based scoring for commonsense adherence, tracking- and aesthetic-based metrics for smoothness and integrity, keypoint/displacement for amplitude, and anomaly detection for temporal coherence (Ling et al., 13 Mar 2025).
• Robot grasping control: Real-time control integrates sensor fusion of actuator torque (SEA), joint thermal state, IMU-based acceleration, and visual/ToF alignment, generating continuous PMM for controller thresholding and grasp-planning (Bianco et al., 7 Apr 2025).

3. Physical, Perceptual, and Task-Driven Ground Truth

PMM frameworks bridge the gap between subjective human experience and physical feasibility by:

• Physical annotation: Simulator-derived minimal perturbations produce continuous error labels accurately reflecting physical law violation degrees (Zhao et al., 11 Aug 2025).
• Human supervision: Binary or paired preference labeling from human raters provides perceptual high/low-fidelity ground truth, integrated into loss via ranking or margin-based objectives (Zhao et al., 11 Aug 2025, Ling et al., 13 Mar 2025).
• Cost/risk propagation: In agent/planner scenarios, ground truth is defined as the change in expected mission/planning cost under varying levels and types of perception error, directly reflecting downstream behavioral implications (Philion et al., 2020).
• Heuristic and analytical models: In grasping and manipulation, physically meaningful PMM are developed from first principles of friction, force, inertia, and thermal limits, tightly coupled to dynamic control loops (Bianco et al., 7 Apr 2025).

4. Evaluation Protocols and Empirical Results

PMM are benchmarked across several domains:

Domain	Main Metric(s)	Human Correlation	State-of-the-Art Performance	Notable Baselines
Human motion	Pearson, SROCC, KROCC, "better/worse" accuracy	0.727 (PLCC), 85%+ acc (Zhao et al., 11 Aug 2025)	Exceeds prior by up to +0.4	Pose/distance-based, MotionCritic
Autonomous driving	S safety score [0,1]	Full risk ranking through scenario analysis	Distinguishes safety-relevant errors under same IoU	mAP, IoU, TTC
Video generation	Aggregate PMM vector (CAS/MSS/OIS/PAS/TCS)	Spearman ρ = 0.622 (Ling et al., 13 Mar 2025)	+35.3 pp over best baseline	FID, FVD, CLIP
Grasping	Payload, force, endurance metrics	N/A, but empirically verified for safe operation	No failures under rated PMM	None (first with real-time PMM feedback)

Ablation studies in PP-Motion confirm the necessity of combining physical and perceptual losses: replacing Pearson correlation with MSE or removing prompt conditioning significantly degrades both physical and perceptual alignment (Zhao et al., 11 Aug 2025). Video-benchmark PMM vastly outperform FVD and prior motion metrics on alignment with human Likert judgments.

5. Comparative Analysis with Prior Motion and Perception Metrics

Classical geometric or kinematic metrics (e.g., average per-joint/pose error, FID/FVD for video, mAP/IoU for object detection) fail to account for three core aspects:

• Task/safety relevance: They treat all errors equally, ignoring spatial, velocity, or task-induced importance. PMM rectify this via object weighting, collision geometry, and planner-induced cost differentials (Volk et al., 16 Dec 2025, Philion et al., 2020).
• Physical feasibility: They may score physically implausible outputs highly if they are close in pose space to ground truth; PMM rigorously enforce physical validity via minimal-simulation corrections (Zhao et al., 11 Aug 2025).
• Perceptual realism: They are not tuned to human perception; PMM include explicit psychometric supervision or perceptual feature alignment (Ling et al., 13 Mar 2025).

In safety assessment and real-time robotics, PMM uniquely integrate perception parameters (rate, quality, latency, uncertainty) with collision risk via formal probability models (e.g., CCP/ACP) and flexible processing strategies (e.g., attentive cropping) (Zhang et al., 2023).

6. Mathematical and Algorithmic Properties

PMM frameworks are designed for:

• Continuity and differentiability: Under smooth cost and planner models, PMM vary continuously with perception errors, making them suitable for end-to-end learning or system optimization (Philion et al., 2020).
• Task-aligned ablation sensitivity: They decompose errors to attribute PMM changes to individual detections or motion primitives, guiding targeted system improvement (Philion et al., 2020).
• Scalability and efficiency: Metrics such as physical error or cost differential are computed batch-wise or in highly parallel (per-pixel, per-object) fashion. Complexity is dominated by combinatorial matching or planning-inference step for large N, but sublinear approximations or hardware acceleration are often feasible (Volk et al., 16 Dec 2025, Philion et al., 2020).
• Human-aligned normalization: All scalar metrics are typically mapped to $x$0 for comparability, average aggregation, or threshold-based evaluation (Ling et al., 13 Mar 2025).

7. Application Scope, Best Practices, and Future Directions

Perception-Driven Motion Metrics have been adopted and validated for:

• Human motion generation (realistic digital humans, AR/VR, film, rehabilitation)
• Autonomous driving and multi-agent navigation (perception modules, risk/mission assessment)
• Video synthesis and generative modeling (multi-dimensional quality benchmarking)
• Robotic manipulation and grasping (payload monitoring, safety envelopes)
• Perception-enhanced planning and uncertainty-adaptive control (UAVs, safety-critical planning)

Best practices include joint supervision with physically continuous and perceptual binary signals (Zhao et al., 11 Aug 2025), per-prompt or scenario batch partitioning, and leveraging attentive or hierarchical processing to reduce latency and collision risk (Zhang et al., 2023). Open questions remain around sample efficiency, domain-adaptive weighting, scalable cost propagation, and integration with uncertainty quantification (Philion et al., 2020).

A plausible implication is that as PMM formulations are incorporated into training objectives (e.g., as differentiable loss components), they will drive systematic, perception-aligned improvement in generative and control modules across disciplines where motion realism and safety are paramount.