2000 character limit reached
Planning with Sketch-Guided Verification for Physics-Aware Video Generation (2511.17450v1)
Published 21 Nov 2025 in cs.CV, cs.AI, and cs.CL
Abstract: Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview: What is this paper about?
This paper is about helping AI make short videos from a picture and a text prompt where objects move in realistic, physics-friendly ways. The authors introduce a method called “SketchVerify.” Instead of immediately generating a full video (which is slow and often makes mistakes), SketchVerify first plans motion using simple “video sketches,” checks if those motions make sense, and only then creates the final, high-quality video. The goal is to get better, more believable motion while using less computer power.
Key Questions the paper tries to answer
- How can an AI plan object movements in a video so they follow the instructions and obey basic physics (like gravity and not passing through walls)?
- Can we catch and fix bad motion plans before spending lots of time generating full videos?
- Is there a faster way to improve motion quality without retraining the model or repeatedly re-generating videos?
How they did it (methods explained simply)
Think of making a mini-movie by moving a cut-out sticker across a background to preview the action before filming. SketchVerify follows a three-part process:
- Plan the steps and find the moving objects
- The AI reads your prompt (like “move the apple toward the basket, then pick it up”) and breaks it into smaller steps (sub-instructions).
- It looks at the starting image and “cuts out” the object that will move (this is called segmentation—like tracing and cutting around the object).
- It also creates a clean background by filling the hole where the object was (inpainting—like patching the area so the background looks normal).
- Try out several motion plans as lightweight “video sketches”
- For each step, the AI samples multiple possible paths for the object. Each path is just a series of rectangles showing where the object would be in each frame (these rectangles are called bounding boxes; they’re like a tight outline around the object).
- Instead of making full videos, it makes quick “sketch videos” by pasting the cut-out object onto the background along each path. This is fast and lets the system preview motion like sliding a sticker frame-by-frame.
- Use a smart “judge” to pick the best motion
- A multimodal verifier (a powerful AI that can understand both images and text) watches each sketch and scores it in two ways:
- Does it follow the instruction? For example, if the instruction says “move toward the basket,” does the object actually move toward the basket?
- Does it obey physics? The verifier checks simple rules you already know from school:
- Newtonian consistency: Speeds and accelerations look natural, not jerky or teleporting.
- No penetration: Objects don’t pass through walls, tables, or other solid things.
- Gravity coherence: Up-and-down motion makes sense (things don’t float without reason).
- Shape stability: Objects don’t stretch or squish unrealistically.
- The system picks the highest-scoring motion plan. If none are good enough, it tries again with feedback until it finds a solid plan.
- A multimodal verifier (a powerful AI that can understand both images and text) watches each sketch and scores it in two ways:
Finally, once the best plan is chosen, a high-quality video generator follows that path to produce the finished video. Think of it like filming after you’ve rehearsed with your sticker storyboard.
Main findings and why they matter
- Better motion quality: Videos follow instructions more accurately and obey physics more consistently (fewer “ghosting through walls,” less floating, more natural movement).
- More stable over time: Motions look smooth and consistent across many frames, not just in the beginning.
- Much faster planning: Verifying sketch videos is way cheaper than repeatedly generating full videos. In tests, planning took about five minutes instead of well over half an hour, and sometimes close to 10× speed-ups compared to methods that regenerate full videos multiple times.
- Scales with more options: Sampling more candidate paths leads to better results because the verifier can choose from a wider variety of motions.
- Seeing beats guessing: A visual+text verifier outperforms text-only checks, because it can directly “see” the motion, not just read a description.
These results were measured on two big benchmarks that test instruction-following and physics realism. SketchVerify scored higher than strong baselines and did it more efficiently.
Why this matters (implications and impact)
- Smarter video tools: Content creators and game designers can get more believable motion without spending tons of time or computing power.
- Safer robot and simulation planning: Systems that need realistic motion (like robots or self-driving simulations) can plan and verify movements faster and more reliably.
- Better physics awareness in AI: This “plan with sketches, then verify” idea helps AIs respect simple physical rules, making their outputs feel more natural and trustworthy.
- Training-free and practical: SketchVerify works at test time—no retraining needed—so it can plug into existing video generators and start improving results right away.
In short, SketchVerify makes AI video generation both smarter and faster by rehearsing motion with simple sketches and using a visual judge to pick the best plan before filming the final scene.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a concise list of concrete gaps and unresolved questions that future work could address:
- Multi-object concurrency and interactions: The planning and sketch verification are described primarily for a single moving object per sub-instruction; scalability to simultaneous multi-object planning, collision avoidance, and coordinated interactions (e.g., hand–object, object–object contacts) is not demonstrated.
- Static background and 2D-only reasoning: Sketches are composited over a static inpainted background, ignoring camera motion and dynamic scene elements; verification is performed in image-plane 2D without depth or 3D layout awareness.
- Camera motion support: The framework does not model or generate camera trajectories (pan/tilt/zoom/ego-motion); extending SketchVerify to jointly plan object and camera motion and to verify under moving-camera conditions remains open.
- Occlusion and depth-order handling: The verifier cannot reason about occlusions, depth ordering, or objects moving behind/around 3D structures; integrating depth maps or 3D scene reconstructions could enable depth-aware verification.
- Sketch fidelity limitations: The sketch uses a single static crop “sprite” pasted across frames, which cannot express rotation, articulation, foreshortening, or perspective-induced scale changes; this can mislead both semantic and deformation checks.
- Physically realistic rotation and angular dynamics: Verification currently ignores torque, angular momentum, and rotational stability; adding checks for orientation dynamics and rotational consistency is an open direction.
- Collision checking with moving entities: Penetration checks are only against static scene elements; collisions and contacts with other moving objects are not verified.
- Coverage of physical laws: The internal verifier covers four dimensions (Newtonian, penetration, gravity, deformation) but omits friction, elasticity, restitution, momentum/energy conservation, support forces, and contact durations; richer physics priors or differentiable physics integration are unexplored.
- Fluids and deformables: Although PhyWorldBench evaluates fluid/deformable behavior, the internal verifier lacks fluid/cloth-specific checks; specialized surrogate renderers and criteria for nonrigid dynamics are needed.
- Domain-aware physics: The verifier assumes Earth-like gravity and everyday physics; handling domains with altered rules (e.g., video games, microgravity) or prompt-conditioned physics priors is not addressed.
- Planner–verifier black-box dependence: The method relies on proprietary MLLMs (GPT-4.1 as planner, Gemini 2.5 as verifier); reproducibility, stability under API/model updates, and feasibility of open-source replacements or distillation are not studied.
- Verifier reliability and calibration: There is no formal audit of the verifier’s accuracy, calibration, or failure modes on labeled trajectory-sketch datasets; robustness to adversarial or confounding sketches is unknown.
- Hand-crafted scoring and thresholds: Mappings from textual judgments to numeric scores, weight coefficients λ, and the quality threshold τ are heuristic; sensitivity analyses, principled tuning, or learned weighting/calibration are missing.
- Search strategy and sampling budget: Candidate generation uses fixed K and simple rejection; adaptive or active search (e.g., beam search, Bayesian optimization, MCTS) guided by verifier uncertainty/confidence is unexplored.
- Long-horizon/global consistency: Planning proceeds sequentially per sub-instruction using only the last sketch frame as context; joint optimization over the entire plan to avoid myopic choices and cross-step inconsistencies is not considered.
- Post-generation drift and lack of closed-loop correction: The final diffusion output is not re-verified; discrepancies between the verified plan and synthesized video are not detected or corrected with lightweight post-hoc editing/feedback.
- Segmentation and object discovery robustness: The pipeline depends on GroundedSAM and prompt-derived object lists; failure cases (small/transparent/occluded objects, category mismatches) and recovery strategies (uncertainty-aware detection, iterative discovery) are not analyzed.
- Identity consistency and tracking: Ensuring persistent object identity across sub-instructions and in the final video (avoiding swaps/drift) is not enforced; identity-aware constraints or tracking losses are absent.
- Metric scale and time realism: Speeds/accelerations are not normalized to scene geometry or metric scale; estimating scene scale (e.g., via depth) to enforce unit-consistent kinematics is an open problem.
- Evaluation dependence on MLLM scorers: Both benchmarks rely on MLLM-based assessment; correlation with human judgments and physics ground truth, and potential evaluation bias, are not quantified.
- Generalization and OOD robustness: Performance under dynamic/handheld cameras, cluttered indoor/outdoor scenes, different art styles, and extreme OOD conditions is not reported; broader stress-testing benchmarks are needed.
- Integration with richer controls and 3D generators: The approach conditions only on 2D trajectories; extending to 3D object poses, scene graphs, contact events, and camera paths—and integrating with 3D-aware generators—remains open.
- Resource and latency reporting: Planning-time gains are reported, but hardware/API latency, cost variability, and parallelism assumptions are not standardized; reproducible cost–quality trade-off reporting is lacking.
Glossary
- Ablation study: A controlled analysis that removes or varies components to measure their effect on performance. "Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance."
- Background inpainting: Filling removed or occluded regions of an image to reconstruct a plausible background. "fill them using Omnieraser, a background inpainting model fine-tuned from FLUX"
- Bounding box: A rectangle defined by coordinates that localizes an object in an image frame. "represented as a sequence of bounding boxes capturing the object's location at each frame."
- Compositing: Layering visual elements onto a background to form a single image or video. "by compositing objects over a static background"
- Commonsense consistency: Alignment of generated content with everyday causal and logical expectations across frames and time. "overall commonsense consistency"
- Dense trajectory: A time-continuous sequence of positions specifying object motion at each frame. "We temporally interpolate this sequence to produce a dense trajectory"
- Denoising process: The iterative removal of noise in diffusion models to reveal a coherent image or video. "modulates the denoising process by injecting object trajectory latents."
- Diffusion-based synthesis: Generating images or videos by iteratively denoising from noise using diffusion models. "bypasses the need for expensive, repeated diffusion-based synthesis"
- FLUX: A generative model used as a base for fine-tuning specialized image tools. "Omnieraser~\cite{wei2025omnieraser}, a background inpainting model fine-tuned from FLUX~\cite{flux2024}"
- GroundedSAM: A detector–segmenter model pair used for precise object mask extraction. "we apply a detector–segmenter pair, GroundedSAM~\cite{ren2024grounded,kirillov2023segment,liu2023grounding}, for precise mask extraction"
- Gravity-coherent vertical motion: Motion in the vertical direction that adheres to realistic effects of gravity. "gravity-coherent vertical motion"
- Image-to-Video (I2V) generation: Transforming a single image into a temporally coherent video. "Image-to-Video (I2V) generation has demonstrated strong potential across a wide range of applications"
- In-context learning: Guiding model reasoning with a few examples provided directly in the prompt. "few-shot in-context learning"
- Iterative refinement: Repeatedly updating prompts or control signals to improve generated results. "iterative refinement which requires multiple calls to the video generator, incuring high computational cost."
- Layout-guided diffusion synthesis: Using spatial layouts (e.g., trajectories, keyframes) to condition diffusion-based generation. "which are then used for layout-guided diffusion synthesis"
- MLLM (Multimodal LLM): A LLM that can process and reason over both text and visual inputs. "Recent work increasingly leverages LLMs and MLLMs to provide structured planning for video generation."
- Motion priors: Assumptions or learned regularities about how objects typically move in the physical world. "based on real-world motion priors."
- Multimodal verifier: A model that evaluates candidate plans using both vision and language inputs. "SketchVerify integrates a multimodal verifier with a test-time search procedure"
- Newtonian consistency: Adherence of motion to Newton’s laws (e.g., realistic acceleration/deceleration). "Newtonian Consistency: Acceleration and deceleration should reflect plausible physical dynamics."
- Non-penetration: The physical constraint that objects should not pass through other solid elements. "non-penetration with scene elements"
- Object segmentation: Partitioning an image to isolate specific objects via pixel-level masks. "identifies the corresponding movable objects through segmentation."
- Omnieraser: A fine-tuned inpainting model used to reconstruct static backgrounds. "Omnieraser~\cite{wei2025omnieraser}, a background inpainting model fine-tuned from FLUX"
- Penetration Violation: A failure mode where a moving object passes through scene elements incorrectly. "Penetration Violation: Moving objects should not pass through static scene elements."
- Physical plausibility: The degree to which motions and interactions conform to realistic physical behavior. "physical plausibility through structured reasoning"
- PhyWorldBench: A benchmark that tests fine-grained physical realism in generative video models. "Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality"
- Semantic alignment: The consistency between generated motion/content and the given textual instruction. "semantic alignment with the instruction"
- Sparse trajectory: A set of key positions (e.g., centers) defining object motion over time with limited points. "resulting in a sparse trajectory"
- Structured prompts: Carefully designed prompt templates to elicit targeted reasoning from models. "using structured prompts and few-shot in-context learning"
- Temporal coherence: Consistent and stable motion and appearance across consecutive video frames. "to improve temporal coherence and motion fidelity."
- Test-time planning: Generating and refining control signals during inference without additional training. "a test-time planning framework"
- Trajectory-conditioned generator: A video generator that uses provided object trajectories to control motion. "which is then passed to the trajectory-conditioned generator for final synthesis."
- Trajectory latents: Encoded representations of trajectories injected into a model to guide generation. "injecting object trajectory latents"
- Trajectory sampling: The process of generating multiple candidate motion paths for evaluation. "Trajectory Sampling."
- Video diffusion models: Generative models that synthesize videos via iterative denoising. "state-of-the-art video diffusion models often violate even basic physical laws"
- Video sketch: A lightweight visualization of motion by pasting segmented objects onto a static background. "render each trajectory as a lightweight video sketch"
- Vision-language verifier: A verifier that jointly considers visual input and text to score candidate plans. "vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility."
- WorldModelBench: A benchmark evaluating instruction following, physics, and commonsense in video generation. "Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality"
Practical Applications
Immediate Applications
The following applications can be deployed now by leveraging the paper’s training-free, sketch-based verification loop and multimodal trajectory ranking. Each item notes sectors, likely tools/workflows, and feasibility assumptions.
- Summary: Preflight physics and instruction checks for existing video generation pipelines
- Sectors: Software, media/entertainment, advertising
- Use case: Integrate SketchVerify as a preflight module that samples and ranks motion trajectories via lightweight sketches, then passes the selected plan to any trajectory-conditioned I2V/T2V model (e.g., Wan-2.1, CogVideoX, Open-Sora).
- Tools/workflows: “Verifier-as-a-Service” API; “Sketch Composer” for object segmentation and static background inpainting; “MotionPlanRanker” that outputs a PhysicsScore and semantic alignment score; SDK plugins for major video gen tools.
- Assumptions/dependencies: Access to an MLLM/VLM verifier (e.g., Gemini 2.5), a trajectory-conditioned video generator, reliable segmentation (GroundedSAM) and inpainting (Omnieraser). Works best with static backgrounds and single/multiple object motions without moving cameras.
- Robotics pre-visualization and task scripting
- Sectors: Robotics, manufacturing
- Use case: Generate physically plausible “storyboards” of manipulation tasks (approach, grasp, place) as verified motion plans before simulation or real execution; validate visual demos for imitation learning.
- Tools/workflows: Robot task authoring “SketchVerify Coach”—plan-verify-synthesize pipeline to produce physics-consistent training clips; QC gate that flags penetration and gravity violations prior to data ingestion.
- Assumptions/dependencies: Static or weakly changing scenes; the physics criteria focus on Newtonian consistency, non-penetration, gravity, and deformation; complex contacts and compliance are only implicitly reasoned.
- Game content pipelines and cutscene pre-visualization
- Sectors: Gaming, animation/VFX
- Use case: Rapidly prototype motion sequences (characters, items) that adhere to instructions and basic physics, using sketches to iterate in minutes instead of full renders; reduce re-render cycles and art-direction overhead.
- Tools/workflows: “SketchBoards” for layout-level motion reviews; automated trajectory ranking; batch generation of candidate motions with K>1 sampling and verifier scoring.
- Assumptions/dependencies: Static backgrounds or locked cameras during pre-vis; verification quality depends on segmentation accuracy and prompt clarity.
- Creative/video apps that “fix motion” before synthesis
- Sectors: Consumer software, prosumer creators
- Use case: In mobile/desktop video apps, offer a “Physics Check” toggle that previews motion via a sketch and auto-corrects implausible trajectories (e.g., floating, tunneling through objects) before final generation.
- Tools/workflows: Client-side lightweight sketch renderer; server-side verifier; UI surfacing physics warnings and suggested corrections; presets for sports plays, object moves, and narrative beats.
- Assumptions/dependencies: Cloud verifier access; user-provided initial frame; reasonable compute for sampling multiple trajectories.
- Benchmarking and QC for academic datasets and models
- Sectors: Academia, ML tooling
- Use case: Use SketchVerify to audit motion trajectories and curate physics-consistent subsets for training/evaluation; report aggregated PhysicsScore to compare model variants and planning strategies.
- Tools/workflows: Batch verification scripts; dataset filters using semantic and physical thresholds (τ); integration with WorldModelBench/PhyWorldBench scoring pipelines.
- Assumptions/dependencies: Access to benchmark prompts/frames; reproducible verifier prompts and score mappings; calibration of weight coefficients λ for semantic vs physics scores.
- Policy and platform safety checks for generative content
- Sectors: Policy, content moderation, platform governance
- Use case: Use physics plausibility checks to flag misleading generative clips (e.g., impossible object interactions) and to enforce platform guidelines for realism in certain categories (education, health).
- Tools/workflows: Moderation triage using sketch-level verification prior to distributing full videos; “Realism Label” metadata informed by PhysicsScore.
- Assumptions/dependencies: Policy definitions for acceptable physical realism; human-in-the-loop review for borderline cases; limited scope for highly stylized or intentionally non-physical content.
- STEM education: interactive physics demonstrations
- Sectors: Education
- Use case: Teachers/students generate short videos from an initial image and prompt (e.g., projectile motion), with built-in physics checks ensuring gravity-coherent arcs and non-penetration; quick formative feedback on motion correctness.
- Tools/workflows: Classroom app with motion sketches, per-law score breakdown (Newton, Gravity, Penetration, Deformation), and corrective hints; export verified trajectories to full clips.
- Assumptions/dependencies: Simplified scenes and clear prompts; verification calibrated to curriculum-level physics rather than advanced dynamics.
Long-Term Applications
These applications require further research, scaling, or development—especially around dynamic backgrounds, camera motion, multi-object interactions, and tighter integration with simulators or real systems.
- Real-time, closed-loop robot motion planning with visual verification
- Sectors: Robotics, industrial automation
- Use case: Integrate verifier-guided planning into robot control loops to visually vet candidate trajectories before execution; reject plans with predicted collisions or implausible dynamics.
- Tools/workflows: “Visual Safety Gate” connecting task-level planners to on-robot execution; hybrid pipelines that fuse SketchVerify with physics simulators (e.g., MuJoCo/Isaac) for higher-fidelity checks.
- Assumptions/dependencies: Real-time MLLM/VLM inference at edge or low-latency cloud; robust segmentation and scene understanding; expanded verifiers covering contact-rich manipulation, compliance, and dynamic backgrounds.
- Physics-aware synthetic data engines at scale
- Sectors: Software, robotics, autonomous vehicles
- Use case: Generate large libraries of verified motion videos for training perception, prediction, and policy models—covering long horizons, multi-agent interactions, and moving cameras.
- Tools/workflows: Distributed sampling-verification farms; curriculum generators that escalate scenario complexity; automatic labeling with semantic + physics scores and failure taxonomies.
- Assumptions/dependencies: Efficient large-batch verification; improved handling of occlusions, camera motion, and complex fluids; cost-effective compute.
- Standards and audits for physical realism in AI media
- Sectors: Policy, media regulation, legal tech
- Use case: Establish a standardized “Physical Realism Score” (and per-law breakdown) for generative videos; require disclosures or minimum scores for certain contexts (education, news, healthcare).
- Tools/workflows: Open scoring specs (prompt templates, mappings); certified verifier models; compliance dashboards for platforms and publishers.
- Assumptions/dependencies: Consensus on metrics and acceptable thresholds; governance around verifier integrity and bias; exemptions for artistic content.
- Advanced pre-visualization in film/TV with dynamic scenes
- Sectors: Media/entertainment
- Use case: Extend sketches to support moving cameras, dynamic lighting, and complex multi-object choreography; reduce costly re-shoots and VFX iterations by validating motion logic ahead of full production.
- Tools/workflows: 3D-aware sketch compositing; depth/occlusion reasoning; multi-object trajectory verification with collision maps; integration with virtual production toolchains.
- Assumptions/dependencies: 3D scene understanding and camera tracking; richer physics heuristics or learning-based physical validators.
- Smart city and urban planning simulations
- Sectors: Public sector, civil engineering
- Use case: Prototype traffic, pedestrian flow, and crowd behaviors with physics-aware generative videos; vet candidate interventions (crossings, signals) for plausibility before expensive simulations.
- Tools/workflows: Scenario generators combining GIS imagery with motion plans; multi-agent trajectory verifiers; comparison dashboards for policy alternatives.
- Assumptions/dependencies: Multi-agent dynamics and interaction laws in the verifier; camera motion and large-scale scene parsing; alignment with domain simulators.
- Healthcare training and surgical robotics visualization
- Sectors: Healthcare, medical education
- Use case: Produce physics-aware procedural videos for training and rehearsal; visually verify tool-tissue trajectories for consistency with expected forces and constraints before sim or teleop.
- Tools/workflows: Domain-adapted verifier prompts for medical motions; integration with surgical simulators; feedback loops that suggest corrections to motion plans.
- Assumptions/dependencies: Specialized segmentation and scene understanding in medical contexts; expanded physics criteria for soft tissue, fluids, and compliance.
- Multi-object, multi-modal reasoning and dynamic backgrounds
- Sectors: All sectors using complex scenes (gaming, robotics, AV)
- Use case: Extend SketchVerify to handle several moving objects, occlusions, non-rigid deformations, and camera motion; verify interactions (contact timing, momentum transfer).
- Tools/workflows: Enhanced sketch rendering (depth layers, parallax), multi-object verifiers with interaction graphs, physically informed priors learned from simulators.
- Assumptions/dependencies: Stronger VLMs with 3D and temporal reasoning; scalable sampling for combinatorial trajectory spaces; tighter coupling to physics engines.
- Integrated co-design of planners and verifiers inside generative models
- Sectors: Software/AI
- Use case: Train or finetune generative models with embedded verifier-guided objectives (reward models or auxiliary losses) for physics consistency at generation time, reducing reliance on external test-time loops.
- Tools/workflows: RLHF/RLAIF pipelines using PhysicsScore; curriculum training on trajectory plans; joint optimization of planner and denoiser.
- Assumptions/dependencies: Access to training data and compute; reliable automatic scoring; preventing mode collapse or over-regularization that harms creativity.
Cross-cutting assumptions and dependencies
- Verifier access and capability: Results depend on strong multimodal verifiers (Gemini 2.5, GPT-4.1), clear prompts, and calibrated score mappings; smaller models yield weaker gains.
- Scene preparation: Accurate segmentation (GroundedSAM) and background inpainting (Omnieraser) are prerequisites; failure here degrades sketch fidelity and verifier judgments.
- Generator compatibility: A trajectory-conditioned diffusion model (e.g., ATI-14B) is needed to consume selected plans; non-conditioned models require adaptation.
- Scope limitations: Current physics checks cover Newtonian consistency, non-penetration, gravity coherence, and deformation stability; complex fluid dynamics, elastic contacts, and camera motion are limited.
- Efficiency vs fidelity: Sketch-based verification trades full appearance for layout-level motion fidelity; diffusion artifacts in full videos can mislead verifiers, whereas sketches remain clean but assume static backgrounds.
- Human oversight: For safety-critical or policy applications, keep a human-in-the-loop for edge cases and for defining acceptable realism thresholds.
Collections
Sign up for free to add this paper to one or more collections.