Epipolar Geometry Improves Video Generation Models (2510.21615v1)

Abstract: Video generation models have progressed tremendously through large latent diffusion transformers trained with rectified flow techniques. Yet these models still struggle with geometric inconsistencies, unstable motion, and visual artifacts that break the illusion of realistic 3D scenes. 3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks. We explore how epipolar geometry constraints improve modern video diffusion models. Despite massive training data, these models fail to capture fundamental geometric principles underlying visual content. We align diffusion models using pairwise epipolar geometry constraints via preference-based optimization, directly addressing unstable camera trajectories and geometric artifacts through mathematically principled geometric enforcement. Our approach efficiently enforces geometric principles without requiring end-to-end differentiability. Evaluation demonstrates that classical geometric constraints provide more stable optimization signals than modern learned metrics, which produce noisy targets that compromise alignment quality. Training on static scenes with dynamic cameras ensures high-quality measurements while the model generalizes effectively to diverse dynamic content. By bridging data-driven deep learning with classical geometric computer vision, we present a practical method for generating spatially consistent videos without compromising visual quality.

• The paper introduces a post-training DPO approach using epipolar geometry constraints, achieving a 31% reduction in Sampson error for enhanced 3D consistency.
• The methodology ranks and fine-tunes outputs from Wan-2.1 using SIFT, RANSAC, and a temporal variation penalty to balance geometric adherence with motion diversity.
• Experimental results demonstrate improved motion stability and 3D reconstruction fidelity, with significant gains in both static and dynamic scene evaluations.

Epipolar Geometry Constraints for 3D-Consistent Video Generation

Introduction

The paper "Epipolar Geometry Improves Video Generation Models" (2510.21615) addresses a persistent challenge in video diffusion models: the lack of geometric consistency in generated sequences, manifesting as artifacts, unstable motion, and perspective errors. Despite extensive training on large-scale, 3D-consistent datasets, state-of-the-art models frequently violate fundamental geometric principles, undermining their utility for downstream tasks such as 3D reconstruction and novel view synthesis. The authors propose a post-training alignment strategy that leverages classical epipolar geometry constraints as preference signals within a Direct Preference Optimization (DPO) framework, demonstrating that mathematically grounded metrics yield more stable and effective optimization than learned or subjective reward models.

Methodology: Epipolar Geometry Optimization

The core contribution is a pipeline that aligns pretrained video diffusion models (specifically Wan-2.1) to generate 3D-consistent scenes by enforcing epipolar constraints. The approach consists of three stages: (1) generating diverse video samples for each prompt, (2) ranking these samples using the Sampson epipolar error, and (3) finetuning the model via DPO to prefer geometrically consistent outputs.

Figure 1: Epipolar Geometry Optimization pipeline: diverse video generation, geometric ranking via Sampson error, and DPO-based policy finetuning.

The Sampson error quantifies the degree to which point correspondences between frames satisfy the epipolar constraint $\mathbf{x}'^T\mathbf{F}\mathbf{x} = 0$, where $\mathbf{F}$ is the fundamental matrix estimated via SIFT feature matching and RANSAC. Lower error values indicate better adherence to projective geometry, serving as a reliable proxy for 3D consistency. The DPO framework is well-suited for this setting, as it requires only relative rankings rather than absolute reward values, circumventing the non-differentiability of classical geometric algorithms.

To prevent degenerate solutions where the model suppresses motion to trivially satisfy geometric constraints, a temporal variation penalty is introduced, encouraging dynamic yet consistent camera trajectories.

Implementation Details

The alignment is performed on Wan-2.1 (1.3B parameters) using LoRA adapters (rank 64, $\alpha=128$), with training conducted on static scenes featuring dynamic cameras. Prompts are sourced from DL3DV and RealEstate10K, expanded using Gemma-3 VLM to increase geometric complexity. Rigorous data filtering ensures that only pairs with meaningful differences in 3D consistency are used for training. The finetuning process is computationally efficient, requiring 1,980 GPU hours for dataset generation and two days for model training on 4 A6000 GPUs.

Experimental Results

Geometric Consistency and Visual Quality

The aligned model exhibits substantial improvements in 3D consistency, as evidenced by a 31% reduction in Sampson error (from 0.190 to 0.131) and increased human-labeled consistency rate (from 54.1% to 71.8%). 3D reconstruction fidelity, measured via Gaussian Splatting, also improves (PSNR +3.6%, SSIM +3.2%, LPIPS -8.2%), confirming that the gains are not merely superficial but translate to enhanced spatial structure.

Figure 2: Baseline vs. aligned model outputs: geometric artifacts and unnatural motion are mitigated, yielding smoother, more consistent camera trajectories.

Figure 3: Qualitative comparison: finetuned model reduces artifacts and improves motion smoothness in both text-to-video and image-to-video settings.

Motion Stability and Generalization

Motion quality metrics (VBench, VideoReward) indicate improved smoothness and reduced flickering, with a tradeoff in dynamic degree (motion amplitude). Human evaluators strongly prefer the aligned model in cases with initial inconsistency (60.4% vs. 7.5% win rate). Notably, the model generalizes beyond its static scene training domain, maintaining performance on benchmarks with dynamic objects (VBench 2.0, MiraData, VideoReward), suggesting that enforcing geometric constraints on camera motion benefits overall video quality even in the presence of object motion.

Figure 4: Dynamic scene evaluation: epipolar-aligned model maintains geometric consistency and smooth trajectories in videos with both camera and object motion.

Ablation and Scaling Analysis

Ablation studies reveal that classical descriptors (SIFT) and metrics (Sampson error) provide cleaner optimization signals than sophisticated learned alternatives, which can be gamed or produce noisy preferences. The temporal variation penalty is essential for preserving motion diversity. Compared to other alignment strategies (SFT, Flow-RWR, DRO), the DPO-based epipolar alignment achieves the highest win rates and lowest geometric error. Scaling analysis shows that geometric alignment partially compensates for model size limitations, narrowing the gap to much larger models (Wan-2.1-14B) with significantly lower computational cost.

Implications and Future Directions

This work demonstrates that integrating classical computer vision principles into the alignment of generative models yields tangible improvements in 3D consistency, visual quality, and downstream utility. The approach is practical, requiring only relative rankings and avoiding the need for differentiable reward models or explicit 3D supervision. The release of a large-scale preference dataset annotated with geometric metrics further enables research in geometry-aware video generation.

The findings suggest several avenues for future research: extending geometric alignment to scenes with dynamic objects by disentangling camera and object motion, integrating additional physical constraints (e.g., photometric or physical soundness), and exploring hybrid reward models that combine classical and learned metrics. The demonstrated generalization and efficiency of the method position it as a promising direction for enhancing the physical realism and utility of generative video models in applications such as robotics, simulation, and 3D vision.

Conclusion

The paper establishes that classical epipolar geometry constraints, when used as preference signals in DPO-based alignment, significantly improve the 3D consistency and visual quality of video diffusion models. The approach is robust, generalizes beyond its training domain, and is computationally efficient. By bridging data-driven deep learning with mathematically principled computer vision, the work advances the state of geometry-aware video generation and sets a foundation for further integration of physical world understanding into generative AI systems.