CamPVG: Panoramic Video Generation

• The paper introduces CamPVG, a diffusion-based framework that generates multi-frame 360° panoramic videos by leveraging spherical geometry and precise pose control.
• CamPVG employs a novel panoramic Plücker embedding that transforms pixel coordinates into spherical rays, addressing mismatches in traditional perspective models.
• The framework integrates spherical epipolar attention to constrain cross-view feature aggregation, resulting in improved metrics like PSNR, SSIM, and reduced hallucination.

Searching arXiv for CamPVG and directly related camera-controlled video generation papers. arxiv_search(query="CamPVG Camera-Controlled Panoramic Video Generation with Epipolar-Aware Diffusion", max_results=5) arxiv_search(query="camera-controlled video generation MotionCtrl CameraCtrl CamI2V DynamiCrafter", max_results=10) CamPVG is a camera-controlled panoramic video generation framework that synthesizes a multi-frame $360^\circ$ panoramic video from a conditional panoramic frame, typically the first frame, and a target camera trajectory represented by per-frame extrinsics. It is presented as the first diffusion-based framework specifically designed for precise pose-guided panoramic video generation, with the central claim that panoramic video synthesis requires geometry tailored to spherical and equirectangular imaging rather than direct reuse of perspective-camera machinery (Ji et al., 24 Sep 2025).

1. Problem formulation and geometric motivation

CamPVG addresses camera-controlled panoramic video generation under four simultaneous constraints: geometric consistency with the specified camera motion, temporal coherence, visual realism, and fidelity to panoramic image geometry. The setting differs from ordinary image-to-video generation because the output is not a perspective clip but a 16-frame panoramic sequence at $256 \times 512$, and the conditioning signal is not merely motion style or text but an explicit camera pose sequence (Ji et al., 24 Sep 2025).

The framework is motivated by the failure of perspective-domain camera-control methods such as MotionCtrl, CameraCtrl, CamCo, and CamI2V when transferred to equirectangular panoramas. Two mismatches are identified. The first is a pose-encoding mismatch: perspective methods assume pinhole intrinsics and standard ray parameterizations, whereas a panorama has no single perspective intrinsic matrix over the full image and each pixel corresponds to a direction on the sphere. The second is a geometric-correspondence mismatch: perspective epipolar geometry yields straight image-plane epipolar lines, but in equirectangular panoramas valid correspondences follow curved trajectories induced by spherical projection. Reported failure modes of naively adapted perspective methods include content loss, cross-view misalignment, frame-to-frame deformation, poor geometric consistency, and hallucinated content in complex scenes (Ji et al., 24 Sep 2025).

This geometric diagnosis is the conceptual core of CamPVG. The method does not treat panoramas as a mere data-domain variant of perspective video generation; it reformulates both pose conditioning and cross-view feature aggregation around spherical projection.

2. Panoramic pose encoding via panoramic Plücker embedding

CamPVG replaces perspective Plücker ray construction with a panoramic variant derived from spherical coordinates. For reference, the perspective formulation is recalled as

$$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$

For a panorama pixel $(u,v)$ in an image of size $H \times W$, CamPVG computes spherical coordinates as

$$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$

These are mapped to Cartesian direction components by

$$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$

Using camera extrinsics $\mathbf E = [\mathbf R, \mathbf t]$, the panoramic Plücker embedding is then

$$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$

where $$(\hat x_{(u,v)}, \hat y_{(u,v)}, \hat z_{(u,v)})^\top$$ is the normalized direction vector. The full trajectory tensor is

$256 \times 512$0

The representation is dense: every pixel in every frame receives a line representation $256 \times 512$1. This differs from perspective Plücker embeddings not by reparameterizing the extrinsics themselves, but by changing the ray-construction stage. Pixel coordinates are first mapped to spherical angles, then to 3D directions, and only then transformed by the extrinsics. The paper is explicit that extrinsics are not transformed into spherical coordinates; rather, spherical rays are constructed first and then rotated and translated in world coordinates (Ji et al., 24 Sep 2025).

Following CameraCtrl, the resulting tensor is passed through a trainable pose encoder with a linear projection layer and injected into the diffusion U-Net. The available text does not specify hidden dimensionality, exact normalization beyond normalized direction vectors, or precise U-Net insertion sites. What is explicit is that the pose encoding is intended to provide the model with a dense, panorama-aware description of camera geometry that perspective-centric encodings cannot express (Ji et al., 24 Sep 2025).

3. Spherical epipolar geometry and masked cross-view attention

CamPVG’s second major contribution is the spherical epipolar module, which reformulates cross-view correspondence under equirectangular projection. For views $256 \times 512$2 and $256 \times 512$3 with extrinsics $256 \times 512$4 and $256 \times 512$5, the relative transform is

$256 \times 512$6

A source-view ray direction $256 \times 512$7 is transformed to target-view coordinates as

$256 \times 512$8

and the source camera origin as seen in view $256 \times 512$9 is

$$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$0

The corresponding epipolar locus is defined by the plane $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$1 containing $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$2, $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$3, and $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$4. In a panorama, the epipolar line is the intersection of this plane with the unit sphere, subsequently mapped back to equirectangular image coordinates. The result is a curve rather than a straight line. The paper states the plane in target-view camera coordinates as

$$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$5

with $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$6, but the printed expressions for $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$7 are corrupted in the provided source. Likewise, the derivation of the auxiliary parameters $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$8 and $$\mathbf{d} = \mathbf{R} \left( \mathbf{K}^{-1} \left(u,v,1\right)^\top \right), \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}. \tag{1}$$9 is incomplete. The explicit panorama-space parametrization that is recoverable is

$(u,v)$0

This establishes the key operational fact: epipolar constraints in equirectangular images are curved and must be handled as such (Ji et al., 24 Sep 2025).

CamPVG converts these curves into binary attention masks by uniformly sampling $(u,v)$1 points $(u,v)$2 along each curve and measuring the minimum distance from a candidate pixel $(u,v)$3: $(u,v)$4 A pixel is treated as a valid correspondence when this distance is below a threshold equal to half the feature grid’s diagonal length. The resulting mask for query frame $(u,v)$5 is

$(u,v)$6

Attention is then restricted by this mask before temporal attention. For query features $(u,v)$7, with keys and values from all frames $(u,v)$8, the module is written as

$(u,v)$9

with the intended meaning that query-key similarities are masked by $H \times W$0 before softmax so that feature aggregation is concentrated on geometrically valid cross-view regions. The paper characterizes this as a fine-grained cross-view aggregation mechanism that complements the global pose awareness provided by the panoramic Plücker embedding (Ji et al., 24 Sep 2025).

4. Diffusion architecture, conditioning, and training pipeline

CamPVG is built on DynamiCrafter as a latent image-to-video diffusion backbone, with DynamiCrafter’s text conditioning removed. The standard denoising objective is

$H \times W$1

where $H \times W$2 is the input video, $H \times W$3 is the autoencoder encoder, $H \times W$4 is the clean latent sequence, $H \times W$5 is the noisy latent sequence at step $H \times W$6, $H \times W$7, $H \times W$8 is the denoiser, and $H \times W$9 denotes conditioning embeddings. In CamPVG, conditioning includes the encoded panoramic Plücker tensor, while cross-view consistency is imposed architecturally by spherical epipolar attention rather than by an explicit extra geometric loss term (Ji et al., 24 Sep 2025).

The model conditions on the input panoramic frame and the target camera trajectory. The framework figure described in the paper proceeds by transforming camera trajectories into panoramic Plücker embeddings, injecting pose features into the U-Net, computing spherical epipolar masks, applying spherical epipolar attention before temporal attention, and denoising latent video to produce the panoramic sequence. The base DynamiCrafter parameters are frozen; only the panoramic position encoder and the spherical epipolar module are trained (Ji et al., 24 Sep 2025).

Training data are constructed from 3D-FRONT. The pipeline is: construct camera trajectories within 3D-FRONT scenes, render cubemaps at trajectory positions, convert cubemaps into panoramas using equirectangular projection, and produce panoramic videos with exact camera poses. The dataset comprises 5,616 scenes, 40-frame sequences per trajectory, and randomly sampled 16-frame training clips at resolution $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$0. Following CamI2V, the paper uses randomized conditional frame selection as data augmentation, and ablations report this as critical for robustness and generalization (Ji et al., 24 Sep 2025).

Optimization uses Adam with learning rate $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$1, batch size 16, 300 epochs, and $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$2 NVIDIA A800 GPUs, with total training time about 4 days. At inference, a conditional panoramic frame and a novel camera trajectory are converted into panoramic Plücker embeddings and spherical epipolar masks, after which diffusion denoising generates a 16-frame panoramic video following the specified motion (Ji et al., 24 Sep 2025).

5. Evaluation protocol and empirical results

CamPVG is evaluated primarily on the rendered 3D-FRONT panoramic dataset, with additional real-world panoramic images used as conditional inputs in user studies. For fairness, MotionCtrl, CameraCtrl, and CamI2V are adapted and retrained on the same panoramic dataset and settings, all using DynamiCrafter as the base model; MotionCtrl is modified to keep only camera control and disable object motion components (Ji et al., 24 Sep 2025).

Because standard perspective pose-estimation tools such as COLMAP and GloMap are stated to be unsuitable for panoramas, camera-trajectory consistency is evaluated by comparing the fifth generated frame to the corresponding ground-truth frame using PSNR, SSIM, and LPIPS. Panoramic frame fidelity is measured by FAED, and overall video quality by FVD with VideoGPT and StyleGAN feature extractors, together with VBench scores for Aesthetic Quality, Subject Consistency, and Temporal Flickering. Evaluation is performed on 1,000 randomly sampled video clips (Ji et al., 24 Sep 2025).

Quantitatively, CamPVG improves over all adapted baselines on all reported metrics. The reported scores are LPIPS $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$3, SSIM $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$4, PSNR $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$5, FAED $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$6, FVD(VideoGPT) $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$7, FVD(StyleGAN) $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$8, VBench Aesthetic $$\phi = \frac{u}{W}\cdot 2\pi,\quad \theta =\frac{v}{H}\cdot \pi. \tag{2}$$9, Subject Consistency $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$0, and Temporal Flickering $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$1. Relative to the strongest baseline on each metric, the gains include $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$2 lower LPIPS than MotionCtrl, $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$3 higher SSIM than MotionCtrl, $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$4 dB higher PSNR than MotionCtrl, $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$5 lower FAED than CamI2V, $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$6 lower FVD(VideoGPT) than CamI2V, and $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$7 lower FVD(StyleGAN) than MotionCtrl (Ji et al., 24 Sep 2025).

User studies with 20 volunteers also favor CamPVG. In the primary study, it obtains 3.518 for camera consistency, 3.598 for condition consistency, and 3.548 for video quality, compared with next-best scores of 2.753, 2.763, and 2.770 respectively. In the real-world-input study, it achieves condition consistency 3.817 and video quality 3.783 (Ji et al., 24 Sep 2025).

The ablation study isolates three components: panoramic Plücker embedding, spherical epipolar module, and randomized conditional-frame selection. Removing the panoramic Plücker embedding degrades LPIPS from $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$8 to $$x_{\left(u,v\right)} =\cos(\theta) \cdot \sin(\phi),\; y_{\left(u,v\right)} =\sin(\theta),\; z_{\left(u,v\right)} =\cos(\theta) \cdot \cos(\phi). \tag{3}$$9, SSIM from $\mathbf E = [\mathbf R, \mathbf t]$0 to $\mathbf E = [\mathbf R, \mathbf t]$1, PSNR from $\mathbf E = [\mathbf R, \mathbf t]$2 to $\mathbf E = [\mathbf R, \mathbf t]$3, FAED from $\mathbf E = [\mathbf R, \mathbf t]$4 to $\mathbf E = [\mathbf R, \mathbf t]$5, and FVD(VideoGPT) from $\mathbf E = [\mathbf R, \mathbf t]$6 to $\mathbf E = [\mathbf R, \mathbf t]$7. Removing the spherical epipolar module yields smaller but still substantial drops, for example LPIPS $\mathbf E = [\mathbf R, \mathbf t]$8, SSIM $\mathbf E = [\mathbf R, \mathbf t]$9, and FVD(VideoGPT) $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$0. Removing randomized conditional-frame selection is particularly destructive, worsening LPIPS to $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$1 and FVD(VideoGPT) to $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$2, which the paper attributes to overfitting to a fixed frame order (Ji et al., 24 Sep 2025).

A further ablation studies the epipolar sampling density $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$3. The best reported setting is $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$4, with the full system’s headline scores. The interpretation given is that too few samples miss valid correspondences, whereas too many introduce irrelevant points or noise (Ji et al., 24 Sep 2025).

6. Significance, distinctions, and limitations

CamPVG’s technical significance lies in a two-level geometric reformulation: pose encoding is made panorama-aware through spherical ray construction, and cross-view attention is constrained by spherical epipolar curves rather than perspective epipolar lines. This supports the paper’s central thesis that panoramic video generation requires spherical geometry at both the conditioning and correspondence stages, not merely panoramic training data or a panoramic decoder (Ji et al., 24 Sep 2025).

The method is explicitly framed as the first camera-controlled panoramic video generation framework designed for precise camera poses. A plausible implication is that it establishes a distinct subproblem within controllable video generation: not generic viewpoint animation, but pose-faithful equirectangular video synthesis under exact camera trajectories. The reported qualitative gains in multi-room transitions, scene-detail preservation, reduced deformation, and reduced hallucination suggest that the benefits are most pronounced when camera motion induces large viewpoint change across a scene whose geometry must remain stable (Ji et al., 24 Sep 2025).

Several misconceptions can be excluded. CamPVG is unrelated to “Periodic Vibration Gaussian” (PVG), a dynamic urban scene reconstruction method based on Gaussian splatting rather than panoramic diffusion video generation (Chen et al., 2023). The shared substring “PVG” therefore does not indicate a shared model family. More broadly, CamPVG is not a camouflage-segmentation method or a graph-recognition backbone; its “Cam” prefix refers to camera control, and its technical lineage is latent video diffusion with geometry-aware conditioning (Ji et al., 24 Sep 2025).

The paper states three principal limitations: the limited availability of panoramic datasets with accurate camera poses, reduced performance in outdoor scenes, and the need for future work on better generalization and longer panoramic video generation. Since the training corpus is built from 3D-FRONT and the experiments are correspondingly indoor-heavy, the strongest empirical support is for indoor $$\mathbf{d} = \mathbf{R}\left(\hat{x}_{\left(u,v\right)},\hat{y}_{\left(u,v\right)},\hat{z}_{\left(u,v\right)}\right)^\top , \quad \mathbf{m} = \mathbf{t} \times \mathbf{d}, \tag{4}$$5 scene exploration under controlled trajectories. This suggests that the method’s current state is best understood as a geometry-specialized foundation for panoramic camera control rather than a solved formulation of general in-the-wild panoramic video generation (Ji et al., 24 Sep 2025).