PanFlow: Decoupled Panoramic Video Generation

• PanFlow is a panoramic video generation method that decouples spherical camera rotation from residual scene motion to simplify flow modeling.
• It employs spherical noise warping and circular padding to enforce loop consistency at panorama seams and poles, ensuring artifact-free video generation.
• Empirical results show improved performance metrics (e.g., FVD, EPE, SSIM) compared to prior methods, highlighting sharper textures and accurate motion control.

Searching arXiv for the PanFlow paper and a related base model mentioned in the provided data. PanFlow is a panoramic video generation method designed for motion-controllable synthesis in equirectangular $360^\circ$ video. It addresses two central difficulties in this setting: the entanglement of large camera rotations with residual scene motion, and the requirement of loop consistency at panorama seams and poles. The method decomposes the input optical flow into a rotational component induced by spherical camera motion and a derotated residual flow, conditions a diffusion model on the residual component, and restores the full motion by re-applying the accumulated rotation. It also introduces spherical noise warping to preserve motion continuity across panorama boundaries, and is trained on a motion-rich panoramic video dataset with frame-level pose and flow annotations. The approach is presented in "PanFlow: Decoupled Motion Control for Panoramic Video Generation" (Zhang et al., 30 Nov 2025).

1. Problem Setting and Motivation

Panoramic video generation differs from standard perspective video synthesis because the camera’s field of view wraps around and requires exact loop consistency at the left/right seam and at the poles. Realistic dynamics in this domain arise from a mixture of large camera rotations, camera translations, and object motions, rather than from local motion alone (Zhang et al., 30 Nov 2025).

The method is motivated by three limitations identified for prior diffusion-based panoramic image and video generators. First, loop inconsistencies occur when motion near the panorama boundary causes visible jumps as content reappears on the opposite side. Second, motion is entangled: perspective models treat optical flow, including both camera and object motion, as a single conditioning signal, so large camera rotations can overwhelm the denoiser and lead to blurred or misaligned motion. Third, explicit motion control is limited: methods such as 360DVD or DynamicScaler may accept a coarse panning directive but cannot faithfully reproduce complex user-provided flow fields (Zhang et al., 30 Nov 2025).

PanFlow addresses these issues by decoupling analytically the camera-rotation component in the spherical domain from the residual motion, and by warping diffusion noise on the sphere so that the model respects motion loops across seams. This suggests a division of labor in which analytically tractable global rotation is handled outside the denoiser, while learned generation is concentrated on residual motion and appearance preservation.

2. Spherical Motion Decoupling

The method takes as input an equirectangular panorama $I_0$, a target optical flow sequence $\{\mathbf f_t\}$, and optionally a text prompt, and produces panorama frames $\{I_t\}$ that follow the specified motion, maintain loop consistency, and preserve photorealistic appearance (Zhang et al., 30 Nov 2025).

Its first stage is motion decoupling. A $360^\circ$ panorama is modeled as an equirectangular projection of the unit sphere, with pixel coordinates parameterized by latitude $$\phi \in \left[-\tfrac{\pi}{2}, \tfrac{\pi}{2}\right]$$ and longitude $\theta \in [-\pi,\pi]$. A pure camera rotation $\mathbf R \in SO(3)$ induces a spherical rotation flow

$$\mathbf f_r(\theta,\phi)=\Pi\bigl(\mathbf R\,\mathbf x(\theta,\phi)\bigr)-\Pi\bigl(\mathbf x(\theta,\phi)\bigr),$$

where $\Pi(\cdot)$ reprojects a 3D direction onto image coordinates. In implementation, this is expressed analytically on the sphere as

$I_0$0

followed by conversion of the 3D offset to a 2D displacement in image space via the equirectangular Jacobian (Zhang et al., 30 Nov 2025).

Given an observed or user-provided optical flow field $I_0$1, the method extracts the rotation matrix $I_0$2, for example from SLAM poses or by accumulating differential rotations, and computes the derotated residual flow as

$I_0$3

Conditioning the diffusion model on $I_0$4 instead of $I_0$5 removes large content-agnostic rotational motion from the network’s input. The paper states that this reduces the network’s motion modeling burden to translation plus object motion only (Zhang et al., 30 Nov 2025).

After denoising, full motion is recovered by re-rotating the generated derotated frame $I_0$6 according to the cumulative rotation $I_0$7:

$I_0$8

This restores the original camera dynamics after residual motion has been synthesized in the derotated frame.

3. Spherical Noise Warping and Loop Consistency

The second major component is spherical noise warping. Rather than embedding the full flow field directly, PanFlow warps Gaussian latent noise according to the derotated flow and uses the warped noise as the motion condition for diffusion. For panoramas, this operation must propagate noise coherently across the left/right seam and at the poles (Zhang et al., 30 Nov 2025).

Let $I_0$9 denote the previous latent noise map, and let $\{\mathbf f_t\}$0 denote the derotated 2D flow at the latent resolution. The next noise map is defined by

$\{\mathbf f_t\}$1

where the wrapped coordinates are

$\{\mathbf f_t\}$2

This operation folds coordinates that move beyond the top or bottom edges back by reflection and wraps horizontal coordinates by modulo. In the terminology of the paper, this enforces pole consistency and seam consistency directly in the latent motion-conditioning process (Zhang et al., 30 Nov 2025).

The significance of this construction is that loop consistency is not treated purely as a post hoc correction. Instead, the diffusion transformer is trained on latents that already satisfy spherical boundary conditions, so the generator learns motion trajectories that close across seams. A plausible implication is that this makes panoramic continuity a property of the learned generation process rather than an after-the-fact stitching heuristic.

4. Architecture and Objective Functions

PanFlow builds on CogVideoX-5B-I2V, described in the paper as an image-conditioned video diffusion transformer. Its structure includes a pretrained 3D-VAE with encoder $\{\mathbf f_t\}$3 and decoder $\{\mathbf f_t\}$4, mapping each frame to a latent grid of spatial tokens with $\{\mathbf f_t\}$5 and temporal depth of 49 frames, and a transformer-based denoiser $\{\mathbf f_t\}$6 that receives the noisy latent sequence $\{\mathbf f_t\}$7, warped-noise condition $\{\mathbf f_t\}$8, optional text embeddings from CLIP, and optional image embeddings from the first frame (Zhang et al., 30 Nov 2025).

Two modifications are introduced to adapt the model to panoramas. First, LoRA adapters of rank 128 are inserted on all self- and cross-attention layers, enabling efficient fine-tuning in 2 days on eight A100s without destroying the pretrained weights. Second, circular padding is applied inside $\{\mathbf f_t\}$9 and $\{I_t\}$0: every 2D convolution over the $\{I_t\}$1 dimension is padded by wrapping columns from the opposite side, which guarantees pixel-space continuity at the seam (Zhang et al., 30 Nov 2025).

The training procedure also includes latent rotation. During each diffusion step, the entire latent grid is randomly rotated by $\{I_t\}$2 longitudinally before tokenization and rotated back afterward. The stated purpose is to expose the network to seams at different horizontal positions and reduce seam artifacts (Zhang et al., 30 Nov 2025).

Training uses the standard denoising diffusion objective together with seam and pole consistency regularizers. The denoising term is

$\{I_t\}$3

The seam-consistency loss is

$\{I_t\}$4

and the pole-consistency loss is

$\{I_t\}$5

The total loss is

$\{I_t\}$6

with $\{I_t\}$7 in practice (Zhang et al., 30 Nov 2025).

5. Dataset Curation and Training Data

To support training, the work curates a large-scale panoramic video dataset from 360-1M. The final dataset contains 150 000 clips and is explicitly described as motion-rich, with frame-level camera poses $\{I_t\}$8 and optical flows $\{I_t\}$9 (Zhang et al., 30 Nov 2025).

The curation pipeline consists of several filtering and preprocessing stages:

Step	Procedure	Purpose stated in the data
1	Format check via SSIM across left/right halves and circular masks	Remove stereo and fisheye clips
2	Scene segmentation using PySceneDetect + OpenVSLAM tracking-lost signals	Cut soft fades
3	Pose extraction with OpenVSLAM	Obtain frame-level $360^\circ$0
4	Watermark detection with LAION-5B-WatermarkDetection	Drop high-watermark clips
5	Flow filtering via Farneback optical flow	Discard low-magnitude clips $360^\circ$1 pixels)

The resulting clips have lengths of 3–10 s (Zhang et al., 30 Nov 2025). This training corpus is central to the method because the proposed decomposition requires frame-level pose and flow information. A plausible implication is that the dataset is not only a source of supervision for synthesis quality, but also a structural prerequisite for learning motion-conditioned panoramic generation under explicit rotational decoupling.

6. Empirical Performance and Applications

The reported evaluation uses a held-out 100-clip test set from 360-1M and a 100-clip subset of WEB360. PanFlow is compared against 360DVD, DynamicScaler, HoloTime, MotionClone, and Go-With-The-Flow on Fréchet Video Distance, Fréchet Image Distance, CLIP-similarity to prompt, optical-flow EPE to ground truth, LPIPS, PSNR, SSIM, and Q-Align human-style aesthetic and video quality (Zhang et al., 30 Nov 2025).

On 360-1M, the paper gives the example that PanFlow achieves FVD = 298 versus 477 for Go-With-The-Flow, EPE = 2.01 versus 3.30, and SSIM = 0.723 versus 0.622 (Zhang et al., 30 Nov 2025). Qualitatively, the method is reported to produce sharper textures on ground, sky, and architectural details; perfect seam closure without jumps at $360^\circ$2; accurate large panning, including $360^\circ$3 panning in 3 s; and better geometry, with buildings and horizons remaining straight under roll (Zhang et al., 30 Nov 2025).

Ablation studies isolate the contribution of the main components. Removing decoupling raises EPE by approximately $360^\circ$4. Disabling spherical warping doubles the seam MSE. Disabling LoRA fine-tuning raises FID from 23 to 40 (Zhang et al., 30 Nov 2025). These results support the paper’s claim that motion decoupling, spherical warping, and panoramic adaptation of the backbone each contribute materially to performance.

The paper also presents two application settings. In motion transfer, a source panorama video provides $360^\circ$5, and a target static panorama $360^\circ$6 supplies appearance; the output preserves the appearance or genus of the target while following the exact motion of the source. In video editing, an edited first frame $360^\circ$7, created via text-driven inpainting or external image models, is combined with the original or specified flow sequence to generate an edited $360^\circ$8 video retaining the same motion pattern (Zhang et al., 30 Nov 2025).

7. Limitations, Interpretation, and Future Directions

The paper identifies three principal strengths: fine-grained control over arbitrary flow fields including large rotations, exact loop consistency enforced at training time through spherical noise warping and padding, and a simple analytic decoupling of rotation flow that reduces network complexity (Zhang et al., 30 Nov 2025).

It also states several limitations. The method relies on accurate rotation estimation, whether from SLAM or flow accumulation, and performance may degrade under very low parallax or textureless scenes. The current resolution is $360^\circ$9, and scaling to $$\phi \in \left[-\tfrac{\pi}{2}, \tfrac{\pi}{2}\right]$$0 is described as nontrivial (Zhang et al., 30 Nov 2025).

The future directions listed in the paper are tightly aligned with these limitations. They include learning a self-supervised derotation network to predict $$\phi \in \left[-\tfrac{\pi}{2}, \tfrac{\pi}{2}\right]$$1 directly from a single panorama and thereby remove the SLAM dependency; extending the method to higher resolution, specifically $$\phi \in \left[-\tfrac{\pi}{2}, \tfrac{\pi}{2}\right]$$2, and longer duration videos exceeding 10 s; and jointly modeling depth and optical flow for more accurate 3D-consistent editing (Zhang et al., 30 Nov 2025).

Within the broader landscape of panoramic generation, PanFlow can therefore be understood as a method that redefines motion control in spherical video as a partially analytic problem rather than a wholly learned one. This suggests a methodological trend toward hybrid systems in which geometric factors with known structure, such as camera rotation on $$\phi \in \left[-\tfrac{\pi}{2}, \tfrac{\pi}{2}\right]$$3, are separated from the residual components delegated to diffusion-based synthesis.