OmniRoam: Panoramic Video Generation

• The paper introduces OmniRoam, a framework for generating long-horizon panoramic videos with enhanced spatial and temporal consistency.
• It employs a two-stage preview–refine pipeline, where a coarse panoramic preview is first generated and then refined to achieve higher resolution and smoother temporal density.
• Quantitative evaluations using metrics like FAED, SSIM, and LPIPS demonstrate the method's superiority over conventional perspective-based video generation approaches.

OmniRoam is a controllable panoramic video generation framework for long-horizon scene wandering, introduced in "OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation" (Liu et al., 31 Mar 2026). Given an input panoramic image or video and a user-defined camera trajectory, it generates a long $360^\circ$ video showing camera motion through the scene. The framework is organized as a two-stage preview–refine pipeline: a preview stage first produces a coarse panoramic overview of the scene, and a refine stage then temporally extends and spatially upsamples that preview into a longer, higher-resolution video. The underlying claim is that panoramic representation provides richer per-frame scene coverage and stronger long-term spatial and temporal consistency than perspective video for scene-level generation and loop-like world exploration (Liu et al., 31 Mar 2026).

1. Problem setting and conceptual scope

OmniRoam addresses a limitation of prior scene-generation and controllable video-generation systems that operate primarily on perspective videos. In that setting, each frame contains only a limited field of view, so global scene structure must be accumulated over time, which tends to exacerbate incompleteness, drift, and weak long-term consistency under large camera motion. OmniRoam instead treats panoramic video as the primary scene representation, with the explicit goal of enabling long-horizon wandering rather than short perspective synthesis (Liu et al., 31 Mar 2026).

The framework is positioned as a scene-modeling system rather than merely a view-synthesis module. Its input is a panoramic image or video together with a user-specified trajectory, and its output is a panoramic video that follows that trajectory while preserving continuity with the source. The preview stage generates an 81-frame video at $480 \times 960$ and accelerated playback speed; the refine stage produces a final output at $720 \times 1440$ and can reach 641 frames. This decomposition is motivated by the observation that direct one-shot generation of very long, high-resolution panoramic video would be both computationally expensive and more prone to inconsistency (Liu et al., 31 Mar 2026).

The broader significance of the design lies in its treatment of scene wandering as a global-to-local generation problem. The preview establishes global layout and motion, while the refine stage adds local detail, normal-speed temporal density, and higher spatial fidelity. This suggests a shift from purely autoregressive long-video generation toward scaffolded generation with an explicitly coarse global scene plan.

2. Representation, control space, and trajectory parameterization

OmniRoam operates on panoramic videos in ERP, or equirectangular projection, format. ERP is used because panoramic content can be represented as a 2D image while camera rotation corresponds largely to cyclic pixel shifts rather than the stronger perspective-induced visibility changes that dominate narrow-FoV video. On that basis, the framework introduces a canonical rotation-invariant control coordinate system that removes camera self-rotation and retains only translation relative to the panorama center. The controlled coordinates are therefore $(x,y,z)$, with no roll, pitch, or yaw in the motion control space (Liu et al., 31 Mar 2026).

Trajectory control is factorized into two orthogonal components: flow and scale. Scale is a scalar $s \in \mathbb{R}^+$ representing global displacement magnitude per timestep, encoded in log space as

$z_s = \phi(\log(s)).$

Flow is a sequence of unit 3D direction vectors

$$\mathcal{D} = \{\mathbf{d}_k\}_{k=1}^{f}, \qquad \|\mathbf{d}_k\| = 1,$$

encoded by a zero-initialized camera encoder as

$z_d = \mathcal{E}_c(\mathcal{D}).$

This decomposition is motivated by two assumptions enforced in the data protocol: uniform velocity and fixed orientation. Under those assumptions, direction can be handled framewise through flow, while overall motion magnitude is controlled globally through scale (Liu et al., 31 Mar 2026).

This parameterization simplifies the second-stage refinement problem. Because the preview already encodes scene evolution and directional motion, the refine stage only needs scale alignment rather than full trajectory-vector conditioning. A plausible implication is that OmniRoam uses factorization not only as a control interface but also as a means of separating global motion planning from local video densification.

3. Preview and refine stages

The preview model is fine-tuned from Wan2.1-1.3B and uses frame-dimension conditioning inspired by ReCamMaster. If the input image or video and target video are denoted by

$$V_{in} \in \mathbb{R}^{f \times 3 \times H \times W}, \qquad V_v \in \mathbb{R}^{f \times 3 \times H \times W},$$

they are encoded by a 3D VAE into latent tensors

$$z_{in} = \mathcal{E}_v(V_{in}), \qquad z_v = \mathcal{E}_v(V_v).$$

A key training detail is that noise is added only to the target latent, while the source latent remains clean. Under the rectified-flow formulation inherited from Wan, the interpolated latent is

$480 \times 960$0

where $480 \times 960$1. This makes the input clip or image an explicit visual anchor for continuity while trajectory conditions are injected through the scale and flow embeddings (Liu et al., 31 Mar 2026).

The refine stage takes the accelerated preview video

$480 \times 960$2

and produces temporally denser, higher-resolution segments that are concatenated into the final sequence,

$480 \times 960$3

If the preview runs at scale $480 \times 960$4 and the final video at target scale $480 \times 960$5, temporal expansion is determined by

$480 \times 960$6

In typical use, $480 \times 960$7 for normal-speed playback, so faster previews require more refined segments (Liu et al., 31 Mar 2026).

Refinement is conditioned through a visibility-mask mechanism. For each segment $480 \times 960$8, a binary mask $480 \times 960$9 selects a temporal window of preview frames; the downsampled latent mask is applied to the encoded preview as

$720 \times 1440$0

This provides sparse but strong anchors from the preview while leaving room for synthesis of intermediate temporal detail. The paper’s interpretation is that this localized masked conditioning is what allows simultaneous temporal extension, super-resolution, and preservation of long-range coherence without resorting to naive chunkwise autoregression (Liu et al., 31 Mar 2026).

4. Data construction and training regime

OmniRoam introduces two panoramic video datasets, combining real-world and synthetic material. The real-world dataset contains approximately 2,000 handheld panoramic videos and about 5 million frames, covering environments such as hotels, schools, and outdoor landscapes. These videos are gravity-aligned, their trajectories are estimated with COLMAP, and videos with abnormal scale are filtered so that motion scale remains roughly consistent. The synthetic dataset is rendered from 1,000 3D Gaussian Splatting scenes from InteriorGS (Liu et al., 31 Mar 2026).

The synthetic corpus is designed to provide precise trajectories and physically feasible motion. Valid cruising areas are defined, obstacles in the camera vertical range $720 \times 1440$1 are filtered, candidate waypoints are generated to cover 50% of free space, and constant-speed trajectories are used so that motion scale remains consistent. The hybrid composition is therefore intentionally asymmetric: real data contributes appearance diversity, texture, and lighting realism, whereas synthetic data contributes geometry and trajectory precision (Liu et al., 31 Mar 2026).

Training follows the two-stage structure. The preview model is trained at $720 \times 1440$2 and 81 frames. It is first trained on real data with random temporal slicing for 120k steps using batch size 64 and learning rate $720 \times 1440$3, then fine-tuned on synthetic data for 64k steps with batch size 64 and learning rate $720 \times 1440$4. The refine model takes accelerated 480p preview clips and produces $720 \times 1440$5 clips at $720 \times 1440$6 scale, also of length 81 per refined segment, trained for 120k steps with batch size 64 and learning rate $720 \times 1440$7, with real and synthetic data sampled with equal probability (Liu et al., 31 Mar 2026).

These choices indicate that OmniRoam depends not only on panoramic representation but also on a curated data regime in which camera motion is canonicalized. The paper explicitly notes fixed orientation and near-constant speed as assumptions, so the training pipeline is tightly coupled to that restricted control manifold.

5. Evaluation protocol and reported performance

Evaluation is organized around three axes: visual quality, trajectory controllability, and long-term scene consistency. The paper uses seven trajectories—forward, backward, left, right, s-curve, loop, and ground-truth—and notes that none of them is directly included during training. For each trajectory and method, 24 videos are generated on the test set. Visual quality is evaluated using FAED, SSIM, and LPIPS; trajectory controllability is measured with PSNR over selected temporal windows following CamPVG’s protocol; and long-term consistency is measured using a new Loop Consistency metric (Liu et al., 31 Mar 2026).

Loop Consistency is defined for a generated sequence $720 \times 1440$8 as

$720 \times 1440$9

where

$(x,y,z)$0

$(x,y,z)$1

with $(x,y,z)$2 and $(x,y,z)$3 given by CLIP embedding cosine similarity. Here $(x,y,z)$4 measures loop closure between the beginning and end of the sequence, while $(x,y,z)$5 penalizes trivial similarity between the beginning and the intermediate part of the video (Liu et al., 31 Mar 2026).

The main reported results are explicit. At 480p and 81 frames, OmniRoam reports FAED 5.27, SSIM 0.70, LPIPS 0.18, PSNR'25 23.06, PSNR'55 20.69, PSNR'75 19.87, and Loop Consistency 2.34. Under the same setting, Matrix-3D reports FAED 8.64, SSIM 0.63, LPIPS 0.37, PSNR'25 19.05, PSNR'55 17.35, PSNR'75 17.04, and Loop 1.38, while Imagine360 reports FAED 23.35, SSIM 0.33, and LPIPS 0.61. At 720p, OmniRoam reports 641 frames with FAED 5.07, SSIM 0.66, LPIPS 0.33, PSNR'25 19.75, PSNR'55 18.59, PSNR'75 18.24, and Loop 1.96; Matrix-3D at 720p reports 81 frames with FAED 9.76, SSIM 0.66, LPIPS 0.36, PSNR'25 19.06, PSNR'55 17.63, PSNR'75 17.12, and Loop 1.41 (Liu et al., 31 Mar 2026).

Ablations sharpen the interpretation. A perspective preview baseline reports FAED 16.90, SSIM 0.62, LPIPS 0.44, PSNR'25 17.72, PSNR'55 16.06, and Loop 1.70 for 81 frames; a perspective refine baseline reports FAED 15.48, SSIM 0.63, LPIPS 0.57, PSNR'25 16.60, PSNR'55 14.94, PSNR'615 13.76, PSNR'635 14.07, and Loop 1.42 for 641 frames. A direct autoregressive panoramic baseline at 641 frames reports FAED 16.04, SSIM 0.33, LPIPS 0.64, PSNR'25 15.44, PSNR'55 14.84, PSNR'615 10.14, PSNR'635 10.11, and Loop 0.89. This supports the paper’s argument that panoramic representation and preview–refine generation primarily improve long-term coherence by reducing error accumulation over long trajectories (Liu et al., 31 Mar 2026).

6. Extensions, uses, and limitations

OmniRoam is not presented solely as a generator of panoramic videos; it is also used as a scene representation for downstream tasks. One extension is a real-time previewer obtained through self-forcing distillation. The student autoregressive preview model is trained to match the teacher preview model, and the reported runtime is 81 frames in 7 seconds, compared with about 5 minutes for the original preview stage and about 11 minutes for Matrix-3D. Another extension is 3D reconstruction: a 641-frame generated panoramic wandering video is sampled at 100 intermediate frames, each panoramic frame is cropped into five perspective views with $(x,y,z)$6 FoV and resolution $(x,y,z)$7, and these crops are used for 3D Gaussian Splatting reconstruction (Liu et al., 31 Mar 2026).

These extensions clarify the intended role of the framework. OmniRoam-generated videos are meant to function as globally coherent traversals from which multi-view structure can be recovered, not merely as short-form visual animations. This suggests that the method sits at the boundary between controllable video generation and generative scene modeling.

The stated limitations are structural. The control design assumes fixed orientation and approximately uniform velocity. The original preview stage is computationally heavy, taking about 5 minutes for 81 frames before distillation. Refinement remains segment-wise rather than single-pass, and the approach depends on canonicalized panoramic data with filtered trajectories. The paper also indicates that qualitative generalization to panoramic images from the Internet exists, but does not present that as a guarantee for arbitrary scenes or highly complex trajectories (Liu et al., 31 Mar 2026).

Within the broader literature on controllable video generation, OmniRoam’s distinctive contribution is the combination of panoramic representation, decomposed trajectory control, and global-to-local generation. The paper’s results suggest that long-horizon scene wandering benefits from treating global scene consistency as a first-class modeling objective rather than as an emergent property of autoregressive continuation.