PanoWorld-X: Controllable 360° Panoramic Worlds

• PanoWorld-X is a framework for generating explorable 360° panoramic worlds with explicit camera-route control and sphere-aware diffusion mechanisms.
• It integrates a large-scale synthetic dataset with detailed exploration route pairs to enable precise camera motion and trajectory-conditioned generation.
• The approach leverages sphere-aware attention to ensure geometric consistency and improved visual fidelity, outperforming prior panoramic video methods.

to=arxiv_search 大发快三彩票json {"query":"(Yin et al., 29 Sep 2025) PanoWorld-X Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion", "max_results": 5} to=arxiv_search 彩彩票娱乐json {"query":"PanoWorld-X panoramic video generation sphere-aware diffusion transformer explorable 360 route-conditioned dataset", "max_results": 10} PanoWorld-X is a framework for generating explorable 360-degree panoramic worlds under explicit camera-route control. It is motivated by a specific limitation in prior video generation systems: perspective-video models are optimized for limited field of view, while earlier panoramic generators still exhibit weak motion range, imprecise camera controllability, and a poor fit to the spherical geometry of equirectangular panoramas. PanoWorld-X addresses these issues jointly by combining a large-scale synthetic dataset of panoramic video–exploration route pairs with a sphere-aware diffusion transformer built on a pretrained video diffusion backbone (Yin et al., 29 Sep 2025).

1. Problem formulation and scope

The central problem addressed by PanoWorld-X is not merely panoramic image synthesis, but controllable generation of a complete environment that a user or autonomous agent can move through freely. The paper frames this as a failure mode of both dominant families of prior methods. Perspective-video generators cover only a narrow field of view, which hinders synthesis of a continuous and holistic scene. Earlier panoramic video models alleviate the field-of-view issue, but remain limited in three ways: weak motion range, lack of precise controllability over virtual camera motion, and neglect of the special spherical geometry of equirectangular projection, which degrades continuity and visual fidelity (Yin et al., 29 Sep 2025).

A common misconception is that panoramic generation alone is sufficient for exploration. The PanoWorld-X formulation rejects that assumption. In its design, exploration requires explicit route following, geometric consistency across time, and preservation of a pretrained visual prior. The framework is therefore organized around three coupled requirements: route-conditioned data, route-aware controllability, and geometry-aware attention.

The route signal is represented as a 6-DoF camera sequence,

$$Er_o^T = (x_0^T, y_0^T, z_0^T, \alpha_0^T, \beta_0^T, \gamma_0^T),$$

where $(x_i, y_i, z_i)$ are camera position coordinates and $(\alpha_i, \beta_i, \gamma_i)$ are yaw, pitch, and roll. This representation makes explicit that the target is not generic motion realism, but trajectory-conditioned panoramic world generation.

2. PanoExplorer dataset

A key contribution is the PanoExplorer dataset, constructed in Unreal Engine to provide panoramic video–exploration route pairs at scale. The dataset uses 504 high-fidelity 3D scenes spanning indoor and outdoor environments with varied lighting and weather. For each scene, walkable surfaces such as floors and roads are first extracted, and Delaunay triangulation is then used to create non-overlapping navigation meshes from sparse points (Yin et al., 29 Sep 2025).

Candidate routes are sampled by randomly choosing two mesh points, connecting them via Dijkstra shortest paths, and smoothing the resulting polyline with Laplacian smoothing so that the motion is visually coherent rather than jagged. Only trajectories longer than 18 meters are retained. Because many sampled paths can intersect scene geometry or clip through objects, a collision-detection filter is applied using bounding-box proxies for objects; any path with intersections is discarded.

The dataset construction also normalizes physical movement scale. Depth is extracted from Unreal Engine, and the distance between adjacent frames is fixed to 10 cm. This makes the control signal more consistent across small indoor scenes and large outdoor ones. The significance of this step becomes clear in the ablation study, where removing position normalization substantially worsens both fidelity and control metrics.

After trajectory rendering, the videos undergo a two-stage quality pipeline. Video-LLaMA3 automatically scores videos for semantic richness, motion richness, and quality, and also generates captions. The first frame of each video is then manually checked for rendering defects or missing details. The resulting curated set contains 116,759 high-quality panoramic video sequences paired with 3D exploration routes.

This dataset is structurally important to the method. The paper’s broader claim is that panoramic world generation had been limited not only by model design but also by the scarcity of route-conditioned panoramic data. PanoExplorer is the data-side mechanism used to address that bottleneck.

3. Exploration-aware route conditioning

The model replaces the standard DiT blocks of a pretrained video diffusion model with an “Explorable Sphere-Aware DiT Block.” At the backbone level, the generator follows a standard latent video diffusion formulation: a 3D-VAE encodes videos into latent tokens, text is encoded separately, and a Transformer denoiser operates with self-attention over spatiotemporal tokens. Standard attention is written as

$$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$

where $d$ is the head dimension (Yin et al., 29 Sep 2025).

PanoWorld-X keeps the pretrained global attention branch frozen for stability and efficiency, then adds an Exploration-Aware Attention branch for route conditioning. This branch is inspired by a ControlNet-style design. Rather than encoding the route as a single global vector, the 6-DoF trajectory is converted into pixel-wise Plücker embeddings so that route information is injected at token level.

The paper describes the camera pose using translation $tr_0^T$, rotation $R_0^T$, and camera intrinsics $K$. For a pixel $(u,v)$ at time $t$, the ray direction is

$(x_i, y_i, z_i)$0

and the Plücker embedding is formed as $(x_i, y_i, z_i)$1. Over the full video, the exploration route is represented as

$(x_i, y_i, z_i)$2

which encodes spatiotemporal motion information at pixel granularity.

This route embedding is compressed by several 3D convolution layers to match the shape of the DiT latent, concatenated with video latents, and processed by an Exploration-Aware Attention module initialized with the same parameters as the original block. A zero-initialized linear projection ensures that the new branch initially has no effect on generation. The paper presents this as a stabilization device: it prevents abrupt degradation of the pretrained prior during finetuning. The outputs of the controllable branch and the original branch are then added element-wise.

The broader technical point is that PanoWorld-X conditions on how each pixel ray evolves under intended camera motion, rather than conditioning only on abstract motion labels. This grounding is central to the claim that the generated panorama should move correctly along a specified exploration path.

4. Sphere-aware geometric reasoning

The second major innovation is the explicit treatment of spherical geometry. The paper argues that standard video models rely on Euclidean adjacency, which is inappropriate for ERP panoramas because pixels that are distant in the 2D raster may be adjacent on the sphere. The seam between the left and right edges of a panorama, and the behavior of polar regions, are given as classic examples of this mismatch (Yin et al., 29 Sep 2025).

To model the panorama on its native domain, ERP coordinates $(x_i, y_i, z_i)$3 are converted into spherical coordinates:

$(x_i, y_i, z_i)$4

where $(x_i, y_i, z_i)$5 is longitude and $(x_i, y_i, z_i)$6 is latitude. Great-circle distance is then computed with the Haversine formula. This provides a notion of adjacency defined on the sphere rather than on the planar raster.

The method also treats temporal motion as rotation of the sphere. A point on the sphere at time $(x_i, y_i, z_i)$7 is rotated relative to the initial orientation by Euler angles:

$(x_i, y_i, z_i)$8

Distances between points across time are measured after this rotation. A binary mask

$(x_i, y_i, z_i)$9

is then used so that only tokens within a spherical distance threshold $(\alpha_i, \beta_i, \gamma_i)$0 are emphasized as mutually relevant.

This Sphere-Aware Attention branch also runs in parallel with the original attention and is zero-initialized at the start. The design goal is not to replace the pretrained model’s global prior, but to correct its geometric bias where ERP topology differs from planar token layouts. The paper’s interpretation is direct: geometric coherence improves because the model no longer assumes that planar neighborhood structure is the correct topology for a panorama.

5. Training, inference, and evaluation protocol

The pretrained diffusion backbone is CogVideoX-5B-I2V. The model is fine-tuned to generate 49 frames. Because panoramas have a $(\alpha_i, \beta_i, \gamma_i)$1 aspect ratio, the training output is resized from $(\alpha_i, \beta_i, \gamma_i)$2 to $(\alpha_i, \beta_i, \gamma_i)$3 in order to preserve the pretrained model’s native priors as much as possible (Yin et al., 29 Sep 2025).

Training runs on 8 A100 GPUs for 8,000 iterations for the controllable branch, followed by an additional 2,000 iterations for the sphere-aware attention block. The staged optimization mirrors the architectural philosophy: route conditioning and spherical awareness are added as minimally perturbative parallel branches rather than through wholesale retraining of the original attention mechanism.

At inference time, a panorama image is first generated from text using FLUX with the panorama LoRA from LayerPano3D. That initial panorama image, together with the desired action signals, is then fed into PanoWorld-X to produce the video. The framework therefore separates initialization of the panoramic scene from route-conditioned video rollout.

Evaluation is performed on 200 randomly selected panoramic videos from the curated dataset. The metrics cover three groups of properties. Pixel-level quality is measured by PSNR, SSIM, and LPIPS. Visual realism and temporal coherence are measured by FID and FVD. Camera controllability is measured by Rotation Error $(\alpha_i, \beta_i, \gamma_i)$4 and Translation Error $(\alpha_i, \beta_i, \gamma_i)$5, following CameraCtrl, by comparing predicted camera extrinsics against ground truth. For varying output lengths, the errors are averaged rather than summed.

6. Quantitative results, ablations, and limitations

The reported results indicate that the full model outperforms the panoramic baselines 360DVD, Imagine360, and GenEX across all reported visual metrics. The PanoWorld-X full model achieves PSNR 19.34, SSIM 0.63, LPIPS 0.24, FID 28.01, and FVD 467.18. GenEX, the strongest named panoramic comparison in the table, reports 16.12, 0.59, 0.42, 42.22, and 1113.72 respectively, while 360DVD and Imagine360 are reported as substantially worse (Yin et al., 29 Sep 2025).

For controllability, the paper evaluates perspective-cropped outputs against CameraCtrl and AC3D. The “PanoWorld-X Perspective” variant yields PSNR 16.76, SSIM 0.56, LPIPS 0.42, FID 38.63, FVD 586.51, $(\alpha_i, \beta_i, \gamma_i)$6, and $(\alpha_i, \beta_i, \gamma_i)$7. These control errors outperform CameraCtrl, reported as $(\alpha_i, \beta_i, \gamma_i)$8, and AC3D, reported as $(\alpha_i, \beta_i, \gamma_i)$9, while also improving image and video quality.

The ablation study is structurally revealing. Removing position normalization degrades performance to PSNR 17.11, SSIM 0.55, LPIPS 0.32, FID 40.37, FVD 751.18, $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$0, and $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$1. Removing the controllable branch yields $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$2 and $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$3, which the paper interprets as confirmation that the route-conditioned branch is necessary for precise exploration. Removing Sphere-Aware Attention reduces performance to PSNR 17.59, SSIM 0.56, LPIPS 0.27, FID 29.96, FVD 492.98, $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$4, and $$Attn(Q,K,V)=\operatorname{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$5. The full model is best overall on every metric.

The qualitative interpretation offered by the paper is consistent with these numbers. Compared with panoramic baselines, PanoWorld-X keeps sharper textures and cleaner geometry during motion. Compared with camera-control baselines, it follows trajectories more faithfully while maintaining panoramic consistency.

The paper also states two explicit limitations. First, the model does not yet support long video generation, so exploration capability is limited in duration. Second, the current input is restricted to exploration routes; richer interactive inputs would be needed for more natural user-driven exploration. These limitations bound the scope of the method: it is a route-conditioned explorable panorama generator, not yet a general long-horizon interactive world model.

7. Position within panoramic world modeling

PanoWorld-X occupies a specific point in the emerging landscape of panoramic world models. Its emphasis is route-conditioned panoramic video generation with explicit camera control. Adjacent work has developed neighboring capabilities rather than duplicates of the same task: “PanoVGGT” addresses feed-forward 3D reconstruction from panoramic imagery with joint prediction of camera poses, depth maps, and 3D point clouds (Guo et al., 18 Mar 2026); “PanoWorld” treats whole-house synthesis as autoregressive generation of node-based panoramas with a floorplan-derived 3D shell and a dynamic 3D Gaussian Splatting cache (Jia et al., 18 May 2026); “Pano2World” converts a single indoor panorama into a persistent 3D Gaussian scene for free-viewpoint local navigation (Li et al., 1 Jul 2026); another “PanoWorld” targets long-range memory in real-world 360° video generation through Dense Panoramic Ray-Conditioning and Geometry-aware Memory Augmentation (Li et al., 10 Jul 2026); and a separate “PanoWorld” studies pano-native spatial understanding in multimodal large models with Spherical Spatial Cross-Attention (Wang et al., 13 May 2026).

This suggests a broader taxonomy of panoramic research. One line focuses on controllable panoramic video rollout, which is the role of PanoWorld-X. A second line addresses panoramic 3D reconstruction and persistent scene representation. A third line studies long-horizon memory and real-world rollout. A fourth line studies pano-native spatial understanding rather than synthesis. Within that landscape, PanoWorld-X is distinguished by the combination of a synthetic large-scale route-conditioned dataset, dense per-pixel Plücker route embeddings, and a sphere-aware attention mechanism designed specifically to correct the mismatch between flat token layouts and spherical panoramic topology.