PanoWorld: Panoramic World Modeling

• PanoWorld is defined as systems that use equirectangular panoramas as the primary representation for coherent spatial world modeling.
• It encompasses diverse methods, from geometry-consistent video synthesis to pano-native spatial supersensing and whole-house VR tour generation.
• These approaches leverage explicit geometric conditioning, spherical embeddings, and memory augmentation to preserve spatial structure and enhance immersive experiences.

PanoWorld is a term used in recent panoramic vision and generative modeling literature for systems that treat the equirectangular panorama as a primary representation for world modeling, rather than as a source image to be decomposed into perspective crops. In the arXiv record, the name spans several distinct formulations: a panoramic video world model that generates geometry-consistent $360^\circ$ video from a single image and a caption, a pano-native MLLM for spatial supersensing over ERP panoramas, a diffusion-based real-world panoramic generator with long-range memory, and a generative spatial world model for whole-house VR tour synthesis; an earlier use of the term referred more broadly to immersive environments constructed from 2D panoramas (Jiang et al., 14 May 2026, Wang et al., 13 May 2026, Li et al., 10 Jul 2026, Jia et al., 18 May 2026, Tikhonov et al., 2023).

1. Terminological scope and early usage

In the literature, “PanoWorld” does not denote a single canonical architecture. Instead, it names several panoramic systems with different inputs, objectives, and evaluation protocols. The common denominator is pano-native treatment of the $360^\circ \times 180^\circ$ field of view, usually in equirectangular projection (ERP), and an explicit attempt to preserve spatial structure that is typically obscured when panoramas are split into perspective crops.

Work	Problem setting	Core idea
"Unrolling Virtual Worlds for Immersive Experiences" (Tikhonov et al., 2023)	2D panorama to immersive environment	mathematically precise mapping, distortion correction, inpainting-like restoration
"PanoWorld: Geometry-Consistent Panoramic Video World Modeling" (Jiang et al., 14 May 2026)	single image and caption to $360^\circ$ video	depth consistency, trajectory consistency, spherical-aware adaptation, PanoGeo
"PanoWorld: Towards Spatial Supersensing in 360$^\circ$ Panorama World" (Wang et al., 13 May 2026)	pano-native MLLM reasoning	Spherical Spatial Cross-Attention, capability-aligned instruction tuning, PanoSpace-Bench
"PanoWorld: Real-World Panoramic Generation" (Li et al., 10 Jul 2026)	controllable panoramic video generation	Dense Panoramic Ray-Conditioning, Geometry-aware Memory Augmentation, World360
"PanoWorld: A Generative Spatial World Model for Consistent Whole-House Panorama Synthesis" (Jia et al., 18 May 2026)	whole-house VR tour synthesis	floorplan-derived 3D shell, dynamic 3D Gaussian Splatting cache, Room-aware Group Attention

An early conceptual use appears in "Unrolling Virtual Worlds for Immersive Experiences," where the “PanoWorld” denotes a class of immersive environments created by converting a 2D panoramic image into a navigable 3D setting through equirectangular projection, distortion correction, and inpainting-like restoration (Tikhonov et al., 2023). Later papers operationalize the term as named models, but the underlying tasks diverge sharply: video world modeling, panoramic reasoning, controllable generation, and node-based VR tour synthesis.

2. Geometry-consistent panoramic video world modeling

"Geometry-Consistent Panoramic Video World Modeling" defines PanoWorld as a panoramic video world model that generates geometry-consistent $360^\circ$ video from a single image and a caption. Its central claim is that panoramic video generation should be treated as a geometry- and dynamics-consistent latent state modeling problem rather than pure visual synthesis. The system builds on Cosmos Predict 2.5, described as a pre-trained perspective video “world model” based on a Diffusion Transformer with a VAE tokenizer, and adds two lightweight regularizers: a depth consistency loss against pseudo ground-truth panoramic depth and a trajectory consistency loss supervising the 3D world-frame positions of tracked points across time (Jiang et al., 14 May 2026).

The conditioning is multi-path. Semantic conditioning uses per-frame image features from SigLIP2 injected via cross-attention. Geometric conditioning projects observed perspective frames onto the equirectangular canvas and encodes them as reference latents together with a visibility mask. Sampling uses blended denoising to preserve observed input pixels. The spherical adaptation is explicit: latitude-aware positional embeddings compress positions near the poles, spherical area weighting scales losses by $w_h = \cos\phi(h)$, and circular shift augmentation imposes seamless wrap-around at the longitudinal seam. The latitudinal positional embedding is

$$\text{pos}_h = \frac{H-1}{2}\left(\sin\left(\frac{\pi h}{H-1} - \frac{\pi}{2}\right) + 1\right).$$

Training is framed as state completion: the model reconstructs full panoramic video from only a random perspective crop plus a text caption. Offline precomputation of depth and trajectory pseudo-labels enables efficient loss computation. The supporting dataset, PanoGeo, aggregates over 8,000 panoramic video clips from self-captured 4K panoramas, public datasets including x360 and 360DVD, and synthetic renderings with Habitat-Sim. Its unified annotation pipeline supplies automated captions, DAP panoramic depth, and dense CoTracker3 trajectories lifted to world-frame 3D points using depth and registered camera poses via the Umeyama algorithm. Evaluation uses a balanced 150-clip subset comprising 50 PanoGeo held-out clips, 50 Argus clips, and 50 Habitat-Sim clips (Jiang et al., 14 May 2026).

The reported outcome is improved geometric self-consistency without sacrificing appearance. In the main comparison, PanoWorld achieves 3D-Smooth $0.025$, Depth-$\sigma$ $0.013$, Tr-Life $360^\circ \times 180^\circ$0, and CLIP-T $360^\circ \times 180^\circ$1, while remaining best or second-best on visual realism metrics such as FVD, FID, and FAED. Ablations indicate that removing geometry losses degrades 3D-Smooth and Depth-$360^\circ \times 180^\circ$2, adding the depth loss improves global structure, and adding the trajectory loss improves temporal coherence of motion and also benefits depth. The stated conclusion is that both regularizers are synergistic and necessary for high geometric fidelity (Jiang et al., 14 May 2026).

3. Pano-native spatial supersensing

A distinct 2026 paper reuses the PanoWorld name for pano-native spatial understanding by multimodal LLMs. Here the target is not video synthesis but reasoning over an ERP panorama as a continuous, observer-centered space. The work defines four capability families for pano-native understanding: semantic anchoring, spherical localization, reference-frame transformation, and depth-aware 3D spatial reasoning. It constructs instruction-tuning data from a large-scale metadata pipeline over more than 570K panoramas drawn from mixed sources, with geometry-aware detection, language-grounded semantic annotation, depth-aware metadata, and metadata graph construction (Wang et al., 13 May 2026).

The metadata graph encodes entities as nodes $360^\circ \times 180^\circ$3 and pairwise relations as edges $360^\circ \times 180^\circ$4. On the model side, PanoWorld introduces Spherical Spatial Cross-Attention (SSCA). ERP patches are mapped to spherical coordinates

$360^\circ \times 180^\circ$5

then encoded as spherical tokens, and visual patch tokens retrieve geometric information through multi-head cross-attention. The fused representation is

$360^\circ \times 180^\circ$6

with $360^\circ \times 180^\circ$7 a learnable gate (Wang et al., 13 May 2026).

Evaluation centers on PanoSpace-Bench, a diagnostic benchmark for ERP-native spatial reasoning, and transfer tests on H* Bench and R2R-CE Val-Unseen. On PanoSpace-Bench, the paper reports overall accuracy $360^\circ \times 180^\circ$8 for PanoWorld versus $360^\circ \times 180^\circ$9 for a Qwen3.5 baseline, with especially large differences in absolute direction ($360^\circ$0 versus $360^\circ$1), BFOV mIoU ($360^\circ$2 versus $360^\circ$3), and ERP seam reasoning ($360^\circ$4 versus $360^\circ$5). On H* Bench, zero-shot PanoWorld reaches $360^\circ$6 overall compared with $360^\circ$7 for the best view-based baseline, and with H* task fine-tuning it reaches $360^\circ$8. On R2R-CE Val-Unseen, PanoWorld reports NE $360^\circ$9, OSR $^\circ$0, SR $^\circ$1, and SPL $^\circ$2, exceeding the best prior SR and SPL listed in the summary table (Wang et al., 13 May 2026).

This PanoWorld therefore occupies a different methodological niche from panoramic video synthesis. Its central claim is that robust panoramic reasoning requires dedicated pano-native supervision and geometry-aware model adaptation, rather than merely exposing an MLLM to multiple perspective crops from a panorama (Wang et al., 13 May 2026).

4. Real-world panoramic generation and long-range memory

"Real-World Panoramic Generation" uses the same name for a diffusion-based panoramic video generator whose main stated target is long-range memory in panoramic world models. The paper argues that omnidirectional representations are rotation-equivariant and exploits this by simplifying camera trajectories into translations via fixed headings. Two modules implement this design: Dense Panoramic Ray-Conditioning (DPRC) for current-action modeling and Geometry-aware Memory Augmentation (GMA) for long-range memory. The backbone is Wan2.2 (WAN2.2-5B) with LoRA-based parameter-efficient adaptation (Li et al., 10 Jul 2026).

DPRC works at ray level. Each ERP pixel $^\circ$3 is mapped to a spherical ray direction by

$^\circ$4

$^\circ$5

The translational term in the local transformation matrix encodes parallax in the ray’s local frame. GMA places both query and memory features in a shared PRoPE space and uses confidence-guided gating, with retrieval confidence derived from the maximum attention affinity and fusion defined as $^\circ$6 (Li et al., 10 Jul 2026).

Training is progressive. Stage 1 fine-tunes the panoramic video backbone with a latitude-aware reconstruction loss. Stage 2 learns translation-induced parallax via DPRC while LoRA weights are frozen. Stage 3 activates GMA for memory-anchored coherence. The evaluation dataset, World360, contains 120,000 high-quality panoramic video clips: 70,000 real-world UAV-collected clips and 50,000 AirSim360 simulated clips. Its curation pipeline applies rotation decoupling, spatial uniform resampling, and illumination filtering, and captions are produced with Qwen3-VL and filtered with Q-Align (Li et al., 10 Jul 2026).

On World360 at 480p, PanoWorld reports FID $^\circ$7, FID$^\circ$8 $^\circ$9, FID$360^\circ$0 $360^\circ$1, and PSNR$360^\circ$2 $360^\circ$3, compared with Matrix-3D at $360^\circ$4, $360^\circ$5, $360^\circ$6, and $360^\circ$7, and OmniRoam at $360^\circ$8, $360^\circ$9, $w_h = \cos\phi(h)$0, and $w_h = \cos\phi(h)$1. The paper also reports $w_h = \cos\phi(h)$2 PSNR on late windows, and states that a realtime variant using Causal Forcing distillation reduces generation time for a 161-frame video to 8 seconds on H20 GPU, versus 4:48 minutes for the full model (Li et al., 10 Jul 2026). In this usage, PanoWorld denotes a memory-augmented, trajectory-controlled panoramic generator rather than a general reasoning model.

5. Whole-house panorama synthesis

A further 2026 paper defines PanoWorld as a generative spatial world model for consistent whole-house panorama synthesis. The task is different again: generating a consistent whole-house VR tour from a floorplan and a style reference. The paper frames the problem as autoregressive generation of node-based $w_h = \cos\phi(h)$3 panoramas, explicitly matching the discrete navigation used by real VR tour products. The architecture separates global geometry from renderable appearance memory through a floorplan-derived 3D shell and a dynamic 3D Gaussian Splatting cache (Jia et al., 18 May 2026).

The floorplan is converted to a coarse 3D shell encoding room geometry, wall locations, and doorways. Camera nodes are organized in a graph over rooms and door boundaries. At each node $w_h = \cos\phi(h)$4, the system renders shell-derived geometric proxies $w_h = \cos\phi(h)$5, renders the existing cache as visual memory $w_h = \cos\phi(h)$6, and generates a new panorama $w_h = \cos\phi(h)$7, where $w_h = \cos\phi(h)$8 is a nearby panorama. A feed-forward panoramic LRM designed for metric-scale multi-room $w_h = \cos\phi(h)$9 inputs lifts the generated panorama into local 3DGS updates. Room-aware Group Attention suppresses cross-room feature interference, and a topology-aware progressive caching strategy fuses local updates without repeatedly reconstructing the full history (Jia et al., 18 May 2026).

The cache update is restricted to a context neighborhood

$$\text{pos}_h = \frac{H-1}{2}\left(\sin\left(\frac{\pi h}{H-1} - \frac{\pi}{2}\right) + 1\right).$$0

and memory filtering uses shell depth to invalidate cache pixels behind the first wall surface, expressed as

$$\text{pos}_h = \frac{H-1}{2}\left(\sin\left(\frac{\pi h}{H-1} - \frac{\pi}{2}\right) + 1\right).$$1

The panoramic LRM further uses CPRoPE for seam-consistent positional encoding and is trained with image, perceptual, opacity, and depth objectives, including log-scale depth and scale-invariant terms (Jia et al., 18 May 2026).

The reported motivation is that pure 2D generators re-imagine geometry and materials when the viewpoint changes, whereas monolithic 3D generation becomes expensive and loses fine texture at multi-room scale. By decoupling shell-based geometry guidance from cache-rendered visual memory, this PanoWorld is intended to preserve high-frequency 2D synthesis quality while improving cross-node layout and material consistency. The summary states that it achieves the highest Overlap PSNR among compared methods while maintaining competitive or superior per-node visual quality, and that removing the cache, nearby panorama conditioning, or CPRoPE significantly reduces cross-node consistency (Jia et al., 18 May 2026).

6. Position within panoramic world modeling

The various PanoWorld papers sit inside a broader research program on panoramic synthesis, reconstruction, and reasoning. "Moving in a 360 World" introduced OmniNeRF as the first method for parallax-enabled novel panoramic view synthesis from a single RGB-D panorama, using projection-based data augmentation and an omnidirectional NeRF representation (Hsu et al., 2021). "360-Degree Panorama Generation from Few Unregistered NFoV Images" introduced PanoDiff, combining a two-stage angle prediction module with latent diffusion, rotation equivariance loss, rotating denoising, and circular padding for controllable panorama generation from one or more unregistered NFoV images plus optional text prompts (Wang et al., 2023). "PanoWan" lifted a pre-trained text-to-video model to panoramic video with latitude-aware sampling, rotated semantic denoising, and padded pixel-wise decoding, supported by the PanoVid dataset (Xia et al., 28 May 2025).

At the 3D end of the spectrum, "Pano2World" converts a single indoor panorama into a persistent 3D Gaussian scene through a coarse Gaussian proxy, adaptively sampled nearby poses, a panoramic diffusion model with View-Aware Attention Routing, and a Latent Feature Adapter that distills multi-view hidden features into a scene latent (Li et al., 1 Jul 2026). "SphericalDreamer" instead generates fully immersive and long-range 3D outdoor environments from textual prompts by synthesizing multiple panoramas, lifting them into 3D, and fusing them with harmonic blending in transition regions (Schnepf et al., 19 May 2026). "PanoVGGT" addresses feed-forward 3D reconstruction from one or multiple panoramas via a permutation-equivariant Transformer with spherical-aware positional embeddings, panorama-specific three-axis $$\text{pos}_h = \frac{H-1}{2}\left(\sin\left(\frac{\pi h}{H-1} - \frac{\pi}{2}\right) + 1\right).$$2 augmentation, and stochastic anchoring (Guo et al., 18 Mar 2026).

A common simplification is to equate panoramic realism with world modeling. The surveyed PanoWorld works reject that equation in different ways. The geometry-consistent video model introduces explicit depth and trajectory regularization; the spatial supersensing model defines pano-native reasoning capabilities and benchmark tasks; the real-world generator centers long-range memory and trajectory fidelity; and the whole-house model formalizes multi-node coherence through topology-aware 3DGS caching (Jiang et al., 14 May 2026, Wang et al., 13 May 2026, Li et al., 10 Jul 2026, Jia et al., 18 May 2026). This suggests that, within current arXiv usage, “PanoWorld” is best understood not as a single method family but as a recurring label for panoramic systems that attempt to preserve the spherical scene as a coherent spatial state rather than a sequence of visually plausible but weakly grounded ERP frames.