World360: Panoramic World Modeling

Updated 14 July 2026

World360 is a paradigm for full-surround visual representation, combining immersive visualization with geometry-aware processing and robust world modeling.
It integrates real-world and simulated panoramic data, as seen in its benchmark of 120,000 sequences that assess physical consistency across diverse conditions.
Methodologies include ERP, tangent projections, and novel memory augmentations that decouple rotation from translation to enhance spatial and temporal fidelity.

World360 denotes a line of research centered on full-surround visual representation, understanding, generation, and navigation with $360^\circ \times 180^\circ$ sensing, and, in a narrower recent usage, a benchmark for panoramic world modeling under real-world physical variations. In the literature surveyed here, the term spans immersive visualization, viewport-aware video behavior modeling, object and geometry estimation on omnidirectional imagery, controllable panoramic generation, real-time roaming, and hybrid urban digital-twin systems. A recent benchmark explicitly named World360 comprises 120,000 high-quality sequences, unifying 70,000 curated real-world clips with 50,000 high-fidelity simulations, and was introduced to evaluate physical consistency under large-scale spatial variations and diverse illumination conditions (Li et al., 10 Jul 2026).

1. Historical emergence and conceptual scope

A foundational early use of 360° media in technical visualization appeared in astrophysical simulation rendering. A 2017 study introduced full-sky $4\pi$ -steradian video as a visualization mode in which the viewer is placed at a fixed point inside a simulation and freely chooses orientation, rather than watching a conventional external camera path (Russell, 2017). That work distinguished immersive spherical video from fully interactive VR: the camera position is fixed, but the line of sight is viewer-controlled. This distinction remains central across later literature.

In parallel, systems work quickly recognized that omnidirectional media is not merely a wider version of conventional video. A 2017 adaptive streaming system argued that wireless VR delivery is intrinsically wasteful when the entire sphere is transmitted at uniformly high quality even though a head-mounted display renders only a limited instantaneous viewport; it proposed tiled, view-aware delivery over a custom hexaface sphere and reported up to 72% bandwidth saving relative to a tiled-but-non-adaptive baseline (Hosseini et al., 2017). The implication was that full-surround media requires new representations, new runtime control logic, and new quality-allocation strategies.

Subsequent work broadened the scope from immersive viewing to full panoramic world understanding. Recent papers explicitly frame panoramic sensing as a basis for scene understanding, robotics, autonomous driving-like navigation, embodied perception, and world modeling, culminating in World360 as a benchmark for panoramic world models and in PanoWorld as a model designed around long-range memory, fixed-heading motion simplification, and geometry-aware memory augmentation (Li et al., 10 Jul 2026). This suggests that “World360” now refers not only to a media format, but to a research program organized around omnidirectional world state, trajectory control, and physical consistency.

2. Representational geometry and projection regimes

The dominant representation across the literature is the equirectangular projection (ERP). In the astrophysical visualization pipeline, the desired frame is a full-sky map with azimuth on the horizontal axis and polar angle on the vertical axis, with $x$ ranging from $0^\circ$ to $360^\circ$ , $y$ ranging from $0^\circ$ to $180^\circ$ , and a required 2:1 aspect ratio with square pixels (Russell, 2017). ERP is operationally convenient because it maps the full sphere to a single rectangle, but it introduces severe nonuniform distortion near the poles and a wrap-around discontinuity at the left/right boundary.

That geometric tension drives much of the field. For object detection in indoor panoramas, 360-Indoor introduced “bounding field-of-view” annotations parameterized as $(\phi,\theta,h,w)$ , where $\phi$ and $4\pi$ 0 are the latitude and longitude of the object’s tangent plane and $4\pi$ 1 are angular height and width (Chou et al., 2019). The point is that axis-aligned raster rectangles are not native to spherical images; annotations and models must respect angular extent and seam continuity.

Later perception models moved beyond ERP-only processing. Elite360M combines ERP with an icosahedron projection point representation (“ICOSAP”) to reduce distortion while retaining global context, and fuses the two through Bi-projection Bi-attention Fusion and Cross-task Collaboration for depth, normals, and semantics (Ai et al., 2024). Cross360 uses ERP for global continuity and tangent projection patches for less-distorted local structure, then aligns them through cross-attention across scales (Huang et al., 24 Jan 2026). Both approaches treat the core 360° problem as a trade-off between global continuity and local geometric fidelity.

World360 and PanoWorld extend this representational logic from single frames to trajectories. Their preprocessing explicitly decouples rotation from translation, and the model uses dense panoramic rays derived from ERP coordinates, with spherical lifting

$4\pi$ 2

followed by a unit direction embedding (Li et al., 10 Jul 2026). This geometry-aware framing is used to simplify fixed-heading camera motion into translation-dominant latent dynamics and to support memory retrieval in a ray-conditioned space. A plausible implication is that panoramic world models benefit when rotation is handled as an explicit geometric transform rather than as latent appearance change.

3. Benchmark and dataset ecosystem

The World360 benchmark emerged within a broader ecosystem of panoramic datasets that each isolate different aspects of the problem: viewport behavior, detection, multimodal understanding, depth, segmentation, tracking, continuous geometry, and long-horizon world modeling.

Resource	Focus	Scale
World360 (Li et al., 10 Jul 2026)	Panoramic world modeling under real-world physical variations	120,000 sequences; 70,000 real + 50,000 simulated
Holo360D (Ou et al., 24 Apr 2026)	Continuous panoramic 3D reconstruction	109,495 panoramas; 75 scenes
Leader360V (Zhang et al., 17 Jun 2025)	Instance segmentation and tracking in 360 video	10,180 videos; 198 classes
“360° in the Wild” (Park et al., 2024)	Depth prediction and view synthesis	25K images from 273 videos; about 11K image-depth pairs
360-Indoor (Chou et al., 2019)	Object detection in indoor panoramas	3,335 panoramas; 89,148 instances
360+x (Chen et al., 2024)	Multi-view, multi-modal scene understanding	232 data examples; 2,152 videos
“A Taxonomy and Dataset for 360° Videos” (Nasrabadi et al., 2019)	Viewport traces and viewing behavior	28 one-minute videos; 60 participants

The viewport-trace dataset introduced a taxonomy with five camera-motion classes—fixed, horizontal, vertical, rotational, and mixed—and three moving-target classes—none, single, and multiple—yielding 14 populated categories across 28 scene-cut-free videos (Nasrabadi et al., 2019). Its importance lies in making viewer agreement and exploration measurable rather than anecdotal; later World360-style systems inherit the idea that motion structure and object motion jointly shape attention.

Object and scene benchmarks extended this into dense recognition. 360-Indoor established a real-world indoor object detection benchmark with 37 categories and 89,148 BFoVs in $4\pi$ 3 ERP panoramas (Chou et al., 2019). 360+x shifted the emphasis to synchronized viewpoints and modalities—panoramic third-person, front-view, egocentric monocular and binocular video, audio, directional binaural delay, GPS, weather, and text—across 28 scene categories (Chen et al., 2024). “360° in the Wild” introduced real Internet panoramas with poses and pseudo-ground-truth depth from monocular reconstruction, thereby expanding beyond indoor scans to worldwide unconstrained content (Park et al., 2024).

For temporal dense prediction, Leader360V scaled 360 video segmentation and tracking to 10,180 real-world videos with 198 semantic classes, using an automatic labeling pipeline that combines 2D segmentors, LLMs, and SAM2-based refinement (Zhang et al., 17 Jun 2025). Holo360D addressed a different bottleneck: most panoramic geometry datasets provide only discrete viewpoints, whereas feed-forward 3D reconstruction models benefit from continuity. Holo360D therefore contributes continuous trajectories with average inter-frame distance $4\pi$ 4, alignment error $4\pi$ 5 px, and depth completeness $4\pi$ 6 indoor and $4\pi$ 7 outdoor (Ou et al., 24 Apr 2026).

World360 integrates several of these desiderata at once: ERP video, real trajectories, synthetic depth-bearing sequences, multi-altitude motion, 5 cm spatial resampling, text descriptions, and long clips of 81 and 161 frames (Li et al., 10 Jul 2026). Within the surveyed literature, it is the clearest attempt to turn panoramic video from a media format into a benchmark for physically grounded world modeling.

4. Core modeling paradigms

Panoramic perception methods increasingly reject direct transfer from perspective-image architectures. Elite360M demonstrates this in multi-task dense prediction: with ERP plus ICOSAP and cross-task fusion, it jointly predicts depth, normals, and semantics, and on Matterport3D with ResNet-34 achieves depth Abs Rel $4\pi$ 8, normal Mean Angular Error $4\pi$ 9, and semantic mIoU $x$ 0 while remaining far smaller than large generic multi-task transformers (Ai et al., 2024). Cross360 applies the same design logic specifically to monocular depth, using ERP–tangent cross-attention and progressive cross-scale aggregation; on Structured3D it reports Abs Rel $x$ 1, RMSE $x$ 2, and $x$ 3 (Huang et al., 24 Jan 2026).

Generative approaches divide into view interpolation, panorama completion, and panoramic video synthesis. See360 treats fixed-center panoramic exploration as conditional multi-scale affine feature warping between sparse perspective views, rather than explicit 3D reconstruction, and reports real-time-style inference around $x$ 4– $x$ 5 seconds per image across four datasets (Liu et al., 2024). Dream360 reframes panorama outpainting as spherical latent modeling from a narrow field-of-view image, combining SH-informed S-VQGAN codebooks with transformer generation and frequency-aware refinement; it reports outpainting FID $x$ 6, compared with $x$ 7 for Omnidreamer and $x$ 8 for SIG-SS (Ai et al., 2024). 360DVD adapts a pretrained text-to-video diffusion model to ERP panoramic video through a lightweight 360-Adapter, latitude-aware loss, latent rotation, and circular padding, and a user study favors it strongly over adapted AnimateDiff baselines on end continuity, content distribution, and motion pattern (Wang et al., 2024).

Single-image 360° view extrapolation appears in Disentangled360, which uses a Gaussian Splatting backbone with radiance decomposed into isotropic and anisotropic terms,

$x$ 9

and reports interactive rendering at 22.7 FPS and about 44 ms/frame at $0^\circ$ 0 on an RTX 3090 (KV et al., 11 Dec 2025). The paper explicitly targets mixed-reality medical supervision as well as natural scenes, although the literature it belongs to remains closer to 3D-aware novel-view synthesis than to complete world reconstruction.

At the level of world models, PanoWorld is distinctive because it treats panoramic rotation as an implicit geometric transformation, reduces derotated motion to translation, conditions generation on dense panoramic rays, and augments long-range memory with geometry-aware retrieval. On World360 it outperforms Imagine360, Matrix-3D, and OmniRoam by large margins in FID, FAED, and late-window PSNR, including FID $0^\circ$ 1 and FAED $0^\circ$ 2 at 720p (Li et al., 10 Jul 2026). Within the surveyed literature, this is the clearest formulation of “World360” as long-horizon panoramic world modeling rather than as isolated scene understanding or media generation.

A separate systems line focuses on using 360° media for interactive exploration rather than dense reconstruction. The earliest scientific example is the 2017 astrophysical pipeline, where a fixed internal viewpoint inside the Galactic-centre simulation is rendered as an all-sky column-density movie and distributed through standard platforms such as YouTube (Russell, 2017). The core contribution was not a new projection but a practical pipeline—modified Splash rendering, ffmpeg encoding, spherical metadata injection, and platform-compatible playback on desktop, mobile, and VR goggles.

For bandwidth-constrained immersive delivery, the MPEG-DASH SRD system partitions the sphere into six tiles via the hexaface sphere, prioritizes tiles inside the field of view, and reports up to 72% bandwidth saving with minor negative quality impacts relative to a tiled baseline without adaptation (Hosseini et al., 2017). This system already contains several principles that recur in later world-scale platforms: geometry-aware tiling, viewport-aware prioritization, and standards-oriented metadata.

Urban-scale systems go further by integrating geospatial structure. 360CityGML aligns 360° walkthrough videos with a CityGML urban model, dynamically projects selected video frames onto the geometry, and uses semantic building-region agreement for street-level trajectory alignment; across 229 Akihabara street videos, mean misalignment rate decreases from $0^\circ$ 3 to $0^\circ$ 4, a 39% relative reduction (Banno et al., 16 Oct 2025). Its runtime implementation in Unity achieves approximately 100 FPS on a MacBook M2 with 16 GB RAM, while supporting flood-risk overlays, daylight visualization, and attribute queries on LOD1 city models.

A more video-native urban system is 360RVW, which constructs a realistic virtual world directly from 360° street videos through intersection-based video segmentation, videographer removal, semantic collision metadata, and projection onto a distorted moving sphere that follows the avatar along estimated trajectories (Takenawa et al., 13 Oct 2025). It requires only 360° videos and start/end coordinates, uses semantic classes such as Ground, Road, Sidewalk, Parking, and Person as navigable regions, and compresses semantic maps for the entire Akihabara area to 1% of their initial size via run-length encoding. In a 19-participant perceptual study against Google Street View and MovieMap, the system improves spatial presence relative to GSV and is judged especially suitable for virtual tours.

These deployment-oriented systems suggest that a practical World360 platform does not always require full 3D reconstruction. Realistic exploration can also arise from video-native representations anchored by trajectories, sparse geometry, GIS metadata, and constrained interaction models.

6. Recurring limitations and open problems

Several limitations recur across the literature. First, immersive 360° video is often mistaken for fully interactive VR. The astrophysical simulation paper explicitly notes that its videos are not room-scale interactive VR: the viewpoint is fixed and only orientation changes (Russell, 2017). The same restriction appears in many navigation systems, where movement is constrained to recorded trajectories or fixed headings rather than full 6DoF free-space motion.

Second, ERP remains a convenience representation rather than a solved geometric substrate. Distortion near the poles, seam discontinuities, and uneven feature statistics motivate BFoV annotations, tangent patches, cubemaps, ICOSAP point sets, latitude-aware losses, and special seam treatments such as latent rotation and circular padding (Chou et al., 2019). This suggests that “World360” systems remain partly limited by the gap between spherical scene geometry and rectangular neural architectures.

Third, data abundance does not eliminate domain gap. “360° in the Wild” provides 25K real omnidirectional images but only about 11K usable depth pairs, with monocular-scale ambiguity and reconstruction failure in dynamic scenes (Park et al., 2024). Holo360D improves continuity and depth completeness, yet the paper still concludes that current 3D models are not panoramic-native and perform best when panoramas are decomposed into perspective subviews (Ou et al., 24 Apr 2026). This is a substantive caution against assuming that perspective pretraining plus panoramic finetuning is sufficient.

Fourth, generation from sparse observations remains hallucinatory in unseen regions. Dream360, See360, 360DVD, and Disentangled360 all rely on learned priors to synthesize large unobserved portions of the world, and Disentangled360 explicitly inherits single-image scale ambiguity and hidden-region uncertainty (KV et al., 11 Dec 2025). A plausible implication is that generative World360 systems and geometry-grounded World360 systems are still converging from opposite directions: one side prioritizes realism and diversity, the other physical correspondence and trajectory consistency.

Finally, the most recent benchmark literature identifies long-range memory as the outstanding problem. World360 was explicitly introduced because existing datasets are “relatively stable” and do not stress physical consistency under large spatial variations and diverse illumination (Li et al., 10 Jul 2026). PanoWorld’s gains are largest in late PSNR windows and in geometry-aware memory ablations, indicating that panoramic world modeling is now moving from local viewpoint synthesis toward persistent spatiotemporal state. The open problem is no longer merely how to render a 360° image, but how to preserve identity, geometry, and illumination across revisitation, altitude changes, and long trajectories in a full-surround representation.