Panorama Generation Task

• Panorama generation is the process of synthesizing complete 360° images using techniques like diffusion models, transformer architectures, and geometric-aware modules.
• Modern frameworks ensure photorealistic, seamless stitching by employing methods such as cubemap handling, circular padding, and tile-based inpainting.
• Applications span VR, robotic navigation, and 3D scene reconstruction, with future research focusing on multi-modality, real-time scalability, and artifact suppression.

Panorama generation is a foundational task in computer vision, graphics, and robotics that encompasses the synthesis, reconstruction, or manipulation of complete 360°×180° (omnidirectional) images from various input modalities. Modern panorama generation frameworks address not only the synthesis of photorealistic and geometrically consistent spherical images but also related subproblems such as upright correction, multimodal guidance, multi-view stitching, and 3D scene generation. Recent progress is driven by innovations in generative modeling, spatial representation, and geometric consistency mechanisms, yielding methods that support high-resolution outputs for applications ranging from virtual reality (VR) to robotic navigation and photorealistic rendering.

1. Problem Definition and Representation Paradigms

Panorama generation seeks to synthesize seamless spherical images or representations that enable omnidirectional viewing. The dominant formats include equirectangular projection (ERP), cubemaps (CP), and tangent-plane tiles. Formally, the space of panoramas can be parameterized on the unit sphere by longitude $\lambda \in [0, 2\pi)$ and latitude $\phi \in [-\frac{\pi}{2}, \frac{\pi}{2}]$, with pixels mapped as

$$(x, y) = \left( \frac{\cos\phi\,\sin(\lambda-\lambda_0)}{\cos c},\ \frac{\cos\phi_0\,\sin\phi-\sin\phi_0\,\cos\phi\,\cos(\lambda-\lambda_0)}{\cos c} \right),$$

where $c$ is the angular distance from the tangent-point $(\lambda_0,\phi_0)$ (Çapuk et al., 26 Jun 2025). Common tasks include text-to-panorama (T2P), view-to-panorama (V2P), inpainting/outpainting, image-editing, and multi-view fusion (Feng et al., 7 Dec 2025, Tuli et al., 8 Jul 2025).

Different representations favor different computational and perceptual properties. ERP is simple but suffers from severe polar distortion and requires explicit left-right continuity handling. Cubemaps yield six perspective faces (typically $90^\circ$ FOV per face), favoring compatibility with perspective pre-training and facilitating geometric consistency, though they introduce seams at face junctions if not carefully treated (Kalischek et al., 28 Jan 2025, Feng et al., 7 Dec 2025). Tangent-plane tiling (gnomonic projection) enables more uniform distortion distribution and is amenable to parallel generation (Çapuk et al., 26 Jun 2025).

2. Generative Modeling Frameworks

2.1 Diffusion and Transformer Architectures

Diffusion models are the dominant class for high-fidelity panorama generation. These models iteratively denoise latent representations, often equipped with cross-attention conditioning for text, partial images, or structural priors (Feng et al., 7 Dec 2025, Kalischek et al., 28 Jan 2025, Çapuk et al., 26 Jun 2025). Both U-Net-based and transformer-based backbones (e.g., DiT, Stable Diffusion, Flux) are used.

Multi-view transformer methods (CubeDiff, JoPano) adapt off-the-shelf 2D backbones to handle cubemap inputs by inflating attention—allowing self-attention and normalization operations to flow across cube faces and enforce translation equivariance (Feng et al., 7 Dec 2025, Kalischek et al., 28 Jan 2025, Huang et al., 20 Jun 2025). Adapter modules (Joint-Face Adapter) or operator replacement (multi-plane synchronization) facilitate the transfer of 2D pre-trained priors to the spherical or omnidirectional domain (Feng et al., 7 Dec 2025, Huang et al., 20 Jun 2025).

2.2 Autoregressive and Token-based Methods

PanoLlama introduces a paradigm shift by recasting panoramic image generation as autoregressive next-token prediction: a panorama is viewed as a sequence of concatenated image crop-tokens. The first-order Markov assumption, $P(x_i|x_{<i}) \approx P(x_i|x_{i-1})$, enables the use of LLM-style AR generators with training-free token-redirection, sidestepping complex diffusion-chain fusion and enabling endless, seamlessly coherent expansion in both spatial directions (Zhou et al., 2024).

2.3 Geometric-aware and Spherical Consistency Modules

To maintain omnidirectional geometric fidelity, recent models incorporate explicit geometry modules:

• Spherical Epipolar-Aware Attention: DiffPano restricts cross-view attention in the UNet to pairs of points matched along 3D epipolar curves, aligning multi-view feature propagation with the underlying spherical scene structure (Ye et al., 2024).
• Circular Padding and Yaw/Cube Losses: DiT360 enforces seam-free boundary continuity and geometrical equivariance in the latent space using circular padding, rotational consistency (yaw) loss, and distortion-aware cube loss (Feng et al., 13 Oct 2025).
• Multi-Plane Synchronization: DreamCube synchronizes self-attention, convolution, and normalization across cube faces, enforcing seamless content and statistical alignment at both low- and high-level features in the omnidirectional space (Huang et al., 20 Jun 2025).

3. Data Modalities, Conditioning, and Control

Panorama generation systems accept diverse input modalities:

• Textual Prompts: Text-to-panorama is the dominant conditioning paradigm. Cross-attention injects text prompt embeddings into generation, often using CLIP or T5 encoders (Feng et al., 7 Dec 2025, Zhou et al., 2024, Chen et al., 2022).
• Partial Images or NFoV Inputs: For cases where only narrow field-of-view images are available, two-stage pipelines first estimate camera orientations (overlap classification and angular regression), then warp and complete the observed region with diffusion models conditioned on both image and pose information (Wang et al., 2023, Zheng et al., 24 Mar 2025).
• Multi-modal and Semantic Control: Advanced methods support composite textual/image control (multi-guidance, mask-free layout insertion (Zhou et al., 2024)), explicit BEV/semanic priors (Teng et al., 9 Jul 2025), or hierarchical object-layout graphs parsed from language (Zhang et al., 31 Jan 2026).
• Lighting and HDR Supervision: High-fidelity panorama-to-light generation for rendering is addressed using dual-codebook quantization, inversion, and inverse tone-mapping cascades, supervised both in LDR and HDR domains (Wang et al., 2022, Chen et al., 2022).

4. Algorithmic Innovations: Seam Handling, Layering, and 3D Lifting

4.1 Seam Consistency and Post-processing

Cubemap and tiled representations naturally suffer from seams at face boundaries or patch junctions. Techniques employed for mitigation include:

• Poisson Blending: Inter-face Poisson blending solves Laplacian equations on each face with Dirichlet boundary conditions set by neighboring faces, minimizing discontinuities at junctions (Feng et al., 7 Dec 2025).
• Circular Padding: Both in image and latent space, circular wrapping ensures left-right ERP continuity. When combined with patched denoising at high resolutions, circular padding is essential for loop-consistency (Çapuk et al., 26 Jun 2025, Feng et al., 13 Oct 2025).
• Tile-based Outpainting: Generative panoramic stitching produces complete panoramas from multiple reference images with large parallax or exposure variations by iterative tile-wise inpainting with layout-aware initialization and robust context conditioning, significantly reducing ghosting and structure drift (Tuli et al., 8 Jul 2025).

4.2 Layered and Volumetric Representations

Recent advances in 3D panorama generation rely on lifting layered 2D panoramas into volumetric or Gaussian splatting representations:

• Layered Decomposition: LayerPano3D and PanoDreamer decompose the reference panorama into depth-ordered layers via segmentation and disparity clustering. Iterative inpainting reconstructs occluded backgrounds; the resulting RGBD stacks are then projected into 3D Gaussians for neural rendering (Yang et al., 2024, Paliwal et al., 2024).
• Panorama Sliding and Multi-View Stereo: PSGS interleaves text-to-panorama with multi-view perspective sampling, then reconstructs globally consistent 3D Gaussian splat point clouds by fitting to overlapping 90° FOV crops (Zhang et al., 31 Jan 2026). Semantic and depth consistency losses enforce multi-view and volumetric coherence.

These volumetric approaches yield panoramic scenes supporting free camera navigation, immersive VR, and consistent 3D geometry.

5. Objective Metrics and Benchmarking

Panorama generation performance is quantified using both traditional generative metrics and specialized seam-aware or geometry-aware scores:

Metric	Purpose	Used in
FID, KID, IS	Fidelity/diversity of panoramic (or crop) images	CubeDiff, DiT360, JoPano
CLIP-FID, CLIP Score	Semantic-text alignment and compatibility	CubeDiff, DiT360, JoPano, PanoLlama
Seam-SSIM, Seam-Sobel	Cubemap face/junction consistency	JoPano
TangentIS, TangentFID	Local (tangent-tile) accuracy and global panorama faults	TanDiT
Distort-CLIP/Distort-FID	Spherical distortion awareness in generated panoramas	PanoDecouple (Zheng et al., 24 Mar 2025)
Perceptual/No-ref (BRISQUE, NIQE, Q-Align)	Aesthetics, no-reference image quality	DiT360, PSGS

User studies frequently validate preference for realism, coherence, and compatibility.

Datasets used for evaluation span Structure3D, SUN360, Polyhaven, Upright360, Matterport3D, HDR360-UHD, and custom text-prompt corpora (Feng et al., 7 Dec 2025, Çapuk et al., 26 Jun 2025, Yang et al., 2024, Chen et al., 2022).

6. Applications and Practical Considerations

Panorama generation underpins multiple downstream applications:

• Vision-and-Language Navigation: Synthetic panoramas are used to expand training sets, improve generalization, and generate paired instructions in VLN research (Li et al., 2023).
• Autonomous Driving and Robotic Perception: Percep360 and Dual-Projection Fusion advance stitched street-view synthesis and upright correction for mobile agents, supporting both geometric integrity (inclination estimation) and downstream BEV segmentation utility (Teng et al., 9 Jul 2025, Shan et al., 30 Nov 2025).
• Photorealistic Lighting, HDRI, and Editing: StyleLight and Text2Light democratize high-dynamic-range, text-driven panorama generation for physically-based rendering and mixed reality, achieving superior RMSE, FID, and real-time relighting/interactivity (Wang et al., 2022, Chen et al., 2022).
• Immersive 3D Content Creation: Layered and volumetric approaches facilitate unconstrained 3D navigation and exploration in VR/AR, supporting hyper-immersive scene authoring from text prompts alone (Yang et al., 2024, Paliwal et al., 2024).

7. Open Problems and Future Directions

Current limits reside in data scarcity (especially high-resolution, real-world panoramas), polar-region distortion, dynamic illumination, and real-time scalability. Promising frontiers include:

• Domain adaptation to bridge synthetic-to-real and HDR/photoreal domains (Feng et al., 13 Oct 2025, Chen et al., 2022).
• Enhanced multi-modality (joint language–layout–image conditioning, complex guidance, and multi-modal user control) (Zhou et al., 2024, Teng et al., 9 Jul 2025).
• Direct 6-DoF modeling, depth-aware and physically-grounded scene synthesis (Huang et al., 20 Jun 2025, Paliwal et al., 2024).
• Advanced seam and artifact suppression via geometric, spectral, and semantic post-processing.
• Efficient and scalable architectures leveraging foundation model priors and sparse attention (Ye et al., 2024, Feng et al., 7 Dec 2025).

The synthesis of seamless, geometrically consistent, and controllable panoramic content remains a rapidly advancing domain at the intersection of generative modeling, geometric vision, and scene understanding.