JoPano: Joint-Face 360° Panorama Synthesis

• JoPano is a unified 360° panorama synthesis framework that leverages cubemap representation and joint modeling of all six faces to ensure seamless and geometrically consistent outputs.
• It employs advanced diffusion transformers and a Joint-Face Adapter to enable condition switching between text-driven and view-driven tasks, enhancing generation efficiency.
• The approach integrates Poisson blending and novel seam metrics to reduce artifacts, achieving state-of-the-art performance across diverse datasets.

Joint-Face Panorama (JoPano) refers to a unified approach for 360° panorama image generation that leverages the cubemap representation and joint modeling of all six cube faces, achieving seamless, geometrically consistent results for both text-driven and view-driven panorama synthesis. Recent advances in this domain employ latent diffusion models or diffusion transformers augmented with specialized architectural components designed to enforce cross-face coherence, enable condition switching, reduce seam artifacts, and unify task handling. JoPano combines precision in geometric encoding, advanced conditioning mechanisms, and new quantitative seam metrics, establishing state-of-the-art results across multiple datasets and evaluation protocols (Kalischek et al., 28 Jan 2025, Feng et al., 7 Dec 2025).

1. Cubemap Representation and Joint Modeling

JoPano encodes a 360°×180° panorama as a cubemap, comprising six perspective faces with 90° field-of-view each. This lossless transformation enables the panorama to be managed as six spatially local, undistorted image tiles. Both CubeDiff (Kalischek et al., 28 Jan 2025) and JoPano (Feng et al., 7 Dec 2025) stack these faces along a new “view” axis, allowing independent face encoding (e.g., via a frozen VAE encoder or DiT patch embedder) yet facilitating global operations over the whole panorama.

To achieve geometric and color consistency across cube faces:

• Normalization: GroupNorm (Kalischek et al., 28 Jan 2025) or LayerNorm (Feng et al., 7 Dec 2025) layers are synchronized across both spatial and face (view) dimensions, preventing per-face tone drift.
• Joint Face Modeling: Standard attention modules are "inflated" such that self-attention and cross-attention operate over the concatenated spatial tokens of all six faces; in DiT-JoPano, the Joint-Face Adapter applies shared layer normalization, then full self-attention across all face tokens, and reshapes the output back to per-face format, with a learnable (zero-initialized) residual connection (Feng et al., 7 Dec 2025).
• 3D Positional Encoding: Each token’s geometric position is encoded using 3D sphere coordinates or rotary positional encoding (RoPE), imparting directional priors and spherical continuity into the model (Feng et al., 7 Dec 2025).

2. Unified Diffusion Framework and Condition Switching

JoPano unifies text-to-panorama (T2P) and view-to-panorama (V2P) generation tasks within a single architecture. A binary condition switch $\gamma\in\{0,1\}$ selects, at each iteration, either a text-driven or view-driven task:

• Text-to-Panorama (T2P): All faces are denoised from noise with a CLIP-encoded global prompt as conditioning.
• View-to-Panorama (V2P): One clean “view” face is provided as input, and the remaining faces are generated conditionally.

The noise injection and conditioning process for each face $f_{i,t}$ at time $t$ follows:

$$f_{i,t} = (1-t)\,f_i + t\,\epsilon, \quad \epsilon \sim \mathcal N(0,1)$$

Network conditioning operates as:

$$\begin{cases} \gamma=0: & v_\theta(f_{0,t},\ldots,f_{5,t}, t, c_\text{text})\ \gamma=1: & v_\theta(\underline{f_0}, f_{1,t},\ldots,f_{5,t}, t, c_\text{text}) \end{cases}$$

with loss applied only to the denoised faces. This scheme avoids redundant task-specific models, increasing data and computation efficiency (Feng et al., 7 Dec 2025).

3. Seam Consistency and Poisson Blending

Even with joint modeling, minor color or gradient inconsistencies arise at cube face boundaries. JoPano addresses these through classical Poisson blending applied to each face independently. For a face $g_i$, the unknown blended result $f_i$ is solved over domain $\Omega_i$ as:

$$\begin{cases} \Delta f_i = \nabla \cdot \mathbf v_i & \text{in } \Omega_i\ f_i = \frac{1}{2}(g_i + g_j) & \text{on } \partial\Omega_i \end{cases}$$

where $\mathbf v_i$ is the gradient field of $g_i$, and neighbor face $g_j$ supplies the Dirichlet boundary average. In implementation, a 5-point stencil and iterative Gauss–Seidel (200 iterations) are used (Feng et al., 7 Dec 2025).

Two specialized metrics assess seam quality:

• Seam-SSIM: Average SSIM across the left/right bands for each of 12 cube edges; higher values denote consistency.

$$\mathrm{Seam\text{-}SSIM} = \frac{1}{12} \sum_{e=1}^{12} \mathrm{SSIM}(B_e^{(L)}, B_e^{(R)})$$

• Seam-Sobel: Mean magnitude of Sobel gradients across edge-adjacent columns; lower values reflect smoother transitions.

$$\mathrm{Seam\text{-}Sobel} = \frac{1}{12} \sum_{e=1}^{12} \frac{\mathrm{mean} |c_e^{(L)}| + \mathrm{mean}|c_e^{(R)}|}{2}$$

JoPano achieves Seam-SSIM scores close to ground truth (0.831 vs 0.847) and significantly lowers Seam-Sobel values (12.66 vs 11.16 ground truth after blending), substantiating its seam reduction efficiency (Feng et al., 7 Dec 2025).

4. Training, Sampling, and Optimization

JoPano is trained on 41,930 panoramas (Structure3D: indoor, SUN360: outdoor), with automated CLIP-based captioning. The loss function minimizes mean-squared error over supervised faces, applying rectified flow objectives. Only the Joint-Face Adapter parameters (≈400M) are optimized, leveraging a frozen Sana-DiT backbone (≈1.6B), with a learning rate of $1\times10^{-4}$ over 1M steps, batch size 8 (Feng et al., 7 Dec 2025).

Sampling proceeds as follows:

• Initialization: Clean (conditioned) faces are encoded; target faces are sampled from Gaussian noise.
• Iterative denoising (DDIM, T=50 steps) applies classifier-free guidance for both text and image embeddings.
• Overlapping-FoV generation: Each face is rendered at 95° FoV, then centrally cropped to 90°, promoting edge continuity (Kalischek et al., 28 Jan 2025).
• Reassembly: Final crops are merged into equirectangular panoramas or cubemap visuals.

5. Quantitative and Qualitative Performance

JoPano presents state-of-the-art results on both T2P and V2P tasks across multiple datasets, as detailed in the following table:

Task	Dataset	FID ↓	CLIP-FID ↓	IS ↑	CLIP-Score ↑
T2P	SUN360	29.83	10.95	7.80	30.12
T2P	Structure3D	34.44	16.17	3.51	27.96
V2P	SUN360	13.07	4.06	7.05	27.93
V2P	Structure3D	16.75	3.97	3.04	27.33

Comparative ablation highlights indicate that:

• 3D-sphere RoPE positional encoding achieves lower FID than planar UV coordinates.
• Poisson Blending elevates Seam-SSIM and reduces Seam-Sobel values, approaching ground-truth consistency.
• JoPano maintains brush-stroke stylization control and minimal seam artifacts.

CubeDiff likewise demonstrates significant performance advantages over prior models (e.g., FID 10.0 vs. 25.7 for MVDiffusion on LAVAL Indoor), with robust generalization to small data scales (Kalischek et al., 28 Jan 2025).

6. Limitations and Prospective Directions

Current limitations for JoPano include residual blurriness at fine scales, primarily due to low-resolution training images ($1024\times512$ upsampled). Sana-DiT, while efficient, yields visual fidelity trailing the flux DiT backbone (Feng et al., 7 Dec 2025).

Other limitations identified include:

• Quadratic memory scaling in global (inflated) attention layers, hampering higher face resolutions unless sparse/localized attention mechanisms (e.g., epipolar constraints) are incorporated (Kalischek et al., 28 Jan 2025).
• Absence of explicit 3D supervision: dynamic scene handling or viewpoint changes require camera-pose conditioning integration.

Planned future work comprises:

• Large-scale, high-resolution panorama dataset collection
• Backbone replacement with higher-fidelity DiT modules (e.g., Flux)
• Multi-scale or patch-based attention for scalable cubemap generation
• Joint training on cubemap and equirectangular projections, with further pole smoothing
• Integration with ControlNet for controllable object placement
• Extension to video panoramas and time-dependent VR/AR generation sequences

7. Related Work and Conceptual Distinctions

JoPano situates itself relative to autoregressive outpainting, circular-padding, and correspondence-aware methods, superseding these in seam continuity and geometric fidelity by its joint modeling strategy. Both CubeDiff and JoPano demonstrate that high-fidelity, wrap-around-consistent panoramas can be synthesized from text and/or image prompts without specialized stitching or autoregression, primarily through minimal architectural extensions to pretrained latent diffusion or transformer backbones (Kalischek et al., 28 Jan 2025, Feng et al., 7 Dec 2025).

This joint-face approach marks a technical progression in panorama synthesis, consolidating task unification, geometric encoding, advanced seam metrics, and efficient domain adaptation. A plausible implication is broad utility for VR/AR content pipelines and high-resolution, artifact-minimized panoramic visualization.