---
title: 'Panoramic Enhancer (PE): Overview & Applications'
url: https://www.emergentmind.com/topics/panoramic-enhancer-pe
type: topic
---

# Panoramic Enhancer (PE): Overview & Applications

to=arxiv_search.search  天天中彩票中大奖  天天中彩票足球  պարզ query="2507.06971" max_results=5 sort_by="relevance" sort_order="descending"
to=arxiv_search.search  պարզ  diýenjson
{"query":"2507.06971","max_results":5,"sort_by":"relevance","sort_order":"descending"}
to=arxiv_search.search  天天中彩票网络json
{"query":"Panoramic Enhancer panoramic generation autonomous driving Percep360","max_results":10,"sort_by":"relevance","sort_order":"descending"}
to=arxiv_search.search  njhanijson
{"query":"QuaDreamer Panoramic Enhancer quadruped robots panoramic video generation","max_results":10,"sort_by":"relevance","sort_order":"descending"}
to=arxiv_search.search  大发快三怎么json
{"query":"Canvas360 geometry-aware pretraining panoramic in-context generation","max_results":10,"sort_by":"relevance","sort_order":"descending"}
to=arxiv_search.search  天天彩票中大奖json
{"query":"\"Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting\"","max_results":5,"sort_by":"relevance","sort_order":"descending"}
Panoramic Enhancer (PE) is a recurring designation in recent panoramic imaging and generation research for modules or full pipelines that improve the quality, coherence, controllability, or geometric fidelity of 360° data. In different papers, PE refers to a generative enhancer for stitched street-view panoramas, a distortion-aware module for wide-FoV panoramic video, a computational imaging pipeline for compact annular lenses, or a geometry-aware in-context generation system [2507.06971] [2508.02512] [2206.06070] [2306.12992] [2607.08765]. Taken together, these usages suggest that PE is best understood as a task-dependent research motif centered on panorama-specific failure modes—seam discontinuity, latitude-dependent distortion, spatially variant blur, controllability loss, and geometric inconsistency—rather than as a single canonical architecture.

## 1. Terminology, scope, and recurrent design goals

The term “Panoramic Enhancer” is used heterogeneously. In Percep360, it is mapped to a system that both enhances stitched panoramas and generates new 360° street-view samples for autonomous driving; in QuaDreamer, it is a dual-stream subnetwork for correcting panoramic distortions in quadruped-robot video generation; in ACI/PI2RNet and PCIE/PART, it denotes restoration pipelines for compact panoramic optics; and in Canvas360, it denotes a geometry-aware in-context panoramic generation framework [2507.06971] [2508.02512] [2206.06070] [2306.12992] [2607.08765].

| Formulation | Primary role | Core mechanisms |
|---|---|---|
| Percep360 | Enhancement and generation of 360° street-view panoramas | LSDM and PPM |
| QuaDreamer PE | Distortion-aware panoramic video enhancement | SSM and FFC |
| ACI / PI2RNet | Restoration of PAL annular panoramas | Wave-based simulation and physics-informed restoration |
| PCIE / PART | Aberration correction and SR&AC for MPIP | PSF map, PFM, PMAB |
| Canvas360 PE | Geometry-aware in-context panoramic generation | Parallel RGB–depth pretraining, velocity circular padding, token-level concatenation |

Across these formulations, three objectives recur. First, PE commonly targets **global coherence**, especially at the left–right seam and under equirectangular distortion. Second, it often targets **conditional fidelity**, meaning that layout, depth, masks, text, or motion controls remain valid after enhancement. Third, several PE systems are explicitly **projection-aware** or **physics-aware**, incorporating spherical topology, PSF structure, or panoramic camera geometry directly into the model. This suggests that PE research is defined less by a shared backbone than by a shared insistence that panoramic imagery cannot be treated as an ordinary planar image domain.

## 2. Percep360: panoramic enhancement for autonomous driving

Percep360 is presented as “the first panoramic generation method Percep360 for autonomous driving” and is explicitly mapped to Panoramic Enhancer (PE) for “improving quality, coherence, and controllability of 360° street-view panoramas” [2507.06971]. It operates both as an enhancer and a generator: starting from stitched panoramas built from multi-pinhole camera rigs, it reduces seam artifacts, corrects spatial misalignments, and improves visual quality, while also hallucinating coherent, high-quality 360° panoramas conditioned on BEV map, depth, mask, and text.

Its coherence mechanism is the **Local Scenes Diffusion Method (LSDM)**. Percep360 uses Latent Diffusion Models where images are encoded to latent space with a VAE encoder \(E\) and decoded by \(D\). With \(z = E(x)\), forward noising and reverse denoising are written as
\[
q(\mathbf{z}_t \mid \mathbf{z}_{t-1}) = \mathcal{N}\!\big(\mathbf{z}_t;\sqrt{\alpha_t}\,\mathbf{z}_{t-1}, (1-\alpha_t)\mathbf{I}\big),\quad
q(\mathbf{z}_t \mid \mathbf{z}_0) = \mathcal{N}\!\big(\mathbf{z}_t;\sqrt{\bar{\alpha}_t}\,\mathbf{z}_0, (1-\bar{\alpha}_t)\mathbf{I}\big),
\]
\[
p_\theta(\mathbf{z}_{t-1}\mid \mathbf{z}_t,\mathbf{c}) = \mathcal{N}\!\big(\mathbf{z}_{t-1};\mu_\theta(\mathbf{z}_t,t,\mathbf{c}),\Sigma_\theta(\mathbf{z}_t,t,\mathbf{c})\big).
\]
The denoising objective is
\[
\mathcal{L} = \mathbb{E}_{t,\mathbf{z}_0,\boldsymbol{\epsilon}}\big[\|\boldsymbol{\epsilon}-\boldsymbol{\epsilon}_\theta(\mathbf{z}_t,t,\mathbf{c})\|_2^2\big],\quad
\mathbf{z}_t = \sqrt{\bar{\alpha}_t}\,\mathbf{z}_0 + \sqrt{1-\bar{\alpha}_t}\,\boldsymbol{\epsilon}.
\]

LSDM addresses the fact that stitched panoramas mix coherent subregions \(\{\mathcal{R}_i\}\) with aliased seam regions \(\{\mathcal{A}_i\}\). The panorama is treated as a spatially continuous domain on a sphere, and all 2D inputs are circularly shifted by a random angle \(\theta\):
\[
\Delta w=\left\lfloor \frac{\theta}{2\pi} W \right\rfloor,\quad
\text{Roll}(\mathbf{X},\Delta w)[u,v]=\mathbf{X}\big[(u+\Delta w)\bmod W,\;v\big].
\]
This wrap-around diffusion exposes the model to different seam locations and encourages learning continuity across the \(0/2\pi\) boundary. The underlying equirectangular mapping is
\[
u = W\,\frac{\theta+\pi}{2\pi},\qquad v = H\,\frac{\phi+\pi/2}{\pi},
\]
which makes continuity at \(u=0\) and \(u=W\) structurally central rather than incidental.

Its controllability mechanism is the **Probabilistic Prompting Method (PPM)**. Supported controls include BEV layout maps, semantic or mask maps, depth maps from Depth Anything, and text prompts. For prompts \(\{p_i\}_{i=1}^m\) with features \(\{\mathbf{F}_i\}\), relevance scores define
\[
p(p_i\mid \text{context}) = \frac{\exp(s_i)}{\sum_j \exp(s_j)},\qquad \alpha_i = p(p_i\mid \text{context}),\qquad \mathbf{c} = \sum_{i=1}^m \alpha_i\,\mathbf{F}_i.
\]
The instantiated attention further injects depth and mask priors through \(\lambda_1\) and \(\lambda_2\), with
\[
\mathbf{A}_{\text{final}}=\max(\mathbf{A}_d,\mathbf{A}_m),\quad \text{Attn}=\text{softmax}(\mathbf{A}_{\text{final}}),\quad
x_{\text{out}}=\text{Proj}(\text{Attn}\cdot v) + x.
\]
This is interpreted in the paper as region-wise probabilistic selection of control signals.

The implementation uses a side-controlling U-Net, a frozen VAE encoder/decoder, a BEV encoder, and a CLIP text encoder. Training uses circular rotations, depth and mask priors for attention gating, and 20 diffusion steps; the reported dataset size is 28,130 train images and 6,019 validation images, with training on two NVIDIA A6000 GPUs for approximately 10 days [2507.06971].

On the nuScenes-360 validation set, Percep360 reports BRISQUE \(20.24\), PIQE \(11.44\), SSIM \(0.16\), FID \(14.43\), Drivable IoU \(0.25\), Mean IoU \(0.13\), and Rank \(7\), outperforming adapted baselines in best no-reference quality and top overall score. When used to augment OneBEV segmentation, Drivable IoU improves from \(0.593\) to \(0.610\) and Mean IoU from \(0.470\) to \(0.495\), whereas baseline synthetic data did not help [2507.06971]. In this formulation, PE is not merely a renderer; it is a panorama-specific control-and-repair mechanism for replacing stitched artifacts with a learned coherent distribution.

## 3. QuaDreamer PE: dual-stream correction for wide-FoV panoramic video

QuaDreamer introduces a different use of the term: a **Panoramic Enhancer (PE)** designed for controllable panoramic video generation for quadruped robots [2508.02512]. Its starting point is the equirectangular projection, where longitude \(\lambda\) and latitude \(\phi\) map to image coordinates by
\[
u = W \cdot \left(\frac{\lambda}{2\pi} + \frac{1}{2}\right),\qquad
v = H \cdot \left(\frac{\phi}{\pi} + \frac{1}{2}\right),
\]
with sphere coordinates
\[
x = \cos \phi \cdot \cos \lambda,\qquad y = \cos \phi \cdot \sin \lambda,\qquad z = \sin \phi.
\]
Because the surface area element satisfies \(dA \propto \cos \phi\, d\lambda\, d\phi\), apparent horizontal stretching grows like \(1/\cos \phi\) as \(|\phi|\) increases. The paper associates this with latitude-dependent stretching, seam artifacts at \(\lambda=\pm\pi\), and wide-FoV geometric inconsistencies under robot jitter.

PE is therefore formulated as a **distortion-aware, dual-stream module** embedded in the QuaDreamer diffusion backbone. The first stream performs **spatial–structure correction** through State Space Models. For an encoder feature tensor \(D \in \mathbb{R}^{B\times C\times H\times W}\), the SSM update is
\[
D^*[b, c, x, y] = \frac{1}{N} \sum_{d \in scan} F_{S6}(S_d(D[b, c, x, y])).
\]
SSM blocks are injected before the first downsampling and after the final upsampling. The stated effect is long-range, multi-directional stabilization of horizon and large-scale layout, with explicit emphasis on seam continuity and panoramic unwrapping.

The second stream performs **frequency–texture refinement** through Fast Fourier Convolution. After three successive downsamplings, intermediate residual blocks apply
\[
X_{\text{out}} = X + FFC(FFC(X_l, X_g)),
\]
with a local/global split of \(25\%/75\%\). The global pathway uses spectral convolution:
\[
\mathcal{F}(I) = FFT(I),\qquad Y = IFFT(M \odot \mathcal{F}(I)),
\]
and the residual form is described as recovering periodic patterns and sharpening textures while avoiding grid artifacts.

This stage-wise collaboration is central to the paper’s definition of PE: early SSM aligns geometry, middle FFC restores detail, and late SSM suppresses seam drift or ringing that may follow frequency enhancement. Unlike Percep360, which emphasizes multi-prompt controllability, QuaDreamer PE is explicitly a distortion-correction subnetwork that complements Vertical Jitter Encoding and the Scene-Object Controller [2508.02512].

Training is end-to-end under the standard diffusion denoising objective
\[
\mathcal{L}(\theta) = \mathbb{E}_{t \sim U(1, T), \epsilon_t \sim \mathcal{N}(0, I)} \left[\|\epsilon_t - \epsilon_\theta(z_t; t, y)\|^2\right],
\]
without auxiliary adversarial or explicit geometry or frequency losses. The reported implementation uses Stable Video Diffusion initialization, a frozen CameraCtrl camera encoder, a single NVIDIA A6000 48G GPU, 320 epochs, 200k steps, 78 hours, batch size \(2\), learning rate \(3\times 10^{-5}\), and DPM-Solver with 30 sampling steps [2508.02512].

Ablations isolate the contribution of PE. Relative to a baseline with FVD \(= 887.31\), LPIPS \(= 0.2714\), and SSIM \(= 0.3815\), adding PE yields FVD \(= 830.92\), LPIPS \(= 0.2669\), and SSIM \(= 0.3893\). The combined SOC+PE model achieves LPIPS \(= 0.2614\), SSIM \(= 0.3947\), and PTrack \(= 14.1744\). When OmniTrack is augmented with QuaDreamer data, HOTA improves by \(+10.1\%\) and MOTA by \(+14.8\%\) over “Real Only” [2508.02512]. In this branch of the literature, PE denotes a panorama-native corrective prior for wide-FoV video rather than a standalone generator.

## 4. Computational-imaging PE: PAL restoration, PSF priors, and minimalist optics

A restoration-oriented branch of PE is represented by **Annular Computational Imaging (ACI)** and by the **Panoramic Computational Imaging Engine (PCIE)**. In these papers, the central problem is not promptable generation but recovery of high-quality panoramas from minimalist panoramic optics with severe spatially variant degradation [2206.06070] [2306.12992].

In ACI, PE is defined as a computational imaging pipeline that restores raw annular panoramas captured by compact Panoramic Annular Lenses (PALs) [2206.06070]. The forward model is
\[
y(u,v) = \iint h(u,v; x,y)\, x(x,y)\, dx\,dy + n(u,v),
\]
with spatially variant PSF \(h(u,v; x,y)\). Wave-optics simulation uses an annular pupil, Fresnel diffraction, and Zernike wavefront aberration
\[
W(\rho,\theta) = \sum_{n,m} C^m_n\, Z^m_n(\rho,\theta),
\]
with PSF obtained as
\[
K(x,y;\theta,\lambda) = |U(x,y;z;\theta,\lambda)|^2.
\]
The simulation samples \(\theta \in [30^\circ,100^\circ]\) at \(0.7^\circ\) and wavelengths \(\lambda \in [400,700]\) nm at \(10\) nm, keeps the first 37 Zernike terms, and randomizes coefficients by \(\pm 25\%\) to bridge the synthetic-to-real gap [2206.06070].

The restoration network, **PI2RNet**, uses two U-Nets in cascade with a physics-informed single-pass engine, a Physics-informed Bridge, and Dynamic Deformable Kernel Prediction. The total loss is
\[
\mathcal{L}_{\text{total}} = \lambda\, \| \hat{\mathcal{R}}_0 - \mathcal{G} \|_1 + \| \hat{\mathcal{R}} - \mathcal{G} \|_1,
\]
with \(\lambda = 0.5\). On DIVPano validation, PI2RNet reports PSNR \(28.71\) dB and SSIM \(89.70\%\), outperforming SRN, NAFNet, HINet, DeepRFT, and KPN; runtime is approximately \(0.21\) s per \(288\times1504\) panorama on an RTX 3090 [2206.06070].

PCIE/PART advances a related but distinct formulation for a **Minimalist Panoramic Imaging Prototype (MPIP)** with fewer than three spherical lenses [2306.12992]. Here, the image formation model is
\[
A_{\theta}(x,y) = \Gamma\!\left[(\int r_{\lambda}R_{\theta}(x,y)\otimes K_{\theta}(x,y,\lambda)d\lambda)\downarrow+N\right],
\]
with Zernike-based wavefront
\[
\Phi_{\theta}(x',y',\lambda) = \sum_{n,m} {C^m_n}(\theta,\lambda){Z^m_n}(x',y').
\]
PE is built around a **PSF-aware Aberration-image Recovery Transformer (PART)**. A compressed PSF map is constructed by
\[
\hat{K}_{x,y} = \mathrm{AveragePool}(\mathrm{padding}(K_{x,y})),\qquad
x_{psf} = \mathrm{Concat}(x_{int}, x_s).
\]
PSF features then condition both a **PSF-aware Feature Modulator (PFM)** and a **PSF-aware Mix-Attention Block (PMAB)**. The PFM computes
\[
x'_{psf} = E_{psf}(x_{psf}),\qquad x_{kernel} = P(x'_{psf}),
\]
while PMAB mixes standard window attention and varied-size attention:
\[
Q,K,V = \mathrm{Linear}(\mathrm{WinPar}(x'_{img})),\qquad
S,O = \mathrm{WinTrans}(x'_{psf}),
\]
\[
K_{vs},V_{vs} = \mathrm{Sample}(K,S,O),\qquad
\mathrm{Attn}(Q, K_{vs}, V_{vs}) = \mathrm{Softmax}\!\left(\frac{QK_{vs}^{\top}}{\sqrt{d}}\right)V_{vs}.
\]

The reported pipelines are **AC** and **SR&AC**. On PALHQ-SynMPIP-P1 for AC, PART reports PSNR \(33.143\), SSIM \(0.9304\), LPIPS \(0.0435\), and FID \(3.571\); on PALHQ-SynMPIP-P2 for AC, PSNR is \(27.198\), SSIM \(0.8540\), LPIPS \(0.0855\), and FID \(16.436\). On synthetic SR&AC, PART reports PSNR \(29.310\), SSIM \(0.8819\), LPIPS \(0.0681\), and FID \(9.648\). On RealMPIP3K-AC, OIQE rises to \(77.87\%\), which the paper reports as the best result [2306.12992].

Taken together, ACI/PI2RNet and PCIE/PART define a physically grounded understanding of PE: enhancement is treated as inversion of a panoramic optical forward model with explicit priors on PSFs, annular geometry, and sensor behavior.

## 5. Geometry-aware and in-context PE: the Canvas360 formulation

Canvas360 uses PE in yet another sense: a **geometry-aware in-context panoramic generation system** built around pretraining and unified fine-tuning [2607.08765]. It targets style transfer, inpainting, outpainting, and editing on equirectangular panoramas, with main experiments at \(1024\times2048\) and ablations at \(512\times1024\).

The pretraining stage uses **parallel RGB–depth generation** with a Flow Transformer built on FLUX.1-dev. RGB and depth latents are concatenated,
\[
x = [x_{rgb}; x_{depth}],
\]
and trained by flow matching on the linear interpolant
\[
x_t = (1 - t) x_0 + t x_1,\qquad t \in [0,1],
\]
with objective
\[
L_{FM}(\theta) = \mathbb{E}_{t, x_0, x_1} \left[ \| v_\theta(x_t, t) - (x_1 - x_0) \|^2 \right].
\]
To prevent modality collapse, Canvas360 introduces
\[
L_{sim} = \mathbb{E} \left[ \left( \frac{\langle v_{rgb}, v_{depth}\rangle}{\|v_{rgb}\|_2 \|v_{depth}\|_2} \right)^2 \right],
\qquad
L = L_{FM} + \lambda L_{sim}.
\]

Its seam mechanism is **velocity circular padding**. Rather than only padding image columns, the method pads both interpolated latents and target velocities:
\[
\tilde{x}_t^{0} = x_t^{W},\qquad \tilde{x}_t^{W+1} = x_t^{1},
\]
\[
\tilde{v}^{0} = v^{W},\qquad \tilde{v}^{W+1} = v^{1}.
\]
The paper argues that this exposes wrap-around adjacency directly in the learned velocity field and improves seam continuity quantitatively and qualitatively. Its best reported LRCE-RGB is \(0.0063\) [2607.08765].

Fine-tuning removes the depth branch and uses **token-level concatenation**:
\[
x = [x_{tgt}; x_{cxt}],
\]
with positional offsets separating target and context roles. This single model supports the four downstream tasks through unified conditioning, rather than training distinct panorama-specific models for each task.

The data scale is unusually large for a panoramic setting. Canvas360Dataset contains **1M paired panoramic samples**, consisting of a **100K pilot set** of RGB–depth panoramas and **900K downstream in-context samples**: Outpainting \(250K\), Inpainting \(250K\), Style transfer \(200K\), and Editing \(200K\). Pseudo-depth is obtained from DAP and clipped at **100 meters** for outdoor scenes and **10 meters** for indoor scenes before normalization [2607.08765].

For text-to-panorama generation, the paper reports **FAED \(= 2.33\)** and **IS \(= 1.76\)** as best, with **FID \(= 44.17\)**, **FID\(_{pole}=51.02\)**, **FID\(_{equ}=25.96\)**, **QA\(_{aesthetic}=4.20\)**, **QA\(_{quality}=4.71\)**, **BRISQUE \(=17.12\)**, and **NIQE \(=3.70\)**. In a user study with 71 participants and 10 images, the method leads in boundary continuity, panorama awareness, and overall quality [2607.08765]. In this usage, PE denotes a panorama-specific generative prior that is geometry-aware before it is task-aware.

## 6. Evaluation regimes, applications, and open issues

PE systems are evaluated along several non-interchangeable axes. Percep360 combines image quality assessment, controllability, and downstream Bird’s Eye View segmentation through SSIM, FID, BRISQUE, PIQE, Drivable IoU, and Mean IoU [2507.06971]. QuaDreamer measures video fidelity and control through FVD, LPIPS, SSIM, PTrack, HOTA, and MOTA [2508.02512]. ACI/PI2RNet and PCIE/PART evaluate restoration using PSNR, SSIM, LPIPS, FID, OIQE, BRISQUE, and NIQE [2206.06070] [2306.12992]. Canvas360 adds panorama-aware metrics such as FAED and LRCE-RGB, which are explicitly designed to capture ERP-specific artifacts that perspective-trained metrics may underweight [2607.08765].

The application domains are correspondingly broad. Percep360 is tied to autonomous driving data regeneration and BEV segmentation [2507.06971]. QuaDreamer is aimed at quadruped robots and 360° multi-object tracking under vertical jitter [2508.02512]. ACI/PI2RNet and PCIE/PART target mobile and wearable panoramic imaging with PAL or MPIP hardware [2206.06070] [2306.12992]. Canvas360 targets in-context generation tasks, including style transfer, inpainting, outpainting, and editing [2607.08765]. This distribution of tasks indicates that PE has become a bridge term linking restoration, generation, and control in panoramic research.

Several limitations recur across formulations. Percep360 notes domain gaps between stitched panoramas and real panoramic cameras, hallucination risks in occluded or aliased regions, moving-object artifacts, calibration sensitivity, and controllability limits for precise road topology; it also states that generated data should be flagged as synthetic and that perception systems should not be trained solely on hallucinated scenes without adequate validation [2507.06971]. QuaDreamer notes weaker performance at panoramic boundaries, difficulty under extreme jitter and blur, near-field parallax, and sensitivity to equirectangular format and seam alignment [2508.02512]. ACI and PCIE both identify synthetic-to-real gaps, sensitivity to ISP mismatch or calibration error, and residual failures under extreme aberration, low light, or unmodeled noise [2206.06070] [2306.12992]. Canvas360 identifies underrepresented categories such as high-resolution faces and dense text signage, especially in high-distortion ERP regions [2607.08765].

Related research reinforces these themes. PanoWorld reframes panoramic video generation as geometry- and dynamics-consistent latent state modeling with depth and trajectory consistency losses, emphasizing that panoramic generation should be treated as a geometric modeling problem [2605.15391]. SphereDrag addresses boundary discontinuity, trajectory deformation, and uneven pixel density in panoramic editing through Adaptive Reprojection, Great-Circle Trajectory Adjustment, and Spherical Search Region Tracking [2506.11863]. Pano360 extends stitching into 3D photogrammetric space for globally consistent alignment across many views [2603.12013]. These adjacent directions do not use PE identically, but they converge on the same conclusion: panoramic enhancement is fundamentally conditioned by spherical geometry, projection nonuniformity, and cross-view consistency.

Future directions stated across the literature include better transfer of pinhole annotations to panoramic domains, stronger prompt utilization and multi-view geometry constraints, extension to panoramic videos with temporal coherence, and richer geometry-aware pretraining [2507.06971] [2607.08765] [2605.15391]. A plausible implication is that the term “Panoramic Enhancer” will continue to denote systems that couple panorama-specific inductive bias with either generative control, optical inversion, or geometric regularization, rather than converging on a single universal model class.

Source: https://www.emergentmind.com/topics/panoramic-enhancer-pe