Panoramic Visual Feature Model

• Panoramic visual feature models are learning frameworks that encode full-surround imagery using spherical representations, geometric consistency, and sensor fusion.
• They bridge perspective-based priors with advanced adaptations like spherical embeddings and contrastive pretraining to enhance segmentation, reconstruction, and spatial reasoning.
• Empirical studies report improvements in metrics—such as increased panoptic quality and reduced pose RMSE—demonstrating their effectiveness in diverse omnidirectional applications.

The term panoramic visual feature model denotes representations and learning frameworks that encode full- or near-full-surround observations while preserving the geometric and semantic structure of spherical imagery. In the narrow sense used by "Multi-LVI-SAM" (Zhang et al., 6 Sep 2025), it is a canonical panoramic model that unifies observations from multiple fisheye cameras into a single global representation for optimization. In the broader literature, closely related panoramic feature models support panoptic segmentation, multi-task dense prediction, geometry estimation, 3D reconstruction, video generation, vision-language grounding, and domain-specific reasoning on omnidirectional imagery (Jaus et al., 2021, Zhang et al., 5 Feb 2026, Bozic et al., 25 May 2026). Across these settings, the central problem is consistent: perspective-native encoders inherit planar assumptions that break under wrap-around continuity, pole distortion, arbitrary viewing orientation, and extreme field of view.

1. Emergence of the concept

An early operational form of panoramic feature modeling appears in indoor localization, where visual landmarks were modeled with a single panoramic image feature rather than multiple narrow-field images. "GPU-accelerated Hierarchical Panoramic Image Feature Retrieval for Indoor Localization" used a 1D omnidirectional feature derived from a panoramic image and the magnitude of its FFT, yielding a rotation-invariant descriptor linked to floor-plan coordinates; the resulting GPU-accelerated retrieval system reported 14fps real-time response (Hu, 2020). In that formulation, the panoramic feature model was primarily a compact retrieval key.

Later work shifted the term toward learned representations. "Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning" defined panoramic panoptic segmentation as the most holistic scene understanding in both field of view and image-level understanding, and introduced the Panoramic Robust Feature (PRF) framework to transfer features learned on pinhole images to panoramic deployment (Jaus et al., 2021). The same paper introduced WildPPS, described as the first panoramic panoptic image dataset (Jaus et al., 2021).

The phrase becomes explicit in "Multi-LVI-SAM: A Robust LiDAR-Visual-Inertial Odometry for Multiple Fisheye Cameras", where the panoramic visual feature model is a global geometric optimization framework that consolidates multiple fisheye cameras into a single representation, simplifies loop closure and global pose optimization, and avoids redundant handling of individual cameras (Zhang et al., 6 Sep 2025). This narrower usage is important because it treats the panoramic feature model not merely as an image encoder, but as a unifying state space for optimization.

This progression suggests a historical broadening: from rotation-invariant handcrafted descriptors, to robust learned backbones for dense perception, to geometry-aware latent spaces spanning perception, generation, and embodied reasoning.

2. Geometric representations and spherical parameterizations

A panoramic visual feature model is not tied to a single image parameterization. The surveyed literature uses several, each chosen to reconcile 360-degree coverage with the inductive biases of existing models.

Representation	Reported motivation	Example papers
Canonical sphere	Unified multi-camera constraints and loop closure	(Zhang et al., 6 Sep 2025)
ERP with spherical adaptation	Full 360° continuity with explicit seam handling	(Jiang et al., 14 May 2026, Feng et al., 9 Jul 2026)
Cubemap	Reduced ERP distortion and compatibility with perspective backbones	(Bozic et al., 25 May 2026)
ViewPoint map	Global spatial continuity with fine-grained local detail	(Fang et al., 30 Jun 2025)
Multi-modal panoramic renderings	Direct compatibility with 2D VLMs while retaining 3D cues	(Jung et al., 24 Dec 2025)

Several papers emphasize that ERP is convenient but geometrically problematic. "PanoVGGT" identifies wrap-around continuity and pole singularities as core difficulties of equirectangular panoramas and introduces spherical positional embeddings, three-axis $SO(3)$ rotation augmentation, and stochastic anchoring to respect spherical geometry and orientation ambiguity (Guo et al., 18 Mar 2026). "Canvas360" addresses the same seam structure with velocity circular padding, explicitly padding the left and right panorama boundaries as adjacent regions during flow-matching training (Feng et al., 9 Jul 2026). "PanoWorld: Geometry-Consistent Panoramic Video World Modeling" adapts positional encoding with latitude-aware compression and weights losses by $\cos(\phi)$ so that optimization follows true spherical area rather than ERP pixel density (Jiang et al., 14 May 2026).

Other papers avoid ERP’s distortion by changing representation. "Unified Panoramic Geometry Estimation via Multi-View Foundation Models" adopts a cubemap projection, partitioning the panorama into six 90° perspective faces so that a perspective 3D foundation model can be reused with minimal architectural change; it further adds cross-face valid padding to maintain continuity across face boundaries (Bozic et al., 25 May 2026). "ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models" argues that ERP is globally continuous but severely distorted, while cubemap is locally clean but spatially fragmented, and therefore introduces the ViewPoint map, a pseudo-stitching representation designed to preserve both global continuity and fine-grained local fidelity (Fang et al., 30 Jun 2025).

A different geometric strategy appears in optimization and reconstruction. "Multi-LVI-SAM" projects camera observations onto a unit panoramic sphere centered at a global location, then introduces an extrinsic compensation method to correct triangulation bias caused by the offset between each physical camera center and the sphere center (Zhang et al., 6 Sep 2025). "Pano2World" and the whole-house "PanoWorld" both attach Plücker ray descriptors or panoramic ray encodings to tokens so that latent features remain explicitly camera- and geometry-aware during 3D lifting (Li et al., 1 Jul 2026, Jia et al., 18 May 2026).

A common misconception is that a panoramic feature model is simply a perspective feature model applied to a wider image. The literature repeatedly rejects that view by adding spherical positional encoding, circular padding, cross-face synchronization, or ray-aware conditioning precisely because unmodified perspective assumptions generalize poorly to 360° data (Guo et al., 18 Mar 2026, Bozic et al., 25 May 2026, Feng et al., 9 Jul 2026).

3. Robust feature learning for dense panoramic understanding

The dense-prediction branch of the literature treats the panoramic visual feature model as a mechanism for robustness under domain shift, task interference, and spherical distortion.

PRF uses a two-stage pipeline: unsupervised pixel-wise contrastive pretraining of a backbone such as ResNet18, followed by supervised panoptic training with a lightweight version of Seamless-Scene-Segmentation that includes a feature pyramid and semantic and instance heads (Jaus et al., 2021). The pretraining stage combines a spatial contrastive loss with a global propagation loss, and uses a momentum encoder inspired by MoCo (Jaus et al., 2021). On WildPPS, the reported PQ improved from 56.0% to 61.4%, with PQ Stuff rising from 57.9% to 70.3%; on Mapillary Vistas, PQ rose from 30.2% to 31.9%, and on PASS semantic segmentation the PRF-based model achieved 55.0% mIoU (Jaus et al., 2021). The paper explicitly frames this as domain robustness rather than panorama-specific overfitting.

"MTPano: Multi-Task Panoramic Scene Understanding via Label-Free Integration of Dense Prediction Priors" extends the notion from a single task to a label-free multi-task panoramic foundation model (Zhang et al., 5 Feb 2026). It projects panoramas into perspective patches, obtains pseudo-labels from off-the-shelf perspective foundation models, and supervises training patch-wise after reprojection (Zhang et al., 5 Feb 2026). Its Panoramic Dual BridgeNet separates rotation-invariant tasks such as depth and segmentation from rotation-variant tasks such as surface normals, using geometry-aware modulation, ERP token mixers, and a dual-branch BridgeNet with gradient truncation to block conflicting gradients (Zhang et al., 5 Feb 2026). Reported results include 75.66 semantic mIoU on Structured3D, 0.0248 depth AbsRel, 3.85° normal mean error, and an ablation in which dual-stream disentanglement and truncated-gradient bridging bring +13% MTL improvement over naive MTL (Zhang et al., 5 Feb 2026).

"PanoSAMic" uses a frozen SAM encoder modified to expose multi-stage features, then fuses RGB, depth, and normal modalities with a spatio-modal fusion module, followed by spherical attention and dual-view fusion to address ERP distortion and left-right edge discontinuity (Chamseddine et al., 12 Jan 2026). It reports 61.57% mIoU and 74.04% mAcc on Stanford2D3DS with RGB-D-N, and 48.43% mIoU on Matterport3D with RGB-D; at the left and right image edges, dual-view processing improves mIoU by approximately 4.9pp relative to a single view (Chamseddine et al., 12 Jan 2026).

For layout estimation, "PanoViT" adapts the Vision Transformer to panoramic imagery with recurrent position embedding, patch sampling tailored to equirectangular images, a frequency-domain edge enhancement module, and a 3D loss (Shen et al., 2022). The paper’s central claim is that panorama-specific positional encoding and geometry-aware supervision are required for the model to exploit global structure and local boundary cues in room layout prediction (Shen et al., 2022).

4. Multi-camera fusion, geometry estimation, and 3D reconstruction

In geometric pipelines, the panoramic visual feature model often becomes a latent coordinate system that simultaneously supports feature aggregation, depth reasoning, and optimization.

"Multi-LVI-SAM" is the clearest explicit formulation. It fuses observations from multiple fisheye cameras, LiDAR, and IMU by projecting visual features onto a panoramic sphere and integrating the resulting constraints into a tightly coupled factor graph optimized with iSAM2 (Zhang et al., 6 Sep 2025). The extrinsic compensation method corrects triangulation inconsistency caused by misalignment between camera frames and the panoramic model’s frame; in the Newer College "Stairs" sequence, RMSE improved by 40.6% when this compensation was enabled (Zhang et al., 6 Sep 2025). With four cameras, total processing time was approximately 2.15× that of a single-camera system rather than scaling linearly (Zhang et al., 6 Sep 2025).

"PanoVGGT" moves from optimization to feed-forward 3D reconstruction. It jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass through a permutation-equivariant Transformer (Guo et al., 18 Mar 2026). The model uses spherical-aware positional embeddings, tri-axis $SO(3)$ augmentation, and stochastic anchoring, while the accompanying PanoCity dataset spans 134 km² across three virtual cities and includes five illumination conditions: Clear, Cloudy, Snow, Fog, and Night (Guo et al., 18 Mar 2026).

"PaGeR" adapts the Depth Anything 3 transformer to unified panoramic geometry estimation by converting panoramas to cubemaps, conditioning on camera intrinsics and extrinsics, and applying cross-face decoder padding (Bozic et al., 25 May 2026). In a single forward pass it predicts scale-invariant depth, metric depth, surface normals, and sky masks for both perspective and omnidirectional images (Bozic et al., 25 May 2026). Reported scale-invariant depth AbsRel values are 9.67 on Matterport3D360, 5.93 on Stanford2D3DS, and 9.36 on ZüriPano; a full 2K panorama is processed in 0.5s per sample on a single GPU with 12.8 GB RAM (Bozic et al., 25 May 2026).

"Pano2World" converts a single indoor panorama into a persistent, renderable 3D Gaussian scene (Li et al., 1 Jul 2026). Its pipeline reconstructs a coarse 3D Gaussian proxy, renders geometrically aligned guidance panoramas at nearby poses, jointly denoises all target views with View-Aware Attention Routing, and then uses a Latent Feature Adapter to distill hidden diffusion features directly into a scene latent for 3D Gaussian decoding (Li et al., 1 Jul 2026). On the reported multi-position panoramic novel-view synthesis benchmark, it achieves 23.56 PSNR, 0.824 SSIM, and 0.230 LPIPS, outperforming the listed baselines (Li et al., 1 Jul 2026).

5. Generative panoramic feature models and world models

A large recent branch of the literature treats panoramic visual features as latent variables for controllable generation, simulation, or world modeling rather than only as descriptors for recognition.

"PanoGen" generates text-conditioned panoramic environments for vision-and-language navigation by captioning Matterport3D rooms with BLIP-2, generating an initial view with Stable Diffusion v2.1, and completing the panorama through recursive outpainting with Stable Diffusion v1.5-inpainting (Li et al., 2023). Each panorama is discretized into 36 images, and downstream visual features are encoded with CLIP-ViT/B-16 for use by the DUET navigation agent (Li et al., 2023). Reported downstream gains include +2.7% SR and +1.9% SPL on the R2R test leaderboard, and a 1.59m gain in goal progress on CVDN (Li et al., 2023).

"ViewPoint" introduces a new panorama representation, the ViewPoint map, together with Pano-Perspective attention that alternates between global panoramic correlation modeling and local perspective-detail reuse from pretrained video diffusion models (Fang et al., 30 Jun 2025). On VBench, it reports 0.8793 subject consistency, 0.5927 imaging quality, 0.9800 motion smoothness, and 0.9083 dynamic degree, outperforming the listed baselines (Fang et al., 30 Jun 2025).

"Canvas360" approaches in-context panoramic generation through geometry-aware pretraining (Feng et al., 9 Jul 2026). Its main mechanisms are parallel depth generation, velocity circular padding, and similarity loss regularization, supported by Canvas360Dataset, a collection of 1M high-quality paired panoramic samples spanning style transfer, inpainting, outpainting, and editing (Feng et al., 9 Jul 2026). The paper reports FAED 2.33, lower than the compared baselines, and the lowest LRCE-RGB, indicating improved panorama-specific fidelity and seam consistency (Feng et al., 9 Jul 2026).

"OmniRoam" frames panoramic video generation as long-horizon scene wandering with a preview stage and a refine stage (Liu et al., 31 Mar 2026). Training uses approximately 2,000 real panoramic videos comprising 5 million frames, together with 1,000 synthetic scenes from InteriorGS (Liu et al., 31 Mar 2026). The paper introduces Loop Consistency as a metric for return-to-start coherence and reports FAED 5.07 at 720p and 81 frames, as well as stable 641-frame loops (Liu et al., 31 Mar 2026).

"PanoWorld: Geometry-Consistent Panoramic Video World Modeling" argues directly that panoramic video generation must be treated as a geometric modeling problem rather than pure visual synthesis (Jiang et al., 14 May 2026). Built on a pre-trained perspective video world model, it adds a depth consistency loss, a trajectory consistency loss, and spherical-geometry-aware adaptation, and trains on PanoGeo, a dataset of over 8,000 panoramic clips with depth, trajectory, and prompt annotations (Jiang et al., 14 May 2026). The reported finding is that geometry-aware regularization dramatically improves 3D-Smooth, Depth-$\sigma$, and Tr-Life while maintaining competitive visual realism (Jiang et al., 14 May 2026).

Two additional lines extend the generative notion of panoramic feature modeling. "QuaDreamer" generates controllable panoramic robot videos using Vertical Jitter Encoding, Scene-Object Controller, and Panoramic Enhancer, and reports downstream multi-object tracking gains from 14.168 to 15.605 HOTA and from -68.4 to -58.3 MOTA, with PTrack 14.17 versus 25.25 for TrackDiffusion (Wu et al., 4 Aug 2025). "OmniX" repurposes 2D flow-matching generative priors for panoramic perception, generation, and completion with a Separate-Adapter cross-modal design; in the reported adapter ablation, Separate-Adapter reaches 21.7 albedo PSNR, 18.2 roughness PSNR, 0.808 depth $\delta$-1.25, and 0.154 depth AbsRel (Huang et al., 30 Oct 2025).

6. Vision-language grounding, reasoning, and embodied use

Panoramic visual feature models have also become interfaces between 3D structure and large multimodal models.

"PanoGrounder" uses multi-modal panoramic scene representations as an intermediate representation for 3D visual grounding (Jung et al., 24 Dec 2025). A three-stage pipeline first places a compact set of panoramic viewpoints using scene layout and geometry, then grounds a text query on each panoramic rendering with a pretrained 2D VLM, and finally lifts and fuses per-view predictions into a 3D bounding box (Jung et al., 24 Dec 2025). Each selected viewpoint yields an RGB panorama, a depth panorama, and a semantic feature map; scene coverage is achieved with an average of approximately 2.4 cameras per scene (Jung et al., 24 Dec 2025). The paper reports state-of-the-art results on ScanRefer and Nr3D, together with stronger generalization to unseen 3D datasets and text rephrasings (Jung et al., 24 Dec 2025).

"PanoEnv" turns panoramic reasoning into a benchmarked 3D spatial intelligence problem (Lin et al., 25 Feb 2026). Its PanoEnv-QA benchmark contains 14.8K questions across five categories grounded in RGB, depth, segmentation, and 3D annotations, and evaluation of 14 state-of-the-art VLMs yields only 49.34% overall accuracy and 8.36% open-ended accuracy (Lin et al., 25 Feb 2026). The paper then proposes a Group Relative Policy Optimization post-training scheme with geometry-aware rewards and a two-stage curriculum; the reported 7B model reaches 52.93% overall accuracy and 14.83% open-ended accuracy, with Q-Score 6.24 and P-Score 5.95 (Lin et al., 25 Feb 2026). These results quantify how difficult panoramic spatial reasoning remains even for strong VLMs.

A domain-specific variant appears in panoramic medical imaging. "OralGPT-Plus" addresses panoramic dental radiographs with an agentic VLM that uses Zoom-In and Mirror-In tools in a thought-action-observation loop, trained on DentalProbe, a five-thousand-image dataset with expert-curated diagnostic trajectories, and evaluated on MMOral-X, which contains 300 open-ended questions and region-level annotations (Fan et al., 6 Mar 2026). Here the panoramic feature model is less about spherical geometry than about long horizontal context, bilateral symmetry, and iterative localized reinspection.

7. Conceptual synthesis and open directions

The literature suggests that a panoramic visual feature model is best understood not as a single architecture, but as a recurring design pattern. First, it must preserve global continuity across the full visual sphere or long panoramic axis. Second, it must encode explicit geometry—through spheres, cubemaps, ray descriptors, depth, or cross-view constraints—rather than relying on purely planar appearance statistics. Third, it must often bridge a training-deployment mismatch, because panoramic data and dense panoramic labels remain scarcer than their perspective counterparts (Jaus et al., 2021, Zhang et al., 5 Feb 2026, Feng et al., 9 Jul 2026).

A second recurring theme is reuse of perspective priors. PRF transfers pinhole-trained representations through unsupervised contrastive pretraining; MTPano generates patch-wise pseudo-labels from perspective foundation models; PaGeR lifts a perspective 3D foundation model through cubemaps and mixed training; OmniX adapts pretrained 2D generative priors with lightweight cross-modal adapters (Jaus et al., 2021, Zhang et al., 5 Feb 2026, Bozic et al., 25 May 2026, Huang et al., 30 Oct 2025). This suggests that panoramic progress has been driven less by training wholly new models from scratch than by systematic geometric adaptation of strong perspective models.

A third theme is the growing tension between visual quality and geometric consistency. Perspective-derived panoramic generators can produce visually plausible outputs while drifting in depth, motion, or cross-view correspondence. PanoWorld makes this tension explicit by introducing depth and trajectory regularizers, and Pano2World resolves a related problem by transferring multi-view diffusion features directly into a 3D Gaussian latent rather than decoding and re-encoding them through pixels (Jiang et al., 14 May 2026, Li et al., 1 Jul 2026). A plausible implication is that future panoramic feature models will be evaluated increasingly as world models rather than only as image generators.

The concept therefore occupies a strategic position in current computer vision and embodied AI. It links sensor fusion, dense prediction, reconstruction, generation, navigation, and multimodal reasoning under a shared requirement: representing the surrounding world in a form that respects panoramic geometry while remaining compatible with scalable learning systems.