---
title: 'Depth Any Panoramas (DAP): 360° Depth Estimation'
url: https://www.emergentmind.com/topics/depth-any-panoramas-dap
type: topic
---

# Depth Any Panoramas (DAP): 360° Depth Estimation

Depth Any Panoramas (DAP) refers to methodologies, foundational models, and complete pipelines for estimating dense, metric, and/or scale-invariant depth from 360-degree (panoramic or omnidirectional) images. These approaches address the unique geometric, data, and deployment challenges of panoramic imagery, enabling applications that require high-fidelity global 3D structure capture, such as robotics, AR/VR, and autonomous navigation.

## 1. Defining DAP: Scope, Motivation, and Distinctions

Depth Any Panoramas (DAP) encompasses a family of methods designed to produce all-angle, per-pixel depth maps from full-sphere images, typically in equirectangular projection. Unlike conventional depth-from-perspective pipelines—where models are trained on rectilinear images and output depth for a limited field of view—DAP must contend with greater geometric distortion (particularly near the poles), boundary wraparound, and wide domain shifts between synthetic and real-world data. The field has matured from early handcrafted fusion approaches to fully end-to-end vision transformer (ViT) pipelines that exhibit strong zero-shot performance across diverse scene types, domains, and capture setups [2512.16913][2512.22819].

DAP is distinguished by (i) its target of complete 360° × 180° coverage, (ii) explicit geometric handling of equirectangular, spherical, and cylindrical projections, (iii) a focus on both metric and scale-invariant depth recovery, and (iv) pipeline and dataset scalability favoring foundational model paradigms.

## 2. Datasets and Synthetic Data Curation for Panoramic Depth

DAP model success is predicated on access to large-scale, diverse, and high-quality panoramic RGB–depth datasets. Early efforts used synthetic environments such as Structured3D and PanoSUNCG or multi-view reconstructions on cube-mapped or equirectangular imagery, but these frequently suffered scale-ambiguity and limited real-world coverage.

Recent contributions have greatly expanded the available training resources:
- **360° in the Wild** [2406.18898]: 25,000 real-world panoramas spanning indoor, outdoor, and mannequin-frozen scenes, annotated with depth via fused multi-view stereo (COLMAP) on synthetic cube-map faces. This resource supports both single-image depth estimation and view synthesis, with depth ranges 0.5 m–30+ m, and standardized pose conventions.
- **DAP-2M** [2512.16913]: Unifies 18k Structured3D (synthetic indoor), 90k AirSim360 (high-fidelity synthetic outdoor), 200k DiT360 (text-to-image), and 1.7M unlabeled real panoramas, using a three-stage pseudo-label curation pipeline—comprising geometry-grounded initialization, realism-driven pseudo-labeling, and progressive fine-tuning—to generate robust, geometry-consistent depth supervision across 2 million panoramas.
- **DA² curation engine** [2509.26618]: Converts perspective RGB–D datasets to full ERP panoramas via a perspective-to-equirectangular projection, filling missing sides/top/bottom via spherical out-painting models, resulting in over 543k high-quality RGB–D panorama pairs, bringing the scale to 607k when combined with existing data.

All recent DAP pipelines leverage such heterogeneous sources, with sophisticated domain adaptation (e.g., realism-invariant labelers, confidence-driven curation), enabling high generalization even under strong cross-domain and scale shifts.

## 3. Core Model Architectures and Geometric Formulations

DAP architectures bifurcate into several specialized classes, each encoding panoramic geometry and distortion in unique ways:

**A. Spherical- and Distortion-Aware Convolutional Approaches**
- **DAMO (Distortion-Aware Monocular Omnidirectional)** leverages a ResNet-50 backbone, inserting deformable convolution and strip pooling modules to address ERP-induced stretching and distortion—especially near the poles. Spherical-aware weight matrices rebalance supervision according to equirectangular surface area [2010.08942].

**B. Multi-Projection and Fusion-Based Solutions**
- **SphereFusion** [2502.05859] processes each panorama through parallel equirectangular (2D CNN) and spherical-mesh ResNet branches, projecting features to a shared spherical mesh and fusing them with gated attention at each resolution. The final prediction is performed in the spherical domain, balancing the texture detail captured in 2D with geometric fidelity from the mesh branch.

**C. Vision Transformer (ViT) DAP Foundations**
- **DAP Foundation Model** [2512.16913] and **DA² (SphereViT)** [2509.26618] employ ViTs (e.g., DINOv3-L, DINOv2-ViT-L) to learn global panoramic representations. DAP couples a metric depth head with a plug-and-play range mask, while SphereViT introduces fixed spherical coordinate embeddings through cross-attention, enhancing geometric consistency across latitude/longitude. Both models achieve end-to-end inference, high throughput, and strong zero-shot generalization.

**D. Scale-Invariant Transformations**
- **DA360** [2512.22819] adapts Depth Anything V2, learning a global shift parameter from the ViT class token to convert affine-invariant log-disparity to scale-invariant depth. Circular padding in the DPT decoder enforces spherical continuity, mitigating seam artifacts at ERP boundaries.

**E. Stereo and LiDAR Fusion Pipelines**
- **MCPDepth** [2408.01653] maps ERP frames to cylindrical panoramas for pairwise stereo matching using standard convolutional stereo networks extended with circular attention, then fuses depth maps in the ERP domain via lightweight U-Net architectures. This approach is compatible with embedded deployment due to its avoidance of custom kernels.
- **LiDAR–Fisheye Fusion** [2010.14270] utilizes multi-camera rigs and LiDAR, projecting 3D points onto wide-FOV images, upsampling sparse depths through local adaptive least squares, and seamless blending via graph cuts and multi-band blending prior to mapping to spherical coordinates.

| Model/Class              | Projection Domain(s)            | Key Innovations                                  |
|--------------------------|----------------------------------|--------------------------------------------------|
| DAMO                     | Equirectangular                  | Deformable conv, strip pooling, spherical loss    |
| SphereFusion             | ERP & Spherical mesh             | Gated fusion, mesh cache, dual-branch arch        |
| DAP, DA², DA360          | ERP, Spherical (ViT)             | Spherical embedding, SI/metric loss, circular pad |
| MCPDepth                 | Cylindrical, ERP                 | Stereo+fusion, circular attn, deployment ready    |

## 4. Loss Functions, Optimization, and Geometric Consistency

Depth estimation from 360-degree imagery faces unique losses from ERP sampling non-uniformity, ambiguity in metric scale, and inconsistency in high-frequency structure. Representative DAP loss formulations include:

- **SILog Loss** (Scale-Invariant Log, as in MiDaS) [2512.16913][2512.22819]: Enforces correct relative structure under unknown scale/shifts.
- **Dense-Fidelity Loss**: Measures Gram-matrix similarity on tangent-plane projections to penalize global geometric fidelity [2512.16913].
- **Gradient and Normal Losses**: Focused on preserving sharp edges ($\mathcal{L}_{\mathrm{grad}}$) and surface normals ($\mathcal{L}_{\mathrm{normal}}$) for fine spatial accuracy.
- **Point-Cloud Losses**: Penalize 3D Cartesian errors after spherical unwrapping [2512.16913].
- **Spherical-Aware Pixel Weighting**: E.g., $W(x,y) = 1 - \cos \phi(x,y)$ in DAMO balances loss contributions over latitude [2010.08942].
- **Plug-and-Play Range Masks**: Binary masks for specified distance bands ($M_r$) facilitate metric masking and loss restriction [2512.16913].

Optimization incorporates data-in-the-loop feedback, curriculum learning across depth ranges, and robust pseudo-label selection. Recent models achieve stable, generalizable metric depth with per-benchmark AbsRel reductions of 25–50% over earlier state-of-the-art [2512.22819][2512.16913].

## 5. Evaluation Protocols and Comparative Results

Panoramic depth models are evaluated across several large-scale benchmarks with standardized metrics. Key datasets and metrics include:

- **Stanford2D3D**: Indoor, real-world panorama suite.
- **Matterport3D**: Large diverse indoor scenes.
- **Deep360, Metropolis**: Synthetic and real outdoor panoramic sets (Metropolis includes 3,000 panoramic frames with LiDAR/MVS ground truth).
- **360° in the Wild**: 11k frame-depth pairs for training; held-out splits for evaluation [2406.18898].

Metrics routinely reported are AbsRel, RMSE, $\delta_1$ (percentage of pixels with error ratio $<1.25$), and derivatives. For example, DAP Foundation Model reports Stanford2D3D AbsRel = 0.0921 (↓36% versus prior SOTA), RMSE = 0.3820, and Deep360 AbsRel = 0.0659 for zero-shot predictions [2512.16913]. DA360 ViT-L achieves AbsRel = 0.0793 (Matterport3D), 0.0710 (Stanford2D3D), 0.2011 (Metropolis), consistently outperforming PanDA and previous transformer-based models [2512.22819].

Trade-offs are documented for real-time efficiency (SphereFusion: 17 ms per 512×1024 panorama) versus transformer-based approaches (60+ ms), as well as ablations demonstrating the necessity of global shift parameters, circular padding, and cross-attention spherical embeddings for peak accuracy.

## 6. Applications and Deployment Considerations

DAP methods have enabled advances in multiple real-world avenues:
- **Localization and Mapping**: Robust metric depth allows for 3D model construction, SLAM, and robot navigation in environments where monocular cues alone are insufficient [2512.16913].
- **AR/VR**: Accurate panoramic depth supports immersive environment reconstruction, real-time scene understanding, and synthetic view generation.
- **Autonomous Vehicles**: Efficient depth pipelines with low-latency inference (e.g., SphereFusion, MCPDepth) are suitable for embedded, resource-constrained systems [2502.05859][2408.01653].
- **Indoor Scene Completion and Outpainting**: Diffusion-based pipelines such as PanoDiffusion perform joint RGB-D outpainting for reconstruction and semantic SLAM when wide-FOV data is incomplete [2307.03177].

Practical deployment is facilitated by models designed with efficiency (ONNX/TensorRT compatibility [2408.01653]), end-to-end operation (SphereViT, DAP), and explicit distortion handling, enabling hardware-agnostic rollout in fielded systems.

## 7. Limitations, Challenges, and Research Frontiers

Despite progress, prominent limitations are noted:
- **Domain Discrepancy and Pseudo-Label Quality**: Generalization across synthetic/real, indoor/outdoor domains depends on pseudo-label curation quality and coverage [2512.16913]. Out-of-distribution scenes still present accuracy challenges.
- **Seam Artifacts and Resolution**: ERP wraparound remains a source of visible seams; current training resolutions (e.g., 512×1024) may lose fine detail or introduce boundary errors [2509.26618][2512.22819].
- **Scale Ambiguity and Metricity**: Achieving true metric depth remains contingent upon sufficient training diversity, reliable ground truth, and proper loss design. Some models remain only scale-invariant in practice outside robustly labeled domains.
- **Computational Footprint and Resource Demands**: Transformer-based foundations incur significant compute cost for training, though inference has become tractable for most architectures.

Areas for future research include integration of temporal/multi-view constraints, self-supervised adaptation, spherical convolution for seamless geometry, and joint semantic-deep representation learning [2512.16913][2509.26618].

---

References:
- "Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation" [2512.16913]
- "Depth Anything in $360^\circ$: Towards Scale Invariance in the Wild" [2512.22819]
- "DA$^2$: Depth Anything in Any Direction" [2509.26618]
- "SphereFusion: Efficient Panorama Depth Estimation via Gated Fusion" [2502.05859]
- "MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas" [2408.01653]
- "360 in the Wild: Dataset for Depth Prediction and View Synthesis" [2406.18898]
- "Distortion-aware Monocular Depth Estimation for Omnidirectional Images" [2010.08942]
- "PanoDiffusion: 360-degree Panorama Outpainting via Diffusion" [2307.03177]
- "A Method of Generating Measurable Panoramic Image for Indoor Mobile Measurement System" [2010.14270]

Source: https://www.emergentmind.com/topics/depth-any-panoramas-dap