EDA-PSeg: Adaptive Panoramic Segmentation

• The paper introduces EDA-PSeg, which extrapolates segmentation from pinhole images to full 360° panoramas using Euler-Margin Attention and Graph Matching Adapter.
• It leverages multi-level adversarial alignment, distortion-aware attention, and prototype memory to handle severe field-of-view shifts and projection distortions.
• The framework expands domain adaptation by incorporating open-set learning to robustly align both known and unknown classes in panoramic imagery.

Searching arXiv for the cited panoramic domain adaptation papers to ground the article in the latest relevant literature. Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) denotes learning semantic segmentation from labeled local perspective or pinhole images and adapting to unlabeled full $360^\circ$ panoramic images under severe field-of-view, camera, and projection shifts; in the open-set formulation, the target label space contains both known and unknown classes, $Y_t = K \cup U$ (Zheng et al., 16 Mar 2026). Earlier panoramic UDA work formalized the same core transfer as Pinhole$\rightarrow$Panoramic Domain Adaptation (P2PDA), in which a segmentation network $G$ is trained on labeled source images from conventional pinhole cameras and evaluated on panoramic targets for which no labels are available during training (Ma et al., 2021). In current usage, EDA-PSeg therefore designates both a specific 2026 framework based on Euler-Margin Attention and Graph Matching Adapter and a broader research line spanning adversarial alignment, distortion-aware attention, prototype alignment, and memory-based adaptation for extrapolating segmentation beyond the source camera’s field of view (Zheng et al., 16 Mar 2026).

1. Problem formulation and scope

The canonical semantic EDA-PSeg setting uses a labeled source domain $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$ of pinhole or perspective images and an unlabeled target domain $D_t = \{x_j^t\}_{j=1}^{N_t}$ of equirectangular panoramic images (Zheng et al., 16 Mar 2026). DensePASS instantiates this as unsupervised domain adaptation for panoramic semantic segmentation, with a source domain of conventional pinhole camera images and a target domain of $360^\circ$ panoramic images, and explicitly states that no panoramic labels are used during training (Ma et al., 2021). In the open-set formulation introduced later, the source label space is $Y_s = K$, while the target label space is $Y_t = K \cup U$, where $U$ denotes novel categories never seen in the source (Zheng et al., 16 Mar 2026).

The basic supervised term is the standard per-pixel cross-entropy on the source domain. DensePASS writes it as

$Y_t = K \cup U$0

with $Y_t = K \cup U$1 and $Y_t = K \cup U$2 indexing pixels (Ma et al., 2021). Its adversarial formulation attaches discriminators $Y_t = K \cup U$3 at levels $Y_t = K \cup U$4 and optimizes

$Y_t = K \cup U$5

with multi-level alignment performed on features or logits depending on the adaptation module (Ma et al., 2021). The 2026 open-set EDA-PSeg framework instead uses

$Y_t = K \cup U$6

combining supervised source segmentation, DACS-style cross-domain mixing with pseudo-labels, and graph matching with unknown-class separation (Zheng et al., 16 Mar 2026).

A central scope distinction is that EDA-PSeg concerns semantic segmentation under panoramic deployment. Closely related work addresses panoptic segmentation under domain shift, but that is a different task definition. EDAPS studies domain-adaptive panoptic segmentation with a shared transformer encoder and task-specific decoders, but it is not panoramic (Saha et al., 2023). By contrast, WildPPS-based work addresses panoramic panoptic segmentation and extrapolation from pinhole data, yet does so without unlabeled target panoramas and frames the problem as target-free feature transfer rather than UDA in the DensePASS sense (Jaus et al., 2021).

2. Geometric FoV shift, projection distortion, and extrapolation

EDA-PSeg is extrapolative because the source and target differ not merely in appearance but in image formation. DensePASS characterizes the pinhole-to-panoramic shift through three coupled factors: panoramic images cover $Y_t = K \cup U$7 horizontally rather than a narrow FoV; equirectangular panoramas introduce strong, position-dependent distortions, especially near the poles and along the image borders; and panoramic images are long, horizontally wrapping images in which objects recur at different azimuths with different distortions (Ma et al., 2021). The later DensePASS formulation further emphasizes that side and rear views never occur in source images, so transferring beyond the source FoV is extrapolative adaptation rather than ordinary in-domain generalization (Zhang et al., 2021).

The 2026 EDA-PSeg paper makes the geometry explicit. For a pixel $Y_t = K \cup U$8 in an equirectangular image of size $Y_t = K \cup U$9,

$\rightarrow$0

with $\rightarrow$1 the longitude and $\rightarrow$2 the latitude, and the corresponding unit 3D ray is

$\rightarrow$3

Viewpoint shifts are parameterized by Euler angles $\rightarrow$4, so a rotated ray is $\rightarrow$5 (Zheng et al., 16 Mar 2026). This formulation is used to explain why source pinhole supervision and target panoramic testing are coupled by both viewpoint change and non-uniform spherical sampling.

Several later methods operationalize this geometry in different ways. DATR argues that neighboring pixel regions in ERP incur less distortion than long-range interactions and therefore replaces global self-attention with neighborhood attention plus trainable Relative Positional Encoding (Zheng et al., 2023). Trans4PASS and Trans4PASS+ instead learn object deformations and panoramic distortions through Deformable Patch Embedding and Deformable MLP modules, with Trans4PASS+ describing distortion handling as applying “whenever and wherever” in shallow and deep stages (Zhang et al., 2022, Zhang et al., 2022). OmniSAM treats the panorama as an overlapping patch sequence and uses SAM2 memory to recover cross-patch correspondences, thereby embedding cross-FoV dependencies without requiring a single monolithic panoramic attention map (Zhong et al., 10 Mar 2025).

A common misconception is that pinhole-to-panorama transfer is primarily a style-shift problem. DensePASS quantified mIoU drops of approximately $\rightarrow$6–$\rightarrow$7 points across more than fifteen Cityscapes-trained models when evaluated directly on DensePASS panoramas, indicating that projection geometry and spatial-layout change are dominant failure modes rather than a minor appearance mismatch (Ma et al., 2021).

3. Architectural mechanisms and methodological development

The literature has converged on a small set of recurrent mechanisms: multi-level adversarial alignment, distortion-aware or FoV-aware attention, prototype or graph alignment, and pseudo-label-based target adaptation. DensePASS introduced a generic framework around SDAM, ADAM, and RCDAM; DATR replaced global attention with local distortion-aware attention and class-wise feature aggregation; Trans4PASS and Trans4PASS+ centered the adaptation pipeline on distortion-aware transformers and mutual prototypes; OmniSAM imported memory mechanisms from SAM2; and the 2026 EDA-PSeg framework unified viewpoint-invariant attention with open-set graph matching (Ma et al., 2021, Zheng et al., 2023, Zhang et al., 2022, Zhong et al., 10 Mar 2025, Zheng et al., 16 Mar 2026).

Method	Core mechanism	Representative reported result
DensePASS / P2PDA	SDAM, ADAM, RCDAM, multi-level adversarial alignment	DANet: 28.50 $\rightarrow$8 39.76 mIoU on DensePASS (Ma et al., 2021)
DATR	Distortion-aware local attention, RPE, CFA memory bank	56.81% mIoU on Cityscapes$\rightarrow$9DensePASS (Zheng et al., 2023)
Trans4PASS+	DPE, DMLPv2, MPA, SAM rectification	59.43% mIoU on DensePASS (Zhang et al., 2022)
OmniSAM	SAM2 memory, patch sequences, FoV-based prototypes	62.46% on CS13$G$0DP13 (Zhong et al., 10 Mar 2025)
EDA-PSeg	EMA and GMA for open-set FoV adaptation	C2D: Common 56.81, Private 18.86, H-Score 28.32 (Zheng et al., 16 Mar 2026)

DensePASS places discriminators before and after the decoder. At $G$1, a discriminator may use the Regional Context Domain Adaptation Module; at $G$2, it may use the Attentional Domain Adaptation Module. SDAM can be plugged into both branches as a general output- or feature-space adversarial aligner (Ma et al., 2021). ADAM applies self-attention before adversarial alignment and uses the standard scaled dot-product form

$G$3

so that alignment is performed on context-aggregated, attention-weighted features rather than raw features (Ma et al., 2021). RCDAM adds a two-stage path built from a Region Construction Block and a Region Interaction Block, with region representatives aggregating and redistributing context in a way tailored to the panoramic horizontal layout (Ma et al., 2021).

DATR takes a different position. It argues that local neighborhoods in ERP are less distorted than long-range pairings, so the deepest stage uses local neighborhood attention with a trainable Relative Positional Encoding injected into the value pathway. Its Class-wise Feature Aggregation module maintains source and target class centers in a memory bank and aligns them through

$G$4

while target pseudo-labels are generated by hard argmax without a confidence threshold (Zheng et al., 2023).

Trans4PASS and Trans4PASS+ emphasize learnable distortion compensation at the feature-token level. Deformable Patch Embedding predicts bounded spatial offsets

$G$5

with $G$6 used to stabilize training, while Mutual Prototypical Adaptation updates class prototype memory with momentum $G$7 and distills features toward prototype-constructed maps using temperature-scaled KL divergence (Zhang et al., 2022). Trans4PASS+ extends this line with DMLPv2 and SAM-based pseudo-label rectification, retaining the mutual prototype objective

$G$8

with $G$9, $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$0, and $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$1 (Zhang et al., 2022).

The 2026 EDA-PSeg framework introduces two modules explicitly aimed at open-set extrapolation. Euler-Margin Attention forms complex features by pairing channels into real and imaginary parts, writes them as $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$2, and modulates both amplitude and phase:

$D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$3

The Graph Matching Adapter samples class nodes and prototypes, completes missing nodes from a per-class memory bank, computes affinities, and performs Sinkhorn-relaxed matching while penalizing known-unknown correlations through orthogonality-style regularization (Zheng et al., 16 Mar 2026). This is the first formulation in the supplied literature to make unknown-class separation an explicit objective rather than a byproduct of closed-set adaptation.

OmniSAM is distinctive in moving the adaptation problem into a patch-sequence regime. Panoramas are divided into overlapping $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$4 patches, with DensePASS resized to $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$5 and processed as a $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$6-frame sequence with stride $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$7. Source and target prototypes are computed per FoV step $D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$8,

$D_s = \{(x_i^s, y_i^s)\}_{i=1}^{N_s}$9

and aligned by

$D_t = \{x_j^t\}_{j=1}^{N_t}$0

Pseudo-labels are updated dynamically using bidirectional sequence processing and a unanimity-plus-confidence rule (Zhong et al., 10 Mar 2025).

4. Datasets, protocols, and benchmark structure

DensePASS established the benchmark infrastructure for semantic EDA-PSeg. It uses Cityscapes as the labeled pinhole source, with $D_t = \{x_j^t\}_{j=1}^{N_t}$1 labeled training images and $D_t = \{x_j^t\}_{j=1}^{N_t}$2 classes, and a panoramic DensePASS target consisting of $D_t = \{x_j^t\}_{j=1}^{N_t}$3 unlabeled training panoramas and $D_t = \{x_j^t\}_{j=1}^{N_t}$4 labeled test panoramas at resolution $D_t = \{x_j^t\}_{j=1}^{N_t}$5, collected from Google Street View and rendered via equirectangular projection (Ma et al., 2021). The labeled test data and unlabeled train data were drawn from different sets of cities, and the class taxonomy was aligned exactly to the $D_t = \{x_j^t\}_{j=1}^{N_t}$6 Cityscapes categories (Ma et al., 2021).

The benchmark space then broadened. SynPASS introduced $D_t = \{x_j^t\}_{j=1}^{N_t}$7 synthetic panoramic RGB images with pixel-wise labels at resolution $D_t = \{x_j^t\}_{j=1}^{N_t}$8, generated in CARLA and intended for Synthetic-to-Real adaptation toward DensePASS (Zhang et al., 2022). The 2026 open-set EDA-PSeg framework organized four benchmark settings: C2D, Cityscapes$D_t = \{x_j^t\}_{j=1}^{N_t}$9DensePASS; S2D, SynPASS$360^\circ$0DensePASS; G2S, GTA$360^\circ$1SynPASS; and S2A, SynPASS$360^\circ$2ACDC (Zheng et al., 16 Mar 2026). These settings mix Pin2Pan, Pan2Pan, Pan2Pin, synthetic-to-real, and weather shifts within a unified open-set evaluation protocol (Zheng et al., 16 Mar 2026).

Evaluation metrics depend on whether the protocol is closed-set or open-set. DensePASS and most pre-2026 methods report mIoU on the panoramic test set (Ma et al., 2021). The 2026 EDA-PSeg framework instead reports mIoU on Common classes, mIoU on Private classes, and H-Score as a combined indicator in open-set adaptation (Zheng et al., 16 Mar 2026). OmniSAM uses subset protocols such as SPin8$360^\circ$3SPan8 and CS13$360^\circ$4DP13, with mIoU reported on the corresponding class subsets (Zhong et al., 10 Mar 2025).

This benchmark evolution matters because it changes what counts as successful adaptation. In DensePASS, the target taxonomy is deliberately aligned to Cityscapes, so the emphasis falls on geometric and contextual extrapolation (Ma et al., 2021). In the open-set setting, semantic novelty becomes part of the task, and success requires not only recovering known classes under FoV shift but also preventing unknown target categories from collapsing into known source labels (Zheng et al., 16 Mar 2026).

5. Empirical results and ablation evidence

DensePASS quantified the severity of the pinhole-to-panorama gap. Across more than fifteen Cityscapes-trained models, mIoU dropped by approximately $360^\circ$5–$360^\circ$6 points when models were evaluated directly on DensePASS panoramas. Examples reported include DeepLabV3+ (ResNet-101), $360^\circ$7; OCRNet (HRNetV2p-W48), $360^\circ$8; and ERFNet, $360^\circ$9 (Ma et al., 2021). This baseline established panoramic semantic segmentation as a failure case for standard source-only training rather than an incremental extension of street-scene segmentation.

Within the DensePASS framework, attention-augmented context exchange yielded consistent gains. For FANet, source-only performance was $Y_s = K$0 mIoU; SDAM at $Y_s = K$1 and $Y_s = K$2 raised this to $Y_s = K$3; ADAM at $Y_s = K$4 with SDAM at $Y_s = K$5 raised it to $Y_s = K$6; and SDAM+ADAM at $Y_s = K$7 with SDAM at $Y_s = K$8 reached $Y_s = K$9 (Ma et al., 2021). For DANet, source-only performance was $Y_t = K \cup U$0; SDAM at both levels produced $Y_t = K \cup U$1; SDAM+ADAM at $Y_t = K \cup U$2 with RCDAM at $Y_t = K \cup U$3 reached $Y_t = K \cup U$4 (Ma et al., 2021). The extended P2PDA formulation later added FCDAM and uncertainty-aware self-training, reporting DANet improvements from $Y_t = K \cup U$5 source-only to $Y_t = K \cup U$6 with extra WildDash source data and to $Y_t = K \cup U$7 after a second self-training run (Zhang et al., 2021).

Transformer-based and prototype-based methods subsequently pushed closed-set DensePASS performance beyond the adversarial baseline. Trans4PASS with MPA+SSL reached $Y_t = K \cup U$8 mIoU on DensePASS, and multi-scale inference raised this to $Y_t = K \cup U$9, reported as improving over prior state of the art by $U$0 mIoU versus P2PDA-SSL at $U$1 (Zhang et al., 2022). DATR-S reached $U$2 mIoU on Cityscapes$U$3DensePASS, outperforming DAFormer at $U$4 and Trans4PASS-S at $U$5, while reporting an approximately $U$6 parameter reduction for DATR-M relative to Trans4PASS-S (Zheng et al., 2023). Trans4PASS+ with MPA + SAM + multi-scale inference reached $U$7 mIoU on DensePASS and was reported as surpassing prior state of the art by $U$8 over P2PDA-SSL (Zhang et al., 2022). OmniSAM later reported $U$9 on CS13$Y_t = K \cup U$00DP13 and $Y_t = K \cup U$01 on SPin8$Y_t = K \cup U$02SPan8, with ablations showing that supervised only, SSL, FPA, and dynamic pseudo-label updates each contributed measurable gains (Zhong et al., 10 Mar 2025).

The 2026 open-set EDA-PSeg results changed the evaluation lens. On C2D, it reported Common $Y_t = K \cup U$03, Private $Y_t = K \cup U$04, and H-Score $Y_t = K \cup U$05; on S2D, Common $Y_t = K \cup U$06, Private $Y_t = K \cup U$07, H-Score $Y_t = K \cup U$08; on G2S, Common $Y_t = K \cup U$09, Private $Y_t = K \cup U$10, H-Score $Y_t = K \cup U$11; and on S2A, Common $Y_t = K \cup U$12, Private $Y_t = K \cup U$13, H-Score $Y_t = K \cup U$14 (Zheng et al., 16 Mar 2026). Closed-set baselines such as DAFormer and HRDA were reported to fail to recognize unknowns, with Private approximately zero on C2D, whereas adding GMA alone improved C2D from $Y_t = K \cup U$15 to $Y_t = K \cup U$16, EMA alone gave $Y_t = K \cup U$17, and the full model reached $Y_t = K \cup U$18 (Zheng et al., 16 Mar 2026). This suggests that open-set panoramic adaptation is not reducible to closed-set domain alignment plus threshold tuning.

Across papers, the ablation evidence converges on a consistent pattern. Multi-level alignment matters in DensePASS; local or distortion-aware attention helps when global attention is too sensitive to ERP warping; prototype memory improves target consistency; and pseudo-label refinement becomes more effective when coupled to structural or geometric constraints rather than used as a standalone self-training heuristic (Ma et al., 2021, Zheng et al., 2023, Zhang et al., 2022, Zheng et al., 16 Mar 2026).

6. Limitations, misconceptions, and research directions

The most persistent limitation is incomplete geometric modeling. DensePASS explicitly reports no explicit $Y_t = K \cup U$19 wrap-around handling and notes that the method treats panoramas as standard images, leaving potential seams and mismatched unaries at image borders (Ma et al., 2021). The 2026 open-set EDA-PSeg framework still evaluates on full equirectangular panoramas but identifies sensitivity to cropping and thresholds, extra compute from graph construction and Sinkhorn iterations, and continued difficulty under severe polar distortions or extremely scarce unknown instances (Zheng et al., 16 Mar 2026). DATR likewise notes that severe pole distortions remain challenging and suggests dynamic neighborhood sizing or adaptive windows as future work (Zheng et al., 2023).

A second misconception is that stronger panoramic adaptation necessarily requires hand-crafted spherical priors. DATR explicitly positions its method as simpler, easier to implement, and more computationally efficient by avoiding manually encoded geometric constraints and multi-branch designs (Zheng et al., 2023). At the same time, Trans4PASS+ and DensePASS both show that geometry-aware inductive bias remains useful when it is learned rather than hard-coded, whether through deformable patch sampling or region-centric context exchange (Zhang et al., 2022, Ma et al., 2021). A plausible implication is that the field has moved from explicit spherical operators toward learned distortion compensation, not toward geometry-agnostic modeling.

A third point of clarification concerns task boundaries. EDAPS studies domain-adaptive panoptic segmentation with a shared MiT-B5 transformer encoder and reports strong mPQ gains on SYNTHIA$Y_t = K \cup U$20Cityscapes and SYNTHIA$Y_t = K \cup U$21Mapillary Vistas, but panoramas are not covered in that work (Saha et al., 2023). WildPPS-based methods study panoramic panoptic segmentation and show that source-only dense contrastive pretraining can improve PQ by $Y_t = K \cup U$22–$Y_t = K \cup U$23 or by more than $Y_t = K \cup U$24 points depending on the architecture, yet they do not rely on unlabeled target panoramas and therefore instantiate a target-free extrapolative transfer regime rather than the standard UDA formulation of DensePASS or EDA-PSeg (Jaus et al., 2022, Jaus et al., 2021). The distinction is substantive because semantic EDA-PSeg typically assumes unlabeled target panoramas for adaptation, whereas the WildPPS line does not.

The most frequently proposed future directions are remarkably consistent across papers: multi-source extrapolation; explicit wrap-around handling through circular padding or spherical operators; pseudo-labeling or self-training that is more robust to rare categories; geometry-aware attention and convolutions; and stronger target-structure modeling through class-wise, prototype-based, or graph-based alignment (Ma et al., 2021, Zhang et al., 2022, Zheng et al., 16 Mar 2026). The 2026 EDA-PSeg paper also names source-free and test-time adaptation variants for panoramic open-world scenarios, while DensePASS-related analyses point to multi-source enrichment and projection-aware extensions as natural next steps (Zheng et al., 16 Mar 2026, Ma et al., 2021). Together, these directions define the current frontier of EDA-PSeg: robust segmentation under simultaneous FoV extrapolation, geometric distortion, and semantic novelty.