Panoramic Dual BridgeNet Architecture

• Panoramic Dual BridgeNet is a dual-stream, geometry-aware multi-task model integrated in MTPano, addressing ERP distortions and task interference.
• It decomposes dense predictions into rotation-invariant and rotation-variant tasks, using a novel bridge mechanism with truncated cross-gradient flow.
• The system leverages label-free supervision via pseudo-label projections to achieve state-of-the-art performance in panoramic dense scene understanding.

Panoramic Dual BridgeNet, often abbreviated PD-BridgeNet, is the core multi-task prediction architecture inside MTPano, a system for label-free panoramic dense scene understanding. It is designed to address three coupled difficulties of panoramic learning in equirectangular projection (ERP): severe latitude-dependent distortion, task interference across heterogeneous dense predictions, and the mismatch between panoramic geometry and perspective-pretrained representations. Within MTPano, PD-BridgeNet operates after a pseudo-label generation stage that projects unlabeled panoramas into perspective patches, applies off-the-shelf perspective foundation models, and reprojects their outputs as patch-wise supervision. The architecture then learns joint prediction of semantic segmentation, depth estimation, and surface normal estimation, augmented by image gradient, Edge Distance Field (EDF), and metric point map auxiliary tasks, through disentangled invariant and variant streams coupled by a bridge mechanism with truncated cross-stream gradients (Zhang et al., 5 Feb 2026).

1. Position within panoramic scene understanding

PD-BridgeNet is not a standalone training recipe but the learned multi-task panoramic model inside MTPano. MTPano comprises two major parts: a label-free training pipeline and PD-BridgeNet itself. The first part generates supervision by projecting panoramas into perspective crops, running strong perspective dense priors on those crops, and reprojecting their predictions back into ERP as patch-wise pseudo-labels. The second part, PD-BridgeNet, learns from those pseudo-labels and predicts multiple dense outputs jointly (Zhang et al., 5 Feb 2026).

The architecture is motivated by a specific failure mode of naive panoramic multi-task learning. Panoramic images are usually represented in ERP, which introduces severe distortions that vary with latitude. At the same time, dense tasks in spherical space do not share the same geometric behavior under camera rotation. The system therefore treats panoramic multi-task learning not as ordinary dense MTL on a wider image, but as a setting in which distortion handling, task disentanglement, and controlled cross-task interaction must be encoded directly in the model design (Zhang et al., 5 Feb 2026).

The implemented network uses input resolution $512 \times 1024$, a DINOv3-Large backbone, and a DPT decoder/head with embedding dimension 512. The main tasks are semantic segmentation, depth estimation, and surface normal estimation. The auxiliary heads predict Image Gradient, Edge Distance Field (EDF), and Point Map. These auxiliary outputs are not treated as incidental side products; they are explicitly used as structured dense priors within the bridge mechanism (Zhang et al., 5 Feb 2026).

2. Rotation-invariant and rotation-variant task decomposition

A defining principle of PD-BridgeNet is its split between rotation-invariant and rotation-variant dense tasks. The paper defines semantic segmentation and depth as rotation-invariant tasks because they “depend solely on relative spatial context” and “should remain consistent when a transformation is applied.” By contrast, surface normals are treated as rotation-variant because they are tied to absolute observer/camera orientation; a normal expressed in camera coordinates changes when the camera rotates even if the scene remains fixed (Zhang et al., 5 Feb 2026).

This distinction is particularly consequential in ERP or spherical coordinates, where each pixel corresponds to a ray on the sphere rather than to a local planar patch. A shared feature space would have to compromise between orientation-agnostic cues needed by semantics and depth and the orientation-sensitive geometry needed by normals. The paper identifies this as a source of negative transfer. PD-BridgeNet addresses the problem by constructing an Invariant Stream for semantic/depth-like tasks and a Variant Stream for orientation-sensitive outputs such as normals and metric point maps (Zhang et al., 5 Feb 2026).

This decomposition is not presented as a generic MTL heuristic. It is a geometric claim about panoramic vision: some tasks should preserve invariance to camera orientation, while others must encode absolute pose on the sphere. A plausible implication is that PD-BridgeNet’s central novelty lies less in multi-head prediction itself than in the decision to make task equivariance properties first-class architectural constraints.

3. Dual-stream architecture and bridge mechanism

PD-BridgeNet processes shared backbone features $\mathcal{F}_{backbone}$ through two parallel streams. The Invariant Stream $M_{inv}$ is intended for rotation-invariant tasks, focuses on extracting distortion-aware but orientation-agnostic context, uses the ERP Token Mixer directly, and does not inject explicit directional or absolute pose priors. The Variant Stream $M_{var}$ is intended for rotation-variant tasks, also uses distortion-aware ERP processing, and additionally receives explicit spherical geometric priors: absolute position on the panorama and normalized 3D ray direction (Zhang et al., 5 Feb 2026).

To mitigate ERP distortion, the model replaces a standard local token mixing operation with the ERP Token Mixer:

$$\text{Mixer}(X) = (1 - w(\varphi)) \cdot (K_{3\times3} * X) + w(\varphi) \cdot (K_{3\times9} * X),$$

with

$w(\varphi) = |\sin(\varphi)|.$

Near the equator, where ERP distortion is milder, the $3\times3$ kernel dominates; near the poles, where horizontal stretching is severe, the wider $3\times9$ kernel receives greater weight (Zhang et al., 5 Feb 2026).

The variant branch is conditioned on explicit spherical geometry. For latitude $\varphi$ and longitude $\theta$, the normalized 3D ray direction is

$\mathcal{F}_{backbone}$0

The paper states that $\mathcal{F}_{backbone}$1 is concatenated with positional embeddings of $\mathcal{F}_{backbone}$2, passed through a lightweight MLP, and used to generate FiLM-style affine parameters $\mathcal{F}_{backbone}$3 and $\mathcal{F}_{backbone}$4, yielding

$\mathcal{F}_{backbone}$5

This modulation injects absolute sphere coordinate priors and breaks the excessive orientation invariance of the pretrained backbone (Zhang et al., 5 Feb 2026).

Disentanglement alone would isolate streams too strongly, so PD-BridgeNet introduces a Bridge Feature Extractor (BFE) based on Cross-Attention. Intermediate features from task heads are extracted, flattened into tokens, concatenated across tasks, and then used as cross-attention memory: generic backbone or stream features act as queries, while task-specific features act as keys and values. The exchanged sources include features derived from semantic segmentation, depth, surface normals, image gradient, EDF, and point map. The stated purpose is to support cross-task interaction while preserving the stream split (Zhang et al., 5 Feb 2026).

The most distinctive control mechanism is Truncated Gradient Flow. Cross-stream features are used during forward aggregation but are passed through a detach operation so that gradients do not backpropagate across the branch boundary. The paper’s textual formulation is explicit: cross-stream features are aggregated in the forward pass while backward gradients from the opposing branch are strictly blocked. No standalone algebraic detach equation is given in the paper, so the strongest faithful statement is procedural: interaction is allowed in representation space but curtailed in optimization space (Zhang et al., 5 Feb 2026).

4. Label-free supervision, auxiliary tasks, and optimization

The label-free pipeline that supports PD-BridgeNet samples $\mathcal{F}_{backbone}$6 random perspective cameras per panorama, with FoV in $\mathcal{F}_{backbone}$7, yaw in $\mathcal{F}_{backbone}$8, and pitch in $\mathcal{F}_{backbone}$9. The system uses InternImage-H for semantic segmentation pseudo-labels and MoGe-2 for depth and surface normals. Rather than stitching crop predictions into a single dense panorama label map, it reprojects each prediction back into ERP coordinates and computes loss only on valid pixels of those spherical patches. The paper states that this avoids seam artifacts, scale inconsistencies, and projection noise associated with full-map stitching (Zhang et al., 5 Feb 2026).

Task transfer is task-aware. Semantic labels are transferred directly. Depth predictions are adjusted by the projection angle using $M_{inv}$0, and normals are rotated back into panorama coordinates by the corresponding camera rotation. Auxiliary labels are generated directly from image or depth content: image gradient is computed via Sobel,

$M_{inv}$1

and the metric point map is

$M_{inv}$2

The EDF is produced from thresholded gradient plus the Jump Flooding Algorithm (Zhang et al., 5 Feb 2026).

Training is conducted in a strictly label-free setting on 140,884 panoramas from 20,041 Structured3D, 34,260 Sun360, 10,359 Matterport3D, and 76,224 synthetic DiT360 images; the original ground-truth labels are not used for training. The implementation uses PyTorch, Adam, learning rate $M_{inv}$3, weight decay $M_{inv}$4, polynomial decay with power 0.9, batch size 4 per GPU, 8 NVIDIA A100 GPUs, and 100,000 iterations. Stabilization is reinforced by zero-initialized convolutions for injection layers and a progressive warmup in which the backbone is frozen and auxiliary heads are optimized first for the initial 1000 steps (Zhang et al., 5 Feb 2026).

The total objective is

$M_{inv}$5

The paper sets

$M_{inv}$6

and, textually, “for auxiliary tasks we set the weight to 0.003.” The numerical value of $M_{inv}$7 is not provided in the visible text (Zhang et al., 5 Feb 2026).

5. Experimental evidence and ablation structure

On Structured3D, MTPano reports Semseg mIoU 75.66, Depth AbsRel 0.0248, Depth RMSE 0.0968, and Normal Mean Error 3.850. Against panoramic MTL baselines, the paper lists InvPT: 70.02 mIoU / 0.1515 RMSE / 5.991 normal mean, BridgeNet: 70.13 / 0.1492 / 5.988, and TaskPrompter: 70.95 / 0.1428 / 5.962. On Stanford2D3D, MTPano reports 69.47 mIoU, 0.0675 AbsRel, and 9.706 normal mean, which the paper presents as strong evidence that the architecture generalizes beyond synthetic training scenes (Zhang et al., 5 Feb 2026).

The ablation table on Structured3D, run with DINOv3-Small, isolates the contributions of the main PD-BridgeNet components. The reported settings are STL, MTL baseline, w.o. $M_{inv}$8 $M_{inv}$9 Trunc, w.o. $M_{var}$0 $M_{var}$1, w.o. Aux Heads, and Full MTPano. The full system reports 66.64 / 0.1432 / 5.4740 / $M_{var}$2, while removing auxiliary heads yields 65.68 / 0.1435 / 5.4850 / $M_{var}$3, and removing geometric modulation and truncation reduces performance further to 66.36 / 0.1437 / 5.4850 / $M_{var}$4. The paper’s interpretation is explicit: dual-branch geometric disentanglement helps, gradient truncation helps, and auxiliary heads help (Zhang et al., 5 Feb 2026).

Qualitatively, the paper reports that MTPano predictions are visually better than the pseudo-labels used to train it. The cited effects are sharper boundaries, smoother and more coherent geometry, and cleaner point clouds than TaskPrompter. A feature visualization under rotation further shows that backbone features $M_{var}$5 have entangled attributes, invariant features $M_{var}$6 remain stable under rotation, and variant features $M_{var}$7 rotate with camera orientation. This is presented as direct evidence that the two-stream decomposition is not merely nominal but realized in the learned representation (Zhang et al., 5 Feb 2026).

6. Relation to adjacent “dual” and “bridge” formulations

The term Panoramic Dual BridgeNet is specific to MTPano and should not be conflated with earlier panoramic architectures that are merely similar in spirit. In “Spatio-Temporal Proximity-Aware Dual-Path Model for Panoramic Activity Recognition”, the relevant model is SPDP-Net, whose core recognition module is DPATr. That system is explicitly dual-path and the two paths “mutually reinforce” one another across layers, but the paper never uses the word bridge, contains no explicit bridge block, and studies Panoramic Activity Recognition rather than multi-task dense prediction (Lee et al., 2024).

A related but distinct form of bridging appears in DiT360, which bridges perspective-domain realism priors and panoramic-domain geometry priors at image level and token level. DiT360 includes a perspective branch, a panoramic branch, position-aware circular padding, yaw loss, and cube loss, but it does not introduce an architecture called Panoramic Dual BridgeNet (Feng et al., 13 Oct 2025).

The name should also be distinguished from BridgeNet, which is a graph-based dataset of 20,000 form-found bridge structures with rendered images from two canonical camera angles. That work supports a dual-view interpretation only indirectly and does not describe panoramic imaging or a panoramic network architecture (Bleker et al., 16 Dec 2025).

Within panoramic vision more broadly, PD-BridgeNet’s distinctive scope is therefore precise: it is a dual-stream, geometry-aware, bridge-coupled architecture for panoramic dense multi-task learning, embedded in a label-free training pipeline, and formulated around the separation of rotation-invariant and rotation-variant task attributes rather than around generic multi-task feature sharing (Zhang et al., 5 Feb 2026).