UAGLNet: Global-Local Fusion for Building Extraction

• The paper introduces UAGLNet, which integrates local details and global semantics with uncertainty modeling for enhanced building extraction.
• The cooperative encoder and intermediate interaction block bridge CNN and transformer features, yielding superior IoU and F1 performance.
• The uncertainty-aggregated decoder actively modulates fusion to refine ambiguous regions, such as occlusions, shadows, and boundary artifacts.

UAGLNet, short for Uncertainty-Aggregated Global-Local Fusion Network, is an encoder-decoder architecture for building extraction from high-resolution remote sensing images that combines hybrid CNN-transformer feature learning with uncertainty-aware decoding (Yao et al., 15 Dec 2025). The model is designed for the binary semantic segmentation setting in which an RGB remote-sensing image $\mathbf{I}\in \mathbb{R}^{3\times H\times W}$ is mapped to a pixel-wise building mask $\mathbf{S}$ aligned with ground truth $\mathbf{Y}$. Its central premise is that accurate building extraction requires the joint treatment of local detail, global semantics, and pixel-wise uncertainty: CNN-style operations are used to preserve edges, corners, and fine roof structure; transformer-style operations are used to capture long-range dependencies and scene-level context; and an uncertainty-aggregated decoder suppresses unreliable responses in ambiguous regions such as boundaries, shadows, occlusions, and clutter (Yao et al., 15 Dec 2025).

1. Problem setting and conceptual motivation

Building extraction in aerial and satellite imagery is difficult because buildings vary substantially in scale, shape, appearance, context, and boundary ambiguity (Yao et al., 15 Dec 2025). The paper situates UAGLNet against three broad design families. First, CNN-only segmentation models such as UNet, HRNet, and CBRNet are described as strong local feature extractors but limited by stacked convolutions with restricted receptive fields, which weakens modeling of long-range dependencies and large-scale building structure. Second, transformer-only approaches such as SwinUNet and STT improve global context modeling through self-attention, but may rely on hand-crafted token grouping and can lose fine-grained spatial detail, particularly around boundaries. Third, hybrid CNN-transformer systems such as BuildFormer, DSAT-Net, and BCTNet improve the balance between locality and context, yet the paper identifies two residual problems: the inherent gap of the feature pyramids and insufficient global-local feature integration (Yao et al., 15 Dec 2025).

UAGLNet is formulated as a direct response to those two issues. Its architecture is organized around three components: a Cooperative Encoder (CE) that uses CNNs early and transformers later, an intermediate cooperative interaction block (CIB) that narrows the semantic gap between local and global features, and a Global-Local Fusion (GLF) plus Uncertainty-Aggregated Decoder (UAD) that explicitly combines complementary representations and attenuates uncertain regions (Yao et al., 15 Dec 2025). This suggests that the model is not merely a generic hybrid backbone, but a specifically staged mechanism for reconciling shallow local cues with deep global structure.

2. Cooperative encoder and intermediate interaction

The cooperative encoder outputs a feature pyramid

$$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$

with four stages whose output sizes are $128\times128$, $64\times64$, $32\times32$, and $16\times16$ respectively (Yao et al., 15 Dec 2025). The reported stage design is: Stage 1: Conv $3\times3$, Conv $2\times2$, then MKFM blocks; Stage 2: Conv $\mathbf{S}$0, then MKFM blocks; Stage 3: Conv $\mathbf{S}$1, then repeated CIB blocks containing MKFM + MHSA + FFN; Stage 4: Conv $\mathbf{S}$2, then MHSA + FFN blocks. The staging encodes the intended transition from local to global representation learning.

In the early stages, local structure is modeled by the Multi-Kernel Feature Modulator (MKFM). The feature map is split into $\mathbf{S}$3 groups,

$\mathbf{S}$4

processed by depth-wise separable convolutions with different kernel sizes,

$\mathbf{S}$5

and then mixed through point-wise convolution,

$\mathbf{S}$6

The modulated feature is

$\mathbf{S}$7

where $\mathbf{S}$8 denotes Hadamard product and $\mathbf{S}$9 is a linear embedding (Yao et al., 15 Dec 2025). The stated role of MKFM is to enrich local representations through multiple receptive fields while remaining lightweight because it uses depth-wise separable convolutions.

The pivotal bridging mechanism is the CIB, applied in the third stage. Its alternation between local modulation and self-attention is written as

$\mathbf{Y}$0

$\mathbf{Y}$1

followed by

$\mathbf{Y}$2

$\mathbf{Y}$3

The attention operator is

$\mathbf{Y}$4

and

$\mathbf{Y}$5

The paper states that this alternating design helps the third-stage features become “more discriminative while preserving local details” (Yao et al., 15 Dec 2025). Empirically, the cooperative encoder is reported to outperform comparison architectures labeled parallel CNN-transformer, sequential CNN-transformer, and alternative CNN-transformer, with 83.74 IoU versus 81.44, 82.84, and 82.15 respectively on Inria (Yao et al., 15 Dec 2025).

3. Global-local fusion mechanism

After encoding, UAGLNet constructs explicit local and global representations from the pyramid. Before fusion, each feature map is enhanced by a depth-convolutional residual block,

$\mathbf{Y}$6

where $\mathbf{Y}$7 is described as two depth-wise separable convolution layers plus a point-wise convolution (Yao et al., 15 Dec 2025).

The local branch uses early and mid-level features:

$\mathbf{Y}$8

The global branch uses deeper features:

$\mathbf{Y}$9

This construction makes the third-stage feature $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$0 a shared term in both branches. In the paper’s interpretation, $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$1 emphasizes fine details and spatial precision, whereas $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$2 emphasizes semantic completeness and global building structure (Yao et al., 15 Dec 2025).

The shared use of $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$3 is central to the meaning of “global-local fusion” in UAGLNet. Rather than treating local and global streams as disjoint, the model couples them through a mid-level representation already shaped by the cooperative interaction block. The ablation on fusion strategy reports that using $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$4 for the local side and $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$5 for the global side gives the best result, namely IoU 83.74% and F1 91.15% (Yao et al., 15 Dec 2025). This suggests that the architecture benefits from sharing an intermediate feature that is neither purely local nor purely global.

4. Uncertainty-aggregated decoder and learning objective

The Uncertainty-Aggregated Decoder (UAD) is the component that differentiates UAGLNet from a conventional hybrid segmentation network. It is designed for ambiguous pixels occurring at building boundaries, in shadowed rooftops, under occlusion, or in noisy and low-resolution regions (Yao et al., 15 Dec 2025). For the local branch feature $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$6, the model predicts

$$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$7

and each pixel $$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$8 is modeled as

$$\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\}, \qquad \mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},$$9

Here $128\times128$0 is the expected prediction and $128\times128$1 denotes uncertainty or variance (Yao et al., 15 Dec 2025).

The uncertainty map is estimated by repeated sampling:

$128\times128$2

with direct sampling made differentiable through the reparameterization trick,

$128\times128$3

UAD computes local uncertainty $128\times128$4 and global uncertainty $128\times128$5, then uses them as attenuation weights:

$128\times128$6

The stated interpretation is straightforward: uncertain regions are down-weighted, while reliable regions are preserved (Yao et al., 15 Dec 2025). In that sense, uncertainty is not only an auxiliary diagnostic output; it directly modulates the feature composition used for final prediction.

The total objective combines segmentation and uncertainty terms. The segmentation loss is

$128\times128$7

where the boundary term uses building boundaries extracted via Laplacian convolution with kernel

$128\times128$8

The uncertainty loss is

$128\times128$9

and the total loss is

$64\times64$0

These formulas make explicit that both local and global uncertainty are supervised during training (Yao et al., 15 Dec 2025).

5. Experimental setting, quantitative results, and ablations

UAGLNet is evaluated on three public building extraction benchmarks: Inria Aerial Image Labeling, Massachusetts Building Dataset, and WHU Building Dataset (Yao et al., 15 Dec 2025). Inria contains 360 large images of size $64\times64$1 at 0.3 m resolution, with 180 images having ground truth and a split into 9737 training tiles and 1942 validation tiles. Massachusetts contains 151 aerial images of size $64\times64$2 at about 1 m/pixel, split into 137 train, 4 val, and 10 test. WHU contains 8189 tiles of size $64\times64$3 at 0.3 m/pixel, split into 4736 train, 1036 val, and 2416 test, covering 450 km$64\times64$4 and about 187,000 buildings (Yao et al., 15 Dec 2025).

Training is reported in PyTorch on a single NVIDIA RTX A6000 (48GB) with input crop size $64\times64$5, AdamW, learning rate $64\times64$6, weight decay 0.01, Cosine Annealing Warm Restarts, and 105 epochs (Yao et al., 15 Dec 2025). Batch size is 16 for Inria and 8 for Massachusetts and WHU; the loss weights are $64\times64$7 and $64\times64$8; random drop path is 0.2 for Inria and WHU and 0.4 for Massachusetts. Data augmentation includes random crop, random horizontal flip, photometric distortion, and mixup. Evaluation uses the standard metrics

$64\times64$9

$32\times32$0

Dataset	IoU	F1
Massachusetts	76.97%	86.99%
Inria	83.74%	91.15%
WHU	92.07%	95.87%

On Massachusetts, the reported Precision and Recall are 88.28% and 85.73%; on Inria they are 92.09% and 90.22%; on WHU they are 96.21% and 95.54% (Yao et al., 15 Dec 2025). The paper states that UAGLNet outperforms BuildFormer by +1.23 IoU on Massachusetts and +2.30 IoU on Inria, and outperforms UANet by +0.56 IoU on Massachusetts and +0.66 IoU on Inria. On Inria it is also reported as lightweight, with 28.90 GFLOPs, 15.34M parameters, and 27.53 FPS, corresponding to savings of 75.32% computation and 62.14% parameters relative to BuildFormer (Yao et al., 15 Dec 2025).

The ablation studies attribute measurable gains to each proposed module. On Inria, the progression CE only: 82.46 IoU, + GLF: 83.33, + local uncertainty or global uncertainty: about 83.58 / 83.62, and + full UAD: 83.74 supports the claim that both feature fusion and uncertainty modeling contribute (Yao et al., 15 Dec 2025). Additional ablations report that CIB alone helps, GLF alone helps, and their combination is best; that the decoder outperforms FPN, UperHead, and ASPPHead; and that the best MKFM hyperparameters are $32\times32$1 and $32\times32$2, yielding IoU 83.74 and F1 91.15. Robustness experiments show improvement under synthetic low-resolution degradation from 80.32 to 81.19 IoU when UAD is added, and under noisy images from 82.72 to 83.41 IoU (Yao et al., 15 Dec 2025). In cross-domain evaluation, training on Inria and testing on WHU, UAGLNet is reported to degrade less than HRNet and BuildFormer, with IoU drop 7.87 and F1 drop 4.45 (Yao et al., 15 Dec 2025).

6. Interpretation, clarifications, and relation to adjacent work

Several clarifications follow directly from the reported architecture and ablations. First, UAGLNet is not a pure transformer model and not a simple CNN front-end followed by a transformer back-end. The cooperative encoder and CIB are explicitly presented as mechanisms to narrow the semantic gap between local and global representations, and the architecture comparison against parallel, sequential, and alternative hybrids indicates that this staged cooperation is materially different from straightforward composition (Yao et al., 15 Dec 2025). Second, the uncertainty mechanism is not only a confidence visualization tool. Because $32\times32$3 and $32\times32$4 attenuate $32\times32$5 and $32\times32$6 directly, uncertainty participates in the segmentation computation itself rather than being used only for post hoc interpretation (Yao et al., 15 Dec 2025). Third, the model’s “global-local” character should not be reduced to the coexistence of shallow and deep features; the shared use of $32\times32$7 in both branches means that the local and global streams are coupled through a deliberately intermediate representation.

A broader research context is visible in later arXiv work. CroBIM-U for referring remote sensing image segmentation uses a pixel-wise uncertainty map to modulate global / cross-modal fusion through Uncertainty-Gated Fusion (UGF) and local / boundary refinement through Uncertainty-Driven Local Refinement (UDLR), which the paper explicitly describes as an uncertainty-guided controller for global-local behavior (Sun et al., 7 Jan 2026). GLU for sparse spatiotemporal reconstruction and forecasting defines a structured latent state composed of a global token, local sensor tokens, and an uncertainty-driven importance field, and uses that assembly for both reconstruction and forecasting (Wang et al., 27 Mar 2026). These models are not the same architecture as UAGLNet, but they support a wider interpretation: uncertainty-aware control over the interaction between global and local representations has become a recurring design pattern across remote sensing and scientific machine learning. A plausible implication is that UAGLNet’s uncertainty-aggregated decoder belongs to this broader family of methods that treat uncertainty as an active fusion variable rather than a passive estimate.

In practical terms, the paper positions UAGLNet for urban mapping, GIS updating, disaster assessment, population and urban planning, and automated rooftop and building footprint extraction in complex scenes (Yao et al., 15 Dec 2025). Its reported qualitative behavior is consistent with that positioning: better preservation of building integrity, more accurate extraction of small and occluded buildings, fewer false positives in cluttered regions, sharper boundaries, and improved handling of ambiguous neighboring pixels (Yao et al., 15 Dec 2025). Within the scope of the reported experiments, UAGLNet is therefore best understood as a hybrid segmentation framework whose distinctive contribution lies in making uncertainty the explicit mechanism for aggregating complementary global and local evidence.