---
title: 'UAGLNet: Global-Local Fusion for Building Extraction'
url: https://www.emergentmind.com/topics/uncertainty-aggregated-global-local-fusion-network-uaglnet
type: topic
---

# UAGLNet: Global-Local Fusion for Building Extraction

UAGLNet, short for **Uncertainty-Aggregated Global-Local Fusion Network**, is an encoder-decoder architecture for **building extraction from high-resolution remote sensing images** that combines hybrid CNN-transformer feature learning with uncertainty-aware decoding [2512.12941]. The model is designed for the binary semantic segmentation setting in which an RGB remote-sensing image $\mathbf{I}\in \mathbb{R}^{3\times H\times W}$ is mapped to a pixel-wise building mask $\mathbf{S}$ aligned with ground truth $\mathbf{Y}$. Its central premise is that accurate building extraction requires the joint treatment of **local detail**, **global semantics**, and **pixel-wise uncertainty**: CNN-style operations are used to preserve edges, corners, and fine roof structure; transformer-style operations are used to capture long-range dependencies and scene-level context; and an uncertainty-aggregated decoder suppresses unreliable responses in ambiguous regions such as boundaries, shadows, occlusions, and clutter [2512.12941].

## 1. Problem setting and conceptual motivation

Building extraction in aerial and satellite imagery is difficult because buildings vary substantially in **scale**, **shape**, **appearance**, **context**, and **boundary ambiguity** [2512.12941]. The paper situates UAGLNet against three broad design families. First, **CNN-only** segmentation models such as UNet, HRNet, and CBRNet are described as strong local feature extractors but limited by stacked convolutions with restricted receptive fields, which weakens modeling of long-range dependencies and large-scale building structure. Second, **transformer-only** approaches such as SwinUNet and STT improve global context modeling through self-attention, but may rely on hand-crafted token grouping and can lose fine-grained spatial detail, particularly around boundaries. Third, **hybrid CNN-transformer** systems such as BuildFormer, DSAT-Net, and BCTNet improve the balance between locality and context, yet the paper identifies two residual problems: the **inherent gap of the feature pyramids** and **insufficient global-local feature integration** [2512.12941].

UAGLNet is formulated as a direct response to those two issues. Its architecture is organized around three components: a **Cooperative Encoder (CE)** that uses CNNs early and transformers later, an **intermediate cooperative interaction block (CIB)** that narrows the semantic gap between local and global features, and a **Global-Local Fusion (GLF)** plus **Uncertainty-Aggregated Decoder (UAD)** that explicitly combines complementary representations and attenuates uncertain regions [2512.12941]. This suggests that the model is not merely a generic hybrid backbone, but a specifically staged mechanism for reconciling shallow local cues with deep global structure.

## 2. Cooperative encoder and intermediate interaction

The cooperative encoder outputs a feature pyramid
$$
\mathcal{F} = \left\{ \mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3,\mathbf{F}_4 \right\},
\qquad
\mathbf{F}_i \in \mathbb{R}^{C_i \times H_i \times W_i},
$$
with four stages whose output sizes are $128\times128$, $64\times64$, $32\times32$, and $16\times16$ respectively [2512.12941]. The reported stage design is: **Stage 1:** Conv $3\times3$, Conv $2\times2$, then MKFM blocks; **Stage 2:** Conv $3\times3$, then MKFM blocks; **Stage 3:** Conv $3\times3$, then repeated CIB blocks containing MKFM + MHSA + FFN; **Stage 4:** Conv $3\times3$, then MHSA + FFN blocks. The staging encodes the intended transition from local to global representation learning.

In the early stages, local structure is modeled by the **Multi-Kernel Feature Modulator (MKFM)**. The feature map is split into $n$ groups,
$$
\mathbf{Z} = [\mathbf{Z}_1,\mathbf{Z}_2,\dots,\mathbf{Z}_n],
$$
processed by depth-wise separable convolutions with different kernel sizes,
$$
\mathbf{Z}' = \mathrm{Cat}\big(\mathrm{DW}_{3\times3}(\mathbf{Z}_1), \dots, \mathrm{DW}_{k\times k}(\mathbf{Z}_n)\big),
$$
and then mixed through point-wise convolution,
$$
\mathbf{M} = \mathbf{W}_p \ast \mathbf{Z}'.
$$
The modulated feature is
$$
\mathrm{MKFM}(\mathbf{F}_i) = \mathbf{M} \otimes \phi(\mathbf{F}_i),
$$
where $\otimes$ denotes Hadamard product and $\phi(\cdot)$ is a linear embedding [2512.12941]. The stated role of MKFM is to enrich local representations through multiple receptive fields while remaining lightweight because it uses depth-wise separable convolutions.

The pivotal bridging mechanism is the **CIB**, applied in the third stage. Its alternation between local modulation and self-attention is written as
$$
\mathbf{X}_{l}^{*} = \mathbf{X}_{l-1} + \mathrm{MKFM}(\mathrm{Norm}(\mathbf{X}_{l-1})),
$$
$$
\mathbf{X}_{l} = \mathbf{X}_{l}^{*} + \mathrm{FFN}(\mathrm{Norm}(\mathbf{X}_{l}^{*})),
$$
followed by
$$
\mathbf{X}_{l+1}^{*} = \mathbf{X}_{l} + \mathrm{MHSA}(\mathrm{Norm}(\mathbf{X}_{l})),
$$
$$
\mathbf{X}_{l+1} = \mathbf{X}_{l+1}^{*} + \mathrm{FFN}(\mathrm{Norm}(\mathbf{X}_{l+1}^{*})).
$$
The attention operator is
$$
\mathrm{Attention}(Q,K,V)=\mathrm{Softmax}\!\left(\frac{QK^T}{\sqrt{C_i}}\right)V,
$$
and
$$
\mathrm{MHSA}(X)=\mathrm{Concat}[\mathrm{head}_1,\mathrm{head}_2,\dots,\mathrm{head}_h]W^O,
\qquad
\mathrm{head}_j=\mathrm{Attention}(Q_j,K_j,V_j).
$$
The paper states that this alternating design helps the third-stage features become “more discriminative while preserving local details” [2512.12941]. Empirically, the cooperative encoder is reported to outperform comparison architectures labeled parallel CNN-transformer, sequential CNN-transformer, and alternative CNN-transformer, with **83.74 IoU** versus **81.44**, **82.84**, and **82.15** respectively on Inria [2512.12941].

## 3. Global-local fusion mechanism

After encoding, UAGLNet constructs explicit **local** and **global** representations from the pyramid. Before fusion, each feature map is enhanced by a depth-convolutional residual block,
$$
\hat{\mathbf{F}}_i = \mathbf{F}_i + \mathrm{DWConv}(\mathbf{F}_i),
$$
where $\mathrm{DWConv}(\cdot)$ is described as two depth-wise separable convolution layers plus a point-wise convolution [2512.12941].

The **local branch** uses early and mid-level features:
$$
\mathbf{F}_L = \mathrm{Conv}\left(\mathrm{Cat}\left(\hat{\mathbf{F}}_1, \mathrm{UpConv}\left(\mathrm{Cat}\left(\hat{\mathbf{F}}_2, \mathrm{UpConv}(\hat{\mathbf{F}}_3)\right)\right)\right)\right).
$$
The **global branch** uses deeper features:
$$
\mathbf{F}_G = \mathrm{UpConv}\left(\mathrm{Cat}\left(\hat{\mathbf{F}}_3, \mathrm{UpConv}(\hat{\mathbf{F}}_4)\right)\right).
$$
This construction makes the third-stage feature $\hat{\mathbf{F}}_3$ a shared term in both branches. In the paper’s interpretation, $\mathbf{F}_L$ emphasizes fine details and spatial precision, whereas $\mathbf{F}_G$ emphasizes semantic completeness and global building structure [2512.12941].

The shared use of $\mathbf{F}_3$ is central to the meaning of “global-local fusion” in UAGLNet. Rather than treating local and global streams as disjoint, the model couples them through a mid-level representation already shaped by the cooperative interaction block. The ablation on fusion strategy reports that using $\{\mathbf{F}_1,\mathbf{F}_2,\mathbf{F}_3\}$ for the local side and $\{\mathbf{F}_3,\mathbf{F}_4\}$ for the global side gives the best result, namely **IoU 83.74%** and **F1 91.15%** [2512.12941]. This suggests that the architecture benefits from sharing an intermediate feature that is neither purely local nor purely global.

## 4. Uncertainty-aggregated decoder and learning objective

The **Uncertainty-Aggregated Decoder (UAD)** is the component that differentiates UAGLNet from a conventional hybrid segmentation network. It is designed for ambiguous pixels occurring at building boundaries, in shadowed rooftops, under occlusion, or in noisy and low-resolution regions [2512.12941]. For the local branch feature $\mathbf{F}_L \in \mathbb{R}^{c\times h\times w}$, the model predicts
$$
\bm{\mu} = \Phi_\mu(\mathbf{F}_L), \qquad \bm{\sigma} = \Phi_\sigma(\mathbf{F}_L),
$$
and each pixel $p$ is modeled as
$$
\mathbf{x}_p \sim \mathcal{N}(\bm{\mu}_p,\bm{\sigma}_p^2).
$$
Here $\bm{\mu}$ is the expected prediction and $\bm{\sigma}$ denotes uncertainty or variance [2512.12941].

The uncertainty map is estimated by repeated sampling:
$$
\mathbf{U} = \mathrm{Norm}\left(\mathrm{Var}(\mathbf{x}^{(1)}, \mathbf{x}^{(2)}, \dots, \mathbf{x}^{(T)})\right),
$$
with direct sampling made differentiable through the reparameterization trick,
$$
\mathbf{x} = \bm{\sigma}\times \bm{\epsilon} + \bm{\mu}, \qquad \bm{\epsilon}\sim \mathcal{N}(0,\mathbf{I}).
$$
UAD computes **local uncertainty** $\mathbf{U}_L$ and **global uncertainty** $\mathbf{U}_G$, then uses them as attenuation weights:
$$
\mathbf{F}_{\mathrm{out}} = (1-\mathbf{U}_G)\times \mathbf{F}_G + (1-\mathbf{U}_L)\times \mathbf{F}_L.
$$
The stated interpretation is straightforward: uncertain regions are down-weighted, while reliable regions are preserved [2512.12941]. In that sense, uncertainty is not only an auxiliary diagnostic output; it directly modulates the feature composition used for final prediction.

The total objective combines segmentation and uncertainty terms. The segmentation loss is
$$
\mathcal{L}_{\mathrm{seg}}(\mathbf{S}) =
\mathcal{L}_{\mathrm{dice}}(\mathbf{S},\mathbf{Y}) +
\mathcal{L}_{\mathrm{bce}}(\mathbf{S},\mathbf{Y}) +
\gamma \mathcal{L}_{\mathrm{bce}}(|\mathbf{S}|,|\mathbf{Y}|),
$$
where the boundary term uses building boundaries extracted via Laplacian convolution with kernel
$$
\begin{bmatrix}
-1 & -1 & -1\\
-1 & 8 & -1\\
-1 & -1 & -1
\end{bmatrix}.
$$
The uncertainty loss is
$$
\mathcal{L}_{\mathrm{unc}}(\mathbf{U}) =
\mathcal{L}_{\mathrm{bce}}(\mathbf{x}^*,\mathbf{Y}) +
\eta \mathcal{D}\big(\mathcal{N}(\bm{\mu},\bm{\sigma}^2)\,\|\,\mathcal{N}(0,\mathbf{I})\big),
$$
and the total loss is
$$
\mathcal{L}_{\mathrm{total}} =
\mathcal{L}_{\mathrm{seg}}(\mathbf{S}) +
\lambda_1 \mathcal{L}_{\mathrm{unc}}(\mathbf{U}_G) +
\lambda_2 \mathcal{L}_{\mathrm{unc}}(\mathbf{U}_L).
$$
These formulas make explicit that both local and global uncertainty are supervised during training [2512.12941].

## 5. Experimental setting, quantitative results, and ablations

UAGLNet is evaluated on three public building extraction benchmarks: **Inria Aerial Image Labeling**, **Massachusetts Building Dataset**, and **WHU Building Dataset** [2512.12941]. Inria contains **360 large images** of size $5000\times5000$ at **0.3 m** resolution, with **180 images** having ground truth and a split into **9737 training tiles** and **1942 validation tiles**. Massachusetts contains **151 aerial images** of size $1500\times1500$ at about **1 m/pixel**, split into **137 train**, **4 val**, and **10 test**. WHU contains **8189 tiles** of size $512\times512$ at **0.3 m/pixel**, split into **4736 train**, **1036 val**, and **2416 test**, covering **450 km\(^2\)** and about **187,000 buildings** [2512.12941].

Training is reported in **PyTorch** on a **single NVIDIA RTX A6000 (48GB)** with input crop size **$512\times512$**, **AdamW**, learning rate **$5\times10^{-4}$**, weight decay **0.01**, **Cosine Annealing Warm Restarts**, and **105 epochs** [2512.12941]. Batch size is **16** for Inria and **8** for Massachusetts and WHU; the loss weights are $\lambda_1=\lambda_2=0.5$ and $\eta=0.2$; random drop path is **0.2** for Inria and WHU and **0.4** for Massachusetts. Data augmentation includes random crop, random horizontal flip, photometric distortion, and mixup. Evaluation uses the standard metrics
$$
\mathrm{P} = \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}, \qquad
\mathrm{R} = \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},
$$
$$
\mathrm{F1} = \frac{2\mathrm{P}\mathrm{R}}{\mathrm{P}+\mathrm{R}}, \qquad
\mathrm{IoU} = \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}+\mathrm{FP}}.
$$

| Dataset | IoU | F1 |
|---|---:|---:|
| Massachusetts | 76.97% | 86.99% |
| Inria | 83.74% | 91.15% |
| WHU | 92.07% | 95.87% |

On Massachusetts, the reported **Precision** and **Recall** are **88.28%** and **85.73%**; on Inria they are **92.09%** and **90.22%**; on WHU they are **96.21%** and **95.54%** [2512.12941]. The paper states that UAGLNet outperforms BuildFormer by **+1.23 IoU** on Massachusetts and **+2.30 IoU** on Inria, and outperforms UANet by **+0.56 IoU** on Massachusetts and **+0.66 IoU** on Inria. On Inria it is also reported as lightweight, with **28.90 GFLOPs**, **15.34M parameters**, and **27.53 FPS**, corresponding to savings of **75.32%** computation and **62.14%** parameters relative to BuildFormer [2512.12941].

The ablation studies attribute measurable gains to each proposed module. On Inria, the progression **CE only: 82.46 IoU**, **+ GLF: 83.33**, **+ local uncertainty or global uncertainty: about 83.58 / 83.62**, and **+ full UAD: 83.74** supports the claim that both feature fusion and uncertainty modeling contribute [2512.12941]. Additional ablations report that CIB alone helps, GLF alone helps, and their combination is best; that the decoder outperforms FPN, UperHead, and ASPPHead; and that the best MKFM hyperparameters are **$k=9$** and **$n=4$**, yielding **IoU 83.74** and **F1 91.15**. Robustness experiments show improvement under synthetic low-resolution degradation from **80.32** to **81.19 IoU** when UAD is added, and under noisy images from **82.72** to **83.41 IoU** [2512.12941]. In cross-domain evaluation, training on Inria and testing on WHU, UAGLNet is reported to degrade less than HRNet and BuildFormer, with **IoU drop 7.87** and **F1 drop 4.45** [2512.12941].

## 6. Interpretation, clarifications, and relation to adjacent work

Several clarifications follow directly from the reported architecture and ablations. First, UAGLNet is **not** a pure transformer model and **not** a simple CNN front-end followed by a transformer back-end. The cooperative encoder and CIB are explicitly presented as mechanisms to narrow the semantic gap between local and global representations, and the architecture comparison against parallel, sequential, and alternative hybrids indicates that this staged cooperation is materially different from straightforward composition [2512.12941]. Second, the uncertainty mechanism is **not** only a confidence visualization tool. Because $\mathbf{U}_L$ and $\mathbf{U}_G$ attenuate $\mathbf{F}_L$ and $\mathbf{F}_G$ directly, uncertainty participates in the segmentation computation itself rather than being used only for post hoc interpretation [2512.12941]. Third, the model’s “global-local” character should not be reduced to the coexistence of shallow and deep features; the shared use of $\mathbf{F}_3$ in both branches means that the local and global streams are coupled through a deliberately intermediate representation.

A broader research context is visible in later arXiv work. **CroBIM-U** for referring remote sensing image segmentation uses a pixel-wise uncertainty map to modulate **global / cross-modal fusion** through **Uncertainty-Gated Fusion (UGF)** and **local / boundary refinement** through **Uncertainty-Driven Local Refinement (UDLR)**, which the paper explicitly describes as an uncertainty-guided controller for global-local behavior [2601.03490]. **GLU** for sparse spatiotemporal reconstruction and forecasting defines a structured latent state composed of a **global token**, **local sensor tokens**, and an **uncertainty-driven importance field**, and uses that assembly for both reconstruction and forecasting [2603.26023]. These models are not the same architecture as UAGLNet, but they support a wider interpretation: uncertainty-aware control over the interaction between global and local representations has become a recurring design pattern across remote sensing and scientific machine learning. A plausible implication is that UAGLNet’s uncertainty-aggregated decoder belongs to this broader family of methods that treat uncertainty as an active fusion variable rather than a passive estimate.

In practical terms, the paper positions UAGLNet for **urban mapping**, **GIS updating**, **disaster assessment**, **population and urban planning**, and automated rooftop and building footprint extraction in complex scenes [2512.12941]. Its reported qualitative behavior is consistent with that positioning: better preservation of building integrity, more accurate extraction of small and occluded buildings, fewer false positives in cluttered regions, sharper boundaries, and improved handling of ambiguous neighboring pixels [2512.12941]. Within the scope of the reported experiments, UAGLNet is therefore best understood as a hybrid segmentation framework whose distinctive contribution lies in making **uncertainty** the explicit mechanism for aggregating complementary **global** and **local** evidence.

Source: https://www.emergentmind.com/topics/uncertainty-aggregated-global-local-fusion-network-uaglnet