Papers
Topics
Authors
Recent
Search
2000 character limit reached

LawinASPP: Multi-scale Attention for Segmentation

Updated 23 January 2026
  • LawinASPP architecture is a multi-scale decoder for semantic segmentation that combines large window attention with spatial pyramid pooling to capture both local and global context.
  • It fuses multi-resolution features from hierarchical Vision Transformers to enhance segmentation accuracy while reducing computational cost.
  • The design achieves state-of-the-art mIoU results (e.g., 84.4% on Cityscapes) through adaptive pooling and efficient multi-head attention mechanisms.

LawinASPP architecture is a multi-scale decoder head for semantic segmentation Vision Transformers (ViTs) that integrates a novel large window attention mechanism with spatial pyramid pooling. Developed to efficiently capture multi-scale contextual information while maintaining computational economy, LawinASPP serves as the decoder in the Lawin Transformer framework, which establishes new state-of-the-art performance across standard semantic segmentation benchmarks (Yan et al., 2022).

1. Architectural Overview

LawinASPP operates as the decoder within a semantic segmentation pipeline whose encoder is a hierarchical ViT (e.g., MiT or Swin). The encoder provides four multi-resolution feature maps: F1  (O ⁣S=4),F2  (O ⁣S=8),F3  (O ⁣S=16),F4  (O ⁣S=32)F_1\;(O\!S=4),\quad F_2\;(O\!S=8),\quad F_3\;(O\!S=16),\quad F_4\;(O\!S=32) where each FiF_i has shape C×H′×W′C\times H'\times W', with H′=H/O ⁣SH'=H/O\!S, W′=W/O ⁣SW'=W/O\!S.

The features {F2,F3,F4}\{F_2, F_3, F_4\} are upsampled to the spatial resolution of F2F_2, concatenated along the channel axis, and projected via a 1×11\times1 convolution to produce: X∈RC×(H/8)×(W/8)X\in\mathbb{R}^{C\times (H/8)\times (W/8)} This tensor XX is processed by LawinASPP, yielding FiF_i0. Subsequently, FiF_i1 is upsampled to match FiF_i2's resolution, concatenated with FiF_i3, passed through another FiF_i4 projection, and then through the final classifier.

2. Large Window Attention Mechanism

A key innovation in LawinASPP is large window attention, which enables each local patch in FiF_i5 to attend to a substantially broader contextual window with low overhead. The input FiF_i6 is partitioned into non-overlapping patches (“query patches”) of size FiF_i7, with FiF_i8.

For each patch:

  • The query FiF_i9 is defined as C×H′×W′C\times H'\times W'0.
  • The context C×H′×W′C\times H'\times W'1 is a surrounding region of spatial size C×H′×W′C\times H'\times W'2, C×H′×W′C\times H'\times W'3, with ratio C×H′×W′C\times H'\times W'4.

To manage computational growth, context is pooled by an average pooling (downsampling factor C×H′×W′C\times H'\times W'5), producing C×H′×W′C\times H'\times W'6. Multi-head attention is employed with C×H′×W′C\times H'\times W'7 heads. Each head applies a position-mixing MLP across the spatial patch dimension: C×H′×W′C\times H'\times W'8

The processed features are re-concatenated and projected. Per-head queries, keys, and values are computed, and multi-head attention per patch is assembled as: C×H′×W′C\times H'\times W'9 Aggregated output is: H′=H/O ⁣SH'=H/O\!S0

3. Multi-Scale Contextual Representation

Multi-scale context extraction is achieved by repeating large window attention for H′=H/O ⁣SH'=H/O\!S1. This produces feature maps with effective receptive fields of H′=H/O ⁣SH'=H/O\!S2, H′=H/O ⁣SH'=H/O\!S3, and H′=H/O ⁣SH'=H/O\!S4. This multi-scale approach allows the representation of both local and global context crucial for precise semantic segmentation.

4. Spatial Pyramid Pooling Integration

LawinASPP integrates the multi-scale large window attention outputs via a spatial pyramid pooling design. Its branches are:

  • Shortcut: identity mapping of H′=H/O ⁣SH'=H/O\!S5.
  • LWA with H′=H/O ⁣SH'=H/O\!S6.
  • LWA with H′=H/O ⁣SH'=H/O\!S7.
  • LWA with H′=H/O ⁣SH'=H/O\!S8.
  • Global pooling: global average pooling of H′=H/O ⁣SH'=H/O\!S9, followed by W′=W/O ⁣SW'=W/O\!S0 convolution and upsampling to match spatial size.

Each produces a feature map of shape W′=W/O ⁣SW'=W/O\!S1; these are concatenated to form a W′=W/O ⁣SW'=W/O\!S2 tensor. A W′=W/O ⁣SW'=W/O\!S3 convolution reduces channel dimension to W′=W/O ⁣SW'=W/O\!S4, followed by BatchNorm and activation (GELU) produces output W′=W/O ⁣SW'=W/O\!S5: W′=W/O ⁣SW'=W/O\!S6 where W′=W/O ⁣SW'=W/O\!S7 is the activation function.

5. Hyperparameters and Implementation

Selected hyperparameters are:

  • Patch size W′=W/O ⁣SW'=W/O\!S8.
  • Ratios W′=W/O ⁣SW'=W/O\!S9.
  • Number of heads {F2,F3,F4}\{F_2, F_3, F_4\}0.
  • Channels: for MiT‑B3, {F2,F3,F4}\{F_2, F_3, F_4\}1; for MiT‑B5 or Swin‑L, {F2,F3,F4}\{F_2, F_3, F_4\}2.

Pseudocode for LawinASPP and large window attention is:

{F2,F3,F4}\{F_2, F_3, F_4\}9

6. Computational Complexity and Empirical Results

For feature maps {F2,F3,F4}\{F_2, F_3, F_4\}3 of size {F2,F3,F4}\{F_2, F_3, F_4\}4, standard local window attention incurs: {F2,F3,F4}\{F_2, F_3, F_4\}5 Lawin attention modifies this to: {F2,F3,F4}\{F_2, F_3, F_4\}6 Since {F2,F3,F4}\{F_2, F_3, F_4\}7, the increase is marginal.

Empirically, on {F2,F3,F4}\{F_2, F_3, F_4\}8 inputs with MiT‑B3 backbone:

  • SegFormer‑B3 w/ ASPP: ~79.0 GFlops
  • Lawin‑B3: 61.7 GFlops This yields ~22% FLOP reduction with mIoU on ADE20K improving from 48.7% to 49.9%.

7. Benchmark Performance and Applications

LawinASPP as part of Lawin Transformer achieves state-of-the-art mIoU:

  • 84.4% on Cityscapes
  • 56.2% on ADE20K
  • Competitive results on COCO-Stuff

This demonstrates the architecture’s capacity to enhance both segmentation accuracy and efficiency by leveraging large window attention within a multi-scale spatial pyramid pooling framework (Yan et al., 2022).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (1)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to LawinASPP Architecture.