---
title: Multi-Scale Fusion Transformer
url: https://www.emergentmind.com/topics/multi-scale-fusion-transformer
type: topic
---

# Multi-Scale Fusion Transformer

Multi-scale fusion transformer denotes a family of architectures in which representations are extracted at multiple resolutions, temporal granularities, or semantic levels and then fused with transformer-style attention mechanisms. Across the literature, the pattern appears in image fusion, semantic and medical segmentation, visual recognition, speech separation, time-series modeling, trajectory prediction, multimodal sentiment analysis, continuous sign language recognition, and LiDAR place recognition. A common objective is to combine local detail, typically emphasized by convolutional or windowed operators, with long-range dependencies or global context, typically emphasized by self-attention or cross-attention [2107.09011][2209.01620][2212.07163].

## 1. Conceptual scope and evolution

The term emerged in several adjacent but not identical lines of work. In image fusion, the "Image Fusion Transformer" introduced a transformer-based multi-scale fusion strategy in a two-stage framework, with multi-scale encoder features fused by a Spatio-Transformer block that combines a CNN branch and a transformer branch [2107.09011]. In fine-grained visual recognition, AFTrans used a global-and-local multi-scale pipeline supervised by a weight-sharing encoder, with attention-derived region selection rather than explicit box annotations [2110.01240]. In general visual recognition, MAFormer formalized multi-scale attention fusion as a dual-stream transformer block with local window attention, global learning with down-sampling, and attention-based fusion [2209.01620].

By 2022 and after, the idea had broadened beyond images. MSFFT-Net used multiple parallel processing paths at different temporal scales for speech separation, rather than a single dual-path sequence model [2212.07163]. FeSeFormer introduced scale-level feature selection and full-scale feature fusion for semantic segmentation, explicitly selecting informative subsets from the entire multi-scale feature set for each query feature [2203.14124]. Later work extended the same design logic to incomplete vehicle trajectories, skin-lesion segmentation, 3D spine segmentation, spiking vision transformers, variable-length long time series, and cross-view LiDAR descriptors [2409.00904][2503.03327][2503.12853][2505.14719][2509.17845][2604.04513].

This distribution suggests that "multi-scale fusion transformer" is better understood as an architectural principle than as a single standardized network. The principle is the explicit interaction of representations computed at multiple scales, with attention used not merely for encoding but also for scale coupling, adaptive weighting, or query-conditioned selection.

## 2. Architectural archetypes

Several recurrent architectural forms appear across the literature.

| Archetype | Representative papers | Distinguishing mechanism |
|---|---|---|
| Dual-branch local/global fusion | IFT, MATCNN, MAFormer | CNN or local-attention branch plus transformer or global-attention branch |
| Hierarchical encoder-decoder fusion | ScaleFusionNet, WoundFormer, RefineFormer3D | Multi-scale skip integration with cross-attention or spatial aggregation |
| Parallel multi-resolution paths | MSFFT-Net, sign-language MSFT | Simultaneous fine and coarse temporal paths with explicit exchange or concatenation |
| Pyramid or scale-selective fusion | FeSeFormer, PST, MPTF-Net | Query-conditioned selection, sparse tokens, or cross-view pyramid attention |

In dual-branch designs, one branch is responsible for local structure and the other for long-range context. IFT makes this division explicit: the spatial branch consists of convolutional layers plus

Source: https://www.emergentmind.com/topics/multi-scale-fusion-transformer