Dynamic Spatial Fusion: Adaptive Input Fusion
- Dynamic Spatial Fusion (DSF) is a class of adaptive fusion operators that compute instance- and position-dependent parameters to align and merge complementary features.
- It employs deformable offsets, dynamic filters, and attention maps to reconcile misalignments in tasks such as UAV detection, super-resolution, and tracking.
- By enabling spatially adaptive kernel and mask generation, DSF enhances accuracy and robustness across multimodal, multi-view, and temporal applications.
Searching arXiv for the cited DSF-related papers and terminology to ground the article in current sources. arXiv search: "Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection" Dynamic Spatial Fusion (DSF) denotes a class of input-conditioned fusion mechanisms in which the combination of complementary signals is spatially adaptive rather than fixed. In the weakly aligned UAV RGB–IR formulation of CoDAF, “dynamic” means that the model computes instance- and position-dependent parameters such as offsets, weights, and attention maps conditioned on the input; “spatial” emphasizes resolving pixel- or feature-location discrepancies across modalities before fusion; and “fusion” integrates complementary cues so that the detector’s representation is stronger than either modality alone (Zongzhen et al., 20 Jun 2025). Related work extends this idea to dynamic filters in light-field super-resolution, decoupled stereo fusion with spatial re-embedding, spatial–temporal modulation in video-to-HDR conversion, and other task-specific variants. At the same time, the acronym is not used uniformly across the literature: in some papers it instead denotes Dynamic Spatial Filtering, Dynamic-Static Fusion, or Dynamic State Fusion (Sun et al., 2020, Li et al., 2022, Xu et al., 2022, Banville et al., 2021, Li et al., 2022, Cai et al., 15 Mar 2026).
1. Definition, scope, and acronym usage
Across the cited literature, DSF is best understood as a family of adaptive fusion operators rather than a single standardized layer. The common pattern is that a network first extracts complementary representations from different modalities, views, frames, or branches, then synthesizes spatially varying offsets, weights, masks, or filters that determine how those representations should be aligned or combined at each location. This pattern appears explicitly in weakly aligned UAV RGB–IR detection, implicitly in light-field super-resolution and dynamic micro-expression recognition, and in modified form in stereo segmentation, HDR video conversion, diffusion synthesis, EEG robustness, LiDAR detection, and visual tracking (Zongzhen et al., 20 Jun 2025, Sun et al., 2020, Li et al., 2022, Xu et al., 2022, Liu et al., 22 May 2025, Banville et al., 2021, Li et al., 2022, Rong et al., 2023, Chen et al., 16 Jul 2025, Cai et al., 15 Mar 2026).
| Context | Meaning of DSF | Representative mechanism |
|---|---|---|
| Weakly aligned UAV RGB–IR detection | Dynamic Spatial Fusion | OSA offsets plus DAFM gating and dual attention |
| Light-field spatial super-resolution | DSF mapped from MDFN + DFB | Multi-dimension fusion plus per-pixel dynamic filters |
| Stereo superpixel segmentation | DSF mapped from DSFM + DSEM | Decoupled stereo fusion plus dynamic spatial embedding |
| SDRTV-to-HDRTV conversion | DSF mapped from DMFA + STFM | Deformable alignment plus spatial–temporal modulation |
| Dynamic facial expression recognition | Dynamic-Static Fusion | Motion/static branch summation before temporal modeling |
| EEG robustness | Dynamic Spatial Filtering | Predicted spatial filters over channels |
| Visual tracking | Dynamic State Fusion | State-space update plus progressive cross-attention |
| Diffusion image synthesis | Dynamic Spatial Fusion | Spatial mask blending base and refiner latents |
| LiDAR 3D object detection | Dynamic-Static Fusion | Neighborhood Cross Attention plus Dynamic-Static Interaction |
A recurrent misconception is that DSF necessarily refers to multimodal image fusion. The surveyed papers show a broader usage: channel fusion in EEG, dual-latent fusion in diffusion, and state injection in tracking all use closely related adaptive mechanisms, while only some of them use the exact phrase “Dynamic Spatial Fusion.” This suggests that the defining property is not the application domain but the presence of input-dependent, spatially structured fusion.
2. Explicit formulation in weakly aligned UAV RGB–IR detection
The most explicit and technically complete definition is given by CoDAF, a two-stream detector for weakly aligned UAV object detection with RGB and infrared imagery (Zongzhen et al., 20 Jun 2025). RGB and IR images are processed by two ResNet-50 backbones to produce multiscale features at , denoted and . At each scale, CoDAF applies Offset-guided Semantic Alignment (OSA) followed by a Dynamic Attention-guided Fusion Module (DAFM), then feeds the fused pyramidal features into an RT-DETR-style efficient hybrid encoder and task-specific heads. On DroneVehicle, the head is YOLOv11-OBB; on M3FD, it is an RT-DETR-compatible head.
OSA addresses weak alignment by using IR as the spatial reference. It first computes an IR-guided attention map
then forms
and predicts a base offset prior
DCNv2 then aligns the RGB feature map to the IR feature map through deformable sampling:
Here is the number of kernel sampling points, are regular kernel offsets, are learned convolution weights, 0 and 1 are residual offsets and modulation scalars, and bilinear interpolation is used at fractional coordinates. Unlike standard DCNv2, offset learning is conditioned on 2, which is derived from cross-modal attention rather than solely from 3.
OSA further introduces a shared semantic space through SID, using a simple shared feature extractor 4 to map aligned RGB and IR features into modality-invariant embeddings:
5
An InfoNCE loss encourages paired RGB–IR samples to be close and unpaired ones to be far:
6
Alignment is regularized by
7
with 8 and 9, and by attention sparsity and smoothness:
0
with 1. The total alignment objective is
2
DAFM then mitigates modality conflict after alignment. Its Modality-Adaptive Gating Network computes pixel-wise modality weights through
3
splits 4 into 5 and 6, and produces
7
A Dual-Attention Cross-Modulation branch applies channel attention
8
followed by spatial attention
9
The end-to-end loss is
0
with 1 in the main experiments, while an ablation varying 2 reports peak mAP near 3.
Empirically, CoDAF achieves 4 on DroneVehicle and 5, 6 on M3FD. On DroneVehicle, adding OSA to the baseline yields 7 8, adding DAFM yields 9, removing MAGN drops 0, removing DACM drops 1, and removing OSA+DAFM at the earliest stage causes the largest drop of 2. The full model uses 3 parameters and 4 FLOPs, yet runs at 5 FPS.
3. Dynamic filters and spatial masks as DSF operators
In light-field spatial super-resolution, DSF is realized by the combination of a Multi-Dimension Fusion Network (MDFN) and a Dynamic Filters Branch (DFB) (Sun et al., 2020). The input light field is 6, and MDFN extracts complementary features from spatial sub-aperture images, angular micro-lens images, horizontal EPIs, and vertical EPIs. Each of the 7 Multi-Dimension Fusion Blocks applies four parallel 2D convolutional paths, then concatenates their outputs:
8
The DFB maps the fused features 9 to a bank of per-pixel dynamic filters
0
with 1, and applies them to the micro-lens domain:
2
A residual branch predicts 3, and final reconstruction is
4
The softmax-normalized filters are spatially varying, view-specific, and input-adaptive. Quantitatively, the method reports, for 5 super-resolution, 6 on HCI1, 7 on HCI2, 8 on EPFL, and 9 on General; for 0, it reports 1 on HCI1, 2 on HCI2, 3 on EPFL, and 4 on General.
A conceptually related but architecturally simpler DSF appears in dual-latent diffusion synthesis (Chen et al., 16 Jul 2025). There, base latent 5 and refined latent 6 are fused after the SDXL base and refiner stages and before VAE decoding. DSF computes
7
concatenates them, produces a spatial attention map
8
and blends the latents through
9
The added module is extremely small, with approximately 0 parameters in the 1 convolution. On class-conditional ImageNet, DSF reports at 2 an FID of 3, sFID of 4, Inception Score of 5, and precision/recall of 6; at 7, it reports FID 8, sFID 9, Inception Score 0, and precision/recall 1. An extra refiner step after fusion degrades FID, sFID, and IS.
These two cases instantiate different ends of the same design space. One synthesizes full per-pixel filter banks over a structured angular neighborhood; the other predicts a single spatial mask over two latent tensors. This suggests a general DSF operator pattern in which fused or summarized features generate spatially varying kernels or masks that act directly on the representation.
4. Decoupled spatial fusion and spatiotemporal modulation
In stereo superpixel segmentation, DSF is operationalized by decoupling spatial information before stereo fusion and reintroducing it afterward (Li et al., 2022). The Decoupled Stereo Fusion Module (DSFM) first removes XY coordinates from stereo inputs and aligns left and right appearance features by row-wise parallax attention:
2
Occlusion handling is implemented through valid masks such as
3
with 4. The Dynamic Spatiality Embedding Module (DSEM) then re-adds normalized coordinates
5
and uses Dynamic Fusion (DF) to compute a weighting map
6
followed by refinement to 7. On KITTI2015 at 8 superpixels, minimum percentage gains over the best competing methods are ASA 9, UE 0 and BR 1, and maximum gains are ASA 2, UE 3, and BR 4.
In SDRTV-to-HDRTV conversion, DSLNet extends the same principle from stereo to temporal windows (Xu et al., 2022). The input is
5
Dynamic Multi-Frame Alignment (DMFA) uses deformable convolution driven by a Large-kernel Dynamic Offset Estimator, while Spatial-Temporal Feature Modulation (STFM) computes a spatial modulation vector 6, a temporal modulation vector 7, and a current-frame modulation vector 8. The modulation operators are
9
00
01
A parallel residual branch and LKQE with large kernels refine the prediction. DSLNet reports PSNR 02, SR-SIM 03, and 04 05, outperforming HDRTVNET and SR-ITM; removing alignment degrades results to 06, and removing modulation degrades them to 07.
In dynamic micro-expression recognition, the paper does not explicitly use the term DSF, but its “parallel time–space fusion method” is directly mapped to DSF in the provided technical synthesis (Liu et al., 22 May 2025). The temporal branch constructs a difference frame
08
then applies a retention mechanism
09
The spatial branch applies patch slicing, learnable positional embeddings, and two Transformer blocks, and both branches produce tensors of shape 10. Fusion is simply
11
The authors characterize this as complementary “where–how” semantics. Reported results include, on CAS(ME)12, UF1/UAR of 13 for the 7-class setting and 14 for the 4-class setting; on CASME II, 15 ACC and 16 UF1 for 5-class, and 17 ACC and 18 UF1 for 3-class.
5. Related acronyms and domain-specific reinterpretations
In NR-DFERNet, DSF means Dynamic-Static Fusion rather than Dynamic Spatial Fusion (Li et al., 2022). The module operates at the spatial stage of dynamic facial expression recognition. Dynamic features are defined by adjacent-frame feature differences
19
while static features are obtained through a downsampling convolution and a spatial transformer. After processing both branches, fusion is performed by element-wise summation:
20
This yields consistent gains on DFEW: the baseline reports UAR/WAR of 21, while DSF alone yields 22, and the full NR-DFERNet yields 23.
In EEG robustness, the paper’s DSF denotes Dynamic Spatial Filtering, but functionally it performs dynamic spatial fusion of channels (Banville et al., 2021). Given an EEG window 24 and a binary channel mask 25, the masked input is
26
A small MLP predicts a set of filters and biases,
27
which are reshaped and applied as
28
Channel importance is read out by
29
With simulated channel corruption, the method reports up to 30 absolute accuracy improvement versus vanilla baselines under significant corruption, and on naturally corrupted mobile sleep data it raises median balanced accuracy from 31 to 32.
DynStaF in LiDAR 3D object detection is another related reinterpretation, again labeled Dynamic-Static Fusion rather than Dynamic Spatial Fusion (Rong et al., 2023). It separates a dynamic multi-frame branch from a static single-frame branch and fuses them through Neighborhood Cross Attention (NCA) and Dynamic-Static Interaction (DSI). NCA uses static queries and dynamic keys/values within a local BEV neighborhood 33, while DSI performs dense late-stage interaction. On the nuScenes test set, PointPillars improves from 34 to 35 NDS, and CenterPoint with DynStaF reaches 36 mAP and 37 NDS.
A distinct case is Uni-MDTrack, where DSF explicitly means Dynamic State Fusion, not Dynamic Spatial Fusion (Cai et al., 15 Mar 2026). The module maintains a hidden state 38, updates it through a gated SSM-like mechanism using search-region tokens, produces dynamic state features 39, and injects them progressively into multiple backbone stages through cross-attention. The paper states that training only the MCP, DSF, and prediction head, while keeping trainable parameters around 40, yields state-of-the-art results on 41 datasets spanning five modalities. This is the clearest example of acronym divergence and shows that the letters “DSF” alone are not sufficient to determine the underlying mechanism.
6. Empirical regularities, limitations, and future directions
A clear empirical regularity across the surveyed work is that DSF-style modules are most beneficial when the input streams are complementary but not trivially aligned. In CoDAF, weak alignment and modality conflict are central, and OSA plus DAFM provide measurable gains in 42 (Zongzhen et al., 20 Jun 2025). In light-field super-resolution, full four-branch fusion reaches 43 on General for 44 super-resolution, exceeding variants that fuse fewer dimensions (Sun et al., 2020). In stereo segmentation, DSFM and DSEM improve ASA, UE, and BR under disparity and occlusion (Li et al., 2022). In HDR video conversion, removing alignment or modulation sharply worsens PSNR and 45 (Xu et al., 2022). In dynamic micro-expression recognition, temporal-only and spatial-only branches both underperform the parallel fusion design, and late parallel T-S fusion is the best among the tested strategies (Liu et al., 22 May 2025). This suggests that DSF is particularly effective when local correspondence, branch complementarity, or dynamic reliability vary across space.
The limitations reported in the papers are equally consistent. CoDAF notes that extreme misalignment or severe modality noise can exceed the capacity of attention-guided offsets, and that deployment on resource-constrained UAVs may require pruning, quantization, or lighter deformable sampling (Zongzhen et al., 20 Jun 2025). The light-field method is limited by local angular neighborhoods of size 46 with 47, which can be challenged by extremely large disparities, severe occlusions, or noise (Sun et al., 2020). Stereo parallax attention scales with 48, and the occlusion threshold 49 may require tuning (Li et al., 2022). The diffusion DSF uses only a single spatial mask broadcast across channels and therefore lacks per-channel gating (Chen et al., 16 Jul 2025). Uni-MDTrack reports difficulties under extreme occlusions among many nearly identical distractors and under very fast abrupt motion (Cai et al., 15 Mar 2026). EEG robustness depends strongly on corruption-aware augmentation, and the robustness profile reflects the training corruptions used (Banville et al., 2021).
The future directions proposed in the source papers point toward a more explicit treatment of geometry, uncertainty, and temporal persistence. For weakly aligned RGB–IR detection, the cited directions include explicit geometric alignment such as global homography plus local refinement, transformer-based cross-attention to predict offsets with longer-range context, uncertainty-aware gating, mixture-of-experts for modality selection, and temporal modeling for asynchronous capture (Zongzhen et al., 20 Jun 2025). For stereo and video tasks, the cited directions include multi-view extensions, temporal fusion beyond stereo pairs, and stronger multi-scale alignment or attention-enhanced alignment (Li et al., 2022, Xu et al., 2022). For dynamic micro-expression recognition, the suggested directions include transfer learning and contrastive learning to address cross-cultural variation and class imbalance (Liu et al., 22 May 2025). For tracking, proposed extensions include learned motion priors, spatial anchors, and hybrid state-space plus deformable attention (Cai et al., 15 Mar 2026). For EEG, extending augmentation beyond white noise to colored noise, dropouts, motion artifacts, and EMG/EOG bursts is explicitly identified as a path to stronger robustness (Banville et al., 2021).
Taken together, these works portray DSF not as a single algorithmic primitive but as an architectural principle: spatially localized fusion should be conditioned on the actual structure, reliability, or dynamics of the input. Depending on the domain, the resulting mechanism may take the form of deformable offsets, neighborhood attention, dynamic filter banks, spatial masks, gated branch weighting, or adaptive spatial filters. The persistence of that principle across multimodal detection, multi-view reconstruction, temporal restoration, physiological sensing, and generative modeling explains both the breadth of the acronym’s usage and the importance of interpreting it in context.