---
title: 'Dynamic Spatial Fusion: Adaptive Input Fusion'
url: https://www.emergentmind.com/topics/dynamic-spatial-fusion-dsf
type: topic
---

# Dynamic Spatial Fusion: Adaptive Input Fusion

Searching arXiv for the cited DSF-related papers and terminology to ground the article in current sources.
arXiv search: "Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection"
Dynamic Spatial Fusion (DSF) denotes a class of input-conditioned fusion mechanisms in which the combination of complementary signals is spatially adaptive rather than fixed. In the weakly aligned UAV RGB–IR formulation of CoDAF, “dynamic” means that the model computes instance- and position-dependent parameters such as offsets, weights, and attention maps conditioned on the input; “spatial” emphasizes resolving pixel- or feature-location discrepancies across modalities before fusion; and “fusion” integrates complementary cues so that the detector’s representation is stronger than either modality alone [2506.16737]. Related work extends this idea to dynamic filters in light-field super-resolution, decoupled stereo fusion with spatial re-embedding, spatial–temporal modulation in video-to-HDR conversion, and other task-specific variants. At the same time, the acronym is not used uniformly across the literature: in some papers it instead denotes Dynamic Spatial Filtering, Dynamic-Static Fusion, or Dynamic State Fusion [2008.11449][2208.08145][2211.02297][2105.12916][2206.04975][2603.14452].

## 1. Definition, scope, and acronym usage

Across the cited literature, DSF is best understood as a family of adaptive fusion operators rather than a single standardized layer. The common pattern is that a network first extracts complementary representations from different modalities, views, frames, or branches, then synthesizes spatially varying offsets, weights, masks, or filters that determine how those representations should be aligned or combined at each location. This pattern appears explicitly in weakly aligned UAV RGB–IR detection, implicitly in light-field super-resolution and dynamic micro-expression recognition, and in modified form in stereo segmentation, HDR video conversion, diffusion synthesis, EEG robustness, LiDAR detection, and visual tracking [2506.16737][2008.11449][2208.08145][2211.02297][2505.16372][2105.12916][2206.04975][2305.15219][2507.13388][2603.14452].

| Context | Meaning of DSF | Representative mechanism |
|---|---|---|
| Weakly aligned UAV RGB–IR detection | Dynamic Spatial Fusion | OSA offsets plus DAFM gating and dual attention |
| Light-field spatial super-resolution | DSF mapped from MDFN + DFB | Multi-dimension fusion plus per-pixel dynamic filters |
| Stereo superpixel segmentation | DSF mapped from DSFM + DSEM | Decoupled stereo fusion plus dynamic spatial embedding |
| SDRTV-to-HDRTV conversion | DSF mapped from DMFA + STFM | Deformable alignment plus spatial–temporal modulation |
| Dynamic facial expression recognition | Dynamic-Static Fusion | Motion/static branch summation before temporal modeling |
| EEG robustness | Dynamic Spatial Filtering | Predicted spatial filters over channels |
| Visual tracking | Dynamic State Fusion | State-space update plus progressive cross-attention |
| Diffusion image synthesis | Dynamic Spatial Fusion | Spatial mask blending base and refiner latents |
| LiDAR 3D object detection | Dynamic-Static Fusion | Neighborhood Cross Attention plus Dynamic-Static Interaction |

A recurrent misconception is that DSF necessarily refers to multimodal image fusion. The surveyed papers show a broader usage: channel fusion in EEG, dual-latent fusion in diffusion, and state injection in tracking all use closely related adaptive mechanisms, while only some of them use the exact phrase “Dynamic Spatial Fusion.” This suggests that the defining property is not the application domain but the presence of input-dependent, spatially structured fusion.

## 2. Explicit formulation in weakly aligned UAV RGB–IR detection

The most explicit and technically complete definition is given by CoDAF, a two-stream detector for weakly aligned UAV object detection with RGB and infrared imagery [2506.16737]. RGB and IR images are processed by two ResNet-50 backbones to produce multiscale features at $t \in \{3,4,5\}$, denoted $V_t \in \mathbb{R}^{H \times W \times C}$ and $I_t \in \mathbb{R}^{H \times W \times C}$. At each scale, CoDAF applies Offset-guided Semantic Alignment (OSA) followed by a Dynamic Attention-guided Fusion Module (DAFM), then feeds the fused pyramidal features into an RT-DETR-style efficient hybrid encoder and task-specific heads. On DroneVehicle, the head is YOLOv11-OBB; on M3FD, it is an RT-DETR-compatible head.

OSA addresses weak alignment by using IR as the spatial reference. It first computes an IR-guided attention map
$$
M = \sigma(C_{1\times1}(I_t)),
$$
then forms
$$
M_w = M \cdot \mathrm{Cat}(I_t, V_t),
$$
and predicts a base offset prior
$$
\phi_c = C_{3\times3}(\mathrm{ReLU}(C_{3\times3}(M_w))).
$$
DCNv2 then aligns the RGB feature map to the IR feature map through deformable sampling:
$$
y(p) = \sum_{k=1}^{K} w_k \cdot x(p + \phi_c + p_k + \Delta p_k) \cdot \Delta m_k.
$$
Here $K$ is the number of kernel sampling points, $p_k$ are regular kernel offsets, $w_k$ are learned convolution weights, $\Delta p_k$ and $\Delta m_k \in [0,1]$ are residual offsets and modulation scalars, and bilinear interpolation is used at fractional coordinates. Unlike standard DCNv2, offset learning is conditioned on $\phi_c$, which is derived from cross-modal attention rather than solely from $x$.

OSA further introduces a shared semantic space through SID, using a simple shared feature extractor $f_{\mathrm{SID}}$ to map aligned RGB and IR features into modality-invariant embeddings:
$$
V_t^c = \mathrm{Norm}(f_{\mathrm{SID}}(V_t^a)), \qquad I_t^c = \mathrm{Norm}(f_{\mathrm{SID}}(I_t^a)).
$$
An InfoNCE loss encourages paired RGB–IR samples to be close and unpaired ones to be far:
$$
L_{\mathrm{contrast}} = -\frac{1}{N}\sum_i \log \frac{\exp((V_{t,i}^c \cdot I_{t,i}^c)/\tau)}{\sum_j \exp((V_{t,i}^c \cdot I_{t,j}^c)/\tau)}.
$$
Alignment is regularized by
$$
L_{\mathrm{sm}} = \lambda_1 L_{\mathrm{ssim}}(V_t^a, I_t^a) + \lambda_2 L_{\mathrm{mae}}(V_t^a, I_t^a),
$$
with $\lambda_1 = 0.3$ and $\lambda_2 = 0.5$, and by attention sparsity and smoothness:
$$
L_{\mathrm{attn}} = L_{\mathrm{sparse}} + \lambda_3 L_{\mathrm{smooth}},
$$
with $\lambda_3 = 0.1$. The total alignment objective is
$$
L_t = L_{\mathrm{contrast}} + L_{\mathrm{sm}} + L_{\mathrm{attn}}.
$$

DAFM then mitigates modality conflict after alignment. Its Modality-Adaptive Gating Network computes pixel-wise modality weights through
$$
G = \mathrm{Softmax}\!\left(C_{3\times3}\big(\mathrm{ReLU}(C_{3\times3}(\mathrm{Cat}(V_t^a, I_t^a)))\big)\right),
$$
splits $G$ into $G_v$ and $G_i$, and produces
$$
F = G_v \odot V_t^a + G_i \odot I_t^a.
$$
A Dual-Attention Cross-Modulation branch applies channel attention
$$
H_c = \sigma(\mathrm{MLP}(\mathrm{AvgPool}(F) + \mathrm{MaxPool}(F))), \qquad F_c = H_c \odot F,
$$
followed by spatial attention
$$
H_s = \sigma(C_{3\times3}(\mathrm{Cat}(\mathrm{Mean}(F_c), \mathrm{Max}(F_c)))), \qquad F_{\mathrm{fused}} = H_s \odot F_c.
$$
The end-to-end loss is
$$
L_{\mathrm{total}} = L_{\mathrm{det}} + \lambda L_t,
$$
with $\lambda = 0.1$ in the main experiments, while an ablation varying $\lambda \in [0.1,0.9]$ reports peak mAP near $\lambda \approx 0.3$.

Empirically, CoDAF achieves $\mathrm{mAP}@.5 = 78.6\%$ on DroneVehicle and $\mathrm{mAP}@.5:.95 = 61.2\%$, $\mathrm{mAP}@.5 = 90.8\%$ on M3FD. On DroneVehicle, adding OSA to the baseline yields $+5.9\%$ $\mathrm{mAP}@.5$, adding DAFM yields $+3.8\%$, removing MAGN drops $-2.3\%$, removing DACM drops $-0.4\%$, and removing OSA+DAFM at the earliest stage causes the largest drop of $-3.2\%$. The full model uses $67.3\mathrm{M}$ parameters and $224.9\mathrm{G}$ FLOPs, yet runs at $58.1$ FPS.

## 3. Dynamic filters and spatial masks as DSF operators

In light-field spatial super-resolution, DSF is realized by the combination of a Multi-Dimension Fusion Network (MDFN) and a Dynamic Filters Branch (DFB) [2008.11449]. The input light field is $I^{lr} \in \mathbb{R}^{U \times V \times X \times Y}$, and MDFN extracts complementary features from spatial sub-aperture images, angular micro-lens images, horizontal EPIs, and vertical EPIs. Each of the $n = 8$ Multi-Dimension Fusion Blocks applies four parallel 2D convolutional paths, then concatenates their outputs:
$$
F_i = [F_i^S, F_i^A, F_i^{E_h}, F_i^{E_v}].
$$
The DFB maps the fused features $F_n$ to a bank of per-pixel dynamic filters
$$
W = \mathcal{F}_d \in \mathbb{R}^{U \times V \times rX \times rY \times d \times d},
$$
with $d = 5$, and applies them to the micro-lens domain:
$$
I^{U}(u,v,rx+\Delta x,ry+\Delta y) =
\sum_{i=1}^{d}\sum_{j=1}^{d}\mathcal{F}_d(u,v,rx+\Delta x,ry+\Delta y,i,j)\,
I^{lr}(u+i-\lfloor d/2 \rfloor, v+j-\lfloor d/2 \rfloor, x,y).
$$
A residual branch predicts $I^R$, and final reconstruction is
$$
I^{sr} = I^U + I^R = G^{DFB}(F_n)\otimes I^{lr} + G^{RB}(F_n).
$$
The softmax-normalized filters are spatially varying, view-specific, and input-adaptive. Quantitatively, the method reports, for $\times2$ super-resolution, $42.31/0.982$ on HCI1, $37.33/0.964$ on HCI2, $35.96/0.963$ on EPFL, and $43.01/0.986$ on General; for $\times4$, it reports $36.02/0.932$ on HCI1, $31.57/0.886$ on HCI2, $30.62/0.875$ on EPFL, and $36.01/0.940$ on General.

A conceptually related but architecturally simpler DSF appears in dual-latent diffusion synthesis [2507.13388]. There, base latent $L_b$ and refined latent $L_r$ are fused after the SDXL base and refiner stages and before VAE decoding. DSF computes
$$
P_{avg} = \mathrm{AvgPool}(L_r), \qquad P_{max} = \mathrm{MaxPool}(L_b),
$$
concatenates them, produces a spatial attention map
$$
M_{\mathrm{spatial}} = \mathrm{Sigmoid}(\mathrm{Conv}_{7\times7}(P_{\mathrm{spatial}})),
$$
and blends the latents through
$$
L_f = M_{\mathrm{spatial}} \odot L_r + (1 - M_{\mathrm{spatial}}) \odot L_b.
$$
The added module is extremely small, with approximately $99$ parameters in the $7 \times 7$ convolution. On class-conditional ImageNet, DSF reports at $256 \times 256$ an FID of $18.89$, sFID of $48.21$, Inception Score of $232.04$, and precision/recall of $0.87/0.39$; at $512 \times 512$, it reports FID $18.70$, sFID $50.22$, Inception Score $243.62$, and precision/recall $0.85/0.38$. An extra refiner step after fusion degrades FID, sFID, and IS.

These two cases instantiate different ends of the same design space. One synthesizes full per-pixel filter banks over a structured angular neighborhood; the other predicts a single spatial mask over two latent tensors. This suggests a general DSF operator pattern in which fused or summarized features generate spatially varying kernels or masks that act directly on the representation.

## 4. Decoupled spatial fusion and spatiotemporal modulation

In stereo superpixel segmentation, DSF is operationalized by decoupling spatial information before stereo fusion and reintroducing it afterward [2208.08145]. The Decoupled Stereo Fusion Module (DSFM) first removes XY coordinates from stereo inputs and aligns left and right appearance features by row-wise parallax attention:
$$
\mathcal{M}_{R \to L} = \mathrm{softmax}(A \otimes B^T), \qquad
\hat{\mathcal{F}}_L = \mathcal{M}_{R \to L} \otimes \mathcal{F}_R.
$$
Occlusion handling is implemented through valid masks such as
$$
O_{L \to R}(i,j) =
\begin{cases}
1, & \sum_{k=1}^{W}\mathcal{M}_{L \to R}(i,k,j) > \tau,\\
0, & \text{otherwise},
\end{cases}
$$
with $\tau = 0.1$. The Dynamic Spatiality Embedding Module (DSEM) then re-adds normalized coordinates
$$
\hat X = \frac{X}{\max(X)}, \qquad \hat Y = \frac{Y}{\max(Y)},
$$
and uses Dynamic Fusion (DF) to compute a weighting map
$$
\mathcal{W} = g \cdot \sigma\!\left(C\!\left(\mathrm{ReLU}\!\left(\mathrm{LN}(C(g))\right)\right)\right),
$$
followed by refinement to $\mathcal{F}_{out}$. On KITTI2015 at $700$ superpixels, minimum percentage gains over the best competing methods are ASA $+0.4\%$, UE $+9.3\%$ and BR $+1.1\%$, and maximum gains are ASA $+1.4\%$, UE $+33.8\%$, and BR $+4.8\%$.

In SDRTV-to-HDRTV conversion, DSLNet extends the same principle from stereo to temporal windows [2211.02297]. The input is
$$
X = [X_{i-T}, \ldots, X_{i+T}], \qquad Y_i = \mathrm{DSLNet}(X).
$$
Dynamic Multi-Frame Alignment (DMFA) uses deformable convolution driven by a Large-kernel Dynamic Offset Estimator, while Spatial-Temporal Feature Modulation (STFM) computes a spatial modulation vector $V_{SM}$, a temporal modulation vector $V_{TM}$, and a current-frame modulation vector $V_{CM}$. The modulation operators are
$$
Y = \mathrm{TFM}(F_P,V_{TMA},V_{TMB}) = F_P * V_{TMA} + V_{TMB},
$$
$$
Y = \mathrm{CFM}(F_P,V_{CMA},V_{CMB}) = F_P * V_{CMA} + V_{CMB},
$$
$$
Y = \mathrm{SFM}(F_P,V_{SMA},V_{SMB}) = F_P * V_{SMA} + V_{SMB}.
$$
A parallel residual branch and LKQE with large kernels refine the prediction. DSLNet reports PSNR $35.28$, SR-SIM $0.9951$, and $\Delta E_{ITP}$ $11.13$, outperforming HDRTVNET and SR-ITM; removing alignment degrades results to $34.28/0.9935/14.24$, and removing modulation degrades them to $33.36/0.9925/14.04$.

In dynamic micro-expression recognition, the paper does not explicitly use the term DSF, but its “parallel time–space fusion method” is directly mapped to DSF in the provided technical synthesis [2505.16372]. The temporal branch constructs a difference frame
$$
f_t = f_{apex} - f_{onset},
$$
then applies a retention mechanism
$$
o_n = \sum_{m=1}^{n} \gamma^{n-m}\,\big(Q_n e^{i n \theta}\big)\,\big(K_m e^{i m \theta}\big)^\dagger\, v_m.
$$
The spatial branch applies patch slicing, learnable positional embeddings, and two Transformer blocks, and both branches produce tensors of shape $14 \times 14 \times 512$. Fusion is simply
$$
\mathcal{F}_{t-s} = \mathcal{F}_t + \mathcal{F}_s.
$$
The authors characterize this as complementary “where–how” semantics. Reported results include, on CAS(ME)$^3$, UF1/UAR of $53.29\%/62.73\%$ for the 7-class setting and $70.75\%/76.03\%$ for the 4-class setting; on CASME II, $87.50\%$ ACC and $86.17\%$ UF1 for 5-class, and $94.84\%$ ACC and $94.30\%$ UF1 for 3-class.

## 5. Related acronyms and domain-specific reinterpretations

In NR-DFERNet, DSF means Dynamic-Static Fusion rather than Dynamic Spatial Fusion [2206.04975]. The module operates at the spatial stage of dynamic facial expression recognition. Dynamic features are defined by adjacent-frame feature differences
$$
F_i^D = F_{i+1} - F_i,
$$
while static features are obtained through a downsampling convolution and a spatial transformer. After processing both branches, fusion is performed by element-wise summation:
$$
F_i^{DS} = F_i^{D'} + F_i^{S'}.
$$
This yields consistent gains on DFEW: the baseline reports UAR/WAR of $50.22\%/63.52\%$, while DSF alone yields $52.35\%/65.36\%$, and the full NR-DFERNet yields $54.21\%/68.19\%$.

In EEG robustness, the paper’s DSF denotes Dynamic Spatial Filtering, but functionally it performs dynamic spatial fusion of channels [2105.12916]. Given an EEG window $X \in \mathbb{R}^{C \times T}$ and a binary channel mask $m \in \{0,1\}^C$, the masked input is
$$
\tilde X = \mathrm{diag}(m)X.
$$
A small MLP predicts a set of filters and biases,
$$
[W_{\mathrm{flat}}, b] = h_{\Theta_{DSF}}(\Phi(\tilde X)),
$$
which are reshaped and applied as
$$
Y = W'(\mathrm{diag}(m)X) + b\,\mathbf{1}_T^\top.
$$
Channel importance is read out by
$$
\phi_j = \sqrt{\sum_{i=1}^{C'} {W'}_{ij}^2}.
$$
With simulated channel corruption, the method reports up to $29.4\%$ absolute accuracy improvement versus vanilla baselines under significant corruption, and on naturally corrupted mobile sleep data it raises median balanced accuracy from $58.4\%$ to $65.0\%$.

DynStaF in LiDAR 3D object detection is another related reinterpretation, again labeled Dynamic-Static Fusion rather than Dynamic Spatial Fusion [2305.15219]. It separates a dynamic multi-frame branch from a static single-frame branch and fuses them through Neighborhood Cross Attention (NCA) and Dynamic-Static Interaction (DSI). NCA uses static queries and dynamic keys/values within a local BEV neighborhood $N(i)$, while DSI performs dense late-stage interaction. On the nuScenes test set, PointPillars improves from $57.7\%$ to $61.6\%$ NDS, and CenterPoint with DynStaF reaches $61.0\%$ mAP and $67.7\%$ NDS.

A distinct case is Uni-MDTrack, where DSF explicitly means Dynamic State Fusion, not Dynamic Spatial Fusion [2603.14452]. The module maintains a hidden state $h_t$, updates it through a gated SSM-like mechanism using search-region tokens, produces dynamic state features $F_t$, and injects them progressively into multiple backbone stages through cross-attention. The paper states that training only the MCP, DSF, and prediction head, while keeping trainable parameters around $30\%$, yields state-of-the-art results on $10$ datasets spanning five modalities. This is the clearest example of acronym divergence and shows that the letters “DSF” alone are not sufficient to determine the underlying mechanism.

## 6. Empirical regularities, limitations, and future directions

A clear empirical regularity across the surveyed work is that DSF-style modules are most beneficial when the input streams are complementary but not trivially aligned. In CoDAF, weak alignment and modality conflict are central, and OSA plus DAFM provide measurable gains in $\mathrm{mAP}@.5$ [2506.16737]. In light-field super-resolution, full four-branch fusion reaches $43.01/0.986$ on General for $\times2$ super-resolution, exceeding variants that fuse fewer dimensions [2008.11449]. In stereo segmentation, DSFM and DSEM improve ASA, UE, and BR under disparity and occlusion [2208.08145]. In HDR video conversion, removing alignment or modulation sharply worsens PSNR and $\Delta E_{ITP}$ [2211.02297]. In dynamic micro-expression recognition, temporal-only and spatial-only branches both underperform the parallel fusion design, and late parallel T-S fusion is the best among the tested strategies [2505.16372]. This suggests that DSF is particularly effective when local correspondence, branch complementarity, or dynamic reliability vary across space.

The limitations reported in the papers are equally consistent. CoDAF notes that extreme misalignment or severe modality noise can exceed the capacity of attention-guided offsets, and that deployment on resource-constrained UAVs may require pruning, quantization, or lighter deformable sampling [2506.16737]. The light-field method is limited by local angular neighborhoods of size $d \times d$ with $d = 5$, which can be challenged by extremely large disparities, severe occlusions, or noise [2008.11449]. Stereo parallax attention scales with $H \times W \times W$, and the occlusion threshold $\tau$ may require tuning [2208.08145]. The diffusion DSF uses only a single spatial mask broadcast across channels and therefore lacks per-channel gating [2507.13388]. Uni-MDTrack reports difficulties under extreme occlusions among many nearly identical distractors and under very fast abrupt motion [2603.14452]. EEG robustness depends strongly on corruption-aware augmentation, and the robustness profile reflects the training corruptions used [2105.12916].

The future directions proposed in the source papers point toward a more explicit treatment of geometry, uncertainty, and temporal persistence. For weakly aligned RGB–IR detection, the cited directions include explicit geometric alignment such as global homography plus local refinement, transformer-based cross-attention to predict offsets with longer-range context, uncertainty-aware gating, mixture-of-experts for modality selection, and temporal modeling for asynchronous capture [2506.16737]. For stereo and video tasks, the cited directions include multi-view extensions, temporal fusion beyond stereo pairs, and stronger multi-scale alignment or attention-enhanced alignment [2208.08145][2211.02297]. For dynamic micro-expression recognition, the suggested directions include transfer learning and contrastive learning to address cross-cultural variation and class imbalance [2505.16372]. For tracking, proposed extensions include learned motion priors, spatial anchors, and hybrid state-space plus deformable attention [2603.14452]. For EEG, extending augmentation beyond white noise to colored noise, dropouts, motion artifacts, and EMG/EOG bursts is explicitly identified as a path to stronger robustness [2105.12916].

Taken together, these works portray DSF not as a single algorithmic primitive but as an architectural principle: spatially localized fusion should be conditioned on the actual structure, reliability, or dynamics of the input. Depending on the domain, the resulting mechanism may take the form of deformable offsets, neighborhood attention, dynamic filter banks, spatial masks, gated branch weighting, or adaptive spatial filters. The persistence of that principle across multimodal detection, multi-view reconstruction, temporal restoration, physiological sensing, and generative modeling explains both the breadth of the acronym’s usage and the importance of interpreting it in context.

Source: https://www.emergentmind.com/topics/dynamic-spatial-fusion-dsf