---
title: Dynamic Multi-Scale Kernel in Deep Learning
url: https://www.emergentmind.com/topics/dynamic-multi-scale-kernel-dmsk
type: topic
---

# Dynamic Multi-Scale Kernel in Deep Learning

Dynamic Multi-Scale Kernel (DMSK) denotes an input-conditioned multi-scale kernel mechanism in which receptive-field scale is not fixed a priori but is selected, weighted, recalibrated, or recursively updated from global or task-specific context. The label appears explicitly in the Dynamic Skip Connection framework for medical segmentation [2509.14610], but closely related mechanisms are described under other names in speech, vision, and medical imaging, including dynamic multi-scale convolution [2108.07787], Dynamic Pyramid Convolution [2210.13821], Dynamic Multi-Scale Convolution [2410.02129], and Dynamic Large Kernel [2403.10674]. Across these formulations, the central objective is consistent: replace fixed-kernel processing with input-adaptive use of multiple temporal or spatial scales.

## 1. Scope and terminological usage

The most explicit use of the term occurs in the DMSK module inside the Dynamic Skip Connection block, where the module is introduced to mitigate the “intra-feature constraint” of conventional skip connections by adaptively selecting kernel sizes from small-scale and large-scale sets according to global contextual cues [2509.14610]. Earlier and parallel work uses different nomenclature for closely related ideas. In dialect identification, the relevant term is **dynamic multi-scale convolution**; in salient object detection, **Dynamic Pyramid Convolution**; in pancreas segmentation, **Dynamic Multi-Scale Convolution**; and in volumetric segmentation, **Dynamic Large Kernel** [2108.07787] [2210.13821] [2410.02129] [2403.10674].

| Formulation | Dynamic mechanism | Multi-scale mechanism |
|---|---|---|
| DMSK in DSC [2509.14610] | Softmax scoring followed by $\arg\max$ kernel selection | Small-scale and large-scale kernel sets in skip processing |
| Dynamic multi-scale convolution [2108.07787] | Channel-wise softmax branch weighting | Undilated and dilated temporal kernels, local split hierarchy, global multi-layer pooling |
| DPConv [2210.13821] | GAP-MLP-Softmax routing over branches | Parallel kernels $\{K3,K5,K7,K9\}$ |
| DMSC [2410.02129] | Global-context sigmoid recalibration | Parallel $3\times3$ and $5\times5$ branches |
| DLK [2403.10674] | Spatial-wise dynamic selection | Sequential large kernels $5\times5\times5$ and $7\times7\times7$ with dilation $3$ |

This diversity makes DMSK less a single canonical operator than a family of constructions. Some methods perform explicit operator-level scale selection; others retain fixed branch kernels and apply dynamic feature-side weighting. A narrower but important adjacent case is the polyp-segmentation Dynamic Kernel method, whose “multi-scale” character comes from encoder-wide context aggregation and multi-resolution recursive updates rather than from an explicit bank of kernels with different spatial supports [2509.23502].

## 2. Core operator patterns

A first recurrent pattern is **soft routing over fixed kernel branches**. In dynamic kernel convolution for dialect identification, two temporal branches are computed, one undilated and one with dilation factor $2$. Their outputs are summed as
$$
X = h_d^1 + h_d^2,
$$
high-order statistic pooling extracts per-channel mean, standard deviation, skewness, and kurtosis, and two dense layers followed by softmax produce channel-wise branch weights $s_i$. The fused output is
$$
h_{out} = \sum\limits_{i = 1}^2 s_i \otimes h_d^i.
$$
The dynamic behavior is therefore per-sample channel-wise branch reweighting rather than per-frame kernel synthesis [2108.07787]. A related formulation appears in DPConv, where global average pooling produces a descriptor $Z$, routing weights are obtained by
$$
\bm{\alpha} = Softmax\bigg(FC\Big(ReLU\big(FC(Z)\big)\Big)\bigg),
$$
and the final output is
$$
Y = X + Concat\big(\bm{\alpha}_1\times(W_1*X),\; \dots,\; \bm{\alpha}_m\times(W_m*X)\big).
$$
Here all branches are evaluated and dynamically modulated, so routing is soft and differentiable rather than sparse [2210.13821].

A second pattern is **hard kernel selection with staged filtering**. The explicit DMSK module first computes global average pooled context
$$
x^l_{\mathrm{GAP}}=\mathrm{GAP}(x^{l}_{in}),
$$
then predicts probability vectors over small-scale and large-scale candidates,
$$
w_{s}=\mathrm{Softmax}\!\left(\mathrm{Conv}_{s}^{l}(x^l_{\mathrm{GAP}})\right), \qquad
w_b=\mathrm{Softmax}\!\left(\mathrm{Conv}_b^{l}(x^l_{\mathrm{GAP}})\right),
$$
and selects
$$
\hat{k}_s=\arg\max(w_s),\quad \hat{k}_b=\arg\max(w_b).
$$
The selected kernels are cascaded through depthwise convolutions,
$$
x_1^l=\mathrm{DWConv}_{(\hat{k}_s)}(x^l),\qquad
x_2^l=\mathrm{DWConv}_{(\hat{k}_b)}(x_1^l),
$$
after which spatial and channel attention refine the result before residual addition. This design makes DMSK a context-conditioned candidate-kernel selector rather than a continuous mixture [2509.14610].

A third pattern is **post-branch recalibration rather than direct kernel generation**. In DMSC, fixed $3\times3$ and $5\times5$ branches produce $\boldsymbol{F}'_1$ and $\boldsymbol{F}'_2$, which are concatenated and globally pooled. A $1\times1$ projection and sigmoid yield channel-wise gates
$$
\boldsymbol{G} = \mathcal{F}_{Sig}\big(\mathcal{F}_{Conv}^{(1)}(\mathcal{F}_{GAvgPool}(\boldsymbol{F}))\big),
$$
and recalibration proceeds by
$$
\boldsymbol{F}'=\boldsymbol{F}\otimes \boldsymbol{G}, \qquad
\boldsymbol{F}^* = \mathcal{F}_{Conv}^{(1)}(\boldsymbol{F}').
$$
The dynamic component thus lies on feature aggregation rather than on convolution weights themselves [2410.02129]. By contrast, the polyp-segmentation Dynamic Kernel method generates an initial decoder-space kernel from encoder attention,
$$
K_5 = \phi(G),
$$
then updates it recursively via
$$
K_i = g_i \odot A_i^{\text{feat}} + (1-g_i)\odot K_{i+1},
$$
so it is a conditional parameter-generation method whose multi-scale character is inherited from encoder-scale aggregation and decoder-stage refinement [2509.23502].

These operator patterns also differ in where “multi-scale” is instantiated. It may be encoded by different kernel sizes or dilations, by hierarchical channel splitting within a block, or by cross-depth aggregation of bottleneck features. This suggests that DMSK is best understood as a design space spanning operator-scale diversity, context-conditioned routing, and cross-level feature fusion rather than a single fixed template.

## 3. Representative neural realizations

In speech processing, dynamic multi-scale convolution is instantiated on a D-TDNN backbone for dialect identification. The first D-TDNN layer is replaced by a multi-scale Dk block that combines dynamic kernel convolution, local multi-scale learning with channel splitting parameter $s=4$, and global multi-scale pooling from two bottleneck layers. All layers use $64$ filters; the dynamic kernel branches use kernel size $3$ with dilation $1$ and $2$; the first TDNN layer and last six D-TDNN layers use context $[t-5:t+5]$ and other layers use $[t-3:t+3]$; training is performed with AAM-Softmax [2108.07787]. In speaker verification, selective-kernel variants extend ECAPA-family systems through channel-wise SKA, frequency-wise SKA, sequential frequency-plus-channel SKA, and msSKA inside Res2Net. The front-end uses $3\times3$ and $5\times5$ branches, msSKA uses temporal kernels $3$ and $5$, the Res2Net scale is $s=8$, and the channel width is $1024$ [2204.01005].

In dense prediction for natural images, DPNet moves dynamic scale routing into the encoder through Dynamic Pyramid Convolution, then redesigns the decoder with BiCFM and Dynamic Weighted Fusion. The best DPConv branch library is $\{K3,K5,K7,K9\}$, and branch routing is derived from globally pooled statistics by a two-layer MLP with softmax [2210.13821]. In medical segmentation, DMSC uses parallel $3\times3$ and $5\times5$ convolutions with global channel recalibration, while DMRC introduces a separate multi-resolution pathway; both are inserted as replacements for single convolutions in U-Net-style blocks [2410.02129]. D-Net replaces self-attention in a hierarchical encoder-decoder with Dynamic Large Kernel blocks composed of sequential depthwise $5\times5\times5$ and $7\times7\times7$ convolutions, with the second using dilation $3$, followed by spatial-wise dynamic selection and Dynamic Feature Fusion for skip connections [2403.10674].

The explicit DMSK module in Dynamic Skip Connection is architecturally narrower but terminologically decisive. It is applied to skip features, uses global-average-pooled context to choose one small and one large kernel by $\arg\max$, then combines spatial attention, channel attention, and residual propagation. The same paper positions DMSK as the component that addresses insufficient modeling of multi-scale feature interactions inside a skip pathway, while a separate Test-Time Training module addresses dynamic adaptation across samples [2509.14610]. A nearby but technically distinct design is the Encoder-Attention plus Dynamic Kernel framework for polyp segmentation, in which all five encoder stages contribute to a global context vector $G$, a decoder channel width $C_d=32$ is enforced by Unified Channel Adaptation, and a $1\times1$ dynamic kernel is recursively refined from stage to stage using lesion-aware pooled features [2509.23502].

## 4. Empirical behavior

Reported experiments consistently show that fixed single-scale kernels are not uniformly optimal and that dynamic multi-scale mechanisms tend to improve accuracy when discriminative evidence is distributed across heterogeneous temporal or spatial extents.

| Setting | Comparison | Headline result |
|---|---|---|
| AP20-OLR dialect task | D-TDNN + AAM-Softmax vs Global and Local Multi-scale Dk Conv | $C_{avg}$ from $0.0766$ to $0.0670$; EER from $7.63\%$ to $6.52\%$; params from $3.3$M to $2.9$M |
| DUTS-TE / DUT-OMRON | Best fixed kernel vs dynamic $\{K3,K5,K7,K9\}$ | DUTS-TE $F_\beta$ from $0.907$ for $K5$ to $0.916$; DUT-OMRON $F_\beta=0.837$ |
| TCIA pancreas segmentation | 3D U-Net vs 3D DMSC-Net vs 3D DMC-Net | DSC $83.56 \rightarrow 85.67 \rightarrow 87.97$ |
| AMOS / brain tumor segmentation | DLK-Net vs D-Net | AMOS average Dice $88.18 \rightarrow 89.01$; brain tumor average Dice $74.57 \rightarrow 75.70$ |
| ISIC2017 / UNETR ablations | Small-only or large-only vs Small+Large DMSK | mIoU $0.7629$ or $0.7729$ vs $0.7980$; Dice $0.8638$ or $0.8695$ vs $0.8876$ |

On the AP20-OLR-dialect-task, the full dynamic multi-scale convolution model obtains the best average cost performance $C_{avg}=0.0670$ and EER $=6.52\%$, with about $9\%$ relative improvement in $C_{avg}$ and about $45\%$ in EER over the best known result, while using about $91\%$ fewer parameters than the best-known large model. The ablation series shows a clear progression from D-TDNN + Softmax to AAM-Softmax, then to dynamic kernel convolution, local multi-scale Dk Conv, and finally global plus local multi-scale Dk Conv; the largest jump occurs when local multi-scale learning is introduced [2108.07787].

In salient object detection, the kernel ablation is especially informative. Single fixed kernels yield DUTS-TE $F_\beta$ values of $0.891$ for $K3$, $0.907$ for $K5$, $0.905$ for $K7$, $0.900$ for $K9$, and $0.899$ for $K11$, showing that no single receptive field is uniformly optimal. Dynamic multi-kernel routing improves this to $0.916$ on DUTS-TE and $0.837$ on DUT-OMRON with the branch set $\{K3,K5,K7,K9\}$, and explicit routing improves over the same architecture without routing from $0.907$ to $0.916$ on DUTS-TE [2210.13821].

In medical segmentation, the pattern is similar but more heterogeneous. On TCIA pancreas segmentation, 2D U-Net improves from DSC $82.80$ to $84.39$ with DMSC alone, while 3D U-Net improves from $83.56$ to $85.67$ with 3D DMSC-Net; combining DMSC and DMRC yields $87.97$ in 3D [2410.02129]. On AMOS, DLK-Net reaches average Dice $88.18$ and D-Net $89.01$; on the brain-tumor task, DLK-Net reaches $74.57$ and D-Net $75.70$, indicating that dynamic feature fusion and salience refinement add to the gains from the dynamic large-kernel operator itself [2403.10674]. In the explicit DMSK ablations of Dynamic Skip Connection, DMSK alone improves UNETR on Abdomen MRI from Dice $0.6867 \pm 0.1488$ to $0.6943 \pm 0.1408$ and NSD from $0.7440 \pm 0.1627$ to $0.7534 \pm 0.1603$; on Endoscopy instruments, the gains are smaller but still positive. The kernel-configuration study on ISIC2017 shows that combining small and large kernels outperforms either subset alone [2509.14610].

## 5. Relation to adjacent multi-scale kernel traditions

DMSK in deep neural networks should be distinguished from older and mathematically different multi-scale kernel traditions. In LDDMM, mixture-of-kernels formulations define a static RKHS by summing or integrating admissible kernels,
$$
\mathsf{k}=\sum_i \mathsf{k}_i \quad \text{or} \quad \mathsf{k}(x,y)=\int \mathsf{k}_s(x,y)\,d\lambda(s),
$$
and the paper proves equivalence between sum-of-kernels registration, simultaneous multiscale registration, and semidirect-product decompositions of diffeomorphisms. The scales are predefined and the kernel itself is not adaptively learned online, although scale-specific transformations interact dynamically through group composition [1108.2472].

A different numerical-analysis line appears in the dual-space multilevel kernel-splitting framework, where a translation-invariant kernel is decomposed as
$$
K = W_0 + \sum_l D_l + R_L.
$$
There, “dynamic” means adaptive hierarchical processing on an adaptive tree and level-dependent short Fourier transforms, not learned kernel routing. The framework is multiscale, adaptive, and kernel-dependent, but not a neural DMSK in the operator-learning sense [2308.00292].

Kernel methods for topology and RKHS approximation provide further adjacent usage. The stable multi-scale kernel for persistence diagrams defines a scale-parameterized family
$$
k_\sigma(F,G) = \frac{1}{8\pi\sigma} \sum_{p\in F,q\in G} e^{-\frac{\|p-q\|^2}{8\sigma}} - e^{-\frac{\|p-\overline q\|^2}{8\sigma}},
$$
which is positive definite and stable with respect to the $1$-Wasserstein distance, but “multi-scale” there means a tunable diffusion scale rather than dynamic scale selection [1412.6821]. Sparse multiscale learning defines a weighted multiscale RKHS
$$
K(x,y)=\sum_{s\in\mathcal I}\zeta_s\sum_{j\in r_s}\psi_j^s(x)\psi_j^s(y)
$$
and a forward-backward greedy algorithm that adaptively selects active scales and basis functions, again without neural branch routing [2102.07068]. Task-specific Gaussian bandwidth selection for manifold learning and classification likewise provides criteria for choosing $\epsilon$ through intrinsic-dimension consistency, embedding geometry, spectral eigengaps, or within-class Markov transitions, but remains primarily a single-scale selection framework rather than a dynamic multi-kernel mixture [1707.01093].

At the other end of the spectrum, KernelDNA is explicitly dynamic but not explicitly multi-scale. It derives child kernels from shared parent kernels through dynamic channel routing and static filter/spatial modulation while preserving standard convolution structure at inference. Its relevance to DMSK is architectural: it shows that dynamic kernel specialization can be made parameter-efficient without maintaining large per-layer kernel banks, but it does not define multiple kernel sizes or multiple receptive-field scales [2503.23379].

## 6. Misconceptions, limitations, and open issues

A common misconception is that DMSK necessarily means synthesizing a new convolution kernel tensor for every sample. Much of the literature does not do this. Dynamic multi-scale convolution uses softmax-based weighting over two fixed temporal branches; DPConv uses soft routing over a fixed pyramid; DMSC uses channel-wise recalibration after fixed multi-kernel branches; and DLK uses spatial-wise weighting over sequential large-kernel features rather than dynamic kernel generation [2108.07787] [2210.13821] [2410.02129] [2403.10674]. The polyp Dynamic Kernel method is one of the clearer examples of conditional parameter generation, but even there the operator is a stage-recursively updated $1\times1$ kernel whose multi-scale character is inherited from encoder and decoder hierarchy rather than from multiple kernel supports [2509.23502].

Another misconception is that “multi-scale” always refers to multiple kernel sizes. In the surveyed formulations it may instead refer to dilated versus undilated temporal support, Res2Net-style within-block hierarchy, cross-layer pooling of bottleneck features, encoder-stage aggregation, or scale-parameterized RKHS families. This plurality explains why DMSK-like work can look superficially inconsistent while still targeting the same underlying problem: fixed receptive fields are often too rigid when discriminative evidence is distributed across heterogeneous scales [2108.07787] [2509.23502] [1412.6821].

Reproducibility is uneven. The dialect-identification work leaves some internals underspecified, including the exact D-TDNN layout, the exact bottleneck-layer indices used in global pooling, the reduction ratio $r$, and precise tensor dimensions through each block [2108.07787]. The explicit DMSK paper does not specify exact candidate kernel sizes, exact dilation rates, the projection reduction ratio, or DMSK-specific parameter and FLOP counts [2509.14610]. The polyp Dynamic Kernel paper contains notation truncations and brace errors in several equations and does not report ablations isolating Encoder Attention, Dynamic Kernel, and Unified Channel Adaptation [2509.23502].

Efficiency claims also require qualification. DPConv is dynamically routed but not computationally sparse, since all branches are evaluated before weighted concatenation [2210.13821]. DMSC improves segmentation but can increase parameters and FLOPs substantially in 2D unless lightweight substitutions are used; the 2D $5\times5$ DMSC-Net on TCIA uses $89.61$M parameters and $343.13$G FLOPs, compared with $22.80$M and $94.86$G for the baseline U-Net [2410.02129]. Conversely, dynamic multi-scale convolution for dialect identification reduces parameters through local channel splitting, and KernelDNA suggests a plausible route toward more efficient future DMSK designs by combining dynamic specialization with cross-layer weight sharing rather than large branch banks [2108.07787] [2503.23379].

These limitations suggest two broad research directions. A plausible implication is that future DMSK systems may benefit from making the meaning of “scale” more explicit, separating operator-scale diversity from feature-fusion diversity and reporting both independently. Another plausible implication is that explicit multi-scale kernel banks could be combined with lightweight sharing or adapter schemes, so that dynamic scale selection does not force linear growth in kernel-bank parameters. Across current arXiv usage, DMSK is therefore best regarded not as a settled module but as a convergent design principle: multi-scale receptive fields become most effective when their use is conditioned on the input rather than fixed by architecture alone.

Source: https://www.emergentmind.com/topics/dynamic-multi-scale-kernel-dmsk