Dynamic Multi-Scale Kernel in Deep Learning

• Dynamic Multi-Scale Kernel (DMSK) is an adaptive mechanism that conditionally selects and weights convolution kernels from multiple scales using global context.
• It integrates soft routing and hard kernel selection patterns to address the limitations of fixed receptive fields across speech, vision, and medical imaging.
• Empirical studies show that DMSK improves accuracy and efficiency by reducing parameters while enhancing segmentation and classification performance.

Dynamic Multi-Scale Kernel (DMSK) denotes an input-conditioned multi-scale kernel mechanism in which receptive-field scale is not fixed a priori but is selected, weighted, recalibrated, or recursively updated from global or task-specific context. The label appears explicitly in the Dynamic Skip Connection framework for medical segmentation (Cao et al., 18 Sep 2025), but closely related mechanisms are described under other names in speech, vision, and medical imaging, including dynamic multi-scale convolution (Kong et al., 2021), Dynamic Pyramid Convolution (Wu et al., 2022), Dynamic Multi-Scale Convolution (Yang et al., 2024), and Dynamic Large Kernel (Yang et al., 2024). Across these formulations, the central objective is consistent: replace fixed-kernel processing with input-adaptive use of multiple temporal or spatial scales.

1. Scope and terminological usage

The most explicit use of the term occurs in the DMSK module inside the Dynamic Skip Connection block, where the module is introduced to mitigate the “intra-feature constraint” of conventional skip connections by adaptively selecting kernel sizes from small-scale and large-scale sets according to global contextual cues (Cao et al., 18 Sep 2025). Earlier and parallel work uses different nomenclature for closely related ideas. In dialect identification, the relevant term is dynamic multi-scale convolution; in salient object detection, Dynamic Pyramid Convolution; in pancreas segmentation, Dynamic Multi-Scale Convolution; and in volumetric segmentation, Dynamic Large Kernel (Kong et al., 2021, Wu et al., 2022, Yang et al., 2024, Yang et al., 2024).

Formulation	Dynamic mechanism	Multi-scale mechanism
DMSK in DSC (Cao et al., 18 Sep 2025)	Softmax scoring followed by $\arg\max$ kernel selection	Small-scale and large-scale kernel sets in skip processing
Dynamic multi-scale convolution (Kong et al., 2021)	Channel-wise softmax branch weighting	Undilated and dilated temporal kernels, local split hierarchy, global multi-layer pooling
DPConv (Wu et al., 2022)	GAP-MLP-Softmax routing over branches	Parallel kernels $\{K3,K5,K7,K9\}$
DMSC (Yang et al., 2024)	Global-context sigmoid recalibration	Parallel $3\times3$ and $5\times5$ branches
DLK (Yang et al., 2024)	Spatial-wise dynamic selection	Sequential large kernels $5\times5\times5$ and $7\times7\times7$ with dilation $3$

This diversity makes DMSK less a single canonical operator than a family of constructions. Some methods perform explicit operator-level scale selection; others retain fixed branch kernels and apply dynamic feature-side weighting. A narrower but important adjacent case is the polyp-segmentation Dynamic Kernel method, whose “multi-scale” character comes from encoder-wide context aggregation and multi-resolution recursive updates rather than from an explicit bank of kernels with different spatial supports (Chashmi et al., 27 Sep 2025).

2. Core operator patterns

A first recurrent pattern is soft routing over fixed kernel branches. In dynamic kernel convolution for dialect identification, two temporal branches are computed, one undilated and one with dilation factor $2$. Their outputs are summed as

$X = h_d^1 + h_d^2,$

high-order statistic pooling extracts per-channel mean, standard deviation, skewness, and kurtosis, and two dense layers followed by softmax produce channel-wise branch weights $s_i$. The fused output is

$\{K3,K5,K7,K9\}$0

The dynamic behavior is therefore per-sample channel-wise branch reweighting rather than per-frame kernel synthesis (Kong et al., 2021). A related formulation appears in DPConv, where global average pooling produces a descriptor $\{K3,K5,K7,K9\}$1, routing weights are obtained by

$\{K3,K5,K7,K9\}$2

and the final output is

$\{K3,K5,K7,K9\}$3

Here all branches are evaluated and dynamically modulated, so routing is soft and differentiable rather than sparse (Wu et al., 2022).

A second pattern is hard kernel selection with staged filtering. The explicit DMSK module first computes global average pooled context

$\{K3,K5,K7,K9\}$4

then predicts probability vectors over small-scale and large-scale candidates,

$\{K3,K5,K7,K9\}$5

and selects

$\{K3,K5,K7,K9\}$6

The selected kernels are cascaded through depthwise convolutions,

$\{K3,K5,K7,K9\}$7

after which spatial and channel attention refine the result before residual addition. This design makes DMSK a context-conditioned candidate-kernel selector rather than a continuous mixture (Cao et al., 18 Sep 2025).

A third pattern is post-branch recalibration rather than direct kernel generation. In DMSC, fixed $\{K3,K5,K7,K9\}$8 and $\{K3,K5,K7,K9\}$9 branches produce $3\times3$0 and $3\times3$1, which are concatenated and globally pooled. A $3\times3$2 projection and sigmoid yield channel-wise gates

$3\times3$3

and recalibration proceeds by

$3\times3$4

The dynamic component thus lies on feature aggregation rather than on convolution weights themselves (Yang et al., 2024). By contrast, the polyp-segmentation Dynamic Kernel method generates an initial decoder-space kernel from encoder attention,

$3\times3$5

then updates it recursively via

$3\times3$6

so it is a conditional parameter-generation method whose multi-scale character is inherited from encoder-scale aggregation and decoder-stage refinement (Chashmi et al., 27 Sep 2025).

These operator patterns also differ in where “multi-scale” is instantiated. It may be encoded by different kernel sizes or dilations, by hierarchical channel splitting within a block, or by cross-depth aggregation of bottleneck features. This suggests that DMSK is best understood as a design space spanning operator-scale diversity, context-conditioned routing, and cross-level feature fusion rather than a single fixed template.

3. Representative neural realizations

In speech processing, dynamic multi-scale convolution is instantiated on a D-TDNN backbone for dialect identification. The first D-TDNN layer is replaced by a multi-scale Dk block that combines dynamic kernel convolution, local multi-scale learning with channel splitting parameter $3\times3$7, and global multi-scale pooling from two bottleneck layers. All layers use $3\times3$8 filters; the dynamic kernel branches use kernel size $3\times3$9 with dilation $5\times5$0 and $5\times5$1; the first TDNN layer and last six D-TDNN layers use context $5\times5$2 and other layers use $5\times5$3; training is performed with AAM-Softmax (Kong et al., 2021). In speaker verification, selective-kernel variants extend ECAPA-family systems through channel-wise SKA, frequency-wise SKA, sequential frequency-plus-channel SKA, and msSKA inside Res2Net. The front-end uses $5\times5$4 and $5\times5$5 branches, msSKA uses temporal kernels $5\times5$6 and $5\times5$7, the Res2Net scale is $5\times5$8, and the channel width is $5\times5$9 (Mun et al., 2022).

In dense prediction for natural images, DPNet moves dynamic scale routing into the encoder through Dynamic Pyramid Convolution, then redesigns the decoder with BiCFM and Dynamic Weighted Fusion. The best DPConv branch library is $5\times5\times5$0, and branch routing is derived from globally pooled statistics by a two-layer MLP with softmax (Wu et al., 2022). In medical segmentation, DMSC uses parallel $5\times5\times5$1 and $5\times5\times5$2 convolutions with global channel recalibration, while DMRC introduces a separate multi-resolution pathway; both are inserted as replacements for single convolutions in U-Net-style blocks (Yang et al., 2024). D-Net replaces self-attention in a hierarchical encoder-decoder with Dynamic Large Kernel blocks composed of sequential depthwise $5\times5\times5$3 and $5\times5\times5$4 convolutions, with the second using dilation $5\times5\times5$5, followed by spatial-wise dynamic selection and Dynamic Feature Fusion for skip connections (Yang et al., 2024).

The explicit DMSK module in Dynamic Skip Connection is architecturally narrower but terminologically decisive. It is applied to skip features, uses global-average-pooled context to choose one small and one large kernel by $5\times5\times5$6, then combines spatial attention, channel attention, and residual propagation. The same paper positions DMSK as the component that addresses insufficient modeling of multi-scale feature interactions inside a skip pathway, while a separate Test-Time Training module addresses dynamic adaptation across samples (Cao et al., 18 Sep 2025). A nearby but technically distinct design is the Encoder-Attention plus Dynamic Kernel framework for polyp segmentation, in which all five encoder stages contribute to a global context vector $5\times5\times5$7, a decoder channel width $5\times5\times5$8 is enforced by Unified Channel Adaptation, and a $5\times5\times5$9 dynamic kernel is recursively refined from stage to stage using lesion-aware pooled features (Chashmi et al., 27 Sep 2025).

4. Empirical behavior

Reported experiments consistently show that fixed single-scale kernels are not uniformly optimal and that dynamic multi-scale mechanisms tend to improve accuracy when discriminative evidence is distributed across heterogeneous temporal or spatial extents.

Setting	Comparison	Headline result
AP20-OLR dialect task	D-TDNN + AAM-Softmax vs Global and Local Multi-scale Dk Conv	$7\times7\times7$0 from $7\times7\times7$1 to $7\times7\times7$2; EER from $7\times7\times7$3 to $7\times7\times7$4; params from $7\times7\times7$5M to $7\times7\times7$6M
DUTS-TE / DUT-OMRON	Best fixed kernel vs dynamic $7\times7\times7$7	DUTS-TE $7\times7\times7$8 from $7\times7\times7$9 for $3$0 to $3$1; DUT-OMRON $3$2
TCIA pancreas segmentation	3D U-Net vs 3D DMSC-Net vs 3D DMC-Net	DSC $3$3
AMOS / brain tumor segmentation	DLK-Net vs D-Net	AMOS average Dice $3$4; brain tumor average Dice $3$5
ISIC2017 / UNETR ablations	Small-only or large-only vs Small+Large DMSK	mIoU $3$6 or $3$7 vs $3$8; Dice $3$9 or $2$0 vs $2$1

On the AP20-OLR-dialect-task, the full dynamic multi-scale convolution model obtains the best average cost performance $2$2 and EER $2$3, with about $2$4 relative improvement in $2$5 and about $2$6 in EER over the best known result, while using about $2$7 fewer parameters than the best-known large model. The ablation series shows a clear progression from D-TDNN + Softmax to AAM-Softmax, then to dynamic kernel convolution, local multi-scale Dk Conv, and finally global plus local multi-scale Dk Conv; the largest jump occurs when local multi-scale learning is introduced (Kong et al., 2021).

In salient object detection, the kernel ablation is especially informative. Single fixed kernels yield DUTS-TE $2$8 values of $2$9 for $X = h_d^1 + h_d^2,$0, $X = h_d^1 + h_d^2,$1 for $X = h_d^1 + h_d^2,$2, $X = h_d^1 + h_d^2,$3 for $X = h_d^1 + h_d^2,$4, $X = h_d^1 + h_d^2,$5 for $X = h_d^1 + h_d^2,$6, and $X = h_d^1 + h_d^2,$7 for $X = h_d^1 + h_d^2,$8, showing that no single receptive field is uniformly optimal. Dynamic multi-kernel routing improves this to $X = h_d^1 + h_d^2,$9 on DUTS-TE and $s_i$0 on DUT-OMRON with the branch set $s_i$1, and explicit routing improves over the same architecture without routing from $s_i$2 to $s_i$3 on DUTS-TE (Wu et al., 2022).

In medical segmentation, the pattern is similar but more heterogeneous. On TCIA pancreas segmentation, 2D U-Net improves from DSC $s_i$4 to $s_i$5 with DMSC alone, while 3D U-Net improves from $s_i$6 to $s_i$7 with 3D DMSC-Net; combining DMSC and DMRC yields $s_i$8 in 3D (Yang et al., 2024). On AMOS, DLK-Net reaches average Dice $s_i$9 and D-Net $\{K3,K5,K7,K9\}$00; on the brain-tumor task, DLK-Net reaches $\{K3,K5,K7,K9\}$01 and D-Net $\{K3,K5,K7,K9\}$02, indicating that dynamic feature fusion and salience refinement add to the gains from the dynamic large-kernel operator itself (Yang et al., 2024). In the explicit DMSK ablations of Dynamic Skip Connection, DMSK alone improves UNETR on Abdomen MRI from Dice $\{K3,K5,K7,K9\}$03 to $\{K3,K5,K7,K9\}$04 and NSD from $\{K3,K5,K7,K9\}$05 to $\{K3,K5,K7,K9\}$06; on Endoscopy instruments, the gains are smaller but still positive. The kernel-configuration study on ISIC2017 shows that combining small and large kernels outperforms either subset alone (Cao et al., 18 Sep 2025).

5. Relation to adjacent multi-scale kernel traditions

DMSK in deep neural networks should be distinguished from older and mathematically different multi-scale kernel traditions. In LDDMM, mixture-of-kernels formulations define a static RKHS by summing or integrating admissible kernels,

$\{K3,K5,K7,K9\}$07

and the paper proves equivalence between sum-of-kernels registration, simultaneous multiscale registration, and semidirect-product decompositions of diffeomorphisms. The scales are predefined and the kernel itself is not adaptively learned online, although scale-specific transformations interact dynamically through group composition (Bruveris et al., 2011).

A different numerical-analysis line appears in the dual-space multilevel kernel-splitting framework, where a translation-invariant kernel is decomposed as

$\{K3,K5,K7,K9\}$08

There, “dynamic” means adaptive hierarchical processing on an adaptive tree and level-dependent short Fourier transforms, not learned kernel routing. The framework is multiscale, adaptive, and kernel-dependent, but not a neural DMSK in the operator-learning sense (Jiang et al., 2023).

Kernel methods for topology and RKHS approximation provide further adjacent usage. The stable multi-scale kernel for persistence diagrams defines a scale-parameterized family

$\{K3,K5,K7,K9\}$09

which is positive definite and stable with respect to the $\{K3,K5,K7,K9\}$10-Wasserstein distance, but “multi-scale” there means a tunable diffusion scale rather than dynamic scale selection (Reininghaus et al., 2014). Sparse multiscale learning defines a weighted multiscale RKHS

$\{K3,K5,K7,K9\}$11

and a forward-backward greedy algorithm that adaptively selects active scales and basis functions, again without neural branch routing (Shekhar et al., 2021). Task-specific Gaussian bandwidth selection for manifold learning and classification likewise provides criteria for choosing $\{K3,K5,K7,K9\}$12 through intrinsic-dimension consistency, embedding geometry, spectral eigengaps, or within-class Markov transitions, but remains primarily a single-scale selection framework rather than a dynamic multi-kernel mixture (Lindenbaum et al., 2017).

At the other end of the spectrum, KernelDNA is explicitly dynamic but not explicitly multi-scale. It derives child kernels from shared parent kernels through dynamic channel routing and static filter/spatial modulation while preserving standard convolution structure at inference. Its relevance to DMSK is architectural: it shows that dynamic kernel specialization can be made parameter-efficient without maintaining large per-layer kernel banks, but it does not define multiple kernel sizes or multiple receptive-field scales (Huang et al., 30 Mar 2025).

6. Misconceptions, limitations, and open issues

A common misconception is that DMSK necessarily means synthesizing a new convolution kernel tensor for every sample. Much of the literature does not do this. Dynamic multi-scale convolution uses softmax-based weighting over two fixed temporal branches; DPConv uses soft routing over a fixed pyramid; DMSC uses channel-wise recalibration after fixed multi-kernel branches; and DLK uses spatial-wise weighting over sequential large-kernel features rather than dynamic kernel generation (Kong et al., 2021, Wu et al., 2022, Yang et al., 2024, Yang et al., 2024). The polyp Dynamic Kernel method is one of the clearer examples of conditional parameter generation, but even there the operator is a stage-recursively updated $\{K3,K5,K7,K9\}$13 kernel whose multi-scale character is inherited from encoder and decoder hierarchy rather than from multiple kernel supports (Chashmi et al., 27 Sep 2025).

Another misconception is that “multi-scale” always refers to multiple kernel sizes. In the surveyed formulations it may instead refer to dilated versus undilated temporal support, Res2Net-style within-block hierarchy, cross-layer pooling of bottleneck features, encoder-stage aggregation, or scale-parameterized RKHS families. This plurality explains why DMSK-like work can look superficially inconsistent while still targeting the same underlying problem: fixed receptive fields are often too rigid when discriminative evidence is distributed across heterogeneous scales (Kong et al., 2021, Chashmi et al., 27 Sep 2025, Reininghaus et al., 2014).

Reproducibility is uneven. The dialect-identification work leaves some internals underspecified, including the exact D-TDNN layout, the exact bottleneck-layer indices used in global pooling, the reduction ratio $\{K3,K5,K7,K9\}$14, and precise tensor dimensions through each block (Kong et al., 2021). The explicit DMSK paper does not specify exact candidate kernel sizes, exact dilation rates, the projection reduction ratio, or DMSK-specific parameter and FLOP counts (Cao et al., 18 Sep 2025). The polyp Dynamic Kernel paper contains notation truncations and brace errors in several equations and does not report ablations isolating Encoder Attention, Dynamic Kernel, and Unified Channel Adaptation (Chashmi et al., 27 Sep 2025).

Efficiency claims also require qualification. DPConv is dynamically routed but not computationally sparse, since all branches are evaluated before weighted concatenation (Wu et al., 2022). DMSC improves segmentation but can increase parameters and FLOPs substantially in 2D unless lightweight substitutions are used; the 2D $\{K3,K5,K7,K9\}$15 DMSC-Net on TCIA uses $\{K3,K5,K7,K9\}$16M parameters and $\{K3,K5,K7,K9\}$17G FLOPs, compared with $\{K3,K5,K7,K9\}$18M and $\{K3,K5,K7,K9\}$19G for the baseline U-Net (Yang et al., 2024). Conversely, dynamic multi-scale convolution for dialect identification reduces parameters through local channel splitting, and KernelDNA suggests a plausible route toward more efficient future DMSK designs by combining dynamic specialization with cross-layer weight sharing rather than large branch banks (Kong et al., 2021, Huang et al., 30 Mar 2025).

These limitations suggest two broad research directions. A plausible implication is that future DMSK systems may benefit from making the meaning of “scale” more explicit, separating operator-scale diversity from feature-fusion diversity and reporting both independently. Another plausible implication is that explicit multi-scale kernel banks could be combined with lightweight sharing or adapter schemes, so that dynamic scale selection does not force linear growth in kernel-bank parameters. Across current arXiv usage, DMSK is therefore best regarded not as a settled module but as a convergent design principle: multi-scale receptive fields become most effective when their use is conditioned on the input rather than fixed by architecture alone.