---
title: Depthwise Dilated Convolution
url: https://www.emergentmind.com/topics/depthwise-dilated-convolution
type: topic
---

# Depthwise Dilated Convolution

Depthwise Dilated Convolution (DDC) designates a class of neural network operations that extend standard depthwise convolution by introducing dilation into the spatial sampling of each channel’s kernel, enabling dramatic enlargement of the receptive field without any increase in parameter count or computational cost per filter. DDC and its variants are a key building block in lightweight, high-throughput architectures for embedded vision, audio analysis, and remote sensing, allowing competitive or superior accuracy compared to classical dense convolutional paradigms, with orders-of-magnitude reductions in parameter count and energy footprint.

## 1. Mathematical Formulation and Definitions

Let $X\in \mathbb{R}^{C\times H \times W}$ denote the input tensor; $W[c,u,v]$ the per-channel kernel for channel $c$ of spatial dimension $K\times K$; and $r$ the dilation rate. The output $Y[c,i,j]$ of a depthwise dilated convolution is
$$
Y[c,i,j] = \sum_{u=0}^{K-1} \sum_{v=0}^{K-1} W[c,u,v] \cdot X[c,\,i + r\cdot u,\,j + r\cdot v]
$$
as formalized in [2104.14125], [2505.00374], [2512.07391], [2002.00476], and [2306.05624]. Here, each input channel is filtered independently using its own kernel, while the dilation factor $r$ determines the stride by which kernel values are "spread" spatially, expanding the effective receptive field from $K$ to $r(K-1)+1$ with no increase in weights or MACs.

Depthwise separable dilated convolution extends this by (1) applying the above operation per channel, then (2) merging the result with a $1\times1$ pointwise convolution for cross-channel mixing:
$$
y_{c'}(i,j) = \sum_{c=1}^{C} W^{(pw)}(1,1,c,c') \cdot \left[ \sum_{m,n} W^{(d)}_{c}(m,n) x_{c}(i + r\,m,\,j + r\,n) \right]
$$
where $W^{(pw)}$ collects pointwise mixing weights [2505.00374], [2002.00476], [2306.05624].

## 2. Architectural Implementations and Variants

The architectural integration of depthwise dilated convolution differs across domains and objectives but fundamentally exploits the same core principles.

**Unified hardware acceleration:** The design in [2104.14125] proposes a single highly parameterized accelerator CLPU that supports regular, depthwise, and dilated convolutions by exploiting an address generator for stride/dilation emulation with no logic or buffer overhead. Odd-sized filters ($3\times3,\,5\times5,\,7\times7$) and arbitrary dilations are efficiently supported.

**Grouped/multi-rate structures:** GlimmerNet [2512.07391] introduces Grouped Dilated Depthwise Convolution (GDBlock), partitioning the $C$ channels into $m$ groups, each group with its own kernel and dilation factor $d_g$. This enables parallel multi-scale extraction, augmenting spatial diversity at constant parameter cost. An aggregator module based on grouped 1$\times$1 pointwise convolutions fuses these features efficiently.

**Dilated fusion blocks:** DSDCN [2505.00374] implements a parallel fusion module in hyperspectral super-resolution, applying $3\times3$ depthwise separable convolutions with dilation rates $r\in\{1,2,3\}$ in parallel, concatenating, and pointwise fusing the results—a pattern analogously seen in audio [2306.05624] and image contexts.

**Learnable spatial arrangements:** DCLS [2112.03740] replaces regular-grid fixed dilation with learnable floating-point positions within a large kernel grid, constructing sparse, channel-specific kernels via bilinear interpolation. This technique is especially effective in depthwise settings, where each channel’s spatial context can be adaptively shaped during learning.

## 3. Computational and Memory Complexity

The parameter and computational savings of depthwise dilated convolution are substantial:

| Operator             | Parameters                   | MACs per output location                         | Receptive Field           |
|----------------------|-----------------------------|--------------------------------------------------|---------------------------|
| Standard convolution | $K^2\cdot M \cdot N$        | $K^2\cdot M \cdot N$                             | $K$                       |
| Depthwise separable  | $K^2\cdot M + M\cdot N$     | $K^2\cdot M + M\cdot N$                          | $K$                       |
| Depthwise dilated    | $K^2\cdot M + M\cdot N$     | $K^2\cdot M + M\cdot N$                          | $r(K-1)+1$                |
| Grouped DDC (GlimmerNet) | $k^2C + C^2/m$         | $(k^2C + C^2/m)$ (grouped pointwise)             | $\max_g k + (k-1)(d_g-1)$ |

Parameter reduction for depthwise-separable compared to standard convolution is approximately a factor of $1/N + 1/K^2$ (assuming $N \gg K^2$). The introduction of dilation does not change the parameter count or number of MACs, only the spatial pattern of memory access and thus the receptive field ([2104.14125], [2505.00374], [2512.07391], [2306.05624]). Grouped and multi-dilated structures further multiply receptive field diversity at fixed cost.

## 4. Practical Performance and Empirical Studies

Across vision, hyperspectral imaging, and audio domains, DDC layers consistently achieve strong or superior accuracy at a fraction of the computational burden:

- **Face detection (RetinaFace, MobileNetV1-0.25):** Replacing cascaded $3\times3$ context modules with DDC ($3\times3, r=1,2$) plus pointwise reduces context module size from 138KB to 23KB and compute from 708 to 119 MACs/pixel—31% lower total compute. Validation accuracy remains within 1% of baseline. Hardware accelerator delivers 20% higher throughput (180 vs 150 fps) [2104.14125].
- **ImageNet classification:** Swapping $3\times3$ with $5\times5$ depthwise increases Top-1 from 68.39% to 69.44%. Hybrid dilation variants ($r=2,3$ in late layers) yield further small gains [2104.14125].
- **DCLS on ConvNeXt:** Replacing depthwise convolutions with learnable DCLS kernels improves Top-1 accuracy from 82.1% to 82.5%, outperforms both classic dilation (Top-1 80.8%) and large dense kernels (Top-1 82.0%), with only minor throughput hit ($\sim$7% slowdown) [2112.03740].
- **Hyperspectral SR (DSDCN):** Parameter overhead $<$1M, first in PSNR and SSIM on PaviaC dataset, matching or beating larger models, confirming the efficiency and efficacy of DDC for high-dimensional input [2505.00374].
- **Audio event classification:** Multi-scale dilated depthwise-separable conv boosts accuracy by 3.4% in domestic activity recognition compared to non-dilated; for polyphonic SED ensembles, depthwise-dilated blocks yield 85% parameter reduction and >78% faster training per epoch, with +4.6% F1 and –3.8% error [2306.05624], [2002.00476].
- **Ultra-lightweight emergency scene recognition:** GlimmerNet achieves F1=0.966 with only 31.2K parameters and 22.3M FLOPs, outperforming much larger transformer and CNN baselines with more than $10\times$ parameters and $100\times$ FLOPs [2512.07391].

## 5. Receptive Field Control and Multi-Scale Contextualization

Dilation in DDC explicitly decouples receptive field growth from parameter count. For a single $3\times3$ kernel with $d=2$ the receptive field expands from 3 to 5, but uses only 9 parameters rather than 25. Layer stacking, or grouped variants (each group with different $d_g$) further extend context. In GlimmerNet, grouped DDC and staged downsampling yield effective receptive fields up to $7\times7$ at full resolution, $15\times15$ after one pooling layer.

Multi-scale embedding strategies, as in [2306.05624], concatenate pooled outputs from multiple DDC blocks at different depths. Empirical ablation demonstrates both dilation and multi-scale pooling independently boost classification accuracy.

## 6. Advanced Variants: Learnable Spacings and Grouped Structures

DCLS [2112.03740] generalizes DDC by making the spatial positions of kernel weights differentiable and learnable via backpropagation. Each DCLS kernel consists of $m$ weights and continuous ($x$, $y$) positions within an $s_1\times s_2$ grid, using bilinear interpolation to project weights onto the grid:
$$
K = \sum_{k=1}^m f(w_k, p_k^1, p_k^2)
$$
followed by standard depthwise convolution. This can expand the receptive field almost arbitrarily without increasing kernel density, and consistently outperforms both fixed-rate dilation and very large fixed kernels.

In grouped constructs like GlimmerNet [2512.07391], the total parameter and compute cost is held constant by dividing channels into $m$ groups, applying distinct dilation per group, then recombining feature maps via grouped pointwise convolutions. This strategy enables simultaneous extraction of local and global features and leads to significant accuracy improvements in small-footprint networks.

## 7. Limitations, Trade-offs, and Design Best Practices

- **Accuracy vs extreme compression:** While DDC and derivatives offer minimal loss in accuracy compared to conventional or even transformer-based architectures, excessive replacement of context modules or aggressive spatial simplification (e.g., reducing all context to DDC) can result in a sub-percent accuracy drop [2104.14125].
- **Hardware design:** Cost-free support for dilation and large kernels is achievable by integrating the dilation stride logic into the address generation stage. Additional on-chip buffer may be required for multi-channel input in depthwise mode. Implementation of address generation for arbitrary dilations can moderately increase hardware complexity [2104.14125].
- **Kernel learnability and dynamic placement:** In DCLS, learning the spatial positions of active taps enhances expressivity but slightly reduces throughput compared to fixed-grid dilated or separable kernels [2112.03740].
- **Best practices for network design:** Universal recommendations encompass ReLU (or ReLU6) activation, batch normalization after convolutional layers, and max-pooling for dimension reduction between DDC modules. Dropout between blocks is favored for regularization, especially in low-parameter regimes. Multi-scale feature extraction through depth stacking and/or explicit concatenation is empirically validated to improve discriminatory capacity [2306.05624], [2512.07391].

---

Depthwise dilated convolution and its variants constitute foundational operations for highly parameter- and energy-efficient deep neural networks. Through precise control of receptive field, memory usage, and compute via dilation and channel-wise separation, DDC-based modules deliver state-of-the-art accuracy in embedded vision, audio, and remote sensing tasks while supporting real-time inference within severe resource budgets [2104.14125], [2505.00374], [2512.07391], [2002.00476], [2306.05624], [2112.03740].

Source: https://www.emergentmind.com/topics/depthwise-dilated-convolution