---
title: 'Learnable DDC (DCLS-DDC): Efficient Dilated Convolution'
url: https://www.emergentmind.com/topics/learnable-ddc-dcls-ddc
type: topic
---

# Learnable DDC (DCLS-DDC): Efficient Dilated Convolution

Dilated Convolution with Learnable Spacings (DCLS), also widely referred to in the literature as DCLS-DDC, is a convolutional operator that parametrizes the spatial or temporal positions of each nonzero element ("tap") in a convolutional kernel as continuous, learnable offsets. These offsets are optimized end-to-end via backpropagation and handled by differentiable interpolation schemes, such as bilinear or Gaussian kernels, to support non-integer sampling locations. DCLS-DDC directly generalizes both standard dilated and dense convolution, enabling efficient enlargement of the effective receptive field without increasing parameter count or imposing a rigid sampling grid. The DCLS methodology has been validated across computer vision, audio, and neuromorphic computing domains, showing consistent improvements in accuracy and robustness at fixed model size, as well as enhanced alignment between model saliency maps and human visual attention.

## 1. Mathematical Formulation and Interpolation Mechanisms

A DCLS layer defines the convolutional kernel as a superposition of $M$ nonzero elements ("points"), each parameterized by a scalar weight $w_m$ and a continuous position $p_m = (p^x_m, p^y_m)$ (2D) or $p_m = (p^t_m)$ (1D). The core mechanism is as follows:

- The "spokes" of the kernel are assigned learnable offsets $\Delta p_m$ that are (in general) non-integer.
- At any input location, the DCLS output is computed via
  $$
  y[\mathbf{i}] = \sum_{m=1}^M w_m \cdot I(\mathbf{i} + p_m)
  $$
  where $I(\cdot)$ denotes sampling at fractional locations, handled by interpolation.

**Interpolation choices:**
- **Bilinear (triangle):** Fractional positions are distributed to the four closest integer grid points according to the bilinear formula:
  $$
  I(x+\Delta x, y+\Delta y) = (1-\alpha)(1-\beta) I(x+p, y+q) + \ldots
  $$
  for $\Delta x = p+\alpha$, $\Delta y = q+\beta$ with $p,q\in\mathbb{Z}$, $\alpha,\beta\in[0,1)$.
- **Gaussian:** Each point spreads via a normalized Gaussian kernel of learnable (or fixed) width $\sigma$, supporting more global and smooth influence.

**Gradient computation:** The interpolation operation is differentiable, enabling joint optimization of weights and offsets. Explicit closed-form expressions for $\partial\mathcal{L}/\partial w_m$ and $\partial\mathcal{L}/\partial p_m$ exist for both interpolation types, ensuring compatibility with standard deep learning frameworks [2112.03740, 2306.00817, 2408.03164].

## 2. Training, Initialization, and Regularization Strategies

**Parameter learning:** In DCLS-DDC, weights $w_m$, positional offsets $p_m$, and, for Gaussian DCLS, widths $\sigma_m$ (if learnable) are optimized via standard backpropagation, typically without any auxiliary loss terms on the offsets. The primary training objective is the task-specific loss (e.g., classification cross-entropy).

**Initialization protocols:**
- Positions are initialized to a uniform grid with small random perturbations or sampled from a narrow Gaussian centered in the kernel support.
- Learning rates for offsets are usually multiplied by a factor (e.g., $5\times$) relative to weights, accelerating spatial adaptation.
- For Gaussian DCLS, width $\sigma$ may be initialized higher early in training and annealed to a lower value.

**Regularization:**
- Clamp offsets to the kernel support after each update to prevent drift.
- No weight decay is applied to offsets or widths.
- An optional repulsive loss (to avoid overlap of positions) and stage-wise offset sharing are sometimes deployed but yield only marginal improvements [2408.06383].

## 3. Algorithmic Implementation and Integration

DCLS-DDC is implemented as a drop-in replacement for depthwise-separable convolutions in CNNs or related layers in spiking and audio networks. The process consists of:

- At the start of each forward pass, constructing the effective dense kernel by interpolating each tap onto the full support.
- Efficient kernel construction is achieved via vectorized routines or custom CUDA kernels, especially with grouped or synchronized offsets ($P, \sigma$) per layer or per stage.
- During backpropagation, automatic differentiation propagates gradients through both the interpolation and convolution steps.

A typical high-level pseudocode for 2D DCLS in PyTorch is:

```python
class DCLSConv2d(nn.Module):
    def __init__(self, in_ch, kernel_size):
        self.W = nn.Parameter(torch.randn(N,))           # N = kernel_size**2 or arbitrary
        self.O = nn.Parameter(init_grid_offsets(N))      # shape (N,2)
    def forward(self, x):                                # x: (B,C,H,W)
        grid = make_offset_grid(x.shape, self.O)
        sampled = bilinear_sample(x, grid)
        out_dw = (sampled * self.W.view(1,1,1,1,N)).sum(-1)
        return self.pointwise(out_dw)
```

Alignment with existing deep learning frameworks is achieved via in-place substitutions in architectures such as ResNet, ConvNeXt, ConvFormer, FastViT, or as a component in temporal and spiking neural networks [2408.03164, 2309.13972].

## 4. Empirical Evaluation across Vision, Audio, and Spiking Domains

**Computer Vision:**
- DCLS-DDC provides consistent accuracy gains for image classification on ImageNet-1k. For ConvNeXt-T, substituting DSC with DCLS-Gauss in a 23×23 support (26 taps) increases top-1 accuracy from $82.1\%$ to $82.5\%$ [2306.00817, 2408.03164, 2408.06383]. For ResNet-50, DCLS achieves top-1 accuracy of $76.5\%$ ($+4.8\%$ over 3×3 baseline) at constant parameter count [2112.03740].
- Robustness enhancements are observed on ImageNet-C/A/R/Sketch: clean error rates improve and mIoU rises in semantic segmentation downstream benchmarks (e.g., ADE20K: ConvNeXt-T $46.0 \rightarrow 47.1$) [2408.06383].
- Marginal reduction in throughput (e.g., ConvNeXt-T: $775 \rightarrow 725$ img/s) due to interpolation overhead, but parameter count remains unchanged.

**Audio and Spiking Neural Networks:**
- On AudioSet, DCLS-DDC integrated in ConvNeXt-T/ConvFormer-S18/FastViT-SA24 yields absolute mean average precision improvements of $+0.5$ to $+0.7$ mAP versus DSC, with a throughput cost of $13-23\%$ [2309.13972].
- In temporal SNNs, DCLS-style learnable delays (1D DCLS) enable state-of-the-art classification for SHD, SSC benchmarks with models such as DCLS-delays (3L,2KC) reaching $95.35\%$ accuracy with orders of magnitude fewer parameters than dense convolutional delay models [2408.06383].

## 5. Analysis: Why Learnable Spacings Advance Convolutional Modeling

- By allowing the kernel taps to drift off a rigid grid, DCLS-DDC adaptively concentrates sampling in task-relevant regions (e.g., salient image parts, harmonic structure in audio, optimal synaptic delays).
- Visualization of learned offsets reveals data-dependent, anisotropic “attractor patterns” that expand the effective receptive field without overparameterization [2112.03740, 2306.00817].
- Gaussian interpolation provides smoother, more stable gradient flow than triangle-based schemes, facilitating convergence and slightly improving task accuracy (statistically significant, $p < 10^{-4}$ on ImageNet) [2306.00817].
- No increase in parameter count is incurred as offsets/widths are stage-shared and occupy a negligible fraction of the model footprint.

## 6. Interpretability and Human Alignment

DCLS-DDC not only improves accuracy but enhances the human resemblance of model attention, as measured by the alignment between model-generated Grad-CAM heatmaps and human visual attention maps on the ClickMe dataset [2408.03164]. Replacing standard convolutions with DCLS layers increased the Spearman correlation between model and human saliency maps in seven of eight tested architectures (e.g., ConvNeXt-Base: $0.7565 \rightarrow 0.7979$, ResNet-50: $0.6135 \rightarrow 0.6252$ under Grad-CAM). The introduction of Threshold-Grad-CAM, a variant that thresholds Grad-CAM maps post-normalization, further accentuated these gains.

**Summary Table of Results from [2408.03164]:**

| Model              | Top-1  | Grad-CAM | Thr-Grad-CAM |
|--------------------|--------|----------|--------------|
| convnext_base      | 83.83  | 0.7565   | 0.7572       |
| convnext_base_dcls | 84.09  | 0.7979   | 0.7845       |
| resnet50           | 77.84  | 0.6135   | 0.7125       |
| resnet50_dcls      | 78.35  | 0.6252   | 0.7261       |

The improved interpretability is attributed to more flexible and data-aligned sampling patterns, offering a tangible connection between network computations and human vision strategies.

## 7. Limitations and Prospects

While DCLS-DDC consistently provides modest but reliable performance improvements at iso-parameters, several trade-offs and open challenges remain:

- **Inference Overhead:** Kernel construction and on-the-fly interpolation increase per-batch compute cost by $13-23\%$ relative to DSC, especially in large or high-dimensional settings. This remains manageable for most practical deployments, especially on modern GPU hardware [2309.13972].
- **Architecture Search:** All reported results reflect drop-in substitution for existing convolutions; a plausible implication is that networks architected natively for DCLS (via neural architecture search) may offer further gains [2408.06383].
- **Large-scale/3D Modeling:** Efficient kernel construction for very large supports (e.g., in 3D or with high tap count) may require further algorithmic optimization, including sparse matrix multiplications.
- **Cross-task Generalization:** Although improvements are systematic across classification, segmentation, audio tagging, and SNNs, results in domains with extremely irregular data (e.g., video, point cloud) remain to be established.

Continued evolution of DCLS-DDC is anticipated toward hybrid attention–convolutional frameworks, dedicated DCLS-based architectures, and further studies into its relationship with human and animal biologic perception [2408.06383, 2306.00817, 2408.03164].

Source: https://www.emergentmind.com/topics/learnable-ddc-dcls-ddc