Dynamic Grouping Convolution (DGConv) in CNNs

• Dynamic Grouping Convolution (DGConv) is a CNN primitive that dynamically learns or adapts channel partitioning, enabling flexible and efficient connectivity.
• DGConv mitigates the limitations of static convolution by dynamically selecting salient channels per input, thereby enhancing accuracy and adaptability.
• Empirical studies show DGConv achieves 2–4× reductions in parameters and FLOPs while maintaining or improving performance in both 2D and 3D architectures.

Dynamic Grouping Convolution (DGConv) is a family of convolutional neural network (CNN) primitives that dynamically learn or adapt group structures for channel partitioning in convolution operations. Originating as a response to the weaknesses of static group convolution—such as fixed connectivity and limited representational flexibility—DGConv seeks to maximize the computational efficiency advantage of grouping while avoiding the loss of accuracy induced by static, sample-agnostic, or manually determined channel assignments. DGConv approaches have been deployed in 2D and 3D CNNs for vision, remote sensing, and resource-constrained inference scenarios, demonstrating consistent improvements in accuracy-complexity trade-offs, adaptability, and feature diversity.

1. Motivation and Problem Formulation

Standard group convolution partitions the input and output channels into $G$ disjoint groups, applying convolutions independently to each. This reduces computational cost by a factor of $1/G$, but the hard partitioning permanently removes cross-group connections, causing reduced representational capacity and fixed inductive bias regardless of data or task (Su et al., 2020, Zhang et al., 2019).

The core observation motivating DGConv is that, for a given input image (or hyperspectral cube), only a subset of channels are typically salient for producing a particular output activation (Su et al., 2020). Static learned-group approaches (e.g. LGC or FLGC) can train connectivity patterns but freeze them after training, making them inflexible to per-sample variation or evolving data structure. Therefore, DGConv introduces architectures in which group structure is explicitly learnable or even dynamically determined per input sample. This allows the network to adaptively select channel-group assignments or input-channel usage conditioned on the features themselves (Zhang et al., 2019, Yang et al., 2021, Li, 2023, Li et al., 2023).

2. Mathematical Definitions and Core Architectures

DGConv encompasses several technical instantiations. All are unified by the presence of a learnable or input-conditioned mapping—often a binary or soft selector or mask—that determines which input channels and filters participate in each group convolution.

Selector-based Dynamic Group Convolution

One prominent approach, as used in "Dynamic Group Convolution for Accelerating Convolutional Neural Networks" (Su et al., 2020), divides the computation into $H$ heads (groups). For input $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$, each head $g$ uses a feature selector to compute a saliency vector $\mathbf{s}^g \in \mathbb{R}^C$, typically via

$$\mathbf{s}^g = (W^g \ \mathrm{pool}(\mathbf{x}) + b^g)_+$$

where global pooling and a two-layer MLP produce per-channel importances. The top $K = (1-\xi)C$ channels (with $\xi$ as pruning rate) are selected via TopK thresholding, generating a binary mask $m^g \in \{0,1\}^C$. Masked input $1/G$0 is then processed by the group convolution. Each head thus attends only to a dynamically selected subset of channels, conditioned on the sample. Finally, outputs are concatenated, optionally shuffled and passed through BN+ReLU.

DGConv via Learnable Binary Relationship Matrix

"Groupable ConvNet" (Zhang et al., 2019, Yang et al., 2021) proposes a generalization wherein the static block-diagonal channel relation is replaced by a learnable binary matrix $1/G$1. The group structure is parameterized compactly via Kronecker factors: $1/G$2 with

$1/G$3

for $1/G$4. The vector $1/G$5 is binarized from real-valued parameters $1/G$6 using the straight-through estimator, allowing end-to-end learning via gradient descent. This structure unifies dense, group, and depthwise convolutions as special cases.

3D Dynamic Group Convolution (Remote Sensing)

For hyperspectral and volumetric data, DGConv adopts specialized forms. DGCNet (Li, 2023) and LGCNet (Li et al., 2023) extend the selection/masking mechanism to 3D convolutions. At each layer, lightweight selectors (typically global average pooling followed by bottleneck FC layers and sigmoidal/truncated activations) generate channel-wise gating masks for each group, dynamically pruning or allowing channels per input (Li, 2023). Alternatively, groups can be represented by "soft" selection matrices with row-wise softmaxes, replaced by hard assignments at inference for maximal efficiency (Li et al., 2023).

3. Algorithmic Workflow and Training Methods

For sample-adaptive selector-based DGConv (Su et al., 2020, Li, 2023):

• Forward pass: For each group, compute saliency over input channels, select top-K (via thresholding or TopK), apply binary mask, and perform group convolution.
• Backward pass: Gradients are propagated only through selected channels; a small $1/G$7-norm loss on saliency values can encourage sparsity.
• Sparsity scheduling: The pruning rate $1/G$8 is scheduled during training (e.g., via cosine ramp), allowing a gradual transition from dense to pruned structure.
• End-to-end optimization: All selector parameters and convolution weights are trained jointly with standard loss (e.g., cross-entropy).

For differentiable grouping via learnable gates or selection matrices (Zhang et al., 2019, Yang et al., 2021, Li et al., 2023):

• Parameter update: Gates are real-valued, binarized via STE, and updated via backpropagation. The group structure and count can vary per layer.
• Complexity regularization: An explicit FLOPs or parameter-based regularizer is added to control network complexity and favor parsimonious groupings.
• Inference: At deployment, the soft selection is replaced with hard assignment (argmax or threshold), enabling efficient static execution using library group convolution primitives.

For 3D DGConv in hybrid architectures (Li, 2023, Li et al., 2023):

• In DenseNet-style backbones: Each dense block applies 1×1×1 and 3×3×3 DGC layers, with learnable grouping, preserving overall connectivity and growth rate scheduling.
• Selector networks: Typically, selector modules are implemented as two-layer bottleneck networks or via soft row-wise assignment, supporting backpropagation and immediate deployment in standard deep-learning frameworks.

4. Computational Efficiency and Theoretical Analysis

DGConv achieves speedup and parameter reduction through learned or dynamic pruning:

• The theoretical FLOPs scaling for selector-based DGConv is

$1/G$9

with selector cost negligible when $H$0 (Su et al., 2020). The effective computational gain is approximately $H$1.

• For hard-gated approaches, offline reordering of channels enables running as fast as library static group convolutions after grouping is fixed (Li et al., 2023).
• In practical deployments, DGConv modules produce up to 2–4× reductions in parameter count and FLOPs, and halve inference latency compared to dense or even static grouped baselines, with minimal or no loss in accuracy (Li, 2023, Li et al., 2023).
• Complexity regularization explicitly constrains learned group structure to meet a target computation budget (Zhang et al., 2019, Yang et al., 2021).

5. Experimental Results and Empirical Insights

DGConv approaches consistently demonstrate superior accuracy-efficiency trade-offs across standard and domain-specific tasks:

Model/Setting	Accuracy/Metric Gain	Efficiency Gain	Reference
DGC, CIFAR-10/CondenseNet	$H$2 vs $H$3 error	51M vs 66M MACs	(Su et al., 2020)
DGC, ImageNet/ResNet-18	$H$4 vs $H$5 error	2.04$H$6 speedup	(Su et al., 2020)
GroupNet, ImageNet (ResNeXt50)	$H$7 vs $H$8 Top-1	$H$9 ResNeXt50 FLOPs	(Zhang et al., 2019)
SepG-ResNet18, Berlin	OA $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$0 vs $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$1 SOTA	Variance $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$2	(Yang et al., 2021)
DGCNet, Indian Pines (3D DenseNet)	OA $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$3	$\mathbf{x} \in \mathbb{R}^{C \times H \times W}$4 param/FLOP $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$5	(Li, 2023)
LGCNet, Indian Pines (3D DenseNet)	+$\mathbf{x} \in \mathbb{R}^{C \times H \times W}$6–$\mathbf{x} \in \mathbb{R}^{C \times H \times W}$7 OA	2–4$\mathbf{x} \in \mathbb{R}^{C \times H \times W}$8 param/FLOP $\mathbf{x} \in \mathbb{R}^{C \times H \times W}$9	(Li et al., 2023)

Ablations show that:

• DGConv outperforms static group and even static learned-group baselines given fixed parameters/FLOPs (Su et al., 2020, Zhang et al., 2019).
• The optimal number of heads/groups and pruning schedule are dataset and backbone-dependent, but moderate grouping (e.g., $g$0) typically achieves the best balance.
• Grouping regularization is not strictly necessary but can improve load balancing across groups (Li et al., 2023).
• Dynamic per-sample grouping yields input-adaptivity not available to any static partition and is especially beneficial in deeper, wider layers (Su et al., 2020, Yang et al., 2021).

6. Applications and Integrations

DGConv is compatible with a wide range of standard and efficient CNN architectures:

• ResNets and MobileNets: Inserted into residual blocks, replacing standard or pointwise convolutions to accelerate deep models with minimal architectural change (Su et al., 2020).
• CondenseNet and EfficientNets: DGC can replace learned group convolution (LGC) modules for further accuracy and efficiency gain (Su et al., 2020).
• Remote Sensing/Multimodal Fusion: In single-stream architectures for multi-source data (hyperspectral, SAR, LiDAR), DGConv matches or surpasses late-fusion SOTA while reducing variance and enabling data-driven sensor branch allocation (Yang et al., 2021).
• 3D Densenets for HSI: DGCNet and LGCNet replace standard 3D convolutions with dynamic modules, achieving state-of-the-art accuracy and extreme compute reduction on the Indian Pines, Pavia, and KSC benchmarks (Li, 2023, Li et al., 2023).

Implementation requires only a minor extension of standard PyTorch/TensorFlow Conv modules, supporting practical adoption (Yang et al., 2021).

7. Empirical and Practical Insights

Analysis across studies reveals several recurrent properties:

• Self-allocation: Lower layers tend towards dense connectivity, while deeper layers self-organize to be sparser, validating the hypothesis that uniform group numbers are sub-optimal (Zhang et al., 2019).
• Sample-adaptivity: Masks and assignments adapt on a per-image or per-location basis, providing implicit form of conditional computation and pruning (Su et al., 2020, Li, 2023).
• Regularization effect: DGConv reduces test accuracy variance and can regularize CNNs, but excessive grouping or masking in narrow bottleneck layers may over-regularize, degrading accuracy (Yang et al., 2021, Li et al., 2023).
• Efficient inference: After hard assignment, inference time matches static group conv, supporting deployment on edge devices (Li et al., 2023).
• Generalization and fusion: DGConv generalizes standard, grouped, and depthwise conv via its mask or gate, and can be interpreted as differentiably performing network architecture search at the channel-group level (Zhang et al., 2019).

• (Su et al., 2020) Dynamic Group Convolution for Accelerating Convolutional Neural Networks
• (Zhang et al., 2019) Differentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks
• (Yang et al., 2021) Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data
• (Li, 2023) DGCNet: An Efficient 3D-Densenet based on Dynamic Group Convolution for Hyperspectral Remote Sensing Image Classification
• (Li et al., 2023) Spatial-Spectral Hyperspectral Classification based on Learnable 3D Group Convolution