Deep Gabor Convolutional Networks

• Deep Gabor Convolutional Networks are deep learning models that embed fixed or learnable Gabor filters to capture orientation, frequency, and scale features.
• They employ various methods—including filter modulation, constrained convolution layers, and deformable sampling—to improve interpretability and energy efficiency.
• Empirical studies show these networks offer faster convergence, reduced parameter counts, and enhanced performance across image recognition, denoising, and biomedical imaging tasks.

Deep Gabor Convolutional Networks (GCNs) are deep convolutional neural architectures that introduce explicit Gabor structure into feature extraction, typically to encode orientation selectivity, frequency selectivity, scale sensitivity, and spatial locality. In the cited literature, this is realized in several distinct ways: by modulating learned convolution filters with fixed Gabor banks, by constraining one or more convolutional layers to parametric Gabor functions with learnable parameters, by mixing fixed and trainable Gabor kernels for energy-efficient training, by combining Gabor modulation with deformable convolution, by constraining all filters in an unrolled denoising network to be real Gabor functions or mixtures of Gabors, and by using Gabor kernels inside attention modules rather than as the primary convolution operator (Luan et al., 2017, Alekseev et al., 2019, Sarwar et al., 2017, Gong et al., 2020, Janjušević et al., 2022, Bao et al., 2022).

1. Conceptual scope and defining characteristics

The unifying premise is that many visual features learned by early and intermediate CNN layers are well approximated by Gabor-like receptive fields. Several works therefore replace unconstrained convolution kernels with analytically structured filters or with learned kernels modulated by Gabor masks. In the 2017 "Gabor Convolutional Networks" formulation, each learned convolution filter is turned into a Gabor Orientation Filter by element-wise multiplication with a predefined bank of oriented and scaled Gabor functions, yielding explicit orientation channels and layer-wise scale control (Luan et al., 2017). In GaborNet, the first convolutional layer is constrained to fit the Gabor function, and the parameters of the Gabor functions are learnable and updated by standard backpropagation techniques (Alekseev et al., 2019). In "Gabor is Enough," every analysis, synthesis, and final dictionary filter in an ISTA-derived denoising CNN is parameterized as a 2D real Gabor or a sum of Gabors (Janjušević et al., 2022).

This family is not limited to a single insertion point. Some models modify only the first layer, some replace higher-level layers, some operate across all layers, and some compute Gabor-derived attention rather than replacing the backbone convolutions. Rega-Net is explicit on this distinction: it uses Gabor filters inside an attention mechanism rather than as the primary convolution operator (Bao et al., 2022). Deformable Gabor Feature Networks use traditional convolutions in the first two blocks and replace higher-level convolutional layers with Deformable Gabor Convolution, where semantics are more discriminative and spatial variations are stronger (Gong et al., 2020).

A recurring motivation is improved inductive bias under limited supervision, geometric variation, or constrained compute. The 2017 energy-efficiency study replaces certain CNN kernels with fixed Gabor filters to eliminate gradient computation and weight updates for those kernels, reducing training complexity, energy, time, and storage in exchange for tolerable accuracy degradation (Sarwar et al., 2017). Biomedical imaging work emphasizes that many lesions vary greatly in size, shape, and viewing configuration, and uses Gabor priors to improve representativeness, robustness, and interpretability even when only weak labels are available (Gong et al., 2020).

2. Mathematical foundations

Most cited formulations use a real-valued 2D Gabor filter. A canonical form appearing repeatedly is

$$g(x,y)=\exp\!\left(-\frac{x'^2+\gamma^2 y'^2}{2\sigma^2}\right)\cos(2\pi f x' + \phi),$$

with rotated coordinates

$$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$

Here, $f$ is the central frequency, $\theta$ the orientation, $\phi$ the phase offset, $\sigma$ the Gaussian envelope scale, and $\gamma$ the aspect ratio of the Gaussian envelope (Luan et al., 2017, Sarwar et al., 2017). GaborNet uses the same real-valued structure but makes the standard six-parameter family learnable per filter, with amplitude $A$, angular frequency $\omega$, orientation $\theta$, phase $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$0, Gaussian scale $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$1, and aspect ratio $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$2 (Alekseev et al., 2019). GDLNet instead parameterizes a single filter as

$$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$3

and allows each learned filter to be a sum of $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$4 such Gabors (Janjušević et al., 2022).

The 2017 modulation-based GCN introduces Gabor structure through element-wise filter modulation rather than direct parameterization. If $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$5 denotes a learned base filter and $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$6 a fixed Gabor bank over orientation $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$7 and scale $$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$8, the Gabor Orientation Filter is

$$x' = x\cos\theta + y\sin\theta,\qquad y' = -x\sin\theta + y\cos\theta.$$9

The oriented response for the $f$0-th filter is then computed over the input feature channels, and orientation is preserved as an explicit channel dimension through the network (Luan et al., 2017). Backpropagation updates only the stored base filter:

$f$1

This separates fixed steerable structure from learned weights (Luan et al., 2017).

Deformable Gabor Convolution combines two operators. The deformable branch applies learned offsets $f$2 to the regular sampling grid $f$3, while the adaptive Gabor branch modulates a shared bank of Gabor filters $f$4 by learned scale masks $f$5. The resulting modulated filters are

$f$6

and the forward propagation first produces deformable features $f$7 and then orientation-channel outputs $f$8 aggregated over mask scales (Gong et al., 2020). This construction preserves explicit orientation channels while adding DCN-style spatial adaptability.

In GDLNet, Gabor structure is embedded in an unrolled sparse-coding network. For $f$9 ISTA-like layers,

$\theta$0

and the output is $\theta$1. All filters in $\theta$2, $\theta$3, and $\theta$4 are built from learnable 2D real Gabor functions or mixtures of Gabors, and denoising performance depends materially on untied learned scale parameters between layers (Janjušević et al., 2022).

3. Architectural realizations

The earliest deep Gabor CNN formulations are modulation-based. "Gabor Convolutional Networks" replaces the basic convolutional operator with a Gabor-modulated operator that can be inserted into CNNs and ResNets by replacing spatial convolutions with GCConv. The Gabor bank is fixed, the learned filters remain unconstrained except for modulation, and orientation is handled as an explicit channel dimension. Scale is introduced across layers by changing the scale index $\theta$5 with depth (Luan et al., 2017). This design is compatible with standard deep learning architectures and is used on MNIST, MNIST-rot, SVHN, CIFAR-10/100, ImageNet-100, ImageNet-2012, and Food-101 (Luan et al., 2017).

A different line constrains the convolution kernels themselves to the Gabor family. GaborNet constrains the first convolutional layer to be a bank of real-valued 2D Gabor filters with learnable parameters, while all subsequent layers remain standard convolutional, pooling, activation, and fully connected layers (Alekseev et al., 2019). This yields a sharp reduction in first-layer parameter count: on Dogs vs Cats, the GaborLayer has 32 filters of size $\theta$6, stride $\theta$7, and 192 trainable parameters in the first layer, compared with approximately 800 for the baseline $\theta$8 grayscale convolution; on AffectNet, the GaborLayer has 96 filters of size $\theta$9, stride $\phi$0, and 576 first-layer parameters, compared with 11,616 for the baseline $\phi$1 grayscale convolution (Alekseev et al., 2019).

A third realization uses fixed or partially trained Gabor kernels as an efficiency device. In LeNet-style networks for MNIST, FaceDet, and TiCH, the first convolutional layer is replaced entirely by fixed $\phi$2 Gabor filters with equally spaced orientations, while the second convolutional layer can be fully fixed or split into a half fixed / half trainable balanced configuration (Sarwar et al., 2017). On CIFAR-10 with Network in Network, 128 of 192 kernels in each of the three convolutional layers are replaced by fixed Gabor filters, and later blocks can instead be partially trained for 20% or 30% of total cycles and then frozen (Sarwar et al., 2017).

Biomedical imaging work extends the idea beyond rigid kernel placement. DGConv fuses deformable convolution, which makes the sampling locations learnable via offsets, with adaptive Gabor convolution that modulates a bank of shared Gabor filters by learned scale masks. The features are then consumed by a Deformable Gabor Feature Network using multi-instance learning or multi-instance multi-label learning, depending on the dataset (Gong et al., 2020). The architecture keeps traditional convolutions in the first two blocks and uses DGConv at higher layers.

Denoising work pushes Gabor parameterization further. GDLNet constrains all filters in an unrolled sparse-coding CNN to be learnable 2D real Gabor functions or their mixtures, uses no hidden feature-domain $\phi$3 or $\phi$4 convolutions, and performs denoising through analysis/synthesis filterbanks and element-wise soft-thresholding (Janjušević et al., 2022). By contrast, Rega-Net leaves the backbone convolutional layers intact and introduces Retina Gabor attention blocks that build a $\phi$5 Gabor kernel bank with learnable parameters, apply a retina-like binary mask, and use the resulting response to produce an attention map that modulates the backbone features (Bao et al., 2022).

4. Training regimes, optimization strategies, and efficiency trade-offs

Training protocols vary strongly with the formulation. The 2017 modulation-based GCN trains with cross-entropy loss; on MNIST it uses Adadelta, batch size 128, initial learning rate $\phi$6, weight decay $\phi$7, and halves the learning rate every 25 epochs, while ImageNet-100 is trained for 120 epochs with initial learning rate $\phi$8 and decays by $\phi$9 every 30 epochs (Luan et al., 2017). GaborNet uses Adam with learning rate $\sigma$0, $\sigma$1, batch size 64, 100 epochs, ReLU, max-pooling $\sigma$2, and dropout for Dogs vs Cats, and follows AlexNet-style training on ImageNet with random crop, horizontal flip, and learning-rate drops at epochs 30 and 50 (Alekseev et al., 2019).

The efficiency-oriented fixed-kernel line removes gradients and updates for designated Gabor kernels during the fixed phase. For a fixed Gabor kernel $\sigma$3, $\sigma$4; for a partially trained Gabor kernel, updates are applied for $\sigma$5 and then stopped for $\sigma$6, with $\sigma$7 in the reported experiments (Sarwar et al., 2017). The paper’s MAC-centric energy model explicitly separates forward, backpropagation, and update costs and estimates energy using RTL MAC units mapped to IBM 45 nm technology at 1 GHz using Synopsys Design Compiler (Sarwar et al., 2017).

In DGFN, optimization is application-specific. INbreast uses 5-fold cross-validation, Otsu background removal, resize to $\sigma$8, random horizontal flips, rotations up to $\sigma$9, 10% shifts, and random $\gamma$0 zeroed boxes, with Adam, initial learning rate 0.0001 for both $\gamma$1 and $\gamma$2, weight decay 0.00005, 10% learning-rate decay every 100 epochs, total 1000 epochs, and typical choice $\gamma$3 orientations and $\gamma$4 masks (Gong et al., 2020). ChestX-ray14 uses DenseNet-121, resize $\gamma$5 to $\gamma$6, ImageNet normalization, random rotations and horizontal flips, SGD with momentum 0.9, batch size 16, initial learning rate 0.001, and learning-rate decay by $\gamma$7 when validation loss stagnates (Gong et al., 2020).

GDLNet trains on BSD432 with additive white Gaussian noise and mean-squared error loss. The thresholds are learned per subband and can be adapted to the noise level via

$\gamma$8

which the paper identifies as the primary driver of noise-level generalization under train–inference mismatch (Janjušević et al., 2022). Rega-Net uses PyTorch 1.8.0 and Python 3.8 on 4× RTX 3080 Ti GPUs; on ImageNet-1K it trains with SGD, learning rate 0.01, momentum 0.9, weight decay $\gamma$9, a step-down schedule by $A$0 every 30 epochs, total 100 epochs, and batch size 16 per GPU; on COCO 2017 it uses AdamW, learning rate $A$1, momentum 0.9, weight decay $A$2, MultiStepLR, batch size 2 per GPU, and 12 epochs (Bao et al., 2022).

The efficiency implications depend on how Gabor structure is introduced. The fixed-kernel study reports 1.31–1.53x improvement in training energy, up to 1.4x improvement in training time, up to 2.23x in storage requirements, and up to 2.2x in memory access energy, with accuracy degradation within 0–3% of the baseline on simpler tasks and mitigated by partial training on CIFAR-10 (Sarwar et al., 2017). GDLNet reports near state-of-the-art denoising with 2–8× fewer parameters than popular fully convolutional baselines and no inference overhead if realized filters are precomputed and stored (Janjušević et al., 2022). Rega-Net adds modest compute overhead: ResNet-50 grows from 25.557M parameters and 4.122 GFLOPs to 29.325M and 4.230 GFLOPs, and ResNet-101 grows from 44.549M and 7.849 GFLOPs to 50.661M and 7.925 GFLOPs (Bao et al., 2022).

5. Empirical performance across tasks

On rotation- and scale-sensitive recognition tasks, the modulation-based GCN literature reports strong improvements with reduced parameter counts. On MNIST-rot, a baseline CNN reports 2.82% error, while GCN4 variants report 1.28–1.45% with $A$3 kernels, 1.10–1.26% with $A$4 kernels, and 1.20–1.33% with $A$5 kernels; on standard MNIST, the same family reports 0.56–0.63%, 0.48–0.49%, and 0.42–0.46%, respectively (Luan et al., 2017). On SVHN, GCN4-40 reaches 96.9% accuracy with 2.2M parameters, compared with 95.8% for ResNet-110 at 1.7M and 95.88% for ResNet-172 at 2.7M (Luan et al., 2017). On CIFAR-10/100, GCN3-28 reports 3.88% / 20.13% versus 4.00% / 19.25% for WRN-28, with 17.6M versus 36.5M parameters (Luan et al., 2017). On ImageNet-100, GCN-34 reports Top-1 error 11.46% and Top-5 error 3.04% versus 11.94% and 3.16% for ResNet-101, and on full ImageNet the paper reports 73.2% for GCN-34 versus 71.6% for ResNet-34 (Luan et al., 2017). On Food-101, the reported error is 14.2%, compared with 17.9% for ResNet-50, 16.8% for compact bilinear pooling, and 14.5% for kernel pooling (Luan et al., 2017).

GaborNet emphasizes convergence and first-layer compression. On Dogs vs Cats, GCNN achieves up to 6% higher accuracy than the baseline by the end, with representative test accuracy near convergence of approximately 0.79 versus 0.73 (Alekseev et al., 2019). On AffectNet, GCNN shows better accuracy in earlier epochs, with approximately 3% average advantage, while in later epochs the average difference drops to approximately 1% (Alekseev et al., 2019). On ImageNet with AlexNet, Gabor AlexNet enjoys up to approximately 2% advantages in top-1 and top-5 accuracy from epochs 10–30, after which the baseline catches up and both models achieve nearly identical final scores (Alekseev et al., 2019).

The fixed-kernel efficiency line reports detailed trade-offs. On MNIST, fixing only the first convolutional layer yields 20.70% energy savings, 9.47% training-time reduction, and 0.62% accuracy loss. Fixing both convolutional layers yields 48.28% energy savings, 53% training-time reduction, 42.48% storage savings, and 5.85% accuracy loss. The half fixed / half trainable balanced design yields 34.49% energy savings, 22.30% training-time reduction, 23.15% storage savings, and 1.14% accuracy loss (Sarwar et al., 2017). On CIFAR-10 with Network in Network, using 128/192 fixed Gabor kernels per convolutional layer yields 29.1% computational energy savings, 55.2% storage savings, 54.97% memory access energy savings, and 3.34% accuracy loss; partial training in the second and third blocks reduces the accuracy loss to 2.41% or 1.95%, depending on the schedule (Sarwar et al., 2017).

Biomedical imaging results are reported in AUC and accuracy. On INbreast-Deform, DGFN reaches 81.29% AUC with approximately 0.98M parameters for kernel stages 8–16–32–64, and 83.30% AUC with 3.40M parameters for 16–32–64–128, exceeding comparable CNN, GCN, and DCN baselines (Gong et al., 2020). On the original INbreast dataset, Pretrained ResNet-18 + DGFN + MIL reaches 88.05% AUC and 93.18% accuracy; Pretrained AlexNet + DGFN + MIL reaches 87.22% AUC and 91.34% accuracy (Gong et al., 2020). On ChestX-ray14, off-the-shelf DGFN averages 78.39% AUC and exceeds prior off-the-shelf methods on 11/14 pathologies, while fine-tuned DGFN achieves 85.01% average AUC, outperforming CheXNet at 84.17%, Wang et al. at 73.8%, and Baltruschat et al. at 82.0% (Gong et al., 2020).

Denoising results show that strict Gabor parameterization can remain competitive with standard fully convolutional networks. On BSD68 at noise levels $A$6, GDLNet-S (MoG 3, 188k) reports 31.68 / 29.22 / 26.30 dB, compared with CDLNet-S at 31.74 / 29.26 / 26.35 dB, FFDNet at 31.63 / 29.19 / 26.29 dB, and DnCNN at 31.72 / 29.22 / 26.23 dB (Janjušević et al., 2022). On Kodak grayscale with GDLNet-S MoG 1, increasing the filter size from $A$7 to $A$8 improves PSNR from 32.45 / 30.06 / 27.21 to 32.57 / 30.21 / 27.40 (Janjušević et al., 2022). The paper further reports that GDLNet with adaptive thresholds retains near-perfect generalization across $A$9 when trained on a noise range and nearly matches single-noise-level models trained at each $\omega$0 (Janjušević et al., 2022).

Rega-Net demonstrates that Gabor structure can also improve attention modules. On ImageNet-1K, Rega-Net with ResNet-50 reports Top-1 78.852% and Top-5 94.120% versus 76.384% and 92.908% for the baseline, while Rega-Net with ResNet-101 reports Top-1 79.963% and Top-5 95.552% versus 78.200% and 93.906% (Bao et al., 2022). On COCO 2017 object detection, FCOS with ResNet-50 improves from 34.6 to 37.8 mAP, Faster R-CNN with ResNet-50 improves from 36.4 to 39.9 mAP, YOLOv4 improves from 41.2 to 43.1 mAP, and RetinaNet with ResNet-50 improves from 35.6 to 38.6 mAP (Bao et al., 2022).

6. Interpretability, limitations, and terminological ambiguities

Interpretability is one of the most persistent themes in this literature. The explicit orientation-channel structure in modulation-based GCNs exposes oriented responses directly (Luan et al., 2017). GaborNet ties first-layer weights to a small set of physically meaningful parameters rather than unconstrained pixels (Alekseev et al., 2019). DGConv supports patch-level malignant probability maps and orientation-selective responses in biomedical imaging (Gong et al., 2020). GDLNet is explicit that its success suggests that representations used by low-level image processing CNNs can be as simple and interpretable as Gabor filterbanks (Janjušević et al., 2022). Rega-Net treats Gabor selectivity as a mechanism for biologically inspired, foveated attention rather than a replacement for the backbone (Bao et al., 2022).

Several misconceptions are corrected by the literature itself. Deep Gabor CNNs are not a single architecture, and they do not all learn Gabor parameters. In the modulation-based GCN, the Gabor bank is fixed and only the base filters are learned (Luan et al., 2017). In GaborNet, the first-layer Gabor parameters are learned end-to-end (Alekseev et al., 2019). In the energy-efficient study, many Gabor kernels are fixed throughout training or trained only for an initial fraction of cycles (Sarwar et al., 2017). In GDLNet, all filters are parameterized as learnable real Gabor functions or sums of Gabor functions (Janjušević et al., 2022). In Rega-Net, Gabor kernels appear in an auxiliary attention branch rather than in the main convolutional pipeline (Bao et al., 2022).

The limitations are equally model-specific. Fixed Gabor banks may restrict adaptiveness; this is stated directly in DGConv, which notes that $\omega$1 is shared and not updated by backprop and suggests that learning these parameters or adopting complex-valued Gabor filters could further improve adaptiveness (Gong et al., 2020). GaborNet reports that gains are strongest when datasets contain “Gabor-like” features and that ImageNet final metrics become nearly identical to the baseline despite earlier convergence advantages (Alekseev et al., 2019). GDLNet notes that CDLNet-S still edges it slightly at some noise levels, and that color images, non-AWGN noise, and complex-valued Gabor filters are not explored (Janjušević et al., 2022). Rega-Net notes that speed and computational complexity still need further optimization and that mask design may affect performance (Bao et al., 2022).

A final ambiguity is terminological. In computer vision, “GCN” in the cited papers often means “Gabor Convolutional Network” or closely related Gabor-based CNNs. In a separate literature, however, “GCN” denotes Graph Convolutional Network. "DeepGCNs: Making GCNs Go as Deep as CNNs" studies residual and dense connections plus dilated graph convolution on non-Euclidean data (Li et al., 2019). DRGCN introduces Dynamic evolving initial Residual for deep graph convolutional networks to address over-smoothing (Zhang et al., 2023). "Deeper Insights into Deep Graph Convolutional Networks: Stability and Generalization" analyzes uniform algorithmic stability and generalization bounds governed by graph filter spectral radius and depth (Yang et al., 2024). These graph-learning papers are not part of the deep Gabor CNN lineage, even though they share the acronym “GCN.”

Taken together, the literature shows that deep Gabor convolutional networks are best understood as a structured-design family for deep vision models. The shared principle is the insertion of explicit Gabor inductive bias into convolution, dictionary learning, deformable sampling, or attention. A plausible implication is that the main design question is not whether to use Gabor structure at all, but where to place it: as a fixed modulation prior, as a learnable parametric kernel family, as a compression device, as a deformable feature generator, as an interpretable synthesis dictionary, or as a receptive-field-expanding attention operator.