FGFP: Fractional Gaussian Filter & Pruning

• FGFP is a neural network compression framework that unifies fractional Gaussian filter parameterization using few trainable parameters with adaptive unstructured pruning to reduce model size.
• It employs fractional calculus to analytically construct convolutional filter templates, replacing dense kernels in high-dimensional layers for efficient parameter reduction.
• Experimental benchmarks demonstrate that FGFP achieves 70–96% compression with under 2% accuracy drop, making it ideal for resource-constrained deployments.

Fractional Gaussian Filter and Pruning (FGFP) is a neural network compression framework that unifies fractional-order filter parameterization and adaptive unstructured pruning to enable aggressive reduction of model size with minimal predictive performance loss. FGFP minimizes the parameter count of convolutional filters by analytically constructing them as fractional Gaussian templates with limited, learnable parameters, and subsequently applies iterative weight pruning on the remaining dense components. This approach combines advances in fractional calculus-based filter design with automated sparsification pipelines, targeting resource-constrained deployments of deep learning models while retaining competitive accuracy on standard benchmarks (Tu et al., 30 Jul 2025).

1. Mathematical Foundations of Fractional Gaussian Filters

The Fractional Gaussian Filter (FGF) leverages results from fractional-order differential calculus, generalizing the classical Gaussian kernel through non-integer differentiation. Let $$G(x,y)=\exp\left(-\frac{(x-x_0)^2+(y-y_0)^2}{\sigma^2}\right)$$ be the standard 2D Gaussian. Application of fractional Gr\"unwald-Letnikov derivatives of orders $a$ and $b$ in $x$ and $y$ yields:

$$F_{fg}(x,y) = D^a_x D^b_y G(x,y) = [D^a_x G(x)] \cdot [D^b_y G(y)]$$

where $D^a_x$, $D^b_y$ are the left-sided Gr\"unwald-Letnikov derivatives:

$$D^\alpha_{G\!-\!L} f(x) = \lim_{h\to 0} \frac{1}{h^\alpha} \sum_{k=0}^{\lfloor (x-a)/h \rfloor} (-1)^k { \alpha \choose k } f(x-kh)$$

with $h=1$ and truncation giving the finite difference expansion:

$a$0

In convolutional neural networks, replacing learned $a$1 convolutional kernels with $a$2-type templates reduces each filter from $a$3 or $a$4 trainable weights to a low-dimensional parameterization:

• Fractional orders $a$5 and optionally $a$6 for the channel index (3D-FGF)
• Spatial offsets $a$7 and channel offset $a$8
• Gaussian width $a$9

This results in exactly 7 trainable parameters per kernel for the 3D-FGF form (Tu et al., 30 Jul 2025).

2. FGFP Architectural Integration

Within the FGFP pipeline, convolutional layers exhibiting a large number of channels (as determined by a threshold) are targeted for conversion:

• The standard dense kernel tensor $b$0 is replaced by its FGF template, parameterized by $b$1.
• Two main parameterization strategies are defined:
  • CA-FGF: Shares $b$2 across all channels, utilizing channel attention weights for additional flexibility; parameter count per kernel is $b$3.
  • 3D-FGF: Treats channel index as an additional spatial dimension, applying a fractional derivative of order $b$4; exactly seven scalars per filter.

After the FGF conversion, only the low-dimensional filter parameters are learned and updated in affected layers, with the rest of the network structure preserved (Tu et al., 30 Jul 2025).

3. Adaptive Unstructured Pruning Methodology

FGFP complements filter parameterization with Adaptive Unstructured Pruning (AUP) of the non-FGF (dense) layers. This process is iterative:

• At each round, compute the absolute magnitude of all weights in target layers.
• Prune the lowest $b$5 fraction of weights, $b$6 per round.
• Retrain (fine-tune) briefly to recover any lost accuracy, monitoring a validation threshold $b$7.
• If accuracy cannot be restored, revert the mask, reduce $b$8, and retry.

AUP is continued until a global sparsity or compression target is achieved. This iterative, accuracy-aware strategy ensures structural robustness and guards against catastrophic performance degradation (Tu et al., 30 Jul 2025).

4. FGFP Training, Optimization, and Workflow

The end-to-end FGFP procedure comprises:

1. Initializing from a standard pre-trained network.
2. Identifying and converting high-dimensional convolutional layers to FGFs (using CA-FGF or 3D-FGF).
3. Training the network with the new filter parameterization to minimize the cross-entropy loss; no explicit regularizers are required.
4. Freezing the FGF layers and beginning AUP on the remaining dense layers, repeating pruning and re-training rounds.
5. Once sparsity criteria are met, an optional joint fine-tuning phase can be applied at a reduced learning rate, but the primary pipeline generally avoids heavy post-processing and matches or closely approaches the original accuracy (Tu et al., 30 Jul 2025).

5. Empirical Performance and Benchmarks

Experimental results demonstrate that FGFP achieves high compression ratios with minimal impact on top-1 accuracy:

Dataset	Model	Compression Ratio	Acc. Drop (%)	Absolute Accuracy
CIFAR-10	ResNet-20	85.2%	1.52	89.82
CIFAR-10	WRN-28-10	96.8%	0.54	–
ImageNet2012	ResNet-50	69.1%	1.63	74.53

FGFP outperforms sparse pruning (SCOP, ARPruning) and low-rank decomposition approaches (ELRT, PSTRN) at equivalent or higher parameter reductions. As FGF-based convolutional layers are defined by analytical templates, both memory footprint and FLOPs decrease proportionally to parameter count, which translates into practical acceleration for edge deployment scenarios (Tu et al., 30 Jul 2025).

6. Comparative Methods: GDF-Based Structured Pruning

The Pruning Filter via Gaussian Distribution Feature (PFGDF) framework provides a structurally distinct but conceptually related approach (Xu et al., 2020). It models the empirical $b$9-norms of convolutional filters as Gaussian distributed, using this to define centrality and prune filters whose norms lie in the distribution tails. Key points:

• Pruning is performed by keeping filters whose norms satisfy $x$0, adaptively tuning $x$1 to avoid loss in validation accuracy.
• Unlike FGFP, which replaces filter weight tensors with closed-form analytic templates, PFGDF retains full convolutional kernels but removes those deemed statistically redundant.
• PFGDF achieves over 90% parameter reduction and nearly 80% inference speedup with negligible or even slightly improved test accuracy, e.g., VGG-16 on CIFAR-10: 90.81% parameter reduction, 70.27% FLOPs reduction, and 83.73% inference speedup, with accuracy increasing from 93.25% (baseline) to 93.48–93.63% post-pruning (Xu et al., 2020).

A plausible implication is that FGFP and PFGDF represent two ends of a spectrum: FGFP imposes strong analytic bias via parametric filter templates augmented by global unstructured pruning, while PFGDF executes structured, data-driven filter selection based on emergent properties of full networks.

7. Considerations, Limitations, and Future Directions

FGFP's effectiveness derives from the continuous flexibility of fractional-order filtering, which interpolates between Gaussian smoothing and higher-order edge detection; this renders the small parameter set expressive enough for diverse vision tasks. Orders $x$2 are constrained to $x$3 in practice, with settings near $x$4 emphasizing edge features.

Adaptive pruning granularity ($x$5–$x$6) facilitates stable convergence, though very aggressive per-round sparsification can degrade recoverability. While unstructured sparsity delivers high compression, it may present irregular memory footprints, suboptimal for all hardware. Extensions could include structured fractional filters or quantized FGF parameterizations for integer-only inference.

FGFP and related frameworks establish a high-water mark for analytic network compression, supporting reductions in model size of 70–96% with under 2% accuracy drop across a variety of architectures and datasets. This suggests strong applicability for deployment on edge devices and motivates further research into hardware-friendly structured fractional pruning and efficient inference strategies (Tu et al., 30 Jul 2025, Xu et al., 2020).