Learnable Shifting Filters

• Learnable shifting filters are adaptive linear operators that adjust spatial supports and coefficients through end-to-end or meta-optimization.
• They reduce complex convolutional filter spaces into a compact universal kernel set, retaining over 99% ImageNet accuracy with fewer parameters.
• This approach unifies CNN and GNN methodologies, enhancing model interpretability, transferability, and efficiency across image and graph domains.

Learnable shifting filters are parameterized classes of linear operators in neural architectures whose spatial supports and coefficients are not fixed a priori, but are adapted either directly through the primary loss function (as in end-to-end training) or through an explicit meta-optimization, often taking the form of affine, shifted, or metric-based modifications of a small universal set of core kernels. This class unifies advances in both graph neural networks, where metric filters are learned via data-dependent diffusion distances, and depthwise separable convolutional networks, where thousands of spatial filters are reduced to parameterized shifts of a handful of universal templates. Learnable shifting filters thus enable highly adaptive receptive fields and efficient parameter sharing, and facilitate greater interpretability and transferability across domains.

1. Formal Definitions and Canonical Instances

In DS-CNNs, learnable shifting filters take the form:

$$f_i(x, y) \approx U_{k(i)}(x - \Delta x_i, y - \Delta y_i) + b_i,$$

where $f_i$ is a learned $7 \times 7$ spatial filter, $U_{k(i)}$ is one of eight master key filters, $(\Delta x_i, \Delta y_i)$ are learnable (possibly non-integer) spatial shifts, and $b_i$ is a per-filter bias term. The master key filters $U_j$ (for $j = 1,\dots,8$) are closed-form spatial primitives: four central-difference operators (horizontal, vertical, diagonal, anti-diagonal), two first derivatives of Gaussians (in $x$ and $y$), one Difference-of-Gaussians (DoG), and one isotropic Gaussian (Babaiee et al., 15 Sep 2025):

Kernel Type	Mathematical Formulation	Representative $f_i$0
Central Difference	$f_i$1 etc.	$f_i$2 to $f_i$3
Gaussian Derivative	$f_i$4	$f_i$5, $f_i$6
DoG	$f_i$7	$f_i$8
Gaussian	$f_i$9	$7 \times 7$0

In graph neural networks, the analogous objects are the structural “jumps” $7 \times 7$1 constructed from learned diffusion distances $7 \times 7$2 and adaptive supports $7 \times 7$3 (Begga et al., 2023):

$7 \times 7$4

with $7 \times 7$5 defined by sorted diffusion distances and data-driven radii $7 \times 7$6.

2. Principle of Shift-Based Parameterization

Learnable shifting filters exploit the observation that the trained spatial filters in certain architectures—especially those with extreme parameter sharing, such as depthwise separable CNNs—can be effectively described as affine (linear plus bias) transforms and spatially shifted versions of a small universal kernel bank. This is formalized in (Babaiee et al., 15 Sep 2025) as the “master key filters hypothesis,” and operationalized through an unsupervised codebook discovery algorithm. For any filter $7 \times 7$7, its approximation requires only:

• An index $7 \times 7$8 into the universal filter bank;
• Real-valued shifts $7 \times 7$9;
• An additive bias $U_{k(i)}$0;
• (Optionally) a global scale $U_{k(i)}$1 (absorbed by pointwise weights in practice).

This structure enables the drastic reduction of learned spatial filters without sacrificing accuracy—DS-CNNs with only eight frozen spatial kernels plus learnable shifts and biases retain $U_{k(i)}$2 of ImageNet accuracy compared to fully trainable models (Babaiee et al., 15 Sep 2025).

In the context of graphs, shifting occurs over the metric structure implicitly learned by the data-driven diffusion embedding $U_{k(i)}$3, mapping nodes to $U_{k(i)}$4 and defining filter supports and weights via distances in the learned space (Begga et al., 2023).

3. Learning Methodologies

In DS-CNNs

The construction and validation of learnable shifting filters in DS-CNNs employs a three-stage process (Babaiee et al., 15 Sep 2025):

1. Filter Matrix Collection and Normalization: Flatten trained $U_{k(i)}$5 depthwise kernels, zero-center, and normalize.
2. Codebook Learning: Train a 1D autoencoder to represent all filters in a compressed code; decode a large set of candidate filters.
3. Linear Approximation and Backward Elimination: For each filter, solve for optimal scaling and bias to fit each candidate, assigning the best match. Iteratively prune the candidate set while tracking validation accuracy, revealing an “elbow” at eight essential kernels.

In Diffusion-Jump GNNs

The learnable metric filter bank is established via the following pipeline (Begga et al., 2023):

1. Fiedler Environment Learning: Learn $U_{k(i)}$6, satisfying $U_{k(i)}$7, by minimizing the trace-ratio Dirichlet loss $U_{k(i)}$8.
2. Diffusion Distance Computation: Build $U_{k(i)}$9.
3. Jump Definition: For each scale $(\Delta x_i, \Delta y_i)$0, compute support sets $(\Delta x_i, \Delta y_i)$1 via sorted diffusion distances and projection matrices $(\Delta x_i, \Delta y_i)$2 via differentiable top-$(\Delta x_i, \Delta y_i)$3 selection.
4. Structural Filter Application: Form $(\Delta x_i, \Delta y_i)$4, apply to features, and aggregate via nonlinear transformations.
5. End-to-End Joint Optimization: Backpropagate joint Dirichlet and classification loss through all parameters, so both metric structure $(\Delta x_i, \Delta y_i)$5 and the supports/weights of each $(\Delta x_i, \Delta y_i)$6 are updated.

4. Empirical Results and Applications

Empirical evidence demonstrates the sufficiency of a small, universal set of master key filters augmented with shifts and biases for high-accuracy image classification in DS-CNNs (Babaiee et al., 15 Sep 2025). On ImageNet, models trained with only eight frozen universal kernels equal or surpass the accuracy of fully trainable architectures with thousands of filter parameters. These filters generalize across architecture variants (ConvNeXtv2, HorNet) and transfer without modification to diverse datasets, outperforming conventional fine-tuned transfers in low-data regimes.

In graph domains, learnable metric filters in Diffusion-Jump GNNs are capable of dynamically discovering long-range structural dependencies, ideal for the heterophilic regime where classical hop-based aggregation fails. The architecture, via end-to-end optimization, reshapes both the learned metric and the filtering “jumps” to restore homophily in latent space, improving classification quality in graphs where class boundaries cut across local structures (Begga et al., 2023).

5. Interpretability, Model Compression, and Theoretical Implications

The universal eight master key filters in DS-CNNs (central differences, DoGs, and Gaussian derivatives) have strong correspondence to classical image processing operators and to receptive field structures measured in biological vision systems. This suggests that even unconstrained modern architectures naturally converge to the same efficient basis as found in scale-space theory and biological evolution.

Learnable shifting filters support large-scale parameter compression and architectural interpretability: thousands of convolutional kernels can be replaced with eight universal primitives plus spatial shifts, substantially reducing the model's parameter count and offering clearer semantic grounding for each learned operation. Similar parameter reduction and transparency can be achieved in graph settings, as each “jump” filter has explicitly defined supports and metric-induced weights.

A plausible implication is that enforcing a shift-based parameterization or pre-baking master key filters into architecture design can yield highly efficient, robust, and interpretable models without performance degradation—facilitating deployment in resource-constrained or explainable machine learning settings.

6. Connections to Transfer Learning and Biological Vision

The eight universal spatial operators identified via the learnable shifting filter approach are directly transferable across architectures and visual recognition tasks, supporting strong out-of-distribution generalization and transfer learning. The close match between DS-CNN-learned master key filters and known retinal/V1 receptive fields (Hubel & Wiesel 1968; Young 1987; Jones & Palmer 1987) underlines a convergence between artificial and biological visual computation principles (Babaiee et al., 15 Sep 2025).

In graphs, the learnable metric filters enable semi-supervised models to reconstruct piecewise smooth class manifolds in latent space even under severe label–structure misalignment, reflecting the power of end-to-end adaptive metric learning (Begga et al., 2023). This highlights the unifying principle: constraining filter learning to shifted, parameterized transformations of a universal set not only streamlines model design but also ties deep learning systems more tightly to efficient, interpretable, and biologically plausible signal processing foundations.