Prior-Guided Convolution (PGConv) Overview

• PGConv is an innovative convolutional architecture that integrates structural and statistical priors to enhance image feature extraction and compression.
• It decomposes standard convolutions into specialized branches, including AConvs for axis-aligned structure and DConvs for high-frequency detail extraction.
• The method employs re-parameterization to merge multiple branches into a single convolution at inference, maintaining efficiency while reducing biases.

Prior-Guided Convolution (PGConv) is an architectural paradigm for convolutional layers that explicitly incorporates structural and statistical priors about spatial data—particularly images—within the convolutional framework. Motivated by the limitations of conventional convolutions in capturing both local dependencies (such as axis-aligned correlations) and high-frequency content essential for compression and perception, PGConv augments the representational capacity of convolutional networks by decomposing the standard operator into specialized parallel branches. Two prominent instantiations are found in the discrete (learned image compression) setting (Chen et al., 30 Nov 2025) and in the probabilistic numeric (Gaussian process-based) convolutional neural network context (Finzi et al., 2020). These formulations address systematic biases in conventional convolutions and enable more robust feature extraction in settings with regular or irregular domain sampling.

1. Motivation and Rationale

In image analysis and compression, conventional 3×3 convolutions are limited by their spatial homogeneity: all positions within the kernel window are treated identically. This homogeneity induces two main blind spots (Chen et al., 30 Nov 2025):

• Skeleton bias: The four axial neighbors (“skeleton” elements) in a 3×3 patch are under-emphasized, whereas corners are over-weighted, often resulting in blurred thin structures and edge artifacts.
• Low-frequency bias: Standard convolutions preferentially encode smooth, low-frequency regions, while attenuating high-frequency, edge, and texture information that are essential for accurate reconstruction after quantization.

To overcome these limitations, PGConv injects explicit structure by composing the convolutional operator from branches dedicated to skeleton enhancement and high-frequency extraction, resulting in improved feature fidelity, especially for learned image compression and tasks involving spatially irregular samples (Chen et al., 30 Nov 2025, Finzi et al., 2020).

2. Operator Structure and Parametrization

The discrete PGConv operator comprises eight parallel convolutional branches, each with learnable weights. These are grouped as follows (Chen et al., 30 Nov 2025):

Branch Type	Kernel/Filter Shape	Functional Role
Convs	3×3, 1×1	Standard convolution (baseline)
AConvs	3×1, 1×3	Skeleton (axis-aligned) enhancement
DConvs	CDC, ADC, HDC, VDC	High-frequency/edge extraction

• Asymmetric Convolutions (AConvs): Emphasize axis-aligned pixel relationships. For vertical relationships $w^{(3\times1)} = [a_1;a_2;a_3]$, and for horizontal $w^{(1\times3)} = [b_1, b_2, b_3]$, the AConvs separately learn to amplify local structure along cardinal axes, preserving lines and T-junctions.
• Difference Convolutions (DConvs): Employ finite-difference filters—central-difference (CDC), angular-difference (ADC), horizontal-difference (HDC), and vertical-difference (VDC)—to explicitly extract multi-directional edge and texture responses.

The combined output is the sum over the outputs of all branches:

$$F^{\mathrm{out}} = \sum_{i=1}^8 F^{\mathrm{in}} * w_i$$

where $*$ denotes convolution and $w_i$ represents the learnable weights for each kernel.

3. Re-parameterization and Computational Efficiency

A critical aspect of PGConv is the use of re-parameterization to fuse the eight branches into a single, equivalent 3×3 convolution at inference, eliminating any runtime penalty compared to standard convolutions (Chen et al., 30 Nov 2025):

$$F^{\mathrm{out}} = F^{\mathrm{in}} * w_{\mathrm{mrg}}, \quad w_{\mathrm{mrg}}(p, q) = \sum_{i} w_{i}(p, q)$$

where $(p, q)\in \{-1, 0, 1\}^2$.

During training, each branch is parameterized and updated independently through backpropagation, permitting specialization to its associated prior. At inference, the weights are summed into $w_{\mathrm{mrg}}$, yielding both the computational cost and memory footprint of a vanilla convolution. The parameter and FLOP cost during training is approximately 5.1× that of a basic convolution, but zero additional cost at runtime.

4. Integration into Architectures and Empirical Impact

Within learned image compression pipelines, PGConv is integrated following strided (down-sample) or sub-pixel (up-sample) convolutions, forming blocks such as Down-sample Residual Block (DRB↓2)—composed of a 3×3 stride-2 convolution followed by PGConv—and Up-sample Residual Block (URB↑2)—composed of a sub-pixel convolution plus PixelShuffle and PGConv (Chen et al., 30 Nov 2025). PGConv acts as a local feature extractor that prepares high-frequency-rich and structurally robust representations for downstream modules, notably multi-scale gated transformers which capture long-range dependencies.

Ablation experiments using the Kodak dataset indicate:

• Addition of DConvs to vanilla convolutions reduces BD-rate by 1.73%.
• Further addition of AConvs yields a cumulative rate reduction of −3.39%.
• Incorporation into a joint network with multi-scale gated transformers achieves an −11.64% BD-rate reduction over the convolution-only backbone, underscoring the role of enhanced local priors in amplifying global attention modules.

5. Probabilistic Numeric Prior-Guided Convolution

An alternate instantiation of PGConv arises in probabilistic numeric convolutional neural networks, where feature maps are modeled as vector-valued Gaussian processes with RBF kernel priors (Finzi et al., 2020). Here, the convolutional operator is formulated as the evolution of linear partial differential equations (PDEs) on the GP:

$A = \sum_{k=1}^{K} W_k e^{D_k}$

with $D_k = B_k^T\nabla + \frac{1}{2}\nabla^T E_k \nabla$ dictating spatial drift and diffusion. The action of the layer is analytically closed due to Gaussians’ convolution properties, propagating both mean and covariance. Nonlinearities (e.g., ReLU) are incorporated by analytically tracking the first and second moments post-activation. After each nonlinear block, the mean and diagonal variances are projected back to an RBF GP to maintain analytic tractability.

This formulation is able to coherently represent epistemic uncertainty, handle irregular and missing data, and provide steerable group-equivariant convolutions. Empirical evaluation on datasets such as SuperPixel-MNIST and PhysioNet2012 demonstrates improved error rates and enhanced robustness to out-of-distribution sampling densities. The computational complexity is dominated by the GP posterior step ($O(N^3)$ per layer per channel), with forward passes remaining tractable for moderate $N$.

6. Limitations and Open Challenges

Identified limitations include:

• In deep probabilistic numeric networks, uncertainty attenuation is observed due to independent RBF GP resets, leading to potential underestimation of epistemic uncertainty (Finzi et al., 2020).
• Scaling to large sample sizes is challenging due to the cubic GP posterior cost, motivating the investigation into inducing-point GPs and structured kernel approximations.
• In the discrete setting, while training cost increases by >5×, inference cost remains identical to standard convolutions (Chen et al., 30 Nov 2025).
• The methodology depends crucially on the appropriate choice of kernel (GP setting) or predesigned filter shapes (discrete setting), and adaptation to task/domain-specific structure is an area of ongoing investigation.

7. Extensions and Relation to Broader Context

PGConv demonstrates that explicitly encoding local statistical or structural priors into the parameterization of convolutional layers can systematically reduce intrinsic biases and achieve domain-specific objectives (e.g., high-frequency preservation, edge retention) without computational trade-off at inference (Chen et al., 30 Nov 2025, Finzi et al., 2020). In the probabilistic numeric setting, the approach generalizes naturally to arbitrary domains—such as graphs or manifolds—by changing the prior kernel and PDE generator, connecting PGConv to the family of gauge-equivariant and steerable CNN architectures. A plausible implication is that prior-guided constructions could underlie future advances in robust and uncertainty-aware learning for domains with irregular or sparse sampling.