Convolution Arithmetic in Deep Learning

• Convolution arithmetic for deep learning is the mathematical framework that defines spatial and channel-wise computations in CNNs, determining output dimensions and computational complexity.
• Fast convolution methods like Winograd and FFT reduce multiplications and enhance throughput, balancing efficiency with precision in various filter sizes.
• Recent advancements in integer-based and RNS-enabled Winograd algorithms optimize hardware performance while mitigating numerical errors, supporting robust quantized CNN inference.

Convolution arithmetic for deep learning refers to the mathematical framework underpinning the spatial and channel-wise computations in convolutional neural networks (CNNs). It encompasses standard direct convolution, pooling, and transposed-convolution layers, as well as advanced fast convolution algorithms such as Winograd and FFT-based methods. The arithmetic governs output dimensions, computational complexity, precision analysis, and optimized hardware execution. Recent advances, including efficient Winograd variants, integer-arithmetic approaches, and residue number system (RNS) Winograd, have significantly impacted inference throughput and quantized CNN deployment, while detailed error analyses guide practitioners in navigating accuracy-versus-speed trade-offs.

1. Convolutional Layer Arithmetic and Output Shape

Two-dimensional convolution, pooling, and transposed-convolution arithmetic are parameterized by input tensor shape $(H_{in}, W_{in}, D_{in})$, kernel shape $(K_h, K_w)$, stride $(S_h, S_w)$, and padding $(P_h, P_w)$. Output height and width are given by: $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$

$$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$

These formulas ensure full kernel coverage and are adopted by major frameworks. Pooling uses the same sliding window logic, typically omitting padding. Transposed convolution reverses the spatial reduction effect, computed as: $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$

$W_{out} = S_w(W_{in} - 1) + K_w - 2P_w$

for the corresponding height and width axes. These arithmetic relations generalize immediately to non-square kernels, non-unit strides, and multiple channels (Dumoulin et al., 2016).

2. Fast Algorithms: Winograd and FFT-based Convolutions

Classic direct convolution for $R\times R$ filters over $M\times M$ output tiles requires $(K_h, K_w)$0 multiplies. Winograd’s minimal-filter convolution, essential in most fast CNN libraries, exploits polynomial interpolation and the Chinese Remainder Theorem (CRT) to reduce this count. For a 1D convolution with $(K_h, K_w)$1, the Winograd bilinear form is: $(K_h, K_w)$2 with $(K_h, K_w)$3, $(K_h, K_w)$4, $(K_h, K_w)$5 as transform matrices and $(K_h, K_w)$6 element-wise multiplication. In 2D: $(K_h, K_w)$7 The arithmetic reduction for Winograd is: $(K_h, K_w)$8 where $(K_h, K_w)$9 is the tile size. For $(S_h, S_w)$0, the reduction is $(S_h, S_w)$1, for $(S_h, S_w)$2, $(S_h, S_w)$3. FFT-based convolution is asymptotically optimal for large kernels but incurs high overhead and complex arithmetic for $(S_h, S_w)$4 and $(S_h, S_w)$5 kernel sizes (Liu et al., 2020, Barabasz et al., 2018).

3. Advanced Winograd Variants and Arithmetic Reduction

Winograd algorithms can be generalized beyond linear polynomials. Toom–Cook (minimal-filtering) is a special case using only linear CRT factors; extending with higher-degree polynomials (e.g., $(S_h, S_w)$6) enables better conditioning and accuracy, especially in low-precision formats (FP16, BF16) (Barabasz et al., 2019). For a kernel of size $(S_h, S_w)$7 and output tile of size $(S_h, S_w)$8, the per-output arithmetic is:

• Direct: $(S_h, S_w)$9 multiplies
• Winograd/Toom–Cook: $(P_h, P_w)$0 multiplies ($(P_h, P_w)$1)
• Extended Winograd (quadratic or superlinear factors): increased $(P_h, P_w)$2, requiring more per-tile multiplies but permitting larger tiles and superior FP accuracy in quantized scenarios

Integer-based Winograd over $(P_h, P_w)$3 further optimizes: for $(P_h, P_w)$4, complex-based construction achieves a $(P_h, P_w)$5 reduction (46 vs. 144 multiplies) with an efficiency gain up to $(P_h, P_w)$6 over rational arithmetic (Meng et al., 2019).

4. Numerical Stability and Error Analysis

Winograd algorithms exhibit numerical instability with increasing tile size due to the ill-conditioning of Vandermonde matrices. Floating-point (FP) error, characterized by machine epsilon $(P_h, P_w)$7, grows exponentially in tile size $(P_h, P_w)$8. Worst-case FP error bounds are of the form: $(P_h, P_w)$9 where $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$0, $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$1, $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$2 are constants from transform operations (Barabasz et al., 2018).

Mitigation strategies:

• Modified Toom–Cook: use a “$$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$3-point” to reduce error by $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$4–$$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$5
• Heuristics for point selection: favor small integers, sign/reciprocal pairs, and low-precision differences
• Mixed-precision computation: pre/post-process in FP64, accumulate in lower precision
• Canonical Huffman-order summation and pairwise reduction: empirically up to $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$6 total error reduction in multi-channel settings

For extended Winograd, use of quadratic factors (e.g., $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$7) improves error bounds under FP16/BF16 (Barabasz et al., 2019). Tabled empirical results confirm L1-errors and recognition rates consistent with error-analysis predictions.

5. Winograd Convolution in Integer and Quantized Domains

Winograd is difficult to deploy for INT8/low-precision inference due to transform denominators, scaling, and precision overflow. Solutions include:

• Integer-based Winograd with conjugate-pair optimization and integer filter scaling: enables $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$8 bit-width reduction (from 13 bits to 9 bits for $$H_{out} = \left\lfloor \frac{H_{in} + 2P_h - K_h}{S_h} \right\rfloor + 1$$9) with negligible loss in top-1/top-5 classification accuracy (Meng et al., 2019).
• Efficient RNS-based Winograd: transforms are performed in a Residue Number System (RNS) for exact modular arithmetic. Each input/filter is projected to $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$0 independent channels mod $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$1:

$$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$2

Outputs are recombined using CRT/MRC. Arithmetic complexity reduction is up to $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$3, with measured speed-up $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$4–$$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$5 for $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$6 and $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$7 filters, with no degradation in prediction accuracy for quantized networks. RNS-Winograd supports 8-to-16-bit arithmetic, plugs into existing integer-GEMM libraries, and is robust to FP numeric fragility even for large tiles ($$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$8 up to $$W_{out} = \left\lfloor \frac{W_{in} + 2P_w - K_w}{S_w} \right\rfloor + 1$$9) (Liu et al., 2020).

Fast Convolution Method	Mults/Output (for $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$0)	Numeric Stability	Speed-up Factor
Direct	9	High	1×
Winograd F(4×4,3×3)	2.25	Moderate	4×
Winograd–RNS F(12×12,5×5)	n=3, up to 4.69×	Exact INT8/16	up to 7.03×
Integer Winograd–$H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$1	3.13× efficiency gain	Robust	up to 17.4%

6. Empirical Performance and Architectural Considerations

Extensive evaluations on VGG-16, Inception-v3, and ResNet50 have shown that advanced Winograd and RNS-Winograd can provide throughput gains in low-precision inference without accuracy degradation. For example, 8-bit RNS-Winograd ($H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$2) yields $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$3 speed-up over INT8 Im2col+GEMM with top-1 accuracy unchanged at $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$4 on ImageNet; similar trends hold for $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$5 filters (Liu et al., 2020).

Practical considerations:

• Winograd, FFT, and related techniques are ideal for small kernels but require meticulous error control in floating-point implementations.
• RNS-Winograd and integer-based Winograd optimize for hardware supporting quantized GEMMs.
• Extended Winograd with quadratic CRT factors balances throughput and FP accuracy, especially for mixed-precision and truncated formats (FP16, BF16) (Barabasz et al., 2019).

7. Broader Implications, Limitations, and Future Directions

Direct convolution, while robust, is computationally intensive for small kernels. FFT-based convolution offers asymptotic advantages for large tiles, but significant overheads for standard $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$6 or $H_{out} = S_h(H_{in} - 1) + K_h - 2P_h$7 kernels. Strassen-like algorithms are unsuitable for CNN kernel sizes. Winograd’s minimal filtering, when adapted to integer/RNS domains or with rational/complex transforms, offers a compelling trade-off of arithmetic reduction with manageable numerical stability.

Winograd methods have become a cornerstone of efficient CNN inference in modern libraries, with applicability spanning floating-point, integer, and mixed-precision regimes. Future research will continue to explore further extensions to CRT-based convolution, quantization strategies, and hardware-specific optimizations—targeting ever-larger tile sizes, deeper complexity reductions, and new architectures without compromise in empirical accuracy (Liu et al., 2020, Dumoulin et al., 2016, Meng et al., 2019, Barabasz et al., 2018, Barabasz et al., 2019).