Residue Number System Winograd

• Residue Number System (RNS) Winograd is a reformulation of the classical Winograd convolution, embedding modular arithmetic and CRT reconstruction to enable efficient low-precision integer computations.
• It translates conventional Winograd transforms into modular operations using 8–16 bit arithmetic, supporting larger transform tiles without sacrificing model accuracy.
• Empirical evaluations show significant throughput improvements on modern hardware by reducing multiplication counts and mitigating floating-point instability.

Residue Number System (RNS) Winograd convolution is a reformulation of the classic Winograd minimal-filtering convolution algorithm for efficient, exact, low-precision integer computation, leveraging the parallel, carry-free properties of the Residue Number System. Motivated by the limitations of floating-point Winograd implementations for low-bitwidth convolutional neural network (CNN) inference, RNS-Winograd enables large transform tiles (e.g., 10×10 to 16×16) and near-minimal arithmetic complexity with 8–16 bit integer operations and no sacrifice in prediction accuracy. By embedding all Winograd steps into modular arithmetic over several small, pairwise-coprime moduli, and reconstructing full-precision results via the Chinese Remainder Theorem (CRT), RNS-Winograd achieves substantial throughput improvements on modern low-precision hardware (Liu et al., 2020).

1. The Classic Winograd Minimal-Filtering Algorithm

Winograd minimal filtering, denoted as $F(M\times M, R\times R)$, computes the convolution of an $R\times R$ filter $g$ with an input patch $d$ to yield an $M\times M$ output tile $Y$. Let $N = M + R - 1$. The Winograd convolution in matrix terms is:

$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$

where $G$ (size $N\times R$) and $R\times R$0 (size $R\times R$1) are the filter and data transform matrices, $R\times R$2 (size $R\times R$3) the inverse transform matrix, and $R\times R$4 denotes element-wise multiplication.

Operationally:

• Filters are transformed: $R\times R$5.
• Inputs are transformed: $R\times R$6.
• Element-wise multiplication: $R\times R$7.
• The result is inverse transformed: $R\times R$8.

Winograd reduces arithmetic costs. Direct convolution requires $R\times R$9 multiplications, Winograd just $g$0 per tile, yielding a reduction factor $g$1. For example, $g$2 gives a $g$3 reduction.

2. Residue Number System (RNS) and CRT Reconstruction

An RNS is defined by $g$4 pairwise-coprime moduli $g$5 with total dynamic range $g$6. Each integer $g$7 is represented by its residues $g$8, where $g$9.

Arithmetic in RNS is performed independently in each channel (modulus):

• Addition/multiplication: $d$0, $d$1.

CRT reconstruction retrieves $d$2 from its residues: For each $d$3, let $d$4 and $d$5. Then,

$d$6

This enables fully parallel, carry-free, low-precision computation suitable for quantized neural network inference.

3. Mapping Winograd Transforms to RNS

To embed Winograd into RNS, all transform matrices ($d$7, $d$8, $d$9) must be expressed in integer or modular form. Rational entries are cleared via a least common multiple $M\times M$0 of all denominators (e.g., $M\times M$1 with integer $M\times M$2). Each modulus $M\times M$3 must be coprime to $M\times M$4 to permit modular inversion:

• $M\times M$5
• Analogously for $M\times M$6

This process ensures all per-modulus transforms can be conducted with 8- or 16-bit integer arithmetic within each channel, so the full Winograd procedure is mapped to efficient, low-precision modular operations.

4. End-to-End RNS–Winograd Convolution Process

Given quantized INT8 filters and activations and RNS base $M\times M$7, the algorithm proceeds channel-wise as follows:

1. Forward Filter Transform (per residue channel $M\times M$8):

$M\times M$9

Can be precomputed per filter.

2. Forward Data Transform:

$Y$0

3. Pointwise Multiplication:

$Y$1

4. Inverse Output Transform:

$Y$2

5. CRT Reconstruction: For each output element $Y$3, recover

$Y$4

The forward data transform is highly amortized, as patches are reused across channels. Depthwise summations and pointwise multiplies can leverage optimized INT8/INT16 GEMM kernels at each modulus (Liu et al., 2020).

5. Complexity Reduction and Arithmetic Analysis

For output tiles $Y$5, filters $Y$6, tile size $Y$7, and $Y$8 RNS channels, when transform and CRT costs are amortized, the dominant cost is $Y$9 multiplies per output tile, compared to $N = M + R - 1$0 for direct convolution. The theoretical speedup is:

$N = M + R - 1$1

Representative reduction factors are shown in the following table:

Filter $N = M + R - 1$2	Tile $N = M + R - 1$3	$N = M + R - 1$4 (8-bit)	$N = M + R - 1$5 (16-bit)
$N = M + R - 1$6	$N = M + R - 1$7	$N = M + R - 1$8	$N = M + R - 1$9
$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$0	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$1	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$2	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$3
$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$4	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$5	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$6	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$7
$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$8	$Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A$9	$G$0	$G$1
$G$2	$G$3	$G$4	$G$5

In practice, $G$6 is selected in the range $G$7–$G$8 to balance transform cost, reconstruction overhead, and GEMM efficiency.

6. Empirical Performance and Accuracy

Experiments confirm that RNS–Winograd delivers significant acceleration without impairing model accuracy:

• On Arm Cortex-A73 CPU, 8-bit RNS–Winograd ($G$9, $N\times R$0 filters, $N\times R$1) yields $N\times R$2 speedup over INT8 im2col+GEMM baseline for VGG16, with no loss in ImageNet Top-1 accuracy (71.4%).
• 16-bit RNS–Winograd ($N\times R$3) achieves $N\times R$4 speedup over INT16 baseline.
• For $N\times R$5 filters (Inception-v3), $N\times R$6 in 8-bit RNS achieves up to $N\times R$7 speedup.
• Transform and CRT/mixed-radix overheads are modest: forward transforms ($N\times R$8), output transforms ($N\times R$9), CRT ($R\times R$00) (Liu et al., 2020).
• Arithmetic reduction for large tiles ($R\times R$01, $R\times R$02, $R\times R$03) reaches $R\times R$04.

Empirical validation across VGG16, ResNet50, and Inception (v1/v3) shows no observable top-1 accuracy loss.

7. Implementation Guidelines and Efficacy

RNS–Winograd eliminates numerical instability inherent in large-tile Winograd with FP32 by confining all computation to integer modular domains. The approach is well-suited for modern accelerators supporting wide low-precision GEMMs and parallel SIMD execution. Implementation steps are:

1. Select appropriate Winograd tile $R\times R$05 and RNS moduli $R\times R$06, ensuring the product covers dynamic range and each $R\times R$07 is coprime with $R\times R$08 (clearance of transform denominators).
2. Precompute transformed matrices $R\times R$09 mod $R\times R$10.
3. For each layer: pre-transform filters, process input patches, multiply in the Winograd domain channel-wise, and reconstruct results by CRT.
4. Aggregate into batched GEMMs for throughput.

RNS–Winograd thus recovers the arithmetic and wall-clock gains of classic minimal filtering, but mapped to robust, tractable, low-precision integer arithmetic (Liu et al., 2020). This approach removes common sources of Winograd instability, provides a parallelism-friendly substrate, and requires only small increases in transformation overhead relative to the potential acceleration. The method is empirically validated at scale for modern CNNs without accuracy degradation.