---
title: Residue Number System Winograd
url: https://www.emergentmind.com/topics/residue-number-system-rns-winograd
type: topic
---

# Residue Number System Winograd

Residue Number System (RNS) Winograd convolution is a reformulation of the classic Winograd minimal-filtering convolution algorithm for efficient, exact, low-precision integer computation, leveraging the parallel, carry-free properties of the Residue Number System. Motivated by the limitations of floating-point Winograd implementations for low-bitwidth convolutional neural network (CNN) inference, RNS-Winograd enables large transform tiles (e.g., 10×10 to 16×16) and near-minimal arithmetic complexity with 8–16 bit integer operations and no sacrifice in prediction accuracy. By embedding all Winograd steps into modular arithmetic over several small, pairwise-coprime moduli, and reconstructing full-precision results via the Chinese Remainder Theorem (CRT), RNS-Winograd achieves substantial throughput improvements on modern low-precision hardware [2007.12216].

## 1. The Classic Winograd Minimal-Filtering Algorithm

Winograd minimal filtering, denoted as $F(M\times M, R\times R)$, computes the convolution of an $R\times R$ filter $g$ with an input patch $d$ to yield an $M\times M$ output tile $Y$. Let $N = M + R - 1$. The Winograd convolution in matrix terms is:
$$
Y = A^\top [ (G g G^\top) \circ (B^\top d B) ] A
$$
where $G$ (size $N\times R$) and $B^\top$ (size $N\times N$) are the filter and data transform matrices, $A^\top$ (size $M\times N$) the inverse transform matrix, and $\circ$ denotes element-wise multiplication.

Operationally:
- Filters are transformed: $\hat{W} = G g G^\top$.
- Inputs are transformed: $\hat{D} = B^\top d B$.
- Element-wise multiplication: $\hat{S} = \hat{W} \circ \hat{D}$.
- The result is inverse transformed: $Y = A^\top \hat{S} A$.

Winograd reduces arithmetic costs. Direct convolution requires $M^2 R^2$ multiplications, Winograd just $N^2$ per tile, yielding a reduction factor $(M^2R^2)/(M+R-1)^2$. For example, $F(2\times2,3\times3)$ gives a $2.25\times$ reduction.

## 2. Residue Number System (RNS) and CRT Reconstruction

An RNS is defined by $k$ pairwise-coprime moduli $\{ m_1, ..., m_k \}$ with total dynamic range $M = \prod_i m_i$. Each integer $x \in [0, M-1]$ is represented by its residues $\{ x_1, ..., x_k \}$, where $x_i \equiv x \pmod{m_i}$. 

Arithmetic in RNS is performed independently in each channel (modulus):
- Addition/multiplication: $(x \pm y)_i \equiv (x_i \pm y_i) \pmod{m_i}$, $(x \cdot y)_i \equiv (x_i \cdot y_i) \pmod{m_i}$.

CRT reconstruction retrieves $x$ from its residues:
For each $m_i$, let $M_i = M/m_i$ and $y_i \equiv M_i^{-1} \pmod{m_i}$. Then,
$$
x \equiv \sum_{i=1}^k x_i M_i y_i \pmod{M}
$$

This enables fully parallel, carry-free, low-precision computation suitable for quantized neural network inference.

## 3. Mapping Winograd Transforms to RNS

To embed Winograd into RNS, all transform matrices ($A^\top$, $B^\top$, $G$) must be expressed in integer or modular form. Rational entries are cleared via a least common multiple $\alpha$ of all denominators (e.g., $G = (1/\alpha) G'$ with integer $G'$). Each modulus $m_i$ must be coprime to $\alpha$ to permit modular inversion:
- $G_{(m_i)} = (\alpha^{-1} \pmod{m_i}) \cdot (G' \pmod{m_i})$
- Analogously for $B^\top_{(m_i)},\,A^\top_{(m_i)}$

This process ensures all per-modulus transforms can be conducted with 8- or 16-bit integer arithmetic within each channel, so the full Winograd procedure is mapped to efficient, low-precision modular operations.

## 4. End-to-End RNS–Winograd Convolution Process

Given quantized INT8 filters and activations and RNS base $\{m_i\}$, the algorithm proceeds channel-wise as follows:

1. **Forward Filter Transform (per residue channel $i$):**
   $$\hat{W}^{(i)} = G_{(m_i)}\,g\,G_{(m_i)}^\top \pmod{m_i} $$
   Can be precomputed per filter.

2. **Forward Data Transform:**
   $$\hat{D}^{(i)} = B^\top_{(m_i)}\,d\,B_{(m_i)} \pmod{m_i} $$

3. **Pointwise Multiplication:**
   $$\hat{S}^{(i)} = \hat{W}^{(i)} \circ \hat{D}^{(i)} \pmod{m_i} $$

4. **Inverse Output Transform:**
   $$Y^{(i)} = A^\top_{(m_i)}\,\hat{S}^{(i)}\,A_{(m_i)} \pmod{m_i} $$

5. **CRT Reconstruction:** For each output element $y[p,q]$, recover
   $$y[p,q] = CRT( \{ Y^{(i)}[p,q] \} )$$

The forward data transform is highly amortized, as patches are reused across channels. Depthwise summations and pointwise multiplies can leverage optimized INT8/INT16 GEMM kernels at each modulus [2007.12216].

## 5. Complexity Reduction and Arithmetic Analysis

For output tiles $M\times M$, filters $R\times R$, tile size $N = M+R-1$, and $n$ RNS channels, when transform and CRT costs are amortized, the dominant cost is $nN^2$ multiplies per output tile, compared to $M^2R^2$ for direct convolution. The theoretical speedup is:
$$
\mathrm{Speedup} \approx \frac{M^2R^2}{nN^2}
$$

Representative reduction factors are shown in the following table:

| Filter $R$ | Tile $M$ | $n=3$ (8-bit) | $n=2$ (16-bit) |
|------------|----------|---------------|----------------|
| $3$        | $4$      | $1.33\times$  | $2.00\times$   |
| $3$        | $6$      | $1.69\times$  | $2.53\times$   |
| $3$        | $12$     | $2.20\times$  | $3.31\times$   |
| $5$        | $10$     | $4.25\times$  | $6.38\times$   |
| $5$        | $12$     | $4.69\times$  | $7.03\times$   |

In practice, $M$ is selected in the range $8$–$16$ to balance transform cost, reconstruction overhead, and GEMM efficiency.

## 6. Empirical Performance and Accuracy

Experiments confirm that RNS–Winograd delivers significant acceleration without impairing model accuracy:
- On Arm Cortex-A73 CPU, 8-bit RNS–Winograd ($M=14$, $3\times3$ filters, $n=3$) yields $2.02\times$ speedup over INT8 im2col+GEMM baseline for VGG16, with no loss in ImageNet Top-1 accuracy (71.4%).
- 16-bit RNS–Winograd ($n=2$) achieves $\sim 2.2\times$ speedup over INT16 baseline.
- For $5\times5$ filters (Inception-v3), $F(10\times 10, 5\times 5)$ in 8-bit RNS achieves up to $2.31\times$ speedup.
- Transform and CRT/mixed-radix overheads are modest: forward transforms ($\sim7.9\%$), output transforms ($\sim9.2\%$), CRT ($\sim1.1\%$) [2007.12216].
- Arithmetic reduction for large tiles ($M=12$, $R=5$, $n=3$) reaches $4.69\times$.

Empirical validation across VGG16, ResNet50, and Inception (v1/v3) shows no observable top-1 accuracy loss.

## 7. Implementation Guidelines and Efficacy

RNS–Winograd eliminates numerical instability inherent in large-tile Winograd with FP32 by confining all computation to integer modular domains. The approach is well-suited for modern accelerators supporting wide low-precision GEMMs and parallel SIMD execution. Implementation steps are:
1. Select appropriate Winograd tile $M$ and RNS moduli $\{ m_i \}$, ensuring the product covers dynamic range and each $m_i$ is coprime with $\alpha$ (clearance of transform denominators).
2. Precompute transformed matrices $A^\top_{(m_i)},\,B^\top_{(m_i)},\,G_{(m_i)}$ mod $m_i$.
3. For each layer: pre-transform filters, process input patches, multiply in the Winograd domain channel-wise, and reconstruct results by CRT.
4. Aggregate into batched GEMMs for throughput.

RNS–Winograd thus recovers the arithmetic and wall-clock gains of classic minimal filtering, but mapped to robust, tractable, low-precision integer arithmetic [2007.12216]. This approach removes common sources of Winograd instability, provides a parallelism-friendly substrate, and requires only small increases in transformation overhead relative to the potential acceleration. The method is empirically validated at scale for modern CNNs without accuracy degradation.

Source: https://www.emergentmind.com/topics/residue-number-system-rns-winograd