---
title: 'Multi-stage INRN: Hardware-Efficient Pipelines'
url: https://www.emergentmind.com/topics/multi-stage-inrn
type: topic
---

# Multi-stage INRN: Hardware-Efficient Pipelines

Multi-stage INRN refers to a family of hardware-efficient Implicit Neural Representation Networks (INRs) that leverage a unified, multi-stage pipelined architecture for computing activation functions, exemplified by the QuadINR framework employing piecewise quadratic activations. This approach centers on the hardware realization and theoretical advances allowing superior expressivity and energy efficiency for INR models, particularly in image and video reconstruction domains, while radically reducing area, power, and latency demands relative to previous INR activation paradigms [2508.14374].

## 1. Unified N-Stage Pipeline Framework

The central innovation of multi-stage INRN architectures is an $N$-stage pipelined computation of smooth or piecewise activations $\varphi$, implemented as a sequenced set of hardware stages $S_0 \rightarrow S_1 \rightarrow \ldots \rightarrow S_{N+1}$, each performing a fundamental multiply and/or add at a fixed clock rate (e.g., 100 MHz on FPGA, 1 GHz in 28 nm ASIC). For any target activation $\varphi$, a Taylor or piecewise polynomial expansion is apportioned across $N$ steps:

- **Stage $S_0$ ("Power Term Multiplier")**: Computes $p_0 = x^2$ and the linear term $\pm 2x$ in tandem (supporting piecewise formulations).
- **Stage $S_1$ ("Coefficient Multiplier" or "Continued Power Term")**: For even-powered activations, $p_0$ is scaled by $c_2$; odd-powered cases continue with $p_1 = p_0 \cdot x^2$. For piecewise quadratic, the terms are summed at this point.
- **Stages $S_2 \ldots S_N$ ("Multiply + Accumulate")**: Each successive stage increments the polynomial approximation, multiplying by $x^2$, scaling by $c_{2i}$ or $c_{2i+1}$, and accumulating.
- **Stage $S_{N+1}$ ("Final Accumulation")**: Outputs the final result.

In the canonical QuadINR design, $N=2$ suffices, with $S_0$ computing $x^2$ and $\pm2x$ in parallel, and $S_1$ summing them. These activation modules are inserted between the multilayer perceptron (MLP) linear layers, yielding an overall composition:

$$
f(x) = h_L \circ \varphi \circ h_{L-1} \circ \ldots \circ \varphi \circ h_1(x)
$$

where $h_i(z) = W_i z + b_i$ denotes the $i$-th affine transform.

## 2. Piecewise Quadratic Activation and Fourier Analysis

QuadINR's activation function $\varphi(x)$ is defined periodically ($T=4$) on each period as:

\[
\varphi(x) =
\begin{cases}
x^2 + 2x, & -2 < x \leq 0 \\
- x^2 + 2x, & 0 < x < 2
\end{cases}
\]

The derivative is:

\[
\varphi'(x) =
\begin{cases}
2x + 2, & -2 < x \leq 0 \\
-2x + 2, & 0 < x < 2
\end{cases}
\]

Periodic extension ensures smooth, rich spectral content. The closed-form Fourier expansion,

\[
\varphi(x) = \sum_{k \in \mathbb{Z}} 
\left[ \frac{32}{\pi^3 (2k+1)^3} \right]
\sin \left( \frac{(2k+1)\pi x}{2} \right)
\approx 1.032 \sin \frac{\pi x}{2} + 0.129 \sin \frac{3\pi x}{2} + \ldots
\]

provides high harmonic coverage, more effectively mitigating spectral bias in INRs than classical Taylor-truncation or unimodal activations [2508.14374]. The construction achieves exactness over each segment, unlike truncated Taylor series.

## 3. Composition of Pipeline Stages in Network Mapping

Each network layer applies the per-stage transforms $P_i$ such that:

\[
\varphi(x) = P_{N+1} \circ P_N \circ \ldots \circ P_0(x)
\]

Defining $F_0(x) = x$, $F_{2i+1}(x) = h_{i+1}(F_{2i}(x))$, and $F_{2i+2}(x) = \varphi(F_{2i+1}(x))$, the final output becomes:

\[
f(x) = F_{2L}(x) = P_{N+1} \circ \ldots \circ P_0 \bigl(h_L(\ldots \varphi(h_1(x))\ldots)\bigr)
\]

This structural interleaving of affine and activation modules yields deep, expressive function classes within minimal hardware overhead.

## 4. Hardware Realization and Module Metrics

The unified $N$-stage pipeline is directly mapped to hardware for both FPGA and ASIC targets. For a single piecewise quadratic activation ($N=2$):

**FPGA (Xilinx VCU128, 100 MHz):**

| Module             | LUT | FF  | DSP | LUTRAM | BRAM | Power (mW) | Latency (ns) |
|--------------------|-----|-----|-----|--------|------|------------|--------------|
| Single Activation  |1258 | 97  |  2  |   0    |  0   |    28      |    20        |

**Full Five-Layer INR Accelerator:**

| Component       | LUT   | FF      | DSP   | LUTRAM | BRAM | Power (mW) | Latency (ns) |
|-----------------|-------|---------|-------|--------|------|------------|--------------|
| MAC Array       |244759 |442268   |5130   |14394   |  0   |   6350     |   3760       |
| AF ×4 modules   |4×1258 |4×97     |4×2    |   0    |  0   |   112      |   80         |
| Others/Total    |250376 |541726   |5138   |14790   | 231  |   7542     |  14240       |

**ASIC (28 nm, 1 GHz, activation for 768×512 image):**

| Design    | Area (μm²) | Static P (mW) | Dyn. P (mW) | Energy (μJ/img) |
|-----------|------------|---------------|-------------|-----------------|
| QuadINR   |   1914     |    1.54       |    6.14     |     9.69        |
| SIREN     |   8415     |    6.83       |   23.30     |    36.79        |
| Gaussian  |  37099     |   20.60       |  112.25     |   177.30        |
| WIRE      |  74536     |   32.00       |  228.75     |   362.19        |
| FINER     |  25765     |   15.34       |   69.45     |   109.60        |
| Sinc      |   6582     |    5.12       |   16.55     |    26.10        |

This highlights the area, power, and runtime efficiencies accomplished by the piecewise quadratic design over previous sinusoidal (SIREN), Gaussian, wavelet, and Sinc-based AFs [2508.14374].

## 5. Pipelined Hardware Implementation: Pseudocode Description

The hardware mapping corresponds to the following pipeline (shown for FP32 precision):

```c
// Inputs: x   (FP32), coefficients c[0..N],   // for piecewise, c={+1,−1} segments
// Outputs: y = φ(x)
// Pipeline depth = N+2 stages

Stage S0:      // Power Term & Linear Term
    p0 ← x * x           // one FP32 multiplier
    ℓ  ← (x>0 ? +2.0*x : x*2.0)   // sign logic + FP32 multiplier
    pass p0, ℓ to next stage

Stage S1:      // Coefficient Multiply for p0
    t1 ← c[0] * p0       // piecewise c[0]=+1 or −1
    sum← t1 + ℓ          // FP32 adder
    pass sum to S2

Stage S2…S(N): // for general AF only; skipped in QuadINR
    ti ← c[i] * (previous power term)
    sum ← sum + ti       // accumulate
    new_power ← previous_power * x*x
    pass sum,new_power

Stage S(N+1):  // Final
    y ← sum
    output y
```

Each clock cycle performs the necessary floating-point multiplications and additions, resulting in a throughput of one $\varphi(x)$ computation per cycle for minimal pipeline depth.

## 6. Comparative Efficacy and Impact

The multi-stage, piecewise quadratic INRN (QuadINR) achieves substantial quantitative advantages:

- **Area (ASIC, 28 nm):** $1,914\,\mu\mathrm{m}^2$ vs $8,415\,\mu\mathrm{m}^2$ (SIREN), and up to $-97\%$ vs WIRE ($74,536\,\mu\mathrm{m}^2$)
- **Dynamic Power (ASIC, 28 nm):** $6.14$ mW vs $23.30$ mW (SIREN), and up to $-97\%$ vs WIRE ($228.75$ mW)
- **Latency (FPGA):** $20$ ns per $\varphi$ vs $60$ ns (SIREN), $320$ ns (WIRE)
- **Resource Savings (FPGA):** Activation modules cut to $1,258$ LUT ($-65\%$ to $-97\%$ vs SIREN/WIRE) and $28$ mW per instance ($-97\%$ power); end-to-end accelerator area and power are similarly reduced.
- **Signal Reconstruction Performance:** Up to $+2.06$ dB PSNR improvement on Kodak images over prior INR approaches (QuadINR vs FINER).

These results establish the unified N-stage pipeline with piecewise quadratic activation as the most hardware-efficient documented INR framework, achieving high-frequency representational fidelity and minimal power, area, and delay overhead [2508.14374].

Source: https://www.emergentmind.com/topics/multi-stage-inrn