RQP: Resource-Oriented Quantiser Pruning for Neural Networks on FPGAs

Abstract: High granularity quantisation (HGQ) exploits weight-level quantisation and pruning to design resource-efficient neural network accelerators, achieving an attractive trade-off between accuracy and hardware utilisation. HGQ is particularly well suited to FPGA-based edge neural network applications. Standard HGQ workflow starts from a high-precision model and progressively reduces bit width, guided by gradient-based optimisation to outline the Pareto frontier. This monotonic and irreversible pruning process is computationally intensive and can overlook the optimal subnetwork for a given resource level. We propose a resource-oriented one-shot quantiser pruning method that brings the network directly close to the target search space, and then use bidirectional beta scheduling for fine-tuning to enable a more refined scan of the Pareto frontier. Validated on the jet substructure classification, JSC, task, our method reduces the search cost by up to 20.58x compared with monotonic resource reduction in standard HGQ workflows, while achieving a competitive Pareto frontier and final network configuration.

• The paper introduces RQP, a resource-targeted method that directly reallocates bit widths, applies spectral constraints to preserve trainable connectivity, and uses bidirectional β scheduling to refine FPGA neural networks near a target EBOP budget.
• RQP reaches 73.87% accuracy at 399 EBOPs in 6,000 epochs, reducing search cost by up to 20.58× versus a 200,000-epoch HGQ search while outperforming comparable pruning baselines, especially at low budgets.
• Hardware tests on a Virtex UltraScale+ show higher frequency and lower latency at similar LUT counts, including 657.462 MHz versus 521.376 MHz and 21.29 ns versus 23.02 ns, although results remain limited to small fully connected models and single-seed experiments.

Motivation and problem statement

FPGA-based neural network inference for ultra-low-latency applications — high-energy physics triggers, quantum control, high-frequency trading — requires simultaneous optimisation of accuracy and on-chip resource usage. High Granularity Quantisation (HGQ) addresses this by making per-weight bit widths differentiable optimisation variables, unifying quantisation and pruning at the bit-width level and using a differentiable resource proxy, Effective Bit Operations (EBOPs), to estimate LUT utilisation during training. However, the standard HGQ workflow is a monotonic, irreversible process: training starts from a high-precision model and progressively increases the resource-pressure coefficient $\beta$ until the target budget is reached. This has two drawbacks identified by the authors: substantial training effort is spent on intermediate resource regions that never deploy, and the monotonic compression exhibits path dependence — connections pushed to low bit-widths or removed early have restricted gradient paths, so better subnetworks may be missed.

Methodology

RQP replaces progressive compression with two components built on top of HGQ's loss function ($$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$).

One-shot quantiser pruning with spectral constraints. Given a pre-trained high-resource HGQ model and a target EBOPs budget $E^\star$, RQP first performs layer-wise bit-width reallocation: a global contraction ratio $\alpha = E^\star / E_{\text{cur}}$ is computed, and each layer's bit-width matrix is rescaled by a layer-specific factor $\alpha_l = \alpha^{r_l/\bar r}$ derived from mean active bit widths, then clipped to the valid range. A second step applies spectral-constrained topology compensation: a structural score $$\mathbf{S}_l = |\mathbf{W}_l| \odot \mathbf{B}_l^{(0)}$$ selects retained connections under a minimum-degree constraint (degree ≥ 2), followed by greedy local repair enforcing a condition-number bound $\kappa(\mathbf{W}_l \odot \mathbf{M}_l) \le \tau_l$ with $\tau_l = 3\,\kappa(\mathbf{W}_l^{\text{pre}})$. The authors position this as a trainability proxy — suppressing ill-conditioned sparse supports preserves stable signal and gradient paths, unlike a spectral-norm bound that only caps maximum layer gain. Disconnected or zero-bit-width channels are excluded from the active matrix so that intentional pruning is not misread as rank collapse. A final global scalar $\lambda^\star$ closes any residual budget gap. Notably, this operates on continuous quantiser bit widths rather than discrete binary masks, which is what distinguishes it from SNIP-, GraSP-, or magnitude-style one-shot pruning; for comparison purposes, baseline masks are mapped onto quantisers in their experiments.

Bidirectional $\beta$ scheduling. After pruning, fine-tuning uses a closed-loop controller on $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$0. An inner loop compares observed EBOPs $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$1 against $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$2 each epoch and adjusts $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$3 proportionally to $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$4, pushing pressure up when over budget and relaxing when under. The paper identifies a "$$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$5 deadlock": sustained compression pushes many quantisers toward their lower admissible range, weakening straight-through gradients and making suppressed connections hard to reactivate, since restoring capacity is penalised by $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$6. An outer loop detects validation-metric stagnation over $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$7 epochs and relaxes $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$8 toward $$L = L_{\text{base}} + \beta \cdot \text{EBOPs} + \gamma \cdot \sum \text{bit-widths}$$9 with an annealing-style schedule whose timing parameter makes early restarts explore broadly and later restarts refine locally. This differs from prior PID-based $E^\star$0 tracking in that it explicitly targets escape from locked configurations rather than only tracking the budget.

Experimental results

Evaluation uses 5-class jet substructure classification (JSC) on OpenML and CERNBox datasets with a fully connected network (16–64–32–32–5), compiled via da4ml/HGQ and verified with out-of-context place-and-route in Vivado 2025.1 on a Virtex UltraScale+ device.

Search efficiency. Targeting 400 EBOPs, RQP (2,500 epochs of pre-training plus 3,500 epochs of near-budget search, 6,000 total) reaches 73.87% test accuracy at 399 actual EBOPs in 18.12 minutes, versus 73.64% at 398 EBOPs after 200,000 epochs of HGQ progressive search taking 372.96 minutes — a 20.58× reduction in search cost while exceeding the baseline's accuracy. Against equal-epoch (6,000) HGQ, RQP gains +0.23% accuracy with a smaller resource gap (−1 vs −10 EBOPs). The efficiency claim should be read carefully: the 20.58× figure compares against a much longer baseline run; the same-budget HGQ run is only modestly slower (15.27 min), and RQP's advantage there lies in accuracy and frontier quality rather than raw time.

Pruning baselines. Across target budgets from 400 to 6,839 EBOPs, RQP-consistent accuracy recovery dominates SNIP, Random, Reallocation (an ablation isolating the spectral constraint), Magnitude, and GraSP. The gap widens at low budgets, where random and magnitude pruning degrade substantially — supporting the claim that continuous bit-width pruning and the spectral constraint both contribute to subnet trainability.

Pareto frontier and hardware results. On CERNBox, RQP traces a higher accuracy-versus-LUT frontier than HGQ at comparable epochs, and surpasses prior LUT-based designs (LogicNets, PolyLUT/Add, NeuraLUT, AmigoLUT, hls4ml QAT points). Post-place-and-route comparisons show consistent small accuracy gains (e.g., 75.65% vs 75.45%; 76.61% vs 76.59%) at similar LUT counts, with notably higher achievable $E^\star$1: 657.462 MHz vs 521.376 MHz at the ~2.2k-LUT point, reducing end-to-end latency from 23.02 ns to 21.29 ns despite one additional pipeline stage. The authors attribute this to different post-pruning bit-width distributions shortening critical paths, though they concede that increased low-bit-width computation can add pipeline stages in some cases — compensated here but not guaranteed generally.

Limitations and open questions

The evaluation is confined to a single small fully connected architecture on JSC classification tasks; scalability to CNNs and transformers remains future work, and the authors note that gradient-based optimisation difficulty grows with network size. All reported results use a single seed (1998); multi-seed evaluation is explicitly deferred. The spectral constraint relies on an empirical condition-number threshold ($E^\star$2 the pre-trained condition number with minimum degree 2) whose generality across architectures is not established. Additionally, the LUT ≈ exp(0.985 log(EBOPs)) proxy inherits its empirical validity from HGQ, and the latency benefit from reduced bit-width distributions may not transfer to designs where added pipeline stages are not offset by frequency gains.

Conclusion

RQP reframes HGQ accelerator design as direct projection into a target resource region followed by bidirectional near-budget refinement, replacing monotonic compression. It achieves up to 20.58× faster convergence to a target design point with competitive or better accuracy, denser Pareto-frontier coverage, and improved post-implementation frequency, at the cost of control overhead relative to equal-epoch progressive search. Its main open questions concern generalisation beyond small MLPs, robustness across seeds, and the portability of the spectral trainability criterion.