Hardware-Aware Training: Optimization & Co-Design

• Hardware-aware training is a family of deployment-oriented optimization methods that incorporate real hardware constraints and non-idealities into the training process.
• It spans techniques from quantization-aware training under exact bit-precision to end-to-end co-design involving routing, latency, and mixed-precision strategies for FPGAs, ASICs, and neuromorphic systems.
• By integrating hardware forward models and post-training calibration, these methods reduce accuracy gaps, enhance energy efficiency, and tailor models to specific device characteristics.

Searching arXiv for papers on hardware-aware training and related deployment-aware optimization. Hardware-aware training denotes a family of deployment-oriented optimization procedures in which model training, fine-tuning, sparse rewiring, or hardware-specific specialization is aligned with the constraints and non-idealities of a target substrate rather than with ideal software arithmetic alone. In the literature, this ranges from training under the exact bit-precision assumptions used in deployment, to retraining with a realistic analog or mixed-signal forward model, to architecture and routing optimization under explicit latency, memory, or mappability constraints (Blouw et al., 2020, Rasch et al., 2023, Campos et al., 2023). Across these formulations, the unifying premise is that the relevant objective is not the nominal floating-point network, but the network after quantization, distortion, placement, routing, or physical realization.

1. Concept and scope

Hardware-aware training is not a single algorithmic template. One line of work defines it as training under the exact bit-precision assumptions used in deployment, so that reported accuracy is intended to equal deployed accuracy; in the keyword-spotting setting, this includes 7-bit activations, 8-bit or 4-bit weights, and pruning-aware fine-tuning for sparse models (Blouw et al., 2020). A second line treats it as an end-to-end co-designed workflow linking quantization-aware training, a hardware-faithful intermediate representation, and synthesis for FPGA or ASIC targets; an example is the pipeline from PyTorch through HAWQ, QONNX, and hls4ml (Campos et al., 2023). A third line places hardware awareness after super-network training, specializing sub-networks to device- and configuration-specific objectives such as latency, MACs, and accuracy or BLEU, rather than modifying the super-network training rule itself (Sarah et al., 2022).

This diversity matters because the term also covers optimization procedures that are not strictly “training” in the narrow SGD sense. In hardware-aware NAS, the objective may be stated as maximizing validation accuracy under a latency constraint, as in

$$\max \ \mathrm{ACC}_{\mathrm{val}(a)} \quad \textrm{s.t.} \quad \tau(a,h) \le \tau_c^{(h)},$$

with device-specific profiling shaping the search space itself (Zhang et al., 2019). In post-super-network search systems, the optimization is explicitly multi-objective,

$$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$

and hardware awareness enters through predictors, warm-start search, and hardware-specific evaluation rather than through the training loss of the super-network (Sarah et al., 2022). The literature therefore distinguishes parameter-level HAT, architecture-level hardware-aware optimization, and deployment-time specialization; conflating them obscures the different sources of hardware information and the different guarantees they provide.

2. Forward-modeling hardware inside the optimization loop

A common formalism is to treat hardware non-idealities as a perturbation of the forward operator. A controlled diagnostic study writes the clean layer as

$$\mathbf{z}=\mathbf{W}\mathbf{x}, \qquad \mathbf{y}=\phi(\mathbf{z}),$$

and the perturbed layer as

$$\tilde{\mathbf{z}}=\mathcal{G}(\mathbf{W},\mathbf{x};\xi,t), \qquad \tilde{\mathbf{y}}=\phi(\tilde{\mathbf{z}}),$$

or equivalently

$$\tilde{\mathbf{z}}=\mathbf{W}_{\mathrm{eff}}(\mathbf{W},\mathbf{x};\xi,t)\,\mathbf{x},$$

with HAT minimizing expected loss under sampled perturbations in the forward pass (Fang et al., 10 May 2026). This operator view is broad enough to include additive noise, time-dependent drift, discretization, parasitics, and structured defects.

In neuromorphic and analog settings, this often becomes an explicit differentiable hardware surrogate. For inference-only neuromorphic MAC circuits, the forward pass replaces ideal weights by a hardware-description function

$g_{HW}(w, X, \alpha),$

with $\alpha=0$ giving the ideal weight and $\alpha=1$ giving the target hardware model; binary, ternary, asymmetric ternary, quinary, and pruning transfer functions are implemented through smooth sigmoidal or $\tanh$-based surrogates so that ordinary backpropagation remains available (Obradovic et al., 2018). In analog in-memory computing, the forward model includes clipped DAC/ADC quantization, programming error, drift, read noise, output noise, IR-drop, and nonlinearities, and retraining is performed with the analog forward path inside SGD while the backward pass remains in regular floating point (Rasch et al., 2023). That study reports that many large-scale CNNs, RNNs, and transformers can be retrained to show iso-accuracy on AIMC, and that non-idealities acting like noise on inputs or outputs matter more than weight noise, with RNNs particularly robust (Rasch et al., 2023).

Physics-informed forward models also appear in silicon analog neural networks. For a time-domain vector-matrix multiplier with floating-gate cells, the ideal relation

$\Delta V_{\text{out},j} \sim \sum_i I_{i,j}\, t_i$

is replaced by a time-slot model that explicitly accounts for capacitive crosstalk and bit-line voltage drop; retraining then uses the HWA model in the forward propagation and the ideal model in the backpropagation (Filippeschi et al., 8 Dec 2025). On passive TiO$$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$0 ReRAM crossbars, ex-situ transfer is modeled by mapping weights to differential conductances, injecting conductance-dependent tuning imprecision, biasing-scheme disturbance, and HRS/LRS failures, and converting back to effective weights through a reparameterization-style pipeline (Drolet et al., 2023). In that case, 79.5% of the test set’s data points are classified with an accuracy of 95% or higher after transfer for the hardware-aware network, versus 18.5% for the regularly trained network (Drolet et al., 2023).

3. Numerical co-design: quantization, integer semantics, and mixed precision

A major branch of hardware-aware training concerns numerical semantics rather than continuous analog distortion. In FPGA/ASIC deployment, HAWQ is used to choose layerwise mixed precision under a BOPs budget, with QONNX preserving arbitrary and mixed-precision quantization and hls4ml translating the resulting graph into synthesizable firmware (Campos et al., 2023). For the reported jet-tagging classifier, homogeneous quantization below 8 bits degrades accuracy, motivating Hessian-guided mixed precision. The selected mixed-precision solution uses layer bit widths 4, 4, 5, and 4, achieves 75.78% accuracy, 90 ns latency, 34,842 LUTs, 9,622 FFs, 154 DSPs, 33% sparsity, and 182,260 BOPs; relative to baseline, that corresponds to a 42.2% reduction in LUTs, 36.3% reduction in FFs, and 95.7% reduction in DSPs, with about 1.1 percentage points of absolute accuracy loss (Campos et al., 2023).

In always-on keyword spotting, hardware-aware training is defined more narrowly as matching training-time arithmetic to the exact deployed arithmetic. The LMU-based systems are trained with 7-bit activations, 8-bit weights for LMU-1, and 4-bit weights for LMU-2/3/4, with pruning of 80% in LMU-3 and 91% in LMU-4 (Blouw et al., 2020). This yields a family of stateful, online, streaming models ranging from 96.9% accuracy at 1683 kbits to 92.7% accuracy at 49 kbits. For LMU-2, the same model family is analyzed on an ARM M4F at $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$1 and on custom hardware at $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$2, with the latter identified at 92 kHz and 8,052,298 transistors (Blouw et al., 2020).

Recent work extends numerical co-design below conventional weight and activation quantization. In full-integer inference, the rescale multiplicand used after 32-bit accumulation is treated as a hardware bottleneck; standard 32-bit multiplicands can be reduced to 8 bits with no accuracy loss across EfficientNet-Lite0, MobileNet-V1, and MobileNet-V2, and to 4 bits with lightweight fine-tuning on EfficientNet-Lite0, recovering 71.62% from a post-training 65.39% at 4-bit rescaling (Mueller et al., 13 Oct 2025). The associated accelerator synthesis reports over $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$3 area-delay product improvement for 4-bit multiplicands and over $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$4 for 8-bit multiplicands (Mueller et al., 13 Oct 2025). In structured state-space models, quantization-aware training has also been proposed explicitly for hardware deployment: QS4D states that QAT can reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics, enhance robustness to analog noise, enable structural pruning, and support deployment on a memristive analog in-memory computing substrate (Siegel et al., 8 Jul 2025).

4. Structural and architecture-level co-design

Hardware-aware optimization often acts on structure rather than only on tensor precision. In HURRICANE, CNN architecture search is driven by direct hardware profiling on Qualcomm Hexagon DSP, Snapdragon 845 ARM CPU, and Intel Movidius Myriad X VPU, because operator rankings change by hardware and even by layer context (Zhang et al., 2019). The search space is automatically reduced by keeping the top four operators per early layer and five in the last four layers, producing $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$5 architectures. Under matched latency budgets, HURRICANE-DSP reaches 76.67% top-1 at 16.5 ms, HURRICANE-CPU reaches 74.59% at 301.3 ms, and HURRICANE-VPU reaches 75.13% at 35.6 ms, each improving over Proxyless and SPOS variants on the same hardware targets (Zhang et al., 2019).

Post-training hardware specialization can also be systematized. A comprehensive search system built on OFA and HAT uses NSGA-II, lightweight predictors, warm-start search, ConcurrentNAS, and PopDB to accelerate extraction of hardware-specific sub-networks from pre-trained super-networks (Sarah et al., 2022). On OFA MobileNetV3-w1.2, ConcurrentNAS reaches 78.81% top-1 at 478M MACs in 100 validations, compared with WeakNAS at 78.70% and 513M MACs in 800 validations, giving the reported 8x faster search result (Sarah et al., 2022). The method is explicitly hardware- and configuration-specific, with Pareto fronts differing across V100 and CLX, and even across batch size and thread count on the same CPU platform (Sarah et al., 2022).

Other structural cases target routing and temporal parameterization. Routing-aware training for a small-world neuromorphic architecture extends DeepR by replacing a single sparsity target with a hop-conditioned profile

$$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$6

where $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$7 indexes sparsity at hop distance $$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$8 (Weber et al., 2024). The resulting SHD network is fully mappable to the hardware, achieves 5% more accuracy using the same number of parameters, and reaches iso-accuracy with 10x less memory usage than non-routing-aware training (Weber et al., 2024). For digital event-driven neuromorphic hardware, synaptic delays are made hardware-trainable by expanding each connection into parallel delayed synapses over

$$\omega^*=\underset{\omega \in \Omega}{\arg\min}\big(f_1(\omega; W), \ldots, f_m(\omega; W)\big),$$9

then pruning and quantizing before mapping to Loihi or Seneca (Patino-Saucedo et al., 2024). On SHD, the 700-48-48-20 delayed SNN reaches 87% in PyTorch, 87% on Loihi, and 86% on Seneca, while the added SCDQ delay hardware on Seneca reduces energy by 3.1× to 3.5× and latency by 3.5× to 4.3× relative to software-emulated delays (Patino-Saucedo et al., 2024).

5. Hardware domains and empirical breadth

Analog in-memory computing has become a principal testbed because it exposes both numerical and physical mismatch. A large-scale study spanning ResNet-32/CIFAR-10, WideResNet-16/CIFAR-100, multiple ImageNet CNNs, BERT-base and Albert-base on GLUE, LSTM acoustic and LLMs, and RNN-T reports that only one workload reaches the paper’s iso-accuracy criterion at 1 hour after programming without HWA, whereas after HWA all models improve above 96% normalized accuracy and five of eleven achieve iso-accuracy (Rasch et al., 2023). The same work reports that HWA reduces the worst raw accuracy gap from 21.81% to 2.65% and identifies input/output-side non-idealities as more damaging than weight noise (Rasch et al., 2023).

Measurement-driven training on physical hardware pushes the same theme toward defect populations rather than abstract perturbations. On 36 CMOS-integrated MTJ dies, each with 20,000 MTJs, defect-unaware software solutions with mean error 4% degrade sharply when mapped to defective hardware, despite a median yield of 99.2% (Borders et al., 2023). Die-specific hardware-aware training recovers to 5.2% mean error across dies, and statistics-aware training over a defect distribution yields mean MNIST misclassification error differing from the software baseline by only 2% on the 36 physical dies (Borders et al., 2023). This is a distinct form of HAT: optimization against a hardware population rather than a single nominal device.

Photonic and mixed-signal neuromorphic systems broaden the substrate range. For integrated photonic neural networks, hardware-aware training and pruning drive weights toward noise-robust and energy-efficient operating regions of microring and PCM devices; experimentally, handwritten-digit accuracy on an MRR-based PNN improves from 67.0% to 95.0% without a thermoelectric controller, while the approach reports a notable 4-bit improvement and tenfold energy reduction (Xu et al., 2024). On the DYNAP-SE2 mixed-signal processor, gradient-based offline training with DynapSim, mismatch noise injection, and an unsupervised 4-bit restricted connection-specific weight quantization procedure preserves task-level discrimination after deployment, despite coarse parameterization and fabrication mismatch (Çakal et al., 2023). Beyond accelerator-style inference hardware, hardware-aware training has also been extended to federated learning: for speech emotion recognition on session-partitioned IEMOCAP, a hardware-aware FL framework using client profiling, top-$$\mathbf{z}=\mathbf{W}\mathbf{x}, \qquad \mathbf{y}=\phi(\mathbf{z}),$$0 selection, adaptive local epochs, and hardware-weighted aggregation reaches 0.352 validation accuracy, reduces total training time by about 36.5% compared to FedAvg, and lowers cumulative communication cost by 40% (Yuksel et al., 23 May 2026).

6. Limits, misconceptions, and open directions

A central misconception is that any hardware distortion can be compensated by training. A controlled diagnostic study of six perturbation classes—read noise, variability, drift, stuck-at faults, IR-drop, and ADC discretization—identifies three diagnostics for compatibility with gradient-based HAT: gradient expectation consistency, bounded gradient variance, and non-degenerate sensitivity (Fang et al., 10 May 2026). The reported outcome is a clear separation between perturbations that can be compensated by HAT and those that consistently break optimization (Fang et al., 10 May 2026). This directly limits claims about HAT as a universal remedy and reassigns some problems to circuit-, architecture-, or calibration-level mitigation.

A second misconception is that maximal hardware specificity is always desirable. On MTJ arrays, die-specific defect-aware training recovers performance but does so “at the cost of generality,” which motivates statistics-aware training over measured defect distributions instead of per-die retraining (Borders et al., 2023). In full-integer rescaling, the fine-tuned model reaches maximum accuracy only for the specific rescaler bit-width or hardware implementation used during fine-tuning, underscoring that some HAT procedures are strongly target-specific even when their implementation recipe is generic (Mueller et al., 13 Oct 2025).

A third misconception is that hardware adaptation must be concentrated inside the training loop. Post-training optimization can sometimes remove part of the burden from HAT. For PCM AIMC inference on RoBERTa-base across all GLUE tasks, per-column conductance range optimization and per-tile input-range optimization reduce the complexity during training and the amount of information about the underlying hardware with no notable accuracy change ($$\mathbf{z}=\mathbf{W}\mathbf{x}, \qquad \mathbf{y}=\phi(\mathbf{z}),$$1) relative to learning those parameters during HWA finetuning, and further PT optimization of learned parameters improves accuracy (Lammie et al., 2024). This suggests a layered workflow in which training-time robustness, post-training calibration, and hardware-specific search are complementary rather than mutually exclusive.

Taken together, the literature portrays hardware-aware training as a spectrum of co-design strategies rather than a single doctrine. Some methods inject realistic hardware forward models; some enforce exact arithmetic semantics; some optimize routing, search spaces, or delay structures; some shift hardware adaptation to post-training calibration. The stable conclusion is narrower but stronger: when the deployment operator differs materially from the ideal software operator, training that explicitly models or constrains that difference can convert hardware mismatch from a post hoc failure mode into an optimization variable (Rasch et al., 2023).