Hardware-Aware NAS

• Hardware-aware neural architecture search is a technique that jointly optimizes network accuracy with hardware metrics such as latency, energy, and memory consumption.
• Methods include differentiable NAS, evolutionary algorithms, and surrogate modeling, which balance multiple objectives to meet specific device constraints.
• Notable implementations like HANNA and MicroNAS demonstrate significant improvements in inference speed and energy efficiency while reducing overall search costs.

Hardware-aware neural architecture search (HW-NAS) denotes neural architecture search in which task performance is optimized together with hardware metrics such as execution latency, energy consumption, memory footprint, model size, and, in custom-hardware settings, area or hardware configuration variables. In the 2021 survey of the field, HW-NAS is organized along four key dimensions—search space, search strategy, acceleration technique, and hardware cost estimation strategy—and is motivated by the observation that architectures synthesized without hardware constraints are often too complex for IoT, mobile, and embedded deployment (Benmeziane et al., 2021). A representative early formulation extended FBNet-style differentiable NAS by adding an explicit energy term to an accuracy-and-latency loss, producing a Raspberry-Pi-specific child architecture named HANNA and making explicit the multi-objective, device-specific nature of the problem (Srinivas et al., 2019).

1. Optimization objectives and formal problem statements

At its most general, HW-NAS is formulated either as constrained optimization or as multi-objective optimization. The survey formalizes the constrained case as

$$\max_{\alpha \in A} f(\alpha,\delta) \quad \text{s.t.}\quad g_i(\alpha)\leq T_i,\ \forall i\in I,$$

where $g_i$ denotes a hardware metric and $T_i$ its threshold, and the multi-objective case as the simultaneous optimization of accuracy, latency, memory, energy, or related metrics under Pareto optimality rather than scalar accuracy alone (Benmeziane et al., 2021). This distinction is central: many HW-NAS systems either search directly for a Pareto front or scalarize several objectives into one differentiable or evolutionary fitness.

The HANNA formulation exemplifies scalarized differentiable HW-NAS. For architecture $a$ with weights $w_a$, the loss is

$$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$

and the NAS objective over supernet parameters $\theta$ is

$$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$

The latency and energy terms are differentiable because they are written as sums of block-level contributions weighted by Gumbel-softmax masks, and the coefficients $(\alpha,\beta,\gamma,\delta)$ act as application-specific “knobs” for latency-dominant, energy-dominant, or balanced search (Srinivas et al., 2019).

Later systems broadened the objective set. MicroNAS introduces latency and peak-memory penalties that are activated only when target constraints are violated, allowing differentiable search for time-series classification on microcontrollers under user-defined latency and memory limits (King et al., 2023). HW-NAS for early exiting networks formulates a joint optimization of average accuracy and average energy-delay cost, with explicit constraints on exit overhead and final-exit ratio, thereby making quantization and exit placement part of the hardware-aware design problem rather than post hoc deployment choices (Zniber et al., 4 Dec 2025). For co-exploration of network and accelerator design, the optimization target can include both test accuracy and hardware utilization or throughput, again interpreted through a Pareto frontier rather than a single optimum (Jiang et al., 2019).

2. Search spaces: from mobile CNN blocks to heterogeneous transformer layers

Search-space design in HW-NAS is not merely combinatorial bookkeeping; it is a primary mechanism for encoding what hardware efficiency can mean on a given platform. The survey distinguishes global or layer-wise spaces, cell-based spaces, and hierarchical spaces, and also separates architecture search space from hardware search space for accelerators such as FPGAs and ASICs (Benmeziane et al., 2021). In practice, HW-NAS systems often bias or restrict the search space toward hardware-friendly operators before search begins.

In the HANNA system, the macro-architecture is fixed at 24 layers, of which 22 are “To Be Searched” layers, and each searchable layer can instantiate one of 9 candidate MobileNetV2-inspired blocks, including an identity option. This yields a search space of $9^{22}$ architectures, while keeping latency and energy profiling manageable because every block-layer pair can be pre-measured on the Raspberry Pi 3B (Srinivas et al., 2019). HURRICANE expands the operator vocabulary substantially further: it builds a hardware-specific operator pool containing up to 32 unique operators, including SEP, MobileNetV2 bottlenecks, ShuffleNetV2 blocks, and SE variants, then selects the top-$g_i$0 operators for each layer based on real hardware profiling. The resulting per-device space is approximately $g_i$1 architectures over 20 learnable layers (Zhang et al., 2019).

Cell-based regularization becomes especially important in resource-constrained regimes. MicroNAS defines two specialized cell types—Time-Reduce Cells and Sensor-Fusion Cells—and augments them with dynamic convolutions so that channel counts themselves become searchable under MCU latency and memory constraints (King et al., 2023). For ultra-low-power microcontrollers, a much more constrained family has been used: a first convolution with $g_i$2 kernels, followed by $g_i$3 repeated cells, with $g_i$4 and $g_i$5 as the only search variables and the number of kernels in later cells determined recursively. The corresponding constrained optimization includes not only RAM, flash, and MACC limits for the deployment device, but also RAM limits for the machine performing NAS itself (Garavagno et al., 15 Jun 2026). A related embedded-device formulation imposes an explicit host-side RAM constraint $g_i$6, enabling NAS execution on devices under 512MB of RAM (Garavagno et al., 12 Jun 2026).

The search-space question has also moved beyond CNNs. LLMForge introduces Infinite-Head Attention, which decouples the number of query heads, the number of KV groups, and the per-head query/key and value dimensions. Within its stated search-space ranges, this expands the feasible per-layer attention configuration space by approximately $g_i$7 over grouped-query attention, producing hardware-conditioned transformer architectures that differ across GPUs, systolic accelerators, and ring-dataflow substrates (Jiang et al., 17 May 2026). At the tooling level, elasticAI.explorer provides a YAML-based specification that supports layer-wise, cell-based, and hierarchical search spaces through a unified interface, with dynamic translation into executable models during sampling (Maman et al., 28 May 2026).

3. Hardware cost estimation: measurement, lookup tables, analytical models, and learned surrogates

Hardware cost estimation is the defining technical bottleneck of HW-NAS. The survey identifies four principal strategies: real-time measurements on target hardware, lookup tables, analytical estimation, and learned prediction models (Benmeziane et al., 2021). The choice among them affects both fidelity and scalability, and much of the field’s methodological development can be read as an attempt to improve this trade-off.

The Raspberry Pi HANNA workflow is a canonical lookup-table system with direct measurement. Latency for each candidate block at each layer is measured on the Raspberry Pi 3B using actual execution time; energy is measured as block execution time multiplied by active current, with current acquired via a PowerJive stick at fixed 5.1V and idle current subtracted. Both latency and energy are then stored as per-block, per-layer lookup tables and summed during differentiable search (Srinivas et al., 2019). On Intel Movidius VPU, two hardware-cost mechanisms were introduced: direct pre-collected device latency for operators, and VPUNN, a device-specific MLP latency model trained on empirical VPU measurements. The expected block latency during search is expressed as a probability-weighted sum over operator costs, integrating VPU latency directly into a ProxylessNAS-style objective (Xu et al., 2023).

Analytical modeling replaces repeated on-device measurement with parameterized latency formulas. HSCoNAS models total runtime as the sum of operator latencies plus a communication-overhead bias term,

$g_i$8

with $g_i$9 calibrated from measured architectures. The reported estimation errors are RMSE $T_i$0 ms on CPU, $T_i$1 ms on GPU, and $T_i$2 ms on Edge hardware, and the method is used to support dynamic channel scaling and progressive space shrinking during search (Luo et al., 2021). EH-DNAS pushes this idea further by learning a differentiable end-to-end hardware performance model from measured architectures, rather than relying on linear layer-wise aggregation. For customized accelerators, its E2E-Perf benchmarking tool reports estimation error below $T_i$3 for pipeline paradigms and below $T_i$4 in the worst generic-paradigm case (Jiang et al., 2021).

Recent work has concentrated specifically on surrogate-model quality. The ESM framework studies dataset generation, sampling policy, encoding choice, and training strategy for hardware-aware latency predictors on GPU-powered devices. It compares one-hot, statistical, feature-based, and proposed Feature Combination Count encodings, and reports, for ResNet on RTX4090, $T_i$5 accuracy with FCC encoding using $T_i$6k samples and $T_i$7 with $T_i$8k samples, versus $T_i$9 for lookup with bias correction. It also reports that balanced sampling reaches target convergence in $a$0 samples versus $a$1 for random sampling, after profiling each architecture over $a$2 runs and discarding the top and bottom $a$3 outliers under a variance threshold below $a$4 (Nasir et al., 2 Aug 2025).

These results reinforce a persistent HW-NAS lesson: hardware proxies such as FLOPs or parameter count are often insufficient. HSCoNAS explicitly states that PARAMs and FLOPs are hardware-agnostic and do not correlate well with real latency, and MicroNAS quantifies this on MCUs by comparing a latency lookup-table estimator with MAE around $a$5 ms against FLOPs-based proxies above $a$6 ms (Luo et al., 2021, King et al., 2023).

4. Search algorithms and the shift from fixed-hardware NAS to co-design

HW-NAS has adopted the full range of NAS algorithms—reinforcement learning, evolutionary algorithms, gradient-based supernet methods, Bayesian optimization, and random search—but hardware awareness changes the role each method plays. The survey notes that weight-sharing supernets, early stopping, proxy datasets, and performance predictors are common acceleration mechanisms precisely because direct hardware-aware evaluation is expensive (Benmeziane et al., 2021).

Differentiable search remains important where block-level cost is compositional. HANNA inherits FBNet-style DNAS and Gumbel-softmax relaxation to optimize accuracy, latency, and energy jointly over a stochastic supernet (Srinivas et al., 2019). MicroNAS uses alternating optimization of architecture parameters and weights under latency and memory penalties, while EH-DNAS replaces linear hardware surrogates with a learned differentiable end-to-end model so that gradient-based search can target complex hardware platforms, including Edge GPUs, Edge TPUs, Mobile CPUs, and customized accelerators (King et al., 2023, Jiang et al., 2021).

Evolutionary search dominates in settings where objectives are highly non-linear or multi-objective. HSCoNAS combines analytical latency prediction with dynamic channel scaling, progressive space shrinking, and an evolutionary algorithm, searching a space of approximately $a$7 architectures (Luo et al., 2021). SONATA augments NSGA-II with tree-based surrogate models for parameter importance and an RL agent that adapts mutation and crossover probabilities; on ImageNet-1k across several search spaces and NVIDIA edge GPUs, it reports up to $a$8 improvement in accuracy, up to $a$9 gains in latency and energy, and up to approximately $w_a$0 Pareto dominance over native NSGA-II (Bouzidi et al., 2024). MO-HDNAS adds a third objective—hardware cost diversity—to the usual pair of performance proxy and hardware cost, arguing that naive two-objective search under-explores the high-cost, high-performance region. It reports a $w_a$1 lower search cost than repeated single-objective runs while producing a trade-off set in a single run across six edge devices (Sinha et al., 2024).

Reinforcement learning persists primarily where architecture generation and hardware decisions are tightly coupled. Hardware/software co-exploration samples architectures with an RNN controller, uses a fast exploration stage to prune hardware-inefficient candidates without lengthy training, then performs a slow exploration stage with validation accuracy and hardware utilization rewards. On ImageNet, it reports the same accuracy with $w_a$2 higher throughput, $w_a$3 higher energy efficiency, and $w_a$4 reduced search time compared with state-of-the-art hardware-aware NAS that assumes fixed hardware (Jiang et al., 2019). This is a substantive conceptual move: the hardware design itself becomes part of the search space, rather than a fixed constraint boundary.

Several methods explicitly target scalability across diverse devices. “One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search” formalizes latency monotonicity between devices,

$w_a$5

and argues that when Spearman’s rank correlation is strong, one proxy device can replace per-device NAS. When monotonicity is weak, a sparse adaptation model is fitted from a small number of target-device samples, reducing total latency-evaluation cost from $w_a$6 over $w_a$7 devices to effectively $w_a$8 (Lu et al., 2021). HURRICANE addresses a related hardware-diversity problem through a two-stage one-shot search over later and earlier layers, reducing training time by $w_a$9 compared with SPOS (Zhang et al., 2019). HW-EvRSNAS attacks search cost even more aggressively by replacing full training with Representation Mutual Information computed from a single training batch and reports speedups up to $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$0 over MnasNet, $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$1 over ProxylessNAS, and $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$2 over OFA, together with lower CO$$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$3 emissions (Sinha et al., 2023).

5. Representative deployment regimes and empirical outcomes

The empirical literature on HW-NAS is unusually heterogeneous because “hardware-aware” means different things on Raspberry Pi boards, VPUs, mobile CPUs, FPGAs, microcontrollers, and edge accelerators for LLMs. Nevertheless, several benchmarked results have become reference points for the field.

On Raspberry Pi 3B, HANNA was benchmarked against MobileNetv2 and CondenseNet on CIFAR-10. HANNA reached $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$4 accuracy, $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$5 s latency, and $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$6 J energy, compared with $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$7, $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$8 s, and $$L(a, w_a) = CE(a, w_a) + \alpha \cdot LAT(a)^\beta + \gamma \cdot ENER(a)^\delta,$$9 J for MobileNetv2 and $\theta$0, $\theta$1 s, and $\theta$2 J for CondenseNet. The reported effect is about $\theta$3 and $\theta$4 speedup, and $\theta$5 and $\theta$6 lower energy consumption, respectively, at the cost of a $\theta$7–$\theta$8 drop in accuracy (Srinivas et al., 2019).

On programmable hardware, co-exploration shows that relaxing the fixed-hardware assumption can materially shift the Pareto frontier. On ImageNet, the co-exploration framework reports a hardware-aware NAS baseline at $\theta$9 top-1 accuracy, $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$0 FPS, and $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$1 GOPS/W, while its OptSW configuration reaches $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$2, $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$3 FPS, and $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$4 GOPS/W and its OptHW configuration reaches $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$5, $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$6 FPS, and $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$7 GOPS/W (Jiang et al., 2019). HSCoNAS, evaluated on ImageNet under platform-specific latency constraints, reports models such as HSCoNet-GPU-A at $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$8 top-1 error and $$\min_\theta \mathbb{E}_{a \sim P(\theta)}[L(a, w_a)].$$9 ms GPU latency, and HSCoNet-CPU-B at $(\alpha,\beta,\gamma,\delta)$0 top-1 error, $(\alpha,\beta,\gamma,\delta)$1 top-5 error, and $(\alpha,\beta,\gamma,\delta)$2 ms CPU latency, outperforming several manually designed and NAS baselines on GPU, CPU, and edge targets (Luo et al., 2021).

For VPU deployment, NAS has been explicitly adapted to Intel Movidius. On ImageNet classification on KeemBay, the searched model achieves $(\alpha,\beta,\gamma,\delta)$3 fps at $(\alpha,\beta,\gamma,\delta)$4 top-1 accuracy versus MobileNet-v2-1.4 at $(\alpha,\beta,\gamma,\delta)$5 fps with the same $(\alpha,\beta,\gamma,\delta)$6 accuracy, and $(\alpha,\beta,\gamma,\delta)$7 fps at $(\alpha,\beta,\gamma,\delta)$8 top-1 versus ResNet50 at $(\alpha,\beta,\gamma,\delta)$9 fps and $9^{22}$0 top-1. For super-resolution, the searched EDSR variant reaches PSNR $9^{22}$1 with $9^{22}$2 higher fps than EDSR3 at PSNR $9^{22}$3 (Xu et al., 2023).

Microcontroller deployment has generated a distinct branch of HW-NAS. MicroNAS targets time-series classification on Nucleo-F446RE and Nucleo-L552ZE-Q and reports MCU-tailored architectures with quantized F1-scores around $9^{22}$4–$9^{22}$5 and memory footprints between approximately $9^{22}$6K and $9^{22}$7K bytes, while adhering to latency and memory constraints more effectively than DARTS (King et al., 2023). For ultra-low-power vision, an affordable HW-NAS for MCUs with $9^{22}$8–$9^{22}$9 kB RAM reports tiny models under $g_i$00 kB RAM, under $g_i$01 kB flash, and under $g_i$02 million MAC/call, with VWW accuracy up to $g_i$03, CIFAR-10 accuracy up to $g_i$04, melanoma accuracy up to $g_i$05, and search cost typically $g_i$06–$g_i$07 minutes, median approximately $g_i$08 hours, on CPU-class devices (Garavagno et al., 15 Jun 2026). A closely related method constraining both the deployment MCU and the machine running NAS reports VWW results on several embedded devices, including $g_i$09 accuracy, $g_i$10 KB RAM, $g_i$11 KB flash, and $g_i$12M MACC when the search is run on Raspberry Pi 4, and $g_i$13 accuracy, $g_i$14 KB RAM, $g_i$15 KB flash, and $g_i$16M MACC on Raspberry Pi Zero 2 W (Garavagno et al., 12 Jun 2026).

HW-NAS has also extended beyond CNN vision models. A hardware-aware framework across modalities pairs evolutionary algorithms with lightly trained objective predictors to search subnetworks from a once-for-all or one-shot super-network for image classification and machine translation under multiple objectives such as accuracy, latency, and memory (Cummings et al., 2022). LLMForge transfers the paradigm to sub-billion-parameter transformers for edge deployment: on a multi-chip ring substrate, its $g_i$17M-parameter accuracy-optimized model reaches validation loss $g_i$18, its $g_i$19M-parameter energy-optimized model lowers energy per token by $g_i$20, and its $g_i$21M-parameter latency-optimized model reduces TTFT and TPOT by $g_i$22, all relative to matched baselines such as SmolLM2-360M and Qwen-0.5B (Jiang et al., 17 May 2026). PlatformX, targeting mobile platforms with an energy-driven search space and transferable kernel-level energy predictor, reports models with up to $g_i$23 accuracy or as little as $g_i$24 mJ per inference, both outperforming MobileNet-V2 in accuracy and efficiency (Tu et al., 10 Oct 2025).

6. Misconceptions, unresolved issues, and current directions

A recurring misconception in HW-NAS is that hardware awareness can be reduced to FLOPs or parameter count. Multiple studies explicitly reject this. HSCoNAS states that PARAMs and FLOPs are hardware-agnostic and do not correlate well with real latency; MicroNAS reports an MCU latency-estimation MAE around $g_i$25 ms for empirical lookup tables versus above $g_i$26 ms for FLOPs-based proxies; HURRICANE constructs per-hardware operator pools precisely because operator rankings differ across DSP, VPU, and CPU targets (Luo et al., 2021, King et al., 2023, Zhang et al., 2019). The practical consequence is that HW-NAS remains tightly coupled to profiling fidelity, whether through lookup tables, analytical calibration, or learned surrogates.

A second misconception is that “hardware-aware” necessarily means optimizing for a fixed device. The co-exploration literature shows that this is only one regime. Joint search over neural architectures and hardware configurations can produce architecture-hardware pairs that surpass the Pareto frontier available to fixed-hardware NAS, especially on programmable substrates such as FPGAs (Jiang et al., 2019). Conversely, the proxy-device literature argues that per-device search is often unnecessarily expensive because architecture latency rankings can be strongly correlated across devices and adapted with sparse target measurements (Lu et al., 2021). These two lines point in different operational directions—co-design for customizable hardware, transfer for hardware fleets—but both challenge the fixed-device assumption.

Search cost has become a first-class concern. HW-EvRSNAS reports up to $g_i$27 search speedups and lower CO$g_i$28 emissions by replacing training-heavy evaluation with representation similarity (Sinha et al., 2023). MO-HDNAS reports a $g_i$29 reduction in search cost relative to repeated single-objective runs by obtaining a full trade-off set in one multi-objective run (Sinha et al., 2024). SONATA and ESM both treat information gathered during search as reusable structure—either for self-adaptive variation operators or for better hardware surrogates—rather than as disposable evaluation traces (Bouzidi et al., 2024, Nasir et al., 2 Aug 2025). This suggests that future HW-NAS systems may increasingly optimize not only architecture quality but also search-time energy, calibration burden, and measurement reuse.

Finally, engineering and reproducibility remain central unresolved issues. The survey highlights benchmarking difficulty, lack of standardization, transferability problems across tasks and hardware, and the concentration of the literature on image classification (Benmeziane et al., 2021). Recent frameworks respond by expanding end-to-end support rather than only proposing new search algorithms. elasticAI.explorer unifies YAML-based search-space definition, dynamic model instantiation, hardware-specific code generation, Docker-based cross-compilation, and hardware-in-the-loop benchmarking binaries for heterogeneous platforms (Maman et al., 28 May 2026). Embedded-device NAS under 512MB RAM emphasizes privacy-preserving on-device search, arguing that a gateway can tailor CNN architectures to acquired data without external servers (Garavagno et al., 12 Jun 2026). Taken together, these developments indicate that HW-NAS is evolving from a narrow NAS subtopic into a broader systems problem involving measurement infrastructure, deployment pipelines, and portability across both hardware and application modalities.