PEL-NAS: Training-Free Hardware-Aware NAS

• The paper introduces PEL-NAS, a training-free NAS framework that jointly optimizes accuracy and latency under device constraints using complexity-based niche partitioning and LLM-guided co-evolution.
• It employs a zero-cost predictor and hardware lookup to rapidly estimate performance, achieving up to 54% lower latency and superior Pareto front metrics compared to baselines.
• Experiments on CNNs and Vision Transformers demonstrate that PEL-NAS efficiently discovers high-quality Pareto-optimal models in minutes rather than days.

Searching arXiv for the target paper and a few named baseline methods to ground citations. PEL-NAS is a training-free, hardware-aware multi-objective neural architecture search framework that discovers Pareto-optimal CNNs and Vision Transformers by co-evolving prompt heuristics and architectures within complexity-based niches. It is formulated to simultaneously optimize top-1 accuracy $z(A)$ and inference latency $\ell(A)$ under real device constraints, while avoiding both the days-long supernet training of traditional one-shot HW-NAS and the exploration bias observed in naïve LLM-driven NAS. The method is introduced as “search space Partitioned, architecture prompt co-Evolutionary and LLM-driven Neural Architecture Search,” with three core components: a complexity-driven partitioning engine, an LLM-powered architecture prompt co-evolution operator, and a zero-cost predictor for rapid fitness estimation (Zhu et al., 1 Oct 2025).

1. Motivation and optimization setting

PEL-NAS is situated in hardware-aware neural architecture search, where the objective is not merely to maximize predictive performance but to jointly optimize predictive quality and deployment efficiency on concrete devices. In the formulation reported for PEL-NAS, HW-NAS is posed as a bi-objective optimization problem over architectures $A$ in a large search space $\mathcal{S}$:

• maximize $z(A)$, the accuracy,
• minimize $\ell(A)$, the latency,

subject to device constraints such as $\ell(A) \leq L_{\max}$ (Zhu et al., 1 Oct 2025).

The motivating contrast is between two existing paradigms. Traditional one-shot methods such as OFA or FairNAS amortize training through a supernet, but require 2–10 GPU-days per dataset. Recent LLM-driven NAS methods avoid training a large supernet and can provide quick feedback, but PEL-NAS identifies an exploration bias: the LLM repeatedly generates “safe,” familiar patterns and leaves large gaps in latency–accuracy space. Static prompts are described as failing to accumulate knowledge from previous successes or failures, so each generation remains uninformed by prior outcomes, which stalls progress toward the true Pareto front (Zhu et al., 1 Oct 2025).

Within that framing, PEL-NAS is presented as a mechanism for restoring global exploration without reverting to expensive supernet training. Its stated goal is to generate neural networks with high accuracy and low latency with reduced search cost, and the reported empirical setting explicitly targets real device constraints rather than proxy efficiency objectives alone (Zhu et al., 1 Oct 2025).

2. Complexity-driven partitioning of the search space

The first defining component of PEL-NAS is a complexity-driven partitioning engine that divides the search space into disjoint niches. The purpose is to enforce diversity and mitigate the exploration bias or “mode collapse” observed in naïve LLM-driven NAS. Rather than clustering architectures abstractly, the partitioning is based on a tangible complexity metric: the count of parameter-heavy operators, exemplified by $3 \times 3$ convolutions (Zhu et al., 1 Oct 2025).

On HW-NAS-Bench, the reported empirical observation is that the total parameter count—and thus latency—grows step-wise with the number of nor_conv_3×3 blocks. PEL-NAS therefore partitions $\mathcal{S}$ into $K$ disjoint niches $\ell(A)$0, including the explicitly stated examples:

• $\ell(A)$1
• $\ell(A)$2
• $\ell(A)$3

These niches span distinct complexity bands from ultra-lightweight to highly complex architectures (Zhu et al., 1 Oct 2025).

The significance of this design is twofold. First, the partitioning makes diversity an explicit structural constraint rather than a hoped-for consequence of stochastic generation. Second, it aligns the niches with a latency-relevant property, so exploration is distributed across different hardware cost regimes. The paper states that this niching strategy mirrors classic multi-objective evolutionary algorithms such as NSGA-II, but is driven by architecture complexity rather than abstract clustering (Zhu et al., 1 Oct 2025). This suggests that PEL-NAS treats diversity not as a purely population-level property, but as a search-space design principle tied directly to deployment cost.

3. LLM-powered prompt co-evolutionary operator

Within each niche $\ell(A)$4, PEL-NAS assigns the LLM two roles in each generation $\ell(A)$5: knowledge base update and rationale-driven generation. The reported LLM is GPT-4.1, and the operator is described as co-evolutionary because prompts and architectures improve together across rounds (Zhu et al., 1 Oct 2025).

For the knowledge base update, the input consists of the top Pareto parents

$\ell(A)$6

from the previous generation together with their design rationales $\ell(A)$7. The task is to summarize rules into a Co-evolve Knowledge Base $\ell(A)$8. The paper gives representative heuristic summaries such as:

• “Use skip_connect after heavy conv to cut latency.”
• “Avoid avg_pool_3×3 after nor_conv_3×3—it adds latency with little accuracy.” (Zhu et al., 1 Oct 2025)

For rationale-driven generation, the prompt includes the device and dataset, the niche constraint, the current $\ell(A)$9, and $A$0. The LLM then applies crossover and mutation. In the reported crossover example, $A$1 merges blocks by inheriting skip_connect from one parent and pooling from another, while justifying how the accuracy–latency trade-off shifts. In the reported mutation example, $A$2 makes a single change such as replacing avg_pool_3×3 \to skip_connect. Each proposed child $A$3 is accompanied by a rationale $A$4 (Zhu et al., 1 Oct 2025).

The technical point of departure from static prompting is that heuristic information is accumulated explicitly and fed back into subsequent generations. The paper states that this avoids random guesswork and improves efficiency. A plausible implication is that PEL-NAS uses the LLM not simply as a generator of candidate architectures, but as an evolving repository of design regularities extracted from previous Pareto-optimal samples.

4. Search loop, zero-cost prediction, and computational profile

The optimization loop is organized in three phases. Phase 1 initializes a zero-cost predictor $A$5 and seeds each niche population $A$6 randomly. Phase 2 iterates over generations $A$7 and, in parallel for each niche $A$8, selects parents from $A$9, constructs a prompt from the parents, $\mathcal{S}$0, and niche constraints, and queries the LLM to produce a child architecture and rationale. If the child lies in $\mathcal{S}$1 and is novel, PEL-NAS evaluates it by predicting accuracy with $\mathcal{S}$2 and querying latency via HardwareLookup(A_child), then updates $\mathcal{S}$3 by inserting the child and removing dominated solutions. Phase 3 aggregates the final front as the non-dominated union of all $\mathcal{S}$4 (Zhu et al., 1 Oct 2025).

To avoid costly training, PEL-NAS trains an XGBoost ensemble on the 13 zero-cost proxies from NAS-Bench-Suite-Zero. Given a candidate architecture $\mathcal{S}$5, the predictor outputs $\mathcal{S}$6 as an instant accuracy estimate, while hardware latency $\mathcal{S}$7 is obtained through a lookup table. The reported predictive quality is Spearman $\mathcal{S}$8 against ground truth, with fitness feedback in $\mathcal{S}$9 ms per evaluation (Zhu et al., 1 Oct 2025).

Pareto front quality is evaluated with Hypervolume (HV) and Inverted Generational Distance (IGD). The paper defines HV for a reference point $z(A)$0 dominated by all $z(A)$1 as

$z(A)$2

where larger HV indicates broader and superior coverage. IGD is defined for true front $z(A)$3 and discovered set $z(A)$4 as

$z(A)$5

where smaller IGD indicates closer approximation to the optimum (Zhu et al., 1 Oct 2025).

The reported algorithmic complexity is dominated by $z(A)$6 LLM calls, with 120 API calls in the described setup, plus fast predictor and lookup queries. The complete search is reported to finish in approximately 3 minutes on a single GPU host. This is contrasted with supernet methods requiring $z(A)$7 training, including FairNAS at about 10 GPU-days and OFA at about 2 GPU-days, while LLMatic, described as naïve LLM plus from-scratch training, exceeds 17 GPU-days (Zhu et al., 1 Oct 2025).

5. Empirical evaluation on HW-NAS-Bench and ViT search

The evaluation is conducted on HW-NAS-Bench with CIFAR-10, CIFAR-100, and ImageNet16-120. The reported deployment targets are Edge GPU (Jetson TX2), Raspberry Pi 4, Edge TPU, Pixel 3, ASIC (Eyeriss), and FPGA. Baselines include LLMatic, FairNAS, DARTS, PRP-NAS, and AutoFormer for ViT (Zhu et al., 1 Oct 2025).

The main reported result is that PEL-NAS achieves overall higher HV, lower IGD, and up to 54% lower latency than baselines at similar accuracy. On CIFAR-10 with Edge GPU, the stated figures are:

Setting	PEL-NAS	Baseline comparison
HV	0.997	0.892 (FairNAS)
IGD	0.006	0.073
Fastest model latency	1.78 ms	–22% vs. PRP-NAS-BL
Fastest model accuracy	$z(A)$8	$z(A)$9 competitor
FPGA latency example	1.65 ms	3.60 ms

The FPGA example is explicitly described as an up to 54% latency reduction, with 1.65 ms versus 3.60 ms (Zhu et al., 1 Oct 2025).

Averaged across devices and datasets, Table 2 is summarized as showing that PEL-NAS outperforms all baselines in HV by +5–80% and in IGD by –6–54%. Search cost in Table 3 is summarized as 3 min versus 2–10 GPU-days (Zhu et al., 1 Oct 2025). These results are presented as evidence that the framework can recover a high-quality Pareto front while sharply reducing search overhead.

The method is also extended to Vision Transformers on ImageNet-1k. In that setting, PEL-NAS partitions by Embed Dim and Depth Num, described as dominant FLOP factors $\ell(A)$0 and $\ell(A)$1, co-evolves ViT blocks under latency budgets on an A6000 GPU, and uses a ViT accuracy predictor. The reported outcome is discovery of tiny, small, and base ViTs with competitive accuracy–latency trade-offs, including the example of 82.5% at 5.4 ms versus AutoFormer’s 83.4% at 8.4 ms (Zhu et al., 1 Oct 2025). This suggests that the framework’s partition-and-co-evolve principle is not restricted to CNN cell spaces and can be adapted to transformer design variables when those variables are tied to dominant complexity factors.

6. Ablations, interpretation, and place within NAS research

The ablation results directly support the importance of the three-component design. Removing partitioning is reported to collapse HV to approximately 0.52, with IGD around 0.37. Replacing the zero-cost ensemble with a single proxy degrades HV by approximately 16%. The full PEL-NAS is described as consistently dominating in all configurations (Zhu et al., 1 Oct 2025).

These ablations clarify a potential misconception that the framework’s performance is attributable only to the use of an LLM. In the reported evidence, the LLM is one component within a broader search strategy that also depends on complexity-driven niching and a reliable zero-cost predictor. The paper’s summary states that the method combines:

• complexity-driven partitioning to enforce global diversity,
• co-evolution of LLM prompts and architectures to accumulate design heuristics,
• a fast, zero-cost predictor for multi-objective fitness (Zhu et al., 1 Oct 2025).

A second misconception would be to interpret PEL-NAS as eliminating evaluation entirely. The reported workflow does avoid training a large number of candidates from scratch, but it still evaluates candidates through predicted accuracy and real-device latency lookup, and the overall objective remains explicitly hardware-aware. A plausible implication is that PEL-NAS should be understood as training-free in the search loop rather than evaluation-free in any absolute sense.

Within NAS research, PEL-NAS occupies the intersection of hardware-aware search, multi-objective optimization, and LLM-guided design. Its specific contribution is not merely replacing search heuristics with language generation, but restructuring the search process so that architectural diversity is enforced by partitioning and design heuristics are accumulated generation by generation. In that sense, the framework is presented as a practical paradigm for hardware-aware NAS that seeks a near-complete, high-quality Pareto front in minutes rather than days (Zhu et al., 1 Oct 2025).