PhaseNAS: Dynamic NAS Methods

• PhaseNAS is a hybrid NAS framework combining differentiable skip-connection search and LLM-driven dynamic phase adaptation to explore complex design spaces.
• Its differentiable approach optimizes sparse connectivity in phase imaging networks by using continuous mask variables, achieving state-of-the-art PSNR and reduced latency.
• The LLM-based variant toggles between exploration and refinement phases to generate efficient architectures for classification and detection benchmarks.

PhaseNAS denotes two distinct but related methodologies for Neural Architecture Search (NAS) that leverage either differentiable relaxation of skip connections (as in off-axis Quantitative Phase Imaging) or LLM-driven architecture generation with dynamic phase adaptation. Both aim to efficiently explore complex network design spaces and produce models with superior accuracy-resource trade-offs across vision tasks.

1. Motivation for Dynamic Neural Architecture Search

Traditional NAS methods—encompassing evolutionary algorithms, reinforcement learning, or purely gradient-based search—face scalability constraints as the size of the architecture space increases. These limitations are pronounced in high-dimensional design spaces prevalent in tasks such as phase retrieval for microscopy or object detection, where additional constraints on latency, parameter count, and generalization further complicate manual design.

Recent LLM-based NAS approaches enable rapid prototyping driven by large pre-trained models. Nonetheless, they often suffer from static resource allocation (fixed LLM size regardless of search stage) and ambiguous, free-form architectural descriptions that degrade generation reliability and evaluation accuracy. Complex tasks further demand adaptive mechanisms to manage the exploration–exploitation balance, motivating frameworks that match computational resources and descriptive precision to the search phase (Kong et al., 28 Jul 2025).

2. PhaseNAS-PRNet: Differentiable Skip-Connection Search

In NAS-PRNet, also denoted PhaseNAS within off-axis Quantitative Phase Imaging, the search space consists of an $L$-stage encoder (based on MobileNet-v2) paired with a symmetric decoder. Each encoder feature $E_i$ and a pooled ground feature $G$ may connect to all decoder stages $D_j$. A continuous mask variable $\alpha_{ij} \in [0, 1]$ modulates these connections; during search, the forward computation for decoder stage $j$ is

$$T_j = \sum_{i=1}^L \alpha_{ij} \cdot \text{Conv}_{3\times 3}(\text{resize}(E_i)) + \alpha_{0j} \cdot \text{Conv}_{3\times 3}(\text{resize}(G))$$

This continuous relaxation enables joint optimization of standard network weights $W$ and mask parameters $\alpha$ via gradient descent. The objective

$$\min_{W,\alpha} L_{\text{recon}}(W,\alpha) + \lambda \|\alpha\|_1 \quad\text{subject to}\quad 0 \le \alpha_{ij} \le 1$$

encourages a sparse topology, with the $E_i$0 penalty on $E_i$1 yielding efficient architectures. The loss $E_i$2 consists of mean squared error between predicted and ground truth phases plus an unwrapping penalty for $E_i$3 discontinuities. Training involves initial pretraining on $E_i$4, followed by sparsity-induced fine-tuning and final pruning of connections with $E_i$5 (Shu et al., 2022).

3. PhaseNAS: LLM-Driven Architecture Search with Dynamic Phase Adaptation

Distinct from the differentiable approach, the LLM-based PhaseNAS introduces a phase-aware controller that toggles between a small “exploration” LLM ($E_i$6) for rapid, low-cost architecture sampling and a large “refinement” LLM ($E_i$7) for high-precision optimization. Search proceeds in dynamic phases:

• Exploration: $E_i$8 samples architectures conforming to a structured template language;
• Phase Transition: When top-K candidates’ expected score $E_i$9 exceeds threshold $G$0, control shifts to refinement;
• Refinement: $G$1 receives prompts to optimize the best-seen architecture;
• Termination: Search halts when a candidate achieves $G$2.

Candidate architectures are described using an unambiguous grammar (e.g., ConvK3BNRELU(3,8,1,1), ResK3K3(16,32,2,1), SCDown(), PSA()), avoiding failures attributed to ambiguous or free-form representations. Scoring functions, such as Zen-Score for classification or its detection extension, provide fast, reliable proxies for downstream performance (Kong et al., 28 Jul 2025).

4. Quantitative Outcomes and Resource Efficiency

Empirical benchmarks highlight the effectiveness of PhaseNAS methodologies:

PhaseNAS (LLM-driven) (Kong et al., 28 Jul 2025):

• NAS-Bench-Macro: With 32B→72B Qwen2.5 LLMs, achieves 93.11% top-1 accuracy (rank 3), surpassing GUNIUS (best 92.75%, rank 110).
• CIFAR-10/100: Matches or exceeds top accuracy (up to 97.34% on CIFAR-10, 81.36% on CIFAR-100) while reducing search time by up to 86% versus Zen-NAS.
• Object Detection (COCO, YOLOv8*): Discovers variants with higher mAP (46.1 YOLOv8s*) and lower parameter/FLOP counts than hand-designed baselines.

PhaseNAS-PRNet (Differentiable, QPI) (Shu et al., 2022):

• Achieves 36.1 dB PSNR, outperforming U-Net (+1.4 dB, 12× lower latency) and SparseMask (improved accuracy with similar latency) on NIH/3T3 cells.
• Generalizes across fringe patterns, attaining 42.9 dB on WBC interferograms with direct topology transfer.

Model	PSNR (QPI)	Params (M)	Latency (ms)	mAP@50:95 (YOLOv8s*)
U-Net	34.7	37.7	373	—
SparseMask	32.5	4.4	28	—
PhaseNAS	36.1	5.0	31	46.1

The table juxtaposes QPI and detection performance, illustrating the method’s cross-domain efficiency.

5. Structural Features and Scoring Mechanisms

In the LLM-based approach, the transition logic is formally specified:

• Transition: Switch to refinement when $G$3.
• Termination: Conclude search when $G$4.
• Pool management: Maintain a top-K pool, replacing the lowest-scoring candidate as new architectures are proposed.

The template language tokens encode all structural hyperparameters, eliminating inconsistencies and mapping directly to standard frameworks (PyTorch, YOLOv8).

In the differentiable NAS-PRNet, skip-connection masks $G$5 and intra-decoder links $G$6 are discovered automatically, resulting in 61% sparsity—only 39 of 100 possible connections are retained after pruning. Empirically, this yields architectures that tightly fuse low-level spatial detail with deep semantic context.

6. Generalizability and Applicability

PhaseNAS demonstrates robust generalization across domains. In quantitative phase imaging, architectures discovered on one fringe pattern (cell type, interferogram orientation) can be retrained on data with differing properties without altering connectivity. In vision benchmarks, the framework discovers efficient architectures for both classification and detection, validated across CIFAR and COCO.

This suggests PhaseNAS methodologies—whether gradient-based with continuous sparsity relaxation or LLM-guided with dynamic phase adaptation—offer practical and scalable solutions for diverse NAS scenarios, balancing exploration, descriptive rigor, and computational resource efficiency (Shu et al., 2022, Kong et al., 28 Jul 2025).