Dextr: Zero-Shot NAS Proxy

• Dextr is a zero-shot neural architecture search proxy that evaluates architectures using a single unlabeled sample by jointly assessing convergence, generalization, and expressivity.
• It applies singular value decomposition to measure feature-map conditioning and uses extrinsic curvature to capture the network output's expressivity.
• The method is architecture-agnostic, efficiently supporting CNNs and Transformers, and demonstrates high correlation with test accuracy on various NAS benchmarks.

Searching arXiv for the specified paper and closely related zero-shot NAS work. {"query":"arXiv (Asthana et al., 18 Aug 2025) Dextr Zero-Shot Neural Architecture Search with Singular Value Decomposition and Extrinsic Curvature", "max_results": 5} {"query":"zero-shot neural architecture search label-free extrinsic curvature singular value decomposition arXiv", "max_results": 10} {"query":"arXiv (Maninis et al., 2017) Deep Extreme Cut extreme points object segmentation", "max_results": 5} Dextr is a zero-shot neural architecture search (NAS) proxy that scores candidate architectures at initialization without requiring labeled data, using a joint characterization of convergence, generalization, and expressivity (Asthana et al., 18 Aug 2025). Its central construction combines two signals: a singular-value-based measure of feature-map conditioning, intended to capture channel collinearity and its implications for optimization and generalization, and an extrinsic-curvature measure of the network output, intended to capture expressivity. Within the recent literature on zero-cost proxies, Dextr is distinguished by its explicit label-free operation, its use of a single unlabeled sample, and its applicability to both CNN and Transformer search spaces (Asthana et al., 18 Aug 2025).

1. Position within zero-shot NAS

Zero-shot NAS seeks to rank architectures before full training, typically through zero-cost proxies computed at initialization. In the formulation associated with Dextr, two limitations of earlier proxies are foregrounded: many rely on labeled data, and many emphasize either convergence and generalization or expressivity, but not both simultaneously (Asthana et al., 18 Aug 2025). Dextr is proposed as a response to both limitations.

The method is therefore best understood as a ranking functional over randomly initialized architectures. Rather than estimating eventual performance through training dynamics or surrogate predictors, it computes a score from internal feature statistics and output geometry. This design places Dextr in the family of initialization-time NAS criteria, but with an explicitly label-free regime and an explicitly composite objective.

A central claim of the work is that accurate prediction of network performance on test data can be achieved using only a single label-free data sample (Asthana et al., 18 Aug 2025). This makes the method especially relevant in settings where labels are unavailable or costly, including industrial, proprietary, or privacy-sensitive data regimes.

2. Channel collinearity, conditioning, and the SVD term

The first component of Dextr is based on the singular value decomposition of layer feature maps. The motivating premise is that high multi-collinearity among channels can hamper convergence and generalization, whereas architectures producing more linearly independent features tend to train faster and generalize better (Asthana et al., 18 Aug 2025).

For a feature map matrix $X^{\phi}$, Dextr uses the singular values $\sigma_k$ to define a condition number,

$c(X^{\phi}) = \frac{\sigma_{\max}}{\sigma_{\min}}.$

This quantity is used as a proxy for collinearity or ill-conditioning. Smaller condition numbers, equivalently larger inverse condition numbers, are treated as beneficial. The paper connects this claim to eigenvalue analysis of Gram matrices and to generalization and convergence theorems, stating that smaller condition numbers in feature maps improve both convergence rate and generalization error bounds (Asthana et al., 18 Aug 2025).

Operationally, Dextr aggregates the inverse condition numbers across layers. This aggregated term serves as the convergence-generalization component of the final proxy. The method’s interpretation is not merely heuristic: it is presented as a structural diagnostic of feature redundancy throughout the network depth. The layer-wise analysis reported in the paper further states that high-performing networks exhibit higher feature independence across all layers (Asthana et al., 18 Aug 2025).

This suggests that Dextr treats architectural quality partly as a conditioning problem. In this view, an architecture is favored when its initialized feature transformations avoid pathological redundancy and therefore preserve a more favorable optimization landscape.

3. Expressivity through extrinsic curvature

The second component of Dextr measures expressivity through the extrinsic curvature of the network output manifold induced by an arbitrary circular input trajectory (Asthana et al., 18 Aug 2025). The paper adopts a Riemannian-geometry perspective in which curvature quantifies how strongly the output manifold bends; larger curvature is taken to indicate greater expressivity.

The curvature term is defined as

$$\kappa(\theta) = (v \cdot v)^{-3/2} \sqrt{ (v \cdot v)\,(a \cdot a) - (v \cdot a)^2 },$$

where $v(\theta)=\partial_\theta h(\theta)$ is the velocity or tangent, $a(\theta)=\partial_\theta v(\theta)$ is the acceleration, and $h(\theta)$ is the output along the chosen trajectory (Asthana et al., 18 Aug 2025).

This construction is notable because it does not equate expressivity with crude architectural surrogates such as depth, width, or parameter count. Instead, it attempts to quantify expressivity directly from output geometry. In Dextr, the curvature term therefore complements the SVD-based term: the latter rewards feature independence and favorable conditioning, while the former rewards representational flexibility.

The paper’s broader methodological claim is that an architecture search proxy should not optimize one of these desiderata at the expense of the other. Dextr is designed to balance them in a single score rather than treating them as competing post hoc diagnostics (Asthana et al., 18 Aug 2025).

4. Composite proxy and label-free computation

Dextr combines the conditioning and curvature terms through the following score:

$$\textbf{Dextr} = \frac{ \log \left( 1 + \sum_{l=1}^{L} \frac{1}{c_l(X^\phi)} \right) \cdot \log\big(1 + \kappa(\theta)\big) }{ \log \left( 1 + \sum_{l=1}^{L} \frac{1}{c_l(X^\phi)} \right) + \log\big(1 + \kappa(\theta)\big) }.$$

Here, $L$ is the number of layers, $c_l(X^\phi)$ is the condition number of the feature maps in layer $\sigma_k$0, and $\sigma_k$1 is the output extrinsic curvature (Asthana et al., 18 Aug 2025).

The use of logarithms is motivated in the paper by stability and scaling. The resulting score is intended to reward architectures that are simultaneously well-conditioned and expressive, while down-weighting those that are strong on only one axis. A key practical point is that both terms can be computed without labels. The paper further states that they can be computed from a single unlabeled data sample or even from synthetically constructed label-free input (Asthana et al., 18 Aug 2025).

Dextr is also presented as architecture-agnostic across CNNs and Vision Transformers. For CNNs, feature maps directly support the SVD-based analysis. For ViTs, the paper argues that prior work relating multi-head self-attention with sufficient heads and/or relative positional encoding to convolutional structure permits the same condition-number and curvature-based reasoning (Asthana et al., 18 Aug 2025).

5. Empirical evaluation

The empirical study reported for Dextr spans tabular NAS benchmarks and open search spaces. The evaluation includes NAS-Bench-101, NAS-Bench-201, NAS-Bench-301, TransNAS-Bench-101-micro, and NAS experiments in the DARTS and AutoFormer search spaces (Asthana et al., 18 Aug 2025). Effectiveness is primarily measured by Spearman rank correlation between the proxy and final test accuracy, together with end-to-end NAS outcomes after full training.

Setting	Reported outcome	Significance
NAS-Bench-101 / 201	State-of-the-art correlation coefficients; e.g. $\sigma_k$2 on NAS-Bench-201	Strong ranking fidelity
DARTS / ImageNet	Top-1 / top-5 test errors of 24.6% / 7.4% with 0.07 GPU days	Competitive search quality and efficiency
AutoFormer	Competitive or superior top-1 ImageNet error at equivalent or lower cost	Extension beyond CNNs

The paper reports that Dextr outperforms prior zero-shot methods on multiple correlation benchmarks, including NAS-Bench-101, NAS-Bench-201, and TransNAS-Bench-101-micro, and shows strong performance in both DARTS and AutoFormer (Asthana et al., 18 Aug 2025). It also reports robustness across supervised and self-supervised tasks, and ablations indicating that both the SVD-based and curvature-based components contribute materially to performance.

An additional empirical claim is that Dextr’s values are stable across random unlabeled input samples (Asthana et al., 18 Aug 2025). This is important because a label-free proxy based on a single sample would otherwise risk high variance and poor reproducibility.

6. Interpretation, scope, and nomenclature

Dextr’s main significance lies in its attempt to formalize architecture quality as a three-way balance among convergence, generalization, and expressivity, rather than reducing zero-shot NAS to a single optimization proxy (Asthana et al., 18 Aug 2025). In practical terms, this supports rapid architecture selection with minimal data assumptions and low computational overhead.

A plausible implication is that Dextr is particularly suited to early-stage architecture screening, where full supervision is unavailable and one seeks a proxy that is inexpensive yet not purely heuristic. Its reliance on forward-pass-accessible quantities also makes it comparatively easy to implement. The paper explicitly characterizes the method as numerically stable and efficient, requiring only a single unlabeled input and a forward pass per network (Asthana et al., 18 Aug 2025).

The name should be distinguished from the earlier computer-vision system DEXTR, short for “Deep Extreme Cut,” which addresses interactive object segmentation from extreme points rather than architecture search (Maninis et al., 2017). That method augments RGB input with an additional channel encoding left-most, right-most, top, and bottom object points, and was developed for guided segmentation, interactive segmentation, video object segmentation, and dense segmentation annotation (Maninis et al., 2017). The shared acronymic resemblance is nominal rather than methodological.

Within NAS, however, Dextr denotes the SVD-and-curvature proxy introduced for label-free zero-shot ranking. Its contribution is not merely to produce another zero-cost score, but to make explicit a particular architectural thesis: that high-performing networks at initialization should be both well-conditioned in their internal feature organization and sufficiently curved in their output geometry to remain expressive (Asthana et al., 18 Aug 2025).