Robustly surpass the parameter-count baseline across heterogeneous search spaces

Establish a single neural architecture search method that robustly beats the trivial #Params/FLOPs baseline across both structure-varying and size-varying search spaces under one evaluation protocol.

Background

The paper identifies a recurring cross-over in zero-cost neural architecture search: capacity-oriented proxies tend to perform well in size-varying spaces but poorly in topology-varying spaces, whereas structure-oriented proxies exhibit the opposite behavior. Consequently, no individual proxy or simple baseline had been shown to dominate across both regimes under a unified protocol.

The authors present CoRA-NAS as addressing this challenge across four vision benchmarks, while explicitly limiting the scope of the demonstrated no-weak-regime result. The quoted sentence nevertheless frames robustly clearing the #Params/FLOPs bar across structure and size spaces as an open problem in the broader setting.