---
title: 'SiamNAS: Siamese NAS Surrogates'
url: https://www.emergentmind.com/topics/siamnas
type: topic
---

# SiamNAS: Siamese NAS Surrogates

Searching arXiv for papers directly associated with “SiamNAS” and close disambiguations.
SiamNAS is a label used in arXiv literature for Siamese-network-based neural architecture search surrogates rather than for a single canonical algorithm. In current usage, it refers principally to two distinct predictor-oriented NAS lines: “Siamese-NAS: Using Trained Samples Efficiently to Find Lightweight Neural Architecture by Prior Knowledge” [2210.00546], which targets low-budget lightweight CNN search through a two-branch predictor and an early-training “Estimation Code,” and “SiamNAS: Siamese Surrogate Model for Dominance Relation Prediction in Multi-objective Neural Architecture Search” [2506.02623], which reformulates multi-objective NAS around pairwise Pareto-dominance prediction. A related but distinct term, “NASiam,” applies differentiable NAS to projector and predictor heads inside Siamese self-supervised learning frameworks rather than to the NAS surrogate itself [2302.00059].

## 1. Terminological scope and historical placement

The term “SiamNAS” is best treated as polysemous. It does not designate a single benchmark-standard method analogous to DARTS or ENAS; instead, it names at least two separate efforts that use Siamese structure to improve NAS efficiency, but at different loci of the search pipeline. In both cases, the searched architectures are conventional CNN cell designs rather than Siamese backbones. The Siamese component appears in the **surrogate model** that guides search, not in the final searched network [2210.00546][2506.02623].

| Term | arXiv id | Core role |
|---|---|---|
| Siamese-NAS | [2210.00546] | Predictor-based NAS with Estimation Code |
| SiamNAS | [2506.02623] | Multi-objective NAS via dominance surrogate |
| NASiam | [2302.00059] | Differentiable NAS for Siamese SSL heads |

This distinction matters because the literature also contains multiple “Siamese” models that are not NAS methods at all. Search results around the string “Siam” can therefore conflate predictor-based NAS, Siamese self-supervised learning, Siamese tracking, and Siamese metric learning. In the strict NAS sense, the 2022 and 2025 papers are the central referents of “SiamNAS,” while NASiam is relevant mainly as a nearby but narrower head-search method in Siamese SSL.

## 2. Siamese-NAS as low-budget predictor-based search

“Siamese-NAS” [2210.00546] addresses a standard bottleneck in predictor-based NAS: even when the predictor is sample-efficient, the labels used to train it are expensive because each labeled sample requires training an architecture to obtain its final accuracy. The method is explicitly aimed at **low-budget NAS scenarios** and at the search for **lightweight CNNs**, especially when only a very small number of trained architectures can be afforded.

Its central object is the **Siamese-Predictor**, a two-branch model with a **basic branch** and an **estimation branch**. The basic branch takes the architecture adjacency matrix and feature matrix and produces a coarse accuracy prediction. The estimation branch takes the same architecture representation plus the **Estimation Code**, which encodes prior knowledge from the training procedure, and produces a finer prediction. In the notation reported in the paper, the two prediction modes are
\(p_b \gets P_b(s_{adj}, s_{feat})\) and
\(p_e \gets P_e(s_{adj}, s_{feat}, s_{estc})\) [2210.00546].

The Estimation Code is the most distinctive component. It is constructed from the **first three losses** observed when training an architecture for **three epochs**. To reduce cost, this short run is performed not on the full training set but on a **mini-batch of CIFAR-10 by random sampling with a one-tenth ratio**. The paper states that relative to full training, Estimation Code extraction costs only **0.3%**. The authors further argue that this code has a **local positive correlation** with final accuracy, especially within the **high-accuracy group**, and that this makes it valuable for discriminating among promising architectures under a tight budget [2210.00546].

Architecturally, the predictor augments graph-based NAS predictors with two modules. The **Estimation Fusion Module (EFM)** fuses the upsampled Estimation Code with the architecture feature matrix by a cross-attention-like mechanism constrained by the adjacency matrix, while the **Nodes self-Attention Module (NSAM)** improves modeling of intra-graph node relations. The paper reports that **NSAM and BTS both bring considerable improvements**, indicating that the benefit is not solely from the Estimation Code itself but also from how it is injected into the graph representation [2210.00546].

Search-time efficiency is organized through **Siamese-Ranking** and **Batch Top Sampling (BTS)**. Siamese-Ranking first scores the whole search space with the cheap basic branch, then reranks only the top \(c\) candidates with the estimation branch. The paper states that \(c=30\) in BTS and \(c=60\) in the evaluation stage. BTS then updates the training pool progressively toward promising architectures rather than relying on purely random sampling. Runtime comparisons reported in the paper show a substantial gain: on **Tiny-NanoBench (1120)**, **Fully Top Sampling** takes **115.67 s** while **Batch Top Sampling** takes **67.22 s**; on **NASBench-201 (15,625)**, the corresponding numbers are **816.43 s** and **140.62 s** [2210.00546].

Empirically, the method is evaluated on **NASBench-201** and on the proposed **Tiny-NanoBench**, a search space of **1120 architectures** designed for very lightweight CNN discovery. The paper reports that **Siamese-Predictor + NSAM + BTS** surpasses **BRP-NAS** when trained samples exceed **125**, where **125 trained samples** are only about **0.8%** of NASBench-201 and **200 trained samples** are about **1.3%**. It also concludes that under a fixed budget \(N+K\), increasing the number \(N\) of architectures used to train the predictor is more effective than increasing the number \(K\) of top-ranked candidates trained from scratch. This makes Siamese-NAS a strongly **predictor-centric** interpretation of SiamNAS: the main optimization target is sample efficiency of architecture evaluation, not redesign of the downstream neural network itself [2210.00546].

## 3. SiamNAS as a dominance-relation surrogate for multi-objective NAS

The 2025 paper “SiamNAS” [2506.02623] shifts the setting from low-budget single-objective search to **multi-objective NAS**. Its premise is that modern NAS must jointly optimize prediction performance and complexity, and that in this regime the decisive primitive is not exact objective regression but **Pareto dominance comparison**. SiamNAS therefore proposes a surrogate that predicts whether one architecture dominates another, and integrates that comparator into an **NSGA-II-style evolutionary search**.

The paper formulates the optimization problem over three objectives:
\[
\mathbf{f}_{e}\left(\mathbf{x}\right)= 1-acc(\mathbf{x}, D),
\qquad
\mathbf{f}_{c}\left(\mathbf{x}\right)=\left( \#params(\mathbf{x}), FLOPS(\mathbf{x}) \right).
\]
The search therefore minimizes \(1-\text{accuracy}\), \(\#\text{params}\), and \(\text{FLOPs}\) [2506.02623]. Rather than learning these quantities directly, SiamNAS learns a directional binary label: for a pair \((\mathbf{x}_1,\mathbf{x}_2)\), the label is \(1\) if the first candidate dominates the second and \(0\) otherwise.

The surrogate is an **ensemble of Siamese blocks**. Each block receives a pair of NAS-Bench-201 architectures encoded as a **\(6 \times 5\) one-hot matrix**, flattened to a **30-dimensional** binary vector. Each branch is a shared-weight MLP with one hidden fully connected layer of **32 neurons** and **ReLU**, producing embeddings \(\mathbf{b}_1,\mathbf{b}_2 \in \mathbb{R}^{32}\). These are combined by subtraction,
\[
\mathbf{d} = \mathbf{b}_1 - \mathbf{b}_2,
\]
and passed to a classifier MLP with a **32-neuron ReLU layer** and a **1-neuron sigmoid** output. The rounded output \(R \in \{0,1\}\) is interpreted as whether the first architecture dominates the second. A ternary decision—first dominates second, second dominates first, or neither dominates the other—is recovered by applying majority vote across the ensemble, then testing the reversed order if needed [2506.02623].

Training efficiency comes from pair generation. With \(N_s\) evaluated architectures, up to \(N_s \times N_s\) training pairs can be formed. The paper identifies class imbalance as a key issue, since many random pairs are non-dominating, and introduces a reassignment heuristic with
\[
\theta \leftarrow 0.5/(1-\sigma),
\]
where \(\sigma\) is the proportion of positive pairs in a generated batch. The main training configuration uses **\(N_s = 600\)** evaluated architectures, **ensemble size \(N_m = 7\)**, **Adam**, **binary cross-entropy**, **20 epochs**, **learning rate 0.001**, and **batch size 100** [2506.02623].

Within the search loop, SiamNAS modifies NSGA-II in two ways. First, it uses the surrogate for **binary tournament selection** and for **sequential efficient non-dominated sorting**, thereby avoiding true objective evaluation of offspring during the search itself. Second, because standard **crowding distance** depends on unavailable objective values, it replaces crowding distance with a **Biased-Selection** heuristic: when the last non-dominated front overflows the population limit, architectures in that front are sorted in **descending order by number of trainable parameters**, and the largest are retained. The paper justifies this by noting that parameter count is training-free and that larger models often achieve better performance empirically [2506.02623].

The reported computational profile is unusually small. True evaluations are required only for surrogate training and final assessment, so the total number of true evaluations is bounded by **\(N_s + N\)**. In the reported setup, \(N=50\), giving at most **650 true evaluations**. Runtime, excluding benchmark-queried architecture evaluation time, is reported as approximately **0.01 GPU days** on a **single NVIDIA GeForce RTX 4090 GPU** [2506.02623].

The empirical outcome is a strong proof of concept on **NAS-Bench-201**. The surrogate reaches about **92% accuracy** on dominance prediction; more specifically, the paper reports **92.83%** on CIFAR-10 at \(N_m=7\), **91.61%** on CIFAR-100 at \(N_m=7\), and **92.57%** on ImageNet-16-120 at \(N_m=7\). For search performance, SiamNAS achieves **\(5.63 \pm 0.00\)** test error on CIFAR-10, matching the **Oracle** value **5.63**; **\(27.39 \pm 0.31\)** on CIFAR-100; and **\(54.07 \pm 0.08\)** on ImageNet-16-120. The **SiamNAS-transfer** variant, trained on CIFAR-10 and applied to the other tasks, yields **\(27.03 \pm 0.19\)** on CIFAR-100 and **\(53.63 \pm 0.12\)** on ImageNet-16-120, which the paper identifies as the **second-best** result among compared methods on the latter benchmark [2506.02623].

## 4. Shared principles and major divergences

The two SiamNAS lineages share a family resemblance: both are **surrogate-assisted** NAS methods that attempt to reduce expensive architecture evaluation, and both use a **Siamese inductive bias** to improve the search guidance signal. In both cases, the searched architectures lie in a conventional cell-based CNN search space, while the Siamese model serves as an evaluator or ranker rather than as the final architecture to be deployed [2210.00546][2506.02623].

Their divergence is nonetheless fundamental. “Siamese-NAS” [2210.00546] predicts **absolute architecture accuracy** and enriches that prediction with a small amount of prior knowledge from the training process. Its crucial signal is the **first three losses** of a short training run, and its control structure is **coarse-to-fine reranking** plus **focused sample acquisition** through BTS. By contrast, “SiamNAS” [2506.02623] predicts **pairwise dominance relations** and dispenses with direct objective regression. Its crucial signal is not early training dynamics but the **relative Pareto order** induced by objective triples \((1-\text{acc}, \#\text{params}, \text{FLOPs})\), and its control structure is **surrogate-driven non-dominated sorting** inside an evolutionary search.

This difference also changes the semantics of “Siamese.” In the 2022 method, Siamese structure supports a predictor with two coordinated information paths—one structure-only and one structure-plus-prior. In the 2025 method, Siamese structure supports a comparator that embeds two candidate architectures into a common latent space and learns a directional relation on their difference vector. A plausible implication is that the 2022 paper should be placed in the lineage of **accuracy predictors for single-objective NAS**, whereas the 2025 paper belongs more naturally to **ranking/comparator surrogates for Pareto search**.

The efficiency strategies are likewise distinct. The 2022 method spends a tiny amount of extra computation to obtain a more informative predictor and then uses that predictor selectively near the top of the ranking. The 2025 method aims to eliminate true evaluations from the search loop altogether and replaces unavailable crowding information with a simple heuristic based on model size. Both strategies are rational responses to evaluation cost, but they encode different assumptions: one assumes that **small amounts of early optimization signal** are worth paying for, while the other assumes that **pairwise structural relations** can be learned well enough to substitute for direct objective access.

## 5. Disambiguation relative to adjacent “Siamese” literature

A recurrent misconception is that SiamNAS denotes **architecture search over Siamese networks themselves**. The literature does not support that as a general definition. In the two principal SiamNAS papers, the Siamese mechanism is located in the **NAS surrogate**, not in the searched task model. The final architectures are lightweight CNN cell instantiations from search spaces such as NASBench-201 and Tiny-NanoBench, not twin-tower verification networks or template-search tracking models [2210.00546][2506.02623].

A second source of confusion is **NASiam**. “NASiam: Efficient Representation Learning using Neural Architecture Search for Siamese Networks” [2302.00059] is directly relevant to Siamese SSL, but it searches only the **projector and predictor MLP heads** in frameworks such as SimSiam, SimCLR, MoCo v2, and BYOL while keeping the convolutional backbone fixed. It is therefore best described as **head-level differentiable NAS for Siamese self-supervised learning**, not as a general SiamNAS framework.

Other papers that may appear in string-based searches are still further removed. “TimeSiam” [2402.02475] is a Siamese self-supervised framework for time-series pre-training and contains **no architecture search**. “SiamSA” [2211.14564] is a manually designed Siamese tracker with pairwise scale-channel attention for UAM approaching, again **not a NAS paper**. “Advancing Single-Snapshot DOA Estimation with Siamese Neural Networks for Sparse Linear Arrays” [2501.07008] is a Siamese metric-learning model for sparse-array DOA estimation and also **not a NAS method**. These works are relevant only insofar as they broaden the technical uses of Siamese architectures; they do not define SiamNAS in the NAS sense.

## 6. Limitations, unresolved questions, and likely directions

Both SiamNAS variants are explicitly efficient, but both inherit limitations from the kind of surrogate they use. In “Siamese-NAS” [2210.00546], the Estimation Code is cheap rather than free, its correlation with final accuracy is described as **local** rather than uniformly global, and the strongest demonstrations are concentrated on **NASBench-201** and the authors’ **Tiny-NanoBench**. The paper also leaves the exact operator grammar of Tiny-NanoBench only partially specified in the provided text. This suggests that the method’s success depends not only on the predictor architecture but also on the match between early-training signals and the structure of the search space.

In “SiamNAS” [2506.02623], the evidence is entirely benchmark-bound: experiments are performed on **NAS-Bench-201**, where objective values are queried from a benchmark rather than obtained by full real-world training during search. The replacement of crowding distance by a **parameter-count bias** is practical but heuristic, and the quality of the dominance classifier still depends on the coverage of the initial **\(N_s\)** evaluated sample set. The paper itself presents multitask transfer and extension to **Sets of Pareto Sets (SOS)** as future work rather than established capability. This suggests that the method’s main conceptual contribution is not yet a universal recipe for multi-objective NAS, but a proof that **dominance prediction** can substitute for objective regression in at least one standard NAS benchmark.

Taken together, the two SiamNAS lines define a coherent research theme: use Siamese structure not to search for Siamese task models, but to make **architecture evaluation more sample-efficient**. One line injects a compact trace of optimization dynamics into an accuracy predictor; the other learns the pairwise ordering relation required by Pareto search. The common lesson is that in NAS, a surrogate need not reconstruct the full training process to be useful. It may suffice to learn a carefully chosen relational signal—early-loss behavior in one case, dominance structure in the other—provided that the search algorithm is redesigned around that signal [2210.00546][2506.02623].

Source: https://www.emergentmind.com/topics/siamnas