Iterative Architecture Synthesis Loop

• Iterative Architecture Synthesis Loop is a meta-heuristic framework that alternates global search and local refinement to iteratively optimize design architectures.
• It integrates coarse-grained candidate exploration with fine-grained, data-driven adjustments such as neuron growth, pruning, and subcircuit extraction for effective synthesis.
• Applied in both neural networks and quantum circuits, the loop consistently yields architectures that outperform traditional methods in accuracy and resource efficiency.

The Iterative Architecture Synthesis Loop is a meta-heuristic framework for generating network or circuit architectures that alternates between global structure search and local refinement phases in a closed-loop manner. This approach has been applied effectively to neural network synthesis (Hassantabar et al., 2019) and quantum circuit compilation (Martiel et al., 2020), enabling efficient design of architectures under strong accuracy or resource constraints. The core principle is the repeated interplay of coarse-grained search, using lightweight predictors or routing techniques, with fine-grained, data-driven adjustments such as pruning, growth, or local re-synthesis. Each iteration produces architectures that are incrementally more optimal with respect to user-specified metrics, and the process continues until convergence or a predefined computational budget is exhausted.

1. Formal Definition and Scope

The Iterative Architecture Synthesis Loop is defined as a closed iterative procedure comprising two distinct modules:

• Global Search (GS): Explores a reduced hyperparameter space to identify promising candidate architectures using predictors or meta-heuristics.
• Local Search (LS): Refines these candidates at the graph- or subcircuit level, applying operations such as pruning, edge growth, neuron synthesis, or local subcircuit extraction.

The two modules are connected in a loop: after local refinement, the improved architecture is returned to the global phase, where it updates predictors or serves as an initial population member for further search. This process is repeated either for a fixed number of cycles or until the incremental gain in the optimization metric falls below a threshold. This framework is distinguished from one-shot or strictly forward pipelines by this explicit feedback of refined samples into the global search (Hassantabar et al., 2019, Martiel et al., 2020).

2. Application to Neural Network Synthesis

In the STEERAGE methodology for neural network synthesis, the loop structure alternates between:

• Global Search: A Sobol-sampled pool of candidate network architectures is evaluated using a regressor $f: \mathbb{R}^n \to \mathbb{R}$ (typically a boosted decision tree), trained to predict validation accuracy as $L(f) = \sum_{(x_i, y_i)} (f(x_i) - y_i)^2$. This reduces the cost of fully training each candidate, allowing evolutionary algorithms to optimize architecture parameters (layers, widths, connections) using $f(x)$ as the fitness function.
• Local Grow-and-Prune: Given a global search optimum, local routines further refine the network: for feed-forward networks (FFNN), growth steps add connections and neurons based on gradient magnitude, and pruning removes weak edges or redundant neurons. Convolutional subgraphs use analogous mechanisms targeting kernels and receptive fields.

The loop operates as follows: after local refinement, the architecture and its validated accuracy are reintegrated into the global search training set, $f(x)$ is retrained, and the global search phase is relaunched (typically for $C=2$ cycles or until $\Delta\textrm{Acc}<\epsilon$). This results in architectures that outperform traditional baselines both in accuracy and compactness. For example, on CIFAR-10, a STEERAGE-synthesized ResNet-18 exhibits a 2.52% absolute accuracy gain over vanilla ResNet-18, matching or exceeding more parameter-heavy baselines while preserving efficiency (Hassantabar et al., 2019).

3. Application to Quantum Circuit Compilation

For architecture-aware compilation of quantum circuits, the iterative loop blends global SWAP-insertion with local, greedy architecture-aware synthesis:

• Global Routing (SWAP insertion): A logical-to-physical mapping is maintained via a subgroup $S$ (e.g., permutations, CNOTs, or Clifford group). Gates compatible with the current mapping are absorbed into a global "lazy" record $A$.
• Local Synthesis: When a gate is incompatible with the hardware graph $G$, a local extraction routine synthesizes the minimal subcircuit (SWAP cascade, CNOT Steiner-tree, or Clifford fan-in/out) necessary to execute the gate and bring the mapping into compliance.

This procedure is formalized in the following pseudocode:

$\begin{algorithmic}[1] \Require C_{\rm in}=g_1\cdots g_m \State A \leftarrow \operatorname{Id} \State C_{\rm out} \leftarrow \varepsilon \For{i=1,\dots,m} \If{g_i \in S} \State A \leftarrow g_i \cdot A \Else \State (A,c) \leftarrow \operatorname{Extract}(A,g_i) \State \operatorname{Append}(c, C_{\rm out}) \EndIf \EndFor \State \Return (A, C_{\rm out}) \end{algorithmic}$

Performance results demonstrate substantial reductions in entangling two-qubit gate overhead for compiled quantum circuits compared to pure SWAP-based or synthesis-only strategies (relative overhead as low as $0$–$20\%$ versus $50$–$200\%$ for SWAP-only on standard benchmarks) (Martiel et al., 2020).

4. Module Workflows and Heuristic Details

Neural Network Case

• Predictor Construction: Utilizes Sobol sampling to create a candidate pool, iterative sample selection, and boosted decision tree regression.
• Evolutionary Search: Parents are mutated with probability $\rho$ and evaluated by the predictor; offspring replace the worst candidates.
• Local Search: For each iteration:
  • ConnectionGrow: Adds top-$k$ dormant connections with largest gradient magnitude $|g_{ij}|$.
  • NeuronGrow: Inserts new or duplicates neurons.
  • ConnectionPrune: Removes connections with $|w_{ij}| \leq \zeta$.
  • Trains weights and updates validation accuracy.
  • Selects architectures yielding improved performance.

Quantum Circuit Case

• Greedy Synthesis Specializations:
  • SWAP (Permutation): Maintains logical mappings with minimal extra gates by synthesizing optimal SWAP cascades along short paths in $G$.
  • CNOT (Linear): Extracts CNOT circuits using Steiner-tree fan-in/fan-out to accumulate or distribute parity.
  • Clifford: Handles Pauli rotations by diagonalizing via local Clifford gates, performing fan-in, inserting rotations, and reversing the process.
• Key Parameters: Lookahead depth $w$ (controls extraction branches), cost metric for gate minimization, and branching on local extraction choices.

5. Convergence, Complexity, and Stopping Criteria

Neural Networks

• Predictor Pool Size: $2048$ (Sobol-sampled).
• Evolutionary Search: Population size $=100$, $200$ generations, mutation rate $\rho\approx 0.1$.
• Local Search Iterations: $J_{\textrm{max}} \approx 50$.
• Growth/Prune Budgets: Example, add $5\%$ new connections, prune $10\%$ smallest weights per cycle.
• Outer Loop: Runs for $C=2$ cycles or until $\Delta \mathrm{Acc} < \epsilon$ (e.g., $0.01\%$).

Quantum Circuits

• Termination: Guaranteed after $m$ iterations (finite input).
• Complexity:
  • SWAP-only: $O(m\,n^{1+w})$
  • CNOT/Clifford: $O(m\,n^{3+w})$
• Space Complexity: $O(n+m)$ plus subgroup representations ($O(n)$ words, $O(n^2)$ bits).
• Parameters: Lookahead depth $w$ (typically $0$ or $1$).

6. Empirical Performance and Theoretical Insights

Neural Networks

On MNIST, STEERAGE achieves $0.66\%$ error with $8.6\times$ fewer parameters than LeNet-5. On CIFAR-10, its ResNet-18 variant offers $2.52\%$ absolute accuracy improvement over vanilla ResNet-18, with parameter and FLOP counts comparable to the original (Hassantabar et al., 2019). This demonstrates that architecture refinement in the loop is more effective than simply increasing network depth.

Quantum Circuits

Benchmarks on IBM Melbourne and Rigetti Aspen show that the Clifford-based loop with merging and reordering outperforms or matches the best existing methods, particularly on sparse topologies. For QAOA and VQE-style circuits on 14-qubit grids, the approach roughly halves CNOT counts relative to SWAP-only routing (Martiel et al., 2020).

• Pure SWAP-insertion ignores algebraic structure and increases gate count.
• Pure synthesis lacks universality or sacrifices global circuit context.
• The loop leverages a global “lazy” record to commute "easy" gates and applies local extraction for non-subgroup gates, minimizing resource usage.
• Lookahead $w$ enables balancing immediate and future costs, improving over greedy heuristics.

7. Comparative Summary

Problem Domain	Global Phase	Local Phase	Main Optimization Target
Neural Networks (Hassantabar et al., 2019)	Predictor-based evolutionary search	Grow-and-prune (SCANN/NeST)	Accuracy, compactness
Quantum Circuits (Martiel et al., 2020)	SWAP, CNOT, or Clifford mapping	Local subcircuit extraction, fan-in/out	Minimization of two-qubit gates

The Iterative Architecture Synthesis Loop provides a generalizable meta-heuristic for architecture optimization that unifies the merits of global search and local synthesis, demonstrated to yield state-of-the-art efficiency and performance across distinct computational paradigms.