BBOPlace-Bench: Chip Placement Benchmark

• BBOPlace-Bench is a benchmark framework that formalizes chip placement as a family of black-box optimization problems with distinct formulations.
• It decouples problem formulation, optimization algorithms, and evaluation, enabling methods like SA, EA, BO, and HPO to be systematically compared.
• Empirical results show that the choice of formulation greatly influences performance and scalability across varied industrial chip placement scenarios.

Searching arXiv for the benchmark paper and closely related formulation paper to ground the article in the latest relevant literature. BBOPlace-Bench is a benchmark and framework for evaluating and developing black-box optimization (BBO) algorithms for chip placement. It formalizes chip placement as a family of BBO problems, integrates three problem formulations specific to placement, and exposes them through a modular, decoupled, and flexible framework in which problem formulation, optimization algorithm, and evaluation are separated. The benchmark standardizes industrial suites, includes simulated annealing (SA), evolutionary algorithms (EAs), and Bayesian optimization (BO), and supports evaluation at the levels of macro-placement wirelength, global-placement wirelength, and downstream PPA metrics obtained from Cadence Innovus (Xue et al., 27 Oct 2025).

1. Research context and problem setting

Chip placement is a central stage in physical design after logic synthesis, where physical locations are assigned to modules on a chip canvas. In the formulation used by BBOPlace-Bench, the netlist is a hypergraph $\mathcal{N}=(V,E)$, where $V$ denotes modules and $E$ denotes nets; for $k$ movable modules $\{m_i\}_{i=1}^k$, a placement solution is

$$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$

Placement quality directly affects wirelength, congestion, area, and timing. A standard proxy for routing wirelength is Half-Perimeter Wirelength (HPWL), with the single-objective formulation

$$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$

subject to a density constraint $D(\mathbf{s})\le \epsilon$, where for a net $e$,

$$\mathrm{HPWL}_e(\mathbf{s})=(\max_{P\in e} P_x-\min_{P\in e} P_x)+(\max_{P\in e} P_y-\min_{P\in e} P_y).$$

The objective is non-differentiable and optimizing it is NP-complete, which motivates black-box treatments of the problem (Xue et al., 27 Oct 2025).

The benchmark is situated against two historical phases of BBO for placement. Early work relied on sequence-pair-based SA and evolutionary methods, but these formulations suffered from huge combinatorial search spaces, expensive decoding, hard constraint handling, and low scalability. More recent work altered the formulation itself. In particular, WireMask-Guided BBO introduced a grid-based continuous representation and a wire-mask-guided greedy decoder that guarantees no overlap and greedily minimizes incremental HPWL, thereby making macro placement substantially more amenable to BBO (Shi et al., 2023). BBOPlace-Bench incorporates that line of development alongside hyperparameter optimization of DREAMPlace and the classical sequence-pair formulation, and thereby turns disparate strands of placement-oriented BBO into a common experimental object (Xue et al., 27 Oct 2025).

2. Benchmark architecture and problem formulations

BBOPlace-Bench is organized around an explicit decoupling of three components: problem formulation, optimization algorithm, and evaluation. The problem formulation defines the search space and genotype-phenotype mapping; the optimization algorithm interacts through ask() and evaluate(); the evaluation stage transforms decoded placements into metrics such as MP-HPWL, GP-HPWL, and PPA. This design permits the same optimizer to be tested across heterogeneous placement formulations and, conversely, permits new formulations or evaluators to be inserted without modifying the optimization layer (Xue et al., 27 Oct 2025).

Formulation	Search space	Decoder
Sequence Pair (SP)	$V$0, two permutations	Packing from pairwise order relations
Mask-Guided Optimization (MGO)	$V$1	Wire-mask-guided greedy decoding on an $V$2 grid
Hyperparameter Optimization (HPO)	$V$3, mixed configuration space	Run DREAMPlace with hyperparameters $V$4

The sequence-pair formulation uses a purely discrete search space of size $V$5. A solution is $V$6, where $V$7 and $V$8 are permutations of the macros. Decoding computes pairwise horizontal and vertical relations, derives orders with longest-common-subsequence-style logic, computes coordinates consistent with those orders, and adjusts coordinates to ensure non-overlap and minimal area. Its principal strengths are a clear combinatorial encoding and legality by construction, but its decoding is $V$9 and the search space grows super-exponentially, which makes exploration difficult for large designs (Xue et al., 27 Oct 2025).

The MGO formulation uses a continuous coordinate representation of dimension $E$0 and decodes candidates by discretizing the canvas into an $E$1 grid, ordering macros by the total area of all modules connected to each macro, constructing a wire mask for each macro, removing infeasible grids, and placing each macro on a grid with minimal incremental HPWL that is nearest to the original candidate coordinate. The resulting decoded placement is legal, non-overlapping, and greedily improves HPWL at each placement step. This formulation inherits the essential logic of WireMask-BBO, where the genotype is unconstrained continuous coordinates and the phenotype is a legal grid-aligned placement returned by a wire-mask-guided greedy evaluator (Shi et al., 2023).

The HPO formulation does not optimize placements directly. Instead, it tunes the hyperparameters of an analytical placer, specifically DREAMPlace. A candidate is

$E$2

and decoding consists of configuring the analytical placer $E$3 and running it on the netlist. The search dimensions listed in the benchmark include GP_num_bins_x, GP_num_bins_y, GP_optimizer, GP_wirelength, GP_learning_rate, GP_Llambda_density_weight_iteration, GP_Lsub_iteration, GP_learning_rate_decay, stop_overflow, target_density, RePlAce_LOWER_PCOF, RePlAce_UPPER_PCOF, RePlAce_ref_hpwl, density_weight, and gamma; the search space dimension is fixed at around $E$4 variables and is independent of macro count (Xue et al., 27 Oct 2025).

3. Black-box interface and integrated optimization algorithms

The benchmark models placement strictly as black-box optimization. The optimizer sees only query-response pairs: a candidate in the formulation-specific search space is decoded into a placement, and an objective such as MP-HPWL, GP-HPWL, or optionally downstream PPA is returned. No gradients, analytic structure, or internal solver state are exposed. For MP-HPWL, the benchmark uses a budget of 10,000 function evaluations; for GP-HPWL, it uses a budget of 200 evaluations because each evaluation requires placing standard cells via DREAMPlace and is therefore much more expensive (Xue et al., 27 Oct 2025).

SA is implemented as a single-solution stochastic search with initial temperature 100, cooling schedule decay 0.99 every 100 steps, and the standard Metropolis acceptance rule. Its mutation operator depends on the formulation: inversion for SP, shuffle for MGO, and random resetting for HPO. As configured in the benchmark, SA functions primarily as a baseline against population-based methods (Xue et al., 27 Oct 2025).

The principal EA baseline is a standard generational algorithm with population size $E$5 in most experiments. For SP it uses order crossover; for MGO and HPO it uses uniform crossover with probability $E$6 per coordinate or hyperparameter; mutation follows the same formulation-dependent operators used by SA. The benchmark also includes Evolution Strategies through CMA-ES for continuous spaces, with initial sigma $E$7, and PSO for continuous spaces via PyPop7, configured with cognitive learning rate $E$8, social learning rate $E$9, and max velocity ratio $k$0. The PSO update follows

$k$1

$k$2

Bayesian optimization is integrated through BoTorch with a Gaussian Process surrogate using a Matern-5/2 kernel and Expected Improvement as the acquisition function (Xue et al., 27 Oct 2025).

The framework for new algorithms is intentionally minimal. The benchmark provides an Evaluator object exposing n_dim, xl, xu, and evaluate(solutions), and algorithms are expected to implement the standard ask()/tell() loop. This API is the mechanism by which BBOPlace-Bench treats SP, MGO, and HPO as interchangeable black-box tasks, despite their radically different search spaces and decoders (Xue et al., 27 Oct 2025).

4. Datasets, preprocessing, and evaluation protocol

BBOPlace-Bench supports two industrial benchmark suites. ISPD 2005 uses the Bookshelf format and includes the adaptec and bigblue series. ICCAD 2015 uses LEF/DEF and ancillary files and includes the superblue series. The benchmark preprocesses both formats into a unified internal representation comprising module dimensions, nets and pins, canvas dimensions, blockages, and densities. For ICCAD 2015, where macros are not predefined, BBOPlace-Bench defines macros as the 512 largest modules by area, with this threshold being configurable (Xue et al., 27 Oct 2025).

The benchmark spans substantial placement scales. For ISPD 2005, the reported chips adaptec1-4, bigblue1, and bigblue3 contain 543–1329 macros, approximately $k$3k–$k$4M cells, 221k–1.1M nets, and approximately 944k–3.8M pins. For ICCAD 2015, the chips superblue1,3,4,5,7,10,16,18 fix macro count at 512 and contain approximately 768k–1.93M cells, approximately 771k–1.93M nets, and approximately 2.5M–6.37M pins. These scales directly determine the effective dimensionality of SP and MGO, while HPO remains fixed-dimensional (Xue et al., 27 Oct 2025).

Evaluation is multi-level. MP-HPWL measures macro-only wirelength and is comparatively cheap. GP-HPWL measures wirelength after macro placement and subsequent standard-cell placement through DREAMPlace, making it more accurate but more expensive. Downstream PPA is measured with Cadence Innovus and includes routed wirelength rWL, congestion overflow rO-H and rO-V, worst negative slack WNS, total negative slack TNS, and number of violation paths NVP. Results are reported over 5 independent runs as mean $k$5 standard deviation; significance is assessed with a Wilcoxon rank-sum test at the $k$6 level, using “≈” for statistically not worse than best and “-” otherwise. Analytical and reinforcement-learning baselines include DREAMPlace, AlphaChip, MaskPlace, and EfficientPlace (Xue et al., 27 Oct 2025).

5. Empirical findings across formulations and algorithm families

The benchmark’s principal empirical result is that problem formulation matters as much as optimizer choice. On ISPD 2005 for MP-HPWL, SP-based methods are uncompetitive: SP-SA and SP-Vanilla-EA have the worst overall average ranks, 15 and 16, respectively, and on adaptec3 they report $k$7 and $k$8 in units of MP-HPWL $k$9. MGO improves substantially, with examples on adaptec3 such as $\{m_i\}_{i=1}^k$0 for MGO-SA and $\{m_i\}_{i=1}^k$1 for MGO-Vanilla-EA. HPO is strongest on ISPD for macro-only wirelength, with HPO-Vanilla-EA achieving average rank $\{m_i\}_{i=1}^k$2 and HPO-PSO $\{m_i\}_{i=1}^k$3, and on adaptec3 values such as $\{m_i\}_{i=1}^k$4 for HPO-Vanilla-EA (Xue et al., 27 Oct 2025).

For GP-HPWL, HPO is consistently dominant because it optimizes through DREAMPlace and therefore considers macros, standard cells, and density within the underlying analytical placement flow. On ISPD 2005, HPO-Vanilla-EA attains average rank $\{m_i\}_{i=1}^k$5 and HPO-BO $\{m_i\}_{i=1}^k$6; on adaptec1, HPO-Vanilla-EA yields $\{m_i\}_{i=1}^k$7 and HPO-BO yields $\{m_i\}_{i=1}^k$8 in units of GP-HPWL $\{m_i\}_{i=1}^k$9, whereas MGO-Vanilla-EA is much worse at $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$0 (Xue et al., 27 Oct 2025).

On ICCAD 2015, the pattern bifurcates according to objective. For MP-HPWL, MGO is dominant: MGO-PSO achieves average rank $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$1 and MGO-Vanilla-EA $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$2; on superblue4, MGO-PSO attains $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$3 and MGO-Vanilla-EA $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$4, while HPO-Vanilla-EA is markedly worse at $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$5 in units of MP-HPWL $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$6. For GP-HPWL, HPO again becomes best: HPO-PSO attains average rank $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$7 and HPO-Vanilla-EA $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$8; on superblue1, HPO-BO, HPO-Vanilla-EA, and HPO-PSO report $$\mathbf{s}=\{(x_i,y_i)\}_{i=1}^k \in \mathbb{R}^{2k}.$$9, $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$0, and $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$1, respectively, while MGO-Vanilla-EA reports $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$2 in units of GP-HPWL $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$3 (Xue et al., 27 Oct 2025).

Across algorithm families, EAs demonstrate better overall performance than SA and BO, especially in high-dimensional search spaces. Vanilla-EA and PSO are most often best or near-best; CMA-ES is competitive but not dominant; SA is generally weaker because of single-point exploration; and BO is strongly formulation-dependent. Under high-dimensional MGO, BO underperforms, with average ranks around 8–10, which aligns with the benchmark’s observation that GP-based BO scales poorly in high dimensions. Under low-dimensional, expensive HPO, however, BO becomes very competitive and is often runner-up for GP-HPWL (Xue et al., 27 Oct 2025).

These results also revise a common historical expectation that BBO for placement is intrinsically too slow or too poorly structured to compete with analytical and reinforcement-learning methods. A more precise reading of the evidence is that older formulations such as sequence pair are slow and hard to scale, whereas formulation changes such as wire-mask-guided decoding and low-dimensional HPO materially alter the optimization landscape. That interpretation is consistent with the earlier WireMask-BBO finding that a wire-mask-guided evaluator can outperform packing-based, analytical, and RL baselines in macro placement by making legality and incremental HPWL optimization part of the evaluator rather than the optimizer (Shi et al., 2023).

6. Ablations, runtime behavior, and landscape implications

BBOPlace-Bench includes an ablation on the number of macros in MGO, directly linking placement scale to black-box difficulty. On ICCAD 2015, the authors vary macro count as 128, 256, 512, and 1024 for MGO-Vanilla-EA and MGO-PSO. GP-HPWL generally worsens as the number of macros increases; for example, on superblue1, MGO-Vanilla-EA reports $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$4, $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$5, $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$6, and $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$7 as macro count increases from 128 to 1024. The stated interpretation is that, at fixed evaluation budget, increasing dimensionality makes optimization harder, while smaller macro sets concentrate optimization effort on the macros deemed most important by the formulation (Xue et al., 27 Oct 2025).

Runtime analysis on adaptec3 distinguishes optimization overhead from evaluation cost. Per-iteration optimization time is approximately $$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$8–$$\min_{\mathbf{s}} \mathrm{HPWL}(\mathbf{s})=\min_{\mathbf{s}} \sum_{e\in E}\mathrm{HPWL}_e(\mathbf{s})$$9 s for SA, $D(\mathbf{s})\le \epsilon$0–$D(\mathbf{s})\le \epsilon$1 s for Vanilla-EA, $D(\mathbf{s})\le \epsilon$2–$D(\mathbf{s})\le \epsilon$3 s for PSO, $D(\mathbf{s})\le \epsilon$4–$D(\mathbf{s})\le \epsilon$5 s for ES, and $D(\mathbf{s})\le \epsilon$6–$D(\mathbf{s})\le \epsilon$7 s for BO. Evaluation time for MP-HPWL is $D(\mathbf{s})\le \epsilon$8 s for SP, $D(\mathbf{s})\le \epsilon$9 s for MGO, and $e$0 s for HPO; for GP-HPWL it is $e$1 s for SP, $e$2 s for MGO, and $e$3 s for HPO. The resulting characterization is that HPO is evaluation-dominated, SP and MGO permit much larger budgets under MP-HPWL, and BO’s optimization overhead is relatively acceptable only when the evaluation budget is small and each evaluation is expensive (Xue et al., 27 Oct 2025).

The benchmark also articulates representation-level differences in search landscape. SP is described as a very rugged, high-dimensional discrete landscape with complex decoding; MGO as a continuous landscape whose wire-mask decoder enforces legality and locally meaningful incremental HPWL improvement; and HPO as a low-dimensional, heavily structured hyperparameter landscape that is smoother but still non-convex. This suggests that BBOPlace-Bench is not a single optimization problem but a set of regimes: high-dimensional combinatorial search, high-dimensional continuous search with hard geometric constraints, and low-dimensional expensive mixed search (Xue et al., 27 Oct 2025).

7. Software organization, practical usage, and limitations

The implementation is in Python and is organized into a preprocessing layer, problem formulations, an Evaluator, separate optimization algorithms, and evaluation and logging modules. The preprocessing layer converts Bookshelf and LEF/DEF data into a unified representation. The formulation layer contains sequence_pair_formulation, wiremask_guided, and hpo_algorithm. The evaluator holds chip data, the selected formulation, problem dimension, and bounds, and provides evaluate(solutions). The optimization layer includes SA, Vanilla-EA, ES via pycma, PSO via pypop7, and BO via botorch and PyTorch. DREAMPlace provides analytical placement and GP-HPWL evaluation, and Cadence Innovus provides PPA metrics. The code is publicly available at https://github.com/lamda-bbo/BBOPlace-Bench (Xue et al., 27 Oct 2025).

The framework is intended to support several distinct research modes. SP is suitable for permutation-specific BBO in a classic representation. MGO is the natural choice for high-dimensional continuous BBO with geometric constraints and relatively cheap macro-only evaluations. HPO is the preferred choice when the objective is full-chip GP-HPWL or downstream PPA, since it tunes a deterministic analytical placer directly. The benchmark further notes algorithm-selection heuristics: Vanilla-EA and PSO are preferred for high-dimensional MGO under large budgets; BO becomes competitive in the low-dimensional, expensive HPO setting; and PSO has especially low optimization overhead under tight runtime constraints (Xue et al., 27 Oct 2025).

The benchmark’s limitations are also explicit. PPA evaluation depends on Cadence Innovus, a commercial tool, which restricts accessibility. The supported circuit suites are ISPD 2005 and ICCAD 2015, which are substantial but limited in variety. Optimization is primarily centered on scalar HPWL even though the downstream design problem is inherently multi-objective and constrained. The BO integration is generic rather than specialized for high-dimensional or highly structured placement problems. The proposed future directions are correspondingly specific: multi-objective optimization over wirelength, congestion, timing, power, and area; surrogate-assisted expensive optimization for GP-HPWL and PPA; high-dimensional optimization using clustering and hierarchical placement; learning-enhanced optimization and hyper-heuristics; and exploratory landscape analysis for SP, MGO, and HPO (Xue et al., 27 Oct 2025).