OpenBox: Generalized BBO & System Toolkit

• OpenBox is a comprehensive framework for generalized black-box optimization that supports mixed parameters, multiple objectives, and constraints.
• The system features a modular architecture with components like Optimizer, Advisor, Executor, and Visualizer to accommodate both service and toolkit environments.
• It also includes methods for exact PLNN interpretation and a two-stage 3D annotation pipeline, showcasing versatility across machine learning domains.

OpenBox is a research name shared by several systems in machine learning and optimization. The term most commonly denotes an open-source framework for generalized black-box optimization (BBO), introduced first as a generalized BBO service and later as a Python toolkit, with emphasis on applicability, performance stability, efficiency, usability, mixed parameter types, multiple objectives, and constraints (Li et al., 2021, Jiang et al., 2023). The same name is also used for an exact interpretation method for Piecewise Linear Neural Networks (PLNNs) and for a two-stage automatic 3D bounding-box annotation pipeline (Chu et al., 2018, Lee et al., 1 Dec 2025).

1. Research uses of the name

The name has been used for distinct technical artifacts rather than for a single unified software lineage.

OpenBox usage	Domain	Core characterization
OpenBox (Li et al., 2021)	Generalized BBO service	“BBO as a service,” distributed, fault-tolerant, scalable
OpenBox (Jiang et al., 2023)	Python toolkit for generalized BBO	Open-source Python toolkit with visualization and modular deployment
OpenBox (Chu et al., 2018)	PLNN interpretability	Closed-form exact and consistent interpretation for PLNNs
OpenBox (Lee et al., 1 Dec 2025)	3D annotation	Two-stage open-vocabulary 3D bounding-box auto-labeling pipeline

In the BBO literature, OpenBox is presented as a general-purpose optimization system for expensive objectives under limited evaluation budgets. In the interpretability literature, it denotes a closed-form transformation of a PLNN into exact local linear classifiers. In 3D perception, it denotes an automatic annotation pipeline that combines 2D vision foundation models with LiDAR. In bibliographic use, the name therefore requires domain disambiguation (Jiang et al., 2023).

2. Generalized black-box optimization framework

In its best-known usage, OpenBox targets generalized BBO. A BBO problem is described as optimizing an objective under a limited evaluation budget when the objective is expensive to evaluate and its analytical form is unknown. The generalized setting extends traditional BBO by allowing various parameter types, including ordinal and categorical parameters, multiple objectives, and constraints, including safe or feasibility constraints (Jiang et al., 2023).

The 2021 service paper formalizes the objective as a vector-valued black-box function

$$\bm{f}(\bm{x}): \mathcal{X} \rightarrow \mathbb{R}^{p},$$

with Pareto optimal set

$$\mathcal{P}^* = \{\bm{f}(\bm{x}) \text{ s.t. } \nexists \ \bm{x'} \in \mathcal{X} : \bm{f}(\bm{x'}) \prec \bm{f}(\bm{x})\},$$

and black-box inequality constraints

$$\mathcal{C} = \{ \bm{x} : c_1(\bm{x}) \leq 0, \ldots, c_q(\bm{x}) \leq 0 \}.$$

The feasible Pareto set is defined as

$$\mathcal{P}_{\text{feas}} = \{\bm{f}(\bm{x}) \text{ s.t. } \bm{x} \in \mathcal{C}, \ \nexists \ \bm{x'} \in \mathcal{X} : \bm{f}(\bm{x'}) \prec \bm{f}(\bm{x}),\ \bm{x'} \in \mathcal{C}\}.$$

When $p=1$, this reduces to single-objective BBO (Li et al., 2021).

The motivation for OpenBox is framed as a response to three limitations in existing platforms when applied to generalized BBO: restricted application scope, unstable performance across problems, and limited scalability and efficiency. Traditional tools are described as typically assuming continuous or integer parameters, a single objective, and no constraints, while also offering only a small set of algorithms executed sequentially. OpenBox is positioned as addressing these limitations through broader algorithm coverage, support for mixed types and constraints, and both sequential and parallel execution (Jiang et al., 2023).

3. System architecture and interaction models

The 2023 toolkit paper describes a modular architecture built around four core components: Optimizer, Advisor, Executor, and Visualizer. The Optimizer is the central interface that takes an objective function and a search space, runs the optimization, and returns final results. The Advisor implements optimization logic and suggests new configurations from the search space and stored history. The Executor runs the user’s objective on recommended configurations and returns observations. The Visualizer provides convergence curves, SHAP-based parameter importance analysis, and Pareto front and hypervolume charts for multi-objective tasks (Jiang et al., 2023).

The same paper emphasizes two interaction modes. One is a wrapped service abstraction. The other is an ask-and-tell workflow: build an Advisor with the search space, repeatedly request a configuration, evaluate it externally, create an Observation, update the Advisor, and finally generate an HTML visualization page from the optimization history. This interface is described as providing maximum flexibility for custom execution logic and control of the optimization loop (Jiang et al., 2023).

The 2021 service paper presents a more explicitly distributed architecture. Its roles include a Service Master, multiple Suggestion Service nodes, a Task Database, a REST API, and user-owned evaluation workers. The Service Master handles node management, load balancing, failure detection, and recovery; the Suggestion Service generates suggestions for many tasks in parallel; the Task Database stores task states, histories, and snapshots; the REST API exposes calls such as Register, Suggest, Update, StopEarly, and Extrapolate; and evaluation workers execute trials and report objectives, constraints, and metadata (Li et al., 2021).

Task specification in the service version is expressed through a Task Description Language supporting FLOAT, INTEGER, ORDINAL, and CATEGORICAL parameters, along with defaults, bounds, and inter-parameter conditions such as “x1 active only if x3=‘a3’.” The service also exposes time budget and number_of_trials, and supports sync versus async parallel evaluation policies. Taken together, the 2021 and 2023 papers suggest a progression from a managed, distributed service model toward a more directly embeddable Python toolkit, while retaining the same modular advisor-centered abstraction (Li et al., 2021).

4. Algorithms and optimization mechanisms

OpenBox aggregates a broad set of state-of-the-art algorithms. The BBO line includes Bayesian optimization variants, evolutionary and derivative-free methods, and multi-fidelity or bandit-based methods. The papers explicitly mention SMBO with Random Forest surrogates, TPE-style approaches, GP-based BO, multi-objective BO, constrained and safe BO, batch or parallel BO, Differential Evolution, NSGA-II, surrogate-assisted evolutionary computation, CMA-ES, Hyperband, MFES-HB, and BOHB (Jiang et al., 2023).

In the 2021 service paper, surrogate selection and acquisition choice are made automatically from task characteristics. GP surrogates are used for moderate trials, with Matérn kernel (ARD) for continuous variables and Hamming kernel for categorical variables; product kernels are used for mixed spaces. For large trials or complex spaces, probabilistic random forest from SMAC3 is used to avoid GP’s $\mathcal{O}(n^3)$ scaling and incompatibilities with conditions. The acquisition layer uses EI for single objective, EHVI for multi-objective when the number of objectives is less than five, MESMO and USeMO for larger numbers of objectives, and Probability of Feasibility for constraints, with optional EIPS, PI, and UCB (Li et al., 2021).

The EI used in the service paper is written as

$$\alpha_{\text{EI}}(\bm{x}; D_n) = (\eta - \mu_n(\bm{x})) \Phi(z) + \sigma_n(\bm{x}) \phi(z),$$

where

$z = \frac{\eta - \mu_n(\bm{x})}{\sigma_n(\bm{x})}.$

OpenBox also introduces an algorithm-agnostic parallelization mechanism based on local penalization with imputation of pending evaluations: outcomes for configurations currently being evaluated are imputed with the median of observed values, appended to the dataset, and used to discourage redundant acquisition maxima near pending points (Li et al., 2021).

Transfer learning is another major mechanism in the service version. It uses an ensemble of surrogates across prior tasks with ranking-weighted aggregation generalized to multi-objective settings. The transfer-learning surrogate is defined as

$$M^{\text{TL}} = \text{agg}(\{M^1,\dots,M^K,M^T\}; \mathbf{w}),$$

with uncertainty-weighted aggregation and ranking-based weights estimated via MCMC. This is intended to improve sample efficiency while mitigating negative transfer (Li et al., 2021).

Automatic algorithm selection appears in both BBO papers, but with version-specific thresholds. The service paper states that PRF is selected when conditions are present, when the space has more than 50 parameters, or when the budget exceeds 500 trials; the toolkit paper gives an example rule in which PRF is chosen instead of GP when the search space has more than ten parameters or the number of trials exceeds 300. A plausible implication is that automatic policy details evolved across OpenBox versions while preserving the same design principle: algorithm choice should depend on task structure rather than be fixed a priori (Jiang et al., 2023).

5. Empirical evaluation and deployments

The empirical record reported for OpenBox is broad. The 2023 toolkit paper evaluates constrained multi-objective optimization on CONSTR, using hypervolume difference from the optimum, and evaluates hyperparameter tuning of LightGBM on 24 OpenML datasets using the rank of the best-achieved accuracy across optimizers. In the reported figures, OpenBox shows superior convergence speed and stability on CONSTR, achieves a median performance rank of 1.25 on the LightGBM study, and ranks first in 12 out of 24 datasets (Jiang et al., 2023).

The 2021 service paper reports experiments on 12 mathematical functions, constrained and unconstrained, single-objective and multi-objective, as well as AutoML for LightGBM and LibSVM on 25 OpenML datasets. Reported findings include more than 10× speedups over baselines on Ackley 16d and 32d, OpenBox being the only method finding sufficient feasible configurations on Keane 10d, stable behavior on SRN where HyperMapper and BoTorch fail to produce enough feasible suggestions, and best median rank in AutoML across 25 datasets. In parallel experiments on Optdigits with a 600s budget, async execution with 8 workers gives the best results and exhibits approximately 8× speedup over sequential execution. In multi-fidelity experiments on Covtype, OpenBox achieves a 3.8× speedup over SMAC3 with comparable convergence, and additional AutoML cases report 2.0–3.3× speedups over the best baseline (Li et al., 2021).

Beyond benchmarking, OpenBox is described as having been embedded in research and industrial systems such as SGL for graph learning and DBTune for database tuning, with deployments at Tencent and ByteDance. The toolkit is available via PyPI as pip install openbox, has source code at the PKU-DAIR GitHub repository, documentation at open-box.readthedocs.io, and is reported as tested on Linux, macOS, and Windows in version 0.8.3 (Jiang et al., 2023).

6. Competition use in automated hyperparameter optimization

A concrete application of OpenBox appears in the CIKM 2021 AnalyticCup automated hyperparameter optimization challenge, where the system underpinned a first-place solution in both the preliminary and final contests, with reported results of 0.938291 and 0.918753, respectively (Jiang et al., 2021).

In that work, OpenBox is instantiated as a Bayesian optimization pipeline over numerical hyperparameters defined in ConfigSpace and scaled to $[0,1]$. The surrogate is a Gaussian Process with Matérn 5/2 kernel, the acquisition function is Expected Improvement, and acquisition optimization is performed by random_scipy, which combines Monte Carlo global start points and local one-change perturbations before L-BFGS-B refinement. The setup also uses an initial design strategy called random_explore_first, multi-suggestion generation by repeated acquisition optimization restarts, and a fixed 10% random exploration probability (Jiang et al., 2021).

The final contest introduced a heuristic early stopping strategy motivated by noisy partial evaluations. Each configuration had 14 evaluation iterations, with the first 13 providing partial reward estimates and a 95% confidence interval. Early stopping began only after 40 configurations had been fully evaluated; at iteration 7, a running configuration was stopped if its reward was not in the top half relative to previously observed iteration-7 rewards. Fully evaluated configurations were used for BO fitting, while stopped runs were imputed with the median of observed final rewards to discourage revisiting poor regions (Jiang et al., 2021).

This competition paper is narrower than the service and toolkit papers, but it demonstrates how OpenBox’s components can be specialized for a tightly budgeted industrial HPO setting. It also shows that the system’s abstractions were sufficiently concrete to support contest-specific scheduling, batch suggestion, and early-stopping logic without changing the underlying BO framework (Jiang et al., 2021).

7. Other systems named OpenBox

In a distinct line of work, “OpenBox” denotes a closed-form interpretability method for Piecewise Linear Neural Networks. The method transforms a PLNN into a mathematically equivalent set of linear classifiers, one per activation region, so that within a fixed region the logits satisfy

$$\mathcal{P}^* = \{\bm{f}(\bm{x}) \text{ s.t. } \nexists \ \bm{x'} \in \mathcal{X} : \bm{f}(\bm{x'}) \prec \bm{f}(\bm{x})\},$$0

The input space is partitioned into convex polyhedral regions, dominant features can be read from the local linear coefficients, and the method defines both exactness and consistency guarantees. The paper further studies non-negativity and sparsity constraints as means of improving interpretability. Reported experiments include exact recovery of logits on Fashion-MNIST binary tasks, perfect similarity for many nearest-neighbor instances sharing the same region, and representative test accuracies such as 0.999 for PLNN on FMNIST-1 and 0.868 for PLNN on FMNIST-2, with PLNN-NS maintaining comparable accuracy while improving interpretability (Chu et al., 2018).

A further and unrelated usage appears in 3D perception. The 2025 paper presents OpenBox as a two-stage automatic annotation pipeline for open-vocabulary 3D object detection. Stage 1 performs cross-modal instance alignment by associating instance-level cues from 2D images with 3D point clouds, using Grounding DINO, SAM2, Patchwork++, HDBSCAN, context-aware refinement, and surface-aware filtering. Stage 2 categorizes instances into static rigid, dynamic rigid, and deformable using PP score and a one-time ChatGPT-4 rigidity lookup, then generates adaptive bounding boxes with class-specific size statistics, multi-view 3D–2D IoU alignment for static rigid objects, visibility-aware face extension for dynamic rigid objects, and per-frame tight fitting for deformables. Reported results include, on the Waymo Open validation set under camera-frustum-only training, Vehicle AP_3D of 70.49 at IoU 0.5 and 32.41 at IoU 0.7, Pedestrian 57.95 and 17.11, and Cyclist 20.81 and 2.15; on Lyft validation at IoU 0.25, class-agnostic AP_3D of 43.3 in the 0–80m range; and on nuScenes validation, AP_3D of 40.9 for Car, 62.7 for Pedestrian, and 5.2 for Cyclist (Lee et al., 1 Dec 2025).

The coexistence of these systems under one name is a recurrent source of ambiguity. In current research usage, however, the BBO framework is the dominant meaning of OpenBox, especially when the discussion concerns AutoML, database tuning, experimental design, mixed search spaces, multiple objectives, or constrained black-box optimization (Jiang et al., 2023).