Closed-Loop Bayesian Optimization

• Closed-Loop Bayesian Optimization is a sequential method that adjusts controller settings based on real-time feedback using Bayesian surrogates.
• It employs Gaussian processes and acquisition functions like Expected Improvement and Lower Confidence Bound to balance exploration and exploitation.
• This approach has been successfully applied in robotics, HVAC, and manufacturing to reduce experimental costs and improve overall system efficiency.

Searching arXiv for papers on closed-loop Bayesian optimization and controller tuning to ground the article in the cited literature. Closed-loop Bayesian optimization (BO) is a sequential optimization paradigm in which the quantity to be optimized is not an open-loop objective but the observed performance of a closed-loop system under feedback. In this setting, an outer BO loop proposes controller gains, reference setpoints, model hyperparameters, or other design variables; an inner control loop or experimental episode is then executed; and the measured closed-loop cost is fed back to update the surrogate and choose the next evaluation. Across model predictive control, robotics, manufacturing, and process systems, the central motivation is the same: the map from tunable parameters to closed-loop performance is nonlinear, noisy, expensive to evaluate, and often only accessible through experiments or long simulations (Lu et al., 2020, Zheng et al., 14 Dec 2025, Hirt et al., 2024, Kavas et al., 2024).

1. Problem formulation and feedback structure

A canonical formulation is

$\theta^* = \arg\min_{\theta \in \Theta} J(\theta),$

where $\theta$ denotes tunable controller or decision variables and $J(\theta)$ is a closed-loop performance measure obtained from an experiment or simulation (Zheng et al., 14 Dec 2025). In controller tuning, the cost is often time-integrated over an episode,

$J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$

with $j(u_t,y_t) \ge 0$ (Stenger et al., 20 Jan 2025). In contextual and constrained settings, the objective becomes $\ell(\theta,z)$ with unknown constraints $g_i(\theta,z)\le 0$, where $z$ denotes time-varying ambient or task context (Xu et al., 2023).

The defining architectural feature is the separation between an inner closed-loop execution and an outer BO tuner. In laser powder bed fusion (LPBF), the inner loop is a real-time in-layer PI controller acting on laser power from pyrometer feedback, while the outer loop uses BO to tune $K_p$ and $K_i$ either online during the build or offline on a bare metal plate (Kavas et al., 2024). In path following, BO proposes the gains $\theta$0, a full-lap experiment is run under closed-loop control, and the observed lap cost updates a Gaussian-process surrogate (Zheng et al., 14 Dec 2025). In HVAC MPC, each BO query corresponds to a year-long closed-loop simulation, so the optimization target is the operational cost induced by the controller rather than any intermediate model-fit criterion (Lu et al., 2020).

Closed-loop BO is therefore broader than gain tuning alone. It has been used to tune MPC back-off terms (Lu et al., 2020), optimize parameters of cost functions, models, and constraints in sequential decision making (Hirt et al., 2024), select kernel structures and hyperparameters of kernel-based models directly from closed-loop control performance (Beckers et al., 2019), and transfer knowledge across similar systems or related tasks through meta-learning or hierarchical surrogates (Chakrabarty, 2022, Hirt et al., 18 Aug 2025).

2. Surrogates, acquisition functions, and the Bayesian layer

The dominant surrogate in closed-loop BO remains the Gaussian process (GP). Representative formulations model the unknown cost as

$\theta$1

with kernels such as ARD Matérn-$\theta$2, isotropic Matérn, or RBF, depending on the application (Zheng et al., 14 Dec 2025, Lu et al., 2020, Kavas et al., 2024). The GP posterior supplies both a mean prediction and quantified uncertainty, enabling the standard exploration–exploitation mechanism used in BO.

Acquisition rules vary by task structure. Expected Improvement (EI) is used in robotic path following (Zheng et al., 14 Dec 2025). Lower Confidence Bound (LCB) is used in HVAC MPC tuning (Lu et al., 2020) and in the LPBF autotuning study as an example acquisition over the GP surrogate (Kavas et al., 2024). For constrained problems, Constrained Expected Improvement (CEI) and contextual CPEI multiply an improvement term by the probability of satisfying modeled constraints (Xu et al., 2021, Xu et al., 2023). When intermediate time-series data are available within an episode, trace-aware Knowledge Gradient (taKG) operates on a joint parameter–fidelity surrogate in which fidelity is aligned with progress along the trajectory (Hirt et al., 2024).

A central methodological question is how Bayesian the surrogate itself should be. A fully Bayesian treatment of GP hyperparameters marginalizes acquisition values over the posterior $\theta$3 rather than conditioning on a point estimate. Across 15 benchmark problems and 4 observational noise settings, fully Bayesian BO using EI with an ARD kernel gave the best performance in the noise-free setting, whereas combining full hyperparameter marginalization with UCB led to over-exploration (Ath et al., 2021). This is not a minor implementation detail: in closed-loop applications, overconfidence or over-exploration directly affects experimental budgets and, in some domains, operational risk.

Recent work has also loosened the GP-centered formulation. ProBO generalizes BO to arbitrary probabilistic programming LLMs through three primitives—inf, post, and gen—so acquisition functions can be computed by Monte Carlo without model-specific derivations (Neiswanger et al., 2019). BORE reformulates EI as a density-ratio, and then as a binary classification problem, replacing explicit probabilistic regression with calibrated class-probability estimation (Tiao et al., 2021). FIBO eliminates both surrogate refitting and acquisition-function maximization at test time by directly sampling from a learned posterior over the optimum point; the paper shows this is equivalent to Thompson sampling and reports a wall-clock efficiency gain of more than $\theta$4 relative to GP-based BO (Carvalho et al., 29 May 2025). Under model misspecification, LOCBO calibrates GP likelihoods by localized online conformal prediction and then denoises the calibrated likelihood to obtain a posterior over the noiseless objective, with theoretical guarantees and improved robustness in noisy settings (Kim et al., 2024).

3. Beyond black-box BO: temporal structure, partial trajectories, and transfer

A recurrent limitation of standard closed-loop BO is that it treats each rollout as a single terminal scalar. Several recent variants replace this black-box abstraction with richer use of the data produced during a closed-loop episode. Time-Series-Informed BO partitions an episode into $\theta$5 segments, treats intermediate cumulative costs as low-fidelity observations, trains a GP over parameter–fidelity space, and applies probabilistic early stopping to terminate unpromising experiments (Hirt et al., 2024). In simulation on the nonlinear cartpole benchmark, the method achieves baseline performance with approximately half the resources, and both UCB- and EI-based early stopping reach high performance in less than $\theta$6 closed-loop iterations, whereas the baseline requires $\theta$7 (Hirt et al., 2024).

A closely related development is Early Stopping BO for controller tuning. Here, an episode is stopped once the accumulated cost already exceeds the best observed full-episode cost, because no eventual improvement is possible under a nonnegative stage cost (Stenger et al., 20 Jan 2025). Since early stopping produces partial trajectories rather than full episode costs, the paper proposes three heuristics: ESBO-C, ESBO-TR, and ESBO-GP. Across five numerical and one hardware experiment, early stopping BO substantially reduces optimization time; ESBO-GP and ESBO-C achieve up to $\theta$8 reduction in total simulation/interaction time, and ESBO-GP gives a $\theta$9 reduction in experimentation time on a three-tank PI controller (Stenger et al., 20 Jan 2025).

A second major direction is to exploit control structure rather than treat the closed-loop map as an undifferentiated scalar black box. A hierarchical surrogate model learns the closed-loop state evolution under controller parameters and then computes the task-specific accumulated cost exactly via known closed-form expressions (Hirt et al., 18 Aug 2025). Because only the GP dynamics map $J(\theta)$0 is learned, the same surrogate can be reused across tasks with different stage-cost weightings or references; the paper proves sublinear regret of order $J(\theta)$1, matching standard black-box BO asymptotically while enabling multi-task transfer (Hirt et al., 18 Aug 2025).

Transfer can also be learned from previously optimized systems. DKN-BO uses Deep Kernel Networks to meta-learn a GP kernel from data collected on similar source systems, then conditions the target-task GP on a small set of target observations without re-training the encoder online (Chakrabarty, 2022). On a nonlinear control example, DKN-BO achieves median regret two orders of magnitude smaller than classical GP-BO after 10 iterations (Chakrabarty, 2022). ProfBO pushes this logic further by encoding optimization trajectories from related source tasks as MDP priors, fitting them into a prior-fitted neural network, and using MAML for rapid adaptation; on few-shot black-box problems, it is designed for regimes such as $J(\theta)$2 evaluations (Li et al., 2 Nov 2025).

4. Safety, feasibility, and violation management

Closed-loop BO often operates under unmodeled constraints. In such settings, the usual distinction is between classical constrained BO, which may incur unlimited violations during search, and safe BO, which is conservative and attempts near-zero violations. Violation-Aware BO (VABO) occupies the intermediate regime by allowing budgeted cumulative violation cost

$J(\theta)$3

for each constraint, while optimizing closed-loop performance (Xu et al., 2021). In industrial vapor compression systems, VABO with budget $J(\theta)$4 learns faster than safe BO and discovers efficient settings $J(\theta)$5 better than the initial safe points in 20 iterations, while keeping cumulative violation within budget (Xu et al., 2021).

VACBO extends the same principle to time-varying ambient conditions by modeling objective and constraints in joint control–context space $J(\theta)$6 and dynamically allocating per-iteration violation sub-budgets (Xu et al., 2023). In a vapor compression case study with changing ambient temperature and humidity, VACBO achieves major energy savings, reported as about $J(\theta)$7 lower average power versus fixed setpoints, while keeping discharge-temperature violations within prescribed small, short-term bounds such as $J(\theta)$8 K (Xu et al., 2023).

A different safety problem arises when evaluations may crash. Local BO with crash constraints assumes that the controller can only be evaluated in an a priori unknown feasible region $J(\theta)$9, while crashes occur in $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$0 (Rohr et al., 2024). CrashGIBO combines local gradient-informed BO with virtual data points that encode crash events in the GP model rather than through a separate classifier or a hand-crafted penalty. The result is a local optimizer that learns the crash boundary while steering search away from it in both simulation and hardware (Rohr et al., 2024).

Process-dependent safety issues also appear in manufacturing. In LPBF, BO-tuned in-layer control reduces overheating, but over-compensation can drive laser power below the robust processing window identified at $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$1 W, causing lack-of-fusion porosities (Kavas et al., 2024). The same study reports a minimum controllable vector length of about 3 mm, determined by melt pool rise time, so short vectors remain intrinsically difficult for feedback control (Kavas et al., 2024). These examples show that closed-loop BO does not remove process limits; it can instead expose them more clearly.

5. Application domains and reported outcomes

The empirical literature spans manufacturing, robotics, process systems, and model-based control. The common pattern is a data-efficient outer loop wrapped around an expensive or safety-critical inner loop.

Domain	BO formulation	Reported outcome
LPBF additive manufacturing (Kavas et al., 2024)	Online and offline BO of in-layer PI gains $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$2	Both approaches converged in $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$3-112 iterations and substantially reduced overheating; power below $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$4 W induced lack-of-fusion porosities
Central HVAC MPC (Lu et al., 2020)	BO over back-off terms $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$5 using year-long simulations	Optimal back-off terms found after 13 year-long simulations; closed-loop costs were reduced
Lyapunov path-following robot (Zheng et al., 14 Dec 2025)	GP-EI tuning of $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$6 on full-lap hardware experiments	Performance improved within 32 trials, including 15 warm-start initial evaluations; best observed $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$7 from a manually tuned baseline $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$8
Kernel-based model selection (Beckers et al., 2019)	BO over kernel choice and hyperparameters using closed-loop cost	Closed-loop BO selected a Gaussian kernel in simulation despite higher prediction loss; tracking error on a 3-DoF robotic arm was reduced by $J(\theta) = \sum_{t=1}^{T} j(u_t,y_t),$9
Early-stopping controller tuning (Stenger et al., 20 Jan 2025)	BO with partial episodes and virtual costs	Up to $j(u_t,y_t) \ge 0$0 reduction in total simulation/interaction time; $j(u_t,y_t) \ge 0$1 reduction in hardware experimentation time

These results support a narrow but important claim: in the reported studies, BO improves sample efficiency relative to manual tuning, grid search, or fixed-length evaluation protocols when the optimized quantity is the realized closed-loop behavior rather than an offline proxy. They do not imply that a single closed-loop BO template dominates across all regimes.

6. Misconceptions, limitations, and current directions

A persistent misconception is that the best model for control is the model with the best open-loop predictive accuracy. Closed-loop model selection for kernel-based models shows the opposite can occur: data-driven kernel selection may choose a model with good prediction loss but high closed-loop cost, whereas closed-loop BO may choose a model with worse prediction loss and much better task performance (Beckers et al., 2019). A second misconception is that “more Bayesian” is always better. The study of fully Bayesian GP hyperparameter marginalization finds that EI with ARD benefits in low-noise settings, but UCB can over-explore and degrade performance (Ath et al., 2021). A third misconception is that external prior information is uniformly helpful. Knowledge-augmented BO using actively elicited expert preferences can significantly speed optimization, yet the same framework can underperform pure BO when the expert is fully biased, with accuracy near $j(u_t,y_t) \ge 0$2 (Huang et al., 2022).

Current directions follow directly from the identified failure modes. In LPBF, safe BO and adaptive objectives are motivated by the need to keep controller actions within processable energy windows and to replace static references with geometry-dependent targets (Kavas et al., 2024). In multi-task control, hierarchical surrogates and meta-learned kernels aim to transfer information across tasks without re-learning from scratch, while higher-dimensional parameterizations motivate possible use of Bayesian neural networks (Hirt et al., 18 Aug 2025, Chakrabarty, 2022). In expensive sequential decision making, time-series-informed BO and early stopping reduce wasted rollout time by exploiting partial traces (Hirt et al., 2024, Stenger et al., 20 Jan 2025). In generic BO infrastructure, direct optimum sampling makes parallel and distributed BO practical because batch suggestions can be generated without acquisition-function optimization (Carvalho et al., 29 May 2025).

This suggests that closed-loop BO is evolving from a purely black-box, terminal-cost optimizer into a family of feedback-driven learning schemes that incorporate temporal traces, structural models, transfer priors, calibrated uncertainty, and explicit safety budgets. The unifying principle remains unchanged: the objective of interest is the behavior of the closed loop itself.