Closed-Loop Optimization

• Closed-loop optimization is an iterative method that uses real-time feedback to update control inputs and maximize system performance.
• It integrates experimental measurements with computational decision-making to adapt to nonlinearities, disturbances, and parameter uncertainties.
• Its diverse architectures, including hardware-in-the-loop and data-driven approaches, are essential in fields like optics, robotics, and supply chain management.

Closed-loop optimization is an iterative methodology in which an optimization algorithm interacts in real-time with a physical or simulated system, using feedback to sequentially update decisions or control signals in order to optimize a specified performance metric. This paradigm tightly couples experimentation (or actuation) and computational decision-making, enabling adaptive responses to system nonlinearities, disturbances, parameter uncertainty, and time-variation. Closed-loop optimization encompasses a diverse array of architectures and is pivotal in fields ranging from optics to industrial process control, robotics, supply chain design, and large-scale infrastructure management.

1. Fundamental Principles and Formal Structure

At its core, closed-loop optimization can be formally characterized as an iterative process where the optimizer updates a policy, control input, or decision variable by incorporating feedback from the measured output or system performance. In mathematical terms, at each iteration $k$, the system executes an input $u_k$ (often generated by a controller or optimizer) and the resulting system response $y_k$ or cost $J(u_k)$ is observed. This feedback then informs the selection of the next input $u_{k+1}$, via either direct model-based updates, data-driven approaches, or combinations thereof.

A canonical example is the phase optimization problem in fast wavefront shaping (Blochet et al., 2017):

• The system state is encoded by an $N$-dimensional phase vector $\Phi\in[0,2\pi)^N$ applied to a spatial light modulator (SLM).
• The objective is to find $\Phi^* = \arg\max_{\Phi} J(\Phi)$, with $J(\Phi)\equiv|E_\text{out}(\Phi)|^2$ measured via photodetector.
• No analytic model relating $\Phi$ to $J$ exists; instead, an iterative mode-wise update algorithm maximizes $J$ using direct in situ measurements as feedback.

More generally, closed-loop optimization problems frequently lack full knowledge of the plant mapping $u\mapsto J(u)$ or are subject to process/model uncertainties, motivating the need for feedback architectures over pure open-loop (i.e., precomputed) optimization.

2. Representative Closed-Loop Optimization Architectures

Implementation of closed-loop optimization spans a spectrum of methodologies, including:

a. Hardware-in-the-loop iterative feedback

Physical actuators or real-world experiments are embedded directly in the optimization loop. Typical in wavefront shaping or robotic process control, the optimizer performs trial updates, observes the resultant physical or measurement data, and adapts further decisions accordingly. Example: high-speed FPGA-driven feedback controllers for MEMS-SLM phase optimization at kHz rates to focus light through time-varying scattering media (Blochet et al., 2017).

b. Data-driven and surrogate-based closed-loop frameworks

When direct experiments are expensive, data-efficient optimization is used. Bayesian optimization with surrogate modeling (e.g., Gaussian processes) is a prevalent approach, including constrained optimization (CONFIG, VABO) with theoretical guarantees for closed-loop global optimality even in the presence of black-box constraints (Xu et al., 2022, Xu et al., 2021).

c. Closed-loop control policy learning

Control laws are parameterized (e.g., via neural networks) and optimized using system trajectories and performance feedback. For discontinuous control systems, open-loop waveform optimization can be closed by fitting a neural network to observed optimal control actions, yielding a robust state-feedback policy superior to the open-loop solution (Zarychta et al., 2022). Similarly, model-predictive control policies can be optimized by backpropagation through the closed system loop, directly tuning cost and constraints for best realized performance (Zuliani et al., 2023, Zuliani et al., 2024).

d. Closed-loop planning as embedded online optimization

For differentially flat robotic systems and motion planning, the closed-loop trajectory is implicitly the solution of a time-varying (strongly-convex) constrained optimization problem, and the control law enforces the system's state to track the real-time optimum even under disturbances and constraint variations (Zheng et al., 2023).

e. Hybrid gray-box architectures

Feedback optimization routines that interpolate between model-based gradient descent (using a system model to compute sensitivities) and model-free update rules (using only measurements and finite-difference approximations), adaptively combine both sources of information for robustness and improved sample efficiency (He et al., 2024).

3. Algorithmic Structures and Update Mechanisms

Closed-loop optimization update laws are algorithmically diverse, but share a reliance on measurement-based feedback or real-time system response:

• Gradient-free feedback: Mode-wise or coordinate-wise iterative updates (e.g., Hadamard basis cycling for phase masks), estimation of gradient directions via direct system perturbations, or random search guided by black-box returns (Blochet et al., 2017, He et al., 2024).
• Surrogate-based Bayesian optimization: Update acquisition functions (e.g., expected improvement, lower confidence bound) using GPs fit to concrete system evaluations, propose new candidate parameters, and account for constraint violation budgets (Xu et al., 2022, Xu et al., 2021).
• Gradient-based, differentiable closed-loop optimization: Formulate the closed-loop cost $J(p)$ (where $p$ parameterizes the controller or reference) and differentiate through the entire trajectory and constraint pipeline, using chain rules and implicit differentiation of embedded QP solvers for controller parameter update (Zuliani et al., 2023, Zuliani et al., 2024).
• Meta-learning and transfer in closed-loop Bayesian optimization: Use deep kernel surrogates pretrained on related systems for rapid adaptation in closed-loop optimization of new plants under limited data (Chakrabarty, 2022).
• Ensemble-based or PINN-based closed-loop policies: Physics-informed neural networks trained on the HJB equation or system cost-to-go, with ensemble inference to yield robust closed-loop control under perturbations (Barry-Straume et al., 21 Oct 2025).

4. Performance: Metrics, Scalings, and Experimental Results

Empirical quantification of closed-loop optimization emphasizes adaptation to dynamic environments, convergence speed, robustness, and final performance relative to open-loop benchmarks. Highlighted results include:

• Speed and scalability: Hardware-in-the-loop SLM phase optimization achieves 4.1 kHz single-mode rates, with focus enhancements saturating at $\eta\approx210$ in static media and scaling linearly with the medium's speckle decorrelation time in dynamic samples (Blochet et al., 2017).
• Robustness and generalization: NN-based closed-loop policies in discontinuous systems retain high displacement (>–1% relative loss) under $\pm10\%$ parameter uncertainty, vs. $>–1.4\%$ loss for open-loop; largest gains appear in high-uncertainty (up to +7%) (Zarychta et al., 2022).
• Constraint satisfaction and sample efficiency: CONFIG matches or outperforms constrained expected improvement (CEI) and penalty-based methods, converging to feasible and near-globally optimal solutions with sample complexity $O(T^{-1/2})$ (Xu et al., 2022).
• Multi-criteria tradeoffs: In supply-chain applications, closed-loop network design via multi-objective GAs visualizes trade-offs on the Pareto front between cost, CO₂ emissions, and reliability under uncertainty, with closed-loop adaptability to DC failures and demand fluctuation (Abir et al., 2020).
• Closed-loop planning and control: Embedded time-varying optimization frameworks (e.g., in formation control of mobile robots) achieve real-time adaptation to constraint changes and dynamic objectives, with provable exponential convergence to optimizer trajectories (Zheng et al., 2023).

5. Application Domains and System Integration

Closed-loop optimization strategies are widely applied:

• Optics and Photonics: Real-time wavefront shaping through dynamically scattering media with hardware feedback (Blochet et al., 2017).
• Robotics and Manufacturing: Adaptive trajectory planning and reference design for precise machines, 3D printers, and multi-robot systems (Hoteit et al., 18 Dec 2025, Balula et al., 2020, Zheng et al., 2023).
• Energy and Process Systems: Well control for geothermal reservoir management under geological uncertainty via closed-loop optimization alternating between deep learning surrogates and ensemble-based data assimilation (Wang et al., 2022).
• Complex Networks and Infrastructure: Dynamic reserve sizing and load forecasting in power systems through bilevel closed-loop prediction-optimization frameworks (Garcia et al., 2021).
• Intelligent Prompt Engineering: Synthetic data feedback for prompt optimization in LLMs, leveraging a closed-loop between generator (adversarially finds failure cases) and optimizer (learns to patch prompt) (Yu et al., 26 May 2025).
• Sustainable Supply Chains: Multi-objective closed-loop supply-chain network design under demand and reliability uncertainty (Abir et al., 2020).
• 6G Cyber-Physical Integration: Resource allocation in 6G-enabled UAV robotic systems, minimizing control cost through joint communication, computation, and actuation closed-loop optimization (Fang et al., 2024).

6. Theoretical Guarantees and Limitations

Closed-loop optimization approaches are accompanied by diverse theoretical guarantees, depending on the architecture:

• Convergence rates: Nonconvex gray-box feedback optimization matches the $O(p^2/T^{2/3})$ rate of model-free approaches under inexact sensitivities, and recovers $O(1/T)$ rate when sensitivities are accurate (He et al., 2024). Bayesian methods like CONFIG guarantee objective and constraint violation decay at $O(T^{-1/2})$ (Xu et al., 2022).
• Global optimality: Under regularity assumptions (RKHS smoothness, sub-Gaussian noise), CONFIG's LCB acquisition delivers global optimality up to statistical error and converges robustly even with black-box constraints.
• Robust constraint satisfaction: Scenario-based closed-loop MPC tuning delivers probabilistic guarantees on constraint violation under model mismatch and process disturbance (Zuliani et al., 2024).
• Scalability: For large-scale, high-dimensional settings (e.g., power grid reserve optimization over 13,659 buses), scalable decomposition and derivative-free heuristics are essential for computational practicalities (Garcia et al., 2021).
• Assumptions and practical limitations: Most algorithms require some form of strong convexity or Lipschitz smoothness conditions, and performance can degrade if surrogate models or sensitivity approximations are poor or if the number of control variables is extremely high. Unmodeled delays, hysteresis, or nonconvexity can pose further challenges.

7. Synthesis and Research Directions

Closed-loop optimization provides a unified framework for performance maximization, adaptation to nonstationarity, and integrated uncertainty management across domains. Ongoing research is probing:

• Advanced architectures combining model-based and model-free updates, adaptive meta-learning, and transfer across similar systems for improved data efficiency and robustness (Chakrabarty, 2022, He et al., 2024).
• Scalable and theoretically sound approaches for high-dimensional, distributed, or safety-critical closed-loop optimization.
• Embedded learning and feedback optimization methodologies in cyber-physical and networked systems (e.g., 6G robotics, autonomous vehicles, smart infrastructure) with constraints on computation, communication, and actuation resources (Fang et al., 2024).
• Integration of synthetic data generation, adversarial curriculum learning, and feedback-driven knowledge distillation in automated prompt engineering and adaptive reasoning (Yu et al., 26 May 2025).

Closed-loop optimization continues to augment the capabilities of automated systems in both physical and computational domains, enabling adaptive, resilient, and performance-driven operation under increasing complexity and uncertainty.