Generative Refinement for Low-Budget Black-Box Optimization

Abstract: Black-box optimization is a fundamental science and engineering tool that makes it possible to optimize objectives without gradient information. Unfortunately, as it often requires many function evaluations, it can be challenging when each one is costly. This is especially true when the evaluation function is noisy or failure-prone, and when high-performing solutions are confined to thin, curved, or disconnected regions of the search space. Existing methods leveraging generative models to navigate these subspaces are built to sample from reward-aligned distributions. As a result, they require a large number of evaluations to align their sampler effectively, making them impractical in low-budget settings. We propose SPARROW, an algorithm that completely decouples the generative prior from the reward signal. SPARROW can use any sampler with a known corruption process and trained on unevaluated data, as a fixed, structured proposal operator. Optimization proceeds by rank-based guidance over an archive of evaluated candidates. SPARROW can navigate complex geometries, handle unreliable reward signals, and perform effective optimization under very low evaluation budgets. We provide asymptotic convergence guarantees over the sampler support and demonstrate strong empirical performance on problems with unreliable rewards and geometrically complex landscapes.

• The paper introduces SPARROW, a novel framework that decouples generative priors from reward-driven updates to efficiently optimize in low-budget settings.
• It leverages rank-guided mutation and an archival selection mechanism to navigate thin, curved, and disconnected solution manifolds in high-dimensional spaces.
• Empirical results across synthetic, neural policy, and aerodynamic tasks show SPARROW outperforms baselines like CMA-ES and GP with higher median performance and lower variance.

Generative Refinement for Low-Budget Black-Box Optimization

Motivation and Context

Black-box optimization (BBO) is critical in scenarios where derivatives of the objective function are inaccessible or expensive to compute, typified by problems such as materials discovery, engineering design, and aerodynamic simulation. In realistic settings, constraints on the evaluation budget, compounded by unreliable or noisy objective signals and the complexity of the search space, severely limit the applicability of classical strategies. When the solution manifold is thin, highly curved, or disconnected, established BBO routines—including CMA-ES, Gaussian Process (GP)-based Bayesian optimization, and their trust-region derivatives—are fundamentally ill-suited due to poor sampling efficiency and structural misalignment with the feasible subspace.

The proliferation of generative models, notably diffusion and flow matching architectures, has motivated their application as samplers to capture complex geometric regularities in design spaces. However, existing approaches rely on learning reward-aligned distributions, necessitating numerous objective evaluations to shift the sample density toward optimal regions. This renders them impractical for low-budget regimes, where only a handful of evaluations (e.g., 100) is permissible. This paper addresses that limitation by introducing a fundamentally distinct framework, termed SPARROW (Sequential Proposal via Archival Rank-based Refinement for Optimization under Weak feedback), which decouples prior generative modeling from reward-driven adaptation.

Figure 1: One iteration of SPARROW on a thin tube optimization problem, with rank-guided directional mutation and refinement restoring support to the manifold.

Methodology

SPARROW proceeds by leveraging an unconditional, fixed generative sampler trained on unlabeled data. The method utilizes the forward corruption and reverse refinement trajectories inherent to the sampler, treating the generative prior solely as a structured proposal operator. Notably, the generative model is never updated or fine-tuned during optimization; all reliance on reward signal is confined to a rank-based selection and mutation loop over an archive of past evaluations.

The core algorithm maintains an archive of candidate solutions, ranked strictly by objective value. At each iteration, a parent is probabilistically selected with bias toward high-ranking entries (softmax with tunable selection pressure $\beta$). The corruption level for mutation, $t$, is computed as a nonlinear function of parent rank (exponent $\gamma$). A rank-guided directional step is formed by displacing the parent along a direction spanned by a random pair of archive members, with the magnitude coupled to corruption intensity. This child candidate is corrupted to noise level $t$ and deterministically mapped back to data space via the generative trajectory, producing a geometry-consistent proposal. All candidates and evaluations are archived, increasing coverage and enabling robust rank estimation under noise.

This paradigm enforces invariance to monotone transformations of the objective and confers robustness to stochastic and failure-prone evaluations—a salient advantage in high-uncertainty or simulation-based tasks.

Asymptotic Analysis

SPARROW is shown, under mild support and irreducibility conditions, to converge asymptotically (with probability one) to the maximal value attainable within the generative prior support. The exponential selection scheme assigns nonzero probability mass to the worst-ranked archive element at every step, guaranteeing persistent, unbiased exploration of the prior distribution. This ensures that any measurable subset with nonzero prior probability is eventually sampled, conditional on infinite budget and noiseless evaluation (see Appendix for detailed proof).

Experimental Evaluation

The approach is benchmarked on three distinct tasks, incrementally increasing in complexity and realism:

1. Synthetic Thin Tube (D=8–128): Optimization target lies on a highly curved 1D submanifold in high-dimensional ambient space ($D=8$ to $128$). The tube occupies a measure-zero subset; ambient-space strategies yield negligible success.
2. HopperController (D=5126): Optimization in the full parameter space of a neural policy, with a highly fragmentary feasible set.
3. Airfoil Aerodynamic Shape Optimization (D=256): Maximizing $C_l/C_d$ via XFOIL simulation, subject to high simulation failure rates and geometric feasibility constraints.

Key baselines include CMA-ES, GP, TuRBO, and Random Diffusion Sampling (RDS) from the unconditional prior. Methods that require reward-driven retraining or surrogate adaptation are explicitly excluded for exceeding feasible evaluation budgets ($B=100$).

SPARROW demonstrates strong and consistent performance across all tasks, with significant margins over the closest baselines.

Figure 2: Median objective and interquartile range as a function of budget on the 64D tube, HopperController, and airfoil optimization.

For the 64D thin tube, SPARROW is the only method to consistently approach the global optimum under a low evaluation budget; ambient-based optimizers collapse due to infeasible proposal rates. On HopperController, the approach finds high-performing policies despite the curse of dimensionality and disconnectedness. In the airfoil task, SPARROW achieves both the highest median performance and lowest performance variance, indicating improved reliability under frequent solver failures compared to all baselines.

Figure 3: Median objective and IQR as a function of dimensionality $D$ for the tube problem; the performance gap between SPARROW and structure-unaware sampling widens with $D$.

Notably, as dimension increases and the feasible manifold occupies a vanishing fraction of ambient space, performance gains from rank-guided sampling over random diffusion-based proposals become more pronounced. This confirms the central claim that SPARROW's archive-driven rank-based guidance repurposes geometric priors more effectively as the geometry becomes more challenging.

Hyperparameter Sensitivity and Robustness

Ablation analysis reveals low to moderate sensitivity to the key hyperparameters $t$0 (selection pressure), $t$1 (corruption exponent), and $t$2 (step size), with $t$3 being most impactful on performance consistency. Importantly, the approach supports effective transfer of hyperparameters across domains with simple heuristics, circumventing the need for exhaustive tuning.

Figure 4: Hyperparameter sweep for the 64D tube: median objective across budget for different $t$4, $t$5, $t$6 (default values: $t$7, $t$8, $t$9).

Theoretical and Practical Implications

The methodology reframes the intersection of generative models and black-box optimization: rather than incurring high evaluation cost to learn reward-aligned posterior samplers, it advocates for the separation of generative modeling (unsupervised, cheap, data-driven) and optimization (archive-driven, reward-agnostic, rank-based), improving sample efficiency in stringent evaluation settings. The architecture is model-agnostic: it only requires access to corruption and refinement operators and is compatible with any generative prior (diffusion, flow-matching, etc.) equipped with a noise–refine decomposition.

From a theoretical standpoint, the asymptotic consistency over the sampler support demonstrates that, conditional on the expressiveness of the generative prior, optimal solutions are approachable. From a practical perspective, the empirical results indicate that SPARROW may be uniquely suited for optimization of objectives fundamentally misaligned with data distribution, especially when high-performance regions are both scarce and structurally complex.

Limitations and Future Directions

The primary limitation is that reachable optima are restricted to the support of the generative model; optimization cannot transcend the representational limitations of the learned prior. The convergence results assume noiseless evaluations and infinite budget; practical, finite-budget sample efficiency is supported by empirical observations but remains analytically uncharacterized in the noisy case. Hyperparameter selection, while robust in presented cases, could benefit from adaptive strategies. Extending the paradigm to multi-modal optimization, active generative model adaptation, or integration with informative priors about the objective landscape are promising avenues.

Conclusion

SPARROW establishes a new principled regime for low-budget black-box optimization in high-dimensional, geometrically nontrivial domains where classical and existing generative-model-based optimizers falter. By decoupling reward signals from the generative prior and leveraging purely rank-based, archive-guided mutation and selection, the method achieves robust sample-efficient search and favorable scaling with problem complexity. The implications are relevant for real-world design and scientific discovery tasks where structure is present but reward information is insufficient to shape a proposal distribution within feasible budgets.