Self-HPO: Self-driving Hyperparameter Optimization Systems

• Self-HPO systems adapt various aspects of the hyperparameter optimization process (e.g., configuration, resource allocation, and objective weighting) using real-time feedback to enhance performance.
• It encompasses techniques like portfolio construction, surrogate-based adaptation, resource adaptation, and federated learning for seamless mathematical models and ML applications.
• Model options include: zero-shot methods, surfaced adaptive strategies, and portfolio construction to select configurations in a statistically accommodating fashion related to the application-use case.

Self-HPO denotes systems that autonomously adapt some part of their hyperparameter-selection, optimization, execution, or agent-environment interaction process using feedback generated during training or evaluation. The term encompasses several technically distinct regimes: reusable zero-shot portfolios, adaptive surrogate-based HPO, multi-fidelity resource allocation, online bandit control, federated single-shot HPO, self-guided label-free tuning, adaptive optimization objectives, and joint optimization of an agent policy and its execution harness. It is therefore not a single algorithmic family. Its defining property is the delegation of at least part of HPO or optimization-process design to an adaptive mechanism, while the search space, objective, model family, or computational budget may remain externally specified.

1. Conceptual scope and taxonomy

Conventional HPO treats a dataset and algorithm as a black-box optimization problem. Given a search space $\Theta$ and generalization loss $\ell(\theta)$, the objective is

$\min_{\theta\in\Theta}\ell(\theta).$

Bayesian optimization, SMAC, BOHB, random search, and related methods typically select configurations sequentially, using observations from the current task to update a surrogate or allocation policy. Self-HPO extends this paradigm by allowing the system to adapt not only the candidate configuration, but also the search policy, resource allocation, evaluation fidelity, objective weighting, or agent-environment interface.

Several forms of Self-HPO can be distinguished:

1. Portfolio and zero-shot Self-HPO: a fixed portfolio of configurations is learned offline across historical datasets. For a new task, configurations are evaluated in parallel and the best observed member is selected. “Practical and sample efficient zero-shot HPO” formulates this as complementary portfolio construction rather than task-specific online optimization (Winkelmolen et al., 2020).
2. Surrogate-adaptive Self-HPO: a surrogate is updated from observations and selects subsequent configuration–task evaluations according to expected portfolio improvement per training job. The HYPPO framework applies this principle to deep-learning models while explicitly modeling predictive uncertainty and stochastic training variability (Dumont et al., 2021).
3. Resource-adaptive Self-HPO: the system determines how much fidelity or training resource candidates should receive and when the search should terminate. PASHA progressively exposes additional resource levels only when top-ranking configurations remain unstable (Bohdal et al., 2022).
4. In-run optimization control: the optimizer changes hyperparameters during a single training trajectory rather than launching independent runs. ULTHO represents hyperparameter values as hierarchical bandit arms and uses return feedback to select them online within one deep-RL run (Yuan et al., 8 Mar 2025).
5. Objective- or update-adaptive optimization: the system adjusts coefficients inside the optimization objective based on current training statistics. Adaptive Hysteretic Policy Optimization changes the weight assigned to negative-advantage updates according to the batch-level balance of positive and negative advantages (Sana et al., 28 May 2026).
6. Environment- or harness-adaptive optimization: the system optimizes the mechanism through which an agent interacts with its environment as well as the agent policy itself. Self-HPO for coding agents alternates harness search and policy reinforcement learning, using the evolving policy to propose harness modifications (Wei et al., 16 Oct 2026).

These regimes differ in what is adapted. A portfolio method adapts initialization; HYPPO adapts configuration selection; PASHA adapts resource exposure; ULTHO and A-HPO adapt the training process online; and coding-agent Self-HPO adapts both the policy and its execution environment.

2. Portfolio construction and surrogate-guided adaptation

Zero-shot HPO constructs a small set

$S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$

such that at least one configuration is expected to perform well on a previously unseen dataset. Its portfolio objective is

$$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$

where $D$ is the number of meta-training datasets. The minimum over configurations rewards complementarity: different configurations may be effective on different datasets, and no individual configuration must dominate universally.

The offline workflow selects historical datasets, generates $N$ candidate configurations, evaluates configuration–dataset pairs, forms a performance matrix, and selects $K$ rows that jointly minimize the portfolio objective. Exhaustive evaluation requires $ND$ training jobs. The central problem addressed by the proposed methods is reducing this cost without substantially degrading the resulting portfolio.

The portfolio loss is monotone decreasing and supermodular. Equivalently, the utility

$U(S)=L(\varnothing)-L(S)$

is monotone increasing and submodular. Greedy selection repeatedly adds the configuration with the largest marginal improvement. Exact subset selection is NP-hard, whereas greedy selection obtains the standard $\ell(\theta)$0 approximation for monotone submodular maximization under exact marginal selection.

The paper introduces two methods. Offline Bayesian Optimization (OBO) uses a random-forest surrogate to model

$\ell(\theta)$1

with representations for numerical, categorical, and conditional hyperparameters together with a dataset identifier or representation. OBO selects not only a candidate configuration but also its intended portfolio position and the dataset on which to evaluate it. The selection criterion is expected portfolio improvement per remaining training job. Dataset selection then targets the dataset on which evaluating the chosen configuration is expected to reduce portfolio loss most.

This constitutes Bayesian optimization over a set-selection objective rather than a single-configuration objective. The surrogate is used to predict both observed and missing entries of the configuration–dataset performance table. OBO therefore distinguishes between choosing what to evaluate and choosing where to evaluate it.

The second method, multi-fidelity zero-shot HPO, avoids a learned surrogate. Fidelity is the number of meta-datasets on which a configuration has been evaluated. HyperBand provides successive-halving-style allocation, while submodular portfolio selection evaluates a candidate according to its marginal coverage relative to previously selected configurations. For a candidate considered for portfolio position $\ell(\theta)$2,

$\ell(\theta)$3

The method allocates resources asynchronously across portfolio positions. Early positions generally produce larger reductions in loss, whereas later positions operate on residual loss after earlier configurations have covered many datasets. The resource-allocation strategy therefore exploits diminishing returns while retaining asynchronous flexibility.

Across XGBoost, LightGBM, CatBoost, MLP, and AutoML settings, OBO was consistently better than the surrogate baseline under the reported comparisons. Its combined improvements relative to that baseline were $\ell(\theta)$4, $\ell(\theta)$5, and $\ell(\theta)$6 for portfolio sizes $\ell(\theta)$7, respectively. Multi-fidelity selection improved over the naive baseline by $\ell(\theta)$8, $\ell(\theta)$9, and $\min_{\theta\in\Theta}\ell(\theta).$0 for those portfolio sizes, although it was worse on CatBoost. These results support the interpretation that OBO is preferable when a suitable surrogate can be maintained, whereas multi-fidelity selection is useful when the space is highly heterogeneous or rapidly changing (Winkelmolen et al., 2020).

3. Uncertainty, parallelism, and resource adaptation

HYPPO organizes HPO into a deep-learning training layer, an outer HPO layer, a surrogate-selection layer, and an HPC execution layer. Its outer objective can combine predictive loss and variability:

$\min_{\theta\in\Theta}\ell(\theta).$1

where $\min_{\theta\in\Theta}\ell(\theta).$2 is predictive loss and $\min_{\theta\in\Theta}\ell(\theta).$3 is uncertainty or loss variability. The implementation uses radial-basis-function and Gaussian-process surrogates. RBF ensembles estimate uncertainty by sampling response surfaces from observed confidence intervals, while GPs provide predictive means and variances directly.

HYPPO estimates epistemic variability using repeated training and Monte Carlo dropout. Repeated training captures variation among independently optimized models; dropout supplies additional stochastic predictions without extensively retraining the same architecture. Candidate scoring combines predicted performance, predictive uncertainty, and, for RBF models, distance from previously evaluated points. Evaluations are launched asynchronously, and the surrogate is updated whenever an individual training job finishes.

The framework provides three levels of parallelism: different configurations can run concurrently; repeated trials for one configuration can run concurrently; and resources can be distributed across the data or model computation of a single training operation. GNU Parallel and SLURM coordinate execution, while PyTorch distributed and Horovod support framework-specific distributed initialization. This nested parallelism separates configuration exploration, uncertainty estimation, and model-training acceleration.

The reported HYPPO results indicate an approximately order-of-magnitude reduction in the number of evaluations required to reach the most promising hyperparameter region and up to a two-orders-of-magnitude throughput or time-to-solution improvement through asynchronous nested parallelism (Dumont et al., 2021). These gains are conditional on surrogate quality, initial-design adequacy, resource availability, and the suitability of the search space.

PASHA addresses a different limitation: the user must ordinarily provide a maximum resource $\min_{\theta\in\Theta}\ell(\theta).$4 for ASHA-style optimization, although the appropriate convergence resource is unknown. PASHA begins with a small maximum resource and progressively adds rungs when the ranking of strong candidates remains unstable. If $\min_{\theta\in\Theta}\ell(\theta).$5 is the minimum resource and $\min_{\theta\in\Theta}\ell(\theta).$6 is the reduction factor, the initial maximum is

$\min_{\theta\in\Theta}\ell(\theta).$7

and successive expansions follow

$\min_{\theta\in\Theta}\ell(\theta).$8

Candidate-level stopping retains only approximately the best $\min_{\theta\in\Theta}\ell(\theta).$9 fraction of a rung. Search-level stopping occurs when the top-rung ranking is sufficiently stable. PASHA uses soft ranking to tolerate performance differences consistent with observed training noise. The threshold is estimated from repeatedly reversing candidate pairs, with the main experiments using the $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$0th percentile of observed performance differences.

This creates a distinction between fixed-budget multi-fidelity HPO and self-adaptive resource allocation. PASHA does not determine which hyperparameters should be searched or how configurations should be generated; it controls the resource axis. In NASBench-201, it achieved speedups of $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$1, $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$2, and $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$3 on CIFAR-10, CIFAR-100, and ImageNet16-120, respectively, with broadly comparable accuracy. On the WMT task, it achieved a reported $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$4 speedup relative to ASHA while using an average maximum of approximately $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$5 epochs rather than approximately $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$6 (Bohdal et al., 2022).

4. Online and federated Self-HPO

ULTHO formulates online deep-RL hyperparameter adaptation as a multi-armed bandit with clustered arms. A cluster represents a hyperparameter type, such as learning rate or batch size, and the values within the cluster are intra-cluster arms. At each update point, ULTHO first selects a cluster and then selects a value within that cluster using UCB scores:

$S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$7

The feedback signal is a sliding-window average of value-network return estimates. Only one hyperparameter is changed at a time, which is intended to improve attribution and reduce destabilization in a non-stationary RL process. The method uses predefined categorical values for learning rate, batch size, value-loss coefficient, entropy-loss coefficient, and the number of update epochs.

ULTHO operates within one RL training trajectory rather than maintaining a population of independently trained agents. The system collects rollouts, computes generalized advantage estimates and returns, selects a hyperparameter cluster and value, applies the selected value to the next policy/value update, and updates the corresponding FIFO return estimates. Relay-ULTHO further focuses on the most frequently selected and least frequently selected clusters, although this restriction can be harmful in highly non-stationary Procgen environments.

On Procgen, ULTHO achieved aggregate mean performance of $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$8 and aggregate IQM of $S=\{\theta_{i_1},\ldots,\theta_{i_K}\}$9, compared with $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$0 and $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$1 for PPO, using $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$2 million environment steps rather than the $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$3 million total steps used by the population baselines. The method outperformed vanilla PPO in 15 of 16 Procgen environments. Its benefits were mixed on MiniGrid, where normal ULTHO did not outperform PPO overall (Yuan et al., 8 Mar 2025). The supplied results do not provide a ULTHO-specific regret bound, convergence theorem, or formal non-stationary-bandit guarantee.

Adaptive Hysteretic Policy Optimization operates at a finer level: it adapts the weighting of policy-gradient contributions within a GRPO-style objective. In sparse verifiable-reward settings, negative advantages can substantially outnumber positive advantages early in training. HPO assigns full weight to nonnegative advantages and a fixed weight $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$4 to negative advantages. A-HPO replaces the fixed coefficient with

$$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$5

where $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$6 and $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$7 are the batch frequencies of positive and negative advantages. The reported experiments use $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$8.

A-HPO also removes group standard-deviation normalization and replaces per-response length normalization with mean-response-length normalization. This separates two mechanisms: hysteretic weighting reduces domination by abundant negative updates, while common length normalization removes response-level length asymmetry. On TeleLogs, A-HPO achieved reward $$L(S)=\frac{1}{D}\sum_{d=1}^{D}\min_{i\in S}\ell(d,\theta_i),$$9, compared with $D$0 for SAPO, $D$1 for GSPO, and $D$2 for GRPO. Its fixed-weight ablation reached a maximum reward of $D$3 at $D$4, while positive-only and fully symmetric settings achieved $D$5 and $D$6, respectively (Sana et al., 28 May 2026). A-HPO is therefore self-adaptive in a narrow update-control sense, but it does not perform general outer-loop HPO.

FLoRA addresses HPO under federated data constraints. Each party independently performs local HPO, generating pairs $D$7 from its local training and validation data. The parties transmit compact observations to an aggregator, which constructs an approximate global loss surface and selects one shared configuration. Only that configuration is used in the final federated training run.

The aggregator can use a single global model, a global model with uncertainty, a maximum of per-party local models, or an average of per-party local models. The average of per-party local models is generally the strongest aggregate strategy in the reported experiments. Communication during HPO is at most $D$8 records for $D$9 parties and $N$0 local trials, rather than $N$1 complete federated training processes.

FLoRA’s optimality-gap analysis includes three sources of error: distributional mismatch between parties, local search coverage, and local surrogate-fitting error. Under the stated assumptions, the heterogeneity term depends on pairwise Wasserstein distances between local distributions. Experiments on seven OpenML datasets and histogram-based gradient boosting, SVM, and MLP models show that all aggregation methods significantly improved over the default baseline overall, with APLM achieving the strongest aggregate performance and wins/ties/losses of $N$2 (Zhou et al., 2022). FLoRA is single-shot and communication-conscious, but it does not optimize personalized hyperparameters, aggregator hyperparameters, privacy mechanisms, or the model family itself.

5. Label-free and agentic forms of self-adaptation

LFSG applies Self-HPO to subspace clustering without ground-truth labels. It runs a clustering algorithm on an initial hyperparameter grid, obtains pseudo-label partitions, and compares neighboring partitions using pseudo-label ACC or NMI. The interval with the largest neighboring agreement is recursively subdivided. For a single hyperparameter $N$3, the procedure selects

$N$4

where $N$5 is pseudo-ACC or pseudo-NMI.

The method assumes that partition agreement varies smoothly or monotonically in a local region. Ternary refinement evaluates internal points and retains the subinterval with the largest local agreement. Refinement stops when the active interval satisfies a relative-error tolerance; the reported experiments use $N$6. Multiple hyperparameters are handled through coordinate-wise optimization rather than a full joint partition.

LFSG is label-free but does not measure absolute clustering correctness. It uses consistency between algorithm-generated partitions as a proxy for a favorable hyperparameter region. The method is interpretable because subspace bases derived from the resulting partitions can be visualized. Across the reported single-view and multi-view datasets, performance was typically within approximately $N$7–$N$8 of oracle tuning, although the gap reached approximately $N$9 for LRSSC on ORL (Kopriva, 2024). Its limitations include dependence on the initial search range, possible failure under abrupt spectral changes, cluster-label alignment issues for direct ACC, and inability of coordinate-wise search to capture strong parameter interactions.

ASAP combines multiple HPO tools with an LLM-based Judge and a wall-clock-oriented execution system. Its optimizer pool contains GP, TPE, Scikit-Optimize, TuRBO, Optuna, SMAC, DNGO, and LLAMBO. Each tool proposes candidates from the same observation history, and the LLM ranks the candidate pool using in-context performance prediction and an Expected Improvement-style score.

The system also uses prefix-stable prompts and KV-cache reuse, speculative proposal generation during model evaluation, and a Self-Tuner that adjusts the speculation threshold from execution logs. Speculation is accepted when

$K$0

The Self-Tuner updates $K$1 using a quantile of recent relative prediction errors. It adapts only this systems-level execution threshold; it does not alter the optimizer pool, LLM, prompt structure, search space, or underlying optimizer hyperparameters.

Across 28 task–model combinations, ASAP and ASAP-spec reached an aggregate regret floor of approximately $K$2, compared with approximately $K$3 for LLAMBO and approximately $K$4 for TuRBO. ASAP achieved a reported $K$5 median wall-clock speedup on HPOBench relative to the non-speculative system, while the benefit was smaller on PD1 because multi-hour model training dominated the runtime (Guo et al., 23 Jun 2026). ASAP is thus agentic and online-adaptive, but its strongest self-tuning component concerns execution policy rather than the fundamental HPO strategy.

Self-adaptive PSRO extends the same idea to population-based game solving. Parametric PSRO exposes meta-solver weights $K$6, best-response initialization $K$7, and response-training budget $K$8 as hyperparameters. SPSRO learns a policy that maps PSRO history to the next tuple $K$9. An offline Transformer is trained on previously generated PSRO trajectories and then deployed autoregressively during new runs.

The Transformer observes prior solver weights, initialization settings, response-training budgets, and performance values. It predicts new hyperparameters at each PSRO epoch. Experiments on normal-form and extensive-form games show that Transformer SPSRO generally outperforms fixed-solver baselines and online Optuna-based adaptation in final NashConv, although exact numerical final NashConv tables are not reported in the supplied material (Li et al., 2024). The method is self-adaptive because decisions change during the run, but its meta-solver set, action representation, offline data-generation process, and game family remain externally specified.

6. Policy–environment co-optimization and limitations

The strongest use of the explicit name Self-HPO is the recursive co-optimization of coding-agent policies and harnesses. A coding agent is represented as a policy $ND$0 and harness $ND$1, with objective

$ND$2

where $ND$3 is the executable-verifier outcome.

Self-HPO alternates two stages. In the H-step, the current policy is frozen and proposes harness candidates based on observed failures. Six candidates are evaluated on a fixed search set, and the best-so-far harness is retained. In the P-step, the selected harness is frozen and the policy is updated using reinforcement learning. Tasks unresolved during policy training become the next harness-search set. The resulting recursion is

$ND$4

$ND$5

$ND$6

The optimized harness includes a budget valve, empty-patch guard, contract-grounding intervention, and patch fallback. These mechanisms address premature termination, empty submissions, incorrect API assumptions, malformed patches, and omitted files. The policy optimizer uses Balanced Adaptive Rollout and GRPO, with eight rollouts per task and selection of four reward-balanced trajectories.

With Qwen3.5-35B-A3C as the base policy, Self-HPO improved average resolve rate from $ND$7 to $ND$8 across SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro. On SWE-Bench Verified it reached $ND$9, while the policy-only and harness-only baselines reached $U(S)=L(\varnothing)-L(S)$0 and $U(S)=L(\varnothing)-L(S)$1. An external Opus 4.8 harness proposer increased the average to $U(S)=L(\varnothing)-L(S)$2 and the SWE-Bench Verified result to $U(S)=L(\varnothing)-L(S)$3 (Wei et al., 16 Oct 2026).

Cross-evaluation indicates complementarity between components. The optimized harness improved the base policy from $U(S)=L(\varnothing)-L(S)$4 to $U(S)=L(\varnothing)-L(S)$5, and the optimized policy reached $U(S)=L(\varnothing)-L(S)$6 under the seed harness. Together, the optimized harness and policy reached $U(S)=L(\varnothing)-L(S)$7, exceeding the sum of their individual gains by $U(S)=L(\varnothing)-L(S)$8 percentage points. Self-HPO also improved Pass@1 reliability and Pass@8 capability, while the evolving hard-task search set outperformed a fixed search set.

Across these forms, Self-HPO remains constrained by several recurring limitations:

• Search-space dependence: most methods require externally defined hyperparameter domains, candidate values, model families, or harness modification interfaces.
• Objective dependence: validation loss, return, clustering agreement, verifier success, NashConv, or execution time may be imperfect proxies for deployment quality.
• Non-stationarity: online feedback changes as the policy, primitive, dataset, population, or training stage evolves. Sliding windows, soft rankings, adaptive thresholds, and failure-conditioned search mitigate but do not eliminate this issue.
• Distribution shift: zero-shot portfolios and federated loss surfaces depend on transferability between source and target distributions. Large Wasserstein distances, changing software versions, or new architectures can invalidate prior observations.
• Noise and sparse feedback: stochastic training, sparse rewards, binary verifiers, and delayed effects make attribution difficult. A-HPO’s sign-frequency rule, ULTHO’s return windows, and PASHA’s soft ranking are empirical approximations rather than universal guarantees.
• Computational overhead: surrogate fitting, repeated evaluations, offline trajectory generation, harness testing, local federated search, and LLM inference can offset optimization gains when model evaluations are cheap.
• Incomplete autonomy: the systems generally do not autonomously define the objective, search space, model family, privacy policy, stopping semantics, or deployment criterion. Even the most explicit Self-HPO framework alternates optimization under externally supplied task distributions, verifiers, and reinforcement-learning procedures.

Self-HPO is consequently best understood as a layered research area rather than a single optimization algorithm. Its common design pattern is feedback-driven adaptation of the optimization process itself: reusable portfolios provide cold starts; surrogates and bandits select configurations; multi-fidelity schedulers control resource exposure; adaptive objectives modify training updates; and agentic systems co-optimize policies with the environments or harnesses through which those policies act. The degree of self-adaptation depends on which layer is allowed to change, from a single coefficient or execution threshold to an evolving policy–harness pair.