Explore improved constraint strategies for online optimization

Investigate additional boundary constraint strategies for online hyperparameter optimization algorithms to achieve better performance on evaluation metrics under evolving data-stream conditions.

Background

The paper integrates five boundary-handling strategies—Boundary, Centroid, Random, Reflection, and Wrapper—into the SPT heuristic optimizer and the MESSPT evolutionary optimizer for online hyperparameter optimization. These strategies repair configurations that fall outside predefined hyperparameter bounds.

The experiments show that the relative effectiveness of the strategies depends on the optimizer and learning task. The authors also observe that the integrated ADWIN drift detector may introduce latency in detecting concept drift and that the standard boundary strategy can underperform compared with other constraint-handling methods. They therefore explicitly identify further exploration of constraint strategies as necessary to improve online optimization performance.

References

While this design feature may introduce latency in detecting concept drift instances, our findings suggest that the boundary constraint solutions underperform when compared with the other constraint strategies. This makes it imperative for further exploration of constraint strategies to achieve better performance on the evaluation metrics.

— Constrained Hyperparameter Optimization for Streaming Data  (2608.24712 - Veloso et al., 25 Aug 2026) in Section 4, subsection 4.3, immediately before Section 5