Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Bi-objective Ranking and Selection Using Stochastic Kriging (2209.03919v3)

Published 5 Sep 2022 in stat.ML and cs.LG

Abstract: We consider bi-objective ranking and selection problems, where the goal is to correctly identify the Pareto optimal solutions among a finite set of candidates for which the two objective outcomes have been observed with uncertainty (e.g., after running a multiobjective stochastic simulation optimization procedure). When identifying these solutions, the noise perturbing the observed performance may lead to two types of errors: solutions that are truly Pareto-optimal can be wrongly considered dominated, and solutions that are truly dominated can be wrongly considered Pareto-optimal. We propose a novel Bayesian bi-objective ranking and selection method that sequentially allocates extra samples to competitive solutions, in view of reducing the misclassification errors when identifying the solutions with the best expected performance. The approach uses stochastic kriging to build reliable predictive distributions of the objective outcomes, and exploits this information to decide how to resample. Experimental results show that the proposed method outperforms the standard allocation method, as well as a well-known the state-of-the-art algorithm. Moreover, we show that the other competing algorithms also benefit from the use of stochastic kriging information; yet, the proposed method remains superior.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Pareto set estimation with guaranteed probability of correct selection. European Journal of Operational Research, 292, 286–298.
  2. Stochastic kriging for simulation metamodeling. Operations Research, 58, 371–382.
  3. Multi-objective ranking and selection: Optimal sampling laws and tractable approximations via score. Journal of Simulation, 14, 21–40.
  4. Hypervolume-based multiobjective optimization: Theoretical foundations and practical implications. Theoretical Computer Science, 425, 75–103.
  5. A preliminary study on handling uncertainty in indicator-based multiobjective optimization. In Applications of Evolutionary Computing (pp. 727–739). Springer Berlin Heidelberg.
  6. Quantifying uncertainty on pareto fronts with gaussian process conditional simulations. European Journal of Operational Research, 243, 386–394.
  7. A survey on high-dimensional gaussian process modeling with application to bayesian optimization. ACM Transactions on Evolutionary Learning and Optimization, 2, 1–26.
  8. Using ranking and selection to “clean up” after simulation optimization. Operations Research, 51, 814–825.
  9. Identifying efficient solutions via simulation: myopic multi-objective budget allocation for the bi-objective case. OR Spectrum, 41, 831–865.
  10. Multiobjective ranking and selection based on hypervolume. In T. Roeder, P. Frazier, R. Szechtman, E. Zhou, T. Huschka, & S. Chick (Eds.), Proceedings of the 2016 Winter Simulation Conference (pp. 859–870). Piscataway, New Jersey: IEEE.
  11. Stochastic Simulation Optimization volume 1. Singapore: World Scientific. doi:10.1142/7437.
  12. The effects of common random numbers on stochastic kriging metamodels. ACM Trans. Model. Comput. Simul., 22, 7:1–7:20.
  13. Sequential sampling to myopically maximize the expected value of information. INFORMS Journal on Computing, 22, 71–80. doi:10.1287/ijoc.1090.0327.
  14. PyMOSO: software for multiobjective simulation optimization with R-PERLE and R-MinRLE. INFORMS Journal on Computing, 32, 1101–1108.
  15. Applying evolutionary algorithms to problems with noisy, time-consuming fitness functions. In Proceedings of the 2004 Congress on Evolutionary Computation (pp. 1254–1261). IEEE volume 2.
  16. Indifference-zone-free selection of the best. Operations Research, 64, 1499–1514.
  17. Score allocations for bi-objective ranking and selection. ACM Transactions on Modeling and Computer Simulation, 28, 7:1–7:28.
  18. Multiobjective simulation optimization on finite sets: Optimal allocation via scalarization. In Y. Yilmaz, W. Chan, I. Moon, T. Roeder, C. Macal, & M. Rosseti (Eds.), Proceedings of the 2015 Winter Simulation Conference (WSC) (pp. 3610–3621). Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
  19. A bayesian approach to constrained single-and multi-objective optimization. Journal of Global Optimization, 67, 97–133.
  20. Multi-objective optimisation in the presence of uncertainty. In 2005 IEEE Congress on Evolutionary Computation (pp. 243–250). volume 1.
  21. The rolling tide evolutionary algorithm: A multiobjective optimizer for noisy optimization problems. IEEE Transactions on Evolutionary Computation, 19, 103–117.
  22. Frazier, P. I. (2014). A fully sequential elimination procedure for indifference-zone ranking and selection with tight bounds on probability of correct selection. Operations Research, 62, 926–942.
  23. Predictive entropy search for multi-objective bayesian optimization. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 ICML’16 (pp. 1492–1501). JMLR.
  24. First investigations on noisy model-based multi-objective optimization. In International Conference on Evolutionary Multi-Criterion Optimization (pp. 298–313). Springer.
  25. A review of multiobjective test problems and a scalable test problem toolkit. IEEE Transactions on Evolutionary Computation, 10, 477–506.
  26. Hughes, E. J. (2001). Evolutionary multi-objective ranking with uncertainty and noise. In International Conference on Evolutionary Multi-Criterion Optimization (pp. 329–343). Springer.
  27. An introduction to multiobjective simulation optimization. ACM Trans. Model. Comput. Simul., 29, 7:1–7:36. doi:10.1145/3299872.
  28. Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. European Journal of Operational Research, 261, 279 – 301.
  29. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13, 455–492.
  30. A simulation based optimization approach to supply chain management under demand uncertainty. Computers & chemical engineering, 28, 2087–2106.
  31. A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation (TOMACS), 11, 251–273.
  32. Selecting the best system. In Handbooks in Operations Research and Management Science (pp. 501–534). North Holland: Elsevier volume 13.
  33. Kleijnen, J. P. C. (2015). Design and Analysis of Simulation Experiments. (2nd ed.). NY: Springer.
  34. Finding the non-dominated pareto set for multi-objective simulation models. IIE Transactions, 42, 656–674.
  35. MO-COMPASS: a fast convergent search algorithm for multi-objective discrete optimization via simulation. IIE Transactions, 47, 1153–1169.
  36. Miettinen, K. (1999). Nonlinear multiobjective optimization volume 12. Springer Science & Business Media.
  37. A benchmark of kriging-based infill criteria for noisy optimization. Structural and Multidisciplinary Optimization, 48, 607–626.
  38. Gaussian Processes for Machine Learning (Adaptive computation and machine learning). (1st ed.). Cambridge, Massachusetts, USA: The MIT Press.
  39. Multiobjective ranking and selection with correlation and heteroscedastic noise. In 2019 Winter Simulation Conference (WSC) (pp. 3392–3403).
  40. A multiobjective stochastic simulation optimization algorithm. European Journal of Operational Research, 284, 212 – 226.
  41. Staum, J. (2009). Better simulation metamodeling: The why, what, and how of stochastic kriging. In Proceedings of the 2009 Winter Simulation Conference (WSC) (pp. 119–133). IEEE.
  42. Evolutionary optimisation of noisy multi-objective problems using confidence-based dynamic resampling. European Journal of Operational Research, 204, 533–544.
  43. Teich, J. (2001). Pareto-front exploration with uncertain objectives. In International Conference on Evolutionary Multi-Criterion Optimization (pp. 314–328). Springer.
  44. Integration of indifference-zone with multi-objective computing budget allocation. European Journal of Operational Research, 203, 419 – 429.
  45. Pareto-dominance in noisy environments. In Proceedings of the Eleventh Conference on Congress on Evolutionary Computation CEC’09 (pp. 3119–3126). Piscataway, NJ, USA: IEEE Press.
  46. New uncertainty handling strategies in multi-objective evolutionary optimization. In Parallel Problem Solving from Nature, PPSN XI (pp. 260–269). Berlin, Heidelberg: Springer Berlin Heidelberg.
  47. The hypervolume indicator revisited: On the design of pareto-compliant indicators via weighted integration. In Evolutionary multi-criterion optimization (pp. 862–876). Springer.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets