Simulation optimization: A review of algorithms and applications (1706.08591v1)

Abstract: Simulation Optimization (SO) refers to the optimization of an objective function subject to constraints, both of which can be evaluated through a stochastic simulation. To address specific features of a particular simulation---discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs, homogeneous or heterogeneous noise---various algorithms have been proposed in the literature. As one can imagine, there exist several competing algorithms for each of these classes of problems. This document emphasizes the difficulties in simulation optimization as compared to mathematical programming, makes reference to state-of-the-art algorithms in the field, examines and contrasts the different approaches used, reviews some of the diverse applications that have been tackled by these methods, and speculates on future directions in the field.

• The paper presents a comprehensive review of algorithmic solutions that efficiently handle stochastic noise and resource constraints in simulation optimization.
• It categorizes several approaches including ranking and selection, response surface methodology, gradient-based, direct search, random search, and model-based techniques.
• It highlights diverse applications in fields like manufacturing, medicine, and logistics while outlining future research directions for mixed variables and enhanced parallel computing.

Simulation Optimization: A Review of Algorithms and Applications

This paper offers a comprehensive examination of the field of Simulation Optimization (SO), detailing a variety of algorithmic approaches and their applications. Simulation Optimization focuses on optimizing stochastic simulations where the objective function and constraints are evaluated through simulations, often treated as black boxes. Unlike mathematical programming, SO can address discrete or continuous decisions, handle expensive simulations, and cope with stochastic noise.

Core Focus and Challenges

The unique challenges of SO compared to mathematical programming include dealing with the stochastic nature of outputs and the absence of algebraic models for functions. Simulations can often be resource-intensive. The search for optimal parameters must therefore be efficient, typically requiring a minimal number of simulations. This paper emphasizes the complexity of discerning optimal input configurations given the inherent simulation noise and potential constraints.

Algorithmic Approaches

The paper categorizes and reviews several algorithms:

1. Ranking and Selection: Deals with finite sets of solutions, focusing on selecting the best alternative by minimizing simulation replicates.
2. Response Surface Methodology (RSM): Builds and iteratively refines surrogate models to guide exploration through designs of experiments. Bayesian global optimization using Kriging is a notable approach.
3. Gradient-based Methods: Utilizes stochastic approximation techniques like Simultaneous Perturbation Stochastic Approximation (SPSA) for estimating gradients when direct gradients are unavailable or unreliable.
4. Direct Search Methods: Includes techniques like pattern search and the Nelder-Mead simplex, useful for parameter spaces where gradients are difficult to estimate.
5. Random Search Methods: Encompasses metaheuristics such as genetic algorithms, simulated annealing, and scatter search, useful for both global and local solutions.
6. Model-Based Optimization: Estimation of distribution algorithms and cross-entropy methods use probability models to guide searches, particularly in large parameter spaces.
7. Simulation-specific adaptations: Methods like Nested Partitions and COMPASS have been tailored for large or infinite discrete spaces.

Applications and Implications

Simulation Optimization finds diverse applications in operations, manufacturing, medicine, engineering, computer science, and logistics. It is particularly effective in fields utilizing discrete-event simulations or systems of stochastic differential equations.

Theoretical implications include developing more robust frameworks for handling noisy outputs and constraints. Practically, the field is moving towards leveraging increased computational power and improved algorithms to address complex, real-world systems more effectively.

Future Directions

Key areas for further research include:

• Handling combined discrete/continuous variables
• Integration of stochastic and deterministic constraints
• Enhanced parallel computing techniques
• Better utilization of multiple simulation outputs
• Incorporation of non-traditional performance metrics like risk

The paper also highlights the need for accessible simulation optimization libraries to bridge the gap between theory and practice, encouraging more streamlined comparisons and broader adoption in commercial applications.

Conclusion

Simulation Optimization remains a rapidly evolving field, with burgeoning applications across various domains. The ongoing integration of diverse algorithmic approaches, backed by advances in computational power, promises continued growth and sophistication in tackling complex optimization problems. The work underscores the importance of adaptability and robustness in future algorithmic developments to meet the challenges posed by increasingly complex simulations.