NOMAD version 4: Nonlinear optimization with the MADS algorithm

Abstract: NOMAD is software for optimizing blackbox problems. In continuous development since 2001, it constantly evolved with the integration of new algorithmic features published in scientific publications. These features are motivated by real applications encountered by industrial partners. The latest major release of NOMAD, version 3, dates from 2008. Minor releases are produced as new features are incorporated. The present work describes NOMAD 4, a complete redesign of the previous version, with a new architecture providing more flexible code, added functionalities and reusable code. We introduce algorithmic components, which are building blocks for more complex algorithms, and can initiate other components, launch nested algorithms, or perform specialized tasks. They facilitate the implementation of new ideas, including the MegaSearchPoll component, warm and hot restarts, and a revised version of the PSD-MADS algorithm. Another main improvement of NOMAD 4 is the usage of parallelism, to simultaneously compute multiple blackbox evaluations, and to maximize usage of available cores. Running different algorithms, tuning their parameters, and comparing their performance for optimization is simpler than before, while overall optimization performance is maintained between versions 3 and 4. NOMAD is freely available at www.gerad.ca/nomad and the whole project is visible at github.com/bbopt/nomad.

• The paper introduces a redesigned algorithm that enhances modularity, parallel processing, and flexibility in solving complex blackbox optimization tasks.
• The paper details the innovative use of parallelism and progressive barriers to boost performance in high-dimensional and constrained problems.
• Benchmark tests show that NOMAD v4 maintains efficiency compared to version 3 while excelling in computational speed on multi-core systems.

Overview of Nonlinear Optimization with Version 4 Algorithm

The paper presents an advanced foray into the optimization of blackbox problems, focusing on the introduction of version 4 of an unnamed software package. This software, which has been under continuous development since 2001, experienced a major overhaul with version 3 in 2008. The latest iteration, version 4, is highlighted as a comprehensive redesign aimed at enhancing flexibility, functionality, and code reuse in solving complex optimization tasks that do not provide analytic descriptions or available derivatives of objective functions.

Key Developments in Version 4

The development strategy for version 4 revolves around several critical improvements in both the architecture and the algorithmic approach:

1. Algorithmic Components: These are fundamental building blocks for developing more complex algorithms. Such components can initiate other processes, handle iterations, or fulfill specific tasks crucial for complex algorithm performance.
2. Parallelism: One significant enhancement is the utilization of parallelism in optimizing the computational efficiency of blackbox evaluations, maximizing core usage by conducting simultaneous computations across multiple cores. This development is crucial in fully leveraging modern computing resources.
3. Enhanced Modularity: Version 4 emphasizes a modular architecture, allowing easier maintainability and the incorporation of new features without destabilizing existing functionalities.
4. Flexibility in Optimization Approaches: Users can run different algorithms, fine-tune parameters, and compare performance, maintaining optimization efficacy between versions.

Algorithmic Details

The core algorithm employed, known as the Mesh Adaptive Direct Search (MADS), tackles optimization problems by iterating over mesh points derived from a candidate set. The algorithm features:

• Mesh Construct and Exploration: A modifiable mesh framework allows for iterative search and poll steps, where the search entails global exploration and polling focuses on local exploration around the incumbent solution.
• Progressive Barrier: An approach for handling constraint violations, allowing the adjustment of solutions based on the extent of constraint violations.

Comparative Performance & Computational Results

Benchmark tests indicate that version 4 maintains comparable performance with its predecessor (version 3) across both constrained and unconstrained test problems. Remarkably, the new version demonstrates improved performance in high-dimensional and parallel scenarios, partly due to its enhanced handling of multiple computational cores.

For parallel space decomposition characters, version 4's implementation offers superior results over its predecessor, achieving a more efficient and reliable optimization process with significantly improved outcomes in challenging problem sets like the Rosenbrock test problems.

Theoretical and Practical Implications

The improvements in version 4 have significant implications for both theoretical research and practical applications in AI:

• Theoretical Enhancement: By refining algorithmic efficiency and robust handling of non-derivative data, the software contributes valuable insight into solving blackbox problems, a critical subset of optimization tasks.
• Practical Applications: The software's flexibility and enhanced computational efficiency make it particularly beneficial in industries like materials science, aerospace engineering, and pharmacology, where simulation-based optimization is often essential.

Future Directions

Version 4 is poised as a foundational base for future enhancements, aimed at integrating additional methodologies such as multi-objective optimization and robust optimization strategies. The ongoing development will likely focus on exploiting the full potential of emerging computational resources and continuously expanding its application scope across diverse industrial sectors.

In conclusion, version 4 of the software package represents a significant upgrade in nonlinear optimization technology for blackbox optimization problems, asserting itself as a crucial tool for both contemporary research and industrial application.