Inexact Restoration (IR) Methods

• Inexact Restoration (IR) is a two-phase optimization framework that decouples feasibility restoration from objective minimization to handle inexact evaluations effectively.
• The methodology employs a restoration phase for improving feasibility and an optimization phase guided by a dynamic merit function with adaptive penalty parameters.
• IR methods offer theoretical guarantees such as global convergence and complexity bounds, proving effective in derivative-free, stochastic, and large-scale optimization scenarios.

Inexact Restoration (IR) is a two-phase optimization framework designed for problems where achieving feasibility (e.g., constraint satisfaction or high-fidelity evaluation) is distinct and often more expensive than reducing the objective. The methodology formally separates steps for improving feasibility from those for achieving optimality, allowing flexible and principled handling of inexactness in constraints, objective, sampling, or function evaluations. IR has been extensively applied to constrained optimization, stochastic programming, derivative-free methods, large-scale or nonsmooth problems, and noisy or subsampled model contexts.

1. Core Principles and Formalism

Inexact Restoration treats an optimization problem in which some form of feasibility is difficult or expensive to attain—such as strict satisfaction of constraints, exact function values, or high-accuracy oracle access. The framework recasts the original problem

$\min_{x\in\Omega} f(x), \quad \text{s.t. } C(x)=0$

as an equivalent problem with a decoupled accuracy or feasibility index $y$: $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$ Here, $h(y)$ measures the inaccuracy or infeasibility (e.g., probability of constraint violation, noise level, sample size deviation, discretization error, etc.), and $f(x,y)$ is an inexact version of the objective, potentially depending on $y$. The two key steps are:

• Restoration phase (Feasibility improvement): Find a new $y$ such that $h(y)$ is reduced—typically by a multiplicative factor.
• Optimization phase (Optimality improvement): With fixed or improved $y$, perform an (inexact) descent step targeting the objective reduction.

A scalar merit function

$$\Phi(x, y, \theta) = \theta f(x, y) + (1-\theta) h(y)$$

with dynamically updated penalty parameter $y$0 balances objective reduction against accuracy improvement and guides acceptance criteria.

2. Algorithmic Realizations and Method Variants

The IR methodology is instantiated in a variety of algorithmic settings:

• Derivative-Free and Filter-Based Multiobjective Optimization: In "An Inexact Restoration Direct Multisearch Filter Approach to Multiobjective Constrained Derivative-Free Optimization" (Silva et al., 2024), the IR step aims to find $y$1 subject to $y$2, where $y$3 aggregates constraint violations and $y$4 as $y$5. Here, IR is tightly integrated with a filter strategy that views feasibility as an added objective, ensuring that computational effort is not wasted at highly infeasible points and balancing progress toward feasibility and objective minimization.
• Stochastic and Subsampled Trust-Region Methods: For finite-sum minimization and stochastic approximations, IR schemes alternate restoration (increasing sample size or estimator accuracy) with trust-region or line-search-based optimization steps. In (Bellavia et al., 2021, Bellavia et al., 2019), the infeasibility $y$6 quantifies the error due to subsampling, and the algorithm guarantees global convergence and $y$7 complexity under mild requirements. Subsampling strategies are adaptively controlled by the restoration step, maintaining low costs early and increasing accuracy as iterates approach stationarity.
• Stochastic Optimization with Variable Sample Size: In (Krejic et al., 2021), IR is used for convex nonsmooth stochastic optimization, with the restoration phase adaptively adjusting the sample size to maintain statistical accuracy. Convergence is established under standard stochastic approximation assumptions.
• Noisy Function Optimization via Random Models: The IRERM algorithm (Morini et al., 2024) provides a random-model-based trust-region implementation of IR for settings where both the function and gradient are only available via random, unbiased estimates, with restoration enforcing prescribed accuracy-in-probability by adapting sample size.
• Derivative-Free, Nonsmooth, and Expensive Problems: "Economic inexact restoration for derivative-free expensive function minimization and applications" (Birgin et al., 2020) extends IR to metric spaces and black-box evaluations without smoothness, using a monotone optimization algorithm as the "optimality" step, convergence guarantees via merit-based acceptances, and empirical demonstration on real and synthetic problems.
• Inexact Function and Constraint Evaluation: BIRA (Bueno et al., 2022) generalizes IR to problems where both the objective and constraints are evaluated inexactly, with explicit modeling and control of dual sources of inexactness. Restoration and optimization are executed with respect to inexact oracles, and complexity results are derived for feasibility, precision, and projected stationarity simultaneously.

3. Theoretical Guarantees and Complexity

IR methods are equipped with rigorous theoretical results. The essential properties depend on problem regularity and the precise IR implementation but generally include:

• Global convergence: Under mild compactness, continuity, or probabilistic model accuracy conditions, sequences generated by IR converge to feasible, (approximate) stationary points—either Clarke, KKT, or AGP-stationary, depending on the setting (Silva et al., 2024, Bueno et al., 2022).
• Worst-case complexity: For trust-region and similar IR variants, the expected number of iterations to reach $y$8-approximate stationarity is $y$9 (Bellavia et al., 2021, Morini et al., 2024, Bellavia et al., 2019).
• Finite restoration steps: Restoration phases reduce infeasibility measures geometrically, leading to $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$0 bounds on the number of restoration iterations before a desired feasibility level (Birgin et al., 2020, Bueno et al., 2022).
• Economic inexactness: Many IR algorithms avoid expensive full-precision evaluations unless required, enabling early iterations with reduced computational burden and gradual refinement as necessary for convergence (Bellavia et al., 2019, Birgin et al., 2020).

4. Integration with Filters, Penalty, and Subsampling Strategies

IR is highly modular and meshes with diverse constraint-handling and sampling mechanisms:

• Filter methods: Treat feasibility as an additional objective, with IR steps invoked to yield points that can be accepted according to a no-domination rule, often leading to more reliable feasibility recovery than penalty or barrier functions (Silva et al., 2024).
• Sampling and subsampling: Restoration controls the accuracy of stochastic or subsampled models, allowing adaptive sample-size schedules that ensure accuracy requirements for stochastic gradients or function estimates are met, while keeping costs low in early phases (Bellavia et al., 2021, Krejic et al., 2021, Morini et al., 2024).
• Derivative-free contexts: The restoration phase accommodates arbitrary or black-box precision indices, with nonsmooth or regularized surrogate models employed in the optimization phase. In settings without any smooth structure, Armijo or monotone descent mechanisms are used (Birgin et al., 2020).

5. Empirical Performance and Application Domains

Numerical experiments across the literature demonstrate that IR-based methods often match or outperform classical penalty, barrier, or fixed-sample methods, especially in regimes with limited computational budget or expensive evaluations:

• Multiobjective constrained derivative-free problems: DMS-FILTER-IR reliably recovers feasibility from infeasible starts and achieves superior hypervolume and nondomination purity compared to exact-penalty and progressive-barrier approaches (Silva et al., 2024).
• Stochastic finite-sum and machine learning tasks: IR-based trust-region solvers reduce total function evaluations, particularly by economizing on early, less accurate iterations (Bellavia et al., 2021, Bellavia et al., 2019, Krejic et al., 2021).
• Noisy and black-box simulation: IR scheduling of accuracy/precision provides practical savings without sacrificing convergence, as validated on experimental tasks in engineering simulation (Birgin et al., 2020).
• Bilevel and equilibrium-constrained optimization: In traffic demand adjustment problems cast as MPECs, IR enables solution methods with global stationarity guarantees while utilizing established lower-level solvers (Walpen et al., 2018).

6. Inexactness Quantification and Control Mechanisms

Explicit quantification and management of inexactness are fundamental:

• Feasibility measures $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$1 are typically monotone and possess the property $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$2 if and only if feasibility is exact (e.g., $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$3 for sample-size control).
• Reduction schedules impose geometric or controlled contraction of $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$4 at each restoration step; for instance, requiring $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$5, $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$6.
• Adaptive penalty parameters $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$7 in the merit function are dynamically adjusted to balance progress in $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$8 and $$\min_{x\in\Omega,\, y\in Y} f(x,y) \quad \text{s.t. } h(y) = 0, \quad C(x,y)=0.$$9, ensuring overall descent in the combined merit function.
• Complexity and termination metrics are established both for feasibility ($h(y)$0), optimality ($h(y)$1), and total work (function/gradient evaluations) (Bueno et al., 2022, Bellavia et al., 2019).

7. Broader Implications and Methodological Significance

Inexact Restoration unifies disparate themes in modern optimization, including inexact oracles, adaptive accuracy, variable-sample methods, and exact penalty/filter frameworks. Its formal split of restoration and optimization phases enables algorithmic designs well-suited for emerging computational contexts—such as derivative-free optimization, stochastic or simulation-based models, nonsmooth and nonconvex problems, and large-scale learning tasks. By enabling adaptive management of both computational and statistical accuracy, IR provides both theoretical guarantees on convergence and practical efficiency benefits proven across diverse application domains (Silva et al., 2024, Bellavia et al., 2021, Bellavia et al., 2019, Bueno et al., 2022, Krejic et al., 2021, Morini et al., 2024, Birgin et al., 2020, Walpen et al., 2018).