Bi-Local Optimal Solutions in Optimization

• Bi-local optimal solutions are a generalization of local optimality that require two distinct locally optimal solutions in both combinatorial and bilevel optimization settings.
• In combinatorial local search, finding two distinct local optima is often NP-complete, though special cases allow tractable solutions via fixed-size neighborhoods or bounded structures.
• In bilevel programming, bi-local solutions facilitate reformulations into one-level problems, enabling the application of first- and second-order optimality conditions and convergence rate analyses.

Bi-local optimal solutions constitute a fundamental extension of classical local optimality notions in optimization, specifically relevant to both combinatorial local search and bilevel programming paradigms. In combinatorial settings, bi-local optimality concerns the existence of two distinct locally optimal solutions within a prescribed neighborhood structure; in bilevel programs, the concept facilitates tractable necessary and sufficient optimality conditions without recourse to intractable directional derivatives of lower-level value mappings.

1. Formal Definitions of Bi-local Optimality

1.1. Combinatorial Local Search

Let $I$ denote an instance of a combinatorial optimization problem with feasible solution space $\mathcal{S}(I)$. Given a polynomial-time neighborhood function

$$\mathcal{N} : \{(I,s) \mid s \in \mathcal{S}(I)\} \rightarrow 2^{\mathcal{S}(I)},$$

and objective $f(I,s) \in \mathbb{R}$, a solution $s^*$ is locally optimal if

$$\forall\,s' \in \mathcal{N}(I,s^*):\, f(I,s') \leq f(I,s^*).$$

The bi-local optimality problem seeks two distinct locally optimal solutions: $\text{Bi-local-Opt}(I,\mathcal{N},f) = \text{“find }s_1 \neq s_2 \in \mathcal{S}(I)\text{, each locally optimal under }\mathcal{N}.”}$ Its decision version asks whether such a pair exists.

1.2. Bilevel Programming

For the parametric bilevel program

$$(BP)\quad \min F(x,y) \quad \text{s.t.} \quad H(x,y)=0,\; G(x,y) \leq 0,\; y \in S(x)$$

where $S(x) = \arg\min_{z \in Y(x)} f(x,z)$, a point $(x^*, y^*)$ is a bi-local solution if for all sufficiently small neighborhoods, $y^*$ is locally optimal for the lower level and any nearby feasible pair corresponds to a worse or equal upper-level objective.

2. Complexity and the Class PLS

2.1. Polynomial Local Search (PLS)

A problem lies in PLS if:

• Initial solution and neighbors can be found/listed in polynomial time
• Objective values are efficiently computable
• The goal is to find a locally optimal solution (no neighbor has strictly better value).

2.2. PLS-Completeness

A problem is PLS-complete if every problem in PLS can be reduced to it by a polynomial-time, locality-preserving reduction. For example, the Max-Cut problem under arbitrary weights and 1-flip neighborhood is PLS-complete.

2.3. Finding Multiple Local Optima

While finding one local optimum is often polynomial, requiring two distinct local optima fundamentally increases complexity in many settings, often rendering the problem NP-complete.

3. NP-hardness of Finding Two Local Optima

Several canonical unweighted local search problems exhibit NP-completeness when generalized to the search for two distinct locally optimal solutions. For each, the neighborhood structure is specified (e.g., $k$-swap for sets, single-flip for SAT/cut assignments):

Problem	Neighborhood	NP-completeness Statement
Maximum Independent Set	$k$-swap ($k \ge 2$)	Deciding existence of two distinct $k$-maximal independent sets is NP-complete.
Minimum Dominating Set	$k$-swap ($k \ge 2$)	Deciding existence of two distinct $k$-minimal dominating sets is NP-complete.
Max-SAT (2-CNF)	Single-flip	Deciding existence of two distinct unflippable assignments is NP-complete.
Max-Cut (unweighted)	Single-flip	Deciding existence of two distinct stable cuts is NP-complete.

The hardness proofs employ reductions from the Maximal Independent Set Extension problem, constructing gadgets that ensure a unique “trivial” solution and a second nontrivial solution linked to the original instance's witness.

4. Bi-local Solutions in Bilevel Programming: One-level Reformulation

4.1. Jacobian Uniqueness

Under Jacobian uniqueness conditions (lower-level KKT system admits a unique, C$^2$-smooth solution mapping near $x^*$), bi-local solutions of the bilevel program coincide with local minimizers of the following explicit reformulations:

• Implicit program (SP):

$$\min F(x, y(x)) \quad \text{s.t.} \quad H(x, y(x)) = 0,\; G(x, y(x)) \le 0$$

• First-order program (FP):

$$\begin{aligned} &\min_{x,y,\mu,\xi} F(x,y) \ &\text{s.t. } H(x,y)=0,\, G(x,y)\le0 \ &\qquad \nabla_y \mathcal{L}(x; y, \mu, \xi) = 0,\; h(x,y)=0,\; g(x,y)-\Pi_{\mathbb{R}_-^s}(g(x,y) + \xi) = 0 \end{aligned}$$

with $$\mathcal{L}(x; y, \mu, \xi) = f(x, y) + \mu^\top h(x, y) + \xi^\top g(x, y)$$.

4.2. Equivalence Theorem

For the lower-level problem at $(x^*, y^*)$ satisfying Jacobian uniqueness, the following are equivalent:

• $(x^*, y^*)$ is a bi-local solution of (BP)
• $x^*$ is a local minimizer of (SP)
• $(x^*, y^*, \mu^*, \xi^*)$ is a local minimizer of (FP)

5. Optimality Conditions and Algorithmic Implications

5.1. First-order Necessary Conditions

The set of multipliers $\Lambda(x^*)$ for (SP) satisfies the Mangasarian–Fromovitz constraint qualification, yielding KKT-type conditions at bi-local solutions. The analogous (FP) formulations involve multipliers for all constraints, likewise reducible to standard KKT conditions.

5.2. Second-order Conditions

• Critical cone definitions for (SP) and (FP) collect admissible perturbations.
• Second-order necessary conditions: For any $d_x$ in the critical cone,

$$\max_{(\lambda_H, \lambda_G) \in \Lambda(x^*)} d_x^\top \nabla_{xx}^2 L^{SP}(x^*; \lambda_H, \lambda_G) d_x \ge 0$$

• Second-order sufficient conditions: If the previous quantity is strictly positive for all nonzero directions, strict local optimality with quadratic growth is ensured:

$$F(x, y(x)) \geq F(x^*, y(x^*)) + \gamma \|x - x^*\|^2$$

Second-order conditions on (FP) employ the Hessian $\nabla_{uu}^2 L^{FP}(u^*; \lambda)$ and are equivalent via the corresponding transformation $U(x^*)$.

6. Tractable and Intractable Cases

Although NP-completeness prevails for bi-local search in unweighted combinatorial problems, notable exceptions arise:

• For fixed $k$, finding two distinct $k$-maximal matchings in a graph can be decided in time $n^{O(k)}$ via enumeration of short augmenting paths.
• For graphs of bounded cliquewidth, the property of being a 2-maximal independent set is expressible in monadic second-order logic, allowing enumeration in polynomial delay by Courcelle’s theorem.

These tractable regimes contrast sharply with the general NP-completeness for other local search problems when requiring two distinct locally optimal solutions.

7. Algorithmic Rate Results and Limitations

Applying the classical augmented Lagrangian method to (FP), under MFCQ and SOSC, yields Q-linear convergence rates to bi-local solutions of (BP) and may admit Q-superlinear rates if penalties are adaptively increased. This convergence property is significant as it applies directly to bi-local optimality, given the equivalence to explicit one-level programs. The requirements include:

• Sufficiently close initialization to the true solution
• Suitably high (or increasing) penalty parameter $\rho$

Limitations and Open Questions

• Failure of Jacobian uniqueness (e.g., loss of LICQ or strict complementarity) necessitates alternative approaches utilizing generalized derivatives or value-function methods.
• Globalization strategies for the augmented Lagrangian iteration, and quantification of attraction basins, remain open problems.
• Non-smooth, degenerate lower levels, or pessimistic bilevel models, demand further variational analysis.
• Attainment of rates beyond Q-linearity for finite penalties is presently unresolved.

Summary

Bi-local optimality generalizes classical local optimality, addressing both the existence of multiple local optima in discrete settings and tractable necessary/sufficient conditions for bilevel programs. In combinatorial local search, bi-local problems exhibit increased computational complexity, often resulting in NP-completeness, except for specific matching and bounded-structure cases. In bilevel optimization, bi-local solutions enable practical second-order analysis without explicit computation of derivatives of lower-level mappings, yielding explicit one-level reformulations compatible with established optimization techniques. Theoretical and algorithmic investigation of bi-local solutions highlights both computational barriers and opportunities for further research in optimization, complexity, and algorithmic convergence analyses.