GNCGCP Framework for Optimization

• GNCGCP is a deterministic annealing framework for combinatorial optimization over partial permutation matrices, using graduated nonconvexity and concavity phases.
• It gradually transforms the objective from a convex relaxation to a discrete solution using the original function’s gradient and a quadratic penalty via the Frank–Wolfe algorithm.
• The framework has shown competitive performance in NP-hard problems like subgraph matching and the quadratic assignment problem, ensuring efficient and robust convergence.

The Graduated NonConvexity and Graduated Concavity Procedure (GNCGCP) is a deterministic-annealing optimization framework designed for approximate solutions of combinatorial problems defined over the set of partial permutation matrices. GNCGCP structures the optimization through a double phase process—graduated nonconvexity (GNC) and graduated concavity (GC)—that implicitly realizes a convex-concave relaxation procedure (CCRP). The approach allows for efficient and general optimization without the explicit formulation of convex or concave surrogates, relying only on the original objective function and its gradient. Typical applications include canonical NP-hard problems such as (sub)graph matching and the quadratic assignment problem (QAP) (Liu et al., 2013).

1. Problem Class and Motivation

GNCGCP targets optimization problems of the form

$\min_{X \in \Pi} F(X)$

where $\Pi \subset \{0,1\}^{M\times N}$ denotes the set of $M\times N$ partial permutation matrices: $$\Pi = \left\{ X \,\middle|\, \begin{array}{l} X_{ij}\in\{0,1\} \ \sum_{j=1}^N X_{ij}=1\;\forall i=1,\ldots,M \ \sum_{i=1}^M X_{ij}\leq 1\;\forall j=1,\ldots,N \ M \leq N \end{array} \right\}$$ For $M=N$ this is the set of permutation matrices; for $M<N$ it encodes one-to-at-most-one assignment constraints relevant for subgraph matching, assignment, and other combinatorial mapping tasks. The objectives of interest, $F(X)$, are typically nonconvex and nonconcave over the convex hull $\Omega$ of $\Pi$, defined as

$$\Omega = \left\{ X\,\middle|\, X_{ij}\geq 0,\; \text{row sums} = 1,\; \text{column sums} \leq 1 \right\}$$

Classical CCPR approaches relax $F(X)$ into explicit convex and concave surrogates $F_v(X)$ and $F_c(X)$, but these relaxations are often not available in closed form, especially for partial permutations (Liu et al., 2013). GNCGCP was introduced to bypass this bottleneck, using only $F(X)$ and a scalar quadratic regularizer.

2. Mathematical Structure of the GNCGCP Objective

GNCGCP introduces a parametric family of surrogate objectives $F_\zeta(X)$ indexed by $\zeta \in [-1,1]$:

• For $1 \geq \zeta \geq 0$ (GNC phase):

$$F_\zeta(X) = (1-\zeta)F(X) + \zeta\,\mathrm{tr}(X^TX)$$

• For $0 > \zeta \geq -1$ (GC phase):

$$F_\zeta(X) = (1+\zeta)F(X) + \zeta\,\mathrm{tr}(X^TX)$$

with $X \in \Omega$. The process starts with $\zeta=1$ (fully convex quadratic), anneals toward $\zeta=0$ (original objective), and continues to $\zeta=-1$ (introducing a negative convex quadratic). This drives $X$ from a "smeared-out" solution through nonconvex regimes to an extreme point of $\Omega$ (i.e., an integer matrix in $\Pi$). An equivalent formulation is

$$\hat{F}_\zeta(X) = \begin{cases} (1-\zeta)F(X) + \zeta\,\mathrm{tr}(X^TX - JX) & 1 \geq \zeta \geq 0 \ (1+\zeta)F(X) + \zeta\,\mathrm{tr}(X^TX - JX) & 0 > \zeta \geq -1 \end{cases}$$

where $J=1_{N\times M}$ is the all-ones matrix.

3. Operational Procedure: Graduated Nonconvexity and Concavity

GNCGCP executes a continuous sweep of $\zeta$ from $1$ to $-1$, divided into two algorithmic subprocedures: Graduated Nonconvexity (GNC):

• $\zeta$ is initialized to $+1$.
• While $\zeta > 0$:
  • The surrogate $F_\zeta(X)$ is minimized over $\Omega$.
  • The gradient is $$\nabla F_\zeta(X) = (1-\zeta)\nabla F(X) + 2\zeta X$$.
  • The Frank–Wolfe algorithm, with a linearized subproblem solved via a non-square Hungarian algorithm ($\mathcal{O}(N^3)$), and a line search, is employed.
  • $\zeta$ is decreased by $d\zeta$ (typ. $10^{-3}$).
  • Graduated Concavity (GC):
• For $0 > \zeta \geq -1$:
  • The surrogate is $$F_\zeta(X) = (1+\zeta)F(X) + \zeta\,\mathrm{tr}(X^TX)$$.
  • The gradient becomes $$\nabla F_\zeta(X) = (1+\zeta)\nabla F(X) + 2\zeta X$$.
  • Frank–Wolfe minimization continues.
  • As $\zeta$ decreases, the regularizer enforces extremal solutions; upon $X\in\Pi$, the process terminates.

The process does not require explicit convex or concave surrogates: only the original $F(X)$, its gradient, and a simple quadratic term are necessary.

4. Mathematical Equivalence to the Convex–Concave Relaxation Procedure

GNCGCP can be shown to exactly instantiate a convex–concave relaxation procedure without direct construction of $F_v$ and $F_c$. The equivalence is established by considering the regularized form

$$F_\gamma(X) = F(X) + \gamma\,\mathrm{tr}(X^TX - JX)$$

with $\gamma$ sweeping from $-\lambda_{\min}$ to $-\lambda_{\max}$, where $\lambda_{\min},\lambda_{\max}$ are the minimal and maximal eigenvalues of the Hessian $\nabla^2 F$ over $\Omega$. Normalizing the surrogate leads to a path-following optimization in the convex–concave family: $$F_\eta(X) = (1-\eta)[F(X) - \lambda_{\min}\,\mathrm{tr}(X^TX - JX)] + \eta[F(X) - \lambda_{\max}\,\mathrm{tr}(X^TX - JX)]$$ This construction ensures the minimizer transitions from the relaxed space to the extreme discrete points, with all computational steps relying solely on $F(X)$ and the quadratic term (Liu et al., 2013).

5. Implementation, Convergence, and Complexity

Each Frank–Wolfe iteration, for convex $F_\zeta$ (i.e., $$\zeta>\zeta_u = \lambda_{\min}/(\lambda_{\min} - 1)$$), converges sublinearly with an $\mathcal{O}(1/k)$ duality-gap rate. In the nonconvex regime, the process is heuristic but consistently yields high-quality local minima, while in the concave phase, a vertex of $\Omega$ (i.e., an integer partial permutation) is assured.

Key computational aspects:

• Each Frank–Wolfe linear subproblem requires a non-square Hungarian algorithm ($\mathcal{O}(N^3)$ per iteration).
• Gradient calculation is $\mathcal{O}(N^3)$ for typical matrix-multiplication forms.
• Storage is $\mathcal{O}(N^2)$.
• No problem-specific tuning is required; the only needed elements are $\nabla F$, the quadratic regularizer, and annealing parameters ($d\zeta\sim 10^{-3}$–$10^{-2}$, Frank–Wolfe tolerance $\epsilon\sim 10^{-3}$).
• Warm-starting at each $\zeta$ results in modest inner iterations per sweep.

6. Canonical Applications and Empirical Performance

The generality of GNCGCP is demonstrated on two central NP-hard problems:

(a) Subgraph / Graph Matching:

Given graphs $G_M(V_M,E_M), G_D(V_D,E_D)$, adjacency matrices $A_M,A_D$, and $|V_M|=M\leq N=|V_D|$, subgraph matching is phrased as: $F(X) = \|A_M - X A_D X^T\|_F^2,\quad X\in\Pi$ with gradient

$$\nabla F(X) = 2 X (A_D^T X^T X A_D + A_D X^T X A_D^T) - 2(A_M X A_D^T + A_M^T X A_D)$$

GNCGCP applied to this (denoted GNCGCP_SGM or GNCGCP_GM) matches or outperforms state-of-the-art approaches (Graduated Assignment, Spectral Matching, (E)PATH) on both synthetic and real datasets, exhibiting superior robustness to noise, outliers, and graph-size scaling.

(b) Quadratic Assignment Problem (QAP):

For $A,B\in\mathbb{R}^{N\times N}$,

$\min_{X\in P} \mathrm{tr}(A X B^T X^T)$

with gradient $\nabla F(X) = A X B^T + A^T X B$, GNCGCP_QAP achieves best known average error ratios (≤1%) on QAPLib (31 benchmarks), outperforming several established algorithms in assignment accuracy.

These results demonstrate that GNCGCP functions as a plug-and-play annealing algorithm for any differentiable objective on partial permutations, requiring only $\nabla F$, annealing, and the standard Frank–Wolfe/Hungarian subroutine (Liu et al., 2013).

7. Summary Characteristics and Practical Implications

GNCGCP enables general, scalable approximate solutions for a class of difficult combinatorial optimization problems without case-specific surrogates or spectral decompositions. Only implementation of the gradient of the original objective is required, and the annealing of a quadratic penalty suffices to implicitly capture convex–concave relaxations. The efficiency and competitive performance on benchmark graph matching and QAP studies position GNCGCP as a robust and flexible deterministic optimization method for structured discrete assignment tasks (Liu et al., 2013).