Convex–Concave Procedure (CCP)

• The Convex–Concave Procedure (CCP) is a method that decomposes nonconvex optimization problems into the difference of convex functions to enable tractable solutions.
• It iteratively linearizes the concave parts, transforming challenging nonconvex problems into a series of convex subproblems with guaranteed descent and feasibility.
• CCP is widely applied in areas such as control synthesis, statistical learning, and combinatorial optimization, providing a versatile framework for solving complex models.

The Convex–Concave Procedure (CCP) is a general optimization methodology for nonconvex programming, specifically designed for problems where the objective and/or constraints can be written as the difference of two convex functions (also termed difference-of-convex, or DC, functions). CCP has played a decisive role in fields such as combinatorial optimization, control synthesis, learning with indefinite kernels, robust statistical estimation, and parameter synthesis for stochastic systems. The underlying principle is to iteratively convexify nonconvex problems by linearizing their concave components, yielding a sequence of tractable convex subproblems while ensuring descent and feasibility properties relevant to the original nonconvex formulation.

1. Theoretical Foundations of the Convex–Concave Procedure

The classical setting for CCP is a DC program of the form

$$\begin{aligned} \mathrm{minimize} \quad & f(x) - g(x) \ \text{subject to} \quad & f_i(x) - g_i(x) \leq 0, \quad i = 1, \dots, m \end{aligned}$$

where $f, g, f_i, g_i$ are convex (typically differentiable) functions. CCP proceeds by replacing each concave component $-g(x)$ with its first-order Taylor expansion at the current iterate,

$$g(x) \approx g(x_k) + \nabla g(x_k)^\top (x - x_k),$$

yielding a convex surrogate subproblem at each iteration. The same linearization applies to the concave portions of DC constraints. This majorization–minimization approach is conceptually related to the broader class of majorize-minimize algorithms, but tailored to the DC structure.

CCP generalizes to constrained programming, including settings where both the objective and constraints admit DC decompositions (see (Shen et al., 2016, Yurtsever et al., 2022)). Theoretical advances have established non-asymptotic convergence rates for CCP by invoking its equivalence with the Frank–Wolfe method applied to a suitable epigraph reformulation (see (Yurtsever et al., 2022)).

2. Algorithmic Realizations and Practical Enhancements

At each iteration, CCP constructs and solves a convex program in which all concave terms have been replaced by their affine surrogates. The update rule is

$$\begin{aligned} x_{k+1} = \arg\min_x & \quad f(x) - [g(x_k) + \nabla g(x_k)^\top (x - x_k)] \ \text{subject to} & \quad f_i(x) - [g_i(x_k) + \nabla g_i(x_k)^\top (x - x_k)] \leq 0, \ \forall i. \end{aligned}$$

Practical algorithmic advances include:

• Penalty CCP: Slack variables and penalty terms are added to convexified constraints to ensure feasibility throughout iterations (see (Tian et al., 2017)).
• Damping: To address potential nondifferentiability at domain boundaries, CCP can update $$x_{k+1} = \alpha\, \hat{x}_{k+1} + (1-\alpha)\, x_k$$ with $\alpha \in (0,1)$ (see (Shen et al., 2016)).
• Inexact and Stochastic CCP: In large-scale settings, subproblems may only be solved to approximate optimality (e.g., via a few steps of gradient descent or stochastic optimization). Provided residual errors are controlled, such “CCICP” variants preserve convergence guarantees (see (Liu et al., 2017)).

3. Connections: Relation to Frank–Wolfe and Majorization–Minimization

Recent analysis demonstrates that CCP is equivalent to the Frank–Wolfe method applied to a lifted (epigraph) form of the original DC problem. For example, minimizing $f(x) - g(x)$ is recast as

$$\begin{aligned} \min_{x, t} &\quad t - g(x)\ \text{subject to} &\quad f(x) \leq t, \end{aligned}$$

which is concave in $(x, t)$ over a convex feasible region. Frank–Wolfe linearizes the objective and moves along feasible directions, matching the CCP update in terms of linearizing the concave part and solving the convex subproblem exactly with step-size one (Yurtsever et al., 2022).

This viewpoint transfers non-asymptotic convergence theory from FW to CCP: after $k$ iterations, there exists a suboptimality gap on order $O(1/k)$ to a stationary point. The FW connection also unifies CCP with a range of primal–dual and saddle-point algorithms and provides geometric insight into why CCP preserves feasibility and descent properties in the original DC landscape.

4. Extensions and Generalizations

CCP admits important generalizations, including:

• Disciplined Convex–Concave Programming (DCCP): DCCP (Shen et al., 2016) extends disciplined convex programming (DCP) logic to DC settings—verifying curvature, automatically handling domains via indicator functions, and splitting nonaffine equalities into inequalities. The Python package DCCP (an extension of CVXPY) encapsulates these principles, supporting a wide range of nonconvex problems.
• Penalty CCP and DC Constraints: When constraints are themselves DC functions, the “CCCP+” or “FW+” methods linearize both the objective’s concave part and the constraints’ concave parts at each iteration, allowing extensions to feasibility for DC-composed feasible sets (Yurtsever et al., 2022).
• Convex–Concave Backtracking: In nonconvex composite optimization, procedures may simultaneously adapt both step size and inertial parameters by constructing local convex upper and concave lower bounds, generalizing classical majorant/minorant backtracking to the nonconvex setting (Mukkamala et al., 2019).

5. Applications Across Domains

CCP and its variants underpin solution methods in multiple domains:

• Combinatorial Optimization: Graduated NonConvexity and Graduated Concavity Procedure (GNCGCP) realizes a simplified CCRP, interpolating between convex and concave relaxations via a single quadratic term, for tasks such as graph matching and the Quadratic Assignment Problem (Liu et al., 2013).
• Stochastic Systems and Control: Parameter synthesis in parametric Markov Decision Processes is handled by reformulating the quadratic constraints as DC forms and applying CCP to large-scale probabilistic model checking (Cubuktepe et al., 2021).
• Power Systems: OPF problems with quadratic and trigonometric constraints are expressed as DC programs and solved using penalty CCP with linearization of concave parts and slack relaxation (Tian et al., 2017).
• Statistical Learning: When kernel matrices are indefinite, as in Indefinite Kernel Logistic Regression, decomposing the regularizer into difference-of-convex form enables CCP-based and CCICP-based training (Liu et al., 2017).
• Morphological Neural Networks: Training neural architectures based on lattice or morphological operations, e.g., linear dilation-erosion perceptron or regressor, is cast as DC programming, with CCP enabling efficient training while accommodating nonsmooth decision functions (Oliveira et al., 2020, Oliveira et al., 2021, Cunha et al., 4 Jan 2024).
• Chance-Constrained Programming: Both sample-approximate and robust chance constraints are reformulated as DC constraints, enabling DCA or proximal DC algorithms to efficiently handle probabilistic feasibility (Jiang et al., 2020, Wang et al., 2023).
• Bayesian Sampling: For efficient simulation of event times in PDMP-based MCMC samplers, concave–convex decompositions yield additive, modular upper bounds for adaptive thinning (Sutton et al., 2021).
• Control with Temporal Logic: Satisfying signal temporal logic (STL) constraints is facilitated by exploiting DC decompositions of robustness measures, leading to efficient SQP-based CCP algorithms (Takayama et al., 2023).
• Financial Arbitrage: Nonconvex portfolio construction for statistical arbitrage is addressed via sequential convex programming using CCP and band or moving-band constraints (Johansson et al., 12 Feb 2024).
• Localization and Security: Robust localization in the presence of malicious anchors leverages CCP to relax nonconvex MLE estimators into SDPs with auxiliary variance-dilating parameters (Tomic et al., 31 Mar 2025).

6. Computational Considerations and Limitations

Advantages:

• Tractability: By reducing complex nonconvex problems to a sequence of convex or even linear programs, CCP leverages mature optimization solvers and scales to high dimension.
• Modularity: Only the convex–concave structure and linearization need to be specified; problem-specific relaxations are not required.
• Generality and Extensibility: CCP is directly applicable to new DC-structured models, provided subgradients are available.

Limitations:

• Local Convergence: CCP usually converges to a stationary point, which may be a local rather than global optimum.
• Initialization Sensitivity: Solution quality can depend on initial points; good heuristics or multi-start strategies may be necessary.
• Tuning: For penalty and inexact variants, appropriate tuning (e.g., slack penalties, inexactness tolerances) is required to balance feasibility and efficiency.
• Modeling Overhead: For some domains, explicit DC decompositions must be provided and maintained; automated tools (as in DCCP) help but may incur additional overhead for complex models.

7. Impact and Future Directions

CCP and its descendants unify major traction points in nonconvex programming, spanning theoretical modeling, algorithmic design, and engineering applications. The connection to primal–dual and Frank–Wolfe methods opens the way for more advanced variants, including stochastic, distributed, and variance-reduced CCP. Domain-specific algorithmic augmentations (e.g., joint localization and attacker detection via SDPs, or adaptive robustness for arbitrary kernel functions) reveal the capacity of CCP for addressing contemporary challenges in engineering, data science, and operations. The ongoing development of scalable software (notably DCCP, integration in CVXPY and MOSEK), together with deepening convergence theory and new application domains, is likely to drive further impact.

CCP is now established as a fundamental paradigm in modern optimization, enabling principled handling of a wide range of nonconvex problems by iterative reduction to tractable convex subproblems and providing a fertile ground for both theoretical innovation and practical problem solving.