---
title: 'CoCo-MILP: Contrastive & Competitive MILP Prediction'
url: https://www.emergentmind.com/topics/coco-milp
type: topic
---

# CoCo-MILP: Contrastive & Competitive MILP Prediction

Searching arXiv for the target paper and closely related MILP solution-prediction papers to ground the article and retrieve citation IDs.
First, I’ll find the CoCo-MILP paper by its arXiv id.
Now I’ll search for related papers mentioned in the provided data, such as Predict-and-Search, ConPS, Apollo-MILP, and DiffILO, so any contextual mentions can be cited accurately.
CoCo-MILP is a learning-based framework for Mixed-Integer Linear Programming (MILP) solution prediction that addresses two structural misalignments identified in prior Graph Neural Network (GNN) approaches: an objective-level misalignment induced by Binary Cross-Entropy (BCE), and an architecture-level misalignment induced by smoothing in standard message passing. Introduced in "CoCo-MILP: Inter-Variable Contrastive and Intra-Constraint Competitive MILP Solution Prediction" [2511.09209], it combines an Inter-Variable Contrastive Loss (VCL) with an Intra-Constraint Competitive (ICC) GNN layer to produce higher-quality predictions for binary decision variables, which are then used within a predict-and-search solver pipeline. The method is positioned within the MILP solution-prediction line of ML for combinatorial optimization alongside Neural Diving, Predict-and-Search (PS), ConPS, Apollo-MILP, and DiffILO [2511.09209].

## 1. Problem setting and motivation

Mixed-Integer Linear Programming is presented as a central modeling tool for combinatorial optimization. A MILP instance \(\mathcal{I}\) is given as
\[
\min_{\bm{x} \in \mathbb{Z}^p \times \mathbb{R}^{n-p} \left\{ \bm{c}^\top \bm{x} \;\middle|\; \bm{A} \bm{x} \le \mathbf{b},\; \bm{l} \le \bm{x} \le \bm{u} \right\},
\]
with objective coefficients \(\bm{c}\), constraint matrix \(\bm{A}\), right-hand side vector \(\bm{b}\), and bounds \(\bm{l}, \bm{u}\) [2511.09209]. The paper focuses on the practically important case
\[
\bm{x}\in\{0,1\}^p\times\mathbb{R}^{n-p},
\]
with general integers handled by standard preprocessing [2511.09209].

The motivation is computational rather than purely predictive. Even when all integer variables are binary, solving large-scale MILPs via Branch-and-Bound or Branch-and-Cut is NP-hard, and modern solvers such as SCIP and Gurobi can still struggle on large or industrial instances [2511.09209]. A prominent ML4CO approach therefore uses GNNs to predict solutions or variable assignments and then allows a solver to refine them. The standard recipe is to represent the MILP as a bipartite graph, train a GNN to output \(\hat{x}_j \in [0,1]\) as estimated marginals \(p(x_j=1\mid \mathcal{I})\) for binary variables, and use these predictions as warm starts or to restrict the solver to a trust region around \(\hat{\bm{x}}\) [2511.09209].

CoCo-MILP argues that existing solution-prediction methods are structurally misaligned with MILP instances at two levels. At the objective level, BCE treats each variable as an independent binary classification, ignoring that solution quality depends on relative priorities among variables, yielding plausibly calibrated but ambiguously ordered logits [2511.09209]. At the architecture level, standard bipartite GNN message passing smooths node representations, whereas many MILP constraints express competition, such as set-packing \(\sum_{i\in S} x_i \le 1\), budget or knapsack constraints \(\sum_i w_i x_i \le W\), and assignment constraints \(\sum_j x_{ij} = 1\) [2511.09209]. The central thesis is therefore that high-quality MILP solution prediction requires both inter-variable contrast and intra-constraint competition.

## 2. Predict-and-search formulation and graph encoding

Within the Predict-and-Search paradigm, CoCo-MILP adopts an energy-based solution distribution
\[
\begin{aligned}
p(\bm{x}\mid\mathcal{I}) &= \frac{\exp(-E(\bm{x}\mid \mathcal{I}))}{\sum_{\bm{x}'}\exp(-E(\bm{x}'\mid \mathcal{I}))},\\
E(\bm{x}\mid \mathcal{I}) &=
\begin{cases}
\bm{c}^\top \bm{x}, & \text{if }\bm{x} \text{ is feasible,}\\
+\infty, & \text{otherwise,}
\end{cases}
\end{aligned}
\]
and approximates it by a factorized distribution over binary variables,
\[
p_{\bm{\theta}}(\bm{x}\mid\mathcal{I}) = \prod_{i=1}^{p} p_{\bm{\theta}}(x_i\mid\mathcal{I}),
\]
with a GNN producing \(\hat{\bm{x}}=(\hat{x}_1,\dots,\hat{x}_p)^\top \in [0,1]^p\), where \(\hat{x}_j = p_{\bm{\theta}}(x_j=1\mid\mathcal{I})\) [2511.09209]. This factorization is explicitly a modeling approximation. The paper later notes that the underlying distribution model remains fully factorized over binary variables, even though VCL injects relational structure at the loss level [2511.09209].

At inference, the framework reuses the search stage of PS through a trust-region formulation:
\[
\min_{\bm{x} \in \mathbb{Z}^p \times \mathbb{R}^{n-p}}
\left\{\bm{c}^\top \bm{x}\;\middle|\; \bm{A} \bm{x} \le \mathbf{b},\; \bm{l} \le \bm{x} \le \bm{u},\; \bm{x}_{1:p}\in \mathcal{B}(\hat{\bm{x}},\Delta) \right\},
\]
with
\[
\mathcal{B}(\hat{\bm{x}},\Delta) :=\{\bm{x}\in\mathbb{R}^n:\|\bm{x}_{1:p}-\hat{\bm{x}}\|_1\le \Delta\}.
\]
The search pipeline chooses hyperparameters \((k_0,k_1,\Delta)\) to fix top-\(k_0\) variables with highest probabilities to \(1\), potentially some to \(0\), possibly unfix top-\(k_1\) with low confidence, and constrain \(\|\bm{x}_{1:p} - \hat{\bm{x}}\|_1 \le \Delta\) before solving the restricted MILP using Gurobi or SCIP for up to \(1000\) seconds [2511.09209].

Each instance is encoded as a weighted bipartite graph
\[
\mathcal{G} = (\mathcal{V} \cup \mathcal{W}, \mathcal{E}),
\]
where \(\mathcal{V}\) denotes variable nodes, \(\mathcal{W}\) denotes constraint nodes, and \(\mathcal{E}\) contains edges for nonzero entries of \(\bm{A}\) [2511.09209]. Variable-node features include the objective coefficient \(c_i\), lower and upper bounds \(l_i, u_i\), an indicator for integrality or binary status, and possibly statistics such as degree. Constraint-node features include the right-hand side \(b_j\), encoded constraint type \((\le,=,\ge)\), and aggregates such as the number of variables. Edge features include the coefficient \(A_{ji}\) and possibly normalized variants or sign indicators. The feature design and preprocessing follow Gasse et al. (2019) and Han et al. (PS, 2023), with the precise feature lists taken from those works [2511.09209].

## 3. Objective-level design: Inter-Variable Contrastive Loss

The objective-level contribution of CoCo-MILP is the Inter-Variable Contrastive Loss, or VCL. For an instance \(\mathcal{I}\), the GNN produces logits \(\bm{z}\) and predictions \(\hat{\bm{x}}\). For a training solution \(\bm{x}^{(i)}\), the method defines the positive variable set \(\mathcal{V}_{+} = \{v \mid x_v^{(i)}=1\}\), the negative variable set \(\mathcal{V}_{-} = \{v \mid x_v^{(i)}=0\}\), and the full binary-variable set \(\mathcal{V}_{0}\) [2511.09209]. The training objective no longer attempts merely to match each \(x_v\) independently; instead, it seeks logits for which positives systematically outrank negatives with margin, both globally and pairwise.

The first component is a global Multi-Sample Contrastive Loss (MSCL), described as InfoNCE-style:
\[
\mathcal{L}_{\text{MSCL}}(\bm{\theta} \mid \mathcal{I}, \mathbf{x}^{(i)}) = - \log\frac{\sum_{v_i \in \mathcal{V}_{+}} \exp(z_i/\tau)}{\sum_{v_k \in \mathcal{V}_{0}} \exp(z_k/\tau)},
\]
where \(z_i\) is the logit of variable \(v_i\) and \(\tau\) is a temperature parameter [2511.09209]. The numerator is the total mass of positives and the denominator is the total mass of all variables. Minimizing this loss pushes positives collectively to receive higher logits than the rest.

The second component is a local pairwise hinge ranking loss:
\[
\mathcal{L}_{\text{rank}}(\bm{\theta} \mid \mathcal{I}, \mathbf{x}^{(i)}) =
\frac{1}{|\mathcal{V}_{+}| |\mathcal{V}_{-}|}
\sum_{v_i \in \mathcal{V}_{+}, v_j \in \mathcal{V}_{-}}
\max(0, \gamma - (z_i - z_j)),
\]
with margin \(\gamma>0\) [2511.09209]. Whenever \(z_i - z_j < \gamma\), the model is penalized; otherwise no pairwise penalty is incurred. This explicitly enforces a fine-grained ordering between variables labeled \(1\) and \(0\).

The combined VCL is
\[
\begin{aligned}
\mathcal{L}_{\text{VCL}}(\bm{\theta} \mid \mathcal{I})
&=\sum_{i=1}^{N}w_i\cdot\mathcal{L}_{\text{VCL}}(\bm{\theta} \mid \mathcal{I}, \mathbf{x}^{(i)}) \\
&= \sum_{i=1}^{N}w_i\cdot\left[
\mathcal{L}_{\text{MSCL}}(\bm{\theta} \mid \mathcal{I}, \mathbf{x}^{(i)})
+ \lambda_{\text{rank}}\cdot \mathcal{L}_{\text{rank}}(\bm{\theta} \mid \mathcal{I}, \mathbf{x}^{(i)})
\right],
\end{aligned}
\]
where the solution weights \(w_i\) are the same energy-based weights used in PS, satisfying \(w_i\propto \exp(-\bm{c}^\top \bm{x}^{(i)})\) [2511.09209]. The hyperparameter \(\lambda_{\text{rank}}\) controls the weight of the pairwise ranking term.

The paper attributes three key properties to VCL. First, it yields relational gradients: the gradient on a single logit depends on all others through MSCL and on all positive-negative pairs through the ranking term. Second, it is margin-enforcing: positives are not only raised, but explicitly pushed above negatives by a margin. Third, VCL replaces BCE rather than supplementing it; in the main formulation and ablations, no BCE term is retained [2511.09209]. Hyperparameters used in experiments include \(\tau=0.1\), \(\lambda_{\text{rank}}=0.01\), and \(\gamma\in[0.6,0.9]\), with \(0.9\) on CA, SC, and WA [2511.09209].

This design reflects a change in surrogate objective. BCE is criticized because it treats variables independently, does not enforce relative ordering, and can produce overlapping logit distributions in which many positives have lower scores than negatives [2511.09209]. CoCo-MILP instead reframes solution prediction as a ranking and contrastive task aligned with the solver’s downstream use of logits for ranking and fixing.

## 4. Architecture-level design: Intra-Constraint Competitive GNN

CoCo-MILP adopts the GNN encoder from Bachman et al. (2019), also used in PS and Apollo-MILP, and builds on its bipartite half-convolution structure [2511.09209]. Initialization is
\[
h^{(0)}_{v_i} = \mathrm{MLP}_\theta(v_i), \quad
h^{(0)}_{w_j} = \mathrm{MLP}_\theta(w_j), \quad
h_{e_{ij}} = \mathrm{MLP}_\theta(e_{ij}),
\]
followed by alternating updates over constraints and variables. The constraint update is
\[
h^{(k+1)}_{w_i} \leftarrow \mathrm{MLP}_\theta\left(
h^{(k)}_{w_i},
\sum_{j: e_{ij} \in \mathcal{E}}
\mathrm{MLP}_\phi\left(h^{(k)}_{w_i}, h_{e_{ij}}, h^{(k)}_{v_j}\right)
\right),
\]
and the variable update is
\[
h^{(k+1)}_{v_j} \leftarrow \mathrm{MLP}_\phi\left(
h^{(k)}_{v_j},
\sum_{i: e_{ij} \in \mathcal{E}}
\mathrm{MLP}_\phi\left(h^{(k+1)}_{w_i}, h_{e_{ij}}, h^{(k)}_{v_j}\right)
\right)
\]
for layers \(k=0,\dots,K-1\) [2511.09209]. Final variable embeddings use Jumping Knowledge:
\[
h_{v_i} = \mathrm{MLP}_\theta\left(\mathrm{CONCAT}_{k=0}^{K} h^{(k)}_{v_i}\right), \qquad
z_{v_i} = \mathrm{MLP}_\theta(h_{v_i}),
\]
with \(\hat{x}_{v_i} = \sigma(z_{v_i})\) [2511.09209].

The paper argues that this baseline encoder tends to smooth variable embeddings, especially when variables share many constraints. In MILPs, however, many constraints encode exclusionary relations in which only a few variables should become active. The ICC layer is therefore inserted after each message-passing layer on variables to model competition explicitly [2511.09209].

For each layer \(l\), let \(\bm{h}_j^{(l)}\) denote the variable embedding. ICC applies three steps. First, for each constraint \(c_k\), it computes a constraint-level average of competitor features:
\[
\overline{\bm{h}}_k^{(l)} \gets \frac{1}{|\mathcal{N}(c_k)|} \sum_{v_j \in \mathcal{N}(c_k)} \bm{h}_j^{(l)},
\]
where \(\mathcal{N}(c_k)\) is the set of variables in constraint \(k\) [2511.09209]. Second, for each variable \(v_j\), it aggregates competitive context from all attached constraints:
\[
\overline{\bm{h}}_j^{(l)} \gets \frac{1}{|\mathcal{N}(v_j)|} \sum_{c_k \in \mathcal{N}(v_j)} \overline{\bm{h}}_k^{(l)},
\]
where \(\mathcal{N}(v_j)\) denotes the set of constraints incident to \(v_j\) [2511.09209]. Third, it performs a competitive deviation update:
\[
\bm{h}_j^{(l)} \gets \bm{h}_j^{(l)} - \beta \cdot \overline{\bm{h}}_j^{(l)},
\]
where \(\beta\) is a learnable scalar per layer [2511.09209].

The intended effect is anti-smoothing. If a variable embedding is similar to the average embedding of its competitors, the subtraction reduces its salience; if it deviates from its peers, the updated embedding acquires larger magnitude and a more distinctive direction [2511.09209]. The paper emphasizes that ICC is simple, adds only one scalar \(\beta\) per layer, and does not rely on extra attention weights or multi-head parameters [2511.09209]. Empirically, ICC increases the variance of logits among variables within the same constraint, whereas baseline GNNs exhibit low intra-constraint variance.

A common misconception would be to interpret ICC as a global repulsion mechanism between all variables. The paper does not claim this. Competition is localized to the constraint neighborhoods induced by the bipartite graph, and the update is defined via averages over those local competitive groups [2511.09209].

## 5. Training setup, benchmarks, and solver integration

The prediction task is restricted to binary variables. CoCo-MILP outputs a vector \(\hat{\bm{x}}\in[0,1]^p\) over the binary subset of variables, while continuous variables are left to the solver [2511.09209]. Supervision is derived from multiple feasible solutions \(\{\bm{x}^{(i)}\}\) collected by running solvers, with each solution weighted by \(w_i \propto \exp(-\bm{c}^\top \bm{x}^{(i)})\) [2511.09209]. These solutions serve as supervised signals for VCL.

The evaluation uses four canonical benchmark classes following PS: Set Covering (SC), Combinatorial Auctions (CA), Item Placement (IP), and Workload Appointment (WA) [2511.09209]. The train/validation/test split is \(240/60/100\) instances per benchmark.

| Benchmark | Constraints | Variables |
|---|---:|---:|
| SC | 3000 | 5000 binary vars |
| CA | 2593 | 1500 binary vars |
| IP | 195 | 1083 vars (1050 binary + 33 continuous) |
| WA | 64306 | 61000 vars (1000 binary + 60000 continuous) |

In addition to these four benchmarks, the paper evaluates on MIPLIB 2017, including the IIS subset and additional subsets, where both binary and non-binary integer variables appear; training there follows a per-instance-group similarity-based selection [2511.09209]. This broadens the setting beyond purely synthetic or competition-style instances.

The pipeline itself has three stages. First, prediction: the CoCo-MILP GNN runs on the bipartite graph to obtain logits \(\bm{z}\) and marginals \(\hat{\bm{x}}\). Second, trust-region formulation: hyperparameters \((k_0,k_1,\Delta)\) are chosen to fix high-confidence variables, possibly unfix low-confidence ones, and impose the \(L_1\)-ball trust region around \(\hat{\bm{x}}\). Third, search: the reduced MILP is solved by Gurobi or SCIP for up to \(1000\) seconds [2511.09209]. In this sense, CoCo-MILP modifies the predictor and learning objective while preserving the search-side predict-and-search framework.

## 6. Empirical performance and ablation evidence

The baselines include solver-only configurations with Gurobi and SCIP, as well as the learning-based methods PS and ConPS. The appendices additionally report results when CoCo-MILP is plugged into Apollo-MILP [2511.09209]. The main comparison uses PS+Gurobi with a \(1000\)-second time limit and best-known solutions from \(3600\)-second Gurobi runs.

On CA, where the benchmark is maximization and the BKS is \(97524.37\), Gurobi achieves objective \(97228.93\) with absolute gap \(295.44\), PS+Gurobi achieves \(97286.29\) with gap \(238.08\), ConPS+Gurobi achieves \(97315.83\) with gap \(208.54\), and CoCo-MILP+Gurobi achieves \(97338.64\) with gap \(185.73\) [2511.09209]. On SC, with BKS \(125.05\), the corresponding objectives and gaps are \(125.21/0.16\) for Gurobi, \(125.17/0.12\) for PS+Gurobi, \(125.18/0.13\) for ConPS+Gurobi, and \(125.06/0.01\) for CoCo-MILP+Gurobi [2511.09209]. On IP, with BKS \(11.16\), Gurobi yields \(11.43/0.27\), PS+Gurobi yields \(11.40/0.24\), ConPS+Gurobi yields \(11.36/0.20\), and CoCo-MILP+Gurobi yields \(11.26/0.10\). On WA, with BKS \(703.05\), Gurobi, PS+Gurobi, and ConPS+Gurobi each report \(703.47/0.42\), whereas CoCo-MILP+Gurobi reports \(703.14/0.09\) [2511.09209].

The paper summarizes these improvements as a primal-gap reduction of up to \(68.12\%\) compared to traditional solvers across benchmarks [2511.09209]. It also reports runtime-quality curves showing that CoCo-MILP achieves lower average primal gap at almost all time horizons and reaches high-quality solutions within approximately \(100\) seconds [2511.09209]. This suggests that better ranking of binary variables translates into faster solver convergence, although the exact mechanism remains the standard trust-region search reused from PS.

The MIPLIB IIS subset results further indicate that CoCo-MILP is equal or better than PS and ConPS and at least as good as Gurobi on all five test instances [2511.09209]. For example, on ex1010-pi with BKS \(233\), Gurobi obtains \(239\), PS obtains \(241\), ConPS obtains \(239\), and CoCo-MILP obtains \(237\). On fast0507, CoCo-MILP matches the BKS of \(174\), while PS and ConPS overshoot to \(179\) [2511.09209].

The method is also not limited to Gurobi. With SCIP as the solver and a \(1000\)-second limit, CoCo-MILP consistently reduces the gap relative to SCIP and the learning baselines. On CA, for example, SCIP has objective \(96423\) and gap \(1100.47\), PS+SCIP has \(96426.46/1097.91\), ConPS+SCIP has \(96428.83/1095.54\), and CoCo-MILP+SCIP has \(96465.09/1059.28\) [2511.09209].

The ablation study isolates the contributions of VCL and ICC. On SC and CA with Gurobi and a \(1000\)-second limit, replacing VCL with BCE worsens performance to \(125.25\) on SC and \(97240.00\) on CA. Removing the ranking term yields \(125.21\) and \(97272.41\); removing MSCL yields \(125.19\) and \(97217.40\); removing ICC yields \(125.18\) and \(97315.96\); and the full model yields \(125.06\) and \(97338.64\) [2511.09209]. These results support the claim that both VCL components help, and that the ICC layer is crucial for maximal gain, particularly on CA.

Generalization experiments on larger SC and CA instances, with variables and constraints approximately doubled, show that a model trained on smaller instances can still improve solver performance when applied directly to larger ones. On larger CA, with BKS \(115787.97\), CoCo-MILP+Gurobi reaches objective \(115463.49\) and gap \(324.48\), improving on Gurobi’s \(114960.25/827.72\), PS+Gurobi’s \(115228.20/559.77\), and ConPS+Gurobi’s \(115343.23/444.74\). On larger SC, with BKS \(101.45\), CoCo-MILP+Gurobi achieves \(101.91/0.46\), improving on Gurobi’s \(102.29/0.84\), PS+Gurobi’s \(102.27/0.82\), and ConPS+Gurobi’s \(102.18/0.73\) [2511.09209].

The paper also reports modularity. Plugging the CoCo-MILP predictor into Apollo-MILP on SC achieves the BKS exactly, \(125.05\), and adding an intra-constraint competitive normalization at search time further improves SC to exactly the BKS when combined with CoCo-MILP [2511.09209].

## 7. Interpretation, limitations, and broader context

The explanatory argument advanced by CoCo-MILP is that MILP solutions are typically sparse, so the main difficulty is to identify the few variables that should be set to \(1\) [2511.09209]. Solvers use predicted probabilities primarily for ranking and fixing rather than for calibrated probabilistic estimation, so correct ordering matters more than marginal calibration. Constraints such as set-packing, knapsack, and assignment often impose competition, implying that local differentiation among variables within a constraint is more useful than embedding homogenization [2511.09209].

The paper’s analyses are consistent with this interpretation. Logit distribution visualizations show greater separation between positive and negative variables than PS. For sampled positive-negative pairs, CoCo-MILP yields a distribution of \(\Delta_{ij} = \hat{x}_i - \hat{x}_j\) that is more positively skewed, indicating better ranking. It also significantly increases the variance of logits within constraints, revealing stronger differentiation among competitors [2511.09209]. A plausible implication is that the method’s benefit derives less from improved probability calibration than from improved score geometry for downstream fixing and trust-region restriction.

Several limitations are explicitly stated. The distribution model remains factorized over binary variables; although VCL introduces relational structure in training, the model does not explicitly represent joint dependencies beyond what the GNN encoder can encode [2511.09209]. The pairwise ranking term has cost \(O(|\mathcal{V}_+||\mathcal{V}_-|)\), and while it can be subsampled in practice, the paper does not detail subsampling heuristics. VCL introduces hyperparameters \(\lambda_{\text{rank}}, \tau, \gamma\), which require tuning, and performance also depends on the search-side trust-region parameters \((k_0,k_1,\Delta)\), even though the method is described as reasonably robust within ranges [2511.09209]. At the same time, ICC is characterized as lightweight, and the overall overhead is described as small relative to MILP solving time [2511.09209].

Within the broader landscape of ML for MILP, CoCo-MILP belongs specifically to the solution-prediction branch, rather than to learning branching policies, cut selection, separator learning, or large-neighborhood-search policy learning [2511.09209]. Its two conceptual contributions are loss design tailored to relative ordering of variables and architecture design tailored to competition within constraints. The authors emphasize that these ideas are orthogonal to many existing frameworks and can be integrated into stronger systems such as Apollo-MILP, as shown experimentally [2511.09209].

Future directions identified in the paper include extending the contrastive and competitive paradigm to more general integer variables via transformations, broader problem families including the full MIPLIB and industrial domains, tighter integration with solver components such as branch heuristics or node scoring, joint training with search-time feedback, and more efficient sampling strategies for pairwise ranking on very large-scale MILPs [2511.09209]. The work also suggests compatibility with instance-generation frameworks such as G2MILP and MILP-StuDio for robust training [2511.09209]. This suggests a research program in which MILP predictors are designed not only around graph structure, but around the competitive combinatorial semantics encoded by constraints and the ranking-oriented demands of downstream search.

Source: https://www.emergentmind.com/topics/coco-milp