Learning to Control Stabilization in Column Generation

Abstract: Column generation is a widely used decomposition technique for large-scale linear programs, but it often suffers from slow convergence due to poor initial dual estimates and dual oscillations. Stabilization techniques such as smoothing and penalization can mitigate these issues, but their effectiveness depends heavily on parameter selection, which requires careful tuning to avoid degrading performance. This paper presents a common framework for smoothing and penalization, showing that despite their different mechanisms, both are governed by two design choices: a reference point in the dual space and stabilization parameters that regulate how strongly that reference influences pricing. Within this framework, we derive parameter bounds that ensure progress, analyze predicted duals as reference points, and establish convergence guarantees for both methods. These results motivate and guide the design of RLSCG, a reinforcement learning-guided framework that adaptively selects stabilization parameters at each iteration. Computational experiments on the Cutting Stock Problem show that RLSCG substantially reduces solution time and iteration count on most synthetic and benchmark instances relative to traditional column generation, rule-based adaptive stabilization, and learning-based column selection, with the largest gains on large-scale instances.

• The paper introduces RLSCG, which learns to control stabilization parameters in column generation using a reinforcement learning framework.
• It unifies smoothing and penalization techniques to address pricing distortions and converges reliably, offering substantial computational savings.
• Empirical results on large-scale instances show that RLSCG is robust to degeneracy and outperforms traditional and rule-based stabilization methods.

Reinforcement Learning-Guided Stabilization in Column Generation

Introduction and Motivation

Column Generation (CG) is a fundamental decomposition technique for large-scale linear programming and integer programming, especially prevalent in branch-and-price algorithms across diverse domains. However, classical CG suffers from two principal convergence pathologies: the heading-in effect (arising from poor initial dual estimates, causing inefficient exploration early on) and the tailing-off effect (manifested by degeneracy-induced dual oscillations, stalling progress during later iterations). To mitigate these, stabilization strategies—such as smoothing and penalization—are used to regularize dual advancement during pricing. These strategies, however, rely critically on the choice of stabilization parameters and the selection of reference points, necessitating heuristic or case-specific tuning that may fail to generalize across problem instances.

"Learning to Control Stabilization in Column Generation" (2604.23889) introduces an adaptive framework for stabilization by formulating parameter selection as a sequential decision process solved via reinforcement learning (RL). The authors propose RL-guided stabilization column generation (RLSCG), which leverages a unified framework for both smoothing and penalization schemes and learns stabilization policies from structural and algorithmic data during the solution process. Empirical evaluation on the Cutting Stock Problem (CSP) demonstrates strong performance improvements—most markedly on large instances—relative to both classical CG, rule-based stabilization, and RL-guided column selection.

Unified Framework for Stabilization

The paper formalizes both smoothing and penalization within a common stabilization paradigm parameterized by:

• A reference dual point $\pi^{\mathrm{ref}}$,
• Stabilization parameters $\theta$,
• A stabilization operator $f(\pi^{\mathrm{ref}}, \theta; \text{RMP})$ that produces the stabilized dual for pricing.

Smoothing computes the pricing dual as

$$\pi^{\mathrm{sep}} = \alpha \pi^{\mathrm{ref}} + (1 - \alpha) \pi^{\mathrm{out}},$$

where $\pi^{\mathrm{out}}$ is the RMP dual and $\alpha$ is the smoothing parameter.

Penalization augments the RMP objective by penalizing deviations from $\pi^{\mathrm{ref}}$, via either polyhedral or quadratic (proximal) penalties, modifying the RMP and yielding the stabilized dual through its optimal solution.

A critical unifying concept is pricing distortion: when the stabilized dual deviates from $\pi^{\mathrm{out}}$, the reduced cost computation distorts the selection of new columns, with consequences ranging from beneficial exploration to detrimental mispricing. The authors rigorously characterize the safe regions for stabilization (in terms of dual deviations) where convergence speed-up is possible without loss of theoretical guarantees.

Convergence and Parameter Analysis

By analyzing the interplay between reference selection and stabilization parameters, the authors derive:

• Explicit upper bounds on the stabilization parameters that guarantee the pricing oracle returns columns that strictly cut off the current dual (preventing mispricing and progress stalling).
• Convergence conditions for both classes of stabilization: Provided that stabilization is eventually reduced (aggressiveness annealed—either through shrinking $\alpha$ or decaying penalties to zero), global convergence is ensured even in the presence of aggressive initial pricing distortions.
• The impact of reference selection: Historical duals and ML-predicted duals serve as references. Penalization is robust to the choice, but smoothing requires switching to standard updates eventually for convergence (hybrid reference strategy).

RL-Guided Stabilization Column Generation (RLSCG)

RL Formulation

RLSCG frames the selection of stabilization parameters as a Markov Decision Process. At iteration $t$, the agent observes a state $\theta$0 (encapsulating the current RMP and progress metrics), selects an action $\theta$1 (the stabilization parameter), and receives a reward signal $\theta$2 driven by primal/dual progress and iteration count.

State Representation

The state combines:

• A bipartite graph encoding of the current RMP (columns and constraints as nodes, with edge structure induced by nonzero coefficients), processed via a graph neural network to aggregate structural features.
• Global features summarizing algorithmic progress, recent dual moves, mispricing proxies, and stability indicators.

Action and Reward

The discrete action space comprises possible values for the stabilization parameter (e.g., smoothing weight $\theta$3 or symmetric penalty $\theta$4). The reward structure penalizes excessive iteration count, incorporates normalization for progress, and delivers a large terminal reward upon convergence.

Policy Learning and Execution

A deep RL agent is used (via a DQN variant); the model is trained offline across diverse synthetic CSP instances. At deployment, the learned policy adaptively chooses stabilization strength at each CG iteration conditioned on structural and dynamic signals, balancing exploration (through permitting dual deviation for potentially faster progress) and exploitation (by retracting to safe domains when necessary).

Empirical Evaluation

The authors perform extensive experiments on both randomly generated and real benchmark CSP instances.

Convergence Speed and Efficiency

RLSCG—when instantiated for both smoothing and penalization—substantially reduces the number of iterations and total computation time compared to traditional CG. For large-scale and degenerate instances (e.g., when standard CG may fail to terminate within the time limit), RLSCG consistently achieves convergence.

Notable results for large instances:

• On Gen_1 Large (Figure 1), RLSCG(S) and RLSCG(P) provide computational savings exceeding an order of magnitude over traditional and baseline stabilized CG.

Figure 1: Gen_1 Large: RLSCG significantly outperforms baseline stabilization methods and traditional CG in total solution time for large-scale instances.

Adaptive vs. Rule-Based Stabilization

Comparisons with adaptive, non-learning parameter schedules (such as ASCG-1/ASCG-2 for smoothing and APCG-1/APCG-2 for penalization) reveal that RL-learned policies consistently dominate both in iteration count and computation time, especially as instance complexity grows. The learned policy is nontrivial and instance-dependent; fixed or hand-crafted rules are consistently suboptimal.

Impact of Reference Points

A hybrid strategy using ML-predicted duals as initial references (switching to adaptive/historical references upon stagnation) enhances both rule-based and RL-driven algorithms. However, RLSCG’s performance remains less sensitive and more robust across the variety of structures, indicating effective credit assignment in the RL agent’s policy.

RL-Guided Stabilization vs. RL-Guided Column Selection

A head-to-head with RL-based column selection (e.g., RLCG [chi2022deep]) shows that RLSCG achieves superior computational time, even if RLCG occasionally matches or beats iteration count. The key reason: learning to directly distort duals prior to pricing (via stabilization) yields more efficient solution paths than learning to select from a larger candidate pool post-pricing.

Figure 2: Gen_1 Large: Distribution of computation times of RLSCG and rule-based CG with or without ML-predicted duals, showing RLSCG's robust improvement.

Sensitivity to Degeneracy

On highly degenerate CSP instances (e.g., Gen_4), where the separation margin is small, aggressive stabilization is less effective but RLSCG remains competitive, demonstrating generality and reliability.

Distributional Robustness

The boxplots for computation times across datasets (Figures 8, 9, 10) show that RLSCG not only achieves lower mean computation times but also exhibits dramatically reduced variance, indicating robustness to outlier instances.

Figure 3: Computation time distributions for ASCG-1 (rule-based smoothing) across reference strategies, with RLSCG sharply reducing both median and variance.

Figure 4: Computation time distributions for ASCG-2 across reference strategies, showing similar trends favoring RLSCG.

Figure 5: Computation time distributions for RLSCG(S) under various reference strategies, highlighting both lower mean and tighter spread compared to baselines.

Theoretical and Practical Implications

This work reconciles stabilization in CG under a shared theoretical and algorithmic lens and demonstrates for the first time that stateful, RL-guided parameter adaptation outperforms not just static, but also hand-crafted adaptive stabilization rules—even when equipped with high-quality ML-predicted duals as references. The provided safe parameter regimes and convergence proofs ensure that RL-induced exploration is always pulled back to the neutral regime required for global optimality. The proposed structural state encoding (via GNNs) offers a scalable template for integrating complex, combinatorial structure into policy learning—potentially extensible to branch-and-price, multi-row relaxations, and other nontrivial decompositions.

On the practical side, the empirical study suggests RLSCG is particularly effective on:

• Large instances with significant degeneracy,
• Problems with expensive pricing or hard-to-tune stabilization,
• Cases where human-crafted rules are delicate or impractical.

Future Directions

Open avenues include:

• Extending RLSCG to richer problem domains with more complex pricing (e.g., vehicle routing, crew scheduling),
• Integrating multi-column addition and adaptive candidate pool sizing jointly with stabilization control,
• Understanding RL policy behavior for additional interpretability and transfer, and
• Exploring sample-efficiency and generalization for RL agents in other combinatorial optimization regimes.

Conclusion

The RL-guided stabilization paradigm for column generation offers a principled, theoretically sound, and practically impactful advancement for accelerating large-scale LP decomposition. The generality and empirical robustness of RLSCG, combined with unified parameter analysis, position it as a new reference approach for learning-assisted optimization pipelines where dual regularization is a limiting bottleneck.