PID-Controlled Primal Weight Update

• PID-Controlled Primal Weight Update is an adaptive strategy that uses PID feedback to adjust primal weights in iterative optimization algorithms.
• It integrates into primal-dual solvers by dynamically tuning step-size parameters to balance the progression of both primal and dual variables.
• Empirical evaluations on cuPDLP+ show up to 4x speedup and improved robustness against ill-conditioned problems compared to heuristic methods.

A PID-controlled primal weight update is an adaptive control strategy that applies proportional-integral-derivative (PID) feedback principles to the updating of primal weights or step-size weights in iterative optimization algorithms, especially within the context of primal-dual first-order methods and large-scale convex optimization solvers. The main objective is to dynamically adjust these weights to balance and regulate progress between the primal and dual variables, thereby achieving faster convergence and improved numerical stability. This methodology has gained practical traction in high-performance linear programming solvers such as cuPDLP+ (Lu et al., 18 Jul 2025), where efficient hardware utilization and adaptability to problem heterogeneity are critical.

1. Principle and Formulation of PID-Controlled Update

The core mechanism introduces a feedback loop for updating the primal weight, denoted as ω, which plays a pivotal role in controlling the scaling relationships between primal and dual step-sizes in primal-dual splitting methods. The update operates as a PID controller on a logarithmic measure of the mismatch between observed primal and dual progress during an epoch:

• Error Signal: At restart iteration n, define the logarithmic (relative) error

$$e^n = \log\left( \frac{ \sqrt{w^n} \| x^{(n,t)} - x^* \|_2 }{ (1/\sqrt{w^n}) \| y^{(n,t)} - y^* \|_2 } \right)$$

where $x^{(n,t)}, y^{(n,t)}$ are the latest primal and dual iterates, $w^n$ the current primal weight, and $(x^*, y^*)$ the unknown optimums (approximated by moving averages in practice).

• PID Rule: The new primal weight for the next epoch is computed as

$$\log(w^{n+1}) = \log(w^n) - \left[ K_P e^n + K_I \sum_{i=1}^{n} e^i + K_D (e^n - e^{n-1}) \right]$$

where $K_P$, $K_I$, and $K_D$ are tunable proportional, integral, and derivative gains, respectively.

This dynamic balancing ensures that when there is relative lagging in primal or dual progress, the weight is adjusted to re-equilibrate, with the feedback mechanisms dampening oscillations (D), compensating for drift (I), and reacting to present imbalance (P).

2. Integration in Primal-Dual First-Order Solvers

In cuPDLP+ (Lu et al., 18 Jul 2025), the PID-controlled update is embedded in a GPU-based first-order linear programming solver employing the restarted Halpern PDHG algorithm. At each algorithmic restart (epoch), the PID error is computed and the primal weight updated accordingly. The primal and dual stepsizes, $\tau$ and $\sigma$, are reparameterized in terms of this weight:

• $\tau = \eta/\omega$
• $\sigma = \eta \omega$ where $\eta$ is a fixed step-size parameter (e.g., $\eta = 0.99 / \|A\|_2$ with $A$ the constraint matrix).

The PID update process requires only the most recent and prior epoch's error measurements, thus incurring minimal computational and memory overhead on GPU hardware and fitting naturally into the restart-based scheme of the solver.

3. Motivations and Advantages

The motivation for PID-controlled primal weight updates derives from the observation that fixed or heuristically-tuned primal weights fail to adaptively track evolving problem conditioning or local progress between primal and dual variables. Prior approaches in cuPDLP employed exponential averaging without explicit control-theoretic feedback. The PID controller addresses this by providing:

• Adaptive balancing: Rapid correction of imbalances in primal versus dual convergence.
• Stability: Damping oscillations and mitigating divergence risks attributable to poor scaling, especially on ill-conditioned problems.
• Systematic tuning: The use of PID gains brings control-theoretic intuition and allows for explicit tuning procedures.

4. Empirical Effectiveness

Extensive benchmarking on 383 MIPLIB LP relaxations (Lu et al., 18 Jul 2025) demonstrates that the PID-controlled update in cuPDLP+ yields substantial empirical gains:

• Speedup: Achieves overall 2x-4x speedup compared to earlier cuPDLP without PID control, with up to 4.6x improvement on more difficult instances.
• High-accuracy regimes: In settings requiring high-precision solutions and with presolve enabled, the PID-controlled method enables aggressive step-sizes without compromising solution quality.
• Robustness: Maintains solution accuracy and convergence stability across a diverse range of problem scales and numerical properties.

While the paper does not isolate the sole contribution of the PID update from other enhancements, the integration of this feedback mechanism is identified as a key factor in the overall performance gains.

5. Comparison with Previous and Alternative Methods

Earlier implementations (e.g., the original cuPDLP) used either fixed primal weights or heuristic, non-feedback-based exponential averaging of scaling factors. These lacked the systematic dampening and quick adaptation properties of PID feedback.

• PID vs. Heuristic Averaging:
  • PID control introduces dynamic, feedback-driven adjustments informed by past and present imbalances, in contrast to memoryless or static heuristics.
  • The integral and derivative terms permit both compensation for persistent drifts and anticipation of trends, improving geometric convergence.
• Alternative Tuning Schemes:
  • Some literature employs restart heuristics or per-iteration adjustment based on observed convergence rates, but without the formal feedback stabilization of PID.

A plausible implication is that PID-based weight updates offer an extensible framework—parameterizable and potentially automatable—for real-time adaptation in other optimization contexts beyond first-order methods for LP.

6. Implementation Considerations

Efficient implementation on GPU architectures, as realized in cuPDLP+, requires:

• Parallelism: The update rule must be vectorizable and lightweight so as not to degrade the overall throughput of the massively parallel matrix operations.
• Synchronization: PID updates are computed only at the epoch (restart) boundaries, minimizing communication and synchronization overhead.
• Numerical Smoothing: Since distance estimates to the optimum are not directly available, the solver uses exponential averaging to estimate $\|x^{(n,t)} - x^*\|_2$ and $\|y^{(n,t)} - y^*\|_2$ from observed iterates, further stabilizing updates.

Careful selection and tuning of the PID gains ($K_P$, $K_I$, $K_D$) remain important for optimal performance; the control perspective provides guidance for these choices.

7. Impact and Future Directions

The PID-controlled primal weight update concept demonstrates significant practical impact in large-scale optimization, particularly for GPU-based solvers targeting high-throughput and high-accuracy applications. Its feedback-driven adaptivity opens opportunities for further integration with other advanced step-size rules, restart strategies, and hardware-specific optimizations.

A plausible implication is that this control-theoretic mechanism could be synthesized with machine learning–based meta-parameter tuning and be generalized to broader classes of primal-dual and operator splitting algorithms, contributing to the design of autonomous, self-regulating optimization solvers.