Restarted Halpern PDHG Methods

• The paper introduces restarted Halpern PDHG methods that fuse PDHG, Halpern acceleration, and restart mechanisms to achieve linear convergence in LPs under sharpness conditions.
• They utilize algorithmic innovations like reflected operators and adaptive restart criteria in a matrix-free, GPU-friendly framework to enhance large-scale linear programming performance.
• The methods demonstrate practical impact through accelerated solve times, improved complexity bounds, and effective infeasibility detection in high-accuracy LP benchmarks.

Restarted Halpern PDHG methods are a recent class of matrix-free first-order algorithms for linear programming (LP) that blend the primal-dual hybrid gradient (PDHG) scheme, Halpern fixed-point acceleration, algorithmic restarts, and (optionally) reflected operators. These methods are notable for achieving accelerated linear convergence under sharpness (non-degeneracy) conditions, supporting adaptive and theoretically-motivated restarts, and straightforward GPU implementation. Their development and analysis have resolved several open questions on first-order complexity in large-scale linear programs and have advanced state-of-the-art LP solvers, notably in high-accuracy and massively parallel settings (Lu et al., 23 Jul 2024, Lu et al., 18 Jul 2025, Xiong, 5 Oct 2024).

1. Linear Programming Problem Framework

Restarted Halpern PDHG methods target linearly constrained convex programs, especially standard-form or box-constrained LPs. A representative box-constrained LP takes the form:

• Primal variables: $$x \in X = \{x \in \mathbb{R}^n : \ell_v \le x \le u_v \}$$
• Slack and dual variables: $$s = A x \in S = \{s \in \mathbb{R}^m : \ell_c \le s \le u_c \}$$, dual variable $y$
• Objective: $\min_{x \in X} c^\top x$
• Saddle-point reformulation:

$$\min_{x \in X} \max_{y \in Y} \; c^\top x + y^\top (A x) - p(y; -u_c, -\ell_c)$$

where the box-penalty is $p(y; \ell, u) = u^\top y^+ - \ell^\top y^-$ (Lu et al., 18 Jul 2025).

The canonical setup for restarted Halpern PDHG and its variants is the saddle-point problem

$\max_{y \in Y} \min_{x \in X} L(y, x)$

with $L$ structured to accommodate nonnegativity, affine equality, and general box constraints.

2. Algorithmic Structure: Halpern and Reflected PDHG with Restarts

The standard PDHG scheme, foundational for this class, applies a proximal point and explicit-gradient step per iteration:

$$\begin{aligned} x^{k+1} &= \operatorname{proj}_{X}(x^k - \tau(c - A^\top y^k)) \ y^{k+1} &= y^k - \sigma A(2x^{k+1} - x^k) - \sigma \operatorname{proj}_{-S}(\sigma^{-1}y^k - A(2x^{k+1} - x^k)) \end{aligned}$$

with step sizes $\tau, \sigma$ (Lu et al., 18 Jul 2025, Lu et al., 23 Jul 2024).

Halpern-PDHG acceleration introduces an "anchor" point $z^0$ and an averaging sequence: $$z^{k+1} = \frac{k+1}{k+2} T(z^k) + \frac{1}{k+2} z^0$$ where $T$ denotes the PDHG operator. This interpolation sharply accelerates fixed-point convergence, in contrast to standard Polyak–Krasnosel'skiu iterations (Lu et al., 23 Jul 2024).

Reflected operator enhancement: Given the nonexpansive property of $T$, the reflection operator,

$R_\gamma(z) = (1+\gamma)T(z) - \gamma z$

(usually with $\gamma = 1$, i.e., the full reflection $R = 2T - I$), can replace $T$ within the Halpern framework. The reflected variant, called r²HPDHG, achieves a further constant-factor speedup in theory and practice (Lu et al., 23 Jul 2024, Lu et al., 18 Jul 2025).

Restart mechanism: The algorithm alternates inner Halpern (or reflected Halpern) PDHG epochs with outer restarts. Each epoch starts from the last point, performs Halpern updates until a restart criterion is triggered, then resets the anchor to (typically) a PDHG iterate of the last inner point (Lu et al., 23 Jul 2024):

• Restart triggers: Sufficient decay, necessary decay but no local progress, or iteration/time budget exhaustion (artificial timeout) (Lu et al., 18 Jul 2025).
• Residual-based condition: A prevalent adaptive trigger is

$$r(z^{n,k}) \leq \beta_{\mathrm{suf}} \; r(z^{n,0}) \quad \text{where} \quad r(z) = \| z - \mathrm{PDHG}(z) \|_P$$

with $\|\cdot\|_P$ a block-diagonal or canonical PDHG norm (Lu et al., 18 Jul 2025, Lu et al., 23 Jul 2024).

3. Convergence Properties and Complexity Analysis

Restarted Halpern PDHG methods guarantee accelerated linear convergence under LP sharpness or non-degeneracy. The essential sharpness parameter is the constant $\alpha_\eta > 0$ such that for bounded iterates,

$$\alpha_\eta \; \mathrm{dist}(z, \mathcal{Z}^*) \leq \|z - T(z)\|$$

where $\mathcal{Z}^*$ is the solution set (Lu et al., 23 Jul 2024).

Global convergence:

• Geometric decay: The epoch anchors contract to optimality with a geometric factor, yielding

$$\mathrm{dist}(z^{n+1,0}, \mathcal{Z}^*) \leq (1/e)^{n+1} \mathrm{dist}(z^{0,0}, \mathcal{Z}^*)$$

and thus require $O(\frac{1}{\alpha_\eta} \log(1/\varepsilon))$ PDHG-evaluations to reach tolerance $\varepsilon$ (Lu et al., 23 Jul 2024).

Two-stage complexity: The convergence comprises:

• Stage I: Active-set identification, with iteration count depending on a non-degeneracy metric $\delta$ and sharpness over the active cone.
• Stage II: Local convergence near the optimal face, with improved (larger) sharpness, resulting in rapid geometric contraction.

Reflected variant acceleration: The r²HPDHG variant reduces all complexity bounds by a factor of 2 and empirically achieves a 20–30% speedup (Lu et al., 23 Jul 2024).

Infeasibility detection: In the infeasible case, the methods recover Farkas certificates with global linear rates, significantly improving upon the sublinear convergence of vanilla PDHG (Lu et al., 23 Jul 2024).

Condition-measure complexity: For unique-optimum LPs, the accessible iteration bound is

$$\widetilde{O}\left(\kappa \Phi \cdot \ln\left(\frac{\|w^*\|}{\varepsilon}\right)\right)$$

where $\kappa$ is the condition number of $A$, and $\Phi$ a geometric sublevel-set condition number closely related to perturbation stability, degeneracy proximity, and LP sharpness (Xiong, 5 Oct 2024).

4. Restart Criteria and Parameter Selection

The design of restart rules is central for both theoretical and empirical performance. Typical criteria include:

• Sufficient decay: If fixed-point residual has dropped below a fraction of its epoch-initial value.
• Necessary decay with no local progress: If the residual is below a looser threshold but stagnates or increases.
• Artificial epoch timeout: If a maximal inner iteration count is reached before sufficient decay occurs.

The specific thresholds $$\beta_{\mathrm{suf}}, \beta_{\mathrm{nec}} \in (0,1)$$ and iteration budgets $T_n^{\max}$ are user-defined (Lu et al., 18 Jul 2025).

Step-size and reweighting heuristics: Balancing primal and dual updates (e.g., by adaptively tuning step-size ratio to $\|x^k\|_1 : \|s^k\|_1$) minimizes the key complexity term $\Phi$ and aligns with practical success in large-scale LP solvers (Xiong, 5 Oct 2024).

5. GPU-Oriented Implementation: cuPDLPx and HPDLP

Modern variants such as cuPDLPx implement the restarted Halpern PDHG algorithm in Julia + CUDA.jl, exploiting the following features for scalability (Lu et al., 18 Jul 2025):

• Sparse matrix-vector and projection operations fully offloaded to GPU custom kernels.
• Fused proximal steps to minimize global memory traffic.
• Reflection/interpolation logic realized as a single vector-scale–add (VAXPY) kernel for efficiency.
• Warp- and block-reduction patterns optimized for contemporary hardware (e.g., NVIDIA H100).
• Minimal synchronization and constant step-size schedules for kernel efficiency.
• PID-controlled primal weight ($\omega$) update performed on GPU for step-size balancing.

These choices achieve high arithmetic intensity and bandwidth utilization, yielding substantial speedups: 2.5x–5x on standard MIPLIB LP relaxations and up to 6.8x on Mittelmann's benchmark set, with the best gains in high-accuracy and presolve-enabled scenarios (Lu et al., 18 Jul 2025).

6. Comparative Performance and Practical Impact

Experimental results validate the efficacy of restarted Halpern PDHG algorithms:

• rHPDHG matches or surpasses prior state-of-the-art GPU PDHG solvers (cuPDLP.jl) in shifted geometric mean solve time and solution accuracy.
• The reflected variant (r²HPDHG) consistently delivers a 20–30% reduction in solve times relative to both rHPDHG and standard PDHG (Lu et al., 23 Jul 2024, Lu et al., 18 Jul 2025).
• Performance gains are especially notable in moderate to high accuracy and large-scale benchmarks.

Theoretically, these methods offer:

• Accelerated two-stage convergence—rapid active-set identification followed by fast local contraction—without dependence on the global Hoffman constant.
• Linear-time recovery of infeasibility certificates.
• Complexity bounds directly tied to computable geometric and condition measures, providing accessible guidance for problem scaling and parameter selection (Xiong, 5 Oct 2024).

7. Connections to Sharpness, Condition Measures, and Algorithm Selection

The sharpness constant $\alpha_\eta$, geometric condition number $\Phi$, and classical Hoffman constants collectively govern algorithmic efficiency. There is a reciprocal relationship between iteration complexity and measures of:

• Data-stability against cost/right-hand side perturbations;
• Proximity to degeneracy and multiple optima;
• Local and global LP sharpness.

A plausible implication is that tailoring step-sizes and restarting policies based on real-time estimation of these quantities can yield further improvements in convergence robustness and generalization to broader convex-concave saddle-point problems (Lu et al., 23 Jul 2024, Xiong, 5 Oct 2024).

---

References

• "cuPDLPx: A Further Enhanced GPU-Based First-Order Solver for Linear Programming" (Lu et al., 18 Jul 2025)
• "Restarted Halpern PDHG for Linear Programming" (Lu et al., 23 Jul 2024)
• "Accessible Theoretical Complexity of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programs with Unique Optima" (Xiong, 5 Oct 2024)