---
title: Restarted Halpern PDHG Methods
url: https://www.emergentmind.com/topics/restarted-halpern-pdhg-methods
type: topic
---

# Restarted Halpern PDHG Methods

Restarted Halpern PDHG methods are a recent class of matrix-free first-order algorithms for linear programming (LP) that blend the primal-dual hybrid gradient (PDHG) scheme, Halpern fixed-point acceleration, algorithmic restarts, and (optionally) reflected operators. These methods are notable for achieving accelerated linear convergence under sharpness (non-degeneracy) conditions, supporting adaptive and theoretically-motivated restarts, and straightforward GPU implementation. Their development and analysis have resolved several open questions on first-order complexity in large-scale linear programs and have advanced state-of-the-art LP solvers, notably in high-accuracy and massively parallel settings [2407.16144], [2507.14051], [2410.04043].

## 1. Linear Programming Problem Framework

Restarted Halpern PDHG methods target linearly constrained convex programs, especially standard-form or box-constrained LPs. A representative box-constrained LP takes the form:

- **Primal variables:** \( x \in X = \{x \in \mathbb{R}^n : \ell_v \le x \le u_v \} \)
- **Slack and dual variables:** \( s = A x \in S = \{s \in \mathbb{R}^m : \ell_c \le s \le u_c \} \), dual variable \( y \)
- **Objective:** \( \min_{x \in X} c^\top x \)  
- **Saddle-point reformulation:**  
  \[
  \min_{x \in X} \max_{y \in Y} \; c^\top x + y^\top (A x) - p(y; -u_c, -\ell_c)
  \]
  where the box-penalty is \( p(y; \ell, u) = u^\top y^+ - \ell^\top y^- \) [2507.14051].

The canonical setup for restarted Halpern PDHG and its variants is the saddle-point problem
\[
\max_{y \in Y} \min_{x \in X} L(y, x)
\]
with \( L \) structured to accommodate nonnegativity, affine equality, and general box constraints.

## 2. Algorithmic Structure: Halpern and Reflected PDHG with Restarts

The standard PDHG scheme, foundational for this class, applies a proximal point and explicit-gradient step per iteration:

\[
\begin{aligned}
  x^{k+1} &= \operatorname{proj}_{X}(x^k - \tau(c - A^\top y^k)) \\
  y^{k+1} &= y^k - \sigma A(2x^{k+1} - x^k) - \sigma \operatorname{proj}_{-S}(\sigma^{-1}y^k - A(2x^{k+1} - x^k))
\end{aligned}
\]
with step sizes \( \tau, \sigma \) [2507.14051], [2407.16144].

**Halpern-PDHG acceleration** introduces an "anchor" point \( z^0 \) and an averaging sequence:
\[
z^{k+1} = \frac{k+1}{k+2} T(z^k) + \frac{1}{k+2} z^0
\]
where \( T \) denotes the PDHG operator. This interpolation sharply accelerates fixed-point convergence, in contrast to standard Polyak–Krasnosel'skiu iterations [2407.16144].

**Reflected operator enhancement:** Given the nonexpansive property of \( T \), the reflection operator,
\[
R_\gamma(z) = (1+\gamma)T(z) - \gamma z
\]
(usually with \( \gamma = 1 \), i.e., the full reflection \( R = 2T - I \)), can replace \( T \) within the Halpern framework. The reflected variant, called r²HPDHG, achieves a further constant-factor speedup in theory and practice [2407.16144], [2507.14051].

**Restart mechanism:** The algorithm alternates inner Halpern (or reflected Halpern) PDHG epochs with outer restarts. Each epoch starts from the last point, performs Halpern updates until a restart criterion is triggered, then resets the anchor to (typically) a PDHG iterate of the last inner point [2407.16144]:

- **Restart triggers:** Sufficient decay, necessary decay but no local progress, or iteration/time budget exhaustion (artificial timeout) [2507.14051].
- **Residual-based condition:** A prevalent adaptive trigger is
  \[
  r(z^{n,k}) \leq \beta_{\mathrm{suf}} \; r(z^{n,0}) \quad \text{where} \quad r(z) = \| z - \mathrm{PDHG}(z) \|_P
  \]
  with \( \|\cdot\|_P \) a block-diagonal or canonical PDHG norm [2507.14051], [2407.16144].

## 3. Convergence Properties and Complexity Analysis

Restarted Halpern PDHG methods guarantee accelerated linear convergence under LP sharpness or non-degeneracy. The essential sharpness parameter is the constant \( \alpha_\eta > 0 \) such that for bounded iterates,
\[
\alpha_\eta \; \mathrm{dist}(z, \mathcal{Z}^*) \leq \|z - T(z)\|
\]
where \( \mathcal{Z}^* \) is the solution set [2407.16144].

**Global convergence:**
- **Geometric decay:** The epoch anchors contract to optimality with a geometric factor, yielding
  \[
  \mathrm{dist}(z^{n+1,0}, \mathcal{Z}^*) \leq (1/e)^{n+1} \mathrm{dist}(z^{0,0}, \mathcal{Z}^*)
  \]
  and thus require \( O(\frac{1}{\alpha_\eta} \log(1/\varepsilon)) \) PDHG-evaluations to reach tolerance \( \varepsilon \) [2407.16144].

**Two-stage complexity:** The convergence comprises:
- **Stage I:** Active-set identification, with iteration count depending on a non-degeneracy metric \( \delta \) and sharpness over the active cone.
- **Stage II:** Local convergence near the optimal face, with improved (larger) sharpness, resulting in rapid geometric contraction.

**Reflected variant acceleration:** The r²HPDHG variant reduces all complexity bounds by a factor of 2 and empirically achieves a 20–30% speedup [2407.16144].

**Infeasibility detection:** In the infeasible case, the methods recover Farkas certificates with global linear rates, significantly improving upon the sublinear convergence of vanilla PDHG [2407.16144].

**Condition-measure complexity:** For unique-optimum LPs, the accessible iteration bound is
\[
\widetilde{O}\left(\kappa \Phi \cdot \ln\left(\frac{\|w^*\|}{\varepsilon}\right)\right)
\]
where \( \kappa \) is the condition number of \( A \), and \( \Phi \) a geometric sublevel-set condition number closely related to perturbation stability, degeneracy proximity, and LP sharpness [2410.04043].

## 4. Restart Criteria and Parameter Selection

The design of restart rules is central for both theoretical and empirical performance. Typical criteria include:

- **Sufficient decay**: If fixed-point residual has dropped below a fraction of its epoch-initial value.
- **Necessary decay with no local progress**: If the residual is below a looser threshold but stagnates or increases.
- **Artificial epoch timeout**: If a maximal inner iteration count is reached before sufficient decay occurs.

The specific thresholds \( \beta_{\mathrm{suf}}, \beta_{\mathrm{nec}} \in (0,1) \) and iteration budgets \( T_n^{\max} \) are user-defined [2507.14051].

**Step-size and reweighting heuristics:** Balancing primal and dual updates (e.g., by adaptively tuning step-size ratio to \(\|x^k\|_1 : \|s^k\|_1\)) minimizes the key complexity term \( \Phi \) and aligns with practical success in large-scale LP solvers [2410.04043].

## 5. GPU-Oriented Implementation: cuPDLPx and HPDLP

Modern variants such as cuPDLPx implement the restarted Halpern PDHG algorithm in Julia + CUDA.jl, exploiting the following features for scalability [2507.14051]:

- Sparse matrix-vector and projection operations fully offloaded to GPU custom kernels.
- Fused proximal steps to minimize global memory traffic.
- Reflection/interpolation logic realized as a single vector-scale–add (VAXPY) kernel for efficiency.
- Warp- and block-reduction patterns optimized for contemporary hardware (e.g., NVIDIA H100).
- Minimal synchronization and constant step-size schedules for kernel efficiency.
- PID-controlled primal weight (\( \omega \)) update performed on GPU for step-size balancing.

These choices achieve high arithmetic intensity and bandwidth utilization, yielding substantial speedups: 2.5x–5x on standard MIPLIB LP relaxations and up to 6.8x on Mittelmann's benchmark set, with the best gains in high-accuracy and presolve-enabled scenarios [2507.14051].

## 6. Comparative Performance and Practical Impact

Experimental results validate the efficacy of restarted Halpern PDHG algorithms:
- rHPDHG matches or surpasses prior state-of-the-art GPU PDHG solvers (cuPDLP.jl) in shifted geometric mean solve time and solution accuracy.
- The reflected variant (r²HPDHG) consistently delivers a 20–30% reduction in solve times relative to both rHPDHG and standard PDHG [2407.16144], [2507.14051].
- Performance gains are especially notable in moderate to high accuracy and large-scale benchmarks.

Theoretically, these methods offer:
- Accelerated two-stage convergence—rapid active-set identification followed by fast local contraction—without dependence on the global Hoffman constant.
- Linear-time recovery of infeasibility certificates.
- Complexity bounds directly tied to computable geometric and condition measures, providing accessible guidance for problem scaling and parameter selection [2410.04043].

## 7. Connections to Sharpness, Condition Measures, and Algorithm Selection

The sharpness constant \( \alpha_\eta \), geometric condition number \( \Phi \), and classical Hoffman constants collectively govern algorithmic efficiency. There is a reciprocal relationship between iteration complexity and measures of:
- Data-stability against cost/right-hand side perturbations;
- Proximity to degeneracy and multiple optima;
- Local and global LP sharpness.

A plausible implication is that tailoring step-sizes and restarting policies based on real-time estimation of these quantities can yield further improvements in convergence robustness and generalization to broader convex-concave saddle-point problems [2407.16144], [2410.04043].

---

**References**
- "cuPDLPx: A Further Enhanced GPU-Based First-Order Solver for Linear Programming" [2507.14051]
- "Restarted Halpern PDHG for Linear Programming" [2407.16144]
- "Accessible Theoretical Complexity of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programs with Unique Optima" [2410.04043]

Source: https://www.emergentmind.com/topics/restarted-halpern-pdhg-methods