Diagonal-Parallel Picard in Fixed-Point Methods

• Diagonal-Parallel Picard is a fixed-point method variant that leverages diagonal approximations to decouple computations for efficient parallel execution.
• It finds application in state-space Jacobian approximations, collocation preconditioning, and coordinate decoupling in reflected G-BSDEs, enhancing computational tractability.
• The approach improves efficiency via parallel scans while balancing trade-offs between approximation accuracy and potential numerical instabilities.

Searching arXiv for papers directly relevant to “Diagonal-Parallel Picard” and adjacent fixed-point/parallel-Picard formulations. Diagonal-Parallel Picard is not a standardized method name in the recent arXiv literature. The exact phrase does not appear in the principal papers most closely associated with parallel Picard-style methods. Instead, the phrase is best read as a compact label for several closely related constructions: Picard-style fixed-point iterations made parallel by a diagonal approximation, a diagonal preconditioner, or a coordinatewise decoupling that leaves only a scan-, map-, or component-parallel subproblem. In the literature on parallelizing sequential models, the closest explicit meaning is a diagonal-Jacobian quasi-Newton update inside a unifying linear dynamical systems framework; in spectral deferred corrections and collocation, it corresponds more literally to diagonal-preconditioned Picard iteration across collocation nodes; and in multidimensional reflected $G$-BSDEs it denotes Picard iteration under diagonal generators, where each frozen-coordinate update is a one-dimensional reflected equation (Gonzalez et al., 26 Sep 2025, Speck, 2017, Li et al., 2021).

1. Terminological status and scope

Several papers explicitly state that the exact phrase “Diagonal-Parallel Picard” does not appear in their text. In the parallel-sequential-model literature, the nearest named objects are Picard iteration with identity transition, quasi-Newton iteration with diagonal Jacobian approximation, and more generally parallel fixed-point methods obtained by choosing structured approximations of the Jacobian that remain closed under composition. In diffusion and Metropolis sampling, the nearest objects are global Picard updates over an entire trajectory, sometimes with a Jacobi-like time-parallel interpretation or with a moving online frontier rather than a diagonal matrix structure (Gonzalez et al., 26 Sep 2025, So et al., 25 Mar 2025, Grazzi et al., 11 Jun 2025).

This suggests that “Diagonal-Parallel Picard” functions less as a canonical algorithm name than as an umbrella description spanning at least three meanings of “diagonal.” In one meaning, “diagonal” refers to a diagonal Jacobian approximation in state space. In another, it refers to a diagonal preconditioner in collocation-node space. In a third, it refers only informally to a diagonal or frontier-like propagation pattern through the time-index/iteration grid. The common invariant across these uses is a Picard-style fixed-point backbone combined with a structural simplification that exposes parallelism.

2. Parallel fixed-point iteration for sequential models

A central recent formulation studies the sequential nonlinear recursion

$x_{t+1}=f_{t+1}(x_t),$

recasts rollout as a fixed-point problem over the full trajectory, and shows that Newton, quasi-Newton, Picard, and Jacobi all fit the same linear-dynamical-systems update

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$

The method is determined by the choice of $A_{t+1}$: full Newton uses the full Jacobian, quasi-Newton uses the diagonal Jacobian, Picard uses $I_D$, and Jacobi uses $0$. The paper states explicitly that “Picard iterations are in fact a type of quasi-Newton iteration, where we approximate the Jacobian of the dynamics function by the identity matrix,” while the diagonal approximation is

$$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$

Within this framework, the strict paper-grounded interpretation of “Diagonal-Parallel Picard” is therefore not Picard in the paper’s own terminology, but the diagonal quasi-Newton member of the same family (Gonzalez et al., 26 Sep 2025).

The same paper makes the computational reason precise. Once $A_{1:T}$ and the previous iterate are fixed, the update becomes an affine linear time-varying recurrence over $t$, and affine-map composition is associative. The full trajectory can therefore be evaluated by a parallel scan in $\mathcal{O}(\log T)$ depth on $x_{t+1}=f_{t+1}(x_t),$0 processors. The diagonal structure matters because diagonal matrices are closed under multiplication, giving a cheap scan representation. The paper records the cost gap explicitly: full Newton requires $x_{t+1}=f_{t+1}(x_t),$1 memory and $x_{t+1}=f_{t+1}(x_t),$2 work, whereas diagonal quasi-Newton requires $x_{t+1}=f_{t+1}(x_t),$3 space and $x_{t+1}=f_{t+1}(x_t),$4 work (Gonzalez et al., 26 Sep 2025).

A later thesis places the same family inside a broader parallel Newton framework. There the generic update is

$x_{t+1}=f_{t+1}(x_t),$5

with $x_{t+1}=f_{t+1}(x_t),$6 for Newton, $x_{t+1}=f_{t+1}(x_t),$7 for diagonal quasi-DEER, $x_{t+1}=f_{t+1}(x_t),$8 for Picard, and $x_{t+1}=f_{t+1}(x_t),$9 for Jacobi. The thesis is explicit that the phrase “Diagonal-Parallel Picard” is not defined there; the closest explicit method is diagonal quasi-DEER, while Picard itself remains the identity approximation (Gonzalez, 17 Mar 2026).

3. Diagonal-preconditioned Picard in collocation and spectral deferred correction

In collocation-based time integration, Picard iteration arises from the collocation system

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$0

and spectral deferred correction is written as a preconditioned Picard iteration

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$1

Classical SDC uses a lower-triangular $$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$2, which makes the sweep Gauss-Seidel-like and serial across nodes. The diagonal-parallel variant replaces that by a diagonal $$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$3, so that all node updates decouple and can be computed simultaneously. One paper tests three such diagonal preconditioners: $$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$4

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$5

and

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$6

This is the most literal sense in which the literature contains a diagonal-parallel Picard construction: a Picard or SDC sweep whose preconditioner is diagonal in node space (Speck, 2017).

A later paper makes the relationship sharper by stating that plain Picard iteration is recovered by choosing $$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$7, while diagonal SDC is a diagonal-preconditioned Picard iteration. It proposes analytically designed diagonal sweepers, including

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$8

for which

$$x_{t+1}^{(i+1)}=f_{t+1}(x_t^{(i)})+A_{t+1}\bigl(x_t^{(i+1)}-x_t^{(i)}\bigr).$$9

and a nonstationary family

$A_{t+1}$0

for which the stiff-limit propagator product vanishes after $A_{t+1}$1 sweeps. In that paper’s formulation, diagonal-parallel SDC is “Picard iteration for collocation” plus diagonal implicit preconditioning, rather than Picard in the strict unpreconditioned sense (Čaklović et al., 2024).

4. Time-parallel trajectory Picard in diffusion and Metropolis sampling

A separate line of work applies Picard iteration not to state-space Jacobians or collocation matrices, but to an entire denoising or Markov trajectory. For diffusion sampling, the trajectory-form Picard update is

$A_{t+1}$2

This is parallel across denoising time because all timestep states are updated from the previous global iterate. The paper explicitly says that it does not introduce a new diagonal block decomposition; if mapped to “diagonal-parallel” language, the closest analogy is a Jacobi-like global update over the time dimension. Its main contribution is the Picard Consistency Model, trained to predict the fixed-point trajectory more directly, together with model switching to recover exact convergence to the base sampler. The reported headline gains are “up to a 2.71x speedup over sequential sampling” and “a 1.77x speedup over Picard iteration” (So et al., 25 Mar 2025).

For zeroth-order Metropolis chains, the relevant object is the Online Picard algorithm. The chain

$A_{t+1}$3

is rewritten via the Picard map

$A_{t+1}$4

The online scheme advances the largest already-correct prefix and reallocates processors forward, producing what the paper describes as a moving frontier through the iteration-versus-time plane. The exact phrase “Diagonal-Parallel Picard” is not used, but this is the closest object in the paper to a diagonal-front propagation interpretation. The exact algorithm gives $A_{t+1}$5 parallel iterations with $A_{t+1}$6 processors for Random Walk Metropolis under the stated assumptions, while an approximate variant is claimed in the abstract to generate samples from an approximate measure $A_{t+1}$7 in $A_{t+1}$8 parallel iterations and $A_{t+1}$9 processors; the body adds that a rigorous analysis of the invariant-measure error is outside the scope of the work (Grazzi et al., 11 Jun 2025).

5. Diagonal generators and coordinate-parallel Picard for reflected $I_D$0-BSDEs

In multidimensional reflected $I_D$1-BSDEs, “diagonal” has a different but very precise meaning. The $I_D$2-th generators are

$I_D$3

so each component may depend on the full vector $I_D$4, but in the $I_D$5-variable depends only on the scalar $I_D$6. The reflected equation is

$I_D$7

with obstacle $I_D$8 and the Skorohod-type condition that

$I_D$9

is a non-increasing $0$0-martingale. The Picard method freezes the cross-coordinate $0$1-dependence through a process $0$2, defines

$0$3

and then solves $0$4 one-dimensional reflected $0$5-BSDEs independently on a short interval. The associated map $0$6 is a contraction in $0$7 for sufficiently small horizon $0$8, and global existence follows by backward patching (Li et al., 2021).

This is arguably the cleanest instance of a genuinely coordinate-parallel Picard method in the supplied literature. The diagonal assumption on the generators is not merely a convenience. It is the structural reason that each frozen-coordinate update remains a one-dimensional reflected $0$9-BSDE, so the established one-dimensional theory applies componentwise. Without diagonal $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$0-dependence, the per-coordinate reflected solve would not decouple.

6. Convergence regimes, trade-offs, and limitations

In the scan-based sequential-model framework, Newton, diagonal quasi-Newton, Picard, and Jacobi all share a finite-step propagation property: in the sequence-evaluation setting they are guaranteed to converge in at most $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$1 iterations. The practical number of iterations, however, depends on how well the approximation $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$2 or $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$3 matches the true Jacobian and on whether the induced linearized recurrence is stable. The resulting regime picture is explicit: Picard works well when the Jacobian is close to $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$4, diagonal quasi-Newton works well when state coordinates are nearly uncoupled, and full Newton works best when off-diagonal coupling is important (Gonzalez et al., 26 Sep 2025, Gonzalez, 17 Mar 2026).

The same sources emphasize that cheaper structure can increase iteration count and can also create numerical instability. One paper notes that “LDS matrices with spectral norm close to or greater than one can cause numerical instabilities in the parallel scan operation.” The thesis sharpens the theoretical side by giving a convergence-rate decomposition in terms of approximation accuracy and stability, and its abstract states that “the sign of the Largest Lyapunov Exponent of a dynamical system determines whether or not parallel Newton methods converge quickly” (Gonzalez et al., 26 Sep 2025, Gonzalez, 17 Mar 2026).

For collocation, the trade-off is analogous. Diagonal SDC or diagonalized collocation exposes node-level concurrency, but usually gives weaker preconditioning than serial LU-like sweeps. One paper reports that diagonal choices can be nearly as good in non-stiff regimes, while for stiffer problems the diagonal-minimizing choice is usually the strongest among diagonal methods and the LU-based choice is usually best overall. Another shows that carefully optimized diagonal sweepers can recover stability domains and convergence behavior close to serial SDC while preserving stage parallelism (Speck, 2017, Čaklović et al., 2024).

7. Distinction from unrelated uses of “diagonal” and “Picard”

The phrase should not be confused with unrelated subjects that happen to contain the same words. In combinatorics, “parallel diagonals” refers to diagonals of polygons parallel to a fixed edge, as in the enumeration of triangulations by number of parallel diagonals (Regev, 2012). In algebraic geometry, “Picard rank” or “Picard number” refers to the rank of a divisor class group or Néron–Severi group, as in work on diagonal quartic surfaces with Picard number $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$5 and infinitely many rational points (Bremner et al., 2014). In toric geometry, “diagonal” may refer to the diagonal sheaf on $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$6 and “Picard rank $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$7” to the rank of $$A_{t+1}=\operatorname{diag}\!\left[\frac{\partial f_{t+1}}{\partial x_t}(x_t^{(i)})\right].$$8, again with no algorithmic relation to parallel Picard iteration (Brown et al., 2022).

Within numerical analysis and machine learning, by contrast, the most defensible encyclopedic definition is narrower: a Diagonal-Parallel Picard method is a Picard-style fixed-point scheme whose parallelism is exposed by a diagonal or structurally separable approximation. Depending on context, that diagonal structure may sit in a Jacobian, a collocation preconditioner, a generator, or only an informal iteration-versus-time propagation pattern. The literature does not presently standardize the phrase, but it repeatedly standardizes the underlying ingredients.