Parallel-in-Time Preconditioners

• Parallel-in-Time Preconditioners are advanced algorithmic strategies that decouple temporal dependencies to enable scalable computations for time-dependent PDEs and optimal control problems.
• They leverage matrix preconditioners and time-transform techniques like DST/FFT to achieve mesh-independent convergence in Krylov subspace solvers.
• Their design ensures significant parallel efficiency with theoretical O(log N) complexity, robust performance, and minimal overhead on modern high-performance computing architectures.

Parallel-in-time preconditioners are advanced algorithmic strategies designed to accelerate the solution of large-scale time-dependent problems—especially evolutionary partial differential equations and optimal control—by enabling scalable and robust temporal parallelism. These methods are distinguished by the design of matrix preconditioners that exploit the temporal block-structured nature of all-at-once discretizations, allowing efficient application via diagonalization in time (typically using fast Fourier or sine transforms). By decoupling or reducing inter-time-step dependencies, parallel-in-time preconditioners unlock significant concurrency on modern high-performance computing architectures and offer mesh-independent convergence in Krylov subspace solvers, fundamentally enhancing large-scale simulation capabilities for parabolic and related PDEs.

1. Algorithmic Foundation: Saddle-Point Reformulation and Time-Global Structure

Parallel-in-time preconditioners are often developed in the context of all-at-once systems resulting from implicit time discretizations (e.g., implicit Euler) of parabolic evolution equations with spatial finite-element discretization ($M$ for mass, $A_n$ for stiffness, see (1802.08126)). After stacking unknowns for all time steps into a single vector $\mathbf{u} = [u_1, ..., u_N]$, the coupled time-global system takes a nonsymmetric block form: $$\mathbf{B}\mathbf{u} = \mathbf{f},\qquad \mathbf{B} = \mathbf{K} + \mathbf{A},$$ where $\mathbf{K}$ (time-coupling, bidiagonal) and $\mathbf{A}$ (block-diagonal, spatial) result from time-stepping. To facilitate efficient preconditioning and analysis, the system is equivalently reformulated as a symmetric saddle-point problem via inf-sup theory, introducing an auxiliary variable $\mathbf{p}$ and yielding a block system: $$\begin{bmatrix} \mathbf{A} & -\mathbf{K} \ -\mathbf{K}^\top & -(\mathbf{K} + \mathbf{K}^\top + \mathbf{A}) \end{bmatrix} \begin{bmatrix} \mathbf{p} \ \mathbf{u} \end{bmatrix} = \begin{bmatrix} \mathbf{f} \ \dots \end{bmatrix}.$$

The system is then solved iteratively, typically using inexact Uzawa methods, which enable decoupling and parallelization of certain key operations.

2. Construction of Parallel-in-Time Preconditioners

A central technical advance is the design of non-intrusive, easily implementable preconditioners leveraging the temporal structure (1802.08126, 1810.00615):

• Block-diagonalization in time is achieved via the discrete sine transform (DST) or, for other systems, discrete Fourier or circulant transforms.
• For the Schur complement $\mathbf{S}$ in the saddle-point problem,

$$\mathbf{S} = \mathbf{K}^\top \mathbf{A}^{-1} \mathbf{K} + \mathbf{K} + \mathbf{K}^\top + \mathbf{A},$$

the corresponding parallel-in-time preconditioner $\mathbf{H}$ is constructed such that

$$\mathbf{H} = \mathbf{\Phi}^\top \hat{\mathbf{H}} \mathbf{\Phi},\quad \hat{\mathbf{H}} = \frac{N}{2\tau}\operatorname{diag}\left\{H_k A^{-1} H_k\right\}_{k=1}^N,$$

where each $H_k = \mu_k M + \tau A$ is associated with a temporal frequency mode $\mu_k = 2\sin\left(\frac{(2k - 1)\pi}{4N}\right)$.

• The preconditioner acts block-diagonally in the time-transform basis, allowing application of spatial solvers at each time-frequency independently—this is the crucial mechanism enabling high temporal parallelism.

The preconditioner is non-intrusive: it does not modify or intrude upon existing spatial solvers, instead reusing them in a black-box fashion within a new temporal framework.

3. Theoretical Properties: Spectral Bounds and Complexity

Parallel-in-time preconditioners exhibit strong theoretical guarantees:

• Spectral Equivalence: The preconditioner achieves robust spectral equivalence with the Schur complement,

$$\frac{1}{2\alpha}\mathbf{H} \leq \mathbf{S} \leq 3\alpha\mathbf{H},$$

where $\alpha$ is a constant determined by spatial and temporal quasi-uniformity.

• Mesh-Independent Convergence: The convergence factor $U$ for the inexact Uzawa method depends solely on preconditioner quality and remains independent of the number of time steps $N$, time window length, or mesh size.
• Parallel Complexity: With sufficient processors, the theoretical parallel complexity per iteration is $O(\log N)$ in the number of time steps, plus costs for spatial matrix-vector products and preconditioning.
• Stability: The inf-sup-based formulation ensures the solution is stable in norms at least as strong as the sup-in-time energy norm.

These properties guarantee that as the number of time steps or spatial refinements grows, both the number of Krylov iterations and the time-to-solution remain robust—critical for large-scale problems.

4. Practical Implementation: Parallelization and Cost Distribution

Implementation strategies—demonstrated in large-scale numerical experiments (see (1802.08126))—emphasize:

• Parallelization in the time direction: the DST/FFT-based block-diagonalization step requires minimal inter-processor communication and is inexpensive relative to spatial preconditioning.
• Parallel application of spatial solvers (e.g., algebraic multigrid, direct methods) to independent blocks post-diagonalization.
• Time-scaling studies reveal:
  • Weak scaling: With fixed work per processor, runtime per iteration is nearly constant even with massive problem and processor counts (over $100,000$ cores).
  • Strong scaling: Increasing processors for fixed problem size reduces wall-clock time nearly linearly.
  • Combined space-time parallelism: Real-world limits are set primarily by communication costs and the efficiency of the spatial solver.
• Overheads: Experiments demonstrate that the computational cost of the DST/FFT transform is typically below 10% of total iteration time; the majority is spent on spatial solve kernels.

The preconditioning approach is scalable, enabling rapid solution for extremely large space-time systems (up to billions of degrees of freedom).

5. Inf-Sup Theory and Its Algorithmic Impact

The methodology is deeply grounded in inf-sup stability theory for parabolic problems:

• The saddle-point reformulation is inf-sup stable in the natural parabolic norms, with no loss of coercivity or stability constants relative to the underlying continuous problem.
• The preconditioner is designed to match the optimal inf-sup norm, making the spectral equivalence physically meaningful and directly justified by the mathematical structure.
• Unlike classical “normal equation” stabilization (which can introduce unwanted damping or loss of temporal regularity), this approach maintains sharp stability bounds and convergence rates.

This theoretical underpinning permits direct, robust transfer of stability from continuous PDEs to large-scale discretized systems.

6. Comparative Perspective and Extension

Parallel-in-time preconditioning frameworks as in (1802.08126) compare favorably to other strategies:

• Block circulant and $\alpha$-circulant preconditioners (1810.00615, 2003.07020) are similar in using spectral–diagonalization techniques, but the inf-sup-based approach achieves mesh-independent stability for a broader class of parabolic problems with time-dependent operators.
• Serial time-stepping and standard block preconditioning lack the global-in-time viewpoint, leading to limited or absent time parallelism.
• Domain decomposition and Schur complement reduction methods may provide spatial parallelism, but do not inherently address time parallelism.
• The design is non-intrusive and can be layered atop existing spatial solver infrastructures without significant code restructuring.

The inf-sup framework and block-diagonalization are also extensible to more general time-stepping schemes, systems with time-dependent spatial operators, and coupled multiphysics settings.

7. Numerical Results and Performance

A suite of numerical experiments (as reported in (1802.08126)) demonstrates:

• Uniformly low iteration counts: Independent of $N$ (number of time steps), mesh refinement, or spatial solver inaccuracy (e.g., using only a few multigrid cycles).
• Scalability: Good weak and strong scaling up to $130,000$+ compute cores; overall wall clock times under $20$ seconds for over $2\cdot 10^9$ unknowns.
• Dominant cost location: Spatial solve component, with time-transform overhead minor.
• Condition number estimates: Preconditioned system condition numbers below $4$, regardless of discretization.

These results empirically validate the theoretical mesh- and parameter-independent properties, confirming the effectiveness of parallel-in-time preconditioners in real-world large-scale computational contexts.

---

Summary Table

Aspect	Key Points / Results
Formulation	All-at-once, symmetric saddle-point reformulation via inf-sup theory
Preconditioner	DST-based, block-diagonal in time, spatially non-intrusive, robust spectral bounds
Theory	Mesh-, step-size-, and time-independent convergence; rapid $O(\log N)$ parallel complexity
Implementation	Minimal overhead for time transforms, high parallel efficiency, scalable to $100$k+ cores
Numerical	Uniformly small iteration counts, excellent scaling, robust to spatial discretization choice
Inf-Sup Theory	Guarantees optimal norm equivalence, structure-preserving, beyond normal-equation methods

---

Parallel-in-time preconditioners, as developed in this framework, represent a robust, theoretically justified, and practically effective approach for solving large-scale parabolic evolution equations. Their inf-sup-based design yields strong scalability and mesh-independent performance, enabling the practical solution of problems previously considered intractable on extreme-scale parallel architectures.