Double-Preconditioning (DoPr) Techniques

• Double-Preconditioning (DoPr) is a family of two-stage designs that combine complementary preconditioning actions (e.g., left/right scalings, nested Schur complements) to improve algorithm robustness.
• It is applied across numerical linear algebra, interval analysis, saddle-point systems, operator preconditioning, and deep learning to directly control condition numbers and robustness criteria.
• These methods enable tighter control over key metrics—such as spectral bounds and convergence rates—compared to single-sided techniques, enhancing performance in both classical and modern applications.

Searching arXiv for the cited DoPr usages to ground the article in the relevant papers. Double-preconditioning, commonly abbreviated DoPr, is not a single universally standardized construction but a recurring name for several technically distinct preconditioning paradigms across numerical linear algebra, interval analysis, operator preconditioning, saddle-point Krylov methods, and deep-learning optimization. Across these settings, the shared motif is the coordinated use of two complementary transformations—most often left/right matrix scalings, nested Schur complements, or dual perturbation controls—to improve conditioning, regularity checks, solver robustness, or downstream rollout performance. In the optimization literature, DoPr denotes the search for positive diagonal row and column scalings $D_\ell$ and $D_r$ minimizing $\kappa(D_\ell A D_r)$ (Qu et al., 2022). In interval-parametric linear systems, it denotes splitting $(A^c)^{-1}$ into two factors and preconditioning from both sides to sharpen strong-regularity tests and solution enclosures (Skalna et al., 2020). In PDE-constrained optimization, it refers to a double Schur-complement preconditioner for a $3\times 3$ saddle-point system (Pearson et al., 2021). In Petrov–Galerkin operator preconditioning, a bi-parametric framework controls perturbations in both the original form and the preconditioner (Escapil-Inchauspé et al., 2020). In recent deep-learning work, DoPr combines gradient-wise and activation-wise preconditioning to target test-time performance under feedback dynamics rather than one-step validation loss (Zhang et al., 4 Jun 2026).

1. Terminological scope and recurring structure

The term double-preconditioning is used in multiple research programs for constructions that involve two coordinated preconditioning actions rather than a single left preconditioner or a single adaptive scaling. This shared naming convention is substantive rather than merely linguistic: each usage introduces a paired mechanism designed to address a limitation of one-sided or single-parameter conditioning.

In the diagonal-scaling setting, a “double-preconditioner” is explicitly “a pair of positive diagonal matrices $D_\ell\in\mathbb R^{m\times m}$ and $D_r\in\mathbb R^{n\times n}$ so that we replace $A$ by $D_\ell A D_r$” (Qu et al., 2022). In interval-parametric systems, the midpoint inverse is split into two factors, yielding a left-right transformation $H(p)=L\,A(p)\,R$ (Skalna et al., 2020). In the double saddle-point setting, the “double” component arises from two nested Schur complements,

$D_r$0

which define the block-diagonal preconditioner $D_r$1 (Pearson et al., 2021). In bi-parametric operator preconditioning, the dual structure is encoded by two perturbation parameters, $D_r$2 for the original form and $D_r$3 for the preconditioner (Escapil-Inchauspé et al., 2020). In deep learning, DoPr is defined as the composition of activation preconditioning and gradient-wise preconditioning, with the first stage computing $D_r$4 and the second stage applying a base optimizer such as AdamW or Muon (Zhang et al., 4 Jun 2026).

This plurality matters for interpretation. A common misconception is that DoPr refers to one specific algorithm. The literature instead supports a broader encyclopedic reading: DoPr is a family of two-stage or two-sided preconditioning designs whose exact mathematical content depends on domain-specific structure. This suggests that the unifying principle is architectural rather than formal: two complementary conditioning operations are used to target quantities that a one-sided method leaves poorly controlled.

2. Optimal diagonal preconditioning in numerical linear algebra

In "Optimal Diagonal Preconditioning" (Qu et al., 2022), DoPr is the systematic search for the best possible row and column scalings of a full-rank matrix with respect to the spectral condition number. Let $D_r$5 be an $D_r$6 real or complex matrix. The objective is to choose positive diagonal matrices $D_r$7 and $D_r$8 to minimize

$D_r$9

where $\kappa(D_\ell A D_r)$0 for invertible $\kappa(D_\ell A D_r)$1.

The paper formulates the problem as

$\kappa(D_\ell A D_r)$2

Although this objective is nonconvex jointly in $\kappa(D_\ell A D_r)$3, the relevant feasibility sets are quasi-convex in the pair. For any $\kappa(D_\ell A D_r)$4, the constraint $\kappa(D_\ell A D_r)$5 can be rewritten through two spectral-norm inequalities and then as linear matrix inequalities:

$\kappa(D_\ell A D_r)$6

This produces a bisection scheme on $\kappa(D_\ell A D_r)$7, initialized with $\kappa(D_\ell A D_r)$8 and $\kappa(D_\ell A D_r)$9 or another upper bound, with each midpoint checked through the LMI feasibility problem (Qu et al., 2022).

The same work develops an interior-point method for the feasibility subproblem. Writing the diagonal entries of $(A^c)^{-1}$0 and $(A^c)^{-1}$1 as $(A^c)^{-1}$2 and $(A^c)^{-1}$3, and defining

$(A^c)^{-1}$4

$(A^c)^{-1}$5

the barrier is

$(A^c)^{-1}$6

Newton steps are then computed from the Hessian of $(A^c)^{-1}$7, with line search preserving positivity of $(A^c)^{-1}$8, $(A^c)^{-1}$9, and the slack matrices. Because the feasibility problem lives in a self-scaled cone, the Nesterov–Todd direction yields $3\times 3$0 iteration complexity for certifying feasibility to accuracy $3\times 3$1 (Qu et al., 2022).

The paper also identifies a one-sided specialization. If $3\times 3$2 and only right-scaling is allowed, then

$3\times 3$3

reduces to the convex SDP

$3\times 3$4

Empirically, the work reports that optimal diagonal preconditioners can significantly improve on heuristic diagonal preconditioners for reducing condition numbers and speeding up iterative methods, and that customized solvers with random row/column sampling can find near-optimal diagonal preconditioners for matrices up to size $3\times 3$5 in reasonable time (Qu et al., 2022). The associated summary further states that, against standard one-sided equilibration, DoPr typically achieves another $3\times 3$6–$3\times 3$7 reduction in spectral condition number, and that on sparse $3\times 3$8 PDE matrices, DoPr-preconditioned CG ran in roughly $3\times 3$9–$D_\ell\in\mathbb R^{m\times m}$0 of the iterations and wall-clock time needed with row-only scaling (Qu et al., 2022).

3. Double preconditioning for interval-parametric linear systems

In "On preconditioning and solving an extended class of interval parametric linear systems" (Skalna et al., 2020), double preconditioning is introduced for interval-parametric linear systems

$D_\ell\in\mathbb R^{m\times m}$1

with particular emphasis on affine-linear dependence,

$D_\ell\in\mathbb R^{m\times m}$2

The goal is to enclose the united solution set

$D_\ell\in\mathbb R^{m\times m}$3

preferably through a parametric, or $D_\ell\in\mathbb R^{m\times m}$4-solution,

$D_\ell\in\mathbb R^{m\times m}$5

The usual starting point is single left preconditioning by the midpoint inverse. With midpoint parameter $D_\ell\in\mathbb R^{m\times m}$6 and midpoint matrix $D_\ell\in\mathbb R^{m\times m}$7, one sets $D_\ell\in\mathbb R^{m\times m}$8 and forms

$D_\ell\in\mathbb R^{m\times m}$9

In the affine-linear case, this yields an interval hull

$D_r\in\mathbb R^{n\times n}$0

and a standard sufficient condition for strong regularity is $D_r\in\mathbb R^{n\times n}$1. The paper’s central point is that failure of this test under single left preconditioning does not imply failure of the original system; instead, two-sided splitting may recover tractability (Skalna et al., 2020).

DoPr splits the midpoint inverse as

$D_r\in\mathbb R^{n\times n}$2

and then preconditions from both sides:

$D_r\in\mathbb R^{n\times n}$3

In the affine-linear case,

$D_r\in\mathbb R^{n\times n}$4

Strong regularity is again checked by $D_r\in\mathbb R^{n\times n}$5, after which the transformed variable $D_r\in\mathbb R^{n\times n}$6 is solved and mapped back as $D_r\in\mathbb R^{n\times n}$7 (Skalna et al., 2020).

The algorithmic procedure given in the paper comprises midpoint computation, a nonsingularity check on $D_r\in\mathbb R^{n\times n}$8, a choice of factors $D_r\in\mathbb R^{n\times n}$9 and $A$0 such that $A$1, formation of $A$2, rejection if $A$3, application of an interval-affine solver such as Parametric Krawczyk to the transformed system, and final recovery of $A$4 (Skalna et al., 2020).

A key illustrative example uses

$A$5

With midpoint $A$6, single preconditioning gives

$A$7

so the method fails. Choosing instead

$A$8

yields

$A$9

so the transformed system becomes solvable (Skalna et al., 2020).

The paper explicitly argues that splitting $D_\ell A D_r$0 into $D_\ell A D_r$1 and $D_\ell A D_r$2 can reduce the size of the interval coefficients in $D_\ell A D_r$3 relative to $D_\ell A D_r$4, thereby reducing $D_\ell A D_r$5 and decreasing overestimation in interval computations (Skalna et al., 2020). It also notes that the choice of $D_\ell A D_r$6 is not unique, that LU-based splitting is advocated as the best trade-off in practice, and that one may also consider SVD- or spectral-based splittings when the coefficient matrices have special structure (Skalna et al., 2020). A plausible implication is that, in this literature, DoPr functions less as a conditioning tool in the classical $D_\ell A D_r$7 sense than as a mechanism for reshaping interval dependence so that spectral-radius regularity criteria become informative.

4. Double Schur-complement preconditioning for saddle-point systems

In "Double Saddle-Point Preconditioning for Krylov Methods in the Inexact Sequential Homotopy Method" (Pearson et al., 2021), DoPr denotes a block preconditioner for the $D_\ell A D_r$8 system arising after elimination of trivial active-set rows and a symmetric permutation. For the remaining unknowns $D_\ell A D_r$9, the linear system is

$H(p)=L\,A(p)\,R$0

The blocks satisfy

$H(p)=L\,A(p)\,R$1

with corresponding Jacobian blocks $H(p)=L\,A(p)\,R$2 and $H(p)=L\,A(p)\,R$3 (Pearson et al., 2021).

The defining construction is the use of two nested Schur complements,

$H(p)=L\,A(p)\,R$4

From these, the ideal block-diagonal preconditioner is

$H(p)=L\,A(p)\,R$5

Under the assumptions

$H(p)=L\,A(p)\,R$6

one has $H(p)=L\,A(p)\,R$7 and $H(p)=L\,A(p)\,R$8, so that $H(p)=L\,A(p)\,R$9 is SPD and $D_r$00 is diagonalizable with real spectrum (Pearson et al., 2021).

Theorem 3.3, as summarized in the supplied material, gives tight spectral bounds:

$D_r$01

implying

$D_r$02

independently of mesh size, $D_r$03, or $D_r$04 (Pearson et al., 2021). This spectral information is central for Krylov performance. The summary further states that MINRES or CG on the symmetric base system converges at a rate bounded by

$D_r$05

so that in practice fewer than $D_r$06 iterations are needed, independent of mesh size (Pearson et al., 2021).

Application of $D_r$07 requires three block solves: approximately solving with $D_r$08, with $D_r$09, and with $D_r$10. The paper emphasizes that $D_r$11 and $D_r$12 are not formed explicitly. Instead, $D_r$13 is approximated through a solve with

$D_r$14

while $D_r$15 is applied via a matching factorization

$D_r$16

leading to two nested sparse solves or AMG applications (Pearson et al., 2021).

The numerical results reported for a nonlinear $D_r$17D benchmark problem reach up to $D_r$18M degrees of freedom on $D_r$19 cores. The provided table gives, for $D_r$20, $D_r$21 DOFs, $D_r$22 outer iterations, $D_r$23 total MINRES iterations, and $D_r$24 minutes total time; even at this scale, average MINRES iterations remain about $D_r$25 per Newton step (Pearson et al., 2021). By contrast, the summary states that a direct factorization on the largest mesh would require several hundred GB of memory and tens of hours (Pearson et al., 2021). In this usage, DoPr is therefore a structure-exploiting Schur-complement preconditioner rather than a left-right scaling method.

5. Bi-parametric operator preconditioning

In "Bi-Parametric Operator Preconditioning" (Escapil-Inchauspé et al., 2020), the relevant DoPr framework arises in an abstract Petrov–Galerkin setting and formalizes two distinct sources of approximation: perturbations in the original sesquilinear form and perturbations in the preconditioner. Let $D_r$26 be reflexive Banach spaces, with a continuous sesquilinear form $D_r$27 and operator $D_r$28 defined by $D_r$29. The discrete problem on finite-dimensional spaces $D_r$30, $D_r$31 is assumed to satisfy the Banach–Nečas–Babuška condition with discrete inf-sup constant $D_r$32 (Escapil-Inchauspé et al., 2020).

The operator-preconditioning construction introduces a model operator $D_r$33 and pairings $D_r$34, $D_r$35, leading to the exact preconditioner

$D_r$36

Discretely,

$D_r$37

The “double” aspect here is not left-right matrix scaling but the simultaneous treatment of two perturbation parameters:

$D_r$38

An $D_r$39-perturbation $D_r$40 satisfies

$D_r$41

and the analogous condition for $D_r$42 uses $D_r$43 (Escapil-Inchauspé et al., 2020).

The perturbed bi-parametric system is

$D_r$44

with

$D_r$45

In a Krylov method such as GMRES, applying the preconditioner requires three steps:

$D_r$46

so that $D_r$47 (Escapil-Inchauspé et al., 2020).

The principal quantitative statement is a bi-parametric condition-number bound. With

$D_r$48

the perturbed condition numbers satisfy

$D_r$49

where

$D_r$50

In the Hilbert setting, under field-of-values assumptions, GMRES residuals satisfy

$D_r$51

and, when $D_r$52 remains bounded and the forms remain uniformly well posed, this convergence is $D_r$53-independent (Escapil-Inchauspé et al., 2020).

The same paper also develops a super-linear regime for second-kind Fredholm operators. If $D_r$54 with $D_r$55 compact or in a Carleman class $D_r$56, then the no-restart GMRES residual satisfies

$D_r$57

and, for $D_r$58, the rate is $D_r$59 (Escapil-Inchauspé et al., 2020).

The practical discussion emphasizes a cost-accuracy trade-off: one may choose $D_r$60 to match Galerkin discretization error while allowing $D_r$61, including aggressive compression of the preconditioner, without losing $D_r$62-independent convergence (Escapil-Inchauspé et al., 2020). The supplied summary states that this can reduce memory and setup cost by factors of $D_r$63–$D_r$64 in practice while keeping $D_r$65 typically below $D_r$66–$D_r$67 and GMRES iteration counts in $D_r$68–$D_r$69 independent of $D_r$70 (Escapil-Inchauspé et al., 2020). This suggests a domain-general interpretation of DoPr as a framework for separately budgeting approximation error in the operator and in the preconditioner.

6. Double preconditioning in test-time-feedback optimization

The 2026 paper "Double Preconditioning (DoPr): Optimization for Test-Time Performance, not Validation Loss" (Zhang et al., 4 Jun 2026) introduces a distinct use of DoPr in machine learning. The motivating setting is test-time feedback (TTF): training minimizes a one-step supervised loss on expert-distributed samples, but deployment rolls the model out on its own predictions, inducing a shifted state or token distribution. The paper states that this mismatch grows with task length and can degrade downstream metrics such as cumulative reward, pass@k, success rate, or FID even when validation loss is low (Zhang et al., 4 Jun 2026).

In this formulation, DoPr combines two classes of optimizer geometry. Gradient-wise preconditioners such as Adam, Muon, Shampoo, and related methods adapt update magnitudes using gradient statistics but are described as blind to activation distributions. Activation preconditioners such as one-sided KFAC, LoCoProp, or right-side-only Fisher rescale layer gradients by the inverse activation covariance

$D_r$71

promoting uniform feature updates across activation directions (Zhang et al., 4 Jun 2026). The paper’s core claim is that adding activation preconditioning improves downstream performance in TTF settings even when validation loss does not improve correspondingly.

For a layer weight $D_r$72, the method computes the raw minibatch gradient $D_r$73 and the empirical activation covariance

$D_r$74

With damping parameter $D_r$75, the damped covariance is

$D_r$76

Stage 1 applies activation preconditioning:

$D_r$77

Stage 2 applies a base gradient-wise preconditioner:

$D_r$78

for example AdamW with the usual $D_r$79 recursions and bias correction, followed by

$D_r$80

The algorithm is presented as a per-layer wrapper and described as a “plug-in” two-stage update (Zhang et al., 4 Jun 2026).

The paper attributes DoPr’s effect to geometry under feedback dynamics. Standard GD or GP is said to bias feature updates toward directions of large activation variance, potentially neglecting low-variance but TTF-sensitive directions. Activation preconditioning “whitens” activations by making the update equivalent to steepest descent under the inner product weighted by $D_r$81, thereby enforcing more uniform subspace contraction (Zhang et al., 4 Jun 2026). Pure AP is described as potentially numerically unstable; the addition of GP is intended to restore stability, momentum, and adaptive step-size control.

The reported computational overhead per layer is

$D_r$82

with end-to-end overhead approximately $D_r$83–$D_r$84 standard backprop in large-batch regimes, and memory sufficient to store $D_r$85 and the base optimizer state (Zhang et al., 4 Jun 2026). Approximation strategies listed in the summary include spatially uncorrelated activation approximations for Conv2d, rank-1-plus-diagonal inverses via Sherman–Morrison, and diagonal treatment of one-hot embedding activations (Zhang et al., 4 Jun 2026).

Empirical evaluation spans four TTF domains. In continuous-control imitation learning on Humanoid-v5 and Half-Cheetah-v5, DoPr variants are reported to increase terminal return, with examples of $D_r$86–$D_r$87 on Humanoid despite similar or worse validation loss. In Robomimic pixel-based imitation tasks such as Tool-Hang and Transport, DoPr-AdamW and DoPr-Muon improve success rate by $D_r$88–$D_r$89 percentage points. In language-model SFT on GSM8K and OpenMathInstruct-2.1M, DoPr shifts peak accuracy upward by $D_r$90–$D_r$91 percentage points in one setting and by $D_r$92–$D_r$93 percentage points in a high-learning-rate regime, while the baseline AdamW exhibits learning rates whose lower NLL degrades downstream accuracy. In generative flows on SiT-S for ImageNet-256, DoPr-AdamW and DoPr-Muon produce faster FID convergence, including examples of $D_r$94–$D_r$95 FID reduction at the midpoint (Zhang et al., 4 Jun 2026). The central interpretive point is explicit in the paper’s title: optimization is being tuned for test-time performance, not validation loss.

7. Comparison, misconceptions, and open directions

A comparative reading of these papers shows that DoPr is a domain-dependent design pattern rather than a transferable algorithmic object. The numerical linear algebra version minimizes the two-norm condition number by left/right diagonal scalings (Qu et al., 2022). The interval-analysis version sharpens strong-regularity checks by factorizing the midpoint inverse and transforming both sides (Skalna et al., 2020). The saddle-point version exploits nested Schur complements to build a mesh-robust SPD block preconditioner (Pearson et al., 2021). The operator-preconditioning version controls two perturbation channels, $D_r$96 and $D_r$97, within a Petrov–Galerkin framework (Escapil-Inchauspé et al., 2020). The deep-learning version composes activation-wise and gradient-wise preconditioners to improve downstream rollout behavior under train-test mismatch (Zhang et al., 4 Jun 2026).

Several misconceptions are therefore best addressed directly. First, DoPr is not synonymous with diagonal equilibration, even though one influential paper uses that interpretation (Qu et al., 2022). Second, DoPr is not always a matrix preconditioner for a linear system; it may instead be a solver framework, a perturbation calculus, or an optimizer update rule (Escapil-Inchauspé et al., 2020, Zhang et al., 4 Jun 2026). Third, the “double” aspect does not always mean literal left and right multiplication. In different literatures it denotes two-sided matrix actions, two nested Schur complements, or two perturbation parameters (Skalna et al., 2020, Pearson et al., 2021, Escapil-Inchauspé et al., 2020).

At the same time, there is a coherent cross-domain theme. In every case, a one-sided or single-statistic procedure is judged insufficient for the quantity of interest: heuristic diagonal scaling may not minimize $D_r$98, midpoint-inverse left preconditioning may not pass the $D_r$99 test, a single Schur complement may not fully exploit $\kappa(D_\ell A D_r)$00 saddle structure, a single perturbation budget may not describe inexact operator preconditioning, and gradient-wise adaptation alone may not address test-time feedback (Qu et al., 2022, Skalna et al., 2020, Pearson et al., 2021, Escapil-Inchauspé et al., 2020, Zhang et al., 4 Jun 2026). This suggests that the enduring value of the term lies in a methodological principle: pair two complementary preconditioning mechanisms so that the target performance criterion—condition number, strong regularity, Krylov spectrum, perturbation robustness, or rollout quality—is controlled more directly than under a one-sided design.

The open directions also differ by field. The interval paper notes that optimizing $\kappa(D_\ell A D_r)$01 and $\kappa(D_\ell A D_r)$02 to minimize $\kappa(D_\ell A D_r)$03 directly would be a nonconvex problem on the Lie group of invertible matrices, and mentions sparsity-preserving splittings and block-wise DoPr as open directions (Skalna et al., 2020). The operator-preconditioning work highlights aggressive approximation of the preconditioner without sacrificing robustness (Escapil-Inchauspé et al., 2020). The deep-learning paper lists extension to online or active data collection, adaptive damping and low-rank pseudoinverses per layer, and theoretical analysis of generalization and TTF-shift reduction in nonlinear settings (Zhang et al., 4 Jun 2026). A plausible implication is that the future of DoPr research will remain plural: advances are likely to emerge through domain-specific exploitation of paired conditioning structures rather than through convergence toward a single canonical formalism.