Papers
Topics
Authors
Recent
Search
2000 character limit reached

LRQ-Solver: Low-Rank Query PDE Solver

Updated 4 July 2026
  • LRQ-Solver is a framework that replaces globally expensive dense operations with efficient low‐rank surrogates through precomputation and hierarchical updates.
  • It encompasses diverse formulations—boundary-integral updates, HODLR/BDLR multifrontal solvers, and transformer-based neural operators—to handle local geometry perturbations and large point clouds.
  • The approach achieves significant speedups and memory savings by leveraging techniques like the Woodbury formula and covariance-based low-rank attention while preserving accuracy.

Low-Rank Query-based PDE Solver (LRQ-Solver) denotes, in the supplied literature, a class of low-rank, query-oriented PDE solution frameworks rather than a single invariant algorithm. In one formulation, it is a two-stage direct-solver workflow for boundary value problems on locally perturbed geometries: one precomputes a direct factorization of an original boundary-integral system and answers each new query by a low-rank update. In another, it is an HODLR- and BDLR-based solver design for 3D elliptic finite-element matrices within a multifrontal elimination process. In a third, it is the name of a transformer-based neural operator for large-scale 3D PDEs that combines Parameter Conditioned Lagrangian Modeling (PCLM) with Low-Rank Query Attention (LR-QA) (Zhang et al., 2017, Aminfar et al., 2014, Zeng et al., 13 Oct 2025).

1. Scope and unifying idea

Across the supplied sources, the label is attached to three distinct constructions. Their common motif is the replacement of globally expensive dense operations by compressed, low-rank surrogates targeted at repeated solves, multifrontal subproblems, or large point-cloud inference. This suggests a shared organizing principle: expensive global interactions are amortized by a one-time factorization, hierarchical compression, or covariance-based attention.

Formulation Query object Low-rank mechanism
Locally perturbed boundary solver New, locally perturbed geometry Low rank update to the original system plus Sherman–Morrison–Woodbury
3D elliptic multifrontal solver Dense frontal matrices in multifrontal elimination HODLR representation plus BDLR pseudo-skeleton compression
Transformer neural operator Global design vector and large point cloud PCLM plus LR-QA with covariance decomposition

A common misconception is that “low-rank” has a single technical meaning here. In the supplied formulations it refers, respectively, to numerically low-rank cut-and-paste updates in boundary integral equations, off-diagonal low-rank structure in hierarchical dense fronts, and low-rank structure in attention induced by second-order statistics of physical fields (Zhang et al., 2017, Aminfar et al., 2014, Zeng et al., 13 Oct 2025).

2. Boundary-integral LRQ-Solver for locally perturbed geometries

In the boundary-integral formulation, the continuous problem is the Laplace Dirichlet boundary value problem on a simply-connected domain Ω⊂R2\Omega \subset \mathbb{R}^2 with smooth boundary Γ\Gamma:

−Δu(x)=0,x∈Ω,u(x)=g(x),x∈Γ.-\Delta u(x)=0,\quad x\in\Omega,\qquad u(x)=g(x),\quad x\in\Gamma.

The solution is represented by a double-layer potential

u(x)=∫ΓD(x,y) σ(y) ds(y),D(x,y)=∂νyG(x,y),G=−(1/2π)log⁡∣x−y∣,u(x)=\int_\Gamma D(x,y)\,\sigma(y)\,ds(y),\qquad D(x,y)=\partial_{\nu_y}G(x,y),\qquad G=-(1/2\pi)\log|x-y|,

and the boundary limit yields the second-kind integral equation

−12σ(x)+∫ΓD(x,y)σ(y) ds(y)=g(x),x∈Γ,-\frac12 \sigma(x)+\int_\Gamma D(x,y)\sigma(y)\,ds(y)=g(x),\qquad x\in\Gamma,

abbreviated as K[σ]=fK[\sigma]=f on Γ\Gamma (Zhang et al., 2017).

The geometric query is a local perturbation of an original boundary Γ0\Gamma_0. One removes a small patch Γc⊂Γ0\Gamma_c \subset \Gamma_0 with NcN_c points, glues in a new patch Γp\Gamma_p with NpN_p points, and denotes the remaining part by Γk=Γ0∖Γc\Gamma_k=\Gamma_0\setminus \Gamma_c. The perturbed boundary is

Γ=Γ0∖Γc∪Γp=Γk∪Γp.\Gamma=\Gamma_0\setminus \Gamma_c \cup \Gamma_p=\Gamma_k\cup\Gamma_p.

After Nyström or Galerkin discretization on Γ0\Gamma_0, one obtains

A0 σ0=f0,A0∈Rn×n,n=N0.A_0\,\sigma_0=f_0,\qquad A_0\in\mathbb R^{n\times n},\quad n=N_0.

A fast direct solver is then built for A0A_0, using a data-sparse factorization such as HBS, HODLR, or an H\mathcal H-matrix approximation A0≈A0\mathcal A_0\approx A_0 satisfying ∥A0−A0∥≤ϵ\|A_0-\mathcal A_0\|\le \epsilon, together with an approximate inverse A0−1\mathcal A_0^{-1} (Zhang et al., 2017).

The new perturbed system is rewritten as an extended block diagonal matrix plus a small update QQ, acting on the unknown densities (σk;σc;σp)(\sigma_k;\sigma_c;\sigma_p). The update collects the couplings caused by removing Γc\Gamma_c and adding Γp\Gamma_p. By proxy-surface arguments or HBS leaf-box extraction, the subblocks

Akc∈RNk×Nc,Apk∈RNp×Nk,Aop∈RNo×Np,A_{kc}\in\mathbb R^{N_k\times N_c},\qquad A_{pk}\in\mathbb R^{N_p\times N_k},\qquad A_{op}\in\mathbb R^{N_o\times N_p},

as well as the diagonal-with-zero-diagonal block Bcc∈RNc×NcB_{cc}\in\mathbb R^{N_c\times N_c}, are numerically low rank. Hence

Q≈LR,L∈RNext×r,R∈Rr×Next,r≪Next,Q\approx LR,\qquad L\in\mathbb R^{N_{\rm ext}\times r},\quad R\in\mathbb R^{r\times N_{\rm ext}},\qquad r\ll N_{\rm ext},

and, after discarding zero rows and columns, the perturbed system is represented in the compact form

A=A0+UVT,U,V∈Rn×r.A=A_0+UV^T,\qquad U,V\in\mathbb R^{n\times r}.

The significance of this formulation is operational rather than merely algebraic: the local perturbation is treated as a query against a precomputed inverse, so the original factorization is reused instead of rebuilt.

3. Precomputation, Woodbury acceleration, and asymptotics

For the low-rank updated system A=A0+UVTA=A_0+UV^T, the core identity is the Sherman–Morrison–Woodbury formula

(A0+UVT)−1=A0−1−A0−1U(Ir+VTA0−1U)−1VTA0−1.(A_0+UV^T)^{-1} = A_0^{-1} - A_0^{-1}U(I_r+V^T A_0^{-1}U)^{-1}V^T A_0^{-1}.

The efficient application sequence is: compute y1=A0−1by_1=A_0^{-1}b; form M=VTA0−1UM=V^T A_0^{-1}U; factor Ir+MI_r+M once by LU or Cholesky; compute y2=A0−1Uy_2=A_0^{-1}U; solve (I+M)w=VTy1(I+M)w=V^T y_1; and return x=y1−y2wx=y_1-y_2w (Zhang et al., 2017).

The precomputation stage is performed once for Γ0\Gamma_0: discretize K[σ]=fK[\sigma]=f on Γ0\Gamma_0, build the fast direct solver, and form A0−1\mathcal A_0^{-1}. The online stage is executed for each placement of the patch Γp\Gamma_p: discretize the new patch and identify removal indices; assemble U,VU,V so that A=A0+UVTA=A_0+UV^T; compute Y=A0−1UY=A_0^{-1}U at cost O(nr)O(nr); form S=Ir+VTYS=I_r+V^TY at cost O(nr+r2)O(nr+r^2); factor or invert SS in O(r3)O(r^3); and, for a given right-hand side, compute the solution by the Woodbury sequence in O(nr+r2)O(nr+r^2). The total per-query cost is therefore O(nr+r3)O(nr+r^3) (Zhang et al., 2017).

For the original factorization, the stated construction and memory costs are O(nlog⁡n)O(n\log n) or O(n)O(n) for HBS/HODLR, and applying A0−1\mathcal A_0^{-1} to a vector also costs O(nlog⁡n)O(n\log n) or O(n)O(n). The update becomes asymptotically faster when r≪nr\ll n. For fixed-size local patches, where rr is roughly constant, the reported precompute-stage speedup is approximately 3×3\times relative to building a new fast direct solver from scratch, while the solve stage is comparable or slightly slower (Zhang et al., 2017).

The limitations are explicit. If the patch grows proportionally with nn, then r≈O(n)r\approx O(n) and the speedup disappears. In practice the method is most advantageous as long as the patch contains fewer than O(n)O(n) points, often less than 50%50\% of nn. Supported perturbations include translated patches of fixed shape, which are described as ideal, and locally refined meshes, which remain efficient as long as rr stays a small fraction of nn. The same methodology is stated to extend to other boundary-integral operators, including Helmholtz and elasticity, and to 3D provided a fast inverse A0−1\mathcal A_0^{-1} can be built (Zhang et al., 2017).

4. HODLR/BDLR realization for 3D elliptic multifrontal systems

A second LRQ-Solver formulation addresses 3D elliptic PDEs through the dense frontal matrices generated by nested-dissection and multifrontal sparse elimination. At each node of the elimination tree, a dense frontal matrix A∈Rn×nA\in\mathbb R^{n\times n} is partitioned at the top level into

A=[A11A12A21A22],A= \begin{bmatrix} A_{11} & A_{12}\\ A_{21} & A_{22} \end{bmatrix},

with A11A_{11} and A22A_{22} treated recursively and the off-diagonal blocks approximated by rank-rr factors,

A12≈U1V2T,A21≈U2V1T,r≪n.A_{12}\approx U_1V_2^T,\qquad A_{21}\approx U_2V_1^T,\qquad r\ll n.

Recursive subdivision yields a pp-level HODLR tree (Aminfar et al., 2014).

The low-rank compression is constructed by the boundary-distance low-rank approximation (BDLR), a pseudo-skeleton scheme based on graph distance in the sparse matrix graph. For an off-diagonal block B=A(Ir,Jc)B=A(I_r,J_c), one forms a separator graph on the DOFs in Ir∪JcI_r\cup J_c, identifies boundary sets Sr0S_r^0 and Sc0S_c^0, computes graph distances drd_r and dcd_c, and selects rows and columns closest to the separator. With the selected index sets II and JJ, one forms

C=B(:,J),R=B(I,:),B^=B(I,J),C=B(:,J),\qquad R=B(I,:),\qquad \widehat B=B(I,J),

then computes an LU factorization with full pivoting of B^\widehat B, truncates to numerical rank rr, and returns factors C~\widetilde C and R~\widetilde R so that B≈C~ R~B\approx \widetilde C\,\widetilde R (Aminfar et al., 2014).

The direct solve is itself Woodbury-style. For

K=[K1U1V2TU2V1TK2],K= \begin{bmatrix} K_1 & U_1V_2^T\\ U_2V_1^T & K_2 \end{bmatrix},

one introduces auxiliary variables y1=V2Tx1y_1=V_2^Tx_1 and y2=V1Tx2y_2=V_1^Tx_2, factors the diagonal blocks, and solves a small Schur complement

S(1)=[IV2TK2−1U2V1TK1−1U1I].S^{(1)}= \begin{bmatrix} I & V_2^T K_2^{-1}U_2\\ V_1^T K_1^{-1}U_1 & I \end{bmatrix}.

The same procedure recurses to leaf size n0n_0. The stated complexity is O(nlog⁡2n)O(n\log^2 n) if the off-diagonal ranks remain approximately constant, or O(n r02)O(n\,r_0^2) in practice; when rank growth follows the 3D pattern rℓ∼nℓ1/2r_\ell\sim n_\ell^{1/2}, the total can rise to O(n2)O(n^2), though with a small prefactor (Aminfar et al., 2014).

This formulation can also be used as a GMRES preconditioner. The stated strategy is to use a low-tolerance HODLR factorization, with εpre∈[10−1,10−3]\varepsilon_{\rm pre}\in[10^{-1},10^{-3}], so that GMRES converges in $10$–$50$ iterations to machine precision (10−10)(10^{-10}). Practical settings include leaf size n0≈64n_0\approx 64–$128$, direct-solve tolerance 10−610^{-6}, preconditioner tolerance 10−110^{-1}–10−310^{-3}, BDLR depth $1$–$3$ for ϵ≥10−1\epsilon\ge 10^{-1} and up to $5$–$10$ for ϵ≤10−5\epsilon\le 10^{-5} on unstructured meshes, and a rank cap that switches to dense storage when rℓ>0.2 nℓr_\ell>0.2\,n_\ell (Aminfar et al., 2014).

5. Transformer-based neural-operator LRQ-Solver

The 2025 LRQ-Solver is an end-to-end differentiable neural operator built on a transformer-style backbone for large-scale 3D PDEs. Its inputs are a global design parameter vector dd and a large point cloud {xi∈R3}i=1N\{x_i\in\mathbb R^3\}_{i=1}^N representing the geometry. Internally it combines two submodules: Parameter Conditioned Lagrangian Modeling (PCLM), which encodes dd into a latent control field ψ\psi, and Low-Rank Query Attention (LR-QA), which replaces standard O(N2)O(N^2) self-attention by a covariance-based approximation with complexity O(NC2+C3)O(NC^2+C^3), where C≪NC\ll N (Zeng et al., 13 Oct 2025).

The architecture proceeds in five stages. First, the Parameter-Conditioned Encoder (PCE) maps dd to ψ∈RC\psi\in\mathbb R^C by cross-attention. In the stated implementation, the PCE uses $4$-head cross-attention, Nq=10N_q=10 learnable queries, and Dh=128D_h=128. Second, ψ\psi is concatenated with each spatial coordinate xix_i to form an extended input ui(0)=[xi;ψ]u_i^{(0)}=[x_i;\psi]. Third, a stack of L=6L=6 transformer-style layers processes these features; each layer uses LR-QA with h=8h=8 heads and C=64C=64 per head together with a residual MLP of width $256$. Fourth, a $2$-layer MLP decodes the final features into pseudo-physics fields such as u^i=(v^i,p^i,T^i,σ^i)\hat u_i=(\hat v_i,\hat p_i,\hat T_i,\hat \sigma_i). Fifth, system-level quantities such as drag coefficient, total force, and heat flux are obtained by differentiable control-volume integrals (Zeng et al., 13 Oct 2025).

PCLM is defined by treating the local state as

u(x;ψ)=(v(x;ψ),p(x;ψ),T(x;ψ)),u(x;\psi)=(v(x;\psi),p(x;\psi),T(x;\psi)),

with ψ\psi a global latent control vector summarizing dd. The conservation laws are then enforced over the extended input space (x,ψ)(x,\psi) through a physics residual

Lphys=1N∑i=1N[∥∇⋅v^i∥2+∥ρ(v^i⋅∇)v^i+∇p^i−∇⋅τ^i∥2+∥ρcp(v^i⋅∇T^i)−∇⋅(k∇T^i)∥2].L_{\rm phys}=\frac1N\sum_{i=1}^N\left[\|\nabla\cdot \hat v_i\|^2+\|\rho(\hat v_i\cdot\nabla)\hat v_i+\nabla \hat p_i-\nabla\cdot \hat \tau_i\|^2+\|\rho c_p(\hat v_i\cdot\nabla \hat T_i)-\nabla\cdot(k\nabla \hat T_i)\|^2\right].

For aerodynamic tasks, a pair-wise ranking loss

Lrank=∑i<jmax⁡(0,m−(y^i−y^j)sij),sij=sign⁡(yitrue−yjtrue),L_{\rm rank}=\sum_{i<j}\max(0,m-(\hat y_i-\hat y_j)s_{ij}),\qquad s_{ij}=\operatorname{sign}(y_i^{\rm true}-y_j^{\rm true}),

is added to preserve monotonicity in drag predictions (Zeng et al., 13 Oct 2025).

LR-QA is motivated by the observation that, in many PDE fields, the key and value matrices admit a low-rank structure. At layer ℓ\ell, one forms

Q(ℓ)=LQ(X(ℓ−1)),K(ℓ)=LK(X(ℓ−1)),V(ℓ)=LV(X(ℓ−1)),Q^{(\ell)}=L_Q(X^{(\ell-1)}),\qquad K^{(\ell)}=L_K(X^{(\ell-1)}),\qquad V^{(\ell)}=L_V(X^{(\ell-1)}),

each in RN×C\mathbb R^{N\times C}. Instead of forming the N×NN\times N similarity matrix, LR-QA computes the C×CC\times C covariances

Ck=(K(ℓ))TK(ℓ),Cv=(V(ℓ))TV(ℓ),C_k=(K^{(\ell)})^T K^{(\ell)},\qquad C_v=(V^{(\ell)})^T V^{(\ell)},

and thereby reduces attention complexity from O(N2)O(N^2) to O(NC2+C3)O(NC^2+C^3). The stated approximation guarantee is

∥Z−Zstd∥F≤∥Q∥F∥V∥F∥K−KKTK∥F,\|Z-Z_{\rm std}\|_F \le \|Q\|_F\|V\|_F\|K-KK^TK\|_F,

with the interpretation that decaying singular values imply discarding only low-energy modes (Zeng et al., 13 Oct 2025).

The training setup uses AdamW with learning rate 10−410^{-4} and weight decay 10−310^{-3}, a learning-rate decay by ×0.1\times 0.1 at epoch $50$, batch size $4$ per A100 GPU (64 GB), and DDP over $4$ GPUs. The total loss is

Ltotal=α1⋅MSE⁡(y^,y)+α2⋅Lphys+α3⋅Lrank+λ∥θ∥2,L_{\rm total}=\alpha_1\cdot \operatorname{MSE}(\hat y,y)+\alpha_2\cdot L_{\rm phys}+\alpha_3\cdot L_{\rm rank}+\lambda\|\theta\|^2,

with α1=1.0\alpha_1=1.0, α2=0.1\alpha_2=0.1, α3=0.1\alpha_3=0.1, and λ=10−4\lambda=10^{-4} (Zeng et al., 13 Oct 2025).

6. Quantitative behavior, invariance, and stated limitations

The three formulations report different performance regimes. For the locally perturbed boundary solver, fixed-size local patches yield an approximately 3×3\times speedup in the precompute stage, while the solve stage is comparable or slightly slower; the method is most advantageous when the perturbation remains a small fraction of the full geometry (Zhang et al., 2017). For the HODLR/BDLR multifrontal solver, the reported benchmarks include a 3D unstructured beam front of size approximately $2$ K, where HODLR-preconditioned GMRES with ϵpre=10−1\epsilon_{\rm pre}=10^{-1} takes approximately $0.4$ s versus LU at $1.6$ s, reducing iterations from $50$ to $7$; a $1.3$ K cylinder-head front with approximately 10×10\times speedup at ϵpre=10−3\epsilon_{\rm pre}=10^{-3}; and a FETI local cube problem with a $23$ K front and approximately $5$–7×7\times speedup (Aminfar et al., 2014).

For the neural-operator LRQ-Solver, the DrivAer++ benchmark uses 8 0008\,000 car models, $23$ deformable parameters, and $100$ k surface points per sample, with a 70/15/15%70/15/15\% train/validation/test split and metrics MSE, MAE, MaxAE, and MRE. The reported result is MSE 5.56×10−55.56\times 10^{-5}, MAE 5.90×10−35.90\times 10^{-3}, MaxAE 3.22×10−23.22\times 10^{-2}, MRE 2.25%2.25\%, training time $7.2$ h, and inference time $0.005$ s, corresponding to a 38.9%38.9\% relative MSE reduction and a 126×126\times inference speedup over Transolver++ at $0.63$ s. On the 3D Beam benchmark, using 3 0003\,000 beam configurations with point clouds ranging from $250$ to $25$ k points and a 75/5/20%75/5/20\% split, the reported result is MAE_sub $1.66$ MPa, MAE_all $1.66$ MPa, training time $0.76$ h, and inference time $3.5$ ms, corresponding to a 28.8%28.8\% MAE reduction over Geom-DeepONet and 61×61\times faster training. The same source states that RegDGCNN runs OOM on the full geometry, whereas LRQ-Solver scales to $2$ million points on one A100 (Zeng et al., 13 Oct 2025).

The ablations identify complementary contributions from LR-QA and PCLM. On DrivAer++, the baseline MLP has MSE 36.81×10−536.81\times 10^{-5} and MRE 5.63%5.63\%; adding LR-QA reduces this to $8.98$ and 2.82%2.82\%; adding PCLM gives $5.69$ and 2.22%2.22\%; and the full model gives $5.56$ and 2.25%2.25\%. On 3D Beam, the baseline MAE_all is $3.07$ MPa, with $1.99$ for baseline plus LR-QA, $2.38$ for baseline plus PCLM, and $1.66$ for the full model (Zeng et al., 13 Oct 2025).

Discretization invariance is reported by varying the number of points from $250$ to $100$ k, while maintaining stable error, exemplified by MAE ≈1.66\approx 1.66 MPa on 3D Beam, and approximately constant inference latency of about $3.5$ ms (Zeng et al., 13 Oct 2025). The stated limitations are likewise formulation-specific. In the classical update solver, large patches drive rr toward O(n)O(n) and remove the speedup (Zhang et al., 2017). In the HODLR setting, 3D rank growth can increase total cost to O(n2)O(n^2) (Aminfar et al., 2014). In the neural-operator setting, the covariance approximation relies on a rapidly decaying singular spectrum; highly turbulent or multi-scale flows with full-rank interactions may require larger CC or hierarchical rank-adaptation, the current model addresses steady or quasi-steady PDEs, and topology changes or additional multiphysics may require domain-decomposition or hybrid graph/transformer architectures (Zeng et al., 13 Oct 2025).

Taken together, these formulations position LRQ-Solver as a name for PDE solvers that use low-rank structure to answer queries efficiently, whether the query is a local geometric perturbation, a multifrontal dense front, or a parameterized point-cloud configuration.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Low-Rank Query-based PDE Solver (LRQ-Solver).