Dual-Stream Least Squares

• Dual-Stream Least Squares is a design principle that couples two complementary channels within least-squares frameworks to achieve enhanced alignment, stability, personalization, or quasi-optimal approximation.
• Key methodologies include spatially adaptive rigid moving least squares for image stitching, primal–dual and streamline-aware formulations for PDEs, and operator–adjoint coupling for hyperbolic transport problems.
• The approach extends to personalized federated learning and machine learning, where dual streams are used to separately capture global features and local refinements, ensuring efficient and robust performance.

Dual-Stream Least Squares denotes a family of least-squares constructions in which two coupled information channels are fitted jointly. Across the cited literature, those channels are defined in technically different ways: two image streams from a dual-fisheye camera; a primal variable and a dual variable together with explicit streamline control in a first-order system least-squares formulation; a hyperbolic operator $L$ and its adjoint $L^*$ in $\mathcal{L}\mathcal{L}^*$-based approximation; a shared global stream and a client-specific refinement stream in personalized federated learning; or a primal trial variable paired with dual test variables that realize dual norms in a minimax formulation (Ho et al., 2017, Cai et al., 2023, Kalchev et al., 2020, Fan et al., 14 Aug 2025, Monsuur et al., 2024). This suggests that the term is best understood as a structural pattern rather than a single canonical algorithm: least-squares coupling of two complementary streams to recover alignment, stability, personalization, or quasi-optimal approximation.

1. Semantic scope and recurring structure

The cited works use “Dual-Stream Least Squares” in several precise senses. In dual-fisheye video stitching, the two streams are the left and right unwarped equirectangular images, with a position-dependent least-squares warp aligning one stream to the other near narrow overlap bands (Ho et al., 2017). In convection-dominated diffusion-reaction, the method is “Dual-Stream” both because it is primal–dual in the variables $(u,\sigma)$ and because the analysis norm explicitly controls the streamline derivative $D_s u := b\cdot\nabla u$ through mesh-dependent weights tied to local Péclet numbers (Cai et al., 2023). In scalar linear hyperbolic problems, the two streams are the primal residual $L v^h-f$ and the adjoint stream built from $L^*$ and $(L_wL^*)^{-1}$, which reconstructs $L^2$ control (Kalchev et al., 2020). In personalized federated learning, the streams are a shared primary stream for collective generalization and a dedicated refinement stream for local personalization (Fan et al., 14 Aug 2025). In quasi-optimal least squares for PDEs with inhomogeneous boundary data, the streams are a primal approximation and dual test variables realizing dual norms for interior and boundary residuals (Monsuur et al., 2024).

Setting	Two streams	Core least-squares role
Dual-fisheye stitching	Left/right image streams	Spatially adaptive rigid MLS alignment
Convection–diffusion–reaction	Primal–dual $(u,\sigma)$ and streamline derivative control	$L^*$0-robust coercivity and adaptive refinement
Hyperbolic transport	$L^*$1 and $L^*$2	$L^*$3-targeted approximation via $L^*$4
Personalized federated learning	Shared primary and local refinement heads	Analytic generalization plus personalization
QOLS for PDEs	Primal trial and dual supremizer variables	Dual-norm residual minimization without fractional boundary norms

A common source of confusion is to treat the phrase as naming one standardized method. The literature instead assigns the phrase to a family of least-squares architectures in which “dual” may mean two data streams, primal–dual variables, primal–adjoint coupling, global–local decomposition, or primal–dual minimax structure. The unifying feature is not a shared application domain, but a repeated design principle: two complementary channels are coupled so that least-squares minimization acquires a property that a single-stream construction would not provide as directly.

2. Dual-fisheye image streams and rigid moving least squares

In 360-degree video stitching for dual-fisheye cameras, the input consists of two image streams from a dual-fisheye lens camera with limited overlap. Each lens has roughly $L^*$5 FOV. After radiometric compensation and fisheye-to-equirectangular unwarping, the right lens image is placed at the center of the $L^*$6 equirectangular canvas, and the left lens image is split to the two sides. The overlapping regions are narrow vertical bands around the stitching boundaries. The central difficulty is that conventional global parametric warps estimated in a least-squares sense tend to align the center well but leave visible discontinuities near the top and bottom of the panorama when overlap is limited and lens misalignment/parallax exist; a “dual-stream least squares” approach that is spatially adaptive is therefore required (Ho et al., 2017).

The paper instantiates this by moving least squares with a rigid constraint. Moving least squares is an interpolation-based deformation that, at each query location $L^*$7, solves a local weighted least-squares alignment problem using control point correspondences. It ensures interpolation, smoothness, and identity properties, and its “moving” nature stems from weights that decay with distance from $L^*$8. The restriction to rigid transformations, namely rotation plus translation with no scale or shear, is used to avoid local scale changes that can warp geometry, which is especially important when overlap is small and preserves natural appearance away from seams. With control points $L^*$9 sampled on the right unwarped image near the stitching boundary and correspondences $\mathcal{L}\mathcal{L}^*$0 on the left unwarped image, the weights are

$\mathcal{L}\mathcal{L}^*$1

where $\mathcal{L}\mathcal{L}^*$2 controls locality. Larger $\mathcal{L}\mathcal{L}^*$3 increases locality and reduces the influence of distant points.

At each query location $\mathcal{L}\mathcal{L}^*$4, the rigid MLS objective is

$\mathcal{L}\mathcal{L}^*$5

Using weighted centroids

$\mathcal{L}\mathcal{L}^*$6

and centered offsets $\mathcal{L}\mathcal{L}^*$7, $\mathcal{L}\mathcal{L}^*$8, the covariance is

$\mathcal{L}\mathcal{L}^*$9

From the SVD $(u,\sigma)$0, the optimal rotation is

$(u,\sigma)$1

which enforces $(u,\sigma)$2, and the translation is

$(u,\sigma)$3

The warp is then $(u,\sigma)$4, or equivalently $(u,\sigma)$5 for a pixel $(u,\sigma)$6.

Because rigid MLS is evaluated per point, the method precomputes an interpolation grid $(u,\sigma)$7 over the overlap and a margin into each stream. For each grid node, it computes the weights, centroids, covariance, a $(u,\sigma)$8 SVD, and then extracts $(u,\sigma)$9 and $D_s u := b\cdot\nabla u$0. The dense warp is obtained by bilinear interpolation on the equirectangular lattice. The right stream is aligned to the left across the overlap, while the rest remains largely unchanged because the weights decay with distance.

The full alignment pipeline has two stages. Stage 1 is camera-dependent and offline: light fall-off correction, fisheye-to-equirectangular unwarping, acquisition of seam control points via checkerboards and/or feature matches, and construction of rigid-MLS interpolation grids. Stage 2 is scene-adaptive and online: residual misalignment due to depth variation/parallax is corrected with a lightweight template-matching step at both seams using normalized cross-correlation,

$D_s u := b\cdot\nabla u$1

followed by a small affine refinement from eight control point pairs by minimizing

$D_s u := b\cdot\nabla u$2

A separate contribution addresses temporal coherence in video. The problem is that per-frame refined alignment can occasionally produce erroneous affine warps when NCC matches are poor, abruptly shifting the seam and causing visible jitter. The proposed gating algorithm accepts refined affine updates only when both boundaries pass empirical reliability thresholds: NCC peak $D_s u := b\cdot\nabla u$3, vertical displacement $D_s u := b\cdot\nabla u$4 pixels, and horizontal displacement within $D_s u := b\cdot\nabla u$5 of the previous frame’s displacement. If current matches fail but the previous frame had good matches, the previous affine warp is reused; otherwise the affine warp is disabled for that frame. The paper states that results show higher quality stitched images and videos than prior work, and the qualitative examples show patterned backgrounds aligned cleanly with rigid MLS while the previous global least-squares approach displays discontinuities; for video, example sequences show “no jitter” in consecutive frames, whereas the baseline exhibits sudden shifts when a bad match propagates into the warp.

The stated limitations are equally specific. Severe parallax, large moving objects across seams, or extreme lens misalignment can exceed the capacity of rigid deformation and small affine tuning. Very limited overlap reduces control-point support; in such cases, feature detection must be strengthened, and $D_s u := b\cdot\nabla u$6 reduced to avoid over-localization.

3. Primal–dual and streamline-aware least squares for convection-dominated problems

For stationary convection–diffusion–reaction equations, “Dual-Stream Least Squares” is used in two precise senses: primal–dual and streamline-aware. The model problem is

$D_s u := b\cdot\nabla u$7

with homogeneous Dirichlet condition $D_s u := b\cdot\nabla u$8, where $D_s u := b\cdot\nabla u$9 for $L v^h-f$0, $L v^h-f$1, $L v^h-f$2, $L v^h-f$3, and $L v^h-f$4. The boundary is decomposed into

$L v^h-f$5

under the reaction–convection coercivity condition

$L v^h-f$6

The first-order system introduces the dual variable

$L v^h-f$7

and rewrites the PDE as

$L v^h-f$8

(Cai et al., 2023).

The least-squares functionals differ by their outflow treatment:

$L v^h-f$9

$L^*$0

$L^*$1

Here $L^*$2 imposes $L^*$3 strongly through $L^*$4, while $L^*$5 and $L^*$6 use $L^*$7, enforce inflow strongly, and impose the outflow boundary weakly through penalties. Because $L^*$8, $L^*$9 penalizes outflow values more strongly than $(L_wL^*)^{-1}$0.

The second sense of “stream” is encoded in the analysis norm through the streamline derivative

$(L_wL^*)^{-1}$1

The strengthened triple-bar norm is

$(L_wL^*)^{-1}$2

with $(L_wL^*)^{-1}$3 chosen by local Péclet number. A canonical choice uses

$(L_wL^*)^{-1}$4

the splitting $(L_wL^*)^{-1}$5 and $(L_wL^*)^{-1}$6, and

$(L_wL^*)^{-1}$7

under the layer-resolution assumption

$(L_wL^*)^{-1}$8

The paper establishes continuous coercivity

$(L_wL^*)^{-1}$9

with $L^2$0 and $L^2$1 independent of $L^2$2, while $L^2$3. At the discrete level, fractional norms are replaced by edge-weighted $L^2$4 norms,

$L^2$5

$L^2$6

and the discrete constants become independent of $L^2$7 for both $L^2$8. The paper also proves

$L^2$9

which establishes ellipticity in the streamlined norm.

The a priori estimates are stated for Raviart–Thomas spaces $(u,\sigma)$0 and continuous $(u,\sigma)$1, under $(u,\sigma)$2 and $(u,\sigma)$3. They provide two regimes: one estimate with degree $(u,\sigma)$4, and an improved estimate when using degree $(u,\sigma)$5 for $(u,\sigma)$6. In addition, the least-squares framework yields the best-approximation bound

$(u,\sigma)$7

with $(u,\sigma)$8 independent of $(u,\sigma)$9 when boundary layers are sufficiently resolved.

Adaptivity is built directly from the least-squares residual. Elementwise indicators are

$L^*$00

The refinement loop uses maximum marking: mark all $L^*$01 with $L^*$02, then refine marked elements and neighbors to avoid hanging nodes. According to the reported numerical results, for the boundary-layer test with $L^*$03 and also $L^*$04, all three formulations produce adaptive meshes clustered along the outflow boundary layers, layer-resolved solutions without spurious oscillations even on coarse meshes away from layers, and convergence of the triple-bar error at rate $L^*$05, robust with respect to $L^*$06; using tolerance $L^*$07 as stopping criterion, effectivity indices remain close to unity. For the interior-layer test, all three methods capture both boundary and interior layers without visible oscillations.

The paper explicitly compares this framework to SUPG. The inclusion of $L^*$08 parallels the SUPG idea of adding residuals tested along streamlines, but here it appears in the analysis norm, not the bilinear form, preserving symmetric positive definite algebraic systems, freedom from inf–sup constraints on finite element pairs, and a natural a posteriori estimator equal to the residual functional value.

4. Primal operator–adjoint operator coupling and $L^*$09 methods

For scalar linear hyperbolic PDEs, Dual-Stream Least Squares refers to formulations that exploit both the primal operator $L^*$10 and its adjoint $L^*$11 to obtain $L^*$12-targeted approximations. The model problem is written abstractly as

$L^*$13

with $L^*$14 and homogeneous inflow boundary conditions, and concretely for the transport equation $L^*$15 in $L^*$16 with $L^*$17 on the inflow boundary $L^*$18 and outflow $L^*$19. The domains are

$L^*$20

$L^*$21

with Poincaré-type inequalities and surjectivity assumptions ensuring that $L^*$22 and $L^*$23 are Hilbert spaces with graph-norm equivalence (Kalchev et al., 2020).

The central motivation is that the best $L^*$24 approximation

$L^*$25

is generally infeasible because $L^*$26 is unknown. The dual construction replaces this by feasible least-squares principles using the weak operator $L^*$27, defined by

$L^*$28

and the Riesz map $L^*$29, given through

$L^*$30

Theorem 3.1 in the paper shows that $L^*$31 is a bijective isometry,

$L^*$32

and Theorem 3.3 shows that $L^*$33 is self-adjoint and positive definite and induces the dual norm on $L^*$34.

This produces several formulations. In the standard $L^*$35 method, with $L^*$36,

$L^*$37

and the weak form is

$L^*$38

The mixed $L^*$39 dual least-squares method instead minimizes

$L^*$40

or equivalently solves the mixed system

$L^*$41

$L^*$42

The algebraic block system is

$L^*$43

with Schur complement $L^*$44.

A key identity is

$L^*$45

which implies that the dual method minimizes the projected $L^*$46-component of the error:

$L^*$47

The discrete $L^*$48 coercivity condition is the inf-sup inequality

$L^*$49

Under this condition, the paper derives

$L^*$50

If $L^*$51, then $L^*$52 and $L^*$53, recovering the $L^*$54 projection.

The single-stage and two-stage $L^*$55-type methods hybridize the $L^*$56 projection with an additional projection onto $L^*$57. Their error estimate depends on approximation properties of both $L^*$58 and $L^*$59:

$L^*$60

where $L^*$61 and $L^*$62.

The preconditioning discussion is part of the method’s practical identity. For the mixed dual system, a symmetric block preconditioner uses $L^*$63 and a Schur complement preconditioner $L^*$64. For the single-stage SPD system, an SPD block preconditioner uses $L^*$65 and $L^*$66 for the Schur complement $L^*$67, which is spectrally equivalent to $L^*$68 independently of the inf-sup condition. Matrix-free application of $L^*$69 and the right-hand side requires only applications of $L^*$70 and $L^*$71 and sparse matrix-vector products with $L^*$72 and $L^*$73.

The numerical results emphasize the difference between directly targeting $L^*$74 and relying on $L^*$75 approximation quality. For $L^*$76 linear and $L^*$77 quadratic on the same meshes with $L^*$78, the dual $L^*$79 method attains an $L^*$80 rate $L^*$81 in a pre-asymptotic regime with unresolved layers, while $L^*$82, single-stage, and two-stage methods converge more slowly. Across the reported tests, the dual method exhibits less oscillation and better resolution of steep layers, whereas the mixed system is indefinite and more challenging to precondition if the discrete inf-sup constant deteriorates with $L^*$83.

5. Dual-norm least squares, saddle points, and inhomogeneous boundary conditions

In quasi-optimal least squares for elliptic PDEs with inhomogeneous boundary conditions, Dual-Stream Least Squares denotes a least-squares formulation in which primal trial variables are coupled to dual test variables that realize dual norms. The basic setting considers second-order elliptic problems on a bounded Lipschitz domain $L^*$84 with disjoint closed boundary parts $L^*$85 and $L^*$86, where

$L^*$87

with $L^*$88 uniformly positive definite and $L^*$89 a bounded first-order operator. The central issue is that classical least-squares formulations measure boundary residuals in $L^*$90 and $L^*$91, which the paper describes as computationally expensive for finite elements and essentially infeasible in machine learning for $L^*$92 (Monsuur et al., 2024).

The remedy is to replace boundary fractional norms by equivalent dual norms in domain spaces through trace dual mappings. For example,

$L^*$93

and

$L^*$94

This yields the dual-norm least-squares functional

$L^*$95

where

$L^*$96

The dual-stream interpretation is explicit: a primal stream approximates $L^*$97 in $L^*$98, while a dual stream approximates supremizers in the test spaces $L^*$99 and $\mathcal{L}\mathcal{L}^*$00. Through the Fenchel-type identity

$\mathcal{L}\mathcal{L}^*$01

the least-squares problem becomes a saddle-point problem with Lagrangian

$\mathcal{L}\mathcal{L}^*$02

so that

$\mathcal{L}\mathcal{L}^*$03

At stationarity, the dual variables are Riesz representatives of the residuals:

$\mathcal{L}\mathcal{L}^*$04

Quasi-optimality is expressed through an inf-sup constant

$\mathcal{L}\mathcal{L}^*$05

Under the paper’s assumptions, the discrete minimizer satisfies

$\mathcal{L}\mathcal{L}^*$06

With a preconditioner $\mathcal{L}\mathcal{L}^*$07, the constant becomes

$\mathcal{L}\mathcal{L}^*$08

The paper also provides an a posteriori estimator

$\mathcal{L}\mathcal{L}^*$09

and states efficiency and asymptotic reliability.

The finite element realization uses uniformly stable pairs. In the first-order modified mild formulation, the trial spaces are

$\mathcal{L}\mathcal{L}^*$10

with test spaces

$\mathcal{L}\mathcal{L}^*$11

Uniform Fortin operators are built through right inverses for the boundary traces, and linear-time preconditioners are stated to exist for the relevant $\mathcal{L}\mathcal{L}^*$12 and $\mathcal{L}\mathcal{L}^*$13 stiffness matrices.

The same saddle-point logic is carried into machine learning through adversarial networks. The primal variable $\mathcal{L}\mathcal{L}^*$14 or $\mathcal{L}\mathcal{L}^*$15 is represented by a neural network, while dual/test networks parameterize the maximizers. The paper defines four minimax objectives, QOLS1, QOLS1$\mathcal{L}\mathcal{L}^*$16, QOLS2, and QOLS2$\mathcal{L}\mathcal{L}^*$17, differing by first- versus second-order formulation and by whether the Dirichlet boundary dual space is vector-valued $\mathcal{L}\mathcal{L}^*$18 or scalar $\mathcal{L}\mathcal{L}^*$19. Training alternates $\mathcal{L}\mathcal{L}^*$20 minimization steps in the primal parameters and $\mathcal{L}\mathcal{L}^*$21 maximization steps in the dual parameters, with loss integrals approximated by Monte Carlo sampling or adaptive Gauss–Legendre quadrature. For machine learning, the paper states a quasi-best guarantee

$\mathcal{L}\mathcal{L}^*$22

when the dual test networks are sufficiently rich to satisfy an empirical inf-sup condition.

The numerical examples illustrate both FE and ML regimes. For finite elements, adaptive refinement on an L-shaped rectangle with mixed Dirichlet/Neumann data and exact singular solution $\mathcal{L}\mathcal{L}^*$23 achieves optimal rates for the available regularity, with stable effectivity indices and indicators concentrating near boundary segments. For machine learning on Laplace with non-smooth boundary data on an L-shaped domain and exact $\mathcal{L}\mathcal{L}^*$24, the reported outcome is that QOLS formulations outperform DRM, PINN, and WAN in $\mathcal{L}\mathcal{L}^*$25-error over $\mathcal{L}\mathcal{L}^*$26k epochs, often by factors up to $\mathcal{L}\mathcal{L}^*$27; QOLS1/QOLS2 slightly outperform the $\mathcal{L}\mathcal{L}^*$28 counterparts, while the $\mathcal{L}\mathcal{L}^*$29 versions can be advantageous in higher dimension because $\mathcal{L}\mathcal{L}^*$30 uses scalar $\mathcal{L}\mathcal{L}^*$31 rather than vector $\mathcal{L}\mathcal{L}^*$32.

6. Shared global and local refinement streams in personalized federated learning

In “APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares,” the term denotes a federated architecture with two analytic heads after a frozen backbone: a shared primary stream for global generalization across all clients and a dedicated refinement stream for local personalization of each individual client (Fan et al., 14 Aug 2025). Each client $\mathcal{L}\mathcal{L}^*$33 has local data $\mathcal{L}\mathcal{L}^*$34, with matrix notation $\mathcal{L}\mathcal{L}^*$35 and $\mathcal{L}\mathcal{L}^*$36. A pre-trained foundation model $\mathcal{L}\mathcal{L}^*$37 is used purely for feature extraction, and $\mathcal{L}\mathcal{L}^*$38 remains fixed throughout APFL.

The two streams use different random projections and activations:

$\mathcal{L}\mathcal{L}^*$39

The shared head is $\mathcal{L}\mathcal{L}^*$40 and the client-specific head is $\mathcal{L}\mathcal{L}^*$41. Prediction on client $\mathcal{L}\mathcal{L}^*$42 is additive:

$\mathcal{L}\mathcal{L}^*$43

where $\mathcal{L}\mathcal{L}^*$44 balances global generalization and local personalization.

The least-squares objectives are ridge-regularized and admit closed forms. The global primary stream solves

$\mathcal{L}\mathcal{L}^*$45

with analytic solution

$\mathcal{L}\mathcal{L}^*$46

The local refinement stream on client $\mathcal{L}\mathcal{L}^*$47 fits the residual after subtracting the primary-stream contribution:

$\mathcal{L}\mathcal{L}^*$48

with solution

$\mathcal{L}\mathcal{L}^*$49

The federated protocol is built from sufficient statistics rather than raw data. Client $\mathcal{L}\mathcal{L}^*$50 computes

$\mathcal{L}\mathcal{L}^*$51

and transmits $\mathcal{L}\mathcal{L}^*$52 and $\mathcal{L}\mathcal{L}^*$53 to the server. The server aggregates

$\mathcal{L}\mathcal{L}^*$54

and reconstructs the centralized solution

$\mathcal{L}\mathcal{L}^*$55

After receiving $\mathcal{L}\mathcal{L}^*$56, each client computes its local refinement matrix $\mathcal{L}\mathcal{L}^*$57 and performs personalized inference by

$\mathcal{L}\mathcal{L}^*$58

A distinctive theoretical property is heterogeneity invariance. Under a frozen deterministic backbone, fixed random projections $\mathcal{L}\mathcal{L}^*$59, fixed activations $\mathcal{L}\mathcal{L}^*$60, fixed ridge parameters $\mathcal{L}\mathcal{L}^*$61, and two federations having identical union datasets but different client-wise partitions, the paper states that the global $\mathcal{L}\mathcal{L}^*$62 is identical because it depends only on the sums $\mathcal{L}\mathcal{L}^*$63 and $\mathcal{L}\mathcal{L}^*$64. If client $\mathcal{L}\mathcal{L}^*$65’s local dataset is unchanged, then $\mathcal{L}\mathcal{L}^*$66 is also identical, because it depends only on $\mathcal{L}\mathcal{L}^*$67, $\mathcal{L}\mathcal{L}^*$68, $\mathcal{L}\mathcal{L}^*$69, and the invariant $\mathcal{L}\mathcal{L}^*$70. The stated limitation is that if $\mathcal{L}\mathcal{L}^*$71, $\mathcal{L}\mathcal{L}^*$72, $\mathcal{L}\mathcal{L}^*$73, or $\mathcal{L}\mathcal{L}^*$74 differ across clients, or if the backbone is not frozen, this invariance may be affected.

The computational profile is also analytic. Client-side costs include $\mathcal{L}\mathcal{L}^*$75 for $\mathcal{L}\mathcal{L}^*$76, $\mathcal{L}\mathcal{L}^*$77 for inverting $\mathcal{L}\mathcal{L}^*$78, $\mathcal{L}\mathcal{L}^*$79 for $\mathcal{L}\mathcal{L}^*$80, plus the analogous refinement costs. Communication to the server is $\mathcal{L}\mathcal{L}^*$81 of size $\mathcal{L}\mathcal{L}^*$82 plus $\mathcal{L}\mathcal{L}^*$83 of size $\mathcal{L}\mathcal{L}^*$84, for size $\mathcal{L}\mathcal{L}^*$85. The paper recommends Cholesky factorization for $\mathcal{L}\mathcal{L}^*$86 and $\mathcal{L}\mathcal{L}^*$87, and notes that conjugate gradient can solve $\mathcal{L}\mathcal{L}^*$88 without explicit inversion for large $\mathcal{L}\mathcal{L}^*$89.

The empirical evaluation uses CIFAR-100 and ImageNet-R, non-IID Dirichlet partitions with $\mathcal{L}\mathcal{L}^*$90, and $\mathcal{L}\mathcal{L}^*$91 clients, with ViT-MAE-Base as frozen backbone. The reported average per-client test accuracies are as follows. On CIFAR-100 with $\mathcal{L}\mathcal{L}^*$92 clients, APFL achieves $\mathcal{L}\mathcal{L}^*$93 at $\mathcal{L}\mathcal{L}^*$94, $\mathcal{L}\mathcal{L}^*$95 at $\mathcal{L}\mathcal{L}^*$96, and $\mathcal{L}\mathcal{L}^*$97 at $\mathcal{L}\mathcal{L}^*$98; the best baselines are $\mathcal{L}\mathcal{L}^*$99, $(u,\sigma)$00, and $(u,\sigma)$01, giving improvements of $(u,\sigma)$02, $(u,\sigma)$03, and $(u,\sigma)$04. On CIFAR-100 with $(u,\sigma)$05 clients, the reported gains are $(u,\sigma)$06, $(u,\sigma)$07, and $(u,\sigma)$08. On ImageNet-R with $(u,\sigma)$09 clients, APFL reaches $(u,\sigma)$10, $(u,\sigma)$11, and $(u,\sigma)$12, with advantages of $(u,\sigma)$13, $(u,\sigma)$14, and $(u,\sigma)$15. On ImageNet-R with $(u,\sigma)$16 clients, the corresponding gains are $(u,\sigma)$17, $(u,\sigma)$18, and $(u,\sigma)$19.

The paper further states that APFL reaches its final accuracy in a single aggregation round per client, whereas gradient-based baselines require up to $(u,\sigma)$20 rounds. For CIFAR-100 with $(u,\sigma)$21 and $(u,\sigma)$22 clients, APFL achieves at least $(u,\sigma)$23 higher accuracy while using $(u,\sigma)$24 of the baselines’ computation overhead and $(u,\sigma)$25 of their communication overhead. Over overhead-versus-round curves, its total overhead is stated to be at least $(u,\sigma)$26 and $(u,\sigma)$27 lower than baselines within the first $(u,\sigma)$28 rounds, and up to $(u,\sigma)$29 and $(u,\sigma)$30 lower after $(u,\sigma)$31 rounds, in computation and communication respectively.

Ablation studies characterize the interaction between the two streams. On CIFAR-100 at $(u,\sigma)$32, the primary stream alone gives $(u,\sigma)$33, while the dual model at $(u,\sigma)$34 gives $(u,\sigma)$35. On ImageNet-R at $(u,\sigma)$36, the primary stream alone gives $(u,\sigma)$37, while the dual model yields $(u,\sigma)$38. The reported trend is that performance rises then falls as $(u,\sigma)$39 increases; the optimal $(u,\sigma)$40 is about $(u,\sigma)$41–$(u,\sigma)$42 for CIFAR-100 and about $(u,\sigma)$43–$(u,\sigma)$44 for ImageNet-R, and as heterogeneity decreases, the optimal $(u,\sigma)$45 decreases because less personalization is needed. The paper also reports that the primary stream is insensitive to $(u,\sigma)$46, that the refinement stream is more sensitive to $(u,\sigma)$47, that performance rises then falls with increasing $(u,\sigma)$48 and $(u,\sigma)$49, and that the method is relatively insensitive to activation choice.

Across these domains, Dual-Stream Least Squares consistently denotes a least-squares system in which two coupled channels are assigned distinct technical roles and then recombined. In image stitching, the channels are two physically separate image streams and the least-squares machinery is local, geometric, and temporally gated. In convection-dominated PDEs, the channels are primal–dual variables and streamline-aware control, producing $(u,\sigma)$50-robust coercivity, best-approximation properties, and adaptive refinement. In hyperbolic transport, the channels are $(u,\sigma)$51 and $(u,\sigma)$52, giving a precise route from residual minimization to $(u,\sigma)$53-targeted approximation. In QOLS, the channels are primal trial functions and dual supremizers, turning dual norms into computable saddle-point problems without fractional boundary norms. In APFL, the channels are shared and client-specific analytic heads, yielding a single-round personalized learning protocol with a stated heterogeneity invariance property. The recurring principle is therefore not domain-specific nomenclature, but the systematic use of two complementary streams so that least-squares fitting acquires locality, adjoint consistency, streamline stability, boundary realizability, or personalization that would be more difficult to obtain from a single-stream construction alone.