Orthogonal Spherical Smoothing Techniques

• Orthogonal spherical smoothing is a method that couples spherical perturbations with an orthonormal frame to yield unbiased Jacobian approximations in derivative-free nonlinear least-squares.
• It reduces finite-difference variance by averaging over random orthogonal directions, ensuring probabilistically first-order accurate gradient estimates with complexity guarantees.
• The approach is applicable across diverse fields such as optimization, room acoustics, parallel MRI, and manifold smoothing, offering structured regularization and efficient computation.

Searching arXiv for papers on orthogonal spherical smoothing and closely related methods. arXiv search query: orthogonal spherical smoothing Jacobian approximation derivative-free Levenberg-Marquardt nonlinear least squares Orthogonal spherical smoothing denotes a family of constructions in which smoothing on a sphere, or by spherical perturbations, is coupled to an orthogonality structure. In derivative-free nonlinear least-squares, the term refers to approximating Jacobian matrices by averaging directional finite differences along orthonormal random directions drawn uniformly on the Stiefel manifold; in that setting it underpins a derivative-free Levenberg–Marquardt method with probabilistically first-order accurate gradient models and a high-probability complexity bound (Chen et al., 2024). In other parts of the literature, closely related language denotes averaging over orthogonal spherical-harmonic modes in room acoustics, regularization in an orthogonal spherical basis in parallel MRI, orthogonal projection and tight-frame smoothing on $\mathbb S^2$, or coordinate-wise smoothing in a transported orthonormal tangent frame on the sphere (Morgenstern et al., 2024, Zhu et al., 2018, Xiao et al., 2023, Kim et al., 2018).

1. Derivative-free definition in nonlinear least-squares

For nonlinear least-squares, let $r:\mathbb R^n \to \mathbb R^m$ be the vector of residuals and

$f(x)=\tfrac12\|r(x)\|^2.$

Orthogonal spherical smoothing is introduced componentwise through the spherically smoothed residual

$$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$

where $S^{n-1}$ is the unit sphere in $\mathbb R^n$ and $\gamma>0$ is a smoothing radius. Its gradient satisfies

$$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$

In implementation one draws $b$ orthonormal directions

$$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$

uniformly at random on the Stiefel manifold, and forms

$r:\mathbb R^n \to \mathbb R^m$0

Because any column $r:\mathbb R^n \to \mathbb R^m$1 of such a $r:\mathbb R^n \to \mathbb R^m$2 is marginally uniform on $r:\mathbb R^n \to \mathbb R^m$3, this is an unbiased estimator of $r:\mathbb R^n \to \mathbb R^m$4 (Chen et al., 2024).

The defining feature is not merely spherical perturbation, but the use of an orthonormal frame. The crucial point is that the $r:\mathbb R^n \to \mathbb R^m$5, $r:\mathbb R^n \to \mathbb R^m$6, form an orthonormal frame, so that the sample-average gradient is unbiased. This separates the method from generic independent-direction finite-difference estimators and places it within the broader class of zeroth-order methods based on orthogonal random directions.

2. Approximate Jacobian and probabilistic first-order accuracy

Having approximated each $r:\mathbb R^n \to \mathbb R^m$7 by $r:\mathbb R^n \to \mathbb R^m$8, one forms the approximate Jacobian

$r:\mathbb R^n \to \mathbb R^m$9

Its $f(x)=\tfrac12\|r(x)\|^2.$0-entry is, up to the scaling $f(x)=\tfrac12\|r(x)\|^2.$1, a directional finite difference: $f(x)=\tfrac12\|r(x)\|^2.$2 In many presentations one absorbs the factor $f(x)=\tfrac12\|r(x)\|^2.$3 into a weight $f(x)=\tfrac12\|r(x)\|^2.$4, and simply writes

$f(x)=\tfrac12\|r(x)\|^2.$5

with $f(x)=\tfrac12\|r(x)\|^2.$6 (Chen et al., 2024).

Under the usual Lipschitz assumptions on $f(x)=\tfrac12\|r(x)\|^2.$7 and $f(x)=\tfrac12\|r(x)\|^2.$8, one shows

$f(x)=\tfrac12\|r(x)\|^2.$9

and then bounds the bias and variance of the induced gradient model for $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$0: $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$1 Via a multivariate Chebyshev or Bernstein argument, for any $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$2,

$$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$3

Combining bias and variance gives, for a suitably chosen $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$4, that with probability $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$5,

$$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$6

that is, the gradient model is “probabilistically first-order accurate” (Chen et al., 2024).

This formulation makes clear that the smoothing radius $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$7 and the number of orthogonal directions $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$8 jointly control model quality. The data state that the bias is $$r_{s,i}(x) \coloneqq E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,],$$9 and the variance is $S^{n-1}$0; a plausible implication is that $S^{n-1}$1 governs approximation bias while $S^{n-1}$2 governs directional Monte Carlo variability.

3. Embedding in the derivative-free Levenberg–Marquardt iteration

The derivative-free Levenberg–Marquardt method uses the orthogonally smoothed Jacobian at each iterate $S^{n-1}$3. The iteration is:

1. draw an orthonormal frame $S^{n-1}$4;
2. form the Jacobian model $S^{n-1}$5 with $S^{n-1}$6;
3. set the Levenberg–Marquardt step by solving

$S^{n-1}$7

where

$S^{n-1}$8

The step is evaluated by the ratio

$S^{n-1}$9

One accepts $\mathbb R^n$0 whenever $\mathbb R^n$1; otherwise the step is rejected and $\mathbb R^n$2 is increased by a factor $\mathbb R^n$3. If $\mathbb R^n$4 but the model-gradient is “small,” one may also decrease $\mathbb R^n$5 by $\mathbb R^n$6. The practical implementation reuses the same orthonormal directions $\mathbb R^n$7 across all $\mathbb R^n$8 residuals at $\mathbb R^n$9, and updates $\gamma>0$0 to tie the bias $\gamma>0$1 to the actual step-length (Chen et al., 2024).

Within this scheme, orthogonal spherical smoothing is not an auxiliary preprocessing step. It is the mechanism by which the Jacobian model is constructed, and it therefore determines both the linear system used to define the Levenberg–Marquardt step and the model reduction appearing in $\gamma>0$2.

4. Complexity guarantees and relation to orthogonal random-direction methods

Under the Lipschitz and bounded-Jacobian assumptions, and choosing $\gamma>0$3 together with $\gamma>0$4, the random model is $\gamma>0$5-probabilistically accurate with $\gamma>0$6 small enough. Standard trust-region-style and worst-case analysis then yields constants $\gamma>0$7 and $\gamma>0$8 such that, for any $\gamma>0$9, with probability at least

$$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$0

one has

$$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$1

Equivalently, the number of iterations $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$2 needed to drive the gradient norm below $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$3 satisfies, with high probability,

$$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$4

more precisely

$$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$5

for computable $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$6 depending on $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$7 and the Lipschitz constants. The method only requires evaluations of the $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$8 residuals $$\nabla r_{s,i}(x)= (n/\gamma)\;E_{u\sim \mathrm{Uniform}(S^{n-1})}[\,r_i(x+\gamma u)\,u\,].$$9 at $b$0 points per iteration (Chen et al., 2024).

A related zeroth-order framework considers a smooth function $b$1 with Lipschitz gradient and approximates $b$2 by finite differences computed in a set of orthogonal random directions that changes with each iteration. There, the smoothed function is

$b$3

and the two-sided estimator is

$b$4

One efficient sampling scheme draws $b$5 with i.i.d. $b$6 entries, computes a QR decomposition $b$7 with $b$8 diagonal positive, and takes the first $b$9 columns of $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$0. Orthogonality helps because it enforces $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$1 and reduces correlation among directional estimates, lowering variance relative to i.i.d. sampling. The paper recovers spherical smoothing, coordinate descent, and discretized gradient descent as special cases, and establishes convergence rates in convex and Polyak–Łojasiewicz settings (Kozak et al., 2021).

Taken together, these results place orthogonal spherical smoothing within a larger zeroth-order methodology in which spherical perturbations are combined with orthogonality to obtain unbiasedness, lower correlation, and complexity guarantees.

5. Other domain-specific meanings

Taken across the literature, orthogonal spherical smoothing is not a single universal operator. The orthogonality may arise from spherical-harmonic modes, an orthogonal basis in a spherical function space, or an orthonormal tangent frame. The following usages are all explicit in the cited papers.

Domain	Orthogonality structure	Purpose
Derivative-free nonlinear least-squares (Chen et al., 2024)	Orthonormal random directions on $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$2	Jacobian approximation for LM
Room acoustics (Morgenstern et al., 2024)	Orthogonal SH channels of a spherical loudspeaker array	Rank restoration for MUSIC-type DOA estimation
Parallel MRI (Zhu et al., 2018)	Orthogonal spherical-function basis $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$3	Coil-sensitivity regularization
$$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$4 approximation and denoising (Xiao et al., 2023)	Orthogonal spherical harmonics and tight spherical framelets	Projection and multiscale smoothing/denoising
Spherical data smoothing splines (Kim et al., 2018)	Orthonormal basis of $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$5	Coordinate-wise Euclidean spline fitting

In room acoustics, a multiple-input multiple-output system combines a spherical microphone array and a spherical loudspeaker array. After plane-wave decomposition, one forms $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$6 and then defines modal vectors $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$7. Averaging over all $$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$8 loudspeaker modes gives

$$[u_1,\dots,u_b]\in \mathrm{St}(n,b) \coloneqq \{\,U\in\mathbb R^{n\times b}:U^\top U=I_b\,\},$$9

which restores rank in the averaged cross-spectrum. In particular, reflections sharing the same time delay but arriving from different directions are decorrelated. To decorrelate up to $r:\mathbb R^n \to \mathbb R^m$00 reflections arriving simultaneously, one needs $r:\mathbb R^n \to \mathbb R^m$01 (Morgenstern et al., 2024).

In parallel MRI reconstruction, the smoothing object is not a random-direction estimator but a regularization in a spherical-function basis. The basis functions are

$r:\mathbb R^n \to \mathbb R^m$02

which are mutually orthogonal in $r:\mathbb R^n \to \mathbb R^m$03 under suitable boundary conditions. Coil sensitivities are expanded in that basis, and the regularizer is

$r:\mathbb R^n \to \mathbb R^m$04

The paper studies the corresponding joint reconstruction model and a non-linear ADMM solver (Zhu et al., 2018).

For approximation and denoising on $r:\mathbb R^n \to \mathbb R^m$05, spherical $r:\mathbb R^n \to \mathbb R^m$06-designs provide equal-weight quadrature points. If $r:\mathbb R^n \to \mathbb R^m$07 is a $r:\mathbb R^n \to \mathbb R^m$08-design with $r:\mathbb R^n \to \mathbb R^m$09, then quadrature is exact for all spherical harmonics up to degree $r:\mathbb R^n \to \mathbb R^m$10, so the truncated expansion

$r:\mathbb R^n \to \mathbb R^m$11

coincides with the orthogonal projection of $r:\mathbb R^n \to \mathbb R^m$12 onto $r:\mathbb R^n \to \mathbb R^m$13. The same framework is used to construct semi-discrete spherical framelets satisfying a tight-frame property, and thresholding in the framelet domain yields denoising gains on noisy Wendland test functions (Xiao et al., 2023).

For spherical data on $r:\mathbb R^n \to \mathbb R^m$14, one may unroll and unwrap data into a single tangent space $r:\mathbb R^n \to \mathbb R^m$15, pick an orthonormal basis $r:\mathbb R^n \to \mathbb R^m$16, and fit $r:\mathbb R^n \to \mathbb R^m$17 independent scalar smoothing splines to the coordinate functions. This separate, coordinate-wise fitting in an orthonormal frame is explicitly identified as “orthogonal” smoothing, after which the fitted curve is wrapped back to the manifold by parallel transport and the exponential map (Kim et al., 2018).

6. Terminological scope, misconceptions, and boundary cases

A common misconception is to treat orthogonal spherical smoothing as synonymous with any smoothing procedure on a spherical domain. The cited literature does not support that equivalence. In the derivative-free optimization setting, orthogonality refers to the sampling geometry of the perturbation directions; in room acoustics it refers to spherical-harmonic loudspeaker modes; in MRI it refers to an orthogonal spherical basis; and in manifold smoothing it refers to an orthonormal tangent-frame representation (Chen et al., 2024, Morgenstern et al., 2024, Zhu et al., 2018, Kim et al., 2018).

Another misconception is that every spherical smoothing method has an orthogonal formulation already built into it. A counterexample is global-field verification on the sphere. There, the smoothing operator is the area-weighted spherical-cap average

$r:\mathbb R^n \to \mathbb R^m$18

and the paper introduces two acceleration schemes for exactly that same area-size-informed smoothing operator. The paper contains no discussion of orthogonality of the kernel, of diagonalizing the smoothing matrix, or of constructing an orthonormal basis of smoothing functions. It explicitly states that orthogonal kernel derivations, Gram–Schmidt on $r:\mathbb R^n \to \mathbb R^m$19, weighted inner-products on irregular spherical grids, discrete orthonormal-basis construction, missing-data handling in an orthogonal framework, and an FSS example built around such an orthogonal operator do not appear there (Skok et al., 2024).

These distinctions suggest that “orthogonal spherical smoothing” is best understood as a structural description rather than a single standardized algorithm. In the optimization paper that gives the term its clearest algorithmic meaning, the structure is: spherical smoothing of residual components, orthonormal random directions, unbiased Jacobian estimation, probabilistically first-order accurate models, and a derivative-free Levenberg–Marquardt method with high-probability $r:\mathbb R^n \to \mathbb R^m$20 complexity (Chen et al., 2024).