Regularization of Statistical Inverse Problems on Non-Reflexive Banach Spaces

Abstract: Inverse learning within a statistical framework has a wide range of applications. It has garnered significant attention in machine learning, artificial intelligence, and related fields, where the goal is to infer unknown parameters from indirect and noisy observations. This work investigates the stable approximation of $u<sup>{\dagger}$ which solves the equation $Au=g$, with $A$ being a linear operator between appropriate vector spaces. We will consider the domain to be a non-reflexive Banach Space and the co-domain to be a space of real-valued functions on a metric space $X$. The function $g$ is characterized by a finite number of independently and identically distributed data points, which are assumed to follow some unknown probability measure $ρ$. We employ Tikhonov regularization with an arbitrary convex functional to obtain the regularized solution corresponding to the given data point. The convergence analysis is carried out with respect to the Bregman distance, and an upper bound for the error is derived in probability terms. The theoretical findings are then supported by numerical experiments.

• The paper extends statistical inverse learning to arbitrary Banach spaces using general convex penalties and high-probability Bregman-distance bounds without reflexivity or covering-number assumptions.
• For norm penalties on spaces such as ℓ¹, the method achieves the conditional rate O(m^{-β/2(p+1+β)}), while q-uniform convexity improves norm convergence to O(m^{-β/2(1+β(q−1))}).
• The numerical Legendre-expansion experiment demonstrates decreasing empirical Bregman error for sparse recovery, but the general rate depends on a source condition and is not proven optimal for non-reflexive domains.

Problem setting and motivation

The paper studies the statistical inverse learning problem of recovering an unknown element $u_\rho$ in a Banach space $B_1$ from i.i.d. noisy samples $z_i = (x_i, y_i) \in X \times [-M,M]$ drawn from an unknown Borel measure $\rho$, where $u_\rho$ minimizes the $L^p$-type risk $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$ for an injective bounded linear operator $A: B_1 \to V \subseteq \mathcal{F}(X,\mathbb{R})$. The regularized estimator is defined by Tikhonov-type minimization over the empirical risk with penalty $\lambda \Omega(u)$, where $\Omega$ is any continuous convex functional.

The central contribution is the removal of reflexivity assumptions on the domain. Prior work by the authors [(2608.17533), cf. their earlier Journal of Complexity paper] required $B_1$0 to be $B_1$1-uniformly convex — hence reflexive — which excludes spaces such as $B_1$2 that are natural in sparse reconstruction and signal processing. The present work extends the framework to arbitrary Banach spaces, using a generalized definition of reproducing kernel Banach space (RKBS) following Xu's sparse-learning formulation, which requires only continuity of point evaluations rather than reflexivity or compactness of the unit ball. Notably, the authors also drop the covering-number assumption with logarithmic growth used previously, which they state yields a faster rate than before; the earlier results are recovered as special cases.

Because $B_1$3 is generally nonsmooth and no Hilbert structure is available, convergence is measured in the Bregman distance $B_1$4, where $B_1$5.

Main results

High-probability Bregman bound. Under uniform boundedness of evaluation functionals $B_1$6 ($B_1$7), measurability, and an approximation-error assumption $B_1$8, the first theorem establishes that there exists a subgradient $B_1$9 such that, for all $z_i = (x_i, y_i) \in X \times [-M,M]$0,

$z_i = (x_i, y_i) \in X \times [-M,M]$1

with constants depending only on $z_i = (x_i, y_i) \in X \times [-M,M]$2. The proof proceeds via an explicit characterization of $z_i = (x_i, y_i) \in X \times [-M,M]$3, a subgradient identity at $z_i = (x_i, y_i) \in X \times [-M,M]$4, a comparison inequality between regularized solutions under two measures, and a Hoeffding-type concentration argument applied to the empirical deviation of a bounded random variable.

Norm penalty. For $z_i = (x_i, y_i) \in X \times [-M,M]$5, the subdifferential norms equal one, giving $z_i = (x_i, y_i) \in X \times [-M,M]$6 explicitly, and the stochastic term is bounded via $z_i = (x_i, y_i) \in X \times [-M,M]$7. With confidence $z_i = (x_i, y_i) \in X \times [-M,M]$8:

$z_i = (x_i, y_i) \in X \times [-M,M]$9

where $\rho$0. Choosing $\rho$1 yields the rate

$\rho$2

with high probability. This result applies to genuinely non-reflexive domains such as $\rho$3, which is the paper's principal point of generality.

Uniformly convex improvement. When $\rho$4 is $\rho$5-uniformly convex and $\rho$6, the Bregman distance dominates $\rho$7, and reflexivity permits dual-norm attainment. The resulting norm-convergence bound,

$\rho$8

features a constant $\rho$9 independent of $u_\rho$0 — a structural advantage over the general case where $u_\rho$1 grows polynomially in $u_\rho$2. With $u_\rho$3 this gives $u_\rho$4, improving on the previous rate $u_\rho$5, $u_\rho$6. For Hilbert spaces ($u_\rho$7), the rate becomes $u_\rho$8, which coincides with the optimal RKHS rate of Blanchard–Mücke when the source-condition exponents satisfy the stated matching condition.

Numerical illustration

The theory is instantiated on $u_\rho$9 with $L^p$0 a diagonal Legendre-polynomial expansion $L^p$1, $L^p$2. Weak* lower semicontinuity of $L^p$3 and $L^p$4 under $L^p$5, together with Banach–Alaoglu compactness, verifies all structural assumptions except (A2). For coefficients decaying as $L^p$6, the authors verify (A2) explicitly with $L^p$7, computed coordinatewise through soft-thresholding of the population solution. Simulations with $L^p$8 and noise levels $L^p$9 show monotone decay of the empirical Bregman error (e.g., from roughly $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$0 to $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$1 across the sample-size range at $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$2). The authors note they do not invoke the corollary's rate numerically because it is not sharp.

Limitations and open questions

Several caveats bear directly on the strength of the results. First, the approximation assumption (A2) is imposed rather than derived in the abstract setting; it is verified only for the specific diagonal operator in the numerical section, so the rates are conditional on a source-type condition whose validity must be checked per problem. Second, the constants $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$3, $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$4, and especially $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$5 grow polynomially in $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$6 in the general Banach case, degrading the effective sample complexity relative to the uniformly convex setting. Third, the authors state plainly that the upper rate $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$7 is not sharp in general and does not establish optimality on non-reflexive domains; optimality remains open and is the subject of ongoing work. Finally, the analysis is restricted to linear operators $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$8; extension to nonlinear forward maps $$\mathcal{E}_\rho(u) = \int_Z |y - (Au)(x)|^p\, d\rho$$9 is identified as a direction not covered here.

Conclusion

This work extends statistical inverse learning with Tikhonov regularization to arbitrary Banach domains, replacing norm-power penalties with general convex functionals and measuring error in the Bregman distance. It delivers high-probability convergence rates of order $A: B_1 \to V \subseteq \mathcal{F}(X,\mathbb{R})$0 in full generality and $A: B_1 \to V \subseteq \mathcal{F}(X,\mathbb{R})$1 under $A: B_1 \to V \subseteq \mathcal{F}(X,\mathbb{R})$2-uniform convexity, strictly improving prior results while removing both reflexivity and covering-number assumptions. The $A: B_1 \to V \subseteq \mathcal{F}(X,\mathbb{R})$3 numerical example demonstrates applicability to sparse reconstruction settings excluded by earlier frameworks, though the sharpness of the general rate remains unresolved.