Online Regularized Learning

• Online Regularized Learning Algorithm is a sequential model that dynamically adjusts regularization during each stochastic gradient update in RKHS.
• It employs a time-varying Tikhonov regularization path with optimal gain scheduling to balance bias and variance, ensuring strong and weak convergence.
• The approach is computationally efficient for streaming data and underpins nonparametric online learning tasks with theoretical guarantees.

An online regularized learning algorithm is a sequential model update strategy that incorporates explicit regularization (such as penalties or constraints) into the online learning process, often to achieve optimal convergence and generalization in infinite-dimensional settings such as reproducing kernel Hilbert spaces (RKHSs). Unlike batch regularization, which uses a fixed parameter throughout training, the online approach adapts the regularization dynamically, tracking a so-called "regularization path" as the algorithm processes data one sample at a time.

1. Formal Algorithmic Structure

The core of the online regularized learning algorithm as developed in the referenced work is a stochastic gradient descent update in an RKHS that follows a time-varying Tikhonov regularization path. The update at iteration $t$ is given by:

$$f_t = f_{t-1} - \gamma_t \Bigl[(f_{t-1}(x_t) - y_t)K_{x_t} + \lambda_t f_{t-1}\Bigr]$$

• $K_{x_t}$ is the kernel section at sample $x_t$.
• $\gamma_t > 0$ is the learning rate (or gain, step size).
• $\lambda_t > 0$ is the time-dependent regularization parameter.
• $(x_t, y_t)$ is the independent data sample observed at step $t$.

The regularization path is defined, for fixed $\lambda$, by the minimizer of the Tikhonov regularized risk:

$f_\lambda = (L_K + \lambda I)^{-1}L_K f_\rho$

• $L_K : L^2(\rho) \to L^2(\rho)$ is the integral/covariance operator induced by the kernel.
• $f_\rho$ is the (unknown) regression function.

The online algorithm "tracks" the path $\lambda_t \mapsto f_{\lambda_t}$ by dynamically updating both $\gamma_t$ and $\lambda_t$, letting $\lambda_t \to 0$ as $t \to \infty$.

2. Convergence Theory and Optimal Rates

Two principal types of convergence are addressed:

• Strong convergence (in the RKHS norm $\|\cdot\|_K$): This establishes convergence of the iterates to $f_\rho$ at the fastest known rates under sufficient regularity. The main rate is

$$\|f_t - f_\rho\|_K \leq O\Bigl( t^{- \frac{2r-1}{4r+2}} \Bigr) \qquad (\text{high probability})$$

where $r > 1/2$ is the order of smoothness in the source condition $L_K^{-r} f_\rho \in L^2(\rho)$.

• Weak convergence (in $L^2$ norm): The mean square error decays at the minimax optimal rate,

$$\|f_t - f_\rho\|_2 \leq O\Bigl( t^{- \frac{r}{2r+1}} \Bigr) \qquad (\text{high probability})$$

The gain and regularization sequences are chosen as power laws in $t$,

$$\gamma_t = a t^{-\theta},\qquad \lambda_t = b t^{-(1-\theta)}$$

with the optimal decay $\theta = 2r/(2r+1)$. This precisely balances the trade-off between bias and variance.

3. Bias–Variance Decomposition and Error Structure

The total error $f_t - f_\rho$ is decomposed via structural results akin to batch learning:

• Initial error (from $f_0$)
• Approximation error ($f_\rho - f_{\lambda_t}$)
• Drift error ($f_{\lambda_t} - f_{\lambda_{t-1}}$)
• Sample error (from randomness in updates)

This decomposition is formalized using martingale techniques. For instance, with the reversed martingale representation:

$$r_t = \Pi_1^t r_0 - \sum_{j=1}^t \gamma_j \Pi_{j+1}^t (A_j w_j - b_j) - \sum_{j=1}^t \Pi_j^t \Delta_j,\qquad r_t = f_t - f_{\lambda_t}$$

• $\Pi_j^t = \prod_{i=j}^t (I - \gamma_i A_i)$
• $A_t, b_t$ are random (sample-dependent) analogues of $L_K, L_K f_\rho$
• $\Delta_j = f_{\lambda_j} - f_{\lambda_{j-1}}$ (path drift)

Both the approximation and drift errors decay at the rate $$O(\lambda_t^{r-1/2}) \sim O(t^{-(r-1/2)(1-\theta)})$$.

The variance term is controlled using Bernstein-type inequalities for martingales in Hilbert spaces, yielding with high probability:

$$\sup_{1 \leq k \leq t} \left\| \sum_{i=1}^k \xi_i \right\| \leq 2 (M/3 + \sigma_t) \log(2/\delta)$$

for martingale differences $\xi_i$ bounded in norm by $M$ and with conditional second moment $\sigma_t^2$.

4. Implementation and Tuning Considerations

To realize optimal rates, the sequences $\gamma_t$, $\lambda_t$ must be chosen to satisfy the coupled condition $\gamma_t\lambda_t \approx 1/t$ (or within logarithmic factors), exploiting the phase transition at $\theta > 1/2$ for the convergence rates. The sample error terms are computed per-iteration using the incoming data; the approximation and drift errors are handled analytically via the regularization path.

The update is computationally lightweight per iteration (assuming efficient access to the kernel) and amenable to parallelization and streaming data, essential for large-scale online settings.

5. Connections and Practical Impact

This online regularized approach recovers the minimax rates of batch regularization, including for strongly convex objectives in RKHS and in the mean square error sense. By careful tracking of a vanishing regularization parameter and matched gain sequence, the algorithm interpolates between bias-limited and variance-limited regimes, and is robust to initialization. The analysis relies crucially on operator-theoretic properties and bias–variance trade-offs.

The framework is broadly extensible to various nonparametric and kernelized online learning tasks that demand high sample efficiency and theoretical guarantees, matching the best-in-class rates previously only associated with batch learning strategies.

6. Summary Table of Key Quantities

Quantity	Formula / Rate	Role
Update Rule	$$f_t = f_{t-1} - \gamma_t\left[(f_{t-1}(x_t)-y_t)K_{x_t} + \lambda_t f_{t-1}\right]$$	Online gradient with regularization
Regularization Path	$f_\lambda = (L_K+\lambda I)^{-1}L_K f_\rho$	Batch regularized estimator
Optimal Decay	$\theta = 2r/(2r+1)$, $\gamma_t \sim t^{-\theta}$, $\lambda_t \sim t^{-(1-\theta)}$	Bias–variance balancing
Strong Conv. Rate (RKHS)	$O\left( t^{- (2r-1)/(4r+2)} \right)$	If $r > 1/2$
Weak Conv. Rate ($L^2$)	$O\left( t^{- r/(2r+1)} \right)$	Minimax rate

7. Mathematical Summary and Theorem (Editor’s term)

Theorem (Strong and Weak Online Learning Rates, Editor’s term):

Let $f_t$ be generated by the recursive update above, with optimal $\gamma_t$, $\lambda_t$, and data of regularity $r > \frac{1}{2}$. Then with high probability,

$$\|f_t - f_\rho\|_K \leq O(t^{- (2r-1)/(4r+2)}) \qquad \text{and} \qquad \|f_t - f_\rho\|_2 \leq O(t^{- r/(2r+1)})$$

where the rates match the best-known batch learning bounds.

---

In conclusion, the online regularized learning algorithm in RKHS analyzed here attains theoretically optimal convergence (both strong and weak) by tracking a regularization path with matched gain and regularization sequences. Bias–variance decomposition, martingale concentration inequalities, and operator-theoretic rates underpin the analysis and practical design, making this approach a foundational method for nonparametric online learning.