Recursive Kernel Centering in Functional Regression

• Recursive kernel-centering is a method that updates kernel-based estimators sequentially as new data arrive, enabling real-time functional regression with optimal convergence guarantees.
• The approach reduces computational overhead by avoiding full dataset recomputation, while establishing strong consistency, precise bias-variance trade-offs, and a central limit theorem for inference.
• By balancing bandwidth selection and kernel choice, the method achieves efficient, memory-aware estimation suitable for high-dimensional and streaming data contexts.

A recursive kernel-centering procedure is a statistical or computational mechanism in which kernel-based estimators or representations are updated sequentially—typically as new data arrive—by exploiting recursive or incremental update formulas. This methodology is especially relevant in nonparametric regression, density estimation, functional data analysis, and machine learning contexts where kernels define local weighting or similarity and real-time, memory-efficient processing is desirable. In the functional nonparametric regression setting, recursive kernel-centering enables fast updates of the regression operator, and theoretical analysis establishes precise asymptotic rates, convergence properties, and inference frameworks.

1. Formulation of the Recursive Kernel Estimator

The recursive kernel estimator for functional nonparametric regression is constructed as a ratio-form operator:

$$r_{n}^{(\ell)}(\chi) = \frac{\varphi^{(\ell)}_n(\chi)}{f^{(\ell)}_n(\chi)}$$

where, for sample $\left\{(\mathcal{X}_i, Y_i) : i=1,\ldots,n\right\}$ with $\mathcal{X}_i$ in a separable infinite-dimensional semi-normed space and $Y_i\in\mathbb{R}$,

$$\varphi^{(\ell)}_n(\chi) = \frac{\sum_{i=1}^{n} (F(h_i))^{-\ell} Y_i K(\|\chi-\mathcal{X}_i\|/h_i)}{\sum_{i=1}^n F(h_i)^{1-\ell}}$$

$$f^{(\ell)}_n(\chi) = \frac{\sum_{i=1}^{n} (F(h_i))^{-\ell} K(\|\chi-\mathcal{X}_i\|/h_i)}{\sum_{i=1}^n F(h_i)^{1-\ell}}$$

Parameters:

• $K(\cdot)$: kernel function (nonnegative, bounded, supported on $[0, 1]$)
• $h_i$: bandwidth sequence ($h_i \to 0$)
• $F(h_i)$: small-ball probability $\mathbb{P}(\|\chi - \mathcal{X}\| \leq h_i)$ at $\chi$
• $\ell \in [0, 1]$: parameter controlling recursion, $\ell=0$ (fully recursive), $\ell=1$ (semi-recursive)

Upon arrival of a new observation, terms for $n+1$ are computed, and the estimator is updated incrementally without recomputation over the full dataset.

2. Asymptotic Analysis: MSE, Bias, and Variance

Theoretical analysis derives the asymptotic bias and variance for $r_n^{(\ell)}(\chi)$, central to understanding accuracy and efficiency:

$$\mathbb{E}[r_n^{(\ell)}(\chi)] - r(\chi) = \varphi'(0) \frac{\alpha_{[\ell]}}{\beta_{[1-\ell]}} \frac{M_0}{M_1} h_n [1+o(1)] + O\left(\frac{1}{n F(h_n)}\right)$$

$$\operatorname{Var}[r_n^{(\ell)}(\chi)] = \frac{\beta_{[1-2\ell]}}{\beta_{[1-\ell]}^2} \frac{M_2}{M_1^2} \sigma_\varepsilon^2(\chi) \frac{1}{n F(h_n)} [1+o(1)]$$

Constants:

• $M_0, M_1, M_2$: kernel-dependent, e.g., integrals over $K$; $M_0 = K(1) - \int_0^1 (sK(s))' \tau_0(s)\,ds$
• $\beta_{[1-\ell]}, \beta_{[1-2\ell]}$: limits involving small-ball probabilities and bandwidth sequence
• $\alpha_{[\ell]}$: analogous limit for numerator
• $\varphi'(0)$: derivative of $$\varphi(t) = \mathbb{E}[r(\mathcal{X}) - r(\chi)\mid \|\mathcal{X}-\chi\|=t]$$
• $$\sigma_\varepsilon^2(\chi) = \operatorname{Var}(\varepsilon|\mathcal{X}=\chi)$$

The bias is $O(h_n)$; variance is $O((n F(h_n))^{-1})$; $nF(h_n)$ is an effective sample size determined by the local geometry ("small-ball" probability). Selection of $h_n$ that balances these terms is critical.

A key asymptotic trade-off:

$$\lim_{n \to \infty} n F(h_n) \mathbb{E}\left[(r_n^{(\ell)}(\chi) - r(\chi))^2\right] = \left[ \frac{\beta_{[1-2\ell]}}{\beta_{[1-\ell]}^2} \frac{M_2 \sigma_\varepsilon^2(\chi)}{M_1^2} + \frac{c \alpha_{[\ell]}^2}{\beta_{[1-\ell]}^2} \frac{\varphi'(0)^2 M_0^2}{M_1^2} \right ]$$

for bandwidths satisfying $nF(h_n)h_n^2 \to c$.

3. Strong Consistency and Convergence Rates

The estimator exhibits almost sure convergence with optimal rates under regularity conditions:

$$\limsup_n \left [\frac{n F(h_n)}{\ln \ln n} \right ]^{1/2} [r_n^{(\ell)}(\chi) - r(\chi)] = \frac{\left [2 \beta_{[1-2\ell]} \sigma_\varepsilon^2(\chi) M_2 \right ]^{1/2}}{\beta_{[1-\ell]} M_1} \quad \text{a.s.}$$

The deviation is of order $\sqrt{\ln \ln n / (n F(h_n))}$, with all constants directly determined by kernel choice, bandwidth schedule, and the error distribution.

4. Central Limit Theorem and Inference

A central limit theorem (CLT) enables inference with the recursive estimator:

$$\sqrt{n F(h_n)}(r_n^{(\ell)}(\chi) - r(\chi)) \xrightarrow{d} \mathcal{N}(\mu_n, \sigma^2)$$

where

• $$\mu_n = c \frac{\alpha_{[\ell]}}{\beta_{[1-\ell]}} \frac{M_0}{M_1} \varphi'(0)$$
• $$\sigma^2 = \frac{\beta_{[1-2\ell]}}{\beta_{[1-\ell]}^2} \frac{M_2}{M_1^2} \sigma_\varepsilon^2(\chi)$$

Tuning $h_n$ so that $h_n \sqrt{nF(h_n)} \to c$ ensures the properly scaled estimator converges in distribution, facilitating the construction of confidence intervals exploiting empirical estimates of all constituent constants.

5. Practical Implementation: Simulation and Real Data

Simulations and real data analyses demonstrate the method's computational and statistical properties:

• Simulation: Functional covariates constructed as, e.g., Brownian motions; regression operators defined by $\int_{0}^{1}\chi(s)^2 ds$; systematic exploration of kernel types, semi-norm definitions (PCA, Fourier, derivatives, partial least squares), and bandwidth schedules.
• Main empirical finding: Recursive estimators generally yield mean square prediction errors (MSPE) close to non-recursive counterparts but with substantially lower computational overhead, especially when data are sequentially updated. For new samples, the recursive estimator avoids $O(n)$ recomputation, leading to marked speedups.
• Real data: El Niño sea surface temperature curves—recursive kernel estimators provide predictions with empirically validated confidence intervals. Ozone pollution dataset—daily curves as predictors, sequential modeling, competitive error rates, and fast updates.

6. Extensions, Trade-Offs, and Application Guidance

The recursive kernel-centering framework is parameterized by $\ell$: $\ell=0$ corresponds to fully recursive estimators (maximum computational efficiency, possible minor statistical loss), while $\ell=1$ yields semi-recursive formulations (strictly maintains sequential updating). Trade-offs include:

• Slight inflation in prediction error compared to batch kernel regression, compensated by enormous savings in computational reprocessing.
• All tuning is dictated by balance of bias (via $h_n$) versus variance (via $nF(h_n)$), with closed-form guidance for constant estimation.

In all practical contexts requiring sequential estimation of regression (with functional covariates in infinite-dimensional spaces), recursive kernel-centering enables real-time learning, scaling to large datasets, and statistically principled inference.

7. Theoretical Significance and Broader Impact

Recursive kernel-centering extends classic Devroye–Wagner estimators to the functional data regime with rigorous asymptotics, attaining almost sure consistency, CLT-based inference, and precise bias/variance control via kernel and bandwidth constants. The method's capacity to update incrementally, maintain statistically optimal rates, and facilitate confidence statements directly addresses the needs of high-dimensional, real-time, and streaming analysis in modern statistical learning. Simulation and applied studies confirm its robustness and efficiency trade-offs, cementing its utility in functional regression, time series analysis, and real-data prediction with complex covariate structures.