Kernel Estimators for Functional Data

• Kernel estimators for functional data are nonparametric methods that smooth curves using a semi-metric and one-sided kernels to estimate means, covariances, and regressions.
• Bias reduction through linear extrapolation and adaptive bandwidth selection improves accuracy, balancing the O(h) bias and variance in infinite-dimensional spaces.
• Extensions include operator-valued regression, spectral density estimation for time series, and efficient online updating for streaming functional data.

Kernel estimators for functional data are a central class of nonparametric methods for estimating mean, covariance, and regression operators, as well as conditional distributions and quantiles, when the observed variables are curves or elements of a (separable) function space rather than finite-dimensional vectors. Key developments span mean and covariance function estimation, regression (including nonlinear operator and function-on-function), quantile estimation, bias reduction, online updating, and bandwidth selection. This entry provides a comprehensive technical overview of the methodology and theory, referencing foundational and recent research.

1. Foundations and General Formulation

Kernel estimation in the functional context generalizes the Nadaraya–Watson paradigm by using a smoothing kernel and a semi-metric or norm to measure proximity in a functional space. For i.i.d. samples $(Y_i,\mathcal X_i)$ with $\mathcal X_i$ in a separable Banach or Hilbert space $\mathcal E$, the core estimator of the regression operator $r(\chi) = E[Y \mid \mathcal X = \chi]$ is

$$\widehat r(\chi) = \frac{\sum_{i=1}^n Y_i\,K\big(h^{-1}\|\mathcal X_i - \chi\|\big)}{\sum_{i=1}^n K\big(h^{-1}\|\mathcal X_i - \chi\|\big)}$$

where $K:[0,1]\to\mathbb R^+$ is a (typically one-sided) kernel, and $h>0$ is a bandwidth parameter (Birke et al., 20 Nov 2025). The same framework underpins estimation of conditional densities, distributions, and quantiles via appropriate choices of the response transformation $\Phi(Y)$. The estimator reduces to a weighted local average, with weights determined by proximity in the chosen (semi-)metric.

This general structure extends to time series data, where estimators for the lag-$\ell$ covariance kernels, long-run covariance, and spectral density are similarly defined through kernel smoothing over time lags and across functional domains (Berkes et al., 2015, Zhu et al., 2018).

2. Bias, Variance, and Bias Reduction

In functional kernel regression, the leading order of bias is $O(h)$, contrasting with the $O(h^2)$ rate for symmetric kernels in multivariate regression, due to the necessity of one-sided kernels when only small-ball probabilities are available. The bias is: $$E\big[\widehat m_\Phi(\chi)\bigr] - m_\Phi(\chi) = \varphi_{\Phi,\chi}'(0)\,\frac{M_{0,\chi}}{M_{1,\chi}}h + o(h)$$ where the constants $M_{0,\chi}$, $M_{1,\chi}$ involve moments of $K$ and local geometric features of $\mathcal X$ (Birke et al., 20 Nov 2025). The variance is of order $O((nL_\chi(h))^{-1})$ with $L_\chi(h)$ the small-ball probability.

Bias cannot be controlled by increasing kernel smoothness—since, with one-sided kernels, the first moment does not vanish—nor by assuming higher-order derivatives exist for $r(\cdot)$. Linear-extrapolation, as in [Cheng2018] and adapted to functional data, forms an affine combination of estimates at different bandwidths to annihilate the $O(h)$ term: $$\widehat m_{\Phi,B}(\chi) = \sum_{i=1}^B g_i\,\widehat m_{\Phi,h_i}(\chi), \;\;\; \text{with } \sum g_i=1, \;\sum g_i h_i = 0,$$ yielding an $O(h^2)$ bias rate while keeping the variance order unchanged (Birke et al., 20 Nov 2025).

The bias-variance trade-off underlies bandwidth selection, with optimal $h$ balancing $O(h^2)$ bias (after correction) and $O((nL_\chi(h))^{-1})$ variance. In finite samples, bias-corrected estimators exhibit notably reduced MSE versus classical kernel estimators.

3. Covariance and Spectral Density Estimation

Estimation of second-order structure—long-run covariance, autocovariance, and spectral density kernels—is central in functional time series analysis. With strictly stationary $\{X_i\}_{i\in\mathbb Z}$ and lag-$\ell$ covariance kernel $\gamma_\ell(t,s)$, the long-run covariance kernel is

$$C(t,s) = \sum_{\ell=-\infty}^\infty \gamma_\ell(t,s),$$

estimated by

$$\hat C_N(t,s) = \sum_{|\ell|<b_N} K\big(\tfrac{\ell}{b_N}\big)\, \hat \gamma_\ell(t,s),$$

where $\hat \gamma_\ell$ is the empirical lag-$\ell$ covariance and $K$ a symmetric, compactly supported "lag window" (Berkes et al., 2015).

Under Bernoulli-shift and weak dependence, asymptotic $L^2$ normality holds: $$\Big\|(N/b_N)^{1/2}(\hat C_N - E\hat C_N) - I_N\Big\| \xrightarrow{p} 0,$$ $I_N$ being a zero-mean Gaussian process with explicit covariance.

Bias expansion reveals that if $K$ has flatness of order $q$, then

$$E\hat C_N(t,s) - C(t,s) = b_N^{-q}\,\mathfrak{c}_q(t,s) + o(b_N^{-q}),$$

with optimal bandwidth order $b_N \asymp N^{1/(1+2q)}$.

For frequency-domain inference, estimators of the spectral density kernel are obtained by smoothing periodograms with flat-top kernels. If $\lambda(\theta)$ is flat up to order $p$, the estimator achieves bias $O(b^p)$ and IMSE $O(N^{-2p/(2p+1)})$, at the cost that the estimate may not be positive semi-definite; projection onto the psd cone resolves this without changing asymptotic accuracy (Zhu et al., 2018).

4. Regression and Operator-Valued Kernel Estimators

Kernel methods for function-valued outputs and inputs are formalized via operator-valued kernels in an RKHS framework (Kadri et al., 2015). For data $\{(x_i, y_i)\}_{i=1}^n$ with $x_i \in \mathcal X$ and $y_i \in \mathcal Y$ (both possibly functional spaces), the solution to

$$\min_{F\in\mathscr F} \sum_{i=1}^n \|y_i - F(x_i)\|_\mathcal Y^2 + \lambda \|F\|_{\mathscr F}^2$$

is

$$F_\lambda(\cdot) = \sum_{i=1}^n K(x_i, \cdot)\,u_i, \quad u_i \in \mathcal Y,$$

where $$K: \mathcal X \times \mathcal X \to \mathcal L(\mathcal Y)$$ is a positive-definite, operator-valued kernel.

Examples include separable ("Kron") kernels $K(x, x') = g(x, x')T$ with $g$ a scalar kernel and $T$ a positive operator on $\mathcal Y$; and integral operator kernels that encode smoothness in the output domain. Computationally, systems involving block-operator matrices are handled using spectral or iterative methods (Kadri et al., 2015).

Function-on-scalar kernel regression unifies this theory with multitask learning. Using a separable kernel $K((x,t),(x',t')) = k_x(x,x') k_t(t,t')$, the estimator has

$$\hat f(x,t) = \sum_{i=1}^N \sum_{j=1}^L \theta_{ij} k_x(x,x_i) k_t(t,t_j)$$

and admits closed-form (ridge) or Bayesian (GP) solutions; estimation leverages joint regularization in both input and output domains (Kusaba et al., 17 Mar 2025).

5. Adaptive, Online, and High-Order Extensions

Efficient online updating for kernel estimators under streaming functional data is enabled by decomposing estimates into sufficient statistics for multiple candidate bandwidths and merging these adaptively across data blocks. At each block, dynamically updated pseudo-sufficient statistics and a sequence of candidate bandwidths $\{\eta_{\mu,\ell}^{(k)}\}$ (e.g. $$\eta_{\mu,\ell}^{(k)} = ((L-\ell+1)/L)^{1/5}\widetilde h_\mu^{(k)}$$) allow near-batch-optimal accuracy while memory and computation scale with the sequence length $L$ rather than the number of blocks (Yang et al., 2021). Relative efficiency typically exceeds $95\%$ even with $L=5$.

In the functional time series setting, kernel-based autoregression and corresponding bootstrap procedures are established for estimating regression operators, with uniform consistency and asymptotic normality. Bootstrap consistency holds under $m$-dependent approximations and small-ball geometric conditions, permitting construction of confidence regions for functional predictions (Krebs et al., 2018).

Moreover, higher-order accurate estimators—via flat-top smoothing in time or bandwidth extrapolation in the functional domain—yield improved bias rates, though often at the expense of positive-definiteness or increased variance. In practice, positive definiteness can be enforced via spectral truncation, and variance inflation from bias correction is typically outweighed by large bias reductions (Zhu et al., 2018, Birke et al., 20 Nov 2025).

6. Practical Guidance: Kernel/Metric/Bandwidth Choice and Applications

Kernel and metric choice are critical in functional kernel estimation. One-sided, compactly supported kernels are standard for local averages, while second-order symmetric kernels are used in the response $Y$ direction for density estimation. Semi-metrics $d(\cdot,\cdot)$ are generally based on the $L^2$ norm, derivatives, or FPCA projections; metric selection may be formalized and chosen by maximized marginal likelihood in a Bayesian framework (Shang, 2020).

Bandwidth selection may use cross-validation, plug-in, or (for covariance/spectral estimation) empirical rules derived from autocorrelation decay or small-ball probabilities. For bias-corrected estimators, a pilot bandwidth is estimated by cross-validation, and extrapolation weights are computed over $B \geq 15$ nearby bandwidths to minimize finite-sample MSE and confidence region width (Birke et al., 20 Nov 2025).

Applications range from functional principal component analysis (FPCA) with independent Gaussian asymptotics (Berkes et al., 2015), online mean/covariance estimation for streaming data (Yang et al., 2021), conditional quantile estimation under functional covariates (Gardes et al., 2012, Gardes et al., 2011), operator regression for function-valued responses (Kadri et al., 2015), to Bayesian kernel regression in high-dimensional nonlinear settings (Kusaba et al., 17 Mar 2025). Simulation and real-data studies confirm the pronounced improvements from bias correction, metric selection, and multitask inference.

7. Extensions: Extreme Value Theory and Quantile Estimation

Functional kernel estimators have been extended to conditional extreme quantiles, where the target quantile order $\alpha_n \downarrow 0$ as $n \to \infty$. Under regularly varying conditional tails and small-ball conditions, the estimator

$$\hat{q}_n(\alpha \mid x) = \inf\{ t : \widehat{\bar{F}}_n(t \mid x) \le \alpha \}$$

exhibits asymptotic normality at rate $(n \mu_x^{(K)}(h_n) \alpha_n)^{-1/2}$, with variance driven by the local tail index $\gamma(x)$. For extrapolation to rarer quantiles, a Weissman-type formula is used, substituting a kernel-based estimator for $\gamma(x)$; kernelized versions of the Hill and Pickands tail-index estimators are provided (Gardes et al., 2012, Gardes et al., 2011). Finite-sample results underline the necessity of careful semi-metric selection and regularization.

---

Overall, kernel estimators for functional data represent a highly flexible, theoretically rigorous class of tools for mean, covariance, regression, distribution, and quantile estimation in infinite-dimensional spaces, with a substantial body of work establishing their properties, extensions, and optimal tuning strategies across iid, time series, streaming, and extreme-value regimes (Berkes et al., 2015, Shang, 2020, Linke et al., 2022, Kusaba et al., 17 Mar 2025, Krebs et al., 2018, Birke et al., 20 Nov 2025, Zhu et al., 2018, Kadri et al., 2015, Gardes et al., 2012, Gardes et al., 2011, Yang et al., 2021).