Piecewise Local Polynomial Estimator

• Piecewise Local Polynomial Estimator is a nonparametric method that divides the function’s domain into dyadic rectangles and fits local anisotropic polynomials for precise approximation.
• It employs recursive partitioning and penalized least-squares selection to adapt to spatial inhomogeneity and directional smoothness, ensuring near-optimal estimation performance.
• The technique is computationally efficient and widely applicable to high-dimensional density estimation, adaptive regression, and image or signal processing tasks.

A piecewise local polynomial estimator is a function estimation technique that partitions the domain of a multivariate function into disjoint regions—most commonly dyadic rectangles—and approximates the function on each region by a (possibly anisotropic) polynomial whose degree and support can adapt locally. This approach enables simultaneous adaptation to spatial inhomogeneity (variations in local smoothness) and anisotropy (direction-dependent smoothness), which is of central significance in nonparametric function and density estimation, especially in high dimensions. The estimator is constructed through a recursively designed model selection procedure, combining nonlinear approximation theory with penalized least-squares strategies, while maintaining computational feasibility suitable for large sample sizes.

1. Piecewise Polynomial Construction and Selection

The estimator is built on a partition $m$ of the unit cube $[0,1]^d$ into dyadic rectangles—that is, hyperrectangles whose sides are dyadic intervals aligned with the axes. For each rectangle $K$ in the partition $m$, a polynomial of coordinate-wise degrees not exceeding a vector $\mathbf{r} = (r_1, ..., r_d)$ is fitted: $$S_{(m, \mathbf{r})} = \left\{ f: [0,1]^d \to \mathbb{R} : \forall K \in m,\, f|_K \text{ is a tensor product polynomial with degrees } \leq r_l \text{ in } x_l \right\}.$$

The key steps are:

• Recursive Partitioning: Starting from the full domain, regions are split recursively along coordinate axes whenever the fit of the best local polynomial on a region fails to meet a prescribed approximation threshold. This threshold can itself be adapted according to local error estimates.
• Local Polynomial Fitting: Within each region, the best polynomial (in an $L_q$ norm) of prescribed degrees is computed, minimizing the local approximation error

$$\mathcal{E}_\mathbf{r}(s, K)_q := \inf_{P \in \mathscr{P}_\mathbf{r}} \|s - P\|_{L_q(K)}.$$

• Model Selection: Among all pairs (partition, degree sequence), selection is performed by minimizing a penalized empirical least-squares criterion: $$(\hat{m}, \hat{\boldsymbol{\rho}}) = \arg\min_{(m, \boldsymbol{\rho})} \left\{ \gamma(\hat{s}_{(m, \boldsymbol{\rho})}) + \text{pen}(m, \boldsymbol{\rho}) \right\},$$ where the penalty term controls model complexity and adaptivity.

2. Adaptation to Anisotropy and Inhomogeneity

The estimator is designed to be minimax-optimal over classes of functions with anisotropic and/or inhomogeneous smoothness. For a direction-dependent regularity vector $\boldsymbol{\sigma} = (\sigma_1, ..., \sigma_d)$, function classes are defined using

$$N_{\mathbf{r}, \boldsymbol{\sigma}, p, p'}(s) = \begin{cases} \left( \sum_{k=0}^{\infty} \left[2^{k\underline{\boldsymbol{\sigma}}} e_{\mathbf{r}, \boldsymbol{\sigma}, p, k}(s)\right]^{p'} \right)^{1/p'} & 0 < p' < \infty \ \sup_{k} 2^{k\underline{\boldsymbol{\sigma}}} e_{\mathbf{r}, \boldsymbol{\sigma}, p, k}(s) & p' = \infty \end{cases}$$

where $e_{\mathbf{r}, \boldsymbol{\sigma}, p, k}(s)$ is the best $L_p$-approximation on locally dyadic partitions, and $\underline{\boldsymbol{\sigma}}$ involves the harmonic mean of the local smoothness parameters.

Adaptation is achieved via:

• Local Model Complexity: Degrees of polynomials and refinement of partitions are optimized locally, allowing the estimator to adapt to varying smoothness both between coordinates and across space.
• Penalized Selection: The penalization scheme is proven to select models near-optimal for any given smoothness configuration, supporting broad adaptation.

3. Approximation Rates and Minimax Theory

The estimator attains optimal rates in the minimax sense over these complex smoothness classes. The rate for a function class with regularity vector $\boldsymbol{\sigma}$ and partition cardinality $D$ is

$$\sup_{s \in \mathcal{S}(\mathbf{r}, \boldsymbol{\sigma}, p, p', R)} \inf_{t \in \cup_{m \in \mathcal{M}_D} S_{(m, \mathbf{r})}} \| s - t \|_q \leq C R D^{-H(\boldsymbol{\sigma})/d}$$

where $H(\boldsymbol{\sigma})$ is the harmonic mean of the regularities, expressing the effective smoothness faced in multidimensional approximation.

Furthermore, in the statistical estimation setting (e.g., multivariate density estimation), the estimator achieves minimax $L_2$-rates for anisotropic Besov classes,

$$\inf_{\hat{s}} \sup_{s \in \mathcal{P}(\boldsymbol{\sigma}, p, p', R, L)} E_s[\|s - \hat{s}\|_2^2] \sim (R n^{-H(\boldsymbol{\sigma})/d})^{\frac{2d}{d + 2 H(\boldsymbol{\sigma})}},$$

with no logarithmic loss for fixed-degree polynomials, and only a logarithmic factor in the case of growing degrees.

4. Computational Scalability

A notable contribution of this estimator is its computational efficiency. The construction leverages the hierarchical (tree) structure of dyadic partitions, allowing:

• Dynamic Programming for optimal model selection,
• Total computational complexity $\mathcal{O}(n)$ for fixed polynomial degree ($r_* = O(1)$), or $\mathcal{O}(n \log^d n)$ if the degree increases logarithmically with sample size, where $n$ is the sample size.

The estimator's representation for each rectangle $K$ is

$$\hat{s}_{(m, \boldsymbol{\rho})} = \sum_{K \in m} \sum_{\mathbf{k} \in \Lambda(\boldsymbol{\rho}_K)} \left( \frac{1}{n} \sum_{i=1}^n \Phi_{K, \mathbf{k}}(Y_i) \right) \Phi_{K, \mathbf{k}},$$

where $\Phi_{K, \mathbf{k}}$ are local, orthonormal basis polynomials.

5. Statistical and Applied Implications

Applications include:

• High-Dimensional Multivariate Density Estimation: The estimator is capable of fitting multivariate densities that are simultaneously inhomogeneous (varying smoothness over space) and anisotropic (differing smoothness in each direction), matching statistical theory benchmarks for a wide range of regularity classes.
• Adaptive Multivariate Regression and Smoothing: The techniques generalize to nonparametric regression and other multivariate function estimation settings.
• Image and Signal Processing: The flexibility to align with local orientations and smoothness makes the technique suitable for adaptive smoothing in images or signals.

The statistical theory clarifies the scope of achievable adaptivity and computational feasibility in nonparametric estimation—previously plagued by intractability in high dimension with complex smoothness.

6. Penalization and Model Complexity Control

The model selection penalty is carefully constructed to account for variance, local complexity, and partition size. For instance, a representative penalty form is: $$\mathrm{pen}(m, \boldsymbol{\rho}) = \frac{1}{n} \sum_{K \in m} \sum_{\mathbf{k} \in \Lambda(\boldsymbol{\rho}_K)} (\kappa_1 \hat{\sigma}^2_{K, \mathbf{k}} + \kappa_2 \pi(\mathbf{k})) + \left( \dots \right) \frac{\log(\dots)|m|}{n}$$ where terms account for empirical variance, basis size, and partition cardinality.

This penalized selection is central to ensuring both adaptivity and oracle inequalities of the form

$$E_s \left[\| s - \tilde{s} \|_q^q\right] \leq C \inf_{m \in \mathcal{M}} \left\{ \inf_{t \in S_m} \|s - t\|_q^q + (\dim(S_m)/n)^{q/2} \right\},$$

ensuring the estimator's risk mimics the best possible choice within the considered model class.

---

Aspect	Key Points
Construction	Dyadic rectangular partitions, direction-dependent polynomials, adaptive refinement
Adaptation	Accommodates both anisotropic (directional) and inhomogeneous (spatial) smoothness
Approximation Rate	Minimax-optimal over broad function classes (incl. anisotropic Besov)
Computational Complexity	Linear in $n$ for fixed degree; log-factor for adaptive degrees
Applications	High-dimensional density estimation, adaptive smoothing, multivariate regression

---

In summary, the piecewise local polynomial estimator described in this work delivers theoretically optimal, locally and directionally adaptive approximation and estimation for multivariate functions, including densities, in high-dimensional and complex smoothness scenarios, with guarantees on statistical risk and computational tractability. Its foundation in nonlinear approximation over dyadic rectangles, coupled with penalized selection, provides a unifying and scalable solution for sophisticated nonparametric estimation tasks.