WaveMesh: Adaptive Wavelet Methods

• WaveMesh is a multiscale framework that applies wavelet transforms to enable nonparametric regression with irregularly spaced data and content-adaptive image segmentation.
• It utilizes interpolation operators and accelerated proximal gradient descent to achieve near-minimax theoretical guarantees for both univariate and high-dimensional additive models.
• The image processing variant generates adaptive superpixels and integrates with graph neural networks, improving classification performance on benchmarks like MNIST and CIFAR-10.

WaveMesh denotes two prominent, technically distinct methodologies leveraging wavelet transforms to enable either (1) nonparametric regression with irregularly spaced data or (2) content-adaptive superpixel graph construction for image processing. Each instantiation, though unrelated in origin, is unified by its use of multiscale wavelet analysis and mesh-based data partitioning. The following sections systematically address both forms, their core algorithms, theoretical properties, computational workflows, and empirical findings as established by Haris, Simon, and Shojaie (Haris et al., 2019), and Mialon, Gallinari, and Laurens (Vasudevan et al., 2022).

1. WaveMesh in Nonparametric Regression

Construction and Problem Setting

WaveMesh, as formulated in (Haris et al., 2019), addresses nonparametric regression of the form $y_i = f^0(x_i) + \epsilon_i$ for arbitrarily positioned covariate points $x_i \in [0,1]$ where $n$ is not necessarily a power of 2. Classical discrete wavelet transforms (DWTs) are inapplicable because they require regular input grids. WaveMesh circumvents this constraint by introducing an interpolation operator $R \in \mathbb{R}^{n \times K}$ mapping values of a function $f$ defined on a regular grid $\{1/K, 2/K, \dots, 1\}$ to its values at the observed $x_i$. Given an orthonormal DWT matrix $W \in \mathbb{R}^{K \times K}$ and wavelet coefficients $d$, fitted values at sample locations are expressed as

$\hat{y} = R W^\top d.$

The $R$ matrix is generally constructed via linear interpolation with special boundary rules.

Penalized Wavelet Domain Estimation

Estimation proceeds by penalized least squares in the wavelet domain. In the univariate case, the estimator solves

$$\min_{d \in \mathbb{R}^K} \frac{1}{2}\|y - R W^\top d\|_2^2 + \lambda \|d_{-1}\|_1$$

where $d_{-1}$ encodes the mother wavelet coefficients (i.e., all but the coarsest scaling coefficient). Weighted $\ell_1$ penalties can be substituted to yield adaptive minimax rates analogous to SURESHRINK [(Haris et al., 2019), Eq. (4)].

Additive and Sparse Additive Extensions

For high-dimensional input, WaveMesh generalizes to additive models by fitting $p$ univariate wavelet components. The sparse additive variant optimizes

$$\min_{d_j} \frac{1}{2}\left\|y - \sum_j R_j W^\top d_j \right\|_2^2 + \sum_{j=1}^p [\lambda_1 P_s(d_j) + \lambda_2 \|R_j W^\top d_j\|_2],$$

where $P_s$ denotes a discrete Besov norm penalty. Theoretical guarantees for the sparse case depend on a weak compatibility condition, yielding minimax-optimal rates for small active sets [(Haris et al., 2019), Theorem 2].

2. Proximal Gradient Descent and Algorithmic Framework

WaveMesh employs accelerated proximal gradient descent for efficient optimization. For the loss $\ell(d) = \frac{1}{2}\|y - R W^\top d\|_2^2$, the gradient is $\nabla \ell(d) = W R^\top (R W^\top d - y)$. Each iteration consists of

$$d^{(\ell+1)} = \mathrm{prox}_{t \lambda \|\cdot\|_1}(d^{(\ell)} - t \nabla \ell(d^{(\ell)})),$$

where the proximal operator (soft-thresholding) is applied solely to $d_{-1}$. Convergence is $O(1/\ell)$, improving to $O(1/\ell^2)$ with Nesterov acceleration. Block coordinate descent enables tractable extension to additive and sparse additive multivariate settings, with per-iteration cost $O(p K)$ for $p$ covariates (Haris et al., 2019).

3. Theoretical Properties

WaveMesh achieves adaptive minimax convergence rates for functions in Besov spaces $B^s_{q_1,q_2}$. For suitable grid size $K \gtrsim n^{1/(2s+1)}$ and penalty parameter $\lambda \gtrsim \sqrt{\log K}$, the estimator satisfies

$$n^{-1} \| f^0(x_{1:n}) - \hat{y} \|_2^2 \leq C ( \log K / n )^{2s/(2s+1)} + (2/n) \| f^0(x_{1:n}) - R f^0(1:K)/K \|_2^2,$$

with the second term quantifying interpolation bias, order $O(K^{-2s})$ for $s$-smooth $f$.

For sparse additive models, the excess risk is bounded by

$$n^{-1} \| f^0 - \hat{y} \|_2^2 \leq C \max \left\{ |S^*| n^{-2s/(2s+1)}, |S^*| \frac{\log p}{n} \right\} + \text{approximation error},$$

where $|S^*|$ is the true active set size. When a (Besov) compatibility condition holds, the faster $|S^*| n^{-2s/(2s+1)}$ rate obtains [(Haris et al., 2019), Theorems 1–2].

4. Empirical Evaluations and Practical Implementation

Empirical results demonstrate superior or competitive MSE relative to interpolation-based, isometric wavelet, and adaptive lifting methods on simulated and real univariate regression tasks, with advantages most pronounced for irregular designs. Sparse additive WaveMesh considerably outperforms AMlet [Sardy] in multivariate simulations and real-world datasets, such as UCI Boston Housing, demonstrates substantially lower test MSE (e.g., CV MSE 21.2 vs. 25.1).

Recommended grid size is $K = 2^{\lceil \log_2 n \rceil}$ or smaller for computational savings if $f$ is smooth. Cross-validation is standard for $\lambda$ selection, though a universal threshold $\lambda \approx \sigma \sqrt{2 \log K}$ yields nearly minimax results. Software is available as the R package "waveMesh," which provides the DWT/IDWT, interpolation, and the full solver (Haris et al., 2019).

5. WaveMesh in Image-based Graph Construction

Superpixel Mesh Generation

A second, unrelated WaveMesh variant defines a multiscale superpixel algorithm for images (Vasudevan et al., 2022). Starting from an $N \times N$ image, Mallat’s 2D multiresolution analysis (MRA) up to $S = \log_2 N$ is applied using Haar wavelets. Wavelet coefficients are thresholded via the Donoho–Johnstone universal rule to select energetic modes, yielding a binary mask of significant locations.

Coefficients are arranged in a depth-$S$ quadtree where each node covers a $2^s \times 2^s$ block. Nodes are recursively split if any of their children or themselves exhibit strong wavelet energy, resulting in adaptive, content-reflective superpixels (the "WaveMesh"). This mesh partitions the image into nonuniform regions whose number and size automatically adapt to local signal complexity.

Graph Structure and Feature Representation

Each superpixel is mapped to a node in a region adjacency graph (RAG), its feature vector being the mean color/intensity, and the node coordinates the centroids of its pixels. Edges exist between adjacent superpixels. The system encodes edge pseudo-coordinates for compatibility with SplineCNN's spatial convolution, normalized by the maximum observed edge length.

6. GNN Integration, Pooling Schemes, and Experimental Results

Classification proceeds by mapping the RAG to a graph neural network, specifically SplineCNN, which generalizes classical convolution via B-spline kernels over edge pseudo-coordinates. Two backbone architectures are implemented, both employing alternations of Spline conv, pooling (see below), global mean pooling, and fully-connected layers.

Pooling schemes compared include:

• GraclusPool: Kernel clustering to coarsen the node graph using pairwise node matching.
• WavePool: A bespoke pooling that merges four sibling leaves in the WaveMesh quadtree to their parent node, preserving the multiscale structure. On a uniform grid, this replicates $2\times2$ max-pooling.

Empirical evaluation on MNIST, Fashion-MNIST, and CIFAR-10 benchmarks yielded the following test accuracies (mean ± std, %, five runs):

Dataset	WM+NP	WM+GR	WM+WP	SL+NP	SL+GR
MNIST	92.60±0.45	89.63±0.45	95.44±0.12	95.99±0.07	95.51±0.29
FashionMNIST	79.60±0.44	65.35±2.94	76.60±0.83	82.37±0.25	81.49±0.38
CIFAR-10	50.59±0.17	43.36±0.72	52.58±0.21	47.25±0.21	45.87±0.28

NP = no pooling; GR = GraclusPool; WP = WavePool; WM = WaveMesh; SL = SLIC.

Notable conclusions include the consistent degradation of accuracy with GraclusPool on WaveMesh graphs (often –3 to –14 percentage points), and the recovery/improvement of accuracy with WavePool. WaveMesh combined with WavePool matches or exceeds standard SLIC superpixel graphs with their preferred clustering, despite using adaptive superpixel counts. SLIC has marginal advantage with no pooling on MNIST/Fashion-MNIST, but WaveMesh closes this gap with appropriate pooling and outperforms on CIFAR-10.

7. Interpretation and Significance

WaveMesh, in both its regression and graph-based image construction forms, exemplifies content-adaptive multiscale modeling powered by wavelet analysis. In nonparametric regression, it provides the first method to combine wavelet sparsity with irregular input designs at (near-)minimax theoretical optimality over Besov classes, scalable to additive and high-dimensional settings via efficient proximal optimization (Haris et al., 2019). In image processing, WaveMesh offers a principled approach to generating superpixels that respect the multi-resolution content of the image, aligning with widely recognized statistical thresholds (Donoho–Johnstone), and integrates seamlessly into graph-based deep learning pipelines. The compatibility of WaveMesh superpixels with the WavePool operator demonstrates the value of structurally consistent coarsening operations, which is substantiated by superior empirical results on heterogeneous benchmarks (Vasudevan et al., 2022).

A plausible implication is that, across modalities, respecting multiscale structure and spatial hierarchies during both representation and coarsening is crucial for maximizing the utility of wavelet-based discretization in predictive modeling.