Voronoi-type Loss Functions

• Voronoi-type loss functions are defined by partitioning space based on divergences like Bregman, enabling adaptive clustering and TV-regularized regression.
• They underpin methods such as the Voronoigram, achieving density-free and tuning-free estimation while matching discrete to continuum total variation.
• Efficient construction using dual transformations and half-space intersections makes these loss functions valuable for prototype-based clustering and nonparametric regression.

Voronoi-type loss functions are a class of optimization criteria arising from geometric partitioning of spaces according to proximity under specific divergences or variations. These loss functions underpin a variety of statistical learning, regression, and clustering methods, including Bregman Voronoi diagrams for prototype-based clustering and the Voronoigram for total variation-regularized regression. They leverage the geometric structure induced by Voronoi diagrams—partitions of a domain into regions of nearest proximity to a set of sites—generalized by metrics (e.g., Euclidean, Bregman divergence) or statistical regularizers (e.g., discrete total variation). This framework yields algorithms that are inherently adaptive to spatial distribution of data, and, in certain settings, achieve minimax-optimality without density-dependent tuning or neighborhood selection.

1. Geometric Foundations: Voronoi and Bregman Voronoi Diagrams

Classical Voronoi diagrams partition a domain $X\subset \mathbb{R}^d$ into cells $V_i$ consisting of all points closer to site $x_i$ than any other site, under the Euclidean metric. Formally,

$$V_i = \{ x \in X: \|x - x_i\| < \|x - x_j\|,~\forall j \neq i \}.$$

Voronoi-type loss functions arise when replacing the Euclidean distance with a more general divergence, notably the Bregman divergence. Let $F:X\rightarrow\mathbb{R}$ be strictly convex and differentiable; the Bregman divergence for $p,q\in X$ is

$$D_F(p\,\|\,q) = F(p) - F(q) - \langle \nabla F(q), p - q\rangle.$$

Given a set of sites $S=\{x_1,\ldots,x_n\}$, the associated loss functions $\ell_i(x) = D_F(x\,\|\,x_i)$, and the assignment via

$$V_i = \{ x \in X: D_F(x\,\|\,x_i) \le D_F(x\,\|\,x_j),~\forall j \neq i \}$$

define the first-type Bregman Voronoi diagram. Each cell is a convex polyhedron, as the bisectors $D_F(x\,\|\,x_i) = D_F(x\,\|\,x_j)$ induce affine hyperplanes, and explicit descriptions can be given in terms of gradients and the generator $F$ (0709.2196).

2. Voronoigram: Total Variation Regularization on Voronoi Graphs

The Voronoigram estimator arises in nonparametric regression for functions of bounded variation. Given noisy samples $y_i = f_0(x_i) + z_i$ at design points $x_i \in \mathbb{R}^d$, estimation is framed as penalized least squares with a discrete total variation (TV) penalty defined on the Voronoi tessellation. Restricting TV-regularized regression to functions constant on each $V_i$, the TV seminorm is discretized as

$$\text{TV}\left(\sum_i \theta_i\, 1_{V_i}\right) = \sum_{\{i,j\}\in E^V} w_{ij}^V\, |\theta_i - \theta_j|$$

where $E^V$ includes pairs of indices for which $V_i$ and $V_j$ share a $(d-1)$-dimensional boundary (positive Hausdorff measure), and $w_{ij}^V$ is the measure of the shared boundary. The Voronoigram solves

$$\hat{\theta} = \arg\min_{\theta\in\mathbb{R}^n} \frac{1}{n} \sum_{i=1}^n (y_i - \theta_i)^2 + \lambda \sum_{\{i,j\}\in E^V} w_{ij}^V |\theta_i - \theta_j|,$$

defining $\hat f(x) = \hat\theta_i$ for $x \in V_i$ (Hu et al., 2022). This estimand matches the (continuum) TV-regularized regression among piecewise constant functions over Voronoi cells.

3. Large-Sample Properties and Density-Freeness

One distinguishing property of Voronoi-type loss functions is the “density-free” asymptotic behavior. Under random design, with $x_i$ i.i.d. from a Lipschitz-continuous density $p$ bounded away from zero and infinity on $\Omega$, the discrete Voronoi TV converges in probability to a scaled version of the continuum total variation:

$$\sum_{\{i,j\}\in E^V} w_{ij}^V |f(x_i) - f(x_j)| \xrightarrow{p} c_d \int_\Omega \|\nabla f(x)\|_2\,dx,$$

where $c_d$ is a dimension-dependent constant. This asymptotic does not involve $p$, unlike classical graph-TV schemes (e.g., $\varepsilon$-NN or $k$-NN graphs), whose empirical TV limits inherit density weights and hence lack invariance under non-uniform sampling. This density-independence ensures unbiased regularization in the large-sample regime (Hu et al., 2022).

4. Algorithmic Construction and Combinatorial Complexity

Bregman Voronoi diagrams and their induced loss landscapes admit efficient construction through two principal methods: half-space intersection and reduction to power diagrams. For half-space intersection, each site $x_i$ defines a tangent hyperplane to $z = F(x)$ at $(x_i, F(x_i))$, and the associated Voronoi cell results from projecting the intersection of half-spaces onto $X$. Alternatively, in the dual (gradient) coordinates after the Legendre transform $F^*$, the power diagram of points $c_i = \nabla F(x_i)$ with weights $w_i$ yields the Voronoi segmentation, matching the combinatorial cell structure in $x$-space. Both approaches require $O(n \log n + n^{\lfloor (d+1)/2 \rfloor})$ time (fixed $d$) (0709.2196).

For the Voronoigram, efficient computation entails constructing the Voronoi tessellation of the points and computing the measures of shared boundaries. These edge weights—surface area in $d=3$, length in $d=2$—constitute the adjacency graph for the TV penalty in the optimization.

5. Extensions: Higher-Order and Composite Divergence Losses

Voronoi-type loss partitions generalize to $k$-order and $k$-bag diagrams. The $k$-order Bregman diagram assigns each point to the $k$ nearest sites under $D_F$, with cells again characterized by power-diagram structures via specifically constructed centers and weights:

$$c_T = \frac{1}{k} \sum_{j\in T} \nabla F(x_j), \quad w_T = F(c_T) - \langle c_T, \nabla F(c_T)\rangle - \frac{1}{k}\sum_{j\in T}[F(x_j)-\langle x_j,\nabla F(x_j)\rangle].$$

Composite “$k$-bag” diagrams encode mixtures of divergences, defined by convex generators $F_\ell$ and site-dependent mixtures ($\alpha^{(i)}$), leading to partitions in lifted spaces of dimension $d+k$. In both generalizations, the combinatorial and computational properties parallel those of the base diagrams, with complexity scaling as $O(n^{\lfloor (d+k)/2 \rfloor})$ for $k$-bag diagrams (0709.2196).

Bregman triangulations—the analogues of Delaunay triangulations—arise via lower convex hulls of lifted points $(x_i, F(x_i))$ or, equivalently, as geodesic triangulations in the dual space. These structures generalize classical spatial interpolation and nearest-neighbor constructs, admitting rich geometric combinatorics.

6. Statistical and Practical Properties

The Voronoigram provides a conceptually clean, tuning-free, density-free discretization of the continuum total variation functional. Its fitted function admits one-to-one correspondence between discrete and continuum TV, thereby preserving complexity under extrapolation: the number of constant regions in the fitted $f^V$ matches the number of connected components in the Voronoi graph. Importantly, the Voronoigram, with properly chosen regularization parameter $$\lambda\asymp\sigma n^{(d-1)/d}(\log n)^{1/2+\alpha}$$, achieves minimax risk up to logarithmic factors for bounded-variation regression:

$$\mathbb{E}\|f^V - f_0\|_{L^2(P)}^2 \leq C n^{-1/d} (\log n)^{O(1)}.$$

Comparable rates are attainable for other graph-TV and wavelet schemes, but only the Voronoigram achieves this rate without auxiliary neighborhood tuning (e.g., $\varepsilon$, $k$), and with exact preservation of TV under 1-NN extrapolation (Hu et al., 2022).

In first-type Bregman Voronoi diagrams, the minimum-loss assignment $\ell(x) = \min_i D_F(x\,\|\,x_i)$ serves as a natural prototype-based clustering rule, with polyhedral cell structure for computational tractability. The Legendre dual structure further enables handling of curved bisectors and symmetry in statistical learning applications (0709.2196).

7. Comparison and Context

The table below summarizes core distinctions among Voronoi-type loss functions in TV-regularized regression:

Method	Density Dependence	Tuning Parameter	TV-Extrapolation Preservation
Voronoigram	Density-free	None	Exact
$\varepsilon$-NN TV	Density-weighted	$\varepsilon$	No
$k$-NN TV	Density-weighted	$k$	No

The Voronoigram is uniquely tuning-free and density-free, with extrapolation matching the discrete TV structure precisely. In contrast, classical graph-TV regularizers require explicit parameter selection and yield continuum limits biased by sampling density.

Voronoi-type loss function frameworks—spanning Bregman divergence–based partitioning and TV-regularization on Voronoi graphs—offer principled, efficient, and statistically robust methodologies for loss landscape construction, spatial regularization, and high-dimensional function estimation (0709.2196, Hu et al., 2022).