Error Distribution Smoothing (EDS)

• Error Distribution Smoothing (EDS) is a technique for addressing imbalanced low-dimensional regression by quantifying both data density and function complexity.
• It partitions the feature space into simplices and uses a complexity-to-density ratio to identify regions with high prediction errors.
• EDS enhances dataset efficiency by selecting representative subsets through dynamic Delaunay triangulation, reducing training time and worst-case error.

Error Distribution Smoothing (EDS) is a methodology for addressing imbalanced regression in low-dimensional settings, where data are unevenly distributed across regions of varying functional complexity. Unlike conventional class imbalance frameworks, EDS specifically targets the challenges of regression tasks by introducing quantitative measures of both data density and underlying function complexity, and by devising algorithms to construct representative data subsets that balance predictive capacity and sample efficiency (Chen et al., 4 Feb 2025).

1. Imbalanced Regression and the Complexity-to-Density Ratio

Imbalanced regression is characterized by datasets $D = \{(x_i, y_i)\}_{i=1}^N$ in $\mathbb{R}^n \times \mathbb{R}^m$ with regions of both sparse sampling (frequently corresponding to high-complexity underlying functions) and dense, often redundant sampling (low-complexity regions). Traditional density-based imbalance metrics are insufficient because high-complexity regions require proportionally more data for equivalently low error, rendering simple density count inadequate.

To quantify this, EDS partitions the feature space into $k$ non-overlapping simplices $\mathcal{F} = \{\Omega_j\}_{j=1}^k$. Each region $\Omega$ is analyzed for:

• Region size: $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$
• Region complexity: $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$ (Frobenius norm of the Hessian of the regression function $f$)
• Sample count: $|\Omega \cap D|$

The complexity-to-density ratio (CDR) is then

$$\rho(\Omega, D) = \frac{g_c(\Omega)\, g_s(\Omega)}{|\Omega \cap D|}$$

This measure reflects the interplay between target function curvature/complexity and local data support.

Log-CRD values across regions are modeled as a Gaussian $\mathbb{R}^n \times \mathbb{R}^m$0 with

$\mathbb{R}^n \times \mathbb{R}^m$1

$\mathbb{R}^n \times \mathbb{R}^m$2

The pair $\mathbb{R}^n \times \mathbb{R}^m$3 provides a Global Imbalance Metric (GIM), where large $\mathbb{R}^n \times \mathbb{R}^m$4 signals severe imbalance.

2. Error Distribution Smoothing: Rationale and Error Bounds

In sparse or complex regions (high CDR), prediction error bounds are intrinsically large for a given sample density. Conversely, in low-CDR regions, numerous data points introduce redundancy without commensurate reduction in error. EDS seeks to smooth error distribution by reducing redundant samples where errors are already low, while preserving or augmenting support in high-error domains. This procedure maintains or reduces the worst-case regional error bound and enhances dataset efficiency.

Over a simplex $\mathbb{R}^n \times \mathbb{R}^m$5 with $\mathbb{R}^n \times \mathbb{R}^m$6 vertices, the interpolation error satisfies:

$\mathbb{R}^n \times \mathbb{R}^m$7

Given $\mathbb{R}^n \times \mathbb{R}^m$8 per simplex, the local interpolation error is proportional to the CDR.

3. EDS Algorithm for Representative Subset Selection

The EDS algorithm accepts the full dataset $\mathbb{R}^n \times \mathbb{R}^m$9, a batch size $k$0, and an error threshold $k$1. Its objective is to identify a representative subset $k$2, discarding points that do not contribute significantly to reducing regional error.

• Initialization: $k$3 is seeded with $k$4 random points to construct an initial simplex.
• Triangulation: Construct Delaunay triangulation $k$5 over $k$6.
• Streaming insertion: For batches $k$7:
  • For each $k$8, find containing simplex $k$9 in $\mathcal{F} = \{\Omega_j\}_{j=1}^k$0.
  • If none exists, insert $\mathcal{F} = \{\Omega_j\}_{j=1}^k$1 into $\mathcal{F} = \{\Omega_j\}_{j=1}^k$2 and update $\mathcal{F} = \{\Omega_j\}_{j=1}^k$3.
  • Otherwise, predict $\mathcal{F} = \{\Omega_j\}_{j=1}^k$4 via the simplex’s linear model, compute error $\mathcal{F} = \{\Omega_j\}_{j=1}^k$5.
  • If $\mathcal{F} = \{\Omega_j\}_{j=1}^k$6, add to $\mathcal{F} = \{\Omega_j\}_{j=1}^k$7; else assign to the auxiliary set $\mathcal{F} = \{\Omega_j\}_{j=1}^k$8 (redundant points).

Algorithmic growth of $\mathcal{F} = \{\Omega_j\}_{j=1}^k$9 is localized: points are added only where errors exceed the prescribed threshold (Algorithm 1, (Chen et al., 4 Feb 2025)).

4. Theoretical Guarantees and Complexity

The EDS framework guarantees that, under mild smoothness conditions, the maximal regional error bound is proportional to the CDR. This enables direct control over the local approximation error via representative data selection. Upon each new insertion, an $\Omega$0-simplex divides into $\Omega$1 smaller simplices, shrinking the region’s volume and size metric $\Omega$2 by $\Omega$3. The expected reduction in error threshold after $\Omega$4 insertions is

$\Omega$5

Convergence to tight error bounds occurs rapidly for small $\Omega$6, but decelerates with increasing dimension, which underscores the “low-dimensional” focus of EDS.

Updating the Delaunay triangulation has average cost $\Omega$7 per sample, and barycentric interpolation is $\Omega$8. The total streaming complexity is $\Omega$9, and typically $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$0.

5. Hyperparameter Effects and Sensitivity

Key EDS hyperparameters include:

• $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$1 (error threshold): Lower values yield tighter error control but larger $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$2 due to increased sample retention.
• $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$3 (standard deviation multiplier): Dictates GIM threshold for error control; higher values increase tolerance for maximal local error, reducing $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$4.
• $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$5 (batch size): Affects runtime efficiency and update frequency of triangulation.
• Initial $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$6 ($$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$7): Sets the early simplex coverage minimum.

Empirically, $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$8 (corresponding to $$g_s(\Omega) = \max_{x_1, x_2 \in \Omega} \| x_1 - x_2 \|_2^2$$998.85% confidence) is sufficient to encompass all notable errors. The paper does not report a systematic sweep over $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$0 or $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$1, but suggests that tighter settings increase representativeness at additional cost (Chen et al., 4 Feb 2025).

6. Empirical Evaluation and Benchmarks

The EDS approach was evaluated on four datasets:

• Motivational example: $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$2 with $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$3 train/$g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$4 test samples.
• Lorenz system identification (SINDy): $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$5 train/$g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$6 test samples.
• Rectangle inertia: $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$7 train/$g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$8 test, 4D feature space.
• Real-world control:
  • Cartpole: $g_c(\Omega) = \max_{x \in \Omega} \| H_f(x) \|_F$9 train/$f$0 test
  • Quadcopter: $f$1 train/$f$2 test

Baselines include the full dataset ($f$3), EDS representative set ($f$4), and a randomly subsampled minor set ($f$5) equal in size to $f$6. Evaluations used RMSE, maximum error, and training time.

Dataset	RMSE (D)	RMSE ($f$7)	RMSE ($f$8)	Max Err (D)	Max Err ($f$9)	Max Err ($|\Omega \cap D|$0)	Train Time (D)	Train Time ($|\Omega \cap D|$1)	Train Time ($|\Omega \cap D|$2)
Lorenz/SINDy	0.0296	0.0117	0.0485	0.715	0.189	1.161	9.412 s	0.017 s	0.058 s

For the motivational example and MLP regression, $|\Omega \cap D|$3 led to more uniform error histograms, lower maximum error, and RMSE competitive with both $|\Omega \cap D|$4 and $|\Omega \cap D|$5. For rectangle inertia, $|\Omega \cap D|$6 slightly increased RMSE but greatly reduced worst-case error; Cartpole/Quadcopter results were consistent, with $|\Omega \cap D|$7 reducing maximum errors in noisy, imbalanced regimes (Chen et al., 4 Feb 2025).

7. Limitations, Strengths, and Prospective Directions

EDS provides a principled mechanism to control local regression error profiles—crucially through CDR—and yields significant dataset size reductions while preserving or enhancing worst-case performance. Its streaming, incremental construction via Delaunay triangulation accelerates training and improves uniformity of predictive error.

However, convergence is markedly slower in high-dimensional feature spaces, reflecting the intrinsic complexity scaling. The need for dynamic triangulation updates as $|\Omega \cap D|$8 grows can become computationally intensive. Hyperparameters ($|\Omega \cap D|$9) are hand-chosen; no automated or adaptive selection approach is currently included.

Potential extensions include parallel or GPU-accelerated triangulation for higher dimensions, hyperparameter adaptation via cross-validation or bandit optimization, and integration of nonlinear local interpolation schemes (e.g., kernel or polynomial fits) for enhanced performance in high-curvature regimes (Chen et al., 4 Feb 2025).