Granular Regression Ball Framework

• The paper introduces granular balls as compact representatives to reformulate ε-SVR constraints, yielding significant computational speedups and robust accuracy.
• It uses recursive K-means splitting to generate homogeneous subregions, effectively reducing dimensionality and mitigating outlier influence.
• Experimental evaluations across synthetic, UCI, and real-world data demonstrate improvements in R² and error metrics compared to traditional SVR.

The Granular Regression Ball Framework is a data abstraction and algorithmic architecture for efficient robust regression, exemplified by Granular Ball Support Vector Regression (GBSVR). In this framework, high-dimensional regression datasets are summarized by a set of "granular balls," which serve as core representatives for the feature-target structure. The approach yields significant computational advantages and empirically robust accuracy, addressing both scalability and outlier sensitivity inherent in classical support vector regression (Rastogi et al., 13 Mar 2025).

1. Formal Definition of Granular Regression Balls

A granular ball (denoted $GRB_j$) is a compact representation of a subset of data points $\{x_i\}_{i=1}^u \subset \mathbb{R}^l$, characterized by its center, radius, and a "purity" measure. The center is defined as

$c = \frac{1}{u}\sum_{i=1}^u x_i,$

with the radius typically taken as the mean distance,

$$r = \frac{1}{u}\sum_{i=1}^u d(x_i, c), \quad d(x,c) = \sqrt{\sum_{j=1}^l (x_j - c_j)^2},$$

a choice enhancing robustness to outliers compared to maximal-distance radii.

To quantify homogeneity, each response $y_i$ is discretized into $k$ quantiles; data points in $GRB_j$ are assigned pseudo-class labels. The purity or quality of a ball is then

$$\mathrm{quality}(GRB_j) = \frac{|\{x_i \in GRB_j : \text{label}(x_i) = \text{majority}\}|}{|GRB_j|},$$

and a ball is considered "homogeneous" if $\mathrm{quality}(GRB_j) \ge T$ for a threshold $T\in(0,1]$.

2. Construction and Splitting of Granular Balls

The regression ball generation proceeds recursively:

1. The continuous target values $\{y_i\}$ are discretized into $k$ quantile-based intervals, labeling each sample accordingly.
2. The entire dataset is initially covered with a single ball.
3. Balls with insufficient purity ($<T$) or excessive cardinality ($>$ minimum threshold $p$) are split using K-means ($K=2$), recomputing centers, radii, and labels after each split.
4. The process recurses until all granular balls satisfy the stopping criteria.

After splitting, the representative target for each granular ball is set as the mean response: $$\hat y_j = \frac{1}{|GRB_j|} \sum_{x_i \in GRB_j} y_i.$$ This hierarchical splitting condenses the original $m$ samples into $n\ll m$ granular balls.

3. Regression Reformulation Using Granular Balls

In GBSVR, standard $\epsilon$-SVR constraints are imposed not on each data point, but on the furthest point from the center of each granular ball. For a learned regressor $f(x)=w\cdot x+b$, the support point of a granular ball is geometrically represented as

$w\cdot x_i = w\cdot c_i - \|w\| r_i.$

Modified $\epsilon$-insensitive constraints for each ball are: $$\|w\|r_i + \hat y_i - w\cdot c_i - b \le \epsilon + \xi_i,$$

$$w\cdot c_i + b - \hat y_i - \|w\|r_i \le \epsilon + \xi^*_i,$$

for all $i=1, ..., n$, with slack variables $\xi_i, \xi^*_i \ge 0$.

The primal soft-margin objective is given by: $$\begin{aligned} \min_{w,b,\xi,\xi^*}\quad &\frac12\|w\|^2 + C\sum_{i=1}^n(\xi_i+\xi^*_i) \ \text{s.t.}\quad &\|w\|r_i + \hat y_i - w\cdot c_i - b \le \epsilon + \xi_i, \ &w\cdot c_i + b - \hat y_i - \|w\|r_i \le \epsilon + \xi^*_i, \ &\xi_i, \xi^*_i \ge 0, \forall i. \end{aligned}$$ The associated dual form introduces variables $\alpha_i, \alpha^*_i$ and constraints $\sum (\alpha_i-\alpha^*_i)=0$.

4. Computational Considerations

Classical $\epsilon$-SVR scales cubically ($O(m^3)$) in time and quadratically ($O(m^2)$) in space with the dataset size $m$. In contrast, GBSVR complexity stems from two stages:

• Ball generation via recursive K-means splits: $O(m\log m)$
• QP optimization for $n$ balls: $O(n^3)$

Since empirically $n\approx m/3$ to $m/10$, overall speedups of 5–20× and substantial reductions in memory are realized without sacrificing accuracy.

5. Experimental Evaluation and Performance

GBSVR has been rigorously tested on both synthetic and real-world datasets:

• Synthetic Functions: "sinc" and "cos" with heteroscedastic noise; GBSVR achieved $R^2>0.98$ (vs. 0.97 for SVR), consistently lower MAE, MSE, and RMSE.
• UCI Benchmarks: Datasets such as AutoMPG, Servo, Yacht, and Real Estate with 5-fold CV showed GBSVR outperforming SVR/NuSVR in accuracy and ≈10× faster runtime under up to 20% injected noise.
• Stock Price Forecasting: Five-year daily series for Apple, Google, NVIDIA, Tesla; GBSVR $R^2\approx0.98$ (Apple) vs. SVR 0.978, with analogous results for other tickers.
• Wind Speed Prediction: 36,000 samples (25 days, minutely); GBSVR outperformed SVR with $R^2\approx0.85$ vs. 0.81, and achieved lower MAE/MSE.

Across all settings, GBSVR exhibited greater robustness to heteroscedastic and high-magnitude noise, with minimal performance degradation compared to classical SVR (Rastogi et al., 13 Mar 2025).

6. Hyperparameter Specification and Guidance

Several hyperparameters govern GBSVR performance:

• Purity threshold $T$: Choices in $[0.90, 0.997]$; higher values induce more homogeneous balls via additional splitting. Recommended: $T=0.97$–0.99.
• Minimum points per ball $p$: Values in $[2,4]$; $p=4$ empirically optimal to balance over-splitting and impurity.
• Discretization labels $k$: Governs the granularity of target quantization, typically selected in preliminary experiments.
• $\epsilon$-tube width ($\epsilon$): Logarithmic grid $[10^{-1}, 10^{-9}]$ for insensitivity.
• RBF kernel scale ($\sigma$): Searched in $[10^{-3}, 1]$ with step 0.01.
• Regularization ($C$): Determined via validation set grid search.

This hyperparameterization underpins effective construction of granular balls and subsequent robust regression fitting.

7. Context, Generalizations, and Implications

The granular ball abstraction offers a paradigm for scalable, noise-tolerant regression in high-dimensional or large-scale settings, recasting pointwise learning approaches into compact, homogeneous subregion representatives. This suggests that other kernel-based or convex regression frameworks may benefit analogously from ball-based summarization, especially when computational efficiency and robustness to outliers are central (Rastogi et al., 13 Mar 2025). The framework bridges discrete and continuous target settings via quantile-based discretization, enabling "purity"-based supervision in otherwise regression-labeled datasets. A plausible implication is enhanced interpretability and modularity in future regression methods that employ granular data abstractions.