---
title: Granular Ball Support Vector Regression (GBSVR)
url: https://www.emergentmind.com/topics/granular-ball-support-vector-regression-gbsvr
type: topic
---

# Granular Ball Support Vector Regression (GBSVR)

Granular Ball Support Vector Regression (GBSVR) is a machine learning framework for regression that integrates the granular ball data summarization paradigm with support vector regression (SVR) to address the high computational cost and outlier sensitivity inherent in traditional SVR, particularly for large-scale and noisy datasets. GBSVR replaces raw data points with a compact representation via granular balls, each characterized by a centroid, a radius, and an aggregated label, enabling more efficient quadratic programming, reduced memory usage, and enhanced robustness to outliers by leveraging group-level information [2503.10539].

## 1. Definition and Construction of Granular Balls

A granular ball, denoted $B(c,r)$, provides a compact summary of a subset of the input space. Given $X = \{x_1, \dots, x_m\} \subset \mathbb{R}^l$, a granular ball centered at $c \in \mathbb{R}^l$ with radius $r \ge 0$ is defined as
\[
B(c,r) = \{x \in \mathbb{R}^l \mid d(x, c) \le r\}, \qquad d(x, c) = \sqrt{\sum_{j=1}^l (x_j - c_j)^2}.
\]
In the GBSVR framework, each ball $B$ not only clusters proximal data points but is also attached a "ball-label" $\hat{y}$, computed as the average response of its constituent points.

The generation of granular balls proceeds by a top-down splitting strategy governed by a regression-purity metric. Since the regression targets $Y$ are continuous, they are first discretized into $k$ quantiles, assigning each sample a label in $\{1, \dots, k\}$. For any candidate ball $GRB_j$, define
\[
\mathrm{quality}(GRB_j) = \frac{|\text{samples in } GRB_j \text{ having the majority label}|}{|GRB_j|}.
\]
Splitting continues until all balls satisfy a purity threshold $T \in (0,1)$ and a minimum cardinality $p$. Splitting is performed via K-means with $K=2$, recalculating centroids $c$, radii $r$, and ball-labels $\hat{y}$ at each step.

## 2. Discretization and Ball-Generation Process

Discretization is essential for quantifying regression purity. The target values $\{y_i\}$ are sorted, cut-points are defined at $(j/k)$-th quantiles for $j=1, \dots, k-1$, and each $y_i$ is assigned an interval label $\ell_i \in \{1, \dots, k\}$. This quantile-based discretization ensures approximately uniform label distributions, supporting effective purity evaluation during ball construction.

The granular regression ball generation process is summarized as follows:
1. Assign quantile labels to $\{y_i\}$.
2. Initialize a single ball covering all data.
3. Iteratively split any ball with insufficient purity or too few samples using $K$-means, updating $(c, r)$ and $\hat{y}$.
4. Terminate when all final balls satisfy purity and cardinality criteria.

## 3. Integration of Granular Balls into SVR

After grouping, the data is represented as $n \ll m$ granular balls $GRB_i = (c_i, r_i, \hat{y}_i)$. SVR now operates on these representatives, not the original dataset. For classical $\epsilon$-SVR, the constraints for a data point $(x_i, y_i)$ are
\[
y_i - w \cdot x_i - b \le \epsilon + \xi_i, \quad w \cdot x_i + b - y_i \le \epsilon + \xi_i^*.
\]
For a granular ball $B_i$, the constraint considers the furthest member along the normal direction, yielding
\[
w \cdot x_i = w \cdot c_i - \|w\| r_i,
\]
and so the SVR constraints become:
\begin{align}
\|w\| r_i + \hat{y}_i - w \cdot c_i - b &\le \epsilon + \xi_i, \\
w \cdot c_i + b - \hat{y}_i - \|w\| r_i &\le \epsilon + \xi_i^*.
\end{align}

The primal GBSVR formulation is:
\[
\begin{aligned}
\min_{w,b,\xi,\xi^*} \quad & \frac{1}{2}\|w\|^2 + C \sum_{i=1}^n (\xi_i + \xi_i^*) \\
\text{s.t.} \quad
& \|w\| r_i + \hat{y}_i - w \cdot c_i - b \le \epsilon + \xi_i,\quad i=1 \ldots n \\
& w \cdot c_i + b - \hat{y}_i - \|w\| r_i \le \epsilon + \xi_i^*,\quad i=1 \ldots n \\
& \xi_i,\ \xi_i^* \ge 0
\end{aligned}
\]
The dual introduces Lagrange multipliers $\alpha_i, \alpha_i^* \in [0, C]$, yielding the following:
\[
\begin{aligned}
A &= \sum_{i=1}^n (\alpha_i-\alpha_i^*)c_i,\\
B &= \sum_{i=1}^n (\alpha_i-\alpha_i^*)r_i,\\
\|w\| &= \|A\| - B,
\end{aligned}
\]
with the dual objective
\[
\begin{aligned}
\max_{\alpha, \alpha^*} \quad
& -\tfrac{1}{2}\|A\|^2 - \tfrac{1}{2} B^2 + \|A\| B \\
& + \sum_{i=1}^n (\alpha_i - \alpha_i^*) \hat{y}_i
- \epsilon \sum_{i=1}^n (\alpha_i + \alpha_i^*) \\
\text{s.t.} \quad
& \sum_{i=1}^n (\alpha_i - \alpha_i^*)=0,\quad 0 \le \alpha_i, \alpha_i^* \le C.
\end{aligned}
\]

## 4. End-to-End Algorithmic Workflow

The GBSVR methodology consists of the following workflow:
1. Input data $D = \{(x_i,y_i)\}_{i=1}^m$.
2. Discretize response values $\{y_i\}$ into $k$ quantile intervals.
3. Generate granular regression balls $\{GRB_i = (c_i, r_i, \hat{y}_i)\}_{i=1}^n$.
4. Select SVR hyperparameters: $\epsilon$ (tube width), $C$ (regularization), and kernel $K(x,x')$ (e.g., RBF kernel).
5. Solve the dual quadratic program on $n$ representatives.
6. Recover the model parameters $(w, b)$.
7. For new input $x$, predict as $f(x) = w \cdot x + b$.

Granular ball generation proceeds until each ball achieves purity at least $T$ and cardinality at least $p$. Model training concludes once the dual optimization reaches standard SVR convergence thresholds.

## 5. Computational Complexity Considerations

Traditional $\epsilon$-SVR on $m$ datapoints solves a quadratic program (QP) of size $m \times m$, requiring $O(m^3)$ time and $O(m^2)$ memory. In GBSVR, the problem is reduced to $n$ representatives, so the QP is $n \times n$, resulting in $O(n^3)$ time and $O(n^2)$ memory. Given that $n \ll m$, this confers a theoretical and practical complexity reduction on the order of $(m/n)^3$, with empirical results often yielding an order-of-magnitude improvement in runtime [2503.10539].

## 6. Empirical Evaluation and Results

GBSVR demonstrates robust performance across synthetic, benchmark, and real-world forecasting tasks. Experiments employ 5-fold cross-validation and RBF kernels.

- On synthetic datasets (e.g., $y = \sin(\pi x)/(\pi x)+\eta$ and $y = \cos(\pi x)+\eta$ with heteroscedastic Gaussian and uniform noise), GBSVR achieves up to 1–2% higher $R^2$ and 10–20% lower RMSE compared to SVR/NuSVR.
- UCI benchmarks (AutoMPG, Servo, Yacht, Machine, etc.) report the following representative compression rates:

| Dataset                | Samples × Features | # Granular Balls |
|------------------------|-------------------|------------------|
| Real Estate Valuation  | 414 × 6           | 147              |
| AutoMPG                | 392 × 7           | 137              |
| Autos                  | 159 × 25          | 59               |
| Servo                  | 167 × 4           | 56               |
| Yacht                  | 308 × 6           | 56               |
| Machine                | 209 × 7           | 74               |

GBSVR yields the highest $R^2$ (e.g., improvements of 0.05–0.10), lowest error metrics, and $\sim$10$\times$ training speed-up (e.g., 1s vs. 9s on Servo).

- In stock price forecasting (Apple, Google, NVIDIA, Tesla; 5-day sliding window), for example, GBSVR on Apple yields $R^2 = 0.9814$ (vs. 0.9782 for SVR, 0.9784 for NuSVR) and MAE of 0.0623 (compared to 0.0710/0.0737).
- For short-term wind speed forecasting (36,000 samples, sliding window), GBSVR delivers (30-min horizon) $R^2 = 0.8241$ vs. 0.8006 (SVR), RMSE 0.2917 vs. 0.3106, and (20-min horizon) $R^2 = 0.8502$ vs. 0.8091, RMSE 0.2633 vs. 0.2973.

In all scenarios, GBSVR outperforms classical SVR and NuSVR with respect to both predictive accuracy and computational efficiency, attributable to its ability to compress the data through granular balls and to mitigate the impact of outliers [2503.10539].

Source: https://www.emergentmind.com/topics/granular-ball-support-vector-regression-gbsvr