---
title: Weighted Improved Greedy Sampling (WiGS)
url: https://www.emergentmind.com/topics/weighted-improved-greedy-sampling-wigs
type: topic
---

# Weighted Improved Greedy Sampling (WiGS)

Searching arXiv for the cited WiGS-related papers to ground the article.
Weighted improved Greedy Sampling (WiGS) is a pool-based active learning method for regression in which the query rule is defined by a weighted additive combination of feature-space diversity and output-space uncertainty, rather than the static multiplicative coupling used by Improved Greedy Sampling (iGS) [2603.10435]. In this formulation, the exploration–investigation balance is controlled by a weight \(w_x^{(t)} \in [0,1]\) that may be fixed, scheduled over time, or learned adaptively through reinforcement learning. WiGS was introduced to address a specific limitation of iGS in irregular-density regression domains: the multiplicative rule can suppress genuinely informative high-error samples when they lie in dense regions of feature space, a failure mode described as the **density veto** [2603.10435].

## 1. Problem setting and objective

WiGS is formulated for **pool-based active learning for regression**. At iteration \(t\), the learner maintains a labeled training set
\[
D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}
\]
and an unlabeled candidate pool
\[
D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.
\]
A regression model \(f:\mathcal{X}\to\mathcal{Y}\) is trained on \(D_{tr}\), and the active learner selects one point from \(D_{cdd}\) to query next [2603.10435].

The central design problem is to trade off two competing objectives. **Exploration** favors candidates that expand coverage of the feature space \(\mathcal{X}\); **investigation** favors candidates for which the current model indicates high output-space uncertainty or error. In the terminology of the method, these correspond to **feature-space diversity** and **output-space uncertainty**. WiGS treats their relative importance as non-stationary: early iterations may benefit from exploration, whereas later iterations, or locally noisy regions, may favor investigation [2603.10435].

This framing is motivated by heterogeneous regression domains in which data density, functional complexity, and noise level vary across regions. In such settings, a fixed acquisition rule can be systematically misaligned with the local structure of the pool. WiGS therefore recasts the query rule so that the exploration–investigation balance can change over time and depend on the state of the labeled set [2603.10435].

## 2. Place within the Greedy Sampling family

WiGS extends a sequence of greedy regression-query heuristics built from pointwise distances between unlabeled candidates and the labeled set. For a candidate \(\mathbf{x}_n\) and a labeled point \((\mathbf{x}_m,y_m)\), the method uses
\[
d_{nm}^x \equiv \|\mathbf{x}_n - \mathbf{x}_m\|
\quad\text{and}\quad
d_{nm}^y \equiv |f(\mathbf{x}_n) - y_m|.
\]

The simplest variant, **GSx**, selects the candidate that is farthest from the labeled set in feature space:
\[
\mathbf{x}^*_{GSx}
=
\underset{\mathbf{x}_n \in D_{cdd}}{\arg\max}\; d_n^x,
\qquad
d_n^x \equiv \min_m d_{nm}^x.
\]
Its companion, **GSy**, selects the candidate that is farthest from the labeled set in predicted output space:
\[
\mathbf{x}^*_{GSy}
=
\underset{\mathbf{x}_n \in D_{cdd}}{\arg\max}\; d_n^y,
\qquad
d_n^y \equiv \min_m d_{nm}^y.
\]

**Improved Greedy Sampling (iGS)** combines these signals with a static multiplicative rule:
\[
\mathbf{x}^*_{iGS}
=
\underset{\mathbf{x}_n \in D_{cdd}}{\arg\max}\; s_n^{iGS},
\qquad
s_n^{iGS} = \min_m (d_{nm}^x \cdot d_{nm}^y).
\]
This criterion favors candidates that are simultaneously novel in feature space and uncertain in output space [2603.10435].

WiGS departs from iGS at exactly this point. The motivating claim is that the product rule is too rigid because it enforces a **fixed, multiplicative coupling** between the two objectives. When a candidate lies in a dense region, the feature-distance term can be small enough to suppress the entire product even if the candidate is highly uncertain. This is the mechanism later formalized as the density veto [2603.10435].

## 3. Additive weighting and the density-veto result

WiGS replaces the multiplicative iGS score with a **weighted additive score**. Before combination, feature-space and output-space distances are normalized by \(\phi(\cdot)\). The score assigned to candidate \(\mathbf{x}_n\) is
\[
s_n^{WiGS}
=
\min_m
\left(
w_x^{(t)} \phi(d_{nm}^x)
+
(1-w_x^{(t)}) \phi(d_{nm}^y)
\right),
\]
and the selected query is
\[
\mathbf{x}^*_{WiGS}
=
\underset{\mathbf{x}_n \in D_{cdd}}{\arg\max}\; s_n^{WiGS}.
\]
Here \(w_x^{(t)}\) is the weight assigned to feature-space exploration, while \(1-w_x^{(t)}\) is the weight assigned to output-space investigation [2603.10435].

The interpretation is direct. If \(w_x^{(t)} \approx 1\), WiGS behaves as an exploration-oriented policy; if \(w_x^{(t)} \approx 0\), it becomes investigation-oriented; intermediate values interpolate between the two. The significance of the additive form is that one signal need not vanish merely because the other is small. In particular, a candidate with high uncertainty but low feature novelty can still receive a large score if the current state favors investigation [2603.10435].

The paper states a formal proposition under the name **The Density Veto**. Consider a target \(x^*\) with high uncertainty \(u^*\) and low diversity \(d^*\), and a distractor \(x'\) with lower uncertainty \(u' < u^*\) but moderate diversity \(d' > d^*\). Defining
\[
S_{iGS}(x) = d(x)\cdot u(x)
\quad\text{and}\quad
S_{WiGS}(x;w)=w\cdot d(x)+(1-w)\cdot u(x),
\]
the proposition shows that as the feature density around \(x^*\) increases, so that \(d^* \to 0\), there exists a threshold \(\delta>0\) such that if \(d^*<\delta\), then the multiplicative selector strictly prefers the sub-optimal distractor \(x'\), regardless of the uncertainty gap. By contrast, there always exists a weight \(w\in[0,1]\) such that the additive selector prefers the high-uncertainty target \(x^*\) [2603.10435].

In the appendix proof, the multiplicative failure is expressed as
\[
d^*u^* < d'u' \implies d^* < \frac{d'u'}{u^*},
\]
while the additive preference condition
\[
w d^* + (1-w)u^* > w d' + (1-w)u'
\]
can be rearranged to show that sufficiently small \(w\) makes the target preferable. The theoretical role of this result is not to establish a global optimality theorem for active learning, but to isolate a concrete failure mode of iGS and to justify the design choice of a tunable additive rule [2603.10435].

## 4. Weighting policies and reinforcement-learning adaptation

WiGS distinguishes three principal ways of setting \(w_x^{(t)}\): static, time-decay, and adaptive. The adaptive family is further divided into a discrete **multi-armed bandit** instantiation and a continuous-control **Soft Actor-Critic** instantiation.

| Variant | Weight rule | Characteristic |
|---|---|---|
| Static WiGS | \(w_x^{(t)}=w_x\) | Constant exploration–investigation balance |
| Time-decay WiGS | Linear or exponential decay | Dynamic but non-adaptive |
| Adaptive WiGS | MAB/UCB1 or SAC | Feedback-driven balance |

In **Static WiGS**, the weight is fixed for all iterations. The paper gives examples such as \(w_x=0.25\), which is investigation-focused, and \(w_x=0.75\), which is exploration-focused. In **Time-decay WiGS**, the weight changes deterministically with iteration number, using schedules such as
\[
w_x^{(t)} = 1 - (c\cdot t/T)
\quad\text{or}\quad
w_x^{(t)} = \exp(-c\cdot t/T).
\]
These schedules encode the hypothesis that exploration is more valuable early and investigation later, but they do not adapt to the actual state of the learner [2603.10435].

The adaptive versions cast weight selection as a reinforcement-learning problem. A key design choice is that the reward is computed **only from the currently labeled set** using cross-validation, rather than from the unlabeled pool, in order to avoid leakage and the **active learning validation paradox**. In both adaptive variants, the reward is
\[
r_t = CV_{RMSE}^{(t-1)} - CV_{RMSE}^{(t)},
\]
so positive reward indicates improved cross-validated generalization performance after the latest query [2603.10435].

**WiGS-MAB** discretizes the action space to a small set of candidate weights, for example \(\{0.25, 0.50, 0.75\}\), and uses **UCB1** to balance exploration of under-tested weights against exploitation of historically successful ones. The action space is intentionally coarse because a finer grid would consume too much of the labeling budget and the bandit does not generalize across nearby weights [2603.10435].

**WiGS-SAC** treats the weight as a continuous action:
\[
a_t = w_x^{(t)} \in [0,1].
\]
Its state \(s_t\) is computed only from \(D_{tr}^{(t)}\) and includes the current \(CV_{RMSE}\), the progress term \(t/T\), and distributional statistics of the labeled set. The policy is a stochastic policy \(\pi(s_t)\) trained by **Soft Actor-Critic (SAC)**. The entropy term in SAC is emphasized because it encourages stochasticity when the reward signal is ambiguous and reduces premature collapse to a brittle deterministic rule [2603.10435].

Operationally, the WiGS loop proceeds by training the current regressor on \(D_{tr}^{(t)}\), computing \(CV_{RMSE}^{(t)}\), deriving the reward, selecting \(w_x^{(t)}\) according to the chosen strategy, computing normalized feature and output distances, evaluating \(s_n^{WiGS}\) for every candidate, querying the highest-scoring point, and updating \(D_{tr}\) and \(D_{cdd}\). In the SAC variant, state transitions are stored in a replay buffer \(\mathcal{B}\), and the policy is updated online [2603.10435].

## 5. Empirical behavior, evaluation protocol, and limitations

The evaluation uses **2 synthetic datasets** and **18 real-world benchmarks**. The real datasets include AutoMPG, Beer, Body Fat, Burbidge variants, Concrete variants, CPS, Housing, NO2, PM10, QSAR, Wine-Red, Wine-White, and Yacht. For each dataset, the protocol uses **100 independent replications**, with an initial training set equal to **5% of the data**, a candidate pool of **95%**, and an active-learning run that continues until the pool is exhausted. The main predictor is **Ridge Regression** with \(\alpha=0.01\), and the experiments are repeated with **Random Forest Regressor** to examine whether the observed behavior is model-specific [2603.10435].

The baseline set includes Random Sampling, GSx, GSy, iGS, WiGS Static with \(w_x=0.25\), WiGS Static with \(w_x=0.75\), WiGS Linear Decay, WiGS Exponential Decay, WiGS-MAB, WiGS-SAC, Uncertainty Sampling, Query-by-Committee implemented via bagging, Expected Model Change Maximization, and EGAL. Evaluation is separated from the reinforcement-learning reward. Training feedback is based on cross-validated RMSE, whereas empirical comparison uses **Full-Pool RMSE** over the whole domain \(D_{tr}\cup D_{cdd}\), together with the **Area Under the Learning Curve (AUC)**, a relative AUC normalized by iGS, **relative label efficiency** \(N_{rel}\) at 70% and 80% of total gain, full RMSE traces, correlation coefficient traces, and RMSE variance traces [2603.10435].

The synthetic environments are designed to expose the density-veto failure mode. In the two-regime construction, the feature distribution is a non-uniform Gaussian mixture,
\[
p(x) = 0.4 \mathcal{N}(x \mid 0.2, 0.07^2) + 0.3\mathcal{N}(x \mid 0.5, 0.1^2)+ 0.3 \mathcal{N}(x \mid 0.85, 0.05^2),
\]
with regression function
\[
f(x) =
\begin{cases}
\sin(10\pi x) & \text{if } x < 0.5,\\
2x - 1 & \text{if } x \geq 0.5,
\end{cases}
\]
and heteroscedastic noise
\[
\epsilon \sim
\begin{cases}
\mathcal{N}(0,1) & \text{if } 0.8 < x < 0.9,\\
\mathcal{N}(0, 0.1^2) & \text{otherwise.}
\end{cases}
\]
This creates a dense high-uncertainty **noise trap** near \(x \approx 0.85\), precisely the regime in which iGS can suppress informative points and WiGS can adapt toward investigation [2603.10435].

The reported findings are consistent with this construction. On the two-regime synthetic task, GSx and GSy perform poorly, iGS improves on those single-objective baselines but still fails in the high-noise dense region, and WiGS-SAC significantly outperforms iGS; the pairwise Wilcoxon signed-rank tests are reported as statistically significant with \(p<0.05\). Across the benchmark suite, WiGS-SAC matches or outperforms iGS on **15 of 20 evaluated datasets** total. Adaptive WiGS methods generally require fewer labels to reach a fixed fraction of the achievable gain, with WiGS-MAB yielding **roughly a 4% label-cost reduction at the median** in one summary, while WiGS-SAC is reported as especially robust with a narrower efficiency distribution than many baselines [2603.10435].

The method also has explicit limitations. **WiGS-SAC has high computational cost**, with the summary table reporting about **\(416.4\) seconds** for WiGS-SAC versus **\(15.5\) seconds** for iGS. There is an **RL warm-up period**, so early iterations may underperform iGS when labeling budgets are very small. The adaptive policy is most useful when the budget is large enough for the learning cost to amortize. Finally, because the reward depends on cross-validation over the current labeled set, the feedback signal is only as informative as that set permits [2603.10435].

## 6. Antecedents, analogues, and terminological boundaries

Although WiGS in the strict sense denotes the regression active-learning method introduced in 2026, the underlying idea of using weights to reshape a greedy objective has notable antecedents. In weighted least-squares polynomial approximation, **Christoffel-weighted approximate Fekete points** define a deterministic greedy sampling procedure in which point selection is driven by a weighted Vandermonde determinant, with the weight given by the inverse Christoffel function \(1/\sqrt{K_\Lambda(y)}\). The procedure is implemented through pivoted linear algebra, especially column-pivoted QR on the transpose of a weighted Vandermonde-like matrix, and the paper explicitly describes it as a direct precursor or near-equivalent formulation of the weighted greedy sampling idea [1708.01296].

In that setting, the weighted least-squares problem is
\[
\mathbf{\widehat{f}}
=
\arg\min_{\mathbf{v}\in \mathbb{R}^N}
\|\mathbf{V}(A_M,Q)\mathbf{v}-\mathbf{W}\mathbf{f}\|_2,
\]
with diagonal weights
\[
(W)_{m,m}=\frac{1}{\sqrt{K_\Lambda(y_m)}},
\qquad
K_\Lambda(y)=\sum_{\alpha\in\Lambda}\psi_\alpha^2(y),
\]
and a weighted polynomial space
\[
Q(\Lambda)=\mathrm{span}\left\{\frac{\psi_\alpha}{\sqrt{K_\Lambda}}:\alpha\in\Lambda\right\}.
\]
The greedy point-selection rule maximizes the weighted determinant modulus, or equivalently pursues optimal conditioning. In one dimension, the method is reported to almost always generate optimally conditioned linear systems, and when optimal sets exist, the weighted determinant and weighted condition-number formulations coincide at the optimum [1708.01296]. A plausible implication is that WiGS belongs to a broader family of weighted greedy designs in which the weighting scheme is used to stabilize or redirect the underlying selection criterion.

A second analogue appears in **iteratively reweighted greedy set cover**, a heuristic for large sparse set cover instances. There the weighted greedy algorithm is used as an inner solver, and element weights are updated multiplicatively whenever a \(k\)-cover attempt leaves some elements uncovered. The procedure begins with
\[
w \leftarrow \mathbf{1}^n,\qquad k \leftarrow m,
\]
runs weighted greedy set cover, and if an uncovered element \(i\) is found, updates
\[
w_i \leftarrow w_i f.
\]
If all elements are covered, the solution is stored as the current best \(s^*\) and the target cover size is tightened by \(k \leftarrow k-1\). The method is explicitly presented as a heuristic rather than an approximation theorem, with the practical properties that the implementation is trivial, there are essentially no tuning parameters beyond the growth factor \(f\), and more iterations can only improve the result [2010.15476]. This is not WiGS in the active-learning sense, but it exhibits the same structural pattern of a greedy core wrapped in an outer weighting mechanism.

A further methodological analogue appears in **greedy completion for weighted \((\alpha,\beta)\)-spanners**, which begins from a sparse initial graph, identifies pairs with bad stretch, computes a minimal segmentation of a shortest path, and adds only the single-edge bottlenecks that violate the \(\alpha\)-stretch condition. The paper explicitly notes a conceptual overlap with “improved greedy sampling” as an algorithmic philosophy—sparse initialization followed by selective greedy repair—but also states that it is **not in the classical “sampling” sense** [2603.17047].

These related formulations delimit the term. In contemporary usage, **Weighted improved Greedy Sampling** refers specifically to the active learning framework for regression that adaptively balances feature-space exploration and output-space investigation through a weighted additive selector and, in its most advanced form, reinforcement learning [2603.10435]. The older weighted Fekete and iterative reweighting methods are best regarded as antecedents or analogues rather than instances of the same algorithm.

Source: https://www.emergentmind.com/topics/weighted-improved-greedy-sampling-wigs