Weighted Improved Greedy Sampling (WiGS)

• WiGS is a pool-based active learning technique that combines feature-space diversity and output-space uncertainty with a weighted additive rule to mitigate the density veto problem.
• It employs static, time-decay, and adaptive reinforcement learning strategies to dynamically balance exploration and investigation during regression model training.
• Empirical evaluations demonstrate that WiGS enhances label efficiency and robustness across synthetic and real-world benchmarks, despite higher computational costs compared to earlier methods.

Searching arXiv for the cited WiGS-related papers to ground the article. Weighted improved Greedy Sampling (WiGS) is a pool-based active learning method for regression in which the query rule is defined by a weighted additive combination of feature-space diversity and output-space uncertainty, rather than the static multiplicative coupling used by Improved Greedy Sampling (iGS) (Nguyen et al., 11 Mar 2026). In this formulation, the exploration–investigation balance is controlled by a weight $w_x^{(t)} \in [0,1]$ that may be fixed, scheduled over time, or learned adaptively through reinforcement learning. WiGS was introduced to address a specific limitation of iGS in irregular-density regression domains: the multiplicative rule can suppress genuinely informative high-error samples when they lie in dense regions of feature space, a failure mode described as the density veto (Nguyen et al., 11 Mar 2026).

1. Problem setting and objective

WiGS is formulated for pool-based active learning for regression. At iteration $t$, the learner maintains a labeled training set

$D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$

and an unlabeled candidate pool

$D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$

A regression model $f:\mathcal{X}\to\mathcal{Y}$ is trained on $D_{tr}$, and the active learner selects one point from $D_{cdd}$ to query next (Nguyen et al., 11 Mar 2026).

The central design problem is to trade off two competing objectives. Exploration favors candidates that expand coverage of the feature space $\mathcal{X}$; investigation favors candidates for which the current model indicates high output-space uncertainty or error. In the terminology of the method, these correspond to feature-space diversity and output-space uncertainty. WiGS treats their relative importance as non-stationary: early iterations may benefit from exploration, whereas later iterations, or locally noisy regions, may favor investigation (Nguyen et al., 11 Mar 2026).

This framing is motivated by heterogeneous regression domains in which data density, functional complexity, and noise level vary across regions. In such settings, a fixed acquisition rule can be systematically misaligned with the local structure of the pool. WiGS therefore recasts the query rule so that the exploration–investigation balance can change over time and depend on the state of the labeled set (Nguyen et al., 11 Mar 2026).

2. Place within the Greedy Sampling family

WiGS extends a sequence of greedy regression-query heuristics built from pointwise distances between unlabeled candidates and the labeled set. For a candidate $\mathbf{x}_n$ and a labeled point $(\mathbf{x}_m,y_m)$, the method uses

$t$0

The simplest variant, GSx, selects the candidate that is farthest from the labeled set in feature space: $t$1 Its companion, GSy, selects the candidate that is farthest from the labeled set in predicted output space: $t$2

Improved Greedy Sampling (iGS) combines these signals with a static multiplicative rule: $t$3 This criterion favors candidates that are simultaneously novel in feature space and uncertain in output space (Nguyen et al., 11 Mar 2026).

WiGS departs from iGS at exactly this point. The motivating claim is that the product rule is too rigid because it enforces a fixed, multiplicative coupling between the two objectives. When a candidate lies in a dense region, the feature-distance term can be small enough to suppress the entire product even if the candidate is highly uncertain. This is the mechanism later formalized as the density veto (Nguyen et al., 11 Mar 2026).

3. Additive weighting and the density-veto result

WiGS replaces the multiplicative iGS score with a weighted additive score. Before combination, feature-space and output-space distances are normalized by $t$4. The score assigned to candidate $t$5 is

$t$6

and the selected query is

$t$7

Here $t$8 is the weight assigned to feature-space exploration, while $t$9 is the weight assigned to output-space investigation (Nguyen et al., 11 Mar 2026).

The interpretation is direct. If $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$0, WiGS behaves as an exploration-oriented policy; if $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$1, it becomes investigation-oriented; intermediate values interpolate between the two. The significance of the additive form is that one signal need not vanish merely because the other is small. In particular, a candidate with high uncertainty but low feature novelty can still receive a large score if the current state favors investigation (Nguyen et al., 11 Mar 2026).

The paper states a formal proposition under the name The Density Veto. Consider a target $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$2 with high uncertainty $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$3 and low diversity $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$4, and a distractor $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$5 with lower uncertainty $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$6 but moderate diversity $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$7. Defining

$D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$8

the proposition shows that as the feature density around $D_{tr} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{k}$9 increases, so that $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$0, there exists a threshold $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$1 such that if $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$2, then the multiplicative selector strictly prefers the sub-optimal distractor $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$3, regardless of the uncertainty gap. By contrast, there always exists a weight $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$4 such that the additive selector prefers the high-uncertainty target $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$5 (Nguyen et al., 11 Mar 2026).

In the appendix proof, the multiplicative failure is expressed as

$D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$6

while the additive preference condition

$D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$7

can be rearranged to show that sufficiently small $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$8 makes the target preferable. The theoretical role of this result is not to establish a global optimality theorem for active learning, but to isolate a concrete failure mode of iGS and to justify the design choice of a tunable additive rule (Nguyen et al., 11 Mar 2026).

4. Weighting policies and reinforcement-learning adaptation

WiGS distinguishes three principal ways of setting $D_{cdd} = \{\mathbf{x}_j\}_{j=k+1}^{N}.$9: static, time-decay, and adaptive. The adaptive family is further divided into a discrete multi-armed bandit instantiation and a continuous-control Soft Actor-Critic instantiation.

Variant	Weight rule	Characteristic
Static WiGS	$f:\mathcal{X}\to\mathcal{Y}$0	Constant exploration–investigation balance
Time-decay WiGS	Linear or exponential decay	Dynamic but non-adaptive
Adaptive WiGS	MAB/UCB1 or SAC	Feedback-driven balance

In Static WiGS, the weight is fixed for all iterations. The paper gives examples such as $f:\mathcal{X}\to\mathcal{Y}$1, which is investigation-focused, and $f:\mathcal{X}\to\mathcal{Y}$2, which is exploration-focused. In Time-decay WiGS, the weight changes deterministically with iteration number, using schedules such as

$f:\mathcal{X}\to\mathcal{Y}$3

These schedules encode the hypothesis that exploration is more valuable early and investigation later, but they do not adapt to the actual state of the learner (Nguyen et al., 11 Mar 2026).

The adaptive versions cast weight selection as a reinforcement-learning problem. A key design choice is that the reward is computed only from the currently labeled set using cross-validation, rather than from the unlabeled pool, in order to avoid leakage and the active learning validation paradox. In both adaptive variants, the reward is

$f:\mathcal{X}\to\mathcal{Y}$4

so positive reward indicates improved cross-validated generalization performance after the latest query (Nguyen et al., 11 Mar 2026).

WiGS-MAB discretizes the action space to a small set of candidate weights, for example $f:\mathcal{X}\to\mathcal{Y}$5, and uses UCB1 to balance exploration of under-tested weights against exploitation of historically successful ones. The action space is intentionally coarse because a finer grid would consume too much of the labeling budget and the bandit does not generalize across nearby weights (Nguyen et al., 11 Mar 2026).

WiGS-SAC treats the weight as a continuous action: $f:\mathcal{X}\to\mathcal{Y}$6 Its state $f:\mathcal{X}\to\mathcal{Y}$7 is computed only from $f:\mathcal{X}\to\mathcal{Y}$8 and includes the current $f:\mathcal{X}\to\mathcal{Y}$9, the progress term $D_{tr}$0, and distributional statistics of the labeled set. The policy is a stochastic policy $D_{tr}$1 trained by Soft Actor-Critic (SAC). The entropy term in SAC is emphasized because it encourages stochasticity when the reward signal is ambiguous and reduces premature collapse to a brittle deterministic rule (Nguyen et al., 11 Mar 2026).

Operationally, the WiGS loop proceeds by training the current regressor on $D_{tr}$2, computing $D_{tr}$3, deriving the reward, selecting $D_{tr}$4 according to the chosen strategy, computing normalized feature and output distances, evaluating $D_{tr}$5 for every candidate, querying the highest-scoring point, and updating $D_{tr}$6 and $D_{tr}$7. In the SAC variant, state transitions are stored in a replay buffer $D_{tr}$8, and the policy is updated online (Nguyen et al., 11 Mar 2026).

5. Empirical behavior, evaluation protocol, and limitations

The evaluation uses 2 synthetic datasets and 18 real-world benchmarks. The real datasets include AutoMPG, Beer, Body Fat, Burbidge variants, Concrete variants, CPS, Housing, NO2, PM10, QSAR, Wine-Red, Wine-White, and Yacht. For each dataset, the protocol uses 100 independent replications, with an initial training set equal to 5% of the data, a candidate pool of 95%, and an active-learning run that continues until the pool is exhausted. The main predictor is Ridge Regression with $D_{tr}$9, and the experiments are repeated with Random Forest Regressor to examine whether the observed behavior is model-specific (Nguyen et al., 11 Mar 2026).

The baseline set includes Random Sampling, GSx, GSy, iGS, WiGS Static with $D_{cdd}$0, WiGS Static with $D_{cdd}$1, WiGS Linear Decay, WiGS Exponential Decay, WiGS-MAB, WiGS-SAC, Uncertainty Sampling, Query-by-Committee implemented via bagging, Expected Model Change Maximization, and EGAL. Evaluation is separated from the reinforcement-learning reward. Training feedback is based on cross-validated RMSE, whereas empirical comparison uses Full-Pool RMSE over the whole domain $D_{cdd}$2, together with the Area Under the Learning Curve (AUC), a relative AUC normalized by iGS, relative label efficiency $D_{cdd}$3 at 70% and 80% of total gain, full RMSE traces, correlation coefficient traces, and RMSE variance traces (Nguyen et al., 11 Mar 2026).

The synthetic environments are designed to expose the density-veto failure mode. In the two-regime construction, the feature distribution is a non-uniform Gaussian mixture,

$D_{cdd}$4

with regression function

$D_{cdd}$5

and heteroscedastic noise

$D_{cdd}$6

This creates a dense high-uncertainty noise trap near $D_{cdd}$7, precisely the regime in which iGS can suppress informative points and WiGS can adapt toward investigation (Nguyen et al., 11 Mar 2026).

The reported findings are consistent with this construction. On the two-regime synthetic task, GSx and GSy perform poorly, iGS improves on those single-objective baselines but still fails in the high-noise dense region, and WiGS-SAC significantly outperforms iGS; the pairwise Wilcoxon signed-rank tests are reported as statistically significant with $D_{cdd}$8. Across the benchmark suite, WiGS-SAC matches or outperforms iGS on 15 of 20 evaluated datasets total. Adaptive WiGS methods generally require fewer labels to reach a fixed fraction of the achievable gain, with WiGS-MAB yielding roughly a 4% label-cost reduction at the median in one summary, while WiGS-SAC is reported as especially robust with a narrower efficiency distribution than many baselines (Nguyen et al., 11 Mar 2026).

The method also has explicit limitations. WiGS-SAC has high computational cost, with the summary table reporting about $D_{cdd}$9 seconds for WiGS-SAC versus $\mathcal{X}$0 seconds for iGS. There is an RL warm-up period, so early iterations may underperform iGS when labeling budgets are very small. The adaptive policy is most useful when the budget is large enough for the learning cost to amortize. Finally, because the reward depends on cross-validation over the current labeled set, the feedback signal is only as informative as that set permits (Nguyen et al., 11 Mar 2026).

6. Antecedents, analogues, and terminological boundaries

Although WiGS in the strict sense denotes the regression active-learning method introduced in 2026, the underlying idea of using weights to reshape a greedy objective has notable antecedents. In weighted least-squares polynomial approximation, Christoffel-weighted approximate Fekete points define a deterministic greedy sampling procedure in which point selection is driven by a weighted Vandermonde determinant, with the weight given by the inverse Christoffel function $\mathcal{X}$1. The procedure is implemented through pivoted linear algebra, especially column-pivoted QR on the transpose of a weighted Vandermonde-like matrix, and the paper explicitly describes it as a direct precursor or near-equivalent formulation of the weighted greedy sampling idea (Guo et al., 2017).

In that setting, the weighted least-squares problem is

$\mathcal{X}$2

with diagonal weights

$\mathcal{X}$3

and a weighted polynomial space

$\mathcal{X}$4

The greedy point-selection rule maximizes the weighted determinant modulus, or equivalently pursues optimal conditioning. In one dimension, the method is reported to almost always generate optimally conditioned linear systems, and when optimal sets exist, the weighted determinant and weighted condition-number formulations coincide at the optimum (Guo et al., 2017). A plausible implication is that WiGS belongs to a broader family of weighted greedy designs in which the weighting scheme is used to stabilize or redirect the underlying selection criterion.

A second analogue appears in iteratively reweighted greedy set cover, a heuristic for large sparse set cover instances. There the weighted greedy algorithm is used as an inner solver, and element weights are updated multiplicatively whenever a $\mathcal{X}$5-cover attempt leaves some elements uncovered. The procedure begins with

$\mathcal{X}$6

runs weighted greedy set cover, and if an uncovered element $\mathcal{X}$7 is found, updates

$\mathcal{X}$8

If all elements are covered, the solution is stored as the current best $\mathcal{X}$9 and the target cover size is tightened by $\mathbf{x}_n$0. The method is explicitly presented as a heuristic rather than an approximation theorem, with the practical properties that the implementation is trivial, there are essentially no tuning parameters beyond the growth factor $\mathbf{x}_n$1, and more iterations can only improve the result (Alexa, 2020). This is not WiGS in the active-learning sense, but it exhibits the same structural pattern of a greedy core wrapped in an outer weighting mechanism.

A further methodological analogue appears in greedy completion for weighted $\mathbf{x}_n$2-spanners, which begins from a sparse initial graph, identifies pairs with bad stretch, computes a minimal segmentation of a shortest path, and adds only the single-edge bottlenecks that violate the $\mathbf{x}_n$3-stretch condition. The paper explicitly notes a conceptual overlap with “improved greedy sampling” as an algorithmic philosophy—sparse initialization followed by selective greedy repair—but also states that it is not in the classical “sampling” sense (Tzalik, 17 Mar 2026).

These related formulations delimit the term. In contemporary usage, Weighted improved Greedy Sampling refers specifically to the active learning framework for regression that adaptively balances feature-space exploration and output-space investigation through a weighted additive selector and, in its most advanced form, reinforcement learning (Nguyen et al., 11 Mar 2026). The older weighted Fekete and iterative reweighting methods are best regarded as antecedents or analogues rather than instances of the same algorithm.