Importance Reweighting in ML

• Importance reweighting effect is a technique that replaces uniform empirical averages with weighted ones to correct bias under distribution shifts.
• It is applied in varied domains such as covariate/label shift, off-policy reinforcement learning, and deep learning to adjust optimization dynamics.
• Dynamic reweighting influences early training trajectories in deep networks, with effects that diminish under exponential loss tails unless regularization alters convergence.

The importance reweighting effect denotes the change in an estimator, predictor, or sampling procedure induced by replacing a uniform empirical average with a weighted one. In its basic supervised form, empirical risk minimization replaces

$$\hat R(\theta)=\frac{1}{n}\sum_{i=1}^n \ell(f_\theta(x_i),y_i)$$

by

$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$

where the weights may encode density ratios, class imbalance, inverse propensities, validation-derived sample influence, or dynamically estimated importance under a changing policy or proposal distribution (Byrd et al., 2018). Across modern machine learning, the effect is not singular: it can correct sampling bias, change finite-sample optima, alter optimization trajectories, reduce or inflate variance, redistribute gradient mass over data points or tokens, and, in over-parameterized deep networks trained to near-zero training error, largely disappear at convergence unless the loss, architecture, or regularization changes the asymptotic dynamics (Byrd et al., 2018, Wang et al., 2021, Vo, 6 May 2025).

1. Formal foundations

Under distribution shift, importance weighting is introduced by rewriting a target expectation under a source distribution. For covariate shift, with $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$, the target risk

$$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$

can be written as

$$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$

For label shift, the weight reduces to a class-prior ratio $w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$. Analogous density-ratio constructions appear in sample-bias correction, noisy-label correction, and off-policy reinforcement learning (Mathelin et al., 2022, Liu et al., 2014, Asis et al., 2023).

This reweighting is unbiased at the population level when the relevant structural assumption is correct and the density ratio is known. In sample-bias correction, $w(x)=p_t(x)/p_s(x)$ converts source risk into target risk under covariate shift (Mathelin et al., 2022). In spatial modeling, the target error

$R(e,p)=\int_X e(x)p(x)\,dx$

is estimated from biased spatial samples $x_i\sim g(x)$ by

$$R_I(e,p,g)=\frac{1}{n}\sum_{i=1}^n e(x_i)\frac{p(x_i)}{g(x_i)},$$

which is unbiased for $$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$0 when $$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$1 is known and support overlap holds (Prokhorov et al., 2023). In noisy-label classification, the clean-label risk is rewritten under the corrupted distribution using

$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$2

so that arbitrary surrogate losses can be used with noisy labels via samplewise reweighting (Liu et al., 2014).

The main statistical tension is classical: weighting can remove bias with respect to a target distribution or target objective, but it can also increase variance when the weights are heavy-tailed or poorly estimated. Several papers therefore treat the importance reweighting effect not as a purely unbiasedness phenomenon, but as a bias-variance and optimization trade-off whose practical behavior depends on model class, loss tail, regularization, and the procedure used to estimate or adapt the weights (Lu et al., 2021, Sow et al., 10 Feb 2025).

2. Principal regimes of the effect

The same algebraic device produces different empirical effects across domains.

Setting	Weight definition	Characteristic effect
Covariate or label shift	$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$3 or class-prior ratio	Bias correction, often with variance inflation
Noisy labels / biquality	$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$4 or validation-derived sample weights	Downweights corrupted or harmful samples
Off-policy RL / replay	$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$5, value-aware weights, or $$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$6	Off-policy correction or concentration on on-policy states
Adaptive importance sampling	$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$7 under evolving proposal mixtures	Exploration via temporary overweighting, then self-correction
LLM pretraining / unlearning	Loss-dependent or tokenwise weights	Redistributes gradient mass over tokens or samples

In transfer learning, classical importance-weighted ERM is asymptotically correct under the right shift assumptions, but finite-sample performance is often dominated by instability of ratio estimation and weight variance. This motivates flattened or relative weighting, one-step joint estimation of predictor and weights, and dynamic reweighting strategies that are optimized for prediction rather than ratio estimation alone (Lu et al., 2021). In weighted ERM theory, the effect can also be stated through conditional risk bounds: under a general “balanceable” Bernstein condition, a data-dependent weight can improve error bounds in specific sub-regions, with the gain appearing through a data-dependent constant term rather than a universal improvement everywhere (Zhang et al., 4 Jan 2025).

3. Deep learning, separability, and implicit bias

The most prominent modern controversy concerns over-parameterized deep classifiers trained with cross-entropy or logistic loss. On separable data, linear and deep-linear analyses imply that gradient descent drives the parameter norm to infinity while the normalized direction converges to a max-margin solution; in that regime, the final decision boundary can become effectively weight-agnostic (Byrd et al., 2018). Empirically, this appears in logistic regression, MLPs, CNNs, ResNets, and BERT fine-tuning: class weights affect predictions early in training, but their influence shrinks over successive epochs, and models trained with widely different class weights often end with very similar decision boundaries and classification ratios (Byrd et al., 2018).

This fading effect is not tied only to over-parameterization. A later analysis argues that the decisive factor is the tail of the loss. With exponentially tailed losses such as logistic or cross-entropy, gradient descent on separable data converges to a weight-independent maximum-margin direction, so importance weighting has little asymptotic effect. With polynomially tailed losses, the limiting direction instead solves a weighted inverse-margin optimization problem, and the weights continue to shape the interpolating classifier (Wang et al., 2021). In the label-shift analysis of that work, weights obtained by exponentiating the classical unbiased importance weights can improve performance, and neural-network experiments report reweighted polynomially tailed losses outperforming reweighted cross-entropy by as much as $$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$8 in test accuracy (Wang et al., 2021).

Recent experiments on synthetic 2D data and CIFAR-10 largely reinforce the earlier deep-learning picture. On linearly separable and moon-shaped datasets using logistic regression and MLPs, weighting strongly affects decision boundaries early in training but fades with prolonged optimization. On CIFAR-10 with various class imbalances, only L2 regularization, not dropout, helps preserve weighting effects; in a covariate-shift experiment, importance weighting yields no significant performance gain (Vo, 6 May 2025). This suggests that the importance reweighting effect in deep learning is fundamentally dynamic: it is strong on the optimization path, weak in many asymptotic cross-entropy regimes, and recoverable when the loss tail or explicit norm control changes the implicit bias.

A related but distinct perspective appears in weighted kernel ridge regression under dataset shift. There, importance weighting is not uniformly beneficial across shift types: under bounded weights and well-specified models, minimax optimal rates are achievable under both covariate and target shift, but with relaxed boundedness conditions the rates can remain optimal under target shift while slowing under covariate shift; under misspecification, alternative reweighting can outperform standard importance weighting under covariate shift (Gogolashvili, 2022). Although the available record for that work is indirect, it reinforces the broader point that the effect of weighting is highly regime-dependent.

4. Sequential decision making and adaptive sampling

In off-policy reinforcement learning, the canonical importance weight is the policy ratio

$$\hat R_{\mathrm{IW}}(\theta)=\frac{1}{n}\sum_{i=1}^n w_i\,\ell(f_\theta(x_i),y_i),$$9

which gives unbiased one-step and trajectory-level estimators under a target policy $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$0 using data from a behavior policy $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$1 (Asis et al., 2023). The difficulty is extreme variance, especially for multi-step products. One response is to enlarge the space of admissible weights. Value-aware importance weighting defines weights $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$2 by requiring unbiasedness for value-based expectations together with a mean-one constraint, and then choosing the minimum-variance solution. In the one-state analysis, the resulting closed form depends on $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$3, $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$4, $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$5, and $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$6, and the variance is provably no larger than that of standard importance sampling in the feasible set (Asis et al., 2023). The effect is therefore not only reweighting for distribution correction, but reweighting for variance-optimal estimation of the specific functional of interest.

Experience replay produces another version of the effect. A replay buffer contains data from many past policies, whereas the critic in actor-critic methods should be most accurate under the current policy’s stationary distribution $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$7. Reweighting TD loss terms by an estimate of $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$8, where $$p_{\mathrm{train}}(y\mid x)=p_{\mathrm{test}}(y\mid x)$$9 is the replay distribution, emphasizes small approximation error on frequently encountered states and actions. Using a likelihood-free density-ratio estimator with fast and slow replay buffers, this approach improves sample complexity for SAC and TD3 on multiple OpenAI Gym tasks, and yields markedly higher correlation between learned $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$0-values and Monte Carlo returns on on-policy data (Sinha et al., 2020).

Adaptive importance sampling turns the importance reweighting effect into an explicit feedback loop. In Parallel Adaptive Reweighting Importance Sampling, samples are drawn from a sample-centered Gaussian mixture whose component centers are existing samples and whose component weights match current importance weights. When a point lands in a region with low proposal density but non-negligible target density, its initial weight is large; that weight then increases proposal mass in that region; once more neighbors are collected, deterministic-mixture-style reweighting raises the denominator and the original weight is reduced (Liu et al., 22 Mar 2026). The paper explicitly describes this as a dynamic balance between exploratory over-weighting and later self-correction. In that setting, the importance reweighting effect is not the disappearance of weighting but its continual reuse to drive exploration, proposal adaptation, and eventual stabilization.

The same principle appears in sensitivity analysis. For Sobol’ indices under a reference distribution $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$1, samples may be drawn from another distribution $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$2 and reweighted by $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$3. The target index remains unchanged while the asymptotic variance changes, and the optimal proposal is the one that minimizes the weighted asymptotic variance. In deterministic cases, the paper derives a zero-variance optimal proposal; in realistic settings, the framework also enables reverse importance sampling, allowing Sobol’ indices under many alternative input distributions to be computed from a single baseline sample (Boucharif et al., 8 Jul 2025). Here the effect is variance control rather than distributional robustness of a predictor.

5. Noisy labels, validation quality, and transfer learning

Under random classification noise, importance weighting recovers the clean-label risk from corrupted observations. With asymmetric noise rates $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$4 and $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$5, the weight

$$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$6

transforms the noisy empirical risk into the clean surrogate risk, so any surrogate loss can be used with consistency guarantees for the clean Bayes-optimal classifier (Liu et al., 2014). The same paper shows that the noise rates are upper bounded by $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$7 and can be recovered under mild identifiability conditions, making the weighting scheme implementable rather than purely oracle-based.

Biquality learning extends this logic to settings with a small trusted dataset $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$8 and a larger untrusted dataset $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{test}}}[\ell(f_\theta(x),y)]$$9. There the key weight is

$$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$0

estimated by training one probabilistic model on $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$1 and another on $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$2, and assigning weight $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$3 to trusted examples and $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$4 to untrusted ones (Nodet et al., 2020). The effect is selective reuse of noisy data: noncorrupted or trusted-consistent untrusted examples retain substantial weight, whereas corrupted examples are pushed toward zero. Across NCAR and NNAR noise settings, this produces better average accuracy than trusted-only, untrusted-only, mixed training, robust symmetric-loss baselines, and Gold Loss Correction (Nodet et al., 2020).

Interactive reweighting studies the same phenomenon when the validation set itself is imperfect. In validation-driven meta-reweighting, per-training-sample weights are chosen to improve validation performance, but if validation labels are noisy or unrepresentative then the resulting weights can be systematically wrong. Reweighter decomposes the effect into a validation–training bipartite influence graph with entries $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$5, introduces validation weights $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$6, and models the effective training weights as

$$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$7

This makes the reweighting effect inspectable and editable: correcting validation labels, adding representative validation samples, and imposing constraints on $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$8 can improve correctness and balancedness of the resulting training weights, with gains of $$R_{\mathrm{test}}(\theta)=\mathbb{E}_{(x,y)\sim p_{\mathrm{train}}}\!\left[\frac{p_{\mathrm{test}}(x)}{p_{\mathrm{train}}(x)}\,\ell(f_\theta(x),y)\right].$$9–$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$0 points on Clothing and OCT where validation issues are severe (Yang et al., 2023).

Transfer learning provides a broader synthesis. Classical importance-weighted ERM is consistent under covariate shift, but in modern high-dimensional settings the two-step procedure “estimate weights, then train predictor” is often too brittle. Recent work therefore emphasizes flattened or relative weights, one-step upper bounds that jointly optimize predictor and importance model, and dynamic schemes in which the importance weights are updated together with the classifier, sometimes even in loss space rather than input space (Lu et al., 2021). This reframes the importance reweighting effect as a property of coupled learning dynamics, not merely of density-ratio estimation.

6. LLMs, conditional risk, and open problems

In LLM pretraining, dynamic loss-based sample reweighting changes the effective optimization problem by replacing the uniform batch average with

$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$1

where the weights are computed online from current losses and normalized within the minibatch (Sow et al., 10 Feb 2025). Theoretical analysis introduces correction terms such as

$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$2

showing that downweighting low-loss samples can tighten convergence bounds, while weighting schemes that emphasize already-solved examples loosen them (Sow et al., 10 Feb 2025). Empirically, the best-performing strategy deprioritizes redundant or uninformative data, and yields faster convergence and improved performance in pretraining 7B and 1.4B parameter LLMs (Sow et al., 10 Feb 2025).

In LLM unlearning, loss reweighting is analyzed at token level through two distinct criteria: Saturation and Importance. Saturation-based weighting

$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$3

emphasizes high-likelihood tokens that remain strongly memorized, whereas importance-based weighting emphasizes annotated key tokens or low-likelihood tokens through surrogates such as

$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$4

The paper finds that Saturation enhances efficacy more than importance-based reweighting, that the two are complementary, and that smoothness and granularity of the weight distribution strongly influence retention and unlearning. The proposed combination,

$w(y)=p_{\mathrm{test}}(y)/p_{\mathrm{train}}(y)$5

emphasizes intermediate-probability tokens and improves the trade-off between forgetting and retention on TOFU and WMDP benchmarks (Yang et al., 17 May 2025).

Weighted ERM theory provides a complementary viewpoint. Under a “balanceable” Bernstein condition, one can choose data-dependent weights so that the variance term is normalized and conditional excess-risk bounds improve on specific sub-regions, notably large-margin regions in classification and low-variance regions in heteroscedastic regression (Zhang et al., 4 Jan 2025). This does not claim universal improvement of the global risk; instead, it shows that the importance reweighting effect can be localized and structurally beneficial where the problem geometry makes balancing possible.

Across these literatures, several misconceptions are explicitly challenged. One is that specifying correct density-ratio weights automatically yields the desired final classifier in deep learning; long-horizon optimization with cross-entropy may erase that effect (Byrd et al., 2018). A second is that unbiased weights are necessarily the best practical weights; polynomially tailed losses can favor exponentiated weights, and flattened or relative weighting can outperform exact ratios when variance dominates (Wang et al., 2021, Lu et al., 2021). A third is that automatic sample reweighting is self-sufficient; when validation quality is poor, the induced weights can amplify noise rather than suppress it (Yang et al., 2023).

Open questions remain consistent across the surveyed work. For nonlinear deep networks, formal theorems matching the empirical “washout” of weights are still limited (Byrd et al., 2018). For approximate adaptive reweighting schemes such as PARIS, rigorous convergence and bias analysis remains open (Liu et al., 22 Mar 2026). In LLM pretraining and unlearning, the interaction among loss-dependent weights, optimizer dynamics, and implicit regularization is only partly characterized (Sow et al., 10 Feb 2025, Yang et al., 17 May 2025). These unresolved issues indicate that the importance reweighting effect is best understood not as a single theorem, but as a family of phenomena linking weighting, variance, optimization, and inductive bias across modern statistical learning.