Adaptive Importance Weighting in ML

• Adaptive importance weighting is a method that dynamically updates sample weights to reflect target distributions and counteract sampling bias.
• It leverages techniques like density-ratio correction, hardness-aware loss modulation, and self-adaptive weighting in various machine learning and simulation contexts.
• This approach balances bias and variance while improving convergence stability, making it essential for robust performance under distribution shifts.

Adaptive importance weighting is the family of methods that learn or modify sample weights so that weighted objectives better reflect a target distribution, control sampling bias, stabilize optimization, or emphasize difficult yet consequential observations. In the broad survey formulation, importance weighting is “a fundamental procedure in statistics and machine learning that weights the objective function or probability distribution based on the importance of the instance in some sense,” and adaptive variants include density-ratio correction under distribution shift, inverse-probability correction under selective labeling, hardness-aware loss modulation, pointwise residual balancing in PDE solvers, and proposal adaptation in Monte Carlo methods (Kimura et al., 2024). Across these settings, the common mechanism is a dynamically updated weighting function coupled to a target criterion—risk correction, variance control, convergence balancing, or exploration efficiency—rather than a fixed ex ante reweighting scheme.

1. Conceptual scope and mathematical foundations

The classical importance-sampling identity rewrites an expectation under a target distribution using samples from a proposal distribution $q$ and a weight $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$, yielding

$$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$

In supervised learning, the same idea yields importance-weighted empirical risk minimization by replacing each loss term $\ell(h(\mathbf{x}_i),y_i)$ with $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$ (Kimura et al., 2024).

A central distinction in the literature is between distribution correction, sample reweighting, and loss weighting. Under covariate shift, the canonical correction is

$$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$

with $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$. By contrast, focal loss and other uncertainty- or hardness-aware schemes are importance weighting in the broader sense but are not strict density-ratio correction (Kimura et al., 2024). This distinction matters because the theoretical guarantees differ: unbiasedness attaches to correctly specified shift models, whereas optimization-oriented or difficulty-oriented weighting typically targets robustness or convergence rather than exact population correction.

Adaptive variants regularize or modify raw weights to manage instability. The survey describes Adaptive Importance Weighted ERM with

$$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$

and Relative Importance Weighted ERM with

$$w(\mathbf{x})=\frac{p_{te}(\mathbf{x})}{(1-\lambda)p_{tr}(\mathbf{x})+\lambda p_{te}(\mathbf{x})},$$

both of which trade bias for variance and numerical stability (Kimura et al., 2024). A recurring misconception is that adaptive importance weighting is synonymous with exact density-ratio estimation. The broader literature contradicts that identification: many methods are adaptive because they alter weights according to uncertainty, hardness, residual decay, or learned geometry rather than because they recover a literal density ratio (Kimura et al., 2024).

2. Distribution shift, misspecification, and dynamic reweighting

Under covariate shift, importance weighting is often motivated by the identity

$$R_{\mathrm{te}}(f) = \mathbb{E}_{(x,y)\sim p_{\mathrm{te}}}[\ell(f(x),y)] = \mathbb{E}_{(x,y)\sim p_{\mathrm{tr}}}[w(x)\,\ell(f(x),y)], \qquad w(x)=\frac{p_{\mathrm{te}}(x)}{p_{\mathrm{tr}}(x)}.$$

However, the need for the correction depends on both model class and specification. In importance-weighted kernel ridge regression, the decisive issue is whether the hypothesis class is misspecified: in the nonparametric, misspecified case, importance weighting can still be necessary because the approximation induced by the source distribution may differ from the target-distribution approximation (Gogolashvili et al., 2023). This directly qualifies the common claim that sufficiently flexible nonparametric models make importance weighting obsolete.

Dynamic importance weighting extends this logic to deep learning under joint distribution shift. Accelerated Dynamic Importance Weighting treats the test-to-training density ratio

$w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$0

as the ideal target, but estimates weights inside the training loop on transformed representations produced by the current network. Its central optimization update is

$w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$1

with warm-started projected gradient descent rather than solving a kernel mean matching subproblem to convergence in every mini-batch. The framework unifies KMM, KLIEP, LSIF, and Wasserstein-1-based estimators, and the paper proves convergence to a stationary point of the weighted training objective under Assumptions B.1–B.3, with an $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$2 iteration complexity statement (Fang et al., 25 May 2026). Empirically, the same work reports per-epoch time reductions of about 20–75% on Fashion-MNIST and over 95% on ImageNet-1K relative to DIW, while remaining competitive in predictive performance (Fang et al., 25 May 2026).

This suggests that adaptive weighting under distribution shift is increasingly treated as an inner-loop estimation problem coupled to representation learning, rather than as a one-shot pre-processing step. It also sharpens the bias–variance issue: exact density-ratio correction is statistically appealing, but dynamic and stabilized estimators are often designed to remain feasible under the optimization and systems constraints of modern models (Kimura et al., 2024).

3. Selective labeling, inverse-probability correction, and weight-aware optimization

In active learning, adaptive importance weighting appears as inverse-probability correction for biased label acquisition. Importance Weighted Active Learning observes an unlabeled point $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$3, queries its label with adaptive probability $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$4, and stores queried examples with weight $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$5: $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$6 The resulting weighted empirical loss

$w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$7

is an unbiased estimate of the true risk $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$8, since $w(\mathbf{x})=\pi(\mathbf{x})/q(\mathbf{x})$9 (0812.4952). The adaptivity is in $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$0, which depends on the current point and the current restricted hypothesis set $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$1. The main query rule sets

$$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$2

so disagreement in loss over the version space drives label acquisition (0812.4952).

The same paper emphasizes variance control: small $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$3 implies large weights $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$4, so consistency requires either a positive lower bound $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$5 or a refined query rule ensuring concentration. It proves a safety result for noisy, nonseparable settings and gives expected label complexity roughly

$$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$6

with $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$7 the generalized disagreement coefficient and $$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$8 the slope asymmetry of the loss (0812.4952). Here adaptive weighting is not a mechanism for emphasizing hard examples in the optimization sense; it is the algebraic correction that makes biased querying statistically valid.

A different but related issue arises once weighted examples enter online optimization. “Online Importance Weight Aware Updates” argues that the standard update

$$\hat{I}=\frac{1}{T}\sum_{t=1}^T w(\mathbf{x}_t)g(\mathbf{x}_t).$$9

is only a first-order approximation and becomes poor when the importance weight $\ell(h(\mathbf{x}_i),y_i)$0 is large (Karampatziakis et al., 2010). The proposed alternative defines an update along $\ell(h(\mathbf{x}_i),y_i)$1,

$\ell(h(\mathbf{x}_i),y_i)$2

where $\ell(h(\mathbf{x}_i),y_i)$3 satisfies

$\ell(h(\mathbf{x}_i),y_i)$4

This yields importance-invariant updates such that performing one update of weight $\ell(h(\mathbf{x}_i),y_i)$5 is equivalent to two composed updates of weights $\ell(h(\mathbf{x}_i),y_i)$6 and $\ell(h(\mathbf{x}_i),y_i)$7, and it gives closed forms for squared, hinge, quantile, logistic, and exponential losses (Karampatziakis et al., 2010). The paper’s safety property,

$\ell(h(\mathbf{x}_i),y_i)$8

formalizes the claim that such updates do not overshoot and reverse the sign of the residual (Karampatziakis et al., 2010). Together, these results show that adaptive importance weighting is not only about estimating the right weight, but also about using that weight in an update rule that respects curvature.

4. Pointwise self-adaptive weighting in physics-informed neural networks

In PINNs, adaptive importance weighting has been specialized to pointwise residual balancing. “Self-adaptive weighting and sampling for physics-informed neural networks” formulates a weighted loss

$\ell(h(\mathbf{x}_i),y_i)$9

with the normalization constraint $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$0, and updates each pointwise weight through an inverse residual decay rate

$w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$1

Reference weights are normalized IRDR values, then smoothed via

$w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$2

with $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$3 (Chen et al., 7 Nov 2025).

The motivation is explicitly convergence-rate balancing: some residuals shrink quickly, others decay slowly, and “the point with the smallest residual decay rate often dominates the convergence speed of the global solution.” The BRDR-based self-adaptive weighting scheme therefore gives larger weight to training points whose residuals decay slowly, especially in regions with sharp gradients, boundary layers, shocks, or singular structures (Chen et al., 7 Nov 2025).

This weighting is coupled to residual-based adaptive sampling, where candidate points are sampled with probability proportional to squared residual,

$w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$4

after clipping

$w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$5

with $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$6 by default. New sampled points inherit weights and running statistics by inverse distance weighting interpolation, and partial replacement is scheduled every $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$7 steps with fraction $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$8 (Chen et al., 7 Nov 2025).

The empirical picture is deliberately non-universal. Adaptive weighting alone and adaptive sampling alone are reported as problem-dependent and sometimes insufficient. Their combination is consistently best across the perturbation problem, Allen–Cahn, Burgers, and lid-driven cavity flow. On the perturbation problem, the combined method achieves $w(\mathbf{x}_i)\ell(h(\mathbf{x}_i),y_i)$9 relative $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$0 error with 2048 residual points, compared with about $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$1 for the standard PINN and $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$2 for the cited ensemble baseline (Chen et al., 7 Nov 2025). The same work reports less than 20% training-time overhead relative to the non-adaptive PINN under the same batch size and iteration budget, but substantially less time to reach a target error (Chen et al., 7 Nov 2025).

5. Adaptive weighting in self-training and preference optimization for LLMs

Recent LLM work uses adaptive importance weighting to control synthetic-data drift rather than only covariate mismatch. “Importance Weighting Can Help LLMs Self-Improve” argues that self-improvement pipelines should suppress not only incorrect samples but also correct samples with high distribution shift extent relative to a human-written valid set (Jiang et al., 2024). The paper defines a naive sample-wise surrogate

$$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$3

and then symmetrizes it into a distribution shift weight $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$4 whose deviation from 1 measures mismatch in either direction. The final method uses self-consistency first and DS-weight ranking second, retaining the lowest-shift fraction of generated samples rather than directly weighting the loss (Jiang et al., 2024).

The empirical results are framed as filtering rather than exact reweighting. Using only a tiny valid set of up to 5% of the training-set size, IWSI reaches 40.4 average accuracy across six tasks, compared with 34.0 for LMSI, 35.2 for Entropy-filter, and 38.7 for Self-filter, and matches RM-filter on average without using an external reward model (Jiang et al., 2024). The same paper states that the weighted-loss variant IWSI-w is consistently worse than IWSI, explicitly because the DS estimate is intentionally naive (Jiang et al., 2024). This is a concrete example where the importance-weighting viewpoint is retained, but the operational use is ranking and truncation rather than continuous loss scaling.

In self-play fine-tuning, Team-based self-Play with dual Adaptive Weighting introduces two adaptive weights: a target-response reweighting

$$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$5

and a main-player weighting

$$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$6

where $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$7 is the preference margin (Li et al., 11 May 2026). The first counteracts decline in target-response likelihood; the second emphasizes weaker historical players in the team-based scoring ensemble. The full objective is

$$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$8

On Qwen2.5-1.5B and Llama3.1-8B, the paper reports consistent gains over SFT and SPIN on Open LLM Leaderboard V1, V2, and GSM8K, with best settings $$\mathbb{E}_{p_{tr}(\mathbf{x},y)}\!\left[\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\ell(h(\mathbf{x}),y)\right] = \mathbb{E}_{p_{te}(\mathbf{x},y)}\!\left[\ell(h(\mathbf{x}),y)\right],$$9, $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$0, and $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$1 in the main experiments (Li et al., 11 May 2026).

A plausible implication is that adaptive importance weighting in LLM alignment is increasingly functioning as a stability device for recursive training: it is used to modulate synthetic samples, targets, and model snapshots according to current usefulness, rather than solely to estimate a test-to-train density ratio.

6. Adaptive importance sampling, variance control, and proposal learning

In Monte Carlo, adaptive importance weighting is inseparable from proposal adaptation and degeneracy control. “Asymptotic optimality of adaptive importance sampling” shows that AIS can be asymptotically optimal, matching the oracle proposal’s asymptotic variance, and introduces weighted AIS

$$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$2

with stage weights chosen inversely to a proposal-quality proxy,

$$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$3

so that poor early stages are downweighted (Delyon et al., 2018).

Several later methods regularize the weights themselves. Safe and Regularized Adaptive Importance Sampling replaces raw weights $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$4 with powered weights $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$5, $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$6, establishing

$$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$7

and connecting the update

$$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$8

to entropic mirror descent (Korba et al., 2021). Robust Covariance Adaptation in Adaptive Importance Sampling instead gates covariance updates by ESS, using raw weights for mean adaptation and transformed weights for covariance when local ESS is below threshold $$w(\mathbf{x})=p_{te}(\mathbf{x})/p_{tr}(\mathbf{x})$$9 (El-Laham et al., 2018). These are distinct regularization strategies: one tempers weights globally, the other selectively transforms weights only for unstable proposal-parameter updates.

Proposal families have also been generalized. “Variational autoencoder with weighted samples for high-dimensional non-parametric adaptive importance sampling” trains a VAE proposal with a weighted ELBO,

$$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$0

thereby learning a flexible proposal from weighted samples rather than from unweighted data (Demange-Chryst et al., 2023). “Implicitly Adaptive Importance Sampling” avoids explicit proposal fitting altogether by affine moment matching of existing samples to weighted moments, guided by the Pareto $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$1 diagnostic, and applies this to Bayesian LOO-CV (Paananen et al., 2019). PARIS constructs a sample-centered Gaussian mixture proposal whose component weights equal current importance weights and reweights all samples via deterministic-mixture denominators, enabling self-correcting exploration of high-dimensional multimodal posteriors (Liu et al., 22 Mar 2026).

The main controversy in this branch is not whether weighting is needed, but how much of the raw weight signal should survive variance control. The literature includes discarding-re-weighting in AMIS, weight clipping, tempering, power regularization, ESS-gated covariance learning, and stagewise forgetting, all of which are attempts to preserve asymptotic correctness while avoiding catastrophic finite-sample degeneracy (Thijssen et al., 2018).

7. Adaptive balancing, causal estimation, and set-wise accuracy

In causal inference, adaptive importance weighting often appears as balancing rather than explicit propensity inversion. “Data adaptive covariate balancing for causal effect estimation for high dimensional data” proposes a random-forest-induced kernel based on the probability that two observations fall in the same leaf of a forest fit to the bivariate response $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$2. The resulting kernel is used in an MMD objective,

$$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$3

subject to

$$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$4

so that treated and control groups are balanced to the full sample in a confounder-adaptive geometry (De et al., 19 Dec 2025).

The paper’s key asymptotic claim is that, under SUTVA, no unmeasured confounding, positivity, a $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$5-Donsker RKHS unit ball, continuity of $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$6, and universality of the kernel, the estimated balancing weights converge in $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$7 to normalized inverse propensity score weights: $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$8 as $$w(\mathbf{x})=\left\{\frac{p_{te}(\mathbf{x})}{p_{tr}(\mathbf{x})}\right\}^{\lambda},\quad \lambda\in[0,1],$$9 (De et al., 19 Dec 2025). The method is therefore adaptive in finite samples through the forest-induced similarity, while asymptotically targeting the same ideal weights as IPW.

A complementary computational perspective appears in “Multicalibrated Partitions for Importance Weights,” which studies estimation of $$w(\mathbf{x})=\frac{p_{te}(\mathbf{x})}{(1-\lambda)p_{tr}(\mathbf{x})+\lambda p_{te}(\mathbf{x})},$$0 from samples and argues that global multi-accuracy is insufficient for set-wise guarantees (Gopalan et al., 2021). The paper proposes sandwiching bounds and constructs partition-based weights $$w(\mathbf{x})=\frac{p_{te}(\mathbf{x})}{(1-\lambda)p_{tr}(\mathbf{x})+\lambda p_{te}(\mathbf{x})},$$1 on cells $$w(\mathbf{x})=\frac{p_{te}(\mathbf{x})}{(1-\lambda)p_{tr}(\mathbf{x})+\lambda p_{te}(\mathbf{x})},$$2 of a learned multicalibrated partition. This provides set-wise completeness and soundness guarantees over a class $$w(\mathbf{x})=\frac{p_{te}(\mathbf{x})}{(1-\lambda)p_{tr}(\mathbf{x})+\lambda p_{te}(\mathbf{x})},$$3, rather than only global moment matching (Gopalan et al., 2021). This suggests that adaptive importance weighting in high-dimensional observational problems is increasingly mediated by learned geometry—RF kernels, MMD objectives, or multicalibrated partitions—rather than by a single parametric score model.

Taken together, these strands show that adaptive importance weighting is not a single algorithmic motif but a unifying principle: weights are made data-dependent, state-dependent, or optimizer-dependent so that a weighted empirical object better approximates a desired target under constraints imposed by bias, variance, optimization stability, or computational tractability.