Dynamic Uncertainty-driven Sample Expansion (DUSE)

• DUSE is a methodological framework that leverages iterative uncertainty estimation to select and incorporate valuable samples from auxiliary data pools.
• It dynamically integrates real labeled data, pseudo queries, and semantic transformations to enhance training signals beyond traditional augmentation.
• Empirical results in automatic modulation recognition, information retrieval, biomedical learning, and reinforcement learning demonstrate notable gains over standard baselines.

Searching arXiv for the cited papers to ground the article in current arXiv records. Dynamic Uncertainty-driven Sample Expansion (DUSE) denotes a class of iterative training procedures in which uncertainty estimates determine which samples, documents, exemplars, or imagined rollouts should be added, emphasized, or trusted as the model evolves. In its explicit named form, DUSE is a data expansion framework for low-resource automatic modulation recognition (AMR) that imports labeled auxiliary signals into a scarce target domain via active learning (Lu et al., 16 Jul 2025). In the broader syntheses provided for later work, the same design pattern is instantiated by Uncertainty-based Iterative Document Sampling (UnIte) for unsupervised domain adaptation in information retrieval, by uncertainty-guided exemplar selection plus semantic expansion for biomedical class-incremental learning, and, in a softer sense, by uncertainty-weighted value expansion in reinforcement learning (Kim et al., 28 Apr 2026, Yao et al., 2024, Zhou et al., 2019).

1. Definition and conceptual scope

DUSE is organized around a recurring bottleneck: the learner cannot afford to use every potentially useful sample, and the utility of expansion depends more on sample choice than on raw quantity. In low-resource AMR, the bottleneck is a small labeled target set $D_T$ and a much larger labeled auxiliary pool $\mathcal{D}_A$ from which only a budgeted subset can be imported (Lu et al., 16 Jul 2025). In neural information retrieval, the bottleneck is pseudo-query generation over large unlabeled corpora, where only sampled documents receive synthetic supervision (Kim et al., 28 Apr 2026). In biomedical class-incremental learning, the bottleneck is the retention of prior classes under severe class imbalance and limited new-class samples, which makes exemplar selection and class-wise expansion central (Yao et al., 2024). In model-based reinforcement learning, the bottleneck is not label scarcity but whether imagined rollouts should be trusted under stochastic dynamics and model error (Zhou et al., 2019).

A common misconception is to equate DUSE with conventional data augmentation. The AMR formulation explicitly distinguishes the two: DUSE expands the training set by importing labeled, real samples from a large, related dataset, whereas conventional augmentation synthetically alters existing samples and “does not introduce new data” (Lu et al., 16 Jul 2025). The biomedical formulation broadens the picture by combining uncertainty-driven exemplar selection with semantic-preserving augmentations, so expansion can mean both adding retained exemplars and densifying class manifolds through controlled transformations (Yao et al., 2024). The reinforcement-learning interpretation extends the term further: here “expansion” refers to imagined value rollouts, and uncertainty does not necessarily change sample count but changes how much expanded targets influence learning (Zhou et al., 2019).

This suggests that DUSE is best understood as a methodological template rather than a single architecture. Its invariant elements are iterative uncertainty estimation, budgeted selection or weighting, expansion of the effective training signal, and repeated model updates that alter later uncertainty estimates.

2. Uncertainty mechanisms and selection rules

The uncertainty signal in DUSE-style methods is domain-specific. In AMR, the scorer is the top-2 softmax margin

$u = p_1^{*} - p_2^{*},$

where smaller $u$ indicates higher uncertainty and therefore higher informativeness for expansion from the auxiliary pool (Lu et al., 16 Jul 2025). The operational interpretation is decision-boundary proximity: samples with small margins are closer to class boundaries and are more useful for refining the classifier.

UnIte for information retrieval explicitly separates aleatoric uncertainty (AU) from epistemic uncertainty (EU). AU is approximated by a model-agnostic lexical density proxy based on BM25 $k$-NN distance,

$$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$

with $\epsilon = 10^{-6}$ and $k=3$, followed by modified z-score filtering

$$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$

with $z_{\mathrm{thr}} = 1.5$ (Kim et al., 28 Apr 2026). EU is then defined by the mismatch between target-domain token importance and the model’s token predictions through the MLM head:

$\mathcal{D}_A$0

High-EU documents are those for which the current retriever fails to predict high-IDF domain terms, making EU a model-conditioned measure of current knowledge gaps rather than a static property of the corpus (Kim et al., 28 Apr 2026).

In biomedical class-incremental learning, uncertainty is not taken from a single epoch. The method tracks the entropy trajectory

$\mathcal{D}_A$1

and uses cumulative or average entropy over training,

$\mathcal{D}_A$2

to rank candidate exemplars (Yao et al., 2024). The paper argues that final-epoch entropy misses how long and how strongly the model struggled with a sample; cumulative entropy is intended to summarize both immediate and persistent influence on optimization.

In RAVE, uncertainty is distributional and decomposed into epistemic spread across ensemble members plus aleatoric variance within each model. For a function output $\mathcal{D}_A$3,

$\mathcal{D}_A$4

These estimates feed a lower confidence bound

$\mathcal{D}_A$5

and inverse-variance horizon weights $\mathcal{D}_A$6 (Zhou et al., 2019). Unlike the AMR or IR cases, uncertainty here does not select external samples; it modulates how much imagined expansion at each rollout horizon should contribute.

Across these formulations, uncertainty is not a monolithic scalar. It may be margin-based, entropy-trajectory-based, density-based, domain-aware lexical, or distributional over model rollouts. What remains constant is that uncertainty governs the marginal utility of expansion.

3. Canonical named framework in low-resource automatic modulation recognition

The paper “DUSE: A Data Expansion Framework for Low-resource Automatic Modulation Recognition based on Active Learning” formalizes the clearest named instance of the concept (Lu et al., 16 Jul 2025). The target-domain dataset is a small labeled set $\mathcal{D}_A$7, while a large labeled auxiliary dataset

$\mathcal{D}_A$8

serves as the source of imported examples. Under a budget $\mathcal{D}_A$9, the objective is to choose $u = p_1^{*} - p_2^{*},$0 so that the augmented set $u = p_1^{*} - p_2^{*},$1 minimizes classification error under cross-entropy training (Lu et al., 16 Jul 2025).

The algorithm is iterative. At round $u = p_1^{*} - p_2^{*},$2, the AMR model is trained on the current target set, every remaining auxiliary sample is scored by the softmax-margin uncertainty, the $u = p_1^{*} - p_2^{*},$3 most uncertain samples are moved into the target set, and the scorer is retrained on the updated data. The selected set after the first round is

$u = p_1^{*} - p_2^{*},$4

with updates

$u = p_1^{*} - p_2^{*},$5

After $u = p_1^{*} - p_2^{*},$6 rounds, the final expanded set is $u = p_1^{*} - p_2^{*},$7 (Lu et al., 16 Jul 2025).

The experimental protocol uses a 2D-CNN as the default scorer and evaluator on I/Q time-domain inputs, with cross-architecture evaluation on 1D-CNN, AlexNet, SigNet, GRU, and MCLDNN. Training uses learning rate $u = p_1^{*} - p_2^{*},$8, batch size $u = p_1^{*} - p_2^{*},$9, $u$0 epochs per expansion round, and $u$1 epochs for final evaluation. Experiments are conducted on RML2016.10a, Sig2019-12, and RML2018.01a, using only samples with $u$2 dB; $u$3 is a random $u$4 class-balanced subset of the training split and $u$5 is the remaining $u$6 (Lu et al., 16 Jul 2025).

The main empirical claim is that DUSE consistently outperforms eight coreset selection baselines in both class-balance and class-imbalance settings. Under class-imbalanced selection on RML2016.10a, DUSE reaches $u$7, $u$8, $u$9, $k$0, and $k$1 accuracy at expansion rates $k$2; after the smallest budget, gains over the best baseline are reported as $k$3, $k$4, $k$5, and $k$6 (Lu et al., 16 Jul 2025). On Sig2019-12, DUSE is lower than some baselines at $k$7, approximately at parity at $k$8, and then matches or exceeds Forgetting at larger budgets. On RML2018.01a, DUSE is better up to $k$9, but Forgetting exceeds DUSE by about $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$0 at $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$1, showing that the method is not uniformly dominant at all budget regimes (Lu et al., 16 Jul 2025).

The active-learning loop is critical. At $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$2, removing active learning reduces performance from $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$3 to $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$4 on RML2016.10a, from $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$5 to $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$6 on Sig2019-12, and from $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$7 to $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$8 on RML2018.01a (Lu et al., 16 Jul 2025). Cross-architecture generalization is also substantial: using a dataset expanded by 2D-CNN scoring at $$D_k(d) = \frac{1}{\epsilon + \mathrm{BM25}(d, n_k)},$$9 on Sig2019-12, SigNet achieves $\epsilon = 10^{-6}$0, AlexNet $\epsilon = 10^{-6}$1, MCLDNN $\epsilon = 10^{-6}$2, 1D-CNN $\epsilon = 10^{-6}$3, and GRU $\epsilon = 10^{-6}$4 (Lu et al., 16 Jul 2025).

Within the AMR setting, DUSE is therefore a budgeted, active, uncertainty-driven import mechanism over real auxiliary data rather than a synthetic data generator. Its limitations are explicit: dependence on a labeled and relevant auxiliary pool, sensitivity at very low budgets, high-budget saturation on some datasets, and unreported behavior under very low SNR (Lu et al., 16 Jul 2025).

4. UnIte as a DUSE-style pipeline for information retrieval domain adaptation

UnIte operationalizes the DUSE pattern for unsupervised domain adaptation of neural retrievers by deciding which target-domain documents should receive pseudo queries and in what order (Kim et al., 28 Apr 2026). The pipeline begins with AU filtering to remove low-density lexical outliers, then enters an iterative sampling-training loop in which EU is estimated under the current retriever, sampling budget is allocated across clusters with a resampling penalty, documents are ranked by a balance of EU and diversity, pseudo queries are generated for the selected batch, the retriever is fine-tuned, and EU is re-estimated for the next round (Kim et al., 28 Apr 2026).

Two design decisions are central. First, the method separates AU from EU rather than using a single uncertainty measure. The AU stage filters approximately $\epsilon = 10^{-6}$5–$\epsilon = 10^{-6}$6 of off-topic, noisy documents, with the stated purpose of avoiding negative transfer and wasted pseudo-query budget. Second, EU is dynamic: because $\epsilon = 10^{-6}$7 changes as the retriever adapts, a document’s EU can fall after the model learns domain-important terms from similar regions of the corpus. This justifies re-estimation after every training round rather than one-shot selection (Kim et al., 28 Apr 2026).

Selection is not pure uncertainty sampling. Within each semantic cluster, documents are ranked by

$\epsilon = 10^{-6}$8

with $\epsilon = 10^{-6}$9, where $k=3$0 is DUQGen’s diversity score and $k=3$1 denotes z-score normalization. Maximal Marginal Relevance is used inside clusters, and cluster budgets are adjusted by a resampling penalty

$k=3$2

to avoid repeatedly drawing from already-sampled dominant clusters (Kim et al., 28 Apr 2026). The loop terminates when the domain-average EU, smoothed by an EMA with $k=3$3, reaches a local minimum or when the maximum budget is reached.

In the reported implementation, UnIte typically samples $k=3$4 documents per iteration for up to $k=3$5 rounds, with a total budget of $k=3$6k, but often early-stops at $k=3$7–$k=3$8k on average. One pseudo query is generated per selected document using Llama3-8B-Instruct with temperature $k=3$9 and top-p $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$0. AU filtering costs about $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$1s using a PySerini BM25 index, EU estimation costs about $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$2s per iteration, and total adaptation is about $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$3 minutes on a single NVIDIA 3090 GPU when early stopping occurs at $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$4–$$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$5k (Kim et al., 28 Apr 2026).

The main gains are reported on large BEIR corpora. Average improvements over DUQGen are $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$6 nDCG@10 for DPR, $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$7 for coCondenser, $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$8 for COCO-DR, and $$\mathcal{C}' = \{ d \in \mathcal{C} \mid z(d) \le z_{\mathrm{thr}} \},$$9 for Qwen3-Embedding-4B, excluding HQ where Qwen3 was pre-trained. Representative DPR gains are $z_{\mathrm{thr}} = 1.5$0 on TREC-COVID, $z_{\mathrm{thr}} = 1.5$1 on Robust04, $z_{\mathrm{thr}} = 1.5$2 on TREC-NEWS, $z_{\mathrm{thr}} = 1.5$3 on HotpotQA, and $z_{\mathrm{thr}} = 1.5$4 on Quora (Kim et al., 28 Apr 2026). Ablations show that removing EU sampling reduces performance by about $z_{\mathrm{thr}} = 1.5$5 nDCG@10 on average, removing AU filtering further reduces performance by about $z_{\mathrm{thr}} = 1.5$6 nDCG@10, and removing the resampling penalty degrades DPR at $z_{\mathrm{thr}} = 1.5$7k samples by $z_{\mathrm{thr}} = 1.5$8 on TC, $z_{\mathrm{thr}} = 1.5$9 on QR, and $\mathcal{D}_A$00 on TN. The domain-aware EU estimator also improves average nDCG@10 by about $\mathcal{D}_A$01 points over MC-Dropout and entropy in a first-iteration DPR comparison (Kim et al., 28 Apr 2026).

The method’s stated trade-offs are equally important. The EU estimator assumes single-vector embeddings and an MLM head, so extensions to multi-vector architectures such as ColBERT and to seq2seq rerankers require approximations; EU relies on IDF as domain importance; minority topics may remain underrepresented in skewed domains; and per-iteration EU estimation adds compute even if early stopping reduces overall cost (Kim et al., 28 Apr 2026).

5. Biomedical class-incremental learning: uncertainty trajectories, replay, and semantic expansion

In biomedical class-incremental learning, the DUSE formulation combines dynamic exemplar selection with Fine-Grained Semantic Expansion to mitigate catastrophic forgetting and class-imbalance bias under limited samples (Yao et al., 2024). The uncertainty module, termed the Uncertainty Trajectory Analyzer (UTA), ranks samples by predicted average cumulative entropy over training and stores the most uncertain samples in a fixed-size memory bank for later rehearsal. The expansion module generates semantically faithful variants through transformations such as rotations and color channel permutations, with explicit supervision over both class and transformation identity (Yao et al., 2024).

The cumulative-entropy formulation is accompanied by a theoretical justification. Using influence functions and gradient-dynamics arguments, the paper states that samples with large cumulative entropy jointly have high entropy sensitivity and large influence on parameter updates, so prioritizing them maximizes near-term training impact and captures persistent uncertainty that training must resolve. These derivations assume smoothness of $\mathcal{D}_A$02, an invertible local Hessian, and stable gradient dynamics (Yao et al., 2024). Because exact tracking across all epochs is expensive, the framework adds a lightweight prediction module that takes multi-scale intermediate features, outputs a per-epoch predictive distribution matrix $\mathcal{D}_A$03, derives $\mathcal{D}_A$04, and is trained by

$\mathcal{D}_A$05

so that most samples require only one forward pass through the predictor while only shortlisted candidates are fully audited (Yao et al., 2024).

The memory strategy is fixed-budget across increments. After each increment, memory is redistributed so that each seen class holds roughly $\mathcal{D}_A$06 exemplars. Reported practical choices are $\mathcal{D}_A$07 for imbalanced PathMNIST, $\mathcal{D}_A$08 for long-tailed PathMNIST, $\mathcal{D}_A$09 for imbalanced BloodMNIST, and $\mathcal{D}_A$10 for long-tailed BloodMNIST; the dataset table also reports comparable budgets of $\mathcal{D}_A$11 and $\mathcal{D}_A$12 for PathMNIST, and $\mathcal{D}_A$13 and $\mathcal{D}_A$14 for BloodMNIST (Yao et al., 2024).

Semantic expansion is not treated as generic augmentation. The reported rationale is geometric: it densifies intra-class manifolds and creates “room” in the feature space for future classes. Separability is expressed through symmetric KL divergences between class predictive distributions, with inter-class divergence

$\mathcal{D}_A$15

and intra-class divergence

$\mathcal{D}_A$16

The multi-task loss

$\mathcal{D}_A$17

encourages the backbone to encode both class semantics and transformation identity (Yao et al., 2024).

Classification bias from imbalance is handled by a cosine classifier over normalized features and prototypes:

$\mathcal{D}_A$18

or equivalently $\mathcal{D}_A$19 with normalized weights and features (Yao et al., 2024). Training combines cross-entropy, supervised contrastive loss with a MoCo-style momentum key encoder, and optionally the multi-task expansion loss.

The reported setting uses PathMNIST with $\mathcal{D}_A$20 classes and BloodMNIST with $\mathcal{D}_A$21 classes, ResNet-20 and ResNet-18 backbones respectively, SGD with momentum $\mathcal{D}_A$22, and per-session training schedules of $\mathcal{D}_A$23 epochs for PathMNIST and $\mathcal{D}_A$24 epochs for BloodMNIST. In the imbalanced setting, base classes have $\mathcal{D}_A$25–$\mathcal{D}_A$26 samples each and incremental classes have $\mathcal{D}_A$27 samples; in the long-tailed setting, incremental classes have $\mathcal{D}_A$28 samples (Yao et al., 2024).

Empirically, the method achieves optimal performance across four imbalanced data distributions over two datasets and surpasses state-of-the-art methods by as much as $\mathcal{D}_A$29 in accuracy. On imbalanced BloodMNIST, last-session accuracy reaches $\mathcal{D}_A$30 versus $\mathcal{D}_A$31–$\mathcal{D}_A$32 for baselines, with average accuracy gains ranging from $\mathcal{D}_A$33 to $\mathcal{D}_A$34 across methods (Yao et al., 2024). Ablations attribute substantial value to each component: removing semantic expansion drops last-session accuracy by $\mathcal{D}_A$35 and average accuracy by $\mathcal{D}_A$36; replacing UTA with random, NME, pool-based, or committee-based selection reduces average accuracy by $\mathcal{D}_A$37, $\mathcal{D}_A$38, $\mathcal{D}_A$39, and $\mathcal{D}_A$40 respectively; and replacing cosine with dot product, Euclidean, or Mahalanobis classifiers causes severe degradations (Yao et al., 2024).

The stated failure modes remain domain-specific. Miscalibrated uncertainty under heavy shift or label noise can misprioritize exemplars; extremely rare-class drift may require adaptive per-class memory quotas or diversity regularization; aggressive color or intensity perturbations can harm biomedical texture fidelity; and drastic morphological shift may require additional self-supervised pretraining or domain adaptation (Yao et al., 2024).

6. Soft DUSE in reinforcement learning and broader implications

RAVE extends model-based value expansion by making imagined rollouts uncertainty-aware and risk-averse (Zhou et al., 2019). The method trains probabilistic ensembles for transition, reward, and termination, each outputting Gaussian distributions and trained by negative log-likelihood. For each replay-buffer transition and each rollout horizon $\mathcal{D}_A$41, it generates $\mathcal{D}_A$42 stochastic distributional value expansions, computes their mean and variance, forms an $\mathcal{D}_A$43-confidence lower bound, and then interpolates across horizons with inverse-variance weights (Zhou et al., 2019).

The resulting target is

$\mathcal{D}_A$44

with adaptive confidence factor

$\mathcal{D}_A$45

Early in training, large one-step prediction error makes $\mathcal{D}_A$46 small; as the model becomes accurate, $\mathcal{D}_A$47 increases toward its upper bound (Zhou et al., 2019). The critic is trained with

$\mathcal{D}_A$48

while the actor remains standard deterministic policy gradient.

The DUSE connection is explicit but qualified. RAVE does not dynamically alter $\mathcal{D}_A$49 or the number of rollout samples per transition. Instead, it implements a “soft” dynamic expansion policy: uncertain horizons receive less weight, and uncertain rollout returns are penalized by the lower confidence bound. This means uncertainty controls trust in imagined expansion rather than hard expansion depth or count (Zhou et al., 2019).

The reported benefits are robustness and sample efficiency in stochastic control. RAVE performed favorably on MuJoCo and Roboschool tasks, surpassed STEVE, and often matched or exceeded TD3 and SAC on tasks such as Hopper and Walker2d. In the OpenSim-based “Learn to Run” setting, the reported falling rate is about $\mathcal{D}_A$50 for an ensemble-Q DDPG agent versus about $\mathcal{D}_A$51 with RAVE over $\mathcal{D}_A$52 episodes, and the RAVE-based solution achieved first place in the NeurIPS 2019 Learn to Move challenge (Zhou et al., 2019). The computational overhead relative to STEVE is reported as $\mathcal{D}_A$53s versus $\mathcal{D}_A$54s for $\mathcal{D}_A$55 batches of size $\mathcal{D}_A$56, or about $\mathcal{D}_A$57 overhead, with no additional inference-time cost because only the learned policy is used at test time (Zhou et al., 2019).

Taken together, the four formulations imply several general properties of DUSE-style methods. First, uncertainty alone is rarely sufficient: effective systems typically combine uncertainty with diversity, memory balancing, semantic expansion, or inverse-variance weighting. Second, “dynamic” means that uncertainty is re-estimated or re-interpreted after model updates; static scoring is repeatedly shown to be weaker than iterative refinement. Third, expansion need not mean synthetic generation. It can mean importing real labeled samples from an auxiliary pool, generating pseudo queries for selected documents, storing uncertain exemplars for replay, or trusting only the lower-confidence portion of imagined rollouts. Finally, the most stable formulations separate data-inherent noise from model uncertainty whenever the problem permits, as in UnIte’s AU/EU split and RAVE’s aleatoric/epistemic decomposition (Kim et al., 28 Apr 2026, Zhou et al., 2019).