Dual Active Learning with Multimodal (DAM)

• The paper presents a dual supervisory framework that fuses sparse human labels with ViL-based supervision via bidirectional distillation for source-free domain adaptation.
• It employs Dual-Focused active Supervision (DFS) and Alternating Distillation (ADL) to improve pseudo-label quality and mitigate modality imbalance.
• Experimental evaluations on benchmarks like Office and VisDA-C show enhanced performance under low-label regimes with minimal added computational cost.

Searching arXiv for the specified DAM paper and closely related multimodal active learning work. I’ll look up the DAM paper and the two related multimodal active learning papers to ground the article in current arXiv records. Dual Active Learning with Multimodal, abbreviated DAM, denotes a framework for source-free active domain adaptation (SFADA) that integrates sparse human annotations with supervision from a Vision-and-Language (ViL) foundation model into a dual supervisory signal, then couples the two through bidirectional distillation during target-domain adaptation (Chen et al., 29 Sep 2025). In the DAM formulation, a pretrained source model is adapted to an unlabeled target domain without access to source data, under a limited labeling budget, and active querying is used once to select target samples for annotation. DAM is situated within a broader multimodal active learning literature that has emphasized modality imbalance, dominant-modality bias, and the limits of modality-agnostic acquisition, but its mechanism is distinct: rather than balancing multiple raw input modalities within a pool-based classifier, it uses a multimodal foundation model to provide an auxiliary supervision pathway during adaptation (Shen et al., 2023, Eisenhardt et al., 31 Mar 2026).

1. Problem formulation and motivation

DAM is defined in the setting where a source model $S$ has already been pretrained on a labeled source dataset

$\{(x_i,y_i)\}_{i=1}^{m}$

spanning $C$ classes, while the source data are unavailable during adaptation. The available data consist of an unlabeled target set

$\mathcal T=\{x'_i\}_{i=1}^{n}$

and a limited labeling budget $B$. An active query strategy $\pi$ selects a subset $\mathcal T_q\subset\mathcal T$ for oracle annotation, and the objective is to minimize the target risk of the adapted target model $T$ on $\mathcal T$ (Chen et al., 29 Sep 2025).

The paper adopts the one-shot query setting from MHPL, in which all queried samples are selected at once using the raw source model $S$, under

$\{(x_i,y_i)\}_{i=1}^{m}$0

with $\{(x_i,y_i)\}_{i=1}^{m}$1. This choice places DAM in a regime where pseudo-label quality and supervision efficiency are critical, because SFDA already lacks source data and therefore depends on source-model predictions, self-training or pseudo-labeling, entropy or diversity regularization, and a small number of target labels (Chen et al., 29 Sep 2025).

The motivation for DAM is that recent domain adaptation work has introduced ViL models to improve pseudo-label quality or feature alignment, but has often treated ViL-based supervision and human-labeled active samples as separate sources. DAM is proposed to overcome this separation by “integrat[ing] multimodal supervision from a ViL model to complement sparse human annotations,” thereby forming a dual supervisory signal (Chen et al., 29 Sep 2025). The framework is therefore aimed at coordinated fusion rather than a simple additive combination of active labels and foundation-model predictions.

Within the broader multimodal active learning literature, this motivation aligns with a recurring observation: naïvely transferring active learning methods from unimodal settings can produce biased selection toward a dominant information source, and multimodal strategies do not automatically resolve that problem (Shen et al., 2023, Eisenhardt et al., 31 Mar 2026). DAM addresses that general concern in a different application setting—SFADA rather than pool-based multimodal classification—by placing the multimodal component in the supervisory loop.

2. Architecture of the dual supervisory signal

DAM organizes its method around two named components: Dual-Focused active Supervision (DFS) and Alternating Distillation (ADL) (Chen et al., 29 Sep 2025). The duality comes from combining:

1. Sparse human annotations on actively queried target samples $\{(x_i,y_i)\}_{i=1}^{m}$2.
2. ViL-based supervision generated by a prompt-tuned CLIP-like model.

The ViL model $\{(x_i,y_i)\}_{i=1}^{m}$3 is based on CLIP with a frozen image encoder, frozen text encoder, and learnable category prompts only. For each class $\{(x_i,y_i)\}_{i=1}^{m}$4, the prompt $\{(x_i,y_i)\}_{i=1}^{m}$5 is structured as

$\{(x_i,y_i)\}_{i=1}^{m}$6

and initialized as

$\{(x_i,y_i)\}_{i=1}^{m}$7

Text embeddings are computed by

$\{(x_i,y_i)\}_{i=1}^{m}$8

while $\{(x_i,y_i)\}_{i=1}^{m}$9 denotes the image feature from the frozen image encoder (Chen et al., 29 Sep 2025).

The ViL prediction is defined as

$C$0

where $C$1 is cosine similarity and $C$2 is a learned temperature parameter. The paper attributes to this construction robust semantic priors, smoother predictions than a purely target-trained classifier early in adaptation, and better class separation under low-label regimes, while also emphasizing that the ViL model is not task-specific until it is adapted on the actively labeled target samples (Chen et al., 29 Sep 2025).

DFS is the mechanism that turns these ingredients into a stable supervisory source. After one-shot active querying, the prompts $C$3 are optimized on $C$4 using

$C$5

Here, $C$6 is the one-hot label vector, and $C$7 is the original prompt embedding. The term $C$8 is described as anchoring prompts to their pretrained general-purpose semantics, so that prompt tuning remains target-aware without discarding the ViL model’s prior structure (Chen et al., 29 Sep 2025).

A central misconception addressed by the paper is that DAM is merely “active learning + CLIP.” The method is defined instead by the fact that active labels specialize the ViL model itself, and that this specialized ViL model is then used as an additional teacher throughout adaptation (Chen et al., 29 Sep 2025).

3. Alternating distillation and optimization

ADL is the mechanism by which DAM turns the dual supervisory signal into a coupled training process. The framework treats the target model $C$9 and the ViL model $\mathcal T=\{x'_i\}_{i=1}^{n}$0 as mutual teachers. For a student model $\mathcal T=\{x'_i\}_{i=1}^{n}$1 taught by teacher $\mathcal T=\{x'_i\}_{i=1}^{n}$2, teacher outputs and weights are defined by

$\mathcal T=\{x'_i\}_{i=1}^{n}$3

where $\mathcal T=\{x'_i\}_{i=1}^{n}$4 is Softmax, and $\mathcal T=\{x'_i\}_{i=1}^{n}$5 are weighting hyperparameters (Chen et al., 29 Sep 2025). The student prediction is

$\mathcal T=\{x'_i\}_{i=1}^{n}$6

and the paper notes that hard pseudo-labeling is a special case when $\mathcal T=\{x'_i\}_{i=1}^{n}$7.

In the direction $\mathcal T=\{x'_i\}_{i=1}^{n}$8, the target model provides confident samples to refine the ViL model. During each epoch, DAM selects the top-$\mathcal T=\{x'_i\}_{i=1}^{n}$9 most confident samples per class from target predictions, forms $B$0, and uses these together with $B$1 to train the ViL model through

$B$2

The paper specifies $B$3 in this direction and states that $B$4 carries the target model’s most reliable discriminative knowledge (Chen et al., 29 Sep 2025).

In the reverse direction $B$5, the ViL model supervises the target model on unlabeled target data: $B$6 with $B$7 and $B$8 for hard ViL pseudo-labels. This gives the target network a pseudo-label source that is intended to be semantically grounded and more stable than target-only self-training early in adaptation (Chen et al., 29 Sep 2025).

DAM augments this distillation with two regularizers used for the target model. The first is entropy minimization,

$B$9

and the second is diversity regularization,

$\pi$0

where

$\pi$1

The overall objectives are

$\pi$2

$\pi$3

After convergence, the ViL model is discarded and only the adapted target model is used for inference (Chen et al., 29 Sep 2025).

A plausible implication is that DAM treats the multimodal foundation model primarily as a training-time supervisory instrument rather than as part of the deployed inference stack. This is consistent with the paper’s explicit report of zero inference FLOPs from the ViL component after training (Chen et al., 29 Sep 2025).

4. Active querying, multimodality, and relation to adjacent research

DAM is strategy-agnostic with respect to the active query rule. The framework uses one-shot sample selection based on the raw source model $\pi$4, and the paper evaluates DAM as a plug-in with several active selection strategies, including CoreSet, MHPL, ELPT, and BIAS-augmented pipelines (Chen et al., 29 Sep 2025). In this design, actively selected samples play two roles: they provide supervised target labels, and they also “activate and specialize the ViL prompt learner” (Chen et al., 29 Sep 2025).

This role of active samples distinguishes DAM from earlier multimodal active learning methods developed for pool-based multimodal classification. In Balanced Multimodal Active Learning (BMMAL), for example, the objective is to avoid biased acquisition toward the dominant modality by modulating BADGE-style gradient embeddings with modality contributions estimated from Shapley values (Shen et al., 2023). BMMAL operates on multimodal inputs directly, with modality-specific encoders, late fusion by concatenation,

$\pi$5

and a joint training objective

$\pi$6

so its balancing mechanism is located in the acquisition geometry itself (Shen et al., 2023).

The benchmark study “Mind the Gap: A Framework for Assessing Pitfalls in Multimodal Active Learning” formalizes a related but broader critique. It identifies missing modalities, modality imbalance, and different interaction structures—shared, unique, and complementary/synergistic information—as multimodal pitfalls absent in the unimodal case, and reports that “models consistently develop imbalanced representations, relying primarily on one modality while neglecting others,” while “multimodal strategies do not provide clear advantages over unimodal ones in our experiments” (Eisenhardt et al., 31 Mar 2026). That analysis concerns multimodal active learning in the standard pool-based sense, whereas DAM uses multimodality through a foundation-model supervision path.

This comparison clarifies a common ambiguity around the acronym. In DAM, “dual” refers to the fusion of human labels and ViL-based supervision, followed by mutual distillation; it does not denote a classical two-view disagreement-based active learner. This suggests that DAM should be read as a framework for dual supervision in active adaptation, not as a synonym for all modality-balanced query methods.

5. Experimental evaluation and reported performance

DAM is evaluated on Office, Office-Home, and VisDA-C, with ResNet-50 used for Office and Office-Home, ResNet-101 for VisDA-C, and CLIP with ViT-B/32 image encoder for the ViL component (Chen et al., 29 Sep 2025). The default labeling budget is $\pi$7, unless otherwise noted, and the reported hyperparameters include $\pi$8, $\pi$9, prompt length $\mathcal T_q\subset\mathcal T$0, and top-confidence set size $\mathcal T_q\subset\mathcal T$1 (Chen et al., 29 Sep 2025).

The paper compares against five categories of methods: Source-only / zero-shot CLIP, UDA methods, SFDA methods, ADA methods, and SFADA methods. Examples listed include Source-only, ZS-C-B32, RADA-prompt, DAPL, CPGA, SHOT, A2Net, GRAPE, DIFO-C-B32, CLUE, EADA, DUC, CA-ADA, SQAdapt, ELPT, CoreSet, BIAS, and MHPL (Chen et al., 29 Sep 2025).

Under a 5% labeling budget, the paper reports representative results such as:

• DAM + MHPL: Office 94.1, Office-Home 82.0, VisDA-C 92.0.
• DAM + CoreSet: Office 93.4, Office-Home 83.9, VisDA-C 92.0.
• DAM + ELPT: Office 93.5, Office-Home 81.9, VisDA-C 91.9 (Chen et al., 29 Sep 2025).

The comparison against BIAS-augmented baselines is explicit. The paper highlights that CoreSet + DAM improves over CoreSet + BIAS by 2.4% / 7.1% / 3.8% on Office / Office-Home / VisDA-C, and MHPL + DAM improves over MHPL + BIAS by 0.5% / 1.7% / 0.9% on the same benchmarks (Chen et al., 29 Sep 2025).

The ablation study on Office-Home isolates the contributions of the two main components. With Source+CLIP as baseline, the reported results are 69.8 / 85.2 on two transfer tasks; with only $\mathcal T_q\subset\mathcal T$2, 72.7 / 89.3; with only $\mathcal T_q\subset\mathcal T$3, 70.5 / 85.8; and with both, 75.2 / 89.3 (Chen et al., 29 Sep 2025). The paper interprets this as showing that DFS contributes strongly, ADL alone helps modestly, and the combination yields the best result.

The computational-cost analysis reports that the ViL prompt-learning module adds only 0.002M trainable parameters, about 0.48 GB extra peak GPU memory, and zero inference FLOPs because the ViL model is discarded after training (Chen et al., 29 Sep 2025). This makes the training-time supervisory expansion lightweight at deployment.

6. Interpretation, scope, and limitations

The DAM paper explains its gains through five factors: better supervision quality, task-aware activation, mutual reinforcement, complementarity of human labels and multimodal priors, and plug-and-play compatibility with multiple active learning backbones (Chen et al., 29 Sep 2025). In this interpretation, active labels are sparse but accurate, ViL supervision is broad but requires calibration, and the benefit arises from coupling them in an iterative loop rather than using either source in isolation.

The broader multimodal active learning literature provides a useful caution against overgeneralizing this result. The balanced multimodal classification literature argues that standard active learning can over-select samples favored by a dominant modality, and proposes explicit balancing rules to avoid greedy sample selection from that modality (Shen et al., 2023). The 2026 multimodal benchmark study goes further, showing that across synthetic settings for Missing, Share, Unique, and Synergy, modality imbalance consistently emerges and multimodal query methods do not reliably improve the weak modality (Eisenhardt et al., 31 Mar 2026). These findings do not contradict DAM, but they delimit what it establishes. DAM demonstrates the value of multimodal supervision in SFADA; it does not by itself solve the general problem of modality-aware acquisition in multimodal classification.

The paper explicitly notes that DAM is compatible with various active sampling strategies and that performance depends on the chosen sampler. It also states that future work should explore optimal active sampling methods within this collaborative framework, since the present focus is on ViL activation rather than advanced sampling design (Chen et al., 29 Sep 2025). This suggests that the quality of the initial active query remains structurally important, even though the dual-supervision loop can improve several existing query backbones.

In summary, DAM occupies a specific position within active learning research. It is a source-free active domain adaptation framework in which multimodality enters through a foundation-model teacher, not through direct multi-input balancing at query time. Its principal technical contribution is the fusion of active labels and prompt-tuned ViL supervision into a coupled distillation process, while its main conceptual significance lies in showing that multimodal priors can be “carefully activated” to strengthen low-budget adaptation when source data are unavailable (Chen et al., 29 Sep 2025).