Query-by-Committee (QBC)

• Query-by-Committee (QBC) is an active learning paradigm where a committee of models identifies unlabeled examples based on the degree of disagreement among predictions.
• It employs diverse committee constructions—from bootstrapping and Bayesian sampling to dropout variants—and disagreement metrics like vote entropy and KL divergence.
• QBC has been successfully applied in areas such as text processing, molecular modeling, and deep learning to reduce labeling costs while enhancing model performance.

Query-by-Committee (QBC) is an active-learning paradigm in which the learner maintains a committee of plausible models and queries labels for unlabeled instances on which those models disagree most strongly. The method replaces reliance on a single current hypothesis with disagreement-driven sampling over a version space or posterior over hypotheses, and it has been instantiated through posterior sampling, bootstrap committees, boosting, dropout-derived subnetworks, MCMC draws of Gaussian-process hyperparameters, and committee neural-network potentials. Its disagreement criterion has likewise expanded from hard-vote entropy to KL-based soft disagreement, regression variance, robust divergences, weighted kernel discrepancies, and structured losses over atomic questions (Tosh et al., 2018, Bloodgood, 2018).

1. Core operational principle

In its standard pool-based form, QBC maintains a committee $\{M_1,\ldots,M_C\}$ trained on the current labeled set $L$, scores each unlabeled example $x\in U$ by the degree to which committee members disagree on its label, and queries the true label of the top-$K$ most-disagreeing examples (Bloodgood, 2018). In the classical version-space view, one places a prior $\pi$ over a hypothesis class $\mathcal H$, restricts $\pi$ to the current version space, samples classifiers from that restricted distribution, and queries examples on which the sampled hypotheses disagree (Tosh et al., 2018).

For binary classification, a common disagreement score is vote entropy. If $v_+(x)=|\{i:M_i(x)=+1\}|$ and $v_-(x)=C-v_+(x)$, then

$$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$

For multiclass classification, the same idea is written as

$L$0

so disagreement is maximal when committee votes are most evenly split (Chae et al., 2020).

For probabilistic classifiers, disagreement can be measured using hard votes or soft posteriors. Argamon-Engelson and Dagan define normalized vote entropy

$L$1

where each committee member votes for its most probable label, and also define an averaged KL divergence from each member’s predictive distribution to the committee mean (Argamon-Engelson et al., 2011). For regression, a standard QBC score is the sample variance of committee outputs,

$L$2

which is high when the committee’s predicted responses are dispersed (Chae et al., 2020).

The conceptual commonality is that informativeness is identified with disagreement among plausible models rather than with uncertainty of a single fitted model. This suggests that QBC is best understood as a family of acquisition rules rather than as a single algorithm.

2. Committee construction and model uncertainty

The defining object in QBC is the committee, and the literature differs most sharply in how that committee is generated. In SVM active learning, two standard constructions are bagging and boosting: bagging trains each committee member on a bootstrap sample of the labeled set, while boosting trains committee members sequentially with re-weighting; these yield the QBag and QBoost families, respectively (Bloodgood, 2018). In bagging-based pool sampling, the committee is re-created each round from bootstrap replicates of the labeled set, and disagreement is computed from their votes (Ghai et al., 2020).

For probabilistic classifiers, committee generation is explicitly Bayesian. Let $L$3 denote a model with parameters $L$4, and let $L$5 be sufficient statistics from previously labeled data. Then plausible committee members are drawn from

$L$6

For multinomial parameter groups, posterior sampling reduces to Dirichlet draws such as

$L$7

so the committee is a posterior sample over parameter settings rather than a heuristic ensemble (Argamon-Engelson et al., 2011).

Fully Bayesian Gaussian-process QBC makes the committee correspond to posterior draws of hyperparameters. With $L$8 and priors $L$9 and $x\in U$0, MCMC over $x\in U$1 yields $x\in U$2, each defining a GP predictive mean

$x\in U$3

The committee is therefore a sample from the hyperparameter posterior, and disagreement is the posterior variance of those mean predictions rather than a disagreement among independently retrained models (Riis et al., 2022).

Deep-learning variants modify the construction to avoid the cost of training many networks from scratch. QBDC begins with a single full CNN $x\in U$4, applies batchwise dropout to obtain partial CNNs $x\in U$5, freezes the first $x\in U$6 layers of each partial network, and retrains only the final classification layer on the labeled set so that the committee remains approximately consistent with current labels (Ducoffe et al., 2015). In atomistic machine learning, the committee may instead consist of independently initialized neural-network potentials that share descriptors but differ in random seeds; fourth-generation high-dimensional committee neural-network potentials use $x\in U$7 in early generations and $x\in U$8 in the final generation, with the reduction validated to give identical accuracy (Faruque et al., 3 Nov 2025).

A compact comparison is useful:

Setting	Committee construction	Representative disagreement
Classical / SVM QBC	Bagging or boosting over labeled data	Vote entropy
Probabilistic classification	Posterior sampling of parameter settings	Vote entropy or averaged KL
Fully Bayesian GP	MCMC draws from $x\in U$9	Variance of predictive means
Deep CNN QBDC	Dropout-derived partial CNNs from one full CNN	Non-majority vote count
Multi-kernel learning	Single-kernel predictors weighted by reliabilities	Expected-kernel-discrepancy

These constructions differ in computational cost and in what kind of uncertainty they represent: subsampling variability, posterior parameter uncertainty, hyperparameter multimodality, or architecture-level stochasticity.

3. Disagreement measures and generalizations

Although vote entropy is the most recognizable QBC score, the literature broadens disagreement well beyond unweighted hard votes. For probabilistic models, one can compare each model’s soft posterior $K$0 to the committee average $K$1 through

$K$2

which captures disagreement in predictive distributions rather than only in argmax labels (Argamon-Engelson et al., 2011). In robust-divergence QBC, the consensus model $K$3 is defined by minimizing average divergence to committee members, and the standard acquisition score is

$K$4

This formulation is then generalized by replacing KL divergence with Bregman divergences, including the $K$5-divergence and the dual $K$6-power divergence (Hino et al., 2022).

The robust variants introduce two changes. First, the disagreement measure becomes

$K$7

where $K$8 and $K$9 are divergence-based consensus models. Second, the influence-function analysis shows that the KL-based acquisition has unbounded influence under an outlying committee member, while the $\pi$0- and dual $\pi$1-based variants have bounded influence functions for $\pi$2 and $\pi$3 (Hino et al., 2022). This directly targets instability caused by misbehaving committee members.

Multiple-kernel learning supplies another generalization. In random-feature MKL, the committee consists of $\pi$4 single-kernel predictors $\pi$5 combined by reliabilities $\pi$6 with $\pi$7. The expected-kernel-discrepancy criterion is

$\pi$8

which measures weighted pairwise discrepancy in the kernel-induced feature spaces. When $\pi$9 and $\mathcal H$0 for all $\mathcal H$1, EKD becomes a constant-multiple upper bound of classical variance-based QBC by Jensen’s inequality (Chae et al., 2020).

Structural QBC generalizes the unit of disagreement from single labels to structured queries. A structure $\mathcal H$2 is viewed as a function on a finite set of atomic questions $\mathcal H$3, and a query $\mathcal H$4 exposes a small subset $\mathcal H$5. Disagreement between structures on a query is

$\mathcal H$6

with corresponding uncertainty

$\mathcal H$7

For general convex losses, the binary disagreement indicator is replaced by squared distances in prediction space (Tosh et al., 2018). This turns QBC into a framework for interactive clustering, metric learning, and structured-output problems rather than only pool-based label querying.

A further extension weights committee members by agreement with annotator rationales. Active Learning++ computes a local explanation for each committee member, compares the resulting feature ranking to the annotator’s ranking using Kendall’s $\mathcal H$8, converts the similarities into weights $\mathcal H$9, and replaces the unweighted vote margin with

$\pi$0

The committee is therefore no longer exchangeable: members whose local explanations better match the annotator carry larger influence in subsequent selection (Ghai et al., 2020).

4. Algorithmic forms and theoretical properties

QBC can be implemented sequentially, in batches, or through posterior-weight updates. A particularly simple algorithm is the two-member sequential variant for probabilistic classification: draw $\pi$1, let each model assign its most probable label to the incoming example, and request annotation only when the two labels differ (Argamon-Engelson et al., 2011). More general sequential algorithms use arbitrary committee size $\pi$2, compute vote entropy, and either threshold it or sample an example with probability proportional to disagreement. Batch variants compute disagreement scores across a set of unlabeled candidates and select the top $\pi$3 examples for labeling (Argamon-Engelson et al., 2011).

The two-member variant is notable because it has exactly zero tunable parameters. Argamon-Engelson and Dagan report that it reduces annotation costs almost as much as larger committees while remaining computationally lightweight, since it requires only two posterior draws and two labelings per example (Argamon-Engelson et al., 2011). On stochastic part-of-speech tagging with a first-order HMM and approximately $\pi$4 tags, complete training needed approximately $\pi$5 ambiguous-word labels to reach $\pi$6 accuracy, whereas two-member sample selection needed only approximately $\pi$7-$\pi$8 labels for the same accuracy; at $\pi$9 accuracy, complete training stored approximately $v_+(x)=|\{i:M_i(x)=+1\}|$0 lexical counts and approximately $v_+(x)=|\{i:M_i(x)=+1\}|$1 bigram counts, while two-member selection needed only approximately $v_+(x)=|\{i:M_i(x)=+1\}|$2 lexical counts and approximately $v_+(x)=|\{i:M_i(x)=+1\}|$3 bigram counts (Argamon-Engelson et al., 2011).

Structural QBC gives explicit posterior updates and consistency guarantees under noise. In the $v_+(x)=|\{i:M_i(x)=+1\}|$4-$v_+(x)=|\{i:M_i(x)=+1\}|$5 loss case, if the expert returns correction $v_+(x)=|\{i:M_i(x)=+1\}|$6, the posterior is updated by

$v_+(x)=|\{i:M_i(x)=+1\}|$7

where $v_+(x)=|\{i:M_i(x)=+1\}|$8 controls how aggressively inconsistent structures are down-weighted (Tosh et al., 2018). Under bounded noise of Massart type and a correction-feedback assumption, if $v_+(x)=|\{i:M_i(x)=+1\}|$9 then $v_-(x)=C-v_+(x)$0 almost surely. The paper establishes an analogous almost-sure consistency result for general bounded, Lipschitz convex losses under a margin condition (Tosh et al., 2018).

Bayesian GP QBC retains the disagreement principle but changes its target. The acquisition function

$v_-(x)=C-v_+(x)$1

directly measures disagreement among plausible bias-variance regimes of the GP, because each committee member corresponds to a different hyperparameter draw (Riis et al., 2022). The paper reports that across six benchmark simulators B-QBC consistently yields the largest average improvement in marginal log-likelihood, reducing NLML by $v_-(x)=C-v_+(x)$2 relative to the standard ALM baseline on average, while QB-MGP achieves the best predictive performance (Riis et al., 2022).

Theoretical and algorithmic work therefore portray QBC not simply as “ask where models disagree,” but as a family of sequential Bayesian or quasi-Bayesian procedures in which committee generation, disagreement scoring, and posterior updating can all be formalized.

5. QBC in SVM active learning and imbalanced text processing

A prominent comparative study of QBC in text processing evaluates QBagPA and QBoostPA against a margin-based alternative, ClosestPA, under class imbalance (Bloodgood, 2018). All three methods integrate Positive Amplification (PA) through asymmetric SVM cost factors $v_-(x)=C-v_+(x)$3 and $v_-(x)=C-v_+(x)$4, with

$v_-(x)=C-v_+(x)$5

when positives are the minority, and with InitPA setting

$v_-(x)=C-v_+(x)$6

from an unbiased initial estimate $v_-(x)=C-v_+(x)$7 of positive prevalence. The cost ratio is set at initialization and then used for all subsequent SVM training (Bloodgood, 2018).

QBagPA forms the committee by bagging. After drawing an initial random sample of size $v_-(x)=C-v_+(x)$8, estimating $v_-(x)=C-v_+(x)$9, and setting $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$0, each round draws with-replacement bootstrap samples $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$1 from the labeled set, trains $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$2 SVMs with those asymmetric costs, computes vote entropy for each $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$3, and queries the top-$$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$4 examples with the largest disagreement (Bloodgood, 2018). QBoostPA uses AdaBoost-style reweighting to train committee members sequentially, but scores unlabeled examples exactly as in QBagPA using vote counts (Bloodgood, 2018).

The empirical results are unfavorable to QBC in this setting. On the AImed relation-extraction corpus, with $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$5 positives and $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$6 negatives, the average number of labeled examples required to reach $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$7 was $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$8 for RandomPA, $$\mathrm{Disagree}(x)=- \left[ \left(\frac{v_+}{C}\right)\log\left(\frac{v_+}{C}\right) + \left(\frac{v_-}{C}\right)\log\left(\frac{v_-}{C}\right) \right].$$9 for QBagPA, $L$00 for QBoostPA, and $L$01 for ClosestPA, with ClosestPA significantly better than QBagPA at $L$02 (Bloodgood, 2018). On Reuters-21578, the average data-utilization ratios over categories were $L$03 for QBagPA, $L$04 for QBoostPA, and $L$05 for ClosestPA; on Ohsumed, the corresponding averages were $L$06, $L$07, and $L$08 (Bloodgood, 2018).

The paper also reports that in the region where a practical stopping criterion would halt, ClosestPA consistently achieves higher $L$09 than both QBagPA and QBoostPA, and that increasing committee size from $L$10 to $L$11 yields only very small gains for the QBC variants, on the order of $L$12-$L$13 $L$14 points, at the cost of training becoming three times slower (Bloodgood, 2018). The proposed explanation is threefold: margin-based selection better avoids redundancy once a queried example becomes a support vector, ClosestPA requires training only one SVM per round rather than $L$15 models, and QBC can dilute the effect of positive amplification across committee members (Bloodgood, 2018).

The practical recommendation in that study is correspondingly specific: start with an unbiased random sample of approximately $L$16 points to estimate $L$17, set $L$18, train all subsequent SVMs with cost ratio $L$19, use closest-to-hyperplane selection, and stop when an SVM-based stopping criterion signals convergence (Bloodgood, 2018). A plausible implication is that QBC should not be treated as a uniformly dominant acquisition rule: in imbalanced margin-based text tasks, the committee mechanism can be outperformed by a single well-calibrated margin model.

6. Applications across scientific and deep-learning domains

In molecular machine learning, QBC has been used to guide sampling of chemical space for universal neural-network potentials. The ANI active-learning workflow trains an ensemble of $L$20 ANI neural networks on a seed dataset, predicts candidate molecules and conformers, and computes committee disagreement from the standard deviation of predicted energies. For structure $L$21 with $L$22 atoms,

$L$23

where $L$24 is the committee standard deviation and the $L$25 scaling corrects for error cancellation in larger molecules (Smith et al., 2018). A structure is queried when $L$26, with $L$27 kcal$L$28mol$L$29(atom)$L$30 chosen so that $L$31 of large-error points satisfy the threshold (Smith et al., 2018). Using this procedure, the AL-trained ANI with approximately $L$32 million conformers, approximately $L$33 of ANI-1, already outperforms ANI-1 on the full COMP6 benchmark, while the final ANI-1x potential uses $L$34 million conformers, $L$35 of ANI-1, and achieves $L$36 kcal/mol versus $L$37, $L$38 kcal/mol versus $L$39, and $L$40 kcal/(mol$L$41) versus $L$42 (Smith et al., 2018).

A related but distinct scientific use appears in fourth-generation high-dimensional committee neural-network potentials for water and air-water interfaces. There, disagreement is based on force predictions:

$L$43

and each iteration selects the top $L$44 configurations with largest force disagreement from a candidate pool (Faruque et al., 3 Nov 2025). Validation on $L$45 held-out AIMD configurations shows error convergence plateaus after approximately $L$46 QbC picks, classical mean absolute force errors over generations improve $L$47 meV/\AA, PIMD force errors improve $L$48 meV/\AA, and reducing the committee from $L$49 to $L$50 yields identical RDFs and spectral results at approximately half the cost (Faruque et al., 3 Nov 2025).

In multiple-kernel regression, EKD generalizes QBC by weighting disagreement using learned kernel reliabilities. On UCI regression datasets, with $L$51 Gaussian kernels and random-feature dimension $L$52 per kernel, EKD achieves the best test MSE$L$53 in the excerpted Tom’s hardware results: $L$54 and $L$55 on Tom(O) at $L$56 and $L$57, and $L$58 and $L$59 on Tom(S) at $L$60 and $L$61, outperforming random sampling, classical QBC, EMC, and EKL in those columns (Chae et al., 2020). The paper interprets this as evidence that explicitly modeling kernel-wise reliabilities is advantageous in an MKL setting where classical QBC and EMC can do no better, and sometimes worse, than random (Chae et al., 2020).

Deep-learning QBC also appears in image classification. QBDC evaluates a dropout committee on MNIST by starting from $L$62 labeled samples, querying in rounds of $L$63, and selecting less than $L$64 of the annotated database. After approximately $L$65 labels, QBDC reaches a mean test error of $L$66 with best error $L$67, while random selection at the same label fraction yields $L$68 mean error and full training on all $L$69 labels yields $L$70 mean error (Ducoffe et al., 2015). The same study reports only a slight increase in adversarial vulnerability, approximately $L$71 more misclassifications at $L$72 under FGSM, despite discarding $L$73 of labels (Ducoffe et al., 2015).

Computational-physics applications combine QBC with Bayesian neural networks. A committee of BNNs defines a committee-averaged predictive density

$L$74

predictive entropy

$L$75

and disagreement

$L$76

In high-dimensional Monte Carlo integration, the resulting method yields a relative-error prefactor approximately $L$77 smaller than uniform sampling, corresponding to two orders of magnitude fewer samples to reach the same accuracy (Chen et al., 2020).

Across these applications, QBC is used less as a single canonical recipe than as a mechanism for steering expensive labeling, simulation, or electronic-structure evaluation toward regions where currently plausible models disagree most strongly.