Adaptive Group Elicitation: Methods & Insights

• Adaptive group elicitation is a framework of sequential queries designed to reduce uncertainty about latent group properties for improved collective outcomes.
• It encompasses methods ranging from group preference aggregation to fair metric elicitation and predictive polling in budget-constrained settings.
• The approach balances query cost and informational gain, emphasizing decision-aware selection and structured group-level inference.

Adaptive group elicitation denotes a family of sequential information-acquisition problems in which an elicitor adaptively chooses questions, comparisons, prompts, or respondent subsets to reduce uncertainty about a latent group-level property or a group-aware objective. In the literature considered here, the term is not uniform: “group” may refer to a decision-making group whose preferences must be aggregated, sensitive demographic groups in a fairness metric, a respondent population in adaptive polling, or multiple co-present participants in a conversation. Across these settings, the common structure is closed-loop querying under limited budgets, partial observations, noisy or selective disclosure, and a downstream target such as a collective decision, a population-level prediction, or a fairness-aware evaluation criterion (Zhao et al., 2018, Hiranandani et al., 2020, Ding et al., 15 Feb 2026).

1. Meanings of “group” in the literature

Adaptive group elicitation is best understood as a cluster of related problem classes rather than a single formalism. One line studies group decision support: given a key group of agents and a budget, the system adaptively asks cost-sensitive preference questions in order to improve the group’s eventual winner distribution under a randomized voting rule (Zhao et al., 2018). A second line studies group-fair evaluation criteria: here “group” means sensitive demographic groups in multiclass classification, and the goal is to recover a latent fair performance metric from pairwise feedback about classifier behavior (Hiranandani et al., 2020). A third line studies population-level response prediction: the elicitor must decide both which question to ask and which subset of respondents to observe, while imputing missing answers for the rest of the population (Ding et al., 15 Feb 2026). A fourth line addresses group-sensitive interaction design in multi-party human-agent conversation, where the central issue is whether an agent is perceived as addressing several people rather than one (Müller et al., 12 Jun 2025).

Sense of “group”	Elicitation target	Representative work
Key decision-making agents	Winner distribution under a randomized voting rule	(Zhao et al., 2018)
Sensitive demographic groups	Fair performance metric over group-conditioned rates	(Hiranandani et al., 2020)
Respondent population	Population-level responses to held-out questions	(Ding et al., 15 Feb 2026)
Co-located participants	Group-sensitive conversational interaction	(Müller et al., 12 Jun 2025)

Several important papers are methodologically adjacent rather than direct group models. Local utility elicitation in GAI models is a single-user framework, but it contributes local-query semantics and myopic value-of-information ideas that are reusable in adaptive elicitation more broadly (Braziunas et al., 2012). Robust active preference elicitation formalizes offline and online active querying as robust optimization with decision-dependent information discovery, but again for a single decision-maker (Vayanos et al., 2020). Personalized recourse elicitation via adaptive pairwise comparisons is also single-subject, though it offers a clean confidence-set perspective on adaptive preference refinement (Nguyen et al., 2024). This suggests that adaptive group elicitation has developed partly by extending individual elicitation primitives into settings where the object of inference is collective, structured, or group-conditioned.

2. Formal objects of inference

What is elicited varies substantially across formulations. In budgeted group preference elicitation under the Plackett–Luce model with features, the latent object is a feature-based parameter matrix $B$ governing utilities

$u_{ji}=\vec x_j^\top B \vec z_i,$

where $\vec x_j$ and $\vec z_i$ are agent and alternative features. The posterior over $\mathrm{vec}(B)$ is approximated by an asymptotic Gaussian using composite marginal likelihood, and the elicitor scores candidate questions by expected information gain divided by monetary cost (Zhao et al., 2018).

In fair performance metric elicitation, the latent object is not an agent ranking but a scale-invariant metric over group-conditioned confusion behavior. For multiclass classification with $m$ sensitive groups and $q=k^2-k$ off-diagonal rate coordinates, the metric is

$$\Psi(r^{1:m};a,B,\lambda) = (1-\lambda)\,\langle a,r\rangle + \lambda \sum_{u,v=1,\ v>u}^m \langle b^{uv}, d^{uv}\rangle,$$

where $a$ weights predictive performance, $B=\{b^{uv}\}$ weights pairwise group discrepancies, and $u_{ji}=\vec x_j^\top B \vec z_i,$0 sets the performance–fairness trade-off. The elicitation target is therefore a latent group-fair metric rather than a direct preference ordering over items (Hiranandani et al., 2020).

In natural-language latent-information elicitation, the latent object is treated through a predictive / missing-data view of uncertainty. Instead of specifying an explicit posterior over an abstract latent entity $u_{ji}=\vec x_j^\top B \vec z_i,$1, the system models uncertainty through entropy over unobserved future answers and uses a meta-learned autoregressive LLM $u_{ji}=\vec x_j^\top B \vec z_i,$2 to simulate future trajectories (Wang et al., 5 Apr 2025). Population-level adaptive group elicitation adopts a closely related predictive view, but lifts it to respondents in a graph-structured population. For respondent $u_{ji}=\vec x_j^\top B \vec z_i,$3, uncertainty about the latent entity is defined by entropy on held-out target questions,

$u_{ji}=\vec x_j^\top B \vec z_i,$4

and the selected question maximizes summed expected information gain across all respondents (Ding et al., 15 Feb 2026).

A recurring distinction is between uncertainty about individual preferences and uncertainty about group-level observables. In the collective-choice setting, the elicitor learns $u_{ji}=\vec x_j^\top B \vec z_i,$5 and then computes a winner distribution under probabilistic plurality or Borda (Zhao et al., 2018). In dynamic polling, the goal is not necessarily to recover latent ideology directly, but to improve prediction of held-out answers for the same respondent or demographic profile (Wang et al., 5 Apr 2025). This suggests that adaptive group elicitation often replaces latent-variable recovery by prediction of downstream group-relevant observables.

3. Adaptive query design

The defining property of the field is sequential, state-dependent query choice. In cost-sensitive group preference elicitation, the next design $u_{ji}=\vec x_j^\top B \vec z_i,$6 is chosen by the one-step Bayesian experimental-design criterion

$u_{ji}=\vec x_j^\top B \vec z_i,$7

where $u_{ji}=\vec x_j^\top B \vec z_i,$8 may be D-optimality, E-optimality, or minimum pairwise certainty (MPC), and $u_{ji}=\vec x_j^\top B \vec z_i,$9 is an empirically estimated question cost from Mechanical Turk timing studies (Zhao et al., 2018). The framework permits pairwise comparisons, top-choice questions, and full rankings over subsets, and its group-aware MPC criterion explicitly targets the least certain pairwise preference across all key-group agents.

Fair performance metric elicitation uses a different adaptive mechanism. The oracle supplies only relative pairwise preferences between classifiers, and the algorithm constructs structured query regions where the nonlinear metric becomes linear or unimodal in a lower-dimensional variable. It then applies binary-search-style procedures in three stages: first to recover predictive weights $\vec x_j$0, next to recover fairness weights $\vec x_j$1 by activating controlled sensitive-group disparities, and finally to recover the trade-off parameter $\vec x_j$2 through a strictly quasiconcave objective over $\vec x_j$3. The resulting query complexity is $\vec x_j$4 for $\vec x_j$5, $\vec x_j$6 for $\vec x_j$7, and $\vec x_j$8 for $\vec x_j$9 (Hiranandani et al., 2020).

Natural-language elicitation replaces analytic geometry by model-based forward simulation. Greedy expected information gain chooses

$\vec z_i$0

while a Monte Carlo Tree Search–style procedure simulates future answers autoregressively from $\vec z_i$1, keeps top-$\vec z_i$2 first-step questions, and evaluates them by rollout information gain (Wang et al., 5 Apr 2025). Reinforcement Prompt Selection adopts yet another adaptive mechanism: prompt selection is treated as sequential decision-making, the state is the current extracted information plus dialogue history, the action is a prompt type from a finite prompt pool, and the reward is normalized information gain

$\vec z_i$3

The legal-domain implementation uses DQN over four prompting strategies: exploratory, precise, corroborative, and confrontational (Wang et al., 15 Apr 2026).

The most explicit answer to the question “whom to query for what” appears in population-level adaptive group elicitation. The LLM scores candidate questions by expected information gain,

$\vec z_i$4

while a heterogeneous GNN imputes missing responses and provides respondent embeddings. After choosing a question, the method clusters respondents in embedding space and queries the $\vec z_i$5 cluster centers, thereby coupling what to ask with whom to observe (Ding et al., 15 Feb 2026). A plausible implication is that respondent selection becomes a first-class design variable once participation budgets, not only question budgets, are binding.

4. Aggregation, propagation, and group-level outputs

Adaptive group elicitation is not only about asking questions; it is also about how elicited information is turned into a group-relevant output. In group preference elicitation under Plackett–Luce with features, the aggregation layer is a randomized voting rule. If $\vec z_i$6 is the score of alternative $\vec z_i$7 under voting rule $\vec z_i$8, then the randomized rule chooses winners proportionally to scores, and the paper proves that an alternative’s winner probability is proportional to its expected total score across key agents. Under probabilistic plurality, the winner probability is the average top-choice probability over key agents; under probabilistic Borda, it is proportional to summed pairwise preference probabilities (Zhao et al., 2018).

In fair performance metric elicitation, the output is itself an evaluation functional. Pairwise discrepancies between sensitive-group rate vectors,

$\vec z_i$9

are aggregated into a fairness-violation term, which is then combined with overall predictive cost. The elicited metric can subsequently rank classifiers, and the paper’s real-data study compares rankings induced by the elicited metric with rankings under the true oracle metric (Hiranandani et al., 2020).

Population-level adaptive group elicitation uses graph propagation rather than voting. The heterogeneous graph contains respondent nodes, feature nodes, and query-choice nodes. After message passing, missing responses are imputed by

$\mathrm{vec}(B)$0

and these imputed responses are appended to histories for future rounds (Ding et al., 15 Feb 2026). Here the group output is not a winner distribution but improved prediction of held-out population responses.

A different aggregation regime appears in adaptive visual design elicitation. The adaptive stage is individual-level first: respondents answer a bi-level sequence of form-only and purchase questions, individual form and utility models are estimated with shrinkage and hierarchical Bayes, and only then are respondents clustered into four groups based on form, price, and MPG importance. The reported outputs include group-level “optimal” designs, group-level willingness-to-trade-off and willingness-to-pay type summaries, and sensitivity profiles for design elements (Kang et al., 2019). This suggests a statistical route to group elicitation in which aggregation is deferred until after personalized adaptive inference.

In human-agent interaction, aggregation may be social rather than mathematical. The museum study distinguishes group-adaptive conversation design, the technical capability to adjust responses based on whether interaction is dyadic or group-based, from group-sensitive conversation design, the user-facing perception that the agent is addressing the group. The tested intervention was primarily linguistic pluralization, with group detection via computer vision, but the relevant output variable was conversational satisfaction rather than a formal preference aggregate (Müller et al., 12 Jun 2025).

5. Empirical findings

Empirical results across the literature generally favor adaptive over random or static elicitation, but they also show that the source of the gain depends on the setting. In budgeted group preference elicitation, adaptive information-based methods substantially outperform random querying, and at budget $\mathrm{vec}(B)$1, MPC achieves about 15% less total variation distance than random querying for both probabilistic plurality and Borda; to achieve total variation distance $\mathrm{vec}(B)$2, MPC uses about 20% less money under probabilistic plurality and 23.5% less money under probabilistic Borda than random querying (Zhao et al., 2018). In fair performance metric elicitation, synthetic recovery experiments report small errors for $\mathrm{vec}(B)$3, $\mathrm{vec}(B)$4, and $\mathrm{vec}(B)$5, and on real datasets the elicited metric achieves the best NDCG and Kendall-$\mathrm{vec}(B)$6, often nearly perfect (Hiranandani et al., 2020).

Natural-language adaptive elicitation also shows consistent advantages when uncertainty estimates are decision-relevant. Across 20 Questions, dynamic opinion polling, and adaptive student assessment, the greedy EIG policy with a meta-trained model improves target-question accuracy, perplexity, Brier score, and calibration relative to a base LLM and an embedding-based baseline. The gains are especially large on “hard” targets: relative gains over random are more than 10x larger for EEDI and 20 Questions and about 5x larger for OpinionQA. The same paper reports that planning with a Base LLM can actually hurt, reducing accuracy by almost 15% on hard questions versus random, while planning substantially helps when the underlying model is the meta-trained uncertainty estimator (Wang et al., 5 Apr 2025).

The joint respondent-question formulation yields some of the clearest group-level gains. Across CES, OpinionQA, and Twin-2K, the proposed method consistently improves population-level response prediction under constrained budgets, including a $\mathrm{vec}(B)$7 relative gain on CES at a 10% respondent budget; the main text reports CES improvements ranging from 17.1% at round 1 to 12.6% by round 4 under the same 10% budget. The gains concentrate on highly sensitive respondents, and at 50% budget the paper reports, for the extreme sensitivity tier, accuracy improvements from 0.720 to 0.826 on CES and from 0.465 to 0.529 on OpinionQA when moving from random to group-relational observation (Ding et al., 15 Feb 2026).

By contrast, multi-party conversational adaptation in the museum study yields a negative result. Across two in-the-wild studies with $\mathrm{vec}(B)$8 participants interacting with either Furhat or MetaHuman, the results did not reveal a significant effect of the group-sensitive conversation design on perceived satisfaction. For Furhat, a one-way ANOVA gave $\mathrm{vec}(B)$9, $m$0, and for MetaHuman a Kruskal-Wallis test gave $m$1, $m$2 (Müller et al., 12 Jun 2025). This is important because it shows that adaptivity at the system level does not necessarily translate into a measurable group-sensitive experience.

6. Limitations, controversies, and open directions

A central limitation of the area is terminological and conceptual heterogeneity. Some papers are genuinely about collective decision support, some are about sensitive-group fairness metrics, some are about population inference, and some are about multi-party interaction design. Several adjacent contributions remain single-user: local GAI elicitation, robust active preference elicitation, adaptive recourse elicitation, and reinforcement prompt selection all provide transferable mechanisms without solving the full multi-agent aggregation problem (Braziunas et al., 2012, Vayanos et al., 2020, Nguyen et al., 2024, Wang et al., 15 Apr 2026). This suggests that “adaptive group elicitation” names a methodological family more than a settled formal problem class.

The technical limitations are equally clear. Budgeted group preference elicitation relies on a greedy one-step lookahead policy, a Gaussian posterior approximation based on composite marginal likelihood, a shared feature-based parameterization $m$3, and experiments with only a few query types and voting rules (Zhao et al., 2018). Fair performance metric elicitation restricts the metric family to a linear predictive term plus a linear penalty on pairwise absolute rate discrepancies; groups must be known, disjoint, and stable; the fairness notion is group-based and rate-based; and the guarantees require a common feasible sphere across groups together with regularity conditions (Hiranandani et al., 2020). Natural-language latent-information elicitation depends heavily on historical trajectories, simulator quality, and calibration of predictive entropy; it does not provide a full group aggregation framework, and respondent sampling is not part of its action space (Wang et al., 5 Apr 2025).

Population-level adaptive group elicitation adds respondent selection, but it inherits dependence on LLM quality, graph quality, and attribute availability, and the paper explicitly notes potential bias and representativeness issues when selecting “informative” respondents. Its evidence is based on offline replay of real survey datasets rather than live human multi-turn studies (Ding et al., 15 Feb 2026). Multi-party conversational adaptation highlights a different gap: the museum study concludes that pluralization alone may not improve the quality of interaction and emphasizes the need for multimodal strategies beyond linguistic pluralization, including gaze, turn-taking, prosodic adjustment, and stronger cues of collective address (Müller et al., 12 Jun 2025).

The literature therefore points toward several open directions already identified within the papers themselves: respondent sampling and population estimands beyond single-trajectory modeling (Wang et al., 5 Apr 2025); shared priors, transfer, or clustering across users rather than learning each user from scratch (Nguyen et al., 2024); joint optimization of what to ask and whom to ask under explicit participation budgets (Ding et al., 15 Feb 2026); and group-aware conversational systems that manage attention, participation equity, and dynamic entry or exit rather than merely pluralizing prompts (Müller et al., 12 Jun 2025). A plausible synthesis is that the most mature future formulations of adaptive group elicitation will have to combine at least three ingredients that are currently distributed across separate subliteratures: decision-aware query selection, explicit group-level aggregation or fairness objectives, and structured modeling of who should be queried, not only what should be asked.