Elicitation Loop in Human-in-the-Loop Systems

• Elicitation loop is a structured, interactive process that iteratively queries experts to uncover latent parameters and preferences.
• The process employs Bayesian updates and information gain-based query selection to refine model inferences efficiently.
• It is applied in domains such as personalized recommendation, causal discovery, and algorithmic recourse to enhance system alignment with human criteria.

An elicitation loop is a structured, interactive process that iteratively queries a human expert or end-user to infer hidden parameters, preferences, or latent knowledge relevant to a computational problem. Elicitation loops are foundational in preference elicitation, human-in-the-loop optimization, algorithmic recourse, and other domains requiring the alignment of automated systems with unobserved human criteria. The loop comprises repeated cycles of targeted query selection, user response collection, belief updating (often Bayesian), and adaptive query regeneration based on current uncertainty or information gain.

1. Formal Structure and Functional Elements

At its core, an elicitation loop consists of the following elements:

• Parameterization: A latent parameter vector $\theta$ (e.g., user cost weights, utility parameters, rule sets) governs observable phenomena or system outcomes.
• Prior Distribution: An initial distribution $P(\theta)$ or set of priors reflects population-level or subjective beliefs.
• Query Generation: At each iteration, the algorithm designs a targeted query (e.g., choice set, local comparison, rule request) aimed at distinguishing between plausible values of $\theta$.
• Response Model: A formal model $P_R(\cdot|\theta)$ links $\theta$ to the user’s observable response, accommodating noise or stochasticity in real-world settings.
• Posterior Update: Using Bayes’ rule, the prior is refined into a posterior $P(\theta|D^{(t)})$ after $t$ rounds, where $D^{(t)}$ includes all observed queries and responses up to iteration $t$.
• Query Selection Criterion: The next query is typically chosen to maximize a measure of information gain (e.g., Expected Utility of Selection, value of information, predictive entropy reduction) or minimize uncertainty relative to the decision objective.
• Termination: The loop may stop when sufficient certainty is reached, a query budget is exhausted, or the user accepts a recommended plan.

This interaction is algorithmically structured in pseudo-code representations (e.g., PEAR Algorithms 1–2 (Toni et al., 2022), GAI Algorithm loops (Braziunas et al., 2012), particle-based elicitation (Bonilla et al., 1 Feb 2026)) that expose each functional block and guarantee reproducibility.

2. Bayesian Update and Information Gain in Elicitation

The central mechanism of most elicitation loops is Bayesian inference driven by adaptive query selection. At round $t$, the posterior is recursively updated:

$$P(\theta|D^{(t)}) \propto P(D^{(t)}|\theta)P(\theta)$$

where $P(D^{(t)}|\theta)$ composes the likelihood of all observed query–response pairs, typically modeled as a product of response probabilities per round. In PEAR, these take the form (noiseless and logistic):

$$P_{NL}(O\!\leadsto\!I|\theta) = \prod_{I'\in O, I'\neq I} \mathbf1[C(I|\theta) < C(I'|\theta)]$$

$$P_{L}(O\!\leadsto\!I|\theta) = \frac{\exp(-\lambda C(I|\theta))}{\sum_{I'\in O} \exp(-\lambda C(I'|\theta))}$$

For query selection, information gain-based policies are dominant. In PEAR, the Expected Utility of Selection (EUS) for a choice set $O$ is

$$\mathrm{EUS}(O|D^{(t)}) = \sum_{I\in O} P_R(O\leadsto I|D^{(t)}) \, U(I|D^{(t)})$$

where $$U(I|D^{(t)}) = -\int_\theta C(I|\theta) P(\theta|D^{(t)}) d\theta$$. In Causal Preference Elicitation, the Expected Information Gain is

$$\mathrm{EIG}_t(i,j) = H_t^{ij} - \mathbb E_{W\sim q_t}[H(p_\theta(Y_{ij}|W))]$$

with $H_t^{ij}$ the entropy of the posterior-predictive distribution and $Y_{ij}$ an edge-orientation judgment (Bonilla et al., 1 Feb 2026).

3. Instantiations Across Domains

Elicitation loops appear in numerous applied frameworks:

Framework	Latent Object	Query Type
PEAR (Personalized AR)	Action cost $\theta$	Choice-sets (interventions)
GAI Utility Models	Local utilities $v_j$	Local threshold queries
CaPE Causal Discovery	DAG structure $W$	Edge existence/orientation
Plackett-Luce Aggregation	Ranking model $B$	Agent ranking/top–$k$ queries
Elicitron (LLM)	User needs	Simulated agent interview
PGPlanner	Planning preferences	Task-method query

In PEAR, algorithmic recourse actions are tailored by learning user-specific effort parameters through interactive choice queries, using greedy submodular optimization for choice set selection and updating a mixture-Gaussian posterior (Toni et al., 2022). In CaPE, each iteration queries the expert on a highly uncertain edge of a causal DAG, updating a particle approximation to the posterior and aggressively collapsing entropy over the combinatorial space (Bonilla et al., 1 Feb 2026). In GAI utility elicitation, VOI-guided local threshold comparisons tune the marginals of subutility functions, allowing tractable update and selection even in high-dimensional multiattribute domains (Braziunas et al., 2012).

4. Query Generation Algorithms and Submodularity

Efficient selection of queries (choice sets, pairs, etc.) is paramount due to combinatorial explosion and cognitive constraints. In PEAR, under the noiseless response model, the EUS objective is submodular, motivating a greedy construction with $1-1/e$ approximation guarantees for optimal set selection (Algorithm 2: SUBMOD-CHOICE). In GAI models, the maximization of the expected-value-of-information (EVOI) over local suboutcomes and thresholds is performed for each factor, with the maximal EVOI query deployed in each loop (Braziunas et al., 2012). In cost-aggregating preference settings, the ratio of information gain to question cost determines the optimal query under budget constraints (Zhao et al., 2018). Elicitron leverages diversity metrics and context-aware agent generation to ensure that the pool of simulated user agents and needs spans maximal region of the design space before analyzing coverage and re-generating as necessary (Ataei et al., 2024).

5. Convergence, Stopping Criteria, and Empirical Guarantees

Elicitation loop convergence is characterized by posterior concentration, diminishing regret, or achieving a reduction in the candidate/uncertainty set to a prescribed threshold. In PEAR, as the number of interaction rounds $T \to \infty$, the Bayesian posterior concentrates on the ground-truth parameter $\theta^{GT}$ and decision regret vanishes. Empirically, normalized regret drops below $0.2$ after five queries for $k=2$ (choice set size), and below $0.1$ for $k=4$ (Figure 1 in (Toni et al., 2022)). PEAR’s personalized plans are $30$–$50\,\%$ more cost-efficient compared to non-personalized baselines after $T=10$. In CaPE, the average predictive entropy and structural Hamming distance (SHD) to the ground-truth DAG decrease monotonically with each query, outperforming random and uncertainty-based policies (Bonilla et al., 1 Feb 2026).

6. Variations: Noiseless, Noisy, and Adaptive Response Models

A key axis of distinction is the user response model:

• Noiseless (deterministic best-response): user always selects the least-cost or dominant action.
• Logistic/noisy: user acts according to a softmax or probabilistic model over costs/utilities.

The noiseless model enables efficient submodular maximization and exact pruning (as in PEAR and GAI when the reward function is submodular), whereas the logistic/noisy models require Bayesian inference or Monte Carlo approaches (ensemble slice sampling in PEAR, particle filtering in CaPE). Empirical studies consistently demonstrate that a handful of rounds suffice for posterior concentration or decision optimality even under moderate response noise (Toni et al., 2022, Bonilla et al., 1 Feb 2026).

7. Impact and Theoretical Foundations

The elicitation loop paradigm grounds a wide class of human-in-the-loop optimization systems in a feedback-driven process, making explicit the connection between query selection, user adaptation, and model identification. Theoretical guarantees (submodularity, converge-to-truth, anytime approximate optimality) support rigorous deployment, while empirical results across domains (recourse, preference modeling, causal discovery, planning, requirements engineering) validate rapid convergence, improved alignment, and reduced user or expert burden.

The approach makes no a priori assumptions regarding the observability of user preferences or parameters, enabling principled interaction where information is most valuable. This general framework is extensible to complex, structured models, multiple agents, probabilistic and deterministic feedback, and settings where the cost of querying itself must be explicitly incorporated into the loop (Toni et al., 2022, Braziunas et al., 2012, Zhao et al., 2018, Bonilla et al., 1 Feb 2026).