Bayesian Elicitation with LLMs

Updated 28 May 2026

Bayesian elicitation with LLMs is an emerging paradigm that integrates probabilistic reasoning with language models to extract expert knowledge and quantify uncertainties.
The methodology incorporates direct Bayesian estimation, surrogate modeling, and active querying to design experiments and refine probabilistic priors.
Empirical findings reveal challenges in LLM calibration and scalability, underscoring the need for robust uncertainty adjustment and protocol-specific optimizations.

Bayesian elicitation with LLMs denotes a growing set of methodologies that harness probabilistic reasoning, structured uncertainty, and active information-gathering algorithms within LLM workflows. These approaches leverage LLMs both as sources and surrogates of expert knowledge, integrating Bayesian inference principles into tasks such as preference modeling, causal discovery, prior extraction, uncertainty quantification, and optimal information acquisition. This article surveys the dominant paradigms, mathematical frameworks, and empirical results in state-of-the-art Bayesian elicitation with LLMs, as surfaced across major recent research on arXiv.

1. Foundations: Bayesian Elicitation and LLMs

Bayesian elicitation transforms latent expert knowledge, preferences, or priors into explicit, probabilistic objects, typically by querying human experts. LLMs substitute or augment humans both by encoding implicit world-knowledge acquired through pretraining and by generating, scoring, and adapting evidence in response to structured queries.

A general Bayesian elicitation problem involves an unknown parameter $\theta$ (e.g., prevalence rate, utility function, graph structure), a prior $p(\theta)$ (possibly to be elicited), and a data-generation or feedback process $p(d\mid\theta)$ . LLMs can be placed in several roles:

Direct Bayesian estimator: Given a prompt, LLMs generate point or interval estimates with an associated uncertainty claim, which serves as a "Bayesian expert opinion" (Hobor et al., 2 Apr 2026).
Probabilistic knowledge extractor: LLMs are queried for conditional, marginal, or structural relationships (e.g., Bayesian network CPTs) (Nafar et al., 21 May 2025).
Uncertainty-aware surrogate: LLMs are framed as posterior approximators or surrogates for Bayesian inference or active learning (Lei et al., 6 May 2026, Choudhury et al., 28 Aug 2025).
Prior sampler: Iterated in-context learning is used to sample from the model's implicit Bayesian prior (Zhu et al., 2024).

LLM-based Bayesian elicitation strictly requires evaluation and calibration, since LLMs may exhibit systematic overconfidence, miscalibration, or domain-dependent biases (Hobor et al., 2 Apr 2026).

2. Mathematical and Algorithmic Frameworks

2.1 Probabilistic Modeling and Inference

The core of Bayesian elicitation is updating beliefs over unknowns using Bayes' rule as new evidence (real or LLM-generated) accumulates.

Expert knowledge extraction: For Bayesian network parameterization, LLMs are prompted per-node/parent configuration, returning $P(X_i = x \mid \mathrm{Pa}(X_i) = \pi)$ , which are normalized over all $x$ (Nafar et al., 21 May 2025).
Prior elicitation: Iterated learning alternates between sampling $\theta$ from $p(\theta|d)$ using the LLM and generating new data $d$ via $p(d|\theta)$ , producing samples from the LLM's implicit prior (Zhu et al., 2024).
Latent-state models: LLMs' variable outputs (e.g., on repeated classification) are treated as noisy measurements of latent variables, and Bayesian latent class models are applied to calibrate and recover the ground truth (Zhang et al., 27 Oct 2025).

2.2 Active Querying and Bayesian Design

Bayesian experimental design frameworks select queries or items to maximize expected information gain (EIG) about $\theta$ (Choudhury et al., 28 Aug 2025), or related mutual information criteria:

EIG computation: For candidate query $p(\theta)$ 0,

$p(\theta)$ 1

Practical acquisition: Approximate EIG using Rao–Blackwellization, sample candidate $p(\theta)$ 2 from current belief $p(\theta)$ 3, estimate predictive distributions via LLM logits or outputs (Choudhury et al., 28 Aug 2025).

For Bayesian preference elicitation, frameworks such as BAL-PM (Melo et al., 2024) and PEBOL (Austin et al., 2024) use composite acquisition functions—combining epistemic uncertainty, predictive entropy, or diversity in semantic feature space—to select the most informative queries in each batch.

3. Practical Implementations and Workflows

3.1 Elicitation Protocols

LLM-as-expert workflows: Chain-of-request or dual-expert strategies use one LLM to propose causal/structural elements and another to critique or verify (e.g., structure learning in BNs) (Shaposhnyk et al., 14 Apr 2025).
Prompt orchestration: Systematic prompting is used to extract conditional probabilities, feature sets, or model confidences. For example, each CPT entry in a BN is queried independently at low temperature, with responses normalized to valid distributions (Nafar et al., 21 May 2025).
Iterated Gibbs samplers: For prior elicitation, LLMs are alternately sampled for $p(\theta)$ 4 and new data $p(\theta)$ 5, with sufficient chains and iterations guaranteeing convergence to the implicit prior (Zhu et al., 2024).

3.2 Bayesian Optimization and Surrogates

Frameworks such as ReElicit (Lin et al., 18 May 2026) use LLM-elicited semantic feature spaces coupled with a Gaussian process Bayesian optimizer. The LLM dynamically proposes interpretable features, maps past prompts into feature space, and GP models guide the selection of next candidate prompts, which the LLM then realizes/refines into deployable prompts.

Surrogate modeling with LLMs requires careful alignment between prompt protocol (the LLM's "prior" and inference rules) and traditional Bayesian surrogate models (e.g., GPs), as different prompt designs and query protocols induce different posterior predictions and uncertainty behaviors (Lei et al., 6 May 2026).

4. Empirical Findings: Calibration, Uncertainty, and Scalability

4.1 Calibration and Overconfidence

Out-of-the-box LLM uncertainty estimates are almost universally overconfident. Across eleven contemporary LLMs, 95% credible intervals contain the true value only 9–44% of the time, far below the nominal coverage (Hobor et al., 2 Apr 2026). This overconfidence can be corrected post hoc using split conformal prediction, which expands intervals to restore coverage, albeit at the cost of sharpness (Hobor et al., 2 Apr 2026).

4.2 Surrogate-Specific Uncertainty and Protocol Effects

LLM surrogates' uncertainty is tightly linked to prompt type and elicitation protocol. Structural prompts act as effective priors (e.g., specifying function family shape), and pointwise vs. joint query protocols induce markedly different predicted posteriors and uncertainty surfaces (Lei et al., 6 May 2026). The uncertainty-alignment criterion, a rank correlation between surrogate uncertainty and sample-consistent reference ambiguities, diagnoses whether LLM uncertainty tracks true epistemic ambiguity.

4.3 Scalability and Task-Specific Observations

Scalability of Bayesian elicitation with LLMs is bounded by prompt throughput, feature-space dimensionality, and LLM context limitations. In structure learning for BNs, performance and precision decrease as graph size grows (Babakov et al., 2024). Preference modeling frameworks remain sample-efficient for moderately large item sets, but require adaptation—e.g., pre-clustering, sparse updates—at scale (Austin et al., 2024).

Practical robustness varies by task: LLMs perform well and produce probabilistically meaningful priors on well-covered domains (e.g., demographic statistics, personality, standard BNs), but may display anchor and miscalibration effects in specialized or data-sparse domains (Hobor et al., 2 Apr 2026, Nafar et al., 21 May 2025).

5. Applications: Causal Modeling, Preference Elicitation, Evaluation, and More

LLM-driven Bayesian elicitation is being deployed in a wide array of applications:

Probabilistic Causal Modeling: LLMs can generate, critique, and refine BN structures, often yielding equal or lower entropy than human-elicited or statistically induced graphs, albeit with a risk of hallucinated edges or inherited training biases (Shaposhnyk et al., 14 Apr 2025).
Preference Elicitation and RLHF: Bayesian active learning via EIG-guided selection minimizes human feedback requirements in large-scale model alignment and RLHF datasets (Melo et al., 2024, Handa et al., 2024).
Automated Evaluation Calibration: Bayesian Dawid–Skene and related models post hoc calibrate LLM-as-evaluator win-rates, improving alignment with human preferences for automatic text generation evaluation (Gao et al., 2024).
Measurement under Stochasticity: Bayesian latent-state models account for LLM output variation and error, producing calibrated estimates, credible intervals for individual predictions, and base-rate uncertainty quantification (Zhang et al., 27 Oct 2025).

6. Best Practices, Limitations, and Future Directions

Best Practices

Explicit protocol specification: Treat prompt wording and query protocol as part of the Bayesian model—empirically, these act as effective priors and inference operators (Lei et al., 6 May 2026).
Uncertainty calibration: Apply diagnostic tools (e.g., ECE, uncertainty-alignment), and statistical recalibration (e.g., conformal prediction) before deploying LLM-derived posteriors in decision-making (Hobor et al., 2 Apr 2026).
Integration with data: Combine LLM-derived priors or CPTs with small-sample real data via Bayesian pooling or pseudo-counts for improved BNs (Nafar et al., 21 May 2025).
Iterative design: Use re-elicitation to adapt feature spaces in dynamic tasks (e.g., prompt optimization) (Lin et al., 18 May 2026).

Limitations and Open Problems

Scalability constraints: Context window, feature dimensionality, and token throughput can limit large-graph or large-pool elicitation (Babakov et al., 2024, Austin et al., 2024).
Domain generality: LLMs' priors are strong on widely-covered topics but unreliable in specialized or emerging domains (Hobor et al., 2 Apr 2026).
Bias and hallucination: LLM-generated causal or probabilistic structures may reflect pretraining artifacts or encode implausible relations; mitigation involves cross-model validation, SEM screening, and human review (Shaposhnyk et al., 14 Apr 2025).
Protocol sensitivity: Unaccounted changes in prompt or query format may shift the induced surrogate belief and downstream acquisition trajectories (Lei et al., 6 May 2026).
Simulation-to-human gap: Many protocols are tested with simulated users or evaluators; rigorous human-in-the-loop validation remains outstanding in most applications (Austin et al., 2024).

Future Directions

Approaches under development include richer acquisition criteria (e.g., multi-step active design (Choudhury et al., 28 Aug 2025)), adaptive feature extraction, variational or neural posterior approximators for richer Bayesian representations, open-ended and multi-format feedback, and joint human–AI decision pipelines in high-stakes or data-scarce settings.

Table: Illustrative Bayesian Elicitation Protocols with LLMs

Application	Elicitation Protocol	Reference
BN parameterization	Zero-shot or few-shot node × parent CPT query	(Nafar et al., 21 May 2025)
Structure elicitation	Multi-LLM proposal + majority voting + contamination check	(Babakov et al., 2024)
Preference modeling (active)	BAL-PM: Bayesian ensemble + epistemic/diversity	(Melo et al., 2024)
Prior extraction	Iterated in-context Gibbs sampler	(Zhu et al., 2024)
Surrogate optimization	Semantic feature elicitation + GP acquisition	(Lin et al., 18 May 2026)
Win-rate calibration (evaluation)	Bayesian Dawid–Skene/BWRS meta-model	(Gao et al., 2024)
Uncertainty quantification over prompts	MHLM proposal MCMC on prompt space	(Ross et al., 11 Jun 2025)

This table presents a concise mapping from target application to documented LLM-centric Bayesian elicitation protocol and primary reference.

7. Conclusion

Bayesian elicitation with LLMs establishes a suite of probabilistic frameworks that leverage the semantic, generative, and adaptive capabilities of LLMs for structured uncertainty modeling and sample-efficient expert knowledge extraction. These paradigms blend Bayesian design and inference principles with modern large-model architectures, yielding adaptive, statistically interpretable systems that improve alignment, evaluation, and decision support across diverse domains. Researchers must attend carefully to protocol specification, robust calibration, and the integration of LLM-sourced priors with empirical data and expert review to ensure rigorous, trustworthy outcomes.