Auditing Belief-Conditioned LLM Agents in Hidden-Information Social Deduction Games

Abstract: Evaluating LLM agents in hidden-information multi-agent settings is hard: final outcomes are high-variance and rarely reveal why an agent decided as it did. We study this in a 9-player Werewolf environment where agents act under strict, code-level information isolation, and we build an auditable framework that maintains an external belief state over hidden roles, logs belief updates and belief-action deviations as structured evidence, and supports a defensive offline improvement loop that reviews bad cases before any strategy change. Across 1,080 frozen games spanning belief-disabled, active-belief, kernel-ablation, camp-restricted, consumption-policy, and high-load arms, and including a seed-paired A0/A1 comparison, the active-belief condition is associated with substantially better good-side outcomes: in the 200-seed A0/A1 comparison the good-side win rate rises from 0.205 to 0.390 (paired McNemar $χ² = 16.4$, $p < 0.001$), with fewer irreversible witch-poison errors. We do not, however, attribute this shift to belief content. Direct action-belief consistency is low ($\approx 0.21$), and giving belief only to the werewolves helps the good side more than giving it only to the good side, which argues against a simple holder-benefit account; we therefore report the effect as an association and treat its mechanism as unresolved. The contribution is the audit framework itself: it makes the effect measurable, exposes low direct action-belief consistency, rejects an unreliable forced-consumption intervention with evidence, and separates strategy effects from load confounds. We accordingly position external belief in high-noise hidden-information games primarily as an auditable cognitive baseline that also carries decision-relevant signal, turning opaque agent behavior into replayable evidence for safer, controlled iteration.

• The paper presents an auditable framework for hidden-information LLM agents using code-level information isolation, traceable role beliefs, deviation logs, and human-reviewed offline updates.
• The active-belief condition increased good-side wins from 0.205 to 0.390 across 200 paired seeds, while wrong witch-poison targets fell from 22 to 4, although the causal mechanism remains unresolved.
• The framework shows that stronger belief compliance is not automatically beneficial: forced consumption failed when beliefs were flat, highlighting the need for confidence-gated interventions and randomized controls.

This paper presents an auditable evaluation framework for LLM agents operating under hidden information, instantiated in a 9-player Werewolf environment with strict, code-level information isolation. The central contribution is not an improved agent but a measurement infrastructure: an external belief layer that maintains role-probability distributions over hidden roles, structured logging of belief updates and belief–action deviations, and a defensive offline improvement loop that requires human-reviewed, versioned strategy changes rather than online updates from single noisy games. Across 1,080 frozen games, the paper reports a large outcome association — the active-belief condition raises good-side win rate from 0.205 to 0.390 under a 200-seed paired comparison — while explicitly declining to attribute this shift to belief content, because direct action–belief consistency is low ($\approx 0.21$) and camp-restricted arms contradict a simple holder-benefit explanation.

Motivation and positioning

The paper's starting point is that final win rate is a high-variance and diagnostically impoverished signal in hidden-information multi-agent games. A locally correct vote may lose due to downstream teammate errors; a locally poor decision may win because of opponent mistakes. When an agent poisons a teammate or ignores a strong public signal, outcome-based evaluation cannot distinguish missing evidence, poor belief tracking, prompt-induced behavior, infrastructure failures, or justified strategic override. Werewolf is chosen as a compact testbed because it combines asymmetric private information, potentially deceptive public language, and role-specific night actions whose quality is only revealed post-hoc.

Relative to prior work, the paper's emphasis is deliberately different. LLM-agent benchmarks such as AgentBench and WebArena evaluate long-horizon tasks with largely recoverable state (Liu et al., 2023, Zhou et al., 2023); Werewolf-focused systems optimize play quality or deception, including RL-guided candidate-action selection (Xu et al., 2023), tournament evaluation (Bailis et al., 2024), latent-space policy optimization (Xu et al., 7 Feb 2025), and deception measurement against human baselines (Kao et al., 20 Jan 2026). Classical belief-state machinery (POMDPs, I-POMDPs, Bayesian Action Decoder, ReBeL) uses belief for optimal decision making or equilibrium computation [kaelbling1998pomdp, foerster2019bad, brown2020rebel]. This paper instead engineers belief as a lightweight, hand-specified, auditable intermediate state whose purpose is traceability rather than optimality, and it treats the game as a trajectory-level diagnostic environment rather than a win-loss benchmark.

System design and information isolation

The environment implements a 9-player configuration (three werewolves, one seer, one witch, one hunter, three villagers) with an eight-action interface ($speak$, $vote$, $night\_kill\_nominate$, $check$, $save$, $poison$, $hunter\_shoot$, $skip$). Information isolation is enforced at the code level rather than by prompt instruction: agents receive serialized JSON contexts containing only public events, role-visible private events, alive/dead status, and legal actions, and ground-truth role maps are consulted only during post-game evaluation. A parser–canonicalizer–validator–fallback pipeline sits between LLM output and the game engine, rejecting role leakage, meta-AI phrases, and chain-of-thought text while logging all repairs.

The belief layer maintains, per observer, a full role distribution over other players, updated in log-odds space from structured events. Each update is factorized as a product of base evidence weight, evidence direction, and source credibility, so that every belief transition is attributable to a small number of named factors. The authors are explicit that this is a Bayesian-inspired heuristic with hand-specified factors, not a calibrated posterior estimator. Belief can run in shadow mode (diagnostics only) or active mode (injected into agent contexts as subjective signal), and beliefs can be locked when an observer holds private certainty.

Two further components complete the framework. Deviation-guided diagnosis records when an agent's action diverges from the belief-recommended target, with a conceptual taxonomy of harmful deviation, justified override, and strategic deviation — though large-scale automatic classification of these categories is deferred to future work. The defensive offline improvement loop aggregates batches of games, mines bad cases, generates candidate strategy insights, and leaves versioned updates for human review, deliberately refusing to rewrite prompts or rules after single trajectories.

Main empirical result: an outcome association with unresolved mechanism

The frozen corpus comprises 1,080 games under a single fixed configuration (deepseek-chat, temperature 0.6, recorded contract version and commit), spanning eight arms with zero failed games and 0% LLM-error rate. The headline result comes from the seed-paired A0/A1 comparison ($n=200$ per arm, seeds 7000–7199), where the only controlled difference is whether factorized belief summaries are injected into agent contexts. Good-side win rate rises from 0.205 to 0.390, with 58 seeds flipping to a good-side win versus 21 the other way (paired McNemar $speak$0, $speak$1), and non-overlapping 95% confidence intervals ($speak$2pp versus $speak$3pp). Both arms record 0% LLM errors, ruling out asymmetric infrastructure failure as an explanation.

Arm	Description	$speak$4	Good WR	Top-1	Top-2	Consistency
A0	Belief-disabled	200	0.205	–	–	–
A1	Active belief	200	0.390	0.686	0.825	0.207
A2	Good-side belief only	150	0.253	0.493	0.773	0.370
A3	Wolf-side belief only	150	0.313	0.899	0.899	0.092
ABL-add	Additive kernel	120	0.333	0.668	0.804	0.201
C1 / C2	Gated / forced consumption	80 each	0.412 / 0.362	0.653 / 0.688	0.802 / 0.840	0.250 / 0.280
L1	High-concurrency replica	100	0.300	0.706	0.825	0.284

The paper is unusually careful about what this result does not establish, and the caveats bear directly on interpretation. First, the belief-disabled baseline A0 retains the belief-oriented prompt structure with only the belief content removed, so prompt-structure effects are not isolated. Second, direct action–belief consistency is only $speak$5, meaning agents match the top belief recommendation at roughly one in five relevant decisions; the paper argues this is a lower bound on belief usage (it misses candidate-set narrowing and speech-level effects) but concedes the large gap between the outcome shift and this figure leaves the causal pathway open. Third, and most tellingly, the camp-restricted arms contradict a holder-benefit account: giving belief only to werewolves raises the good-side win rate more (0.313) than giving belief only to the good side (0.253), the opposite of what improved holder-side inference would predict. The authors accordingly report the effect as an association of the active-belief condition, not as evidence that belief content improves hidden-role inference, and propose two missing controls — a native no-belief baseline and a randomized-belief arm preserving prompt structure with permuted content — as the necessary next step.

Aggregate belief diagnostics indicate usable but not sharp signal: top-1 hit rate 0.686, top-2 hit rate 0.825, wolf-versus-good separability 0.356, and Brier score 0.160. The paper cautions that these sidecar aggregates pool camps and action types; werewolves trivially know teammates, inflating the pooled hit rate, so the good-side-only vote-time rate is substantially lower and only modestly above the $speak$6 chance level. Separability remains moderate, with the top suspect often only weakly ahead of the next candidates.

Concrete error-class reduction: witch poison

The most interpretable behavioral change co-occurring with the outcome shift concerns the witch's poison, an irreversible high-risk action. Without belief, the witch attempts poison in 29 of 200 games, and 76% of attempts (22 of 29) hit a good-side player. With belief, attempts drop to 11 of 200 and the wrong-poison fraction falls to 36% (4 of 11), a two-sided Fisher exact test of $speak$7. The metric cross-validates against an independently recorded poison-target-is-wolf count in the belief-signal sidecar. The paper correctly notes that per-arm Wilson intervals are wide and partly overlapping ($speak$8 versus $speak$9), so this should be read as directional; and because poison is infrequent, it cannot fully explain the aggregate win-rate shift. Its significance is that it links the outcome association to one concrete, auditable, irreversible error class rather than leaving the shift opaque.

Negative result: forced consumption fails when belief is flat

A natural response to low consistency is to force stronger alignment with belief recommendations. A seed-paired consumption-policy A/B ($vote$0 per arm) tested a gated baseline against a prompt variant requiring stronger belief-following. The intervention failed: consistency rose only from 0.250 to 0.280, and good-side win rate did not improve (0.412 versus 0.362, well within the $vote$1pp confidence intervals at this sample size). The paper's explanation — supported but not established — is that belief is often flat at key decision points, so there is frequently no sharp recommendation to enforce, and mandating obedience to weak evidence risks converting uncertainty into false confidence. The correct conclusion, as the authors frame it, is not "make agents obey belief more" but "the current signal is often not separable enough to deserve stronger control." This negative finding is presented as evidence that the framework can reject plausible-sounding interventions, not merely justify belief guidance, and it motivates confidence-gated rather than unconditional consumption policies.

Ruling out load confounds

The paper treats concurrency as an explicit experimental variable, motivated by the observation that fallback behavior is role-asymmetric: a failed witch action degrades to a skip, weakening the good side, while a failed werewolf nomination may be compensated by teammates. A high-concurrency replica (concurrency 32) of the active-belief arm matched the low-concurrency arm (concurrency 4) at 0% LLM-error rate with no context-budget violations, and its win rate remained within the range of other belief arms. On this stack, the main A0/A1 effect is therefore not explained by observed load-induced failures; more importantly, the instrumentation would detect and quarantine such an artifact if a less robust provider reintroduced it.

Stability of the substrate

The frozen corpus doubles as an infrastructure validation. In the A1 arm alone, across 11,724 recorded decisions, LLM-error and retry rates were both 0%, rule-validation fallback fired on 0.24% of decisions, no role/meta/CoT leakage passed through canonicalization, and no context exceeded the 4,000-token budget. The belief infrastructure logged 82,967 belief saves and 746,703 belief-curve points, and the frozen run records 1,138 passing tests covering information isolation, contract validity, and game termination. These results establish that the hidden-information assumptions are enforced by the experimental substrate, which is a precondition for the belief and deviation analyses to be meaningful at all.

Limitations and open questions

The paper is candid about the limits of its claims. The mechanism behind the win-rate gain is unresolved: low direct consistency, the counterintuitive camp-restricted pattern, and the prompt-structure confound in A0 all remain open, and the proposed A0-native and A-rand controls have not been run. The belief update rules are hand-specified and may miss linguistic subtleties, long-horizon deception patterns, and role-specific tactics. The belief layer is not a theory-of-mind model — it tracks role likelihoods but not nested beliefs — and the offline loop is deliberately not a fully autonomous learning system. The deviation taxonomy (harmful/justified/strategic) is conceptual only, without quantitative validation or inter-rater agreement. Smaller arms ($vote$2–$vote$3) are reported as exploratory without multiple-comparison correction, and the single-model evaluation leaves cross-model robustness untested. Finally, Werewolf is a controlled game, so claims about transfer to domains such as fraud detection or negotiation are architectural hypotheses rather than empirical findings.

Conclusion

The paper demonstrates that hidden-information LLM-agent evaluation can be made auditable: strict code-level information isolation, factorized and traceable belief updates, deviation instrumentation, and a defensive offline improvement loop together turn opaque agent decisions into replayable evidence. Its strongest quantitative finding — the active-belief condition's association with a good-side win-rate rise from 0.205 to 0.390 ($vote$4) and a wrong-poison reduction from 22 to 4 — is consistently paired with the admission that the causal mechanism is not isolated. The framework's demonstrated value is diagnostic: it surfaces a concrete irreversible error-class reduction, rejects a forced-consumption intervention with evidence, and separates strategy effects from load confounds. The central open question the paper leaves is whether the outcome shift reflects belief content, prompt structure, or indirect behavioral change, and the proposed randomized-belief and native-baseline controls are the specific experiments required to answer it.