ABBEL: Belief Bottlenecks in Agent Planning

• ABBEL is a framework that condenses internal and environmental knowledge into concise, human-readable belief states, facilitating efficient and interpretable decision-making.
• It uses modular architectures that separate belief summarization from action generation, enabling safe, transparent multi-agent communication and cognitive auditability.
• Reinforcement learning optimizes belief accuracy, length penalties, and communication protocols, resulting in improved success rates, reduced memory usage, and enhanced task performance.

Acting through Belief Bottlenecks Expressed in Language (ABBEL) is a paradigm for agent reasoning and planning that enforces the compression of internal state and environmental knowledge into natural-language “belief states.” ABBEL refactors sequential or multi-agent decision processes so that, instead of leveraging full memory or raw sensorimotor traces, agents operate through interpretable, summary bottlenecks—structured linguistic representations of task-relevant uncertainties, commitments, and hypotheses—with the dual objectives of maximizing decision efficiency and aligning cognitive transparency with epistemic control.

1. Formal Foundations of Belief Bottlenecks

ABBEL generalizes the notion of belief state $b_t$ to explicit, human-readable linguistic summaries. At time $t$, the agent maintains $b_t$—a textual condensation of information gathered so far—updated via

$$b_t = \mathrm{Update}(b_{t-1}, o_t) \approx p(\theta \mid b_{t-1}, o_t),$$

where $\theta$ denotes latent, task-relevant variables and $o_t$ the most recent observation (Lidayan et al., 23 Dec 2025). This belief bottleneck intervenes between perception and action selection: all downstream policies operate exclusively on $b_t$ via

$a_t \sim \pi(a \mid b_t).$

Representations may take the form of structured natural language (e.g., “Letters ruled out: C, O; contains A, R at position 3, 4.”) or symbolic belief grammars (see CoBel-World) (Wang et al., 26 Sep 2025).

Belief bottlenecking extends to multi-agent communication, where IEC and related protocols encode and decode belief state into compressed utterances, typically via low-dimensional vectors or discrete message sets that mediate coordination (Ye et al., 2022).

2. Architectures and Modular Pipelines

ABBEL architectures bifurcate agent cognition into distinct “belief summarization” and “action generation” modules. In the standard LLM agent pattern:

• The belief summarizer receives the prior belief, last action, new observation, and a fixed summarization prompt, returning $b_t$.
• The actor module (possibly an independent LLM prompt) receives only the current $b_t$, selecting $a_t$ (Lidayan et al., 23 Dec 2025).

This interface enforces a strict information bottleneck: all agent reasoning, plan formation, and environment interaction is mediated through the latest textual belief state. In symbolic approaches (CoBel-World), agents jointly evolve a belief-world ontology, representing both zero-order (facts about the environment) and higher-order (other agents’ beliefs) states. Internal structures include:

• zeroBeliefs: static facts (e.g., “Alice BELIEVE <cup>(123) IN <kitchen>(2000)”)
• firstBeliefs: nested models of collaborator beliefs All communication, miscoordination detection, and planning are gated exclusively by queries over these belief sets (Wang et al., 26 Sep 2025).

In multi-agent RL, intention-embedded communication (IEC) encodes private beliefs as latent vectors, compresses these into fixed-size messages, and evolves grammar and coordination through alternated “babbling” phases—jointly tuning sender and receiver to maximize reward under communication bottlenecks (Ye et al., 2022).

3. Reinforcement Learning and Epistemic Control

ABBEL incorporates reinforcement learning post-training to optimize both the faithfulness and efficiency of belief updates. The composite reward at each timestep includes:

• $r_{\rm task}(o_t, a_t)$: external task performance
• $r_{\rm grade}(b_t)$: alignment of $b_t$ with ground-truth or high-likelihood posterior (belief grading)
• $r_{\rm len}(b_t) = -\lambda |b_t|$: explicit penalty for belief verbosity

Agents are trained to maximize the cumulative discounted sum

$$J(\pi) = \mathbb{E}_\pi\left[\sum_{t=1}^T \left( r_{\rm task}(o_t, a_t) + r_{\rm grade}(b_t) + r_{\rm len}(b_t) \right)\right]$$

(Lidayan et al., 23 Dec 2025). RL tuning—especially with domain-specific belief graders and length penalties—mitigates error propagation (incorrect or bloated beliefs leading to compounding failures) and can outperform agents with access to raw history or uncompressed state (Lidayan et al., 23 Dec 2025). IEC similarly integrates RL with a communication-bottleneck ELBO term, balancing reward maximization and compactness/accuracy of belief state transmission (Ye et al., 2022).

Belief filtering—epistemic control through content-aware, modular operators on linguistic belief fragments—further regulates the update and admissibility of internal beliefs, enhancing both safety and cognitive auditability. Filters act on semantic sectors and abstraction levels, enabling selective acceptance, normalization, or suppression of state fragments (Dumbrava, 8 May 2025).

4. Inter-agent Communication and Theory-of-Mind Modeling

ABBEL subsumes key advances in belief-embedded communication. Agents maintain latent, evolving models of partner intentions—authorized belief vectors continually updated from partner messages and environmental traces. In IEC, a variational module infers these from compressed broadcasts, while policy and RNN states co-evolve to maximize joint reward (Ye et al., 2022). Empirical findings confirm that such joint modeling:

• Accelerates convergence and coordination efficiency (up to 50% faster than baseline methods)
• Ensures agents learn a compact “grammar” of intentions, harmonizing message entropy and mutual information between belief and hidden state
• Is highly sensitive to ablation—removal of explicit belief inference or restriction of communication channels yields dramatic drops in final reward

In CoBel-World, the collaborative belief-world is initialized through a propose-and-revise procedure, parsing task descriptions into consensus templates and belief rules. Zero-shot and Bayesian-style belief updates synchronize agent plans and detect misaligned or redundant actions, reducing communication cost and boosting collective task success (Wang et al., 26 Sep 2025).

5. Experimental Benchmarks and Quantitative Results

ABBEL efficacy has been demonstrated across diverse sequential and collaborative environments:

• In sequential puzzles (Wordle, Mastermind), ABBEL with belief grading and length penalties exhibits a 20% absolute increase in combination lock success rates and substantially lower cumulative regret versus full-history baselines (Lidayan et al., 23 Dec 2025).
• In collaborative coding (ColBench), ABBEL-style agents operate near full-context test pass rates, while halving the memory footprint and maintaining concise belief representations.
• Multi-agent environments (Predator–Prey, Traffic Junction, Level-based Foraging) show 20–30% improvements in final returns and accelerated learning (≈2×) when belief modeling and communication bottlenecks are active (Ye et al., 2022).
• In embodied multi-agent setups (TDW-MAT, C-WAH), CoBel-World reduces communication costs by 22–60% and increases task completion efficiency by up to 28% (Wang et al., 26 Sep 2025).

A tabular summary outlines ABBEL’s performance vs. baselines:

Task	ABBEL Success Rate	Memory Use	Improvement (Baseline)
Combo Lock	+20%	<50% MEM1	Lower regret
ColBench	Near full-pass	~50% tokens	Matched baseline
TDW-MAT/C-WAH	+4-28%	-22–60% comms	Above SOTA baseline

6. Safety, Alignment, and Cognitive Auditability

Belief bottlenecks, especially in frameworks deploying belief filtering over a semantic manifold, are architecturally suited for AI safety and alignment. Filters attach to semantic sectors of internal belief fragments, enabling fine-grained containment and intervention—unsafe ideas are blocked upstream of planning, and all expressive content and update steps are transparent for audit (Dumbrava, 8 May 2025). In practical scenarios (e.g., delivery drone navigation), filter modules enforce route compliance by redacting belief fragments advocating risky behavior (e.g., entering restricted zones), resulting in robust epistemic control.

Epistemic bottlenecks thus serve not only to enforce interpretability and memory efficiency but embed operational safety through principled modular governance. All ABBEL pipelines preserve detailed logs of semantic filtering decisions, supporting forensic traceability and continuous alignment monitoring.

7. Design Principles and Open Questions

ABBEL is grounded in the rational modeling of communication, integrating belief- and action-oriented utility functions into speaker-listener dynamics (Sumers et al., 2021). The design recipe entails:

• Defining listener models over latent beliefs induced by utterances
• Specifying task reward and belief accuracy objectives
• Training agents to maximize combined epistemic and instrumental utility through RL and communication bottleneck regularization

Key design guidelines emphasize the composition of tailored belief prompts and grader functions, careful tuning of length penalties, and modular filters to steer summary sufficiency versus compression. Open questions include extending ABBEL to pragmatic listener recursion (RSA-style), iterated dialogues, meta-learned epistemic/action balances, and domain-specific symbolic grammars for open-world deployment (Sumers et al., 2021).

A plausible implication is that ABBEL, with modular belief filtering and RL optimization, may enable scalable, verifiable cognitive architectures for safe, transparent, and efficient agent reasoning, generalizing across diverse environments and collaborative tasks.