- The paper introduces EPC-AW, a novel workflow that improves planning reliability by calibrating epistemic judgments during multi-agent planning.
- It demonstrates an average success rate improvement of 9.75% over non-repair methods across benchmarks like HotpotQA and GAIA.
- The study reveals that cross-agent consistency and dynamic epistemic refinement are key to suppressing latent planning failures.
Characterizing Epistemic Miscalibration in Multi-Agent Planning
The paper develops a formal framework for understanding failures in LLM-driven multi-agent systems that occur despite perfect execution, identifying epistemic miscalibration during planning as a distinct root cause. In these systems, the planning agent generates a sequence of actions to fulfill a user query by iterative information acquisition and tool invocation. Even when all actions are correctly executed, the system can fail if its planning feasibility assessments are misaligned with the underlying factual situation. The latent nature of this failure mode means plans can appear coherent and executable while being fundamentally infeasible with respect to task objectives—a property not captured by conventional diagnosis focused on execution errors.
Epistemic miscalibration is further characterized as dynamic: as agents acquire new evidence, their feasibility estimates change, potentially obfuscating prior signs of miscalibration and enabling erroneous planning patterns to recur. The paper formally defines subjective and objective feasibility for any plan, highlighting that miscalibration emerges from a divergence between judgments under partial information and what is justified by complete task knowledge.
EPC-AW: Agentic Workflow for Epistemic Calibration
To address these latent and dynamic failures, the authors introduce the Epistemic Planning Calibration Agentic Workflow (EPC-AW), structurally separating planning, execution, and diagnosis within a multi-agent architecture comprised of a Planner, Executor, and Diagnoser, each maintaining heterogeneous epistemic states via private memory structures. EPC-AW shifts error mitigation from execution-time corrections—typical of prior approaches—to planning-time calibration, with two key components:
Information-consistency-based Plan Selection (IPS): Within each round, the Planner generates a set of candidate plans. IPS does not directly verify plan feasibility but instead selects plans whose evaluations remain stable across agents holding distinct information. Cross-agent consistency scores are computed by contrasting each agent’s assessment with predicted evaluations under alternative epistemic states. Plans demonstrating robustness to epistemic variation are favored, functioning as a direct planning-time diagnostic of latent miscalibration.
Consistency-guided Epistemic State Refinement (CESR): Over rounds, EPC-AW detects divergences between Planner-selected and IPS-selected plans. Such discrepancies signal epistemic miscalibration, triggering induction of lightweight epistemic constraints. These constraints are accumulated in the Planner’s memory, informing subsequent planning and suppressing recurrence of previously identified miscalibration patterns as information evolves.
Empirical Assessment
Experiments are conducted on six benchmarks spanning reasoning-centric and retrieval-centric tasks (e.g., Bamboogle, 2Wiki, HotpotQA, Musique, GAIA, MedQA), all rooted in multi-step agentic workflows. Baselines include No-Repair (single forward plan, no recovery), Retry (local planning re-generation), Rollback (global state reversal), with all methods implemented under the AgentFlow framework.
EPC-AW achieves an average absolute success rate improvement of 9.75% compared to No-Repair and outperforms Retry and Rollback by 4.64% system-wide. This advantage holds across all datasets, demonstrating that planning-time calibration via IPS and CESR robustly suppresses latent epistemic failures and delivers consistent gains over execution-level repairs.
Ablation studies isolate the contributions of IPS and CESR. IPS alone offers gains in compositional reasoning but can induce excessive conservatism in search-heavy tasks, while CESR rectifies this by guiding planning via cross-round feedback, maintaining exploration where justified and steering away from epistemically unsupported paths. Full EPC-AW consistently surpasses both partial variants, with improvements exceeding +15% in retrieval-intensive settings.
The paper further contrasts IPS with Mean-Score Aggregation (naive cross-agent averaging), revealing that IPS delivers stronger mitigation of epistemic variance on tasks with pronounced cross-agent disagreement (e.g., HotpotQA, GAIA), whereas simple aggregation is insufficient to suppress systemic latent miscalibration.
Generalization tests across backbone LLMs (Qwen3-14B, DeepSeek-R1-32B) indicate that EPC-AW’s improvements (+11.18% and +11.13% over No-Repair, respectively) hold across architectures, evidencing that calibration gains stem from workflow design rather than model specifics.
Cost analysis establishes that EPC-AW preserves asymptotic time and token complexity, with modest overhead from multi-plan evaluation and constraint accumulation offset by substantial reliability improvements.
Practical and Theoretical Implications
The formalization and mitigation of epistemic miscalibration contribute to a shift in designing robust LLM-based multi-agent systems. Unlike prior approaches that focus on execution anomalies, EPC-AW targets epistemic fragility in the planning phase, demonstrating that calibration of feasibility assessments is as critical as correctness of actions. Information-consistency signals and memory-driven constraint feedback offer a principled, scalable mechanism to detect and suppress latent failures that would otherwise persist in real-world deployments.
On a practical level, EPC-AW enables system-level planning reliability under evolving information, supporting applications in reasoning-intensive QA, open-world navigation, and tool-based assistant workflows. It is extensible to different LLM backbones and accommodates heterogeneous agent architectures.
Theoretically, the work raises new questions regarding optimal aggregation of epistemic states in cooperative agent systems, persistent calibration under open-ended evidence acquisition, and dynamic constraint induction. Future developments may explore integration with probabilistic epistemic logic, adaptive constraint learning, and scalable calibration in deeply nested agent workflows—potentially setting new standards for reliability guarantees in agentic AI.
Conclusion
The paper systematically formalizes epistemic miscalibration during planning as an underexplored but significant source of failures in LLM-based multi-agent systems. By introducing EPC-AW, which prioritizes information-consistent plan selection and persistent calibration via constraint memory, the authors demonstrate robust gains in task success and system reliability. These findings establish epistemic calibration as a central concern in the future refinement and deployment of agentic AI workflows, with practical solutions now available for mitigation at the planning stage (2605.23414).