- The paper introduces Vivaldi, a five-agent clinical pipeline that combines deterministic safety calculations, role-based reasoning, visualization selection, critique, and evidence synthesis for emergency-department physiological time series.
- The evaluation shows agentic orchestration improves non-thinking models’ relevance and justification, while reducing strong thinking models’ performance; tool-based computation raises qSOFA F1 to 100.0 and identifies 30% of ESI Level 1 cases versus none with zero-shot inference.
- The framework improves clinical utility but increases latency 5×–14× and token use 13×–38×, supporting adaptive orchestration that applies agents and tools selectively rather than uniformly.
Vivaldi is a role-structured multi-agent system that generates clinical explanations for multivariate physiological time series in emergency department (ED) settings, and the paper reports a controlled expert evaluation comparing its agentic pipeline against zero-shot LLM inference (2603.04142). The central finding is deliberately counter to prevailing assumptions: agentic orchestration is not a uniform performance multiplier. It substantially improves explanation quality for non-thinking and medically fine-tuned models while often degrading relevance and trust for strong thinking models, and its clearest gains come from externalizing deterministic computation rather than from added reasoning depth.
Motivation and problem framing
The authors identify three underexplored challenges in deploying LLM-based explanation systems for continuous physiological monitoring: structuring explanations to match ED workflows (safety screening, hypothesis formation, critique, evidence synthesis), determining when externally decomposed reasoning actually helps relative to zero-shot inference, and aligning explanations with clinician expectations regarding conciseness and familiar visual conventions. Prior multi-agent clinical systems such as MACD and MedLA evaluate diagnostic correctness but not interactive interpretability or visualization usability; Vivaldi addresses both gaps by deterministically computing safety-critical indices in a sandboxed coder agent and using role-conditioned agents to generate and select visual evidence.
System architecture
Vivaldi instantiates an ED team as five specialized agents coordinated by an orchestrator that maintains a Shared Memory Buffer (SMB) of clinical state:
- TriageAgent computes deterministic safety metrics (Shock Index, MAP, pulse pressure, qSOFA, SIRS, SpO₂ trend features, HR volatility) with local code rather than an LLM, then uses a role-conditioned LLM to produce a structured context summary with personalized thresholds (e.g., adjusted blood-pressure bands for chronic hypertension).
- DoctorAgent, prompted as a senior attending, runs up to three iterative rounds of hypothesis formation over longitudinal vital-sign trends, employing a rank-select-prune strategy that scores each plot 1–10 and retains only the top three.
- ConsultantAgent plays a "curbside consult" role, surfacing blind spots, alternative diagnoses, and inconsistencies, and explicitly requesting additional visualizations.
- CoderAgent writes Python code in a restricted sandbox for each prescribed analysis, returning results, interpretations, figures, and executed code for auditability.
- SynthesizerAgent produces the final assessment in zero-shot mode, instructed to preserve the doctor's decisions unless there is an obvious inconsistency.
The pipeline is built on LangChain primitives with Pydantic-enforced output schemas, temperature 0.2 (except Claude Opus 4.5 at its API-mandated default), and all system prompts released for reproducibility.
Evaluation design
Experiments use the MC-MED dataset (ED visits from an academic medical center), restricted by a static benchmarking index enforcing valid ground-truth labels (integer ESI, non-null LOS, pain score) and minimum data density (30 samples per vital category within a continuous 24-hour window), plus physiological plausibility filters. Five model families are benchmarked under identical prompts: GPT 5.2, Claude 4.5 Opus, Gemini 3 Pro, Llama 4 Maverick, and MedGemma 27B. Six board-certified experts in emergency and internal medicine produced 109 anonymous evaluations across six dimensions: factuality, justification, relevance, trust (ternary scales normalized to 0–100), chart comprehensibility, and clinical utility (5-point Likert). The authors justify the small cohort via early-stage evaluation guidelines (DECIDE-AI) and "informational power," a concession worth noting: the sample supports formative assessment, not large-scale statistical claims.
Explanation quality is model-dependent
Aggregated by model class, the divergence between thinking and non-thinking models is stark:
| Metric |
Thinking: zero-shot |
Thinking: agentic |
Non-thinking: zero-shot |
Non-thinking: agentic |
| Factuality |
88.6 |
83.3 |
87.0 |
91.0 |
| Justification |
93.2 |
83.3 |
88.9 |
96.2 |
| Relevance |
95.5 |
81.0 |
87.0 |
97.4 |
| Trust |
90.9 |
83.3 |
90.7 |
93.6 |
Non-thinking models gain +10.4 points in relevance and +7.3 in justification under agentic execution; thinking models lose −14.5 in relevance, −9.9 in justification, and −7.6 in trust. At the per-model level, Gemini 3 Pro drops 17.2 points in relevance and Claude 4.5 Opus degrades on all four metrics, while GPT 5.2 improves modestly (relevance +7.1) despite being classified as non-thinking here — indicating the effect is not reducible to a single binary "thinking" flag. The authors' interpretation is that models with strong internal reasoning already perform implicit multi-step abstraction, so external structure diffuses attention or encourages over-generation. The implication is direct: agentic scaffolding should be matched to model capability rather than applied uniformly.
Delegating numerical reasoning to executable tools yields near-perfect performance on deterministic metrics — qSOFA F1 rises from 63.6/48.1 to 100.0 for thinking/non-thinking models, and MAP MAE falls to 0.0 — confirming that these gains stem from explicit computation, not improved language modeling. ESI prediction improves from F1 61.0 to 64.6 (thinking) and 40.7 to 65.4 (non-thinking). A confusion-matrix analysis exposes a safety-relevant failure of zero-shot inference: zero-shot models detect none of the ESI Level 1 cases (80% misclassified as Level 2, 20% as Level 3), whereas the agentic framework identifies 30% of them. This is arguably the most consequential result in the paper, since missed high-acuity patients are the costliest error class in triage.
In contrast, subjective targets show no consistent benefit: pain-score MAE worsens for non-thinking models (2.4 → 2.8) and LOS MAE worsens for both classes under agentic inference. The authors attribute this to the poorly observable nature of these outcomes, where over-structured reasoning without additional signals introduces noise.
Visualization trade-offs
Expert ratings show consistent gains in perceived clinical utility across models, but small and inconsistent changes in chart comprehensibility. Claude 4.5 Opus increases utility while reducing visual suitability, conveying important signals through unconventional chart forms; GPT 5.2 and MedGemma improve utility while preserving familiar presentation conventions; Llama 4 Maverick improves neither dimension. The authors conclude that gains in utility depend on adherence to clinical visualization conventions, with medically specialized models achieving the best trade-off.
Computational overhead and failure modes
The agentic pipeline carries a substantial "reliability tax": latency increases 5×–14× and token consumption 13×–38× relative to zero-shot. GPT 5.2 is the worst offender (average 478 s, ~132k tokens), driven largely by the CoderAgent repeatedly generating Unicode mathematical symbols (e.g., ≤) instead of valid Python operators, triggering retry loops. Gemini 3 Pro's bottleneck differs: ~17% of its runtime is spent in the TriageAgent's internal thinking during threshold personalization. These observations make instruction-following robustness a first-order efficiency concern for production agentic systems, independent of reasoning quality.
Limitations and open questions
Several constraints qualify the findings. Regulatory constraints precluded live deployment, so the study is a controlled pilot rather than prospective clinical use; the six-expert cohort provides thematic saturation but limited statistical power; and the classification of GPT 5.2 as non-thinking, which drives much of the aggregate narrative, sits uneasily with its partial gains. Subjective targets (pain, LOS) remain unresolved, and the mechanism by which agentic decomposition disrupts thinking models is hypothesized (attention diffusion, over-generation) but not empirically isolated. The paper leaves open whether dynamic agent-selection policies can capture tool-computation benefits without the narrative-quality costs observed in strong zero-shot models, and whether the ESI Level 1 detection gap generalizes beyond this dataset.
Conclusion
Vivaldi demonstrates that the value of agentic AI in safety-critical healthcare lies in selective externalization of computation and structure, not maximal reasoning complexity. Agentic pipelines reliably fix numerical fragility (near-perfect qSOFA/SI/MAP, improved high-acuity detection) and lift smaller or medically fine-tuned models, but they can degrade relevance and trust in frontier thinking models while imposing 13×–38× token overhead. The actionable principle is adaptive orchestration: invoke roles and tools only where they address concrete model limitations, a design question the authors explicitly defer to future work on dynamic agent selection and prospective evaluation.