Dr. Sai: Autonomous Multi-Agent HEP Analysis

• Dr. Sai is a large language model-powered multi-agent system that transforms natural language queries into rigorous, autonomous high-energy physics workflows.
• It employs role-specific agents for task decomposition, coding, execution, numerical calculation, and reflective validation to ensure percent-level benchmarking accuracy.
• By integrating with established HEP tools like BOSS, ROOT, and HTCondor, Dr. Sai automates complex analyses, paving the way for future autonomous pipelines across data-intensive sciences.

Dr. Sai is a LLM-powered, multi-agent system for autonomous physics analysis in high-energy physics (HEP), with initial deployment and validation on the BESIII experiment. By translating free-form natural language queries into full, rigorous workflows—including Monte Carlo (MC) simulation, event reconstruction, statistical fitting, and uncertainty propagation—Dr.Sai operationalizes end-to-end physics analyses formerly requiring months of manual effort. It directly integrates with the full HEP computing stack (BOSS, ROOT, HTCondor) and demonstrates benchmarking consistency at the percent level, positioning itself as a blueprint for future autonomous discovery pipelines across HEP and other data-intensive sciences (He et al., 24 Apr 2026).

1. System Architecture and Agent Design

Dr.Sai is structured as a Multi-Agent System (MAS) comprising six role-specialized agents coordinated via the AutoGen “Talker–Reasoner” framework and a lightweight daemon for message routing and cluster job monitoring. The core agents and their responsibilities are:

Agent	Function	Signature Inputs/Outputs
Host	Orchestrates user NLQ ingestion, state management, sub-task delegation	Receives NLQ, dispatches to Planner, maintains global state
Planner	Decomposes goals into ordinal sub-tasks (using fixed templates)	Outputs a 12-step workflow plan
Coder	Fills C++/JSON domain templates (e.g., AlgorithmCard) under HepScript rules; leverages HEP-RAG	Outputs code/JSON to Tester
Tester	Executes code and scripts on BESIII (via HaiDDF, HTCondor); streams back logs and outputs	Reports job status, produces ROOT files
Calculator	Computes physics quantities (e.g., cross sections, branching fractions), emits Python for error propagation	Produces final numerical results
Reflector	Performs physics and sanity checks (charge conservation, fit diagnostics)	Flags anomalies, confirms result validity

Inter-agent communication occurs exclusively via strictly typed JSON messages over a persistent socket, each with sender, receiver, task_id, payload, and a terminal “Task–Conclusion” marker. The Host agent maintains a minimal shared “global state” (e.g., file paths, physics constants) to prevent context overload, offloading obsolete messages to disk and ensuring agents receive only context-relevant information.

The typical system state transition is schematically rendered:

$\text{User} \xrightarrow{\text{NLQ}} \text{Host} \xrightarrow{\text{select\_expert}} \begin{cases} \text{Planner} \to \text{Coder} \to \text{Tester} \to \text{Calculator} \to \text{Reflector} \ \qquad \quad\rotatebox{90}{\looparrowright} \text{ (reflection)} \end{cases} \xrightarrow{\text{final answer}} \text{User}$

LLM backends—including Qwen3-max, DeepSeek-v3.2, and GLM-4.7—execute agent prompts, augmented by retrieval-augmented generation (RAG) with HEP-specific document stores.

2. Natural-Language Workflow Decoding

Dr.Sai’s user interface accepts high-level NLQ (Natural Language Queries) such as “Measure the branching fraction of $J/\psi\to\mu^+\mu^-$ using $\psi(2S)\to\pi^+\pi^-J/\psi$ data.” The Host agent performs physics-law checks (charge, energy) via Anchor Word constraints and invokes the Planner agent.

Workflow generation follows a fixed 12-step template, where each template step corresponds to a concrete sub-task, such as algorithm JSON creation or fit summary. For example:

Step	Task Description
1	Create Analysis Algorithm JSON Variable Card for process Psi(2S)→π+π−J/ψ(→μ+μ−)
2	Execute built-in script to generate C++ algorithm under BOSS
...	...
12	Summarize the task

Once the Planner outputs the structured plan, the Coder agent instantiates C++/JSON templates according to HepScript rules and RAG hints (through Qdrant+LlamaIndex), delivering schema-compliant cards or scripts for execution.

A typical algorithm JSON:

$J/\psi\to\mu^+\mu^-$5

3. BESIII and HEP Software Stack Integration

Dr.Sai performs live code execution and data analysis using actual BESIII computing resources. Tester agent submits code and jobcards using:

• bossrun for BOSS C++ algorithm execution,
• root -l script.C for ROOT-based TTree and histogram queries,
• HTCondor for distributed MC and data processing jobs.

Representative shell snippet:

$J/\psi\to\mu^+\mu^-$6

This automated invocation pipeline permits Dr.Sai to directly navigate all data tiers (from MC generation to final fit extraction) without human intervention.

4. Analysis Pipeline Components and Mathematical Workflow

The Dr.Sai system encodes the canonical HEP analysis flow:

a) Monte Carlo simulation:

Uses GEANT4 for detector model, KKMC for ISR, EvtGen for decay modeling, PHOTOS for final state radiation. For the validation task, $N_{\psi(2S)} = 107.7 \times 10^6$.

b) Reconstruction & Event Selection:

Employs BOSS recon algorithms to analyze data, constructing AOD and TTree structures suitable for mass/histogramming queries, e.g.,

$$h(m_{\mu\mu}) = \text{TTree::Draw("M\_inv","...","goff")}$$

c) Statistical Analysis (Signal Extraction):

Performs unbinned/binned fits of invariant mass spectra,

$$f(m) = N_{\mathrm{sig}}\,[ S_{\mathrm{MC}}(m)\otimes G(\mu,\sigma) ] + \sum_{k=0}^n c_k\,T_k(\tilde m)$$

where $T_k$ are Chebyshev polynomials.

d) Branching Fraction Computation:

Final physics observable extraction obeys

$$\mathcal{B} = \frac{N_{\mathrm{sig}}}{N_{\psi(2S)}\,\varepsilon\,\mathcal{B}_{\mathrm{sub}}}, \quad \sigma_{\mathcal{B}} = \mathcal{B} \sqrt{ \left(\frac{\sigma_{N_{\mathrm{sig}}}}{N_{\mathrm{sig}}}\right)^2 + \left(\frac{\sigma_{N_{\psi}}}{N_{\psi}}\right)^2 + \left(\frac{\sigma_{\varepsilon}}{\varepsilon}\right)^2 + \left(\frac{\sigma_{\mathcal{B}_{\mathrm{sub}}}}{\mathcal{B}_{\mathrm{sub}}}\right)^2 }$$

with $\varepsilon$ estimated from signal MC and $\mathcal{B}_{\mathrm{sub}}$ and its uncertainty taken from PDG values.

5. Validation and Model Performance

Dr.Sai underwent validation by reproducing branching-fraction measurements for ten $J/\psi\to\mu^+\mu^-$0 decay modes, comparing autonomous outputs ($J/\psi\to\mu^+\mu^-$1) to reference inputs ($J/\psi\to\mu^+\mu^-$2). Performance is quantified via

$J/\psi\to\mu^+\mu^-$3

The distribution of $J/\psi\to\mu^+\mu^-$4 is tightly clustered around zero; pull values follow a unit-Gaussian. This demonstrates absence of systematic bias and faithful end-to-end pipeline implementation.

A comparative multi-model study (Qwen3-max, DeepSeek-v3.2, GLM-4.7, GPT-4o, GPT-4 Turbo) benchmarked agent success rates, error modes (Tool-Call Mismatch, Structural Inconsistency), and token consumption as retry counts varied from 1 to 9. Qwen3-max, DeepSeek-v3.2, and GLM-4.7 exhibited near-100% completion rates; GPT-4o was chiefly limited by schema synthesis errors despite correct tool naming.

6. Limitations and Forward Outlook

Principal limitations currently include residual imprecision in “last-mile” tool-calling and failure to always instantiate parameter-compliant schema. Expert heuristics (such as dynamic fit strategies and cut tuning) remain incompletely captured. Planned developments target:

• Construction of a “synthesis layer” or dedicated LLM fine-tuning on HepScript for enhanced schema compliance,
• Expansion of the HepScript ontology (“HepScript-2.0”) to encode expert knowledge and analysis heuristics,
• Domain adaptation (astronomy, genomics) via custom RAG backends, tool bridges (e.g., Astropy/PyCBC), and domain-specific languages such as “AstroScript” and “GeneScript”.

Dr.Sai evidences that a multi-agent LLM-based framework can autonomously map open-ended physics queries to complete, validated analyses within complex software ecosystems, suggesting a pathway for the scalable, automated extraction of physics results from petabyte-scale data (He et al., 24 Apr 2026).