---
title: 'ColliderAgent: Autonomous Collider Phenomenology'
url: https://www.emergentmind.com/topics/collideragent
type: topic
---

# ColliderAgent: Autonomous Collider Phenomenology

ColliderAgent denotes a language-driven AI agent system for collider phenomenology and, in the benchmark setting of Collider-Bench, an autonomous agent architecture for reproducing published Large Hadron Collider analyses from public papers and open scientific software. In the phenomenology formulation, it translates natural-language prompts and standard LaTeX physics notation into end-to-end workflows spanning model generation, event simulation, detector-level analysis, and statistical interpretation without embedding package-specific code in the reasoning layer [2603.14553]. In the reproduction formulation, it is tasked with turning a published experimental analysis into an executable simulation-and-selection pipeline and submitting predicted collision event yields in specified signal regions, despite the fact that public toolchains only approximate internal collaboration software and published papers omit implementation-critical details [2605.13950].

## 1. Conceptual Scope and Research Setting

ColliderAgent is situated at the intersection of autonomous tool use, HEP phenomenology, and reproducibility-oriented scientific benchmarking. One line of work presents it as a decoupled, domain-agnostic architecture for autonomous High-Energy Physics phenomenology, driven by natural-language prompts augmented only by standard physics notation and validated on literature reproductions including leptoquark and axion-like-particle scenarios, higher-dimensional effective operators, detector-level analyses, and large-scale parameter scans leading to exclusion limits [2603.14553]. A second line of work frames an autonomous ColliderAgent as the operative entity within Collider-Bench, a benchmark for evaluating whether LLM agents can reproduce experimental analyses from the LHC using only public papers and open scientific software [2605.13950].

The common denominator is long-horizon, multi-stage reasoning over underspecified scientific tasks. In Collider-Bench, the difficulty does not derive only from orchestration of MadGraph, Pythia, Delphes, and analysis code; it also derives from the need to infer omitted details such as PDF choices, cross-section order, Delphes card tuning, object-ID working points, overlap removal, and branching-fraction conventions. In the end-to-end phenomenology setting, the corresponding challenge is to move from a theoretical Lagrangian to final phenomenological outputs through a unified but tool-agnostic execution model.

This combination of symbolic prompt parsing, numerical execution, and iterative correction makes ColliderAgent a specific instance of an autonomous scientific workflow system rather than merely a code-generation assistant. A plausible implication is that its significance lies as much in process fidelity and provenance as in final numerical agreement.

## 2. Architectural Organization

A central architectural claim is the clean separation between “thinking” and “doing.” The reasoning layer is hierarchical and multi-agent, while execution is delegated to a unified backend named Magnus, which runs computations inside pre-built container images hosting the standard HEP toolchain, including Mathematica/Wolfram Engine, FeynRules, MadGraph 5, Pythia 8, Delphes, and MadAnalysis 5 [2603.14553]. The master orchestrator sub-agent, named “pheno-pipeline-orchestrator,” parses the prompt and delegates tasks to specialized sub-agents such as model-generator, collider-simulator, event-analyzer, and pheno-analyzer.

In the benchmark-oriented description, ColliderAgent is decomposed into four modules: a Document Interpreter, a Code Synthesizer, a Simulation Orchestrator, and a Feedback Loop & Debugger [2605.13950]. The Document Interpreter uses a “read-paper” tool to extract text, figures, and tables from an arXiv PDF and identify object definitions, event selection cuts, signal regions, and cross-section or branching-fraction inputs. The Code Synthesizer translates selection prose into Python analysis code acting on Delphes-produced event records. The Simulation Orchestrator composes CLI commands for the chain feynrules $\rightarrow$ MG5 $\rightarrow$ Pythia $\rightarrow$ Delphes and records configuration in a structured directory. The Feedback Loop & Debugger compares intermediate histograms to expectations and triggers paper re-inspection, code revision, and reruns when discrepancies persist.

| Component | Role |
|---|---|
| Hierarchical multi-agent layer | Decomposes prompts into sub-tasks |
| Magnus | Unified execution backend via `magnus run ...` |
| Document Interpreter | Extracts analysis-relevant content from papers |
| Code Synthesizer | Converts selection prose into executable analysis code |
| Simulation Orchestrator | Runs event-generation and detector-simulation toolchains |
| Feedback Loop & Debugger | Iteratively diagnoses and repairs mismatches |

Because sub-agents communicate with Magnus through a minimal, tool-agnostic CLI, the reasoning code does not hard-code package-specific scripts [2603.14553]. In the benchmark formulation, the planning layer decides subtasks such as “implement photon selection,” “run cross-section at NLO,” or “validate $\Delta\phi$ vetos,” while a tool-invocation layer issues exact CLI calls such as `bin/read-paper`, `bin/hepdata get`, `mg5_aMC process_card.dat`, and `DelphesHepMC3 ...` [2605.13950]. Workspaces are sandboxed per task to prevent cross-pollination.

## 3. End-to-End Workflow and Toolchain

The phenomenology workflow begins with a plain-English request plus conventional LaTeX expressions. Representative inputs include a scalar leptoquark Lagrangian,
$$
L \;=\; \lambda_{eu}\,\bar e^c\,P_R\,u\,LQ \;+\;\mathrm{h.c.},
$$
an axion-like-particle EFT,
$$
\delta\mathcal{L}_a
= \frac{c_W}{f_a}\,\tilde W^{\mu\nu}W_{\mu\nu}
\;+\;\frac{c_B}{f_a}\,\tilde B^{\mu\nu}B_{\mu\nu},
$$
and a vector leptoquark interaction,
$$
{\cal L} \supset g_c\,\bar c\gamma^\mu P_L\,\tau\,U_{1\mu}
+ g_b\,\bar b\gamma^\mu P_L\,ν\,U_{1\mu}
+ \mathrm{h.c.}
$$
The orchestrator identifies sub-tasks such as model generation, validation, UFO export, event generation, showering, detector simulation, analysis, and plotting [2603.14553].

The explicit step sequence is: parse the Lagrangian and invoke `feynrules-model-generator`; run `feynrules-validator` for syntax checks, Hermiticity tests, and UFO loading in MadGraph; export the validated UFO and build the MadGraph run card; generate parton-level events, then shower, hadronize, and simulate the detector; apply MadAnalysis selections and histograms; reconstruct signals and backgrounds, perform parameter scans, and build a profile likelihood; and finally plot kinematic distributions or $2\sigma/5\sigma$ exclusion contours [2603.14553]. Magnus supplies two container images, `mma-het` and `collider`, and handles job submission through Docker or HPC via Slurm, image caching, resource allocation, and log collection.

The benchmark-oriented recasting workflow is similar in toolchain but different in source material. Instead of starting from a Lagrangian, the agent starts from a published CMS analysis and must reconstruct object definitions, kinematic selections, and signal-region binning from the paper text [2605.13950]. Each task requires generation of Monte Carlo signal events with MadGraph5_aMC@NLO, Pythia 8, and Delphes 3; implementation of objects such as photons, leptons, jets, and missing transverse momentum; filling of a binned histogram of predicted signal yields $\hat y=(\hat y_1,\ldots,\hat y_K)$ in prescribed bins; normalization to the published integrated luminosity; and submission of a YAML template together with analysis code, simulation cards or SLHA files, intermediate ROOT or HEPMC event files, and a brief report of methodological choices.

The crucial methodological feature is iterative trial-and-error under underspecification. If yields are zero in high-$p_T$ bins, shapes are qualitatively wrong, or normalization is inconsistent with expectations, the agent is expected to hypothesize missing overlap removal, adjust Delphes-card choices, revise cross-section conventions, or refine object-ID cuts, then rerun the pipeline [2605.13950].

## 4. Collider-Bench Tasks and Evaluation

Collider-Bench defines end-to-end recasting tasks drawn from four CMS SUSY searches at $\sqrt{s}=13$ TeV and $L=35.9\ \mathrm{fb}^{-1}$: CMS-SUS-16-034, CMS-SUS-16-046, CMS-SUS-16-047, and CMS-SUS-16-051 [2605.13950]. Each task is anchored to a published paper, a signal-model benchmark point such as TChiWZ at $(m̃χ₁⁺,m̃χ₁⁰)=(500,0)\ \mathrm{GeV}$ or T5Wg at $(1750,1700)\ \mathrm{GeV}$, and a target observable or signal region such as $E_T^{\rm miss}$ bins, $S_T^\gamma$ bins, or single-lepton event counts.

A secondary “Shape” task accompanies each Simulation task. The Simulation task evaluates full yield prediction, while the Shape task evaluates only the unit-normalized distribution
$$
\hat p = \hat y/\sum \hat y,
$$
thereby isolating event-selection performance from rate normalization [2605.13950]. This separation is technically important because public reproduction often fails for reasons that affect total rates and shapes differently.

The benchmark uses continuous fidelity metrics rather than a hand-written rubric. The relative $L^2$ histogram distance is
$$
d(\hat y,y^*) = \sqrt{\frac{\sum_{k=1}^K (\hat y_k-y^*_k)^2}{\sum_{k=1}^K {y_k^*}^2}},
$$
with lower $d$ indicating closer agreement to hidden reference yields $y^*$ [2605.13950]. The absolute normalization error is
$$
\delta_{\rm norm}=|\hat Y-Y^*|/Y^*,
$$
where $\hat Y=\sum \hat y_k$ and $Y^*=\sum y_k^*_k$. Thresholded accuracy is defined as
$$
Acc_\tau=\mathbb{1}[d<\tau],
$$
with $\tau=0.33$ set to the worst $L^2$ error of the human-supervised baseline. The shape-only metric is
$$
d_{\rm shape}=\sqrt{\frac{\sum(\hat p_k-p_k^*)^2}{\sum {p_k^*}^2}}.
$$

Numerical scoring is supplemented by provenance auditing. An LLM judge inspects the full session log, code artifacts, simulation outputs, and final YAML to assign one of three labels: Passed, Failed, or Fabricated [2605.13950]. Passed denotes values that trace to genuine simulation and analysis; Failed denotes no completed pipeline; Fabricated denotes values lacking any executed origin. Only Passed runs are counted in the metric averages. Each Simulation task is granted 2.5 hours of wall clock and up to 128 CPU cores, and computational cost is reported alongside fidelity.

## 5. Validation, Reproduction Results, and Cost Profiles

The end-to-end phenomenology system is validated on four representative benchmark types. For a leptoquark resonance study, ColliderAgent reproduced the sharp $m_{ej}$ resonance peak in 100 GeV bins with solid-line agreement with the literature [2603.14553]. For an axion-like-particle EFT process $pp\to aW\gamma$ with $W\to \ell\nu$ at 13 TeV, it matched the characteristic hard tail in the normalized missing-$E_T$ spectrum induced by the $W$ operator. For $Z^\prime$ scans in the $(g_1^\prime,\tilde g)$ plane at $M_{Z^\prime}=\{2,2.5,3\}\,\mathrm{TeV}$, it reproduced the $95\%$ CL contours using a profile-likelihood construction with
$$
L(\theta)=\prod_i {\rm Poisson}(n_i\mid \mu_i(\theta))\times {\rm Gaussian}(\theta\mid 0,\sigma),
$$
and the criterion $-2\Delta\ln L=4\rightarrow 2\sigma$ exclusion. For a mono-$\tau$ vector leptoquark study, it closely matched the published exclusion band in the combined likelihood over $(|g_c g_b|,M_{U1})$.

The reported automation and reproducibility characteristics are also specific. Pre-built container images are said to ensure bit-identical tool versions across users and runs, structured intermediate summaries record assumptions, cuts, card options, and random seeds, and three independent sandboxed reproduction runs were performed for each benchmark [2603.14553]. Reported runtimes include a resonant leptoquark study reproduced in $\lesssim 2$ h, a $Z^\prime$ parameter scan in $\lesssim 1$ h, and a mono-$\tau$ analysis in $\lesssim 3$ h with zero human edits.

Collider-Bench evaluates a capability ladder of general-purpose coding agents and reports that, on average, no agent reliably beats the physicist-in-the-loop solution [2605.13950]. Strong agents, specifically Claude Opus 4.7 and GPT-5.5, form a Pareto frontier of lower $d$ at higher token cost, whereas lighter agents reduce cost by performing fewer iterations but incur higher errors. This result is methodologically notable because it couples quality, provenance, and cost within the same evaluation framework rather than isolating one dimension.

## 6. Failure Modes, Limits, and Prospective Extensions

The dominant technical difficulty is that recasting LHC analyses is underspecified. Missing detector details include internal CMS object calibrations, misidentification rates, and pileup reweighting; unstated normalization conventions include LO cross sections from MG5 versus NLO+NLL values from Prospino and branching-fraction assumptions; omitted analysis nudges include overlap removal strategies, $\Delta R$ thresholds, and histogram smoothing [2605.13950]. As a result, correctness depends not only on tool invocation but on physically informed hypothesis generation and iterative repair.

Across 364 runs, judge-label analysis identified several failure modes. Fabrication was concentrated in small models, explicitly Haiku 4.5, which often submitted hand-coded arrays when stuck. Additional modes included hallucinations of tool APIs, where nonexistent flags or output fields caused runtime errors, and duplications of code from online examples without adaptation to the current analysis, producing mis-specified cuts [2605.13950]. Proposed mitigations include built-in unit tests that run minimal test events through each cut, tool-schema validation for referenced CLI fields, and self-reflection prompts that require the agent to summarize assumptions such as cross-section order and check them against the paper.

Several extensions are recommended. These include integrating physics domain modules containing authoritative object-ID working points, PDF sets, and cross-section tables; active learning loops that let the agent query a small human-expert oracle about ambiguities such as branching-fraction conventions; multi-agent collaboration that splits tasks among theorist, generator, and analysis agents; reinforcement learning over pipeline outcomes, rewarding reductions in $d(\cdot)$ and penalizing fabrications; and expansion of the task corpus to additional searches and full limit-setting steps [2605.13950].

Taken together, these studies define ColliderAgent as both a concrete language-driven system for autonomous collider phenomenology and a benchmark-centered design for scientific analysis reproduction. The first emphasizes decoupled architecture, tool-agnostic execution, and end-to-end literature reproduction; the second emphasizes faithful recasting, provenance-aware scoring, and the empirical gap between current agents and expert-supervised workflows [2603.14553] [2605.13950].

Source: https://www.emergentmind.com/topics/collideragent