- The paper introduces AMID, a multi-agent framework that combines data-conditioned method planning with independent verification to create executable, submission-ready, and auditable medical imaging models.
- AMID produced reviewer-accepted results on all 20 ReX-MLE tasks and outperformed the strongest listed baseline on 19, matching or exceeding human references on several challenges.
- The framework exposes remaining challenges, including high GPU and token costs, limited ablation evidence, short-horizon training failures, and difficult coupled objectives such as topology-aware classification.
AMID (Autonomous Multi-agent framework for medical Imaging model Development) is a multi-agent system that automates end-to-end development of medical imaging models, from raw task data to submission-ready, auditable artifacts. The paper addresses two failure modes that the authors identify when generic machine-learning-engineering (MLE) agents are applied to medical imaging: coarse-grained search spaces that ignore modality-, anatomy-, and protocol-specific constraints, and unreliable experimental feedback under expensive GPU-bound training where shortcut experiments, leakage, metric mismatches, or malformed prediction artifacts can masquerade as progress. AMID's response is a combination of Data-Conditioned Method Planning (DCMP) and Verification-Guided Two-Stage Optimization, evaluated on 20 tasks from the ReX-MLE benchmark (Kenia et al., 19 Dec 2025).
Problem framing: accountable engineering, not one-shot code generation
The paper formalizes autonomous medical imaging model development around a "task and artifact contract." The input is deliberately minimal — a dataset plus a task definition specifying target output, evaluation metric, and submission protocol. The required output is not a score but a complete model package: executable training and inference code, weights, prediction files, validation scores, submission artifacts, and an audit trail demonstrating that results were produced under the correct data, split, metric, and format. This framing makes verification a first-class requirement rather than an afterthought: a high local score is admissible only if the metric direction is correct, the split is legal, the prediction schema matches the evaluator, and model lineage is traceable.
The authors argue that generic MLE agents fail here for structural reasons. A nominal task type (e.g., segmentation) can demand radically different pipelines depending on voxel spacing, annotation density, patient grouping, and output contract; conversely, literature-derived methods are often non-executable because code, checkpoints, or data interfaces are unavailable. Additionally, feedback loops are slow and costly relative to CPU-oriented benchmarks, biasing agents toward incomplete training and weak validation. These observations motivate both components of the framework.
Data-conditioned method planning
DCMP converts a task into a portfolio of executable method lanes through two stages. First, task and data profiling produces an evidence-backed inventory of file organization, image geometry (shape, spacing, anisotropy, affine consistency), supervision statistics (class balance, foreground ratio, box counts), and evaluation constraints (metric direction, fold/group structure, leakage risks, submission format). These diagnostics act as search predicates — sparse 3D vessel masks imply patch sampling and topology-aware post-processing; histopathology implies tiling and slide-level aggregation.
Second, resource-grounded method search grounds candidate methods in runnable resources rather than open-ended brainstorming. Two sources are used: a pre-seeded library of toolkits such as nnU-Net [isensee2021nnu] and MONAI [cardoso2022monai], treated as optional implementation anchors, and a curated registry of medical imaging foundation models (MedSAM/MedSAM2, TotalSegmentator, VISTA3D, pathology encoders) indexed by access status, license, and downloadable artifacts. The planner checks each candidate against the profiled data before assigning it to a lane. The resulting hybrid method graph yields a portfolio — typically including a conservative baseline lane, a foundation-model adaptation lane, a preprocessing/post-processing-heavy lane, an ensemble lane, and calibration-only lanes that verify data loading and submission mechanics. Lower-confidence or paper-only ideas are downranked or retained as fallbacks rather than silently discarded. Each lane carries its hypothesis, resource requirements, validation obligations, budget estimate, and likely failure modes, so worker agents start from executable routes.
Verification-guided two-stage optimization
The optimization layer moves from broad exploration to selective exploitation, with an independent reviewer gate active throughout. In Stage 1 (behavior-gated exploration), workers distributed across method families submit attempts tied to lanes; each attempt records committed code, artifacts, scores, and lane attribution. Workers cannot certify their own evidence: the reviewer evaluates frozen commits outside the worker session, checking validation splits, metric computation, artifact completeness, payload uniqueness, and code-to-artifact traceability. Only reviewer-accepted attempts count as valid lane coverage; failed attempts remain useful for diagnosis but do not drive promotion. Once lanes plateau, the manager promotes the strongest candidates, optionally retaining a challenger.
In Stage 2 (selective exploitation), optimizer workers improve leading recipes via architecture, inference, test-time augmentation, post-processing, or ensembling changes; challenger workers preserve capacity on alternative lanes; repair workers fix candidates with good raw scores but failing artifact checks; finalizers prepare deliverables. The same gate applies: correct metric direction, locked-fold usage, complete fold outputs, valid prediction schemas, and provenance back to accepted attempts. The final package is selected only from high-performing, reviewer-verified candidates.
Execution rests on a self-organizing agent loop: workers run in isolated git worktrees against shared file-system memory (attempt ledgers, notes, skills, reviewer reports), while a lifecycle manager controls worker creation, interruption, reassignment, and retirement. Heartbeat prompts force reflection, consolidation, and pivot decisions tied to medical-imaging obligations. The substrate is backend-agnostic — Codex, Claude Code, OpenCode, Cursor Agent, and Kiro can participate in mixed-backend runs over the same shared state.
Benchmark results
Evaluation uses ReX-MLE's 20 medical imaging challenges spanning segmentation, detection, classification, image-quality assessment, and enhancement across X-ray, CT, MRI, CTA, MRA, histopathology, ultrasound, and microscopy. Main runs use Codex with GPT-5.5, a 24-hour wall-clock budget, and a single RTX A6000 GPU per challenge. Compared against AIDE (Jiang et al., 18 Feb 2025), ML-Master (Liu et al., 19 Jun 2025), and R&D-Agent (Yang et al., 20 May 2025):
| Challenge |
Metric |
Best baseline |
AMID |
Δ |
Human ref. |
| SEG.A |
Dice |
0.02 |
0.91 |
+0.89 |
0.92 |
| ISLES'22 |
Dice |
0.04 |
0.71 |
+0.67 |
0.79 |
| PUMA-T1-TissueSeg |
Dice |
FAIL/0.00 |
0.56 |
+0.56 |
0.78 |
| PUMA-T1-NucleiDet |
F1 |
0.08 |
0.54 |
+0.46 |
0.66 |
| DENTEX |
AP |
0.09 |
0.49 |
+0.40 |
0.40 |
| NeurIPS-CellSeg |
F1 |
0.36 |
0.90 |
+0.54 |
0.88 |
| LDCT-IQA |
Score |
2.66 |
2.74 |
+0.08 |
2.74 |
AMID produced valid accepted results on all 20 tasks and improved over the strongest listed baseline on 19, tying on TopCoW-CTA-Cls. On DENTEX, NeurIPS-CellSeg, PUMA-T2-Det, and LDCT-IQA it reaches or exceeds the original-competition human reference; the authors note this reference is approximate, since ReX-MLE re-splits public training data and agents train with less data than original competitors. A controlled comparison holding GPT-5.5 fixed across baselines shows the gap persists (e.g., 0.49 vs. 0.08 AP on DENTEX; 0.64 vs. 0.52 Dice on PUMA-T1-Seg), supporting the claim that gains derive from search organization and verification rather than backend strength alone. Runtime-backend swaps within AMID show mixed per-task patterns (Codex best on DENTEX; Claude Code best on PUMA-T1-Seg and TopCoW-MRA-Cls), indicating runtime still shapes individual trajectories.
Case studies attribute success to task-matched decomposition. On DENTEX, a YOLOv8 detector is combined with an anatomical calibration layer mapping boxes to FDI tooth numbering, reaching 0.48825 mean AP. On PUMA tissue segmentation, frozen UNI patch features with lightweight MLP heads and fine-tuned Phikon-v2 blocks convert previously zero-scoring tasks into Dice ≈ 0.56; on nuclei detection, CellViT-256 and PathoSAM proposals with classical classifiers yield macro-F1 of 0.544 and 0.284, the latter ranking first in its official track despite low absolute F1 from rare classes.
Failure modes
Two failures are analyzed explicitly. On USenhance, a five-fold pix2pix-style U-Net ensemble produced valid outputs but low quality (LNCC 0.187); stronger CycleGAN candidates were rejected because unstable adversarial losses gave weak short-horizon evidence under the fixed budget — a case where the verification-driven selection may have prematurely discarded a direction needing longer training. On TopCoW classification, graph-edge labels depend on intermediate small-vessel segmentation and topology extraction; the CTA run collapsed to a fixed topology prior and MRA reached only 0.463 anterior accuracy, exposing a coupled segmentation–topology–classification bottleneck that isolated optimization cannot resolve within budget.
Limitations and open questions
The authors state plainly that cost is the dominant limitation: long-horizon medical MLE consumes substantial GPU time and agent tokens, and full-suite results use only the primary backend, with alternative backends examined on three representative tasks only. Broader ablations over method-lane planning, verification gates, compute budgets, and reviewer validity analyses remain undone, so the contribution of the search design versus the underlying agent runtime is not fully separated. Performance also depends on the quality of the resource registry, data profiling accuracy, and the backend's implementation ability. Challenge performance is explicitly framed as system-level evidence, not clinical validation — it does not establish safety, prospective generalization, or calibration under deployment shift. Open questions include whether cost-aware mechanisms (cheaper early filters, surrogate validation, selective escalation) can preserve verified performance at lower budgets, and whether coupled multi-stage objectives like TopCoW classification can be optimized jointly rather than sequentially.
Conclusion
AMID reframes autonomous medical imaging model development as a data-conditioned, verification-controlled search process, coupling domain-grounded planning of executable method lanes with independent review of every piece of evidence that drives promotion or selection. Across 20 ReX-MLE challenges it outperforms general-purpose MLE agents nearly uniformly and matches expert-level references on several tasks, while its failure analyses delineate where long-horizon budgets and coupled objectives remain difficult. As a preliminary technical report, it establishes the framework and initial evidence, leaving systematic ablations, cost-aware autonomy, and broader deployment-oriented validation as the principal outstanding work.