NeuroAgent: LLM Agents for Multimodal Neuroimaging Analysis and Research

Abstract: Multimodal neuroimaging analysis often involves complex, modality-specific preprocessing workflows that require careful configuration, quality control, and coordination across heterogeneous toolchains. Beyond preprocessing, downstream statistical analysis and disease classification commonly require task-specific code, evaluation protocols, and data-format conventions, creating additional barriers between raw acquisitions and reproducible scientific analysis. We present NeuroAgent, an LLM-driven agentic framework that automates key preprocessing and analysis steps for heterogeneous neuroimaging data, including sMRI, fMRI, dMRI, and PET, and supports interactive downstream analysis through natural-language queries. NeuroAgent employs a hierarchical multi-agent architecture with a feedback-driven Generate-Execute-Validate engine: agents autonomously generate executable preprocessing code, detect and recover from runtime errors, and validate output integrity. We evaluate the system on 1,470 subjects pooled across all ADNI phases (CN=1,000, AD=470), where all subjects have sMRI and tabular data, with subsets also having Tau-PET (n=469), fMRI (n=278), and DTI ($n=620$). Pipeline ablation studies across multiple LLM backends show that capable models reach up to 100% intent-parsing accuracy, with the strongest backend (Qwen3.5-27B) reaching 84.8% end-to-end preprocessing step correctness. Automated recovery limits manual intervention to edge cases where human review is required via the Human-In-The-Loop interface. For Alzheimer's Disease classification using automatically preprocessed multimodal data, our agent ensemble achieves an AUC of 0.9518 with four modalities, outperforming all single-modality baselines. These results show that NeuroAgent can reduce the manual effort required for neuroimaging preprocessing and enable end-to-end automated analysis pipelines for neuroimaging research.

• The paper introduces NeuroAgent, a framework leveraging LLM-based agents to automate complex multimodal neuroimaging pipelines with minimal manual intervention.
• It employs a hierarchical planning module and modality-specific agents in a generate-execute-validate loop to improve preprocessing and diagnostic classification accuracy.
• The system reduces expert manual effort while matching conventional methods on ADNI-scale data through robust error recovery and human-in-the-loop support.

NeuroAgent: Hierarchical LLM Agents for Autonomous Multimodal Neuroimaging Research

Introduction

NeuroAgent addresses a principal operational barrier in neuroscience: the automation of complex, heterogeneous neuroimaging pipelines spanning structural MRI (sMRI), diffusion MRI (dMRI), functional MRI (fMRI), and positron emission tomography (PET), as well as downstream group analyses and classification. The framework leverages LLM-based agentic control to map high-level scientific directives to concrete, context-sensitive workflows, subsuming both preprocessing and analytic stages with minimal manual intervention. The system is explicitly designed to bridge the gap between rigid, static orchestration tools (e.g., Nipype, fMRIPrep) and the highly variable demands of real clinical analysis, supporting Human-in-the-Loop (HITL) supervision and self-reflective error recovery.

Figure 1: The NeuroAgent architecture integrates a Central Orchestrator for planning, specialized Modality Agents for distributed execution, and a feedback-driven Generate-Execute-Validate engine with HITL oversight.

Methodology and Agent Architecture

NeuroAgent’s control hierarchy features a semantic Planning Module, a pool of specialized modality agents, and a feedback-driven execution engine. The Planning Module performs non-trivial intent parsing on natural language research queries, infers implicit modality dependencies, and dispatches tasks into a directed acyclic graph (DAG) respecting both explicit and latent constraints (e.g., enforcing sMRI preprocessing as a prerequisite for fMRI/dMRI).

Each Modality Agent is mapped to a validated, toolchain-wrapped pipeline (e.g., FreeSurfer for sMRI cortical modeling, FSL/MRtrix/DIPY for dMRI, Elastix for PET motion correction) and is capable of dynamic parameter tuning based on local data characteristics. Importantly, the Generate-Execute-Validate loop enables each agent to autonomously generate Python scripts for tool invocation, recover from failures by parsing error logs and modifying parameters, and validate output file structures against strict schemas such as BIDS compliance.

The HITL interface allows users to intervene at escalation points, supervise pipeline logic adaptation, and interactively refine both preprocessing and downstream analysis, ensuring safety and robustness in clinical and scientific settings.

Pipeline Ablation: LLM Benchmarking and Reliability

A structured evaluation across multiple LLM backends demonstrates that pipeline reliability is closely tied to model scale and family, but exhibits significant non-monotonicity with regard to parameter count. Notably, the Qwen3.5-4B and Qwen3-4B models achieve 100% intent parsing accuracy, surpassing much larger models in their family, underscoring that intermediate-sized open source LLMs—when properly prompted—can match or exceed larger backends on controlled neuroimaging domain tasks.

The strongest backend (Qwen3.5-27B) yields an 84.8% "all-pass" preprocessing step correctness rate, while the main failure mode is step-constraint adherence rather than low-level code or API errors. Data integration (assembling subject-level manifests across arbitrarily missing modalities) remains the bottleneck, with the highest performing models achieving only 50% all-pass rates in this critical stage.

Clinical Analysis: Automated Group-Level Neurodegeneration Analytics

Following the successful execution of preprocessing, NeuroAgent supports fully automated, natural-language-driven group analytics on ADNI-scale data. The system autonomously matches subjects between visits, performs age- and sex-adjusted regressions, and surfaces clinically salient neurodegeneration markers.

For cortical thickness, the agent identified canonical AD trajectories: a stepwise decrease in bilateral entorhinal cortex from CN through MCI to AD, with the most negative age slopes in the MCI and AD groups.

Figure 2: Age-related cortical thinning in left and right entorhinal cortex, accentuated in MCI and AD compared to CN, replicates established disease progression trajectories.

Figure 3: Stepwise diagnosis-dependent entorhinal thickness stratification, highlighting clear group separation with bilateral cortex as significant biomarkers.

For dMRI, the agent executes robust regression and group comparison of FA/MD in the inferior parietal cortex, highlighting both the expected monotonic decline in FA and increase in MD across groups, and a ceiling effect on age dependence in advanced AD.

Figure 4: In inferior parietal cortex, FA declines with age in CN and AD; MD increases strongly with age in CN and MCI but not in advanced AD.

Figure 5: MD group comparison demonstrates a clear monotonic increase from CN to MCI to AD, consistent with progressive white matter degeneration.

Downstream Classification and Multimodal Fusion

The agent ensemble, leveraging MLP stacking over per-modality model predictions, achieves superior binary AD-vs-CN classification performance. The four-modality fusion model (sMRI, Tau-PET, fMRI, tabular) obtains an AUC of 0.9518 and MCC of 0.7389, outperforming all unimodal and two-modality baselines. These results establish that LLM-orchestrated, automated pipelines preserve crucial diagnostic information for downstream ML modeling, and that the fused multimodal regime confers significant performance advantages over single-modality analysis.

Crucially, with no manual pipeline tuning, NeuroAgent’s ensemble surpasses conventional CNN backbones on both sMRI and Tau-PET cohorts, and it achieves robust generalization despite the highly heterogeneous and partially incomplete real-world dataset structure.

Practical Implications and Theoretical Outlook

Practically, NeuroAgent quantifiably reduces expert manual effort in pipeline construction, code debugging, and output structuring, democratizing large-scale, multimodal neuroimaging analytics. The system’s adaptive, feedback-driven architecture—combined with HITL escalation—positions it for deployment in both research and clinical workflow environments, especially given its strong performance on standardized ADNI datasets.

On the theoretical front, the demonstrated reliability ceilings and failure modality patterns highlight specific bottlenecks in current LLM-agent architectures, particularly in cross-modality data integration and step-constraint enforcement. Integration of domain-prior guided reasoning, expanded expert-discriminative benchmarks, and direct comparisons to gold-standard manual pipelines will be necessary to further close the automation gap. The systematic ablation framework established here provides a rigorous methodology for future evaluations.

The pipeline’s explicit separation of planning, execution, and validation lays groundwork for more interpretable, auditable, and extendable agent-based scientific computation in medical AI. With parallel developments in autonomous scientific agent benchmarking (e.g., ReX-MLE (Kenia et al., 19 Dec 2025), MedMASLab (Qian et al., 10 Mar 2026)), the NeuroAgent architecture contributes to the standardization and evaluation of LLM-powered agentic workflows in high-stakes biomedical research.

Conclusion

NeuroAgent introduces a highly structured LLM-agent framework that achieves robust, near end-to-end automation of multimodal neuroimaging analysis, excelling in intent parsing, self-correcting error recovery, and supporting interactive statistical workflows. With demonstrated capacity to reproduce canonical disease markers, enable high-fidelity diagnostic classification, and minimize manual overhead, it serves as both an applied tool and a methodological template for next-generation autonomous scientific pipelines. Future extensions will need to address residual failure cases, benchmark against manual scripting standards, and validate generalization in more diverse clinical contexts.

(2605.06584)