SOLVE-Med: Multi-Agent Medical QA

Updated 5 July 2026

SOLVE-Med is a multi-agent medical QA system that integrates a Router Agent, ten specialized 1B-parameter models, and an Orchestrator for expert response synthesis.
It features dynamic specialist selection and modular design to mitigate hallucinations, bias, and high computational demands in healthcare AI.
Empirical results show improved ROUGE-1 and BERTScore F1 performance over larger standalone models, supporting efficient, privacy-preserving local deployment.

Searching arXiv for SOLVE-Med and closely related medical QA multi-agent systems. SOLVE-Med, short for “Specialized Orchestration for Leading Vertical Experts across Medical Specialties,” is a multi-agent medical question-answering system designed to address deployment problems in healthcare AI, including hallucinations, bias, computational demands, privacy concerns, and the need for specialized expertise across diverse domains. The system combines a Router Agent for dynamic specialist selection, ten specialized small LLMs with 1B parameters each fine-tuned on specific medical domains, and an Orchestrator Agent that synthesizes responses. On Italian medical forum data across ten specialties, the reported best configuration, SOLVE-Med (Top-3), achieves ROUGE-1 of 0.3010 and BERTScore F1 of 0.6974, outperforming standalone models up to 14B parameters while enabling local deployment (Marino et al., 5 Nov 2025).

1. Problem framing and conceptual rationale

SOLVE-Med is motivated by a specific tension in medical question answering: a single generalist model may offer broad coverage, but medical deployment requires specialized expertise, reliability, efficiency, and privacy-preserving operation. The system is explicitly presented as a response to hallucinations, bias, high compute cost, and privacy risk in medical QA, with the core design choice being a modular architecture rather than a monolithic model (Marino et al., 5 Nov 2025).

Its conceptual analogy is consultative. A medical query is first routed to the relevant “specialists,” then their outputs are merged into a single response. This suggests a computational analogue of referral and synthesis rather than end-to-end generation by one model. A plausible implication is that SOLVE-Med treats specialization as a first-class systems property rather than a latent capability expected to emerge from scale alone.

The reliability concerns targeted by SOLVE-Med are consonant with broader patterns in medical AI. Work on medical vision-LLMs emphasizes modality misalignment and clinically consequential hallucinations (Zhu et al., 2024), while work on medical reasoning emphasizes the importance of reflection quality and self-correction in high-stakes settings (Yang et al., 11 Jun 2025). Within that broader landscape, SOLVE-Med addresses reliability at the architectural level through specialist selection and answer orchestration rather than through preference optimization or reflection training alone.

2. Multi-agent architecture and inference pipeline

The SOLVE-Med pipeline consists of four stages. First, the system receives the user query together with the conversation state; in the demo, it supports multi-turn chat, and the query is contextually reformulated before routing. Second, a Router Agent performs multi-label prediction over medical specialties. Third, the selected specialist models each receive the query. Fourth, the Orchestrator Agent receives the original question and the specialist outputs and produces one integrated final answer (Marino et al., 5 Nov 2025).

The Router Agent is implemented by fine-tuning DistilBERT-base-multilingual-cased as a multi-label classifier. It outputs label scores for the ten specialties, and specialist selection is performed either by threshold-based selection or by top- $n$ selection. The system design intentionally prioritizes recall in routing, on the assumption that the Orchestrator can later remove irrelevant material (Marino et al., 5 Nov 2025).

Each specialist is a quantized LLaMA-3.2-1B-Instruct model fine-tuned separately for one specialty. The Orchestrator Agent is a quantized Gemma-2-9B-IT model. Its prompt frames it as a professional medical assistant that merges expert outputs into a unified, evidence-based answer. Functionally, the Orchestrator is meant to resolve overlap, remove irrelevant content, reduce incompleteness from any one specialist, and improve coherence and medical soundness (Marino et al., 5 Nov 2025).

This division of labor is central to the system’s claims. The Router distributes queries across domain-specific models; the specialists provide focused responses within their vertical domains; the Orchestrator performs late fusion at the text level. The architecture therefore separates expert selection from expert generation and separates both from answer synthesis.

3. Specialty coverage and component construction

SOLVE-Med consolidates the original forum categories into ten macro-categories:

Cardiology and Hematology
Dermatology and Aesthetics
Gastroenterology
Gynecology
General Medicine and Surgery
Neurology
Eye, ENT and Pulmonology
Orthopedics
Mental Health
Urology and Andrology (Marino et al., 5 Nov 2025)

These categories were derived from Italian medical forum data and are intended to cover broad, high-frequency medical domains. The source corpus comes from Medicitalia and Dica33 and contains approximately 700,000 question–answer pairs annotated with one of 102 platform-defined categories, which were manually consolidated into the ten macro-categories (Marino et al., 5 Nov 2025).

The specialist models are trained independently. For each specialist SLM, the reported split is 10,000 training, 500 validation, and 500 test examples. The sampling procedure computes text embeddings for all QA pairs, applies UMAP for dimensionality reduction, then HDBSCAN clustering, and then samples representative QA pairs within each macro-category. For the Router Agent, the reported split is 5,000 training, 1,000 validation, and 1,000 test examples, evenly distributed across the ten macro-categories. The paper states that disjoint training subsets were built for the Router and the specialists in order to prevent leakage between components (Marino et al., 5 Nov 2025).

Fine-tuning and inference are performed with Unsloth, using PEFT with LoRA, and both specialists and Orchestrator are quantized. These choices support the paper’s emphasis on efficient local deployment. A plausible implication is that the system’s practical claims depend not only on the macro-architecture but also on the use of low-overhead fine-tuning and quantized inference.

4. Routing strategies, evaluation protocol, and metrics

The Router Agent uses two selection strategies. In threshold-based selection, specialists are chosen if their predicted label score exceeds a threshold; thresholds were tuned to maximize the $F_\beta$ score with $\beta = 2$ and $\beta = 3$ , yielding thresholds of 0.15 for $F_2$ and 0.10 for $F_3$ . In top- $n$ selection, the system directly selects the Top-2 or Top-3 scoring labels. The paper reports that increasing $\beta$ emphasizes recall, which is desirable because the Orchestrator can later filter irrelevant details (Marino et al., 5 Nov 2025).

For final system evaluation, the reported test set contains 100 question–answer pairs, with 10 per specialty, manually selected for clarity, relevance, and diversity from a part of the forum not used in training. Text-generation evaluation uses ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-Lsum, BLEU, METEOR, and BERTScore Precision, Recall, and F1. Router evaluation uses Precision, Recall, and the number of specialists selected per query (Marino et al., 5 Nov 2025).

The routing metrics quantify the precision–recall trade-off directly. Top-2 yields Precision 0.4020, Recall 0.8040, and 2.000 specialists per query; Top-3 yields Precision 0.2960, Recall 0.8870, and 3.000 specialists per query; the $F_2$ threshold yields Precision 0.5699, Recall 0.8290, and 1.528 specialists; the $F_3$ threshold yields Precision 0.4157, Recall 0.8900, and 2.277 specialists. The authors prefer recall in routing because the Orchestrator can filter later (Marino et al., 5 Nov 2025).

This evaluation design makes the system’s operating principle explicit: routing is allowed to over-select if downstream orchestration can recover specificity. That differs from a pipeline in which expert selection itself must be highly precise.

5. Reported performance and comparative position

The paper reports that SOLVE-Med outperforms three larger standalone baselines: Llama-3.1-8B-Instruct, Gemma-2-9B-IT, and Velvet-14B. Among SOLVE-Med variants, Top-3 achieves the best overall reported results (Marino et al., 5 Nov 2025).

System	ROUGE-1	BERTScore F1
SOLVE-Med (Top-3)	0.3010	0.6974
SOLVE-Med (Top-2)	0.2974	0.6953
SOLVE-Med ( $F_\beta$ 0)	0.2975	0.6939
SOLVE-Med ( $F_\beta$ 1)	0.2920	0.6928
Llama-3.1-8B-Instruct	0.2183	0.6657
Gemma-2-9B-IT	0.2546	0.6641
Velvet-14B	0.2501	0.6775

The pattern across routing variants is also informative. Selecting more specialists tends to improve performance, and Top-3 produces both the highest recall among fixed- $F_\beta$ 2 strategies and the strongest end-task metrics. This supports the paper’s interpretation that broader expert coverage yields better final answers, even when routing precision decreases (Marino et al., 5 Nov 2025).

The reported gains are not restricted to lexical overlap. The best BERTScore F1, 0.6974, exceeds the 0.6775 reported for Velvet-14B. On the paper’s reading, this indicates that specialization and orchestration improve semantic quality as well as overlap-based metrics. A plausible implication is that domain decomposition can offset, and in this setting surpass, monolithic model scale.

The article’s broader significance lies in how it situates SOLVE-Med relative to medical AI failure modes. Hallucination and factuality problems are also emphasized in clinically aware multimodal preference optimization (Zhu et al., 2024), and the importance of reasoning quality is emphasized in self-corrected reflection for medical QA (Yang et al., 11 Jun 2025). SOLVE-Med addresses the same reliability agenda through model specialization, routing, and synthesis. In that sense, it belongs to a family of methods that treat medical AI as a structured systems problem rather than as a pure scaling problem.

6. Deployment claims, limitations, and significance

SOLVE-Med makes four deployment-oriented claims. First, domain-specialized experts are argued to reduce hallucination risk compared with one generalist model. Second, the architecture is argued to reduce some sources of bias through specialty-focused models, an explicit routing layer, and local deployment. Third, compute demands are reduced because the specialists are 1B models, the Router is DistilBERT, and the Orchestrator is a quantized 9B model, with LoRA and Unsloth used to reduce training and inference cost. Fourth, local execution reduces exposure of sensitive medical questions and is therefore positioned as advantageous for privacy in healthcare settings (Marino et al., 5 Nov 2025).

The paper is also explicit about the current limitations. The final evaluation set is small, with only 100 manually curated test QA pairs. There is no human evaluation. Context handling could be improved. The architecture has not been validated with clinical users or real-world workflows. It is presented as a support tool and not a replacement for medical judgment (Marino et al., 5 Nov 2025).

These limitations are important for interpreting the results. The evidence for reductions in hallucinations and bias is primarily architectural and indirect rather than based on dedicated hallucination benchmarks or direct bias audits. Likewise, the privacy advantage is a deployment property of local execution rather than an empirical privacy measurement. This suggests that SOLVE-Med should be understood as a promising systems architecture with encouraging benchmark performance, rather than as a clinically validated assistant.

In longer historical perspective, SOLVE-Med extends a recurring idea in medical informatics: decision support may benefit from modular structures that combine different forms of expertise while preserving physician oversight. Earlier work on physician-oriented medical information systems proposed a modular combination of a diagnostic expert system and a treatment recommendation engine sharing a patient database (0810.1991). SOLVE-Med differs in technical realization, relying on routed specialist LLMs and an orchestration layer, but it continues the broader line of designing medical AI as coordinated expert support rather than autonomous substitution.

Taken together, SOLVE-Med is best understood as a locally deployable, specialty-aware medical QA architecture in which performance gains are attributed to specialized orchestration rather than model scale alone. Its main contributions are the Router Agent for multi-label specialist selection, ten domain-specific 1B medical SLMs, a quantized Orchestrator Agent for response synthesis, and empirical gains over larger standalone baselines on Italian medical forum question answering (Marino et al., 5 Nov 2025).