The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy

Abstract: The growing ability of LLMs and vision LLMs to jointly interpret and reason over images and text is reshaping medical agents, moving them from task specific predictors toward autonomous systems that perceive, reason, plan, remember, and act in clinical environments. This work departs from the capability first perspective of existing literature and instead begins from clinical deployment, asking what tasks, contamination resistant benchmarks, and interactive training environments are required before medical agents can be trusted in practice. Medical agents are formalized as sequential decision making systems under partial observability, together with a three level autonomy taxonomy spanning assisted, cooperative, and fully autonomous operation. The field is organized along a unified scaling spine consisting of framework scaling, capability scaling, and environment scaling. Within this framework, clinical environment scaling, the integration of tools, data, and clinical gyms, is identified as the most actionable yet underexplored direction for agents operating in PACS, EHR, and FHIR ecosystems. Clinical self evolution, where agents improve through interaction with their environments rather than parameter scaling alone, is further positioned as a key research frontier, drawing insights from self improving agents, agent gyms, and test time compute scaling. Applications across radiology, pathology, ophthalmology, and hospital workflows are examined together with deployment challenges including hallucination, cascading failures, and fairness. By consolidating more than 300 references, with particular emphasis on advances from 2025 to 2026, this work provides a roadmap toward trustworthy, self improving medical imaging systems for real clinical practice.

• The paper presents a systems-level roadmap defining six cognitive modules and a scaling spine to evolve clinical agents from assistant tools to autonomous systems.
• It introduces architectural scaling from single-agent tool use to dynamic, multi-agent orchestration, employing reinforcement learning methods that boost performance by up to 29.94%.
• The study emphasizes real-world integration, continual self-improvement, and the need for rigorous governance to transition safely to fully autonomous clinical workflows.

The Path to Self-Evolving Clinical Systems: A Systems-Level Roadmap for Medical Agents

Introduction: From Passive Predictors to Autonomous Medical Agents

This paper provides an exhaustive systems-level synthesis on the development, scaling, and deployment of medical agents, with an explicit focus on evolving from passive, foundation-model-based predictors toward autonomous, self-improving clinical systems. The work reformulates medical agency not as a collection of loosely connected model gadgets but as a rigorous sequential decision process under partial observability, characterized by six core cognitive modules—perception, reasoning, planning, memory, tool use, and reflection—operating within rich clinical environments and interacting with both tool and human ecosystems. The survey departs from model-centric reviews by foregrounding the clinical deployment context, insisting that readiness is defined by real-world interaction, bench-to-bedside integration, and continual improvement via clinical feedback, rather than by static capability metrics.

The Scaling Spine: Framework, Capability, and Environment Axes

A central conceptual contribution is the introduction of the "scaling spine," a tripartite organizational axis:

• Framework scaling focuses on architectural growth: from a single agent with external tools, through multi-agent, role-based teams, to learnable, dynamic orchestration topologies governing inter-agent and tool-agent collaboration.
• Capability scaling addresses the deepening of the cognitive reasoning/planning loop, migrating from single-pass perception to multi-turn, harness-driven cycles that integrate long-term memory and tool use.
• Environment scaling is posed as the most immediately actionable lever, arguing that the richness of clinical tools, data, and interfaces (e.g., PACS, EHR, FHIR systems) can amplify agentic capability more tractably than continued parameter expansion.

Figure 1: The scaling spine elevates a passive foundation model to a self-evolving clinical agent along three orthogonal axes: framework, capability, and environment.

Taxonomy and Levels of Autonomy

Building on established autonomy taxonomies in medicine and robotics, the paper defines a three-level medical agent autonomy scale:

• Level 1 (Assisted): Agents act strictly under clinician control, serving as suggestion engines or autofill tools with no independent workflow execution.
• Level 2 (Cooperative): Proactive, human-in-the-loop agents decompose tasks and invoke tools semi-autonomously but keep the clinician as the ultimate authority.
• Level 3 (Fully Autonomous): Agents are empowered to run end-to-end clinical workflows, escalating only via exception triggers.

This decisively frames much of the field's open space: Level 3 autonomy, with self-evolving agents, is explicitly not a solved capability but an aspirational goal, supported by recent advances in RL-based self-improvement, agentic gyms, and test-time adaptation.

Architectural Scaling: From Tools to Adaptive Orchestration

The architectural dimension is analyzed through three coordination paradigms:

• Single-agent tool-augmented: A solitary controller wraps a curated toolbox, often reason-acting in a ReAct/CoT loop.
• Multi-agent role-based: Specialized agents, mirroring clinical teams, coordinate via explicit protocols, often with persistent per-case or cross-case memory for longitudinal reasoning.
• Orchestration topologies: Adaptive, query-dependent graphs control agent interaction, allowing run-time-optimized communication patterns—moving beyond fixed trees or stars to fully dynamic, learnable agent networks.

Figure 2: Progression from single tool-augmented agents, through statically coordinated multi-role systems, to orchestration topologies with learnable, query-adaptive coordination.

Capability Scaling: The Harness-Driven Cognitive Loop

The cognitive capability stack is organized into six interleaved modules, executed cyclically under harness/runtime control. The harness, and its loop engineering, is identified as a first-order object that governs reliability, error correction, and real-world adaptivity—subverting the idea that model scale alone suffices. This runtime-centric engineering encompasses scaffolding, context management, memory virtualizing, dynamic action spaces (code-generation vs tool-calling), and end-to-end governance modules. The paper also highlights the emergence of self-evolving agents, which acquire, refine, and consolidate procedural “skills” and tool use from cumulative experience rather than one-off episodes.

Training, Adaptation, and Self-Evolution: A Supervision Signal Ladder

The review provides a nuanced taxonomy of agent adaptation regimes:

• Prompting/in-context learning yields rapid, domain-agnostic deployment but lacks task robustness.
• Supervised fine-tuning (SFT) and preference optimization (DPO, RLHF) introduce implicit value alignment but are capped by available labels.
• Reinforcement learning with verifiable rewards (RLVR) leverages clinically checkable signals (e.g., measurement consistency, guideline compliance) for dense, scalable feedback, with proven sample efficiency and substantial gains over SFT in new RL pipelines for clinical report generation.
• Self-play and agentic RL define the most advanced regime, enabling experience generation in clinical simulators (agentic gyms), continual improvement, and in-place adaptation without retraining.

Strong empirical results document that RLVR and GRPO methods can boost diagnostic VQA and multimodal report accuracy by up to 29.94% over SFT baselines, even when using models that are 36x smaller than conventional alternatives in some modalities.

Figure 3: Medical agent training regimes span from prompt-based elicitation through SFT, DPO, RLVR, to self-play, with each higher rung enabling greater autonomy—orthogonally amplified by parameter, inference-time, and environment scaling axes.

Clinical Workflows: Agentic Integration Across Specialties

Case studies across radiology, pathology, ophthalmology, oncology, and cross-modal domains demonstrate the domain transferability and compositionality of the framework. Notably, radiology is shown to have reached multi-agent, workflow-integrated deployments, supported by agent orchestration and self-evolving capabilities, whereas other fields—such as ophthalmology and neurology—are still in early stages of modular agent support. The systems catalog highlights integration from department-specific agents into orchestration layers capable of end-to-end closed-loop hospital workflows.

Figure 4: Medical agents organized from specialty departments, via shared protocol layers, into end-to-end hospital workflows with self-evolving feedback mechanisms.

Risks, Open Challenges, and Governance

Key unresolved challenges are identified at every pipeline layer:

• Perception: Brittleness in 3D/4D medical data tokenization and cross-modal alignment remain unsolved.
• Reasoning: Hallucination is magnified by the agentic loop; visual grounding and evidence auditability are the most promising rebalancing mechanisms. Empirical evidence indicates that generalist models can outperform medical-specialized models in hallucination resistance, which contradicts prevailing assumptions.
• Coordination: Cascade failure across agents, especially in shared-memory and poorly modularized topologies, presents a unique risk absent from traditional models.
• Deployment: Real-time inference costs, agentic reasoning latency, and evaluation bottlenecks are primary gates for adoption in time-sensitive workflows.
• Governance: Fairness, privacy, and liability under self-evolving regimes are unsolved; auditability, role-based memory, and protocol standardization are required for clinical trust and regulatory approval.

Implications, Theoretical Significance, and Future Work

The paper’s central thesis is that clinical-readiness and agent capability will be most effectively advanced by environment scaling—connecting agents with rich, modular, verifiable clinical tool ecosystems and automated gymnasia for continual improvement—rather than by parameter scaling alone. The resulting agentic systems are argued to be more adaptive, safer, and aligned than monolithic models, provided rigorous evaluation and regulatory protocols mature correspondingly.

Long-term, the field is expected to converge towards embodied, self-evolving agents operating across federated sites and seamlessly integrating longitudinal, multi-modal data streams. The highlighted roadmap predicts three horizons: near-term (evaluation and safety), mid-term (native 3D/federated deployments), and long-term (embodied, continually improving agent ecosystems).

Figure 5: The road ahead for medical agents spans immediate safety and evaluation, mid-term native 3D/federated learning, and aspirational embodied, self-evolving agent deployments.

Conclusion

This survey presents a unified, systematic foundation for scaling, deploying, and governing medical agents in realistic clinical contexts (2607.11175). By synthesizing advances in agent architecture, adaptive harness engineering, novel training regimes (notably RLVR and continual self-play), and standardized environment/tool protocol stacks, it defines both the opportunities and remaining research challenges for trustworthy, self-improving clinical AI. The implications stretch beyond imaging to all structured clinical work, with the potential, upon resolution of current governance pitfalls, to significantly reshape diagnostic and workflow automation in healthcare.