ClinSeek-35B-A3B: Open Agentic Clinical LLM

• ClinSeek-35B-A3B is an open-source 35B parameter LLM designed for automated, agentic clinical reasoning that leverages EHR data and multiple medical tools.
• It utilizes supervised fine-tuning on high-quality, tool-augmented reasoning trajectories, improving SQL-based querying for efficient clinical evidence retrieval.
• The model achieves near state-of-the-art performance, reaching 94.4% of its teacher's F1 score on clinical decision-support tasks across diverse EHR benchmarks.

ClinSeek-35B-A3B is an open-source, 35-billion parameter LLM targeted at agentic clinical reasoning through automated evidence seeking with electronic health records (EHRs) and medical tools. It is derived from the Qwen3.5-35B-A3B base and distilled from high-quality multimodal, tool-augmented reasoning trajectories generated by a state-of-the-art teacher model. This approach produces a model that closely matches the performance of leading closed-source systems in clinical decision-support tasks, while maintaining the openness and extensibility of contemporary transformer LLMs (Wu et al., 19 May 2026).

1. Model Backbone and Agentic Modality

ClinSeek-35B-A3B is architecturally grounded in Qwen3.5-35B-A3B, a decoder-only transformer with approximately 35 billion parameters. This model incorporates multi-query multi-head attention, rotary positional embeddings, and GELU activations, following the standard design of the Qwen3.5 family. No structural modifications were introduced: all agentic and multimodal behaviors are mediated by injecting tool-call and tool-response tokens directly into the model’s context, as well as defining a unified tool namespace at inference time. Tool availability encompasses 20 discrete endpoints, including EHR SQL access, browser-based search, and medical image analysis. The model interacts with tools purely through special tokens, with no adapters or new network layers.

2. Training and Distillation Methodology

ClinSeek-35B-A3B is trained via standard supervised fine-tuning (SFT) on agentic reasoning trajectories generated by Claude Opus 4.6, the teacher model. Each training sample is formatted as a sequence alternating between $\langle\text{tool\_call}\rangle$ and $\langle\text{tool\_response}\rangle$ tokens, concluding with $\langle\text{finish}\rangle$, resulting in flat token sequences up to 52,000 in length:

$$\tau = (x,\,(a_{1},o_{1}),\dots,(a_{K},o_{K}),\hat y)$$

After filtering out-lengthy examples (18.3% dropped), the resulting corpus comprised 7,204 training and 147 validation trajectories. The SFT loss function is standard cross-entropy:

$$\mathcal{L}_{\mathrm{SFT}} = -\sum_{t=1}^T \log p_\theta\bigl(y_t \mid x,\,y_{1:t-1}\bigr)$$

All tool-response content, including SQL results, web page excerpts, and classifier outputs, was incorporated verbatim, subject to token limits. No KL-divergence or temperature-scaled objectives were used.

Table 1: ClinSeek-35B-A3B SFT Training Configuration

Attribute	Value
Base model	Qwen3.5-35B-A3B
Teacher	Claude Opus 4.6
Training / validation split	7,204 / 147 trajectories
Sequence length cap	52,000 tokens
Epochs	3
Optimizer	Megatron (CPU offload)
Peak LR / Final LR	$2\times 10^{-5}$ / $2\times 10^{-6}$
Precision	bfloat16
Hardware	8× H200 GPUs, TP2 PP1 EP8

3. Agent Benchmarking and Quantitative Outcomes

Model evaluation centers on AgentEHR-Bench, a text-only assessment suite comprising 500 held-out questions distributed across diagnoses, laboratory events, microbiology events, procedures, and transfers. The metric is per-sample F1, averaged first at the subtask and then the aggregate level.

Table 2: AgentEHR-Bench Performance (F1 %)

Model	Diagnoses	Labs	Micro	Proc.	Trans	Avg.
Claude Opus 4.6 (teacher)	58.5	42.1	27.2	31.1	20.9	36.0
Qwen3.5-35B-A3B (base)	36.6	17.7	16.2	21.9	18.1	22.1
ClinSeek-35B-A3B	55.4	38.5	27.6	31.7	16.7	34.0
Kimi K2.5 (SOTA OSS)	46.9	33.7	18.9	27.9	22.1	29.9
MiniMax M2.5	51.5	29.0	19.0	22.0	17.0	27.7
GLM-4.7	46.4	28.6	16.6	23.7	22.9	27.6

ClinSeek-35B-A3B achieves an average F1 of 34.0, representing a +11.9 point improvement over its base and attaining 94.4% of the closed-source teacher’s F1. Gains are observed in all categories except “Transfers,” where a minor decrease is noted. ClinSeek-35B-A3B outperforms Claude Sonnet 4.6 by 1.3 points.

4. Tool Use and Agentic Behavior

Tool invocation analysis reveals substantial differences induced by agentic SFT. On 500 AgentEHR-Bench samples, the base model issued 33,043 calls (2.0% free-form SQL vs. templated retrieval); following agentic distillation, ClinSeek-35B-A3B issued 31,446 calls, increasing SQL-based querying to 12.5%. This shift indicates that SFT using agent trajectories encourages procedural use of EHRs as programmable databases, as opposed to fixed retrieval patterns. No direct baseline for “passive SFT on curated inputs” was included; the benchmark comparison is between pretrained and SFT student.

5. Inference Workflow and Current Limitations

At inference, ClinSeek-35B-A3B operates with a temperature of 1.0, an 8,192 output token limit, 200 maximum agent rounds, concurrency set to 6, a maximum tool-result length of 100,000 characters, and a maximum image edge of 1,568 pixels. Tool invocations are implemented via special tokens, with session termination defined by emitting the <finish> token or reaching the round limit.

Documented limitations are as follows:

• Multimodal tasks in ClinSeek-Bench remain relatively simple; complex, multi-hop clinical workflows are not yet evaluated.
• Teacher-generated agentic trajectories may include redundant or low-value tool calls, occupying valuable context window space and potentially leading to inefficiencies.
• On decision-oriented subtasks (e.g., prescribing), the model may prioritize collecting broad evidence, sometimes overlooking the singular key signal present in curated-input baselines.

Suggestions for improvement include constructing more challenging, compositional multimodal benchmarks, filtering or optimizing teacher traces (potentially by reinforcement learning), and applying post-SFT reinforcement or preference optimization to promote concise, high-utility agentic behavior.

6. Context, Impact, and Future Research

ClinSeek-35B-A3B demonstrates the feasibility of distilling high-quality agentic reasoning, including tool usage patterns, from top-performing closed-source models into open-source LLMs. This approach yields near state-of-the-art performance for open models in clinical evidence-seeking and decision support, approaching that of established proprietary systems while maintaining transparency and reproducibility. A plausible implication is that SFT on agentic trajectories, as opposed to passive fine-tuning, enables open-source models to acquire procedural skills in database querying, medical knowledge integration, and tool orchestration, thereby narrowing the performance gap with closed solutions in data-rich, multimodal environments (Wu et al., 19 May 2026).