Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification

Abstract: Vision-LLMs (VLMs) show promise in drafting radiology reports, yet they frequently suffer from logical inconsistencies, generating diagnostic impressions unsupported by their own perceptual findings or missing logically entailed conclusions. Standard lexical metrics heavily penalize clinical paraphrasing and fail to capture these deductive failures in reference-free settings. Toward guarantees for clinical reasoning, we introduce a neurosymbolic verification framework that deterministically audits the internal consistency of VLM-generated reports. Our pipeline autoformalizes free-text radiographic findings into structured propositional evidence, utilizing an SMT solver (Z3) and a clinical knowledge base to verify whether each diagnostic claim is mathematically entailed, hallucinated, or omitted. Evaluating seven VLMs across five chest X-ray benchmarks, our verifier exposes distinct reasoning failure modes, such as conservative observation and stochastic hallucination, that remain invisible to traditional metrics. On labeled datasets, enforcing solver-backed entailment acts as a rigorous post-hoc guarantee, systematically eliminating unsupported hallucinations to significantly increase diagnostic soundness and precision in generative clinical assistants.

• The paper introduces a neurosymbolic verification pipeline that formalizes and audits deductive consistency in radiology VLM-generated reports.
• It autoformalizes free-text findings into structured evidence and uses an SMT solver (Z3) to validate diagnostic claims against a clinical knowledge base.
• Empirical evaluations across multiple VLMs reveal distinct reasoning profiles and show that symbolic filtering improves soundness and completeness with minimal recall loss.

Formal Verification of Clinical Reasoning in Vision-LLMs

Motivation and Problem Statement

Vision-LLMs (VLMs) have emerged as pivotal tools in radiology, offering automated drafting of clinical reports that synthesize visual and language information. Despite advancements, a persistent challenge remains: the logical inconsistency in generated reports, where diagnostic impressions are frequently unsupported by perceptual findings or where necessary clinical conclusions are omitted. Current evaluation schemes—predominantly lexical or entity overlap metrics—are insufficient to detect such deductive failures and offer no reference-free guarantees regarding the internal consistency of reports. In the medical domain, statistical plausibility is inadequate; reports must be formally valid from a deductive standpoint.

Neurosymbolic Verification Framework

The paper proposes a neurosymbolic verification pipeline designed to deterministically audit the logical soundness of VLM-generated radiology reports. This framework decouples perceptual grounding (handled probabilistically by the VLM) from clinical reasoning (audited via symbolic logic), bridging probabilistic language generation with formal verification mechanisms.

Generated findings and impressions are autoformalized into structured propositional evidence, grounded in a curated ontology and clinical knowledge base. The verification process utilizes an SMT solver (Z3) to adjudicate whether each diagnostic claim in the Impression section is mathematically entailed by the stated findings, hallucinated (unsupported), or omitted (missed logical consequences).

Figure 1: Neurosymbolic verification pipeline mapping findings and impressions to structured evidence, enabling formal consistency checks against a clinical knowledge base via Z3.

Ontological Formalization

The verifier operates under a lightweight clinical ontology $$\mathcal{O} = \langle \mathcal{F}, \mathcal{D}, \mathcal{K} \rangle$$:

• $\mathcal{F}$: Atomic observational predicates extracted from findings.
• $\mathcal{D}$: Diagnostic predicates relevant for the impression.
• $\mathcal{K}$: Clinical knowledge base, implemented as a set of propositional formulas capturing sufficient conditions and consistency constraints between findings and diagnoses.

Autoformalization is performed via schema-constrained LLMs, yielding deterministic mappings from free-text to structured binary state vectors (indicating affirmed or negated findings), under a closed-world assumption.

Deductive Consistency Verification

Verification is framed as a formal SAT problem. For each diagnosis in the impression, the SMT solver tests whether it is a logical consequence of the structured evidence and clinical knowledge base. This results in a taxonomy of reasoning errors:

• Entailed diagnoses: Mathematically supported by findings and knowledge base.
• Hallucinated diagnoses: Stated without logical support from findings.
• Omitted diagnoses: Forced by findings but missing in impression.
• Correctly excluded diagnoses: Neither forced nor claimed.

Reliability follows an assume-guarantee paradigm: guarantees are conditional on accurate autoformalization and correctness of the structured ontology.

Experimental Protocol

Model and Dataset Selection

Seven VLMs spanning general-purpose and medical-adapted architectures (e.g., MedGemma, Lingshu, Llava, Qwen3-VL) are evaluated across five chest X-ray benchmarks including MIMIC-CXR, CheXpert, NIH-CXR14, and Indiana-IU. Reports are generated with distinct findings and impression sections, processed into structured evidence for verification.

Limitations of Lexical Evaluation

Lexical overlap metrics (BLEU, ROUGE-L) demonstrate consistently low scores across models, heavily penalizing legitimate clinical paraphrasing and failing to capture deductive reasoning. This underscores their inadequacy for auditing internal consistency or clinical reasoning.

Reference-Free Neurosymbolic Auditing

The neurosymbolic verifier enables rigorous, reference-free audit of internal consistency. Two formal metrics are introduced:

• Soundness ($S$): Proportion of impression claims that are logically entailed by findings.
• Completeness ($C$): Proportion of logically entailed diagnoses that are verbalized in the impression.

These metrics quantify deductive reliability, offering explicit thresholds for safe deployment ($S \ge 0.99$ targeted for clinical use).

Three VLM reasoning profiles emerge:

• Balanced models (e.g., MedGemma-27B): High soundness and completeness; impression diagnoses are reliably supported by findings.
• Conservative observers (e.g., Qwen3-VL-8B): Maximize soundness (rare hallucinations) at the expense of completeness (omitted logical diagnoses).
• Stochastic generators (e.g., Llava-Vicuna-7B): Low precision and completeness, frequently hallucinating unsupported diagnoses.

  Figure 2: Heatmap visualization of multiple VLMs’ performance on deductive reliability metrics, exposing distinct reasoning failure modes among model families.

Symbolic Filtering and Performance Guarantees

Application of symbolic entailment filtering (post-hoc deduction enforcement) on labeled datasets consistently yields increased soundness and precision, with marginal reductions in recall and completeness. Diagnoses not deductively supported by findings are conservatively filtered, systematically eliminating unsupported hallucinations. The completeness loss is minimal, confirming that logically forced diagnoses are largely preserved. This verification delta provides quantifiable, post-hoc guarantees for safe deployment of generative clinical assistants.

Practical and Theoretical Implications

The neurosymbolic verification framework represents a shift from empirical string-matching toward formal internal consistency auditing in clinical AI. By leveraging formal verification and autoformalization, the approach provides deterministic, reference-free assessment of deductive reliability. Conditional guarantees—grounded in symbolic translation and domain knowledge—align with safety standards typical in medical and verification domains.

Practically, this methodology enables integration of generative assistants in clinician workflows with reduced risk of automation bias, hallucinated diagnoses, and logical inconsistencies. Theoretically, the approach generalizes to verification tasks across medical AI, offering scalable formal methods for auditing reasoning in complex generative systems.

Future directions may encompass expansion to broader clinical modalities, improved ontology coverage, integration of uncertainty quantification in autoformalization, and multi-tier hybrid verification schemes.

Conclusion

The neurosymbolic verification framework deterministically audits clinical reasoning in radiology VLMs, addressing deductive failures unrevealed by empirical metrics. By enforcing logical entailment through structured autoformalization and SMT solving, the approach provides post-hoc guarantees of diagnostic soundness and precision, with minimal recall trade-offs. This work advances evaluation and deployment safety for generative clinical assistants, aligning AI verification methodologies with the rigorous needs of medical practice and formal system assurance (2602.24111).