Grounding AI Explanations in Experience: A Reflective Cognitive Architecture for Clinical Decision Support

Abstract: Effective disease prediction in modern healthcare demands the twin goals of high accuracy and transparent, clinically meaningful explanations. Existing machine learning and LLM based approaches often struggle to balance these goals. Many models yield accurate but unclear statistical outputs, while others generate fluent but statistically unsupported narratives, often undermining both the validity of the explanation and the predictive accuracy itself. This shortcoming comes from a shallow interaction with the data, preventing the development of a deep, detailed understanding similar to a human expert's. We argue that high accuracy and high-quality explanations are not separate objectives but are mutually reinforcing outcomes of a model that develops a deep, direct understanding of the data. To achieve this, we propose the Reflective Cognitive Architecture (RCA), a novel framework that coordinates multiple LLMs to learn from direct experience. RCA features an iterative rule refinement mechanism that improves its logic from prediction errors and a distribution-aware rules check mechanism that bases its reasoning in the dataset's global statistics. By using predictive accuracy as a signal to drive deeper comprehension, RCA builds a strong internal model of the data. We evaluated RCA on one private and two public datasets against 22 baselines. The results demonstrate that RCA not only achieves state-of-the-art accuracy and robustness with a relative improvement of up to 40\% over the baseline but, more importantly, leverages this deep understanding to excel in generating explanations that are clear, logical, evidence-based, and balanced, highlighting its potential for creating genuinely trustworthy clinical decision support systems. The code is available at \https://github.com/ssssszj/RCA.

• The paper presents a novel Reflective Cognitive Architecture that unifies predictive accuracy with explainability through iterative, experience-driven rule base optimization.
• It employs a closed-loop system of multiple LLMs to generate evidence-based explanations, validate clinical predictions, and mitigate cognitive bias.
• Experimental results on CRT, Diabetes, and Heart Disease datasets show RCA outperforms 22 baselines, demonstrating high accuracy and robustness to data noise.

Grounding Clinical AI Explanations in Experience: The Reflective Cognitive Architecture

Motivation and Problem Formulation

The deployment of AI in clinical decision support is fundamentally constrained by the dual requirements of predictive accuracy and the generation of explanations that are both transparent and clinically meaningful. Existing approaches, including classical ML models (e.g., Lasso, CatBoost) and LLM-based systems, typically fail to achieve both objectives simultaneously. Classical models provide only statistical artifacts (e.g., feature importances), which are not directly actionable or interpretable for clinicians. LLMs, while capable of generating fluent natural language, often produce explanations that are not grounded in the actual data distribution, leading to statistically unsupported narratives and, in many cases, degraded predictive performance.

The central thesis of this work is that predictive accuracy and high-quality explanations are not orthogonal or competing objectives. Instead, they are mutually reinforcing outcomes of a model that develops a deep, experience-driven understanding of the data. The authors propose that only by forcing the model to engage directly and iteratively with the data—mirroring the experiential learning of human experts—can both accuracy and trustworthy explanations be achieved.

The Reflective Cognitive Architecture (RCA)

The RCA framework is designed to coordinate multiple LLMs in a closed-loop system that iteratively builds, refines, and validates a rule base for clinical prediction and explanation. The architecture is characterized by two core mechanisms: iterative rules optimization and distribution-aware rules checking.

Figure 1: The RCA pipeline, illustrating reflective cycles and distribution-aware checks that drive deep data understanding for both prediction and explanation.

Data Narrative and Distribution Extraction

Structured clinical features are transformed into unstructured text narratives, making them directly accessible to LLMs. In parallel, global data statistics (means, quantiles, frequencies) are extracted to provide a statistical context, ensuring that reasoning is grounded in evidence-based medicine (EBM) and mitigating cognitive bias (CB).

Dynamic Rule Base and Reflective Cycles

At the core of RCA is a dynamic rule base $R$, which serves as the evolving long-term memory of the system. The prediction LLM ($M_{pred}$) generates both a binary disease label and a natural language explanation, explicitly conditioned on the current rule base and the global data distribution. Incorrect predictions are aggregated and passed to a reflection LLM ($M_{ref}$), which updates the rule base by distilling new or revised rules from the error cases. This process emulates experiential learning, converting short-term feedback into generalizable, abstract rules.

Distribution-aware Rules Check

To prevent overfitting to spurious patterns or outliers, a checking LLM ($M_{chk}$) reviews the rule base at the end of each epoch, using the global data distribution as a reference. Low-quality or overly specific rules are pruned, and general rules for outlier detection are summarized. This mechanism ensures that the logical structure of the model remains robust and statistically grounded.

Experimental Evaluation

The evaluation is conducted on three datasets: a proprietary Catheter-Related Thrombosis (CRT) dataset, and public Diabetes and Heart Disease datasets. RCA is compared against 22 baselines, including traditional ML models, standalone LLMs, reasoning-augmented LLMs, and LLM-based agents (tool- and code-based).

Metrics

• Predictive Performance: Accuracy, MCC, F1-score.
• Explanation Quality: Four criteria—Cognitive Load (CL), Logical Argumentation (LA), Evidence-based Medicine (EBM), and Cognitive Biasing (CB)—scored by clinicians on a 1–10 scale.

Main Results

RCA-based approaches consistently achieve the highest predictive performance and explanation quality across all datasets. Notably, on the CRT dataset, RCA+GPT-4.1 achieves an accuracy of 0.8730 and a CL score of 8.16, outperforming all baselines. The results demonstrate that models which develop a deep, experience-driven understanding of the data (as enforced by RCA) occupy the top-right quadrant in the CL vs. Accuracy and CL vs. MCC plots, indicating simultaneous optimization of both objectives.

Figure 2: CL vs. Accuracy and CL vs. MCC on the CRT dataset, showing RCA's superior performance and robustness.

Robustness to Data Noise

RCA exhibits minimal performance degradation under various noise conditions (feature removal, random value deletion, outlier injection), as evidenced by short dashed lines connecting original and noisy data points in the performance plots. In contrast, baselines such as CatBoost and DeepSeek-V3.1 show significant drops in MCC, indicating a lack of robustness and shallow data understanding.

Ablation Study

Systematic removal of any core RCA module (distribution, reflection, or check) leads to a significant collapse in both predictive and explanation metrics, confirming the necessity of each component for deep data understanding.

Qualitative Case Study

A direct comparison of explanations for the same patient by RCA and a strong reasoning LLM baseline (DeepSeek-R1) highlights the practical impact of deep data grounding. DeepSeek-R1 produces a plausible but statistically unsupported narrative, misclassifying the patient due to incorrect thresholding. RCA, in contrast, integrates quantitative thresholds learned from the data and provides a balanced, evidence-based argument, resulting in a correct prediction and a clinically trustworthy explanation.

Figure 3: RCA integrates quantitative thresholds and balanced reasoning, while DeepSeek-R1's explanation is statistically ungrounded and leads to an incorrect prediction.

Implementation Considerations

• LLM Coordination: RCA requires orchestration of multiple LLMs (prediction, reflection, checking) with explicit prompt engineering for each role. The system is agnostic to the specific LLM used, as demonstrated by comparable results with Qwen2.5-72B and GPT-4.1.
• Resource Requirements: The iterative, multi-agent process increases computational overhead compared to single-pass LLM or classical ML approaches. However, the performance gains in both accuracy and explanation quality justify the additional cost in high-stakes clinical settings.
• Scalability: The architecture is modular and can be extended to other structured data domains beyond medicine. The rule base can be persisted and transferred, enabling continual learning and adaptation to new datasets.
• Deployment: For real-world deployment, the RCA pipeline can be integrated into clinical decision support systems, with the rule base and explanation outputs subject to clinician review and regulatory oversight.

Implications and Future Directions

The RCA framework demonstrates that deep, experience-driven data understanding is both necessary and sufficient for achieving state-of-the-art predictive accuracy and trustworthy, actionable explanations in clinical AI. This challenges the prevailing notion of a trade-off between accuracy and explainability, and provides a concrete architectural blueprint for future systems.

Potential future developments include:

• Automated Rule Auditing: Integration of formal verification or statistical auditing tools to further validate the rule base.
• Continual Learning: Online adaptation of the rule base as new data becomes available, with mechanisms for clinician-in-the-loop feedback.
• Generalization to Other Domains: Application of RCA to other high-stakes domains (e.g., finance, law) where both accuracy and explanation are critical.
• Hybridization with Causal Inference: Incorporation of causal reasoning modules to further enhance the robustness and interpretability of the learned rules.

Conclusion

The Reflective Cognitive Architecture establishes a new paradigm for clinical AI, in which predictive accuracy and high-quality explanations are emergent properties of a system that is forced to engage deeply and iteratively with the data. By coordinating multiple LLMs in a closed-loop, experience-driven process, RCA achieves robust, evidence-based, and balanced clinical decision support, setting a new standard for trustworthy AI in medicine.