Identify the mechanism of dual-modality retrieval interaction

Identify why jointly supplying EHR- and ECG-derived retrieval contexts improves PRISM performance whereas supplying either modality-specific memory-bank index alone does not materially improve performance over no retrieval.

Background

The ablation experiments indicate that the retrieval benefit appears only when both genuine modalities are supplied together. However, the experiments do not determine whether the interaction arises from complementary similarity information, the separate projectors, the neighborhood composition, or another aspect of the architecture. The paper explicitly leaves the mechanism unresolved and notes that joint-similarity retrieval was not evaluated.

References

These controls establish an interaction in the deployed PRISM configuration but do not identify its mechanism; an explicit joint-similarity retrieval comparison remains future work.

— Retrieval-Augmented Multimodal Language Model with Heterogeneous Evidence Integration for Clinical Prognosis  (2610.13510 - Shafiq et al., 9 Oct 2026) in Section Population Memory Bank, paragraph 'Same-patient retrieval'; Section 4.5, paragraph 'Population retrieval benefits from dual-modality indexing'