Lightweight end-to-end Bayesian uncertainty propagation in transformers

Develop a comprehensive and lightweight Bayesian framework that propagates uncertainty from token embeddings through attention layers to final decision layers in transformer-based clinical language models, suitable for practical deployment in clinical AI systems.

Background

Within the survey of prior Bayesian approaches for NLP and transformers, the paper notes that existing methods typically provide uncertainty only at the output level or operate on isolated layers (e.g., embeddings or the classifier head), failing to propagate uncertainty throughout the model’s reasoning pipeline.

The authors emphasize that a fully integrated approach—capturing and transmitting uncertainty from embeddings through attention and into decision layers—has not been comprehensively achieved in a manner that is also computationally lightweight for practical clinical use. This motivates the problem of designing an end-to-end Bayesian framework that is efficient enough for deployment in clinical settings.

References

Although some researchers have explored how uncertainty may influence attention mechanisms , current approaches still fall short of achieving a full Bayesian treatment that propagates uncertainty from embeddings through attention layers and into final decision layers. Developing a comprehensive yet lightweight framework that supports this type of end-to-end uncertainty propagation remains an open challenge for practical and clinical AI systems.

— MedBayes-Lite: Bayesian Uncertainty Quantification for Safe Clinical Decision Support  (2511.16625 - Hossain et al., 20 Nov 2025) in Section 2.1 (Foundational Bayesian Methods in Deep Learning), Related Work