Develop a fast implementation of Minimum Bayes Risk decoding

Develop a computationally efficient implementation of sample-based Minimum Bayes Risk decoding that substantially reduces walltime while preserving accuracy in speech-to-text applications.

Background

The paper reports that MBR decoding improves accuracy over beam search but incurs significant computational cost, with walltime dominated by utility computation and hypothesis sampling. Although faster MBR algorithms exist in the literature and Whisper has optimized inference libraries (e.g., faster-whisper, whisper.cpp), integrating these optimizations for MBR requires additional engineering.

The authors explicitly state that creating a fast implementation is future work, highlighting a practical barrier to MBR adoption in latency-sensitive scenarios.