SAE-RNA: A Sparse Autoencoder Model for Interpreting RNA Language Model Representations (2510.02734v1)

Abstract: Deep learning, particularly with the advancement of LLMs, has transformed biomolecular modeling, with protein advances (e.g., ESM) inspiring emerging RNA LLMs such as RiNALMo. Yet how and what these RNA LLMs internally encode about messenger RNA (mRNA) or non-coding RNA (ncRNA) families remains unclear. We present SAE- RNA, interpretability model that analyzes RiNALMo representations and maps them to known human-level biological features. Our work frames RNA interpretability as concept discovery in pretrained embeddings, without end-to-end retraining, and provides practical tools to probe what RNA LMs may encode about ncRNA families. The model can be extended to close comparisons between RNA groups, and supporting hypothesis generation about previously unrecognized relationships.