SAE-RNA Interprets RNA Language Model Representations

Researchers introduce SAE-RNA, a sparse autoencoder model designed to interpret representations from the RiNALMo RNA language model. The study applies sparse autoencoders (SAEs) to RNA language model representations, following similar work on protein language models like ESM. SAE-RNA maps RiNALMo's internal representations to known human-level biological features, serving as a probe for understanding how RNA language models organize biological information. The authors frame this as a representation-level analysis rather than definitive biological concept discovery, acknowledging limitations in the approach.