LC-SEPLM: Long-range contact-supervised adaptation for sequence-only protein representation learning

A new protein language model, LC-SEPLM (Long-range Contact-supervised ESM Protein Language Model), adapts ESM2 using LoRA and long-range residue-pair contact supervision while maintaining sequence-only inference. It employs pair-specific queries with cross-attention over the complete sequence to capture global context for spatial contacts. Trained on 500,000 AlphaFold Swiss-Prot proteins, LC-SEPLM improved all eight protein-level tasks compared to ESM2, with the largest gain in remote-homology recognition (macro-F1 increased). The model addresses the limitation of standard protein language models that do not explicitly learn three-dimensional residue contacts from sequences.