Liquid AI Releases LFM2.5-Encoders for Fast Long-Context Inference on CPU

Liquid AI has introduced the LFM2.5-Encoders, a series of natural language processing models designed for rapid long-context inference on CPUs. This lineup includes the LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, both capable of handling an 8,192-token context and built on the LFM2 architecture. They employ bidirectional attention masks along with 30% masked language modeling. The training process incorporates both short-context (1,024 tokens) and long-context adaptations. In benchmark evaluations, the LFM2.5-Encoder-350M secured the fourth position among 14 models on GLUE and SuperGLUE, while the LFM2.5-Encoder-230M surpassed ModernBERT-base. Notably, LFM2.5-Encoder-230M is the quickest across all sequence lengths on CPU. Liquid AI provides demos for various uses, with models accessible on Hugging Face for fine-tuning.