Soro: Tajik-Specialized LLM Outperforms Gemma 3 on Local Benchmarks

A team of researchers has unveiled Soro, a series of conversational large language models (LLMs) tailored for the Tajik language, which has seen limited AI representation. These models are constructed from open-weight Gemma 3 checkpoints and undergo continual pretraining exclusively in Tajik, utilizing a meticulously curated corpus of 1.9 billion tokens that encompasses filtered web content, PDF files, and educational materials aligned with curricula. This is succeeded by supervised instruction tuning with 40,000 examples modeled after Tajik teachers. To assess their efficacy, the researchers developed a collection of Tajik benchmarks focused on general knowledge, linguistic skills, and academic entrance exams, which are available on Hugging Face. Soro significantly surpasses equivalent Gemma 3 models on these benchmarks while maintaining robust performance in English on standard datasets. The models are optimized for practical use in Tajikistan, considering constraints in compute and connectivity, with enhancements through FP8 and INT4 quantization.