Routing Pipeline Enhances Multilingual Short-Text Classification
Researchers have introduced an innovative routing pipeline designed to enhance multilingual short-text classification effectively and economically. The new method, outlined in an arXiv paper, operates without the need for task-specific fine-tuning. It employs a fixed-list routing technique, allowing strong languages to be processed directly while translating weaker languages into English for zero-shot classification. This system leverages pretrained sentence encoders. Performance evaluations on the SIB-200 and MASSIVE datasets reveal that traditional inference often fails to recognize the differences between high- and low-resource languages, underscoring the necessity of language-aware routing in applications such as content moderation and customer service.
Key facts
- The pipeline uses a fixed-list routing strategy.
- Stronger languages are kept on a direct multilingual path.
- Weaker languages are translated into English before zero-shot classification.
- The pipeline is fully self-hosted and uses pretrained compact sentence encoders.
- No task-specific fine-tuning is required.
- Evaluated on SIB-200 (15 languages, seven-way topic classification) and MASSIVE (15 locales, 60-intent classification).
- The approach addresses performance gaps between high-resource and low-resource languages.
- The paper is available on arXiv with ID 2608.10939.
Entities
Institutions
- arXiv