Biaffine LSTM Outperforms Transformers in Low-Resource Dependency Parsing
A recent investigation published on arXiv (2605.02608) examines four different dependency parsing models—Biaffine LSTM, Stack-Pointer Network, AfroXLMR-large, and RemBERT—across twelve languages with diverse typological backgrounds, particularly emphasizing low-resource African languages. The findings indicate that in low-resource contexts, the Biaffine LSTM consistently surpasses transformer models, which regain their effectiveness as the volume of training data increases. This transition occurs within the resource levels typical for treebanks of under-resourced languages. Additionally, morphological complexity, assessed through MATTR, is identified as a notable secondary factor influencing the transformers' relative shortcomings when accounting for corpus size. These insights are vital for NLP professionals focused on languages with limited resources, underscoring the challenges faced by large transformer models in such settings.
Key facts
- Evaluates four parsers: Biaffine LSTM, Stack-Pointer Network, AfroXLMR-large, RemBERT
- Across twelve typologically diverse languages
- Focus on low-resource African languages
- Biaffine LSTM outperforms transformers in low-resource regimes
- Transformers recover advantage as training data increases
- Crossover falls within resource range typical of under-resourced treebanks
- Morphological complexity (MATTR) is a significant secondary predictor
- Study suggests Biaffine LSTM better for low-resource syntactic tool development
Entities
Institutions
- arXiv