LLMs' Multilingual Generalization Tied to Language Similarity Structures
A recent study published on arXiv (2607.22699) explores the challenges faced by large language models (LLMs) in generalizing across different languages, particularly those that are underrepresented in their training datasets. The authors, utilizing insights from cognitive science, contend that effective generalization relies on suitable representations within similarity space. Their findings indicate that the latent representations of LLMs largely reflect the hierarchical organization of the Indo-European language family tree, clustering similar languages in proximity. This implies that the multilingual capabilities of these models are connected to their ability to accurately represent language-level similarity structures.
Key facts
- arXiv paper 2607.22699 examines LLM multilingual generalization.
- LLMs struggle with languages outside English and those poorly attested in training data.
- Cognitive science suggests generalization relies on similarity space representations.
- LLMs' latent representations recover the Indo-European language family tree hierarchy.
- Languages from the same subfamily are grouped closely in representation space.
- The study aims to understand why some models generalize better multilingually.
- Research focuses on hierarchical similarity structures between distinct languages.
- Findings indicate representation quality impacts cross-lingual performance.
Entities
—