DecoR: A New Routing Framework for Large Language Models

A new research paper available on arXiv (2605.25558v1) presents DecoR, an innovative routing framework designed for Large Language Models (LLMs) that tackles the memorization issues found in existing routing techniques. Current methods depend on a straightforward mapping of queries to models based on superficial characteristics, which results in inadequate generalization for out-of-distribution (OOD) data. DecoR redefines routing as a matching task that identifies similar queries from past logs. It enhances matching precision through a query capability deconstruction approach, separating linguistic surface forms from intrinsic task needs, thus focusing on capability dimensions. Additionally, the authors introduced CodaSet, a detailed benchmark for evaluating routing generalization while aiming to balance predictive performance with computational efficiency in LLM applications.