LLMs Show Quality Degradation in Crisis Translation, Urgency Preservation Asymmetry Found

A recent investigation published on arXiv (2602.13452v2) examines how well large language models (LLMs) and machine translation systems perform in translating crisis-related content, with an emphasis on maintaining a sense of urgency. Utilizing multilingual crisis data (TICO-19, covering 30 languages) alongside a newly created urgency-annotated dataset comprising 100 scenarios translated into 29 languages, the findings indicate that both specialized translation models and LLMs experience significant declines in quality, especially in low-resource languages. Furthermore, a human annotation study uncovers a notable disparity: while human evaluators consistently assess urgency across different prompt languages, LLM-generated urgency evaluations fluctuate. This research underscores the limitations of existing LLMs for critical crisis communication and triage, highlighting the necessity for enhanced models in multilingual crisis contexts.