LLM Ensemble Wins SemEval-2026 Multi-Turn Generation Task

A heterogeneous ensemble of seven large language models (LLMs) won Task B of SemEval-2026 Task 8: MTRAGEval, achieving a conditioned harmonic mean of 0.7827 and ranking 1st out of 26 teams. The system, developed by RaguTeam, uses a GPT-4o-mini judge to select the best candidate response per instance from two prompting variants across diverse model families and scales. Ablation studies confirmed that ensemble diversity is essential, consistently outperforming any single model including the strongest baseline (gpt-oss-120b, 0.6390). The team also introduced Meno-Lite-0.1, a 7B domain-adapted model offering a strong cost-performance trade-off. Their analysis of MTRAGEval highlighted annotation limitations and directions for improvement. The code is publicly available on GitHub.