Evaluating Multi-Turn Information Seeking in LLMs

A new research paper on arXiv (2608.14808) introduces MT-InfoSeek, a controlled evaluation suite for multi-turn information seeking in large language models (LLMs). The study formalizes the task as solving a k-underspecified constraint satisfaction problem, where k measures the degree of missing information. MT-InfoSeek comprises 5,251 problems and 9,006 task instances across mathematics, logic, biology, medicine, and general knowledge. The authors evaluate models on what they ask, when they ask it, and how acquired information affects the final answer. Results show performance degrades as underspecification increases, and models often underestimate the amount of additional information needed. The paper is available at https://arxiv.org/abs/2608.14808.