Bactrainus: Modular Selector-Reader Framework for Multi-hop QA with Llama 3.1

The study presents Bactrainus, a modular framework designed to enhance large language models for complex multi-hop question answering. It tackles the issue of evidence bottlenecks by differentiating between paragraph selection, supporting-sentence identification, and answer generation. In the context of the English HotpotQA distractor setting, findings indicate that providing complete candidate context, rather than relying on gold supporting facts, leads to a decline in answer token-overlap F1 by 17-21 points, highlighting that mere scale is not enough. The framework also features optional question decomposition and teacher-generated rationale supervision to assess the benefits of added reasoning structure. Evaluation includes foundation-model screening, controlled context and prompting ablations, and efficient parameter adaptation of Llama 3.1 8B and 70B Instruct readers. The paper can be found on arXiv under identifier 2501.06286v3, recently updated via replace-cross.