Separating Evidence Interpretation from Decision Aggregation in LLM Systems

A recent paper on arXiv (2608.14509) introduces a novel framework aimed at enhancing the synthesis of information by language models from various sources. The authors contend that existing approaches, which merge all sources into one prompt, mistakenly combine two separate functions: analyzing each source and merging those analyses. They propose a division of these tasks by implementing a four-field evidence tuple (hypothesis, reliability bucket, rationale, provenance) to refine the interface. The paper highlights a failure mode termed 'count-scale drift,' where thresholding an aggregate of unnormalized weights, akin to posterior thresholding, shifts based on the number of sources and increases with reader reliability. Variations in source reliability affect the vote rule and posterior order differently. This research marks a significant advancement in AI and machine learning, particularly in multi-source reasoning and decision-making aggregation.