TIDE Benchmark: Temporal QA on Evolving Documents

A new benchmark called TIDE has been developed by researchers to evaluate the capacity of large language models in managing documents that evolve over time. It consists of 3,050 question-answer pairs sourced from 644 official customs instruments released by the Government of Bangladesh from 1969 to 2025. These documents feature a complex mix of codes, varying layouts, and dual calendar dates, posing significant challenges for AI systems. The primary focus is on version resolution, which involves determining the applicable version of a document on a specific date. Unlike encyclopedic data, legal texts specify what they replace and their effective dates, allowing earlier versions to remain valid. Current temporal QA datasets do not address version resolution adequately. TIDE includes eight task types and aims to address this gap. Verified by experts, it emphasizes the importance of temporal accuracy in customs instruments. This research underscores the shortcomings of existing LLMs in processing time-sensitive data and serves as a valuable resource for future studies. The paper can be found on arXiv with the identifier 2608.08512.