TradeVerse: New Benchmark for LLMs in Political Negotiation

TradeVerse has been launched by researchers as a benchmark aimed at assessing large language models (LLMs) in the context of long-term political negotiations. This benchmark is constructed from specific trade concerns of the World Trade Organisation (WTO) and includes minutes from 1,170 meetings across 5 groups and 89 product categories. It outlines three distinct tasks, with the initial task requiring systems to scrutinize historical meeting records and forecast the harmonized system codes (HS chapters) for the products being discussed. This approach overcomes the shortcomings of current benchmarks that focus on single documents or tasks, acknowledging that actual negotiations are iterative and reliant on context. The findings are published in a paper on arXiv (2608.06549).