Hash-Based Aggregation States for Long-Context Language Models

A new arXiv paper (2607.26448) introduces a model-side aggregation interface using Hash-based HyperLogLog (HLL) sketch states to improve long-context language model performance on non-additive, set-based tasks like cardinality estimation and grouped statistics. The approach maintains compact 2 KiB HLL states alongside a frozen language model, with an extractor mapping records to canonical identities that update the state via hashing. These states can be merged across context segments or read directly for reasoning, avoiding an extra generate-execute-return cycle. The method addresses known limitations in long-context models for tasks involving logs, program outputs, tables, and multi-turn conversations.