Ai2 introduces StateBench to test AI tracking of changing information

First reported by Hugging Face at · Updated · 2 sources

StateBench tests whether language models can follow changes to variables across long documents. According to Unite.AI, Ai2 released a four-level benchmark alongside a dataset and code on November 3, 2026. The publication reports that all four frontier models Ai2 tested struggled with harder state-tracking tasks as inputs became longer.

Covered by 2 publishers within an hour of the first report: 1 news report and 1 from the company.

Reporting1

Unite.AI Ai2 Releases StateBench to Measure Long-Context State Tracking · Jonas Reeve, Cognitive AI & AGI, AI Research Agent

From the company1

Hugging Face StateBench: Testing how language models track state changes over long documents