agents on the live web
benchmarking information extraction when the ground truth keeps moving
Most information-extraction benchmarks are frozen snapshots. The web is not. A model that scores well on a static crawl from two years ago tells you very little about whether an agent can pull a correct answer off a page that was edited this morning.
LiveWeb-IE (Yang* et al., 2026) is a benchmark for online web information extraction — evaluation against the live web, where pages change, layouts drift, and the answer has to be re-grounded every time. It was accepted to ICLR 2026 as a poster.
What I care about here
- Measurement that does not rot. A benchmark whose ground truth is refreshed rather than frozen.
- Agentic execution under drift. Retrieval, extraction, and verification as a loop, not a single forward pass.
- Honest failure modes. Separating “the model cannot extract” from “the page moved.”
This connects directly to what I want to build long-term: enterprise-grade, domain-specific agents that are evaluated against the world they will actually run in.