The nonprofit added longer and more open-ended tasks to test judgment, not just coding and math skills.
METR, the nonprofit AI evaluation group, released an updated version of its time horizon plot. The revised chart adds tasks that take human testers far longer to complete and tasks that are more open-ended, where success is harder to measure automatically. METR rates models by the length of task they can finish.
The original plot showed that the length of tasks frontier models could complete doubled roughly every four months. That chart circulated widely and drew misinterpretation because its measures were complex. METR built the first version from a suite of tasks that humans had also completed, scoring difficulty by how long each person took.
Coding and math have improved quickly because reinforcement learning works well when a system can check success automatically. Judgment-heavy work such as choosing a research direction resists that technique, since success takes longer to confirm. Builders who forecast capability from coding benchmarks risk overestimating performance on open-ended operational work.
METR included some fuzzier tasks in the new suite, asking models to design AI evaluations that the group’s employees then score by hand. A separate, heavily redacted METR report from earlier this month claimed AI might be doubling the pace of AI progress at Anthropic, igniting fears of recursive self-improvement. Watch whether the expanded plot changes how teams estimate timelines for automating research.
What matters
- METR added tasks that take humans far longer and others that are more open-ended.
- Verifiable tasks like coding improved fast, but judgment-heavy work is harder to train and score.
- Watch whether METR’s expanded plot changes how teams forecast progress on research tasks.
Why it matters
Watch whether METR’s expanded plot changes how teams forecast progress on research tasks.
This GenAI News article was prepared in original wording using reporting and materials published by MIT Technology Review AI. Source reference: https://www.technologyreview.com/2026/11/20/1149399/the-most-misunderstood-graph-in-ai-just-got-a-major-update/.
Drafted by the GenAI News review pipeline.
