# METR adds open‑ended tasks to its AI progress graph

> The nonprofit’s updated plot shows AI still doubling on easy tasks every four months, but models lag on judgment‑heavy work like research design.

Oossa · 2026-11-20 · https://oossa.com/en/metr-adds-open-ended-tasks-to-its-ai-progress-graph

MIT Technology Review reports that the AI evaluation nonprofit METR has released an updated version of its time‑horizon plot. The new graph adds tasks that take humans much longer and that require more judgment, such as designing AI evaluations or scoring philosophical arguments. The original plot, which was widely shared, measured progress by how long a human would need to finish a task and showed that frontier models double the length of tasks they can handle every four months. The update keeps that trend for easy, verifiable tasks like coding and math, but it also highlights a growing gap on open‑ended tasks.

The added tasks are meant to probe capabilities that matter for automating research and development. In the evaluation‑design task, even the most advanced models performed only “borderline above the bar,” falling short of being useful for METR staff. In the philosophical‑argument task, models also scored below human experts. Researchers quoted in the article say the results are not surprising; earlier work found AI still cannot reproduce human computer‑science research on its own.

METR’s own staff note that measuring performance on these tasks is hard. Human time estimates replace actual testing for many new tasks because paying people to complete hundreds‑of‑hour jobs is costly. The report therefore includes fewer judgment‑heavy tasks and admits the data are less robust than the original set. Experts like Sayash Kapoor of UC Berkeley and Najoung Kim of Boston University see the inclusion as a positive step but caution that trends for hard‑to‑measure abilities will remain noisy.

Overall, the updated plot confirms rapid gains on tasks where success is easy to verify, while progress on judgment‑laden work remains tentative. The graph will likely shape how the community thinks about the timeline for AI‑driven R&D automation.

## Why the new tasks matter

If AI is to help design new experiments or write research proposals, it must handle tasks that cannot be checked by a simple pass/fail test. The updated METR plot shows that current models are still behind humans on those fronts, suggesting that fully automated AI research is not imminent.

For companies betting on AI to speed up product development, the gap means they should not expect immediate gains from using AI for high‑level strategic decisions. Instead, the technology remains most reliable for concrete, verifiable work such as code generation or data analysis.

## The facts

- METR’s original plot showed AI systems double the length of tasks they can complete every four months.
- The updated plot adds tasks estimated to take humans far longer than the original suite, including designing AI evaluations and scoring philosophical arguments.
- The most advanced models achieved only “borderline above the bar” on the evaluation‑design task, still below METR staff’s usefulness threshold.
- Models scored below human performance on the philosophical‑argument task, where agreement was judged against a single human rater.
- The report’s new human‑time estimates replace actual testing for many tasks because paying for hundreds of hours of work is prohibitively expensive.

## Why it matters

For everyday users, the update means AI tools will keep getting better at clear‑cut jobs like coding, but they are not yet ready to replace human judgment in research planning or philosophical reasoning. Businesses should continue to rely on AI for automating routine tasks while keeping humans in the loop for strategic decisions.

## Sources & references

1. [The most misunderstood graph in AI just got a major update](https://www.technologyreview.com/2026/11/20/1149399/the-most-misunderstood-graph-in-ai-just-got-a-major-update/) – MIT Technology Review, 2026-11-20

Last updated: 2026-11-20
