# Taste-Bench finds AI agents struggle with long-term choices

> A 502-question benchmark tests whether AI can choose the better path before the outcome is known. The top tested model scored 59.7%.

Oossa · 2026-10-18 · https://oossa.com/en/taste-bench-finds-ai-agents-struggle-with-long-term-choices

Researchers introduced Taste-Bench, a 502-question test of whether AI models can choose the better direction during a coding or research task, before later steps reveal the outcome. In tests of 14 models, GPT-5.6 Sol scored highest at 59.7%; questions counted only when the model chose correctly in both answer orders.

Accuracy averaged 62.3% when the clue was already visible, but fell to 21% when it appeared only much later. The researchers also trained a Qwen3.6-27B model to advise a separate agent: on 41 held-out SWE-bench Pro tasks, success rose from 14.6% to 33.7%. The team says the benchmark and code are public.

## The facts

- Taste-Bench contains 502 questions built from agent-run decision points.
- With trained Qwen3.6-27B advice, task success on 41 held-out SWE-bench Pro problems rose from 14.6% to 33.7%.

## Why it matters

For teams using coding agents, the results suggest that passing end-of-task tests may not show whether an agent makes choices that help with later work; whether Taste-Bench predicts performance in everyday deployments remains unknown.

## Sources & references

1. [AI agents have bad taste in long-term decisions and a new benchmark proves it](https://the-decoder.com/ai-agents-have-bad-taste-in-long-term-decisions-and-a-new-benchmark-proves-it/) – The Decoder, 2026-10-18

Last updated: 2026-10-18