A team of Israeli researchers introduced Jev, a decision language model that gives structured judgments on clinical AI output. They tested it on four tasks, including 118 pediatric answers graded by doctors. Jev’s agreement score (Gwet's AC2 0.50) was on par with or better than seven of the 12 large language model (LLM) judges they compared, such as Claude Opus 5.5 (0.49) and GPT‑6 (0.38). Jev also matched or beat LLM judges on binary HealthBench criteria and performed similarly to a three‑LLM jury on MedHELM tasks. The paper notes Jev costs 73‑294× less and runs 7‑57× faster than the frontier LLM judges.
Why it matters
Hospitals could evaluate AI tools for children more affordably and quickly using Jev instead of expensive, slow LLM judges.
