# MA-Bench evaluates language models on military planning tasks

> The new MA‑Bench dataset tests how well LMs turn messy documents into operational plans; the top model answered 62% of rubric items.

Oossa · 2026-10-15 · https://oossa.com/en/ma-bench-evaluates-language-models-on-military-planning-tasks

Researchers released MA‑Bench, a benchmark that measures a language model’s ability to create an operational plan from a set of contradictory documents. The test includes ten expert‑built scenarios, each with ten to twelve synthetic documents and 1,257 pass/fail requirements written by planners. A reasoning‑model judge grades the model‑generated briefs, matching the consistency of a four‑planner panel (Fleiss κ ≈ 0.42). Across twenty frontier and open‑weight models, the best scored 62% correct on the rubric.

## The facts

- 10 scenarios with 1,257 expert‑written requirements
- Best model achieved 62% correct

## Why it matters

It shows current language models still miss many inference‑heavy planning judgments, limiting their use for real‑world mission analysis.

## Sources & references

1. [MA-Bench: Measuring Operational Planning in Language Models](https://arxiv.org/abs/2610.19138) – arXiv cs.LG, 2026-10-15

Last updated: 2026-10-15