Oossa

Metis-1.6 report details two small language models and their failures

Researcher Giulianno Vollmer reports that Metis-1.6’s Logos model scored well in math benchmarks but struggled with code, instruction following and long context.

NoteBy Published by Oossa: 1 min read

Giulianno Vollmer’s Metis-1.6 technical report describes two language models built from scratch: Praxis, with 2.94 billion parameters, and Logos, with 10.23 billion. The report says Logos placed seventh among 17 models across 40 benchmarks, and second among 30 models in mathematics; it also notes weaknesses in code, instruction following and long-context tasks.

Vollmer details several training problems, including an optimizer that never ran and a length-reward bug that could penalize correct, lengthy answers. The report says Logos was trained on 1.02 trillion tokens and was not intended to compete with frontier models.

Why it matters

For developers considering smaller models, the report offers benchmark results and specific training lessons, but its own findings show Logos is weak at code and long-context work; independent validation is not documented here.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.