Oossa

llama.cpp b12549 fixes attention handling for Inkling2

The update gives Inkling2 a dedicated relative-position attention input and fixes mask reuse for hybrid-memory models including LFM2 and LFM3.

NoteBy Published by Oossa: 1 min read

The ggml-org team has released llama.cpp b12549 with fixes to attention-mask handling for Inkling2, a model supported by the project. The update gives Inkling2 a dedicated input that builds its dense attention mask and relative-position bias from the same cells. It also fixes hybrid-memory mask construction that prevented graph reuse with a unified cache and flash attention, affecting LFM2, LFM3 and Inkling2. The release additionally allows Inkling2’s expert-weight scale and final-logit softcap to be zero.

Why it matters

For people running these hybrid-memory models in llama.cpp, the fixes address a condition that stopped graph reuse with a unified cache and flash attention; the release notes do not quantify any speed improvement.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.