Oossa

llama.cpp changes Hexagon flash-attention splitting

Release b11868 combines two Hexagon flash-attention split flags and adds team-splitting when KV heads do not fully use available cores.

NoteBy Published by Oossa: 1 min read

The ggml-org/llama.cpp b11868 release changes how Hexagon handles multicore HMX flash attention. It replaces the separate `FA_HEAD_SPLIT` and `FA_SPLIT` settings with one flag, `GGML_HEXAGON_FA_SPLIT`.

With the flag set to 1, work can be split across KV heads and, when there are fewer KV heads than cores, across teams so otherwise idle cores can participate. The release gives Gemma 4 with two KV heads on four cores as an example.

Why it matters

For people running llama.cpp on Hexagon hardware, the change is intended to let cores participate when a model has fewer KV heads than cores; the release notes do not quantify any speedup.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.