The ggml‑org team released llama.cpp version b12291. It adds a Vulkan‑based optimization that dequantizes the key‑value cache using the coopmat2 instruction. The change is applied only when a heuristic based on query tokens and GQA ratio deems it safe. The update also fixes alignment handling for some head sizes, improving performance when the optimization cannot be used.
Why it matters
Developers running Llama models on Vulkan‑compatible hardware can see faster inference without changing their code.
