The llama.cpp project released a GPU kernel tweak that adds a fast path for a state size of 64. Zamba2 and similar chips use this size, and before the change every scan step fell back to the CPU, limiting generation to about 6‑7 tokens per second. The new template raises throughput by roughly 35%, moving Zamba2‑7B‑Instruct from 11 to 15.1 tokens per second on the same hardware. Some scan calls (about 28%) still run on the CPU, so full parity isn’t reached yet.
Why it matters
Developers using Zamba2 GPUs can generate text noticeably faster without extra hardware.