# llama.cpp adds fast path for Zamba2 GPUs, boosting token speed

> The ggml-cuda update adds a d_state=64 path, raising Zamba2‑7B‑Instruct generation from 11 to 15.1 tokens per second.

Oossa · 2026-11-08 · https://oossa.com/en/llama-cpp-adds-fast-path-for-zamba2-gpus-boosting-token-speed

The llama.cpp project released a GPU kernel tweak that adds a fast path for a state size of 64. Zamba2 and similar chips use this size, and before the change every scan step fell back to the CPU, limiting generation to about 6‑7 tokens per second. The new template raises throughput by roughly 35%, moving Zamba2‑7B‑Instruct from 11 to 15.1 tokens per second on the same hardware. Some scan calls (about 28%) still run on the CPU, so full parity isn’t reached yet.

## The facts

- Added d_state=64 fast path to ggml-cuda SSM_SCAN kernel
- Improved Zamba2‑7B‑Instruct speed from 11 tok/s to 15.1 tok/s (~35% gain)

## Why it matters

Developers using Zamba2 GPUs can generate text noticeably faster without extra hardware.

## Sources & references

1. [ggml-org/llama.cpp b12207](https://github.com/ggml-org/llama.cpp/releases/tag/b12207) – llama.cpp, 2026-11-08

Last updated: 2026-11-08