# llama.cpp adds faster SYCL attention for quantized caches

> Release b11980 adds an Intel GPU kernel for small-batch decoding with q4_0 and q8_0 KV caches; prompts still use existing kernels.

Oossa · 2026-10-29 · https://oossa.com/en/llama-cpp-adds-faster-sycl-attention-for-quantized-caches

The llama.cpp b11980 release adds a SYCL flash-attention kernel for decoding with q4_0 and q8_0 quantized key-value caches. It handles batches of 1 to 31 query tokens, avoiding scalar dequantization or converting the full cache to f16 for each call.

The release notes report that on an Intel Arc B580, Llama 3.1 8B Q4_K_M with a q4_0 cache reached 54.2 tokens per second at a 16K context, up from 31.3. That result is for one stated setup; prompt processing remains on the existing kernels, and the new kernel can be disabled with GGML_SYCL_FA_DPAS=0.

## The facts

- Release: llama.cpp b11980.
- Reported B580 result: 31.3 to 54.2 tokens per second at 16K context, using Llama 3.1 8B Q4_K_M and a q4_0 KV cache.

## Why it matters

For people decoding on an Intel Arc B580 with quantized caches, this could mean faster token generation in the tested setup; the release gives no evidence that prompt processing or other hardware will speed up similarly.

## Sources & references

1. [ggml-org/llama.cpp b11980](https://github.com/ggml-org/llama.cpp/releases/tag/b11980) – llama.cpp, 2026-10-29

Last updated: 2026-10-29