The llama.cpp b11980 release adds a SYCL flash-attention kernel for decoding with q4_0 and q8_0 quantized key-value caches. It handles batches of 1 to 31 query tokens, avoiding scalar dequantization or converting the full cache to f16 for each call.
The release notes report that on an Intel Arc B580, Llama 3.1 8B Q4_K_M with a q4_0 cache reached 54.2 tokens per second at a 16K context, up from 31.3. That result is for one stated setup; prompt processing remains on the existing kernels, and the new kernel can be disabled with GGML_SYCL_FA_DPAS=0.
Why it matters
For people decoding on an Intel Arc B580 with quantized caches, this could mean faster token generation in the tested setup; the release gives no evidence that prompt processing or other hardware will speed up similarly.
