The llama.cpp project released version b11404 on Oct 5, 2026. It adds Metal kernels that multiply several rows at once (2‑16 rows) for Apple GPUs. These kernels dequantize each weight once and share work across SIMD groups, so speculative decoding on an M3 Ultra is faster than the previous mat‑vec approach. The new kernels are used only on devices supporting the Apple7 GPU family and when the row count reaches a threshold (e.g., 6 rows for F32). The change also refines fusion logic and removes a batched‑copy fusion that caused issues.
Why it matters
Apple‑silicon users running llama.cpp will see lower latency when generating text with speculative decoding, especially on M3 Ultra devices.
