# llama.cpp adds few‑row Metal kernels for faster speculative decoding on Apple M3 Ultra

> The open‑source LLM runtime now uses new matrix‑multiply kernels on Apple Silicon, cutting token‑generation time for certain quantized models.

Oossa · 2026-10-05 · https://oossa.com/en/llama-cpp-adds-few-row-metal-kernels-for-faster-speculative-decoding-on-apple-m3

The llama.cpp project released version b11404 on Oct 5, 2026. It adds Metal kernels that multiply several rows at once (2‑16 rows) for Apple GPUs. These kernels dequantize each weight once and share work across SIMD groups, so speculative decoding on an M3 Ultra is faster than the previous mat‑vec approach. The new kernels are used only on devices supporting the Apple7 GPU family and when the row count reaches a threshold (e.g., 6 rows for F32). The change also refines fusion logic and removes a batched‑copy fusion that caused issues.

## The facts

- Release date: Oct 5, 2026 ("Mon Oct 05 2026 08:06:22 GMT+0200").
- New Metal kernels handle 2–16 rows and improve speculative decoding on M3 Ultra.

## Why it matters

Apple‑silicon users running llama.cpp will see lower latency when generating text with speculative decoding, especially on M3 Ultra devices.

## Sources & references

1. [ggml-org/llama.cpp b11404](https://github.com/ggml-org/llama.cpp/releases/tag/b11404) – llama.cpp, 2026-10-05

Last updated: 2026-10-05