The ggml‑org team released llama.cpp version b12345. It adds support for Q5_0 and Q5_1 quantization, a 5‑bit compression that can speed up inference. The update also includes dedicated kernels for these formats on Hexagon devices. Pre‑built binaries are provided for macOS Apple Silicon, macOS Intel, iOS, and several Ubuntu builds, including CPU‑only and Vulkan variants.
Why it matters
Developers can run LLMs faster on low‑power hardware, expanding the range of devices that can host local AI models.
