Oossa

llama.cpp adds new OpenCL kernels for small‑batch workloads

The open‑source Llama inference library now includes several optimized OpenCL GEMM variants for low‑batch runs, expanding support for Snapdragon GPUs.

NoteBy Published by Oossa: 1 min read

The ggml‑org/llama.cpp project released an update (b11763) on 2026‑10‑21. It adds a handful of OpenCL kernels that speed up small‑batch matrix multiplications, including q4_k dp4a GEMM and a narrow mul_mm_f16_f32 variant. The changes also fix an out‑of‑bounds read bug and remove an unnecessary CPU‑fallback guard. The library already runs on many platforms, from macOS Apple Silicon to Windows CUDA and Linux Vulkan, and now better supports Snapdragon CPUs, Adreno GPUs and Hexagon NPUs.

Why it matters

Developers can run Llama models faster on Android devices with Snapdragon chips, making on‑device AI more practical.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.