Oossa

llama.cpp adds smarter prompt cache handling

The new release lets the server keep only reusable slots in RAM, cutting wasted memory for certain task types.

NoteBy Published by Oossa: 1 min read

The llama.cpp project released version b11642 on 2026-10-15. It changes how the server stores prompt‑cache slots. Only slots that can be restored later are saved, so idle slots for rerank, embedding, decision and infill tasks no longer fill the cache. Decision tasks now skip the cache entirely unless the --cache-ram flag is used. This reduces RAM use and avoids unnecessary cache churn.

Why it matters

Developers running llama.cpp on limited hardware will see lower memory usage and fewer cache‑related slowdowns.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.