Oossa

Valkey 9.2 adds shared caching for AI inference

The open-source datastore adds distributed KV caching and changes aimed at cutting backup memory, replication traffic and large sorted-set overhead.

NoteBy Published by Oossa: 1 min read

Valkey announced general availability of version 9.2, alongside an open-source distributed key-value cache for AI inference. It lets inference engines reuse computed prompt and conversation data across a GPU fleet; teams using self-managed open-weight models can connect it through LMCache with vLLM or SGLang.

Valkey says the cache can cut inference costs by up to 40% and time-to-first-token by up to 80%. The release also adds forkless snapshots to avoid backup-related memory spikes, and replication compression that the company says reduces traffic by roughly 50% to 65%.

Why it matters

Teams running open-weight models on their own GPU fleets can try reusing inference work through LMCache, though the advertised savings are company claims, not a guarantee for every workload.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.