Valkey announced general availability of version 9.2, alongside an open-source distributed key-value cache for AI inference. It lets inference engines reuse computed prompt and conversation data across a GPU fleet; teams using self-managed open-weight models can connect it through LMCache with vLLM or SGLang.
Valkey says the cache can cut inference costs by up to 40% and time-to-first-token by up to 80%. The release also adds forkless snapshots to avoid backup-related memory spikes, and replication compression that the company says reduces traffic by roughly 50% to 65%.
Why it matters
Teams running open-weight models on their own GPU fleets can try reusing inference work through LMCache, though the advertised savings are company claims, not a guarantee for every workload.