# llm‑d 1.0 adds in‑house controls for enterprise AI inference

> IBM, Red Hat and partners release llm‑d version 1.0 with QoS, zero‑downtime upgrades and cache routing for self‑hosted large models.

Oossa · 2026-10-20 · https://oossa.com/en/llm-d-1-0-adds-in-house-controls-for-enterprise-ai-inference

IBM and its collaborators announced version 1.0 of the open‑source llm‑d framework on Oct 20 2026 at the PyTorch Conference. The update adds quality‑of‑service controls, zero‑downtime upgrades and smarter cache routing so companies can run large language models on their own Kubernetes clusters without overloading the service. IBM said a demo ran 3,000 coding agents on self‑hosted hardware at five to ten times lower cost than a hosted model. The framework works with inference engines such as vLLM and SGLang.

## The facts

- Version 1.0 released on Oct 20 2026
- Demo: 3,000 agents ran at 5‑10× lower cost than a hosted service

## Why it matters

Enterprises can keep AI workloads in‑house, cutting cloud costs and avoiding dependence on a single provider.

## Sources & references

1. [IBM-backed llm-d framework adds controls for running AI models in-house](https://siliconangle.com/2026/10/20/ibm-backed-llm-d-framework-adds-controls-for-running-ai-models-in-house/) – SiliconANGLE, 2026-10-20
2. [llm-d 1.0 hardens open-source AI inference for day 2 production operations](https://research.ibm.com/blog/llmd-1-0-is-ready-for-production-inference-serving) – IBM Research
3. [Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets](https://www.marktechpost.com/2026/10/20/llm-d-1-0-open-source-distributed-inference-stack/) – MarkTechPost, 2026-10-21

Last updated: 2026-10-21