llm‑d 1.0 adds in‑house controls for enterprise AI inference

IBM and its collaborators announced version 1.0 of the open‑source llm‑d framework on Oct 20 2026 at the PyTorch Conference. The update adds quality‑of‑service controls, zero‑downtime upgrades and smarter cache routing so companies can run large language models on their own Kubernetes clusters without overloading the service. IBM said a demo ran 3,000 coding agents on self‑hosted hardware at five to ten times lower cost than a hosted model. The framework works with inference engines such as vLLM and SGLang.