The open‑source LLM serving library vLLM 0.31.0 introduces a preload CLI that keeps post‑quantized weights resident in GPU memory across engine restarts.
NoteBy OossaPublished by Oossa: 1 min read
The vLLM project released version 0.31.0 on October 5, 2026. The highlight is a new preload command that starts a weight‑cache daemon, letting post‑quantized model weights stay in GPU memory when the server restarts. The feature works with data parallelism and adds a health‑check endpoint. The release also bundles 717 commits from 307 contributors.
Developers can update or restart LLM services without re‑loading large models, cutting downtime for applications that rely on vLLM.
Models
A small language model, Qwen3.5 0.8B, now tracks English learners’ grammar mastery cheaper and more accurately than GPT‑5 prompts.
Models
The new OSS Scanner uses Anthropic’s Claude Mythos model to give open‑source projects periodic, model‑generated vulnerability reports at no cost.
Models
Claude now offers Motion for animated explainer videos and Dashboards for auto‑updating data views, available to paid users.
Oossa · Newsletter
The week in AI, explained
Every Monday: the stories worth knowing, in plain language. Free, no spam.
