Google's new open‑source C++ library cuts KV cache transfer latency by over 90% on large MoE models.
NoteBy OossaPublished by Oossa: 1 min read
Google released TPU Sync, an open‑source C++ library that moves key‑value (KV) caches and model weights between TPU high‑bandwidth memory, host RAM, storage and the network. The company says the library reduces KV transfer latency by more than 90 % on mixture‑of‑experts models ranging from 235 billion to 480 billion parameters.
Developers can refresh large language model state on TPUs much faster, lowering pause time during inference or fine‑tuning.
Models
OpenAI's new ChatGPT Wallet lets its ‘dots’ agents purchase items for U.S. Pro subscribers using tokenized card data.
Models
The AI agent can place phone calls to cable companies, DMVs and restaurants, but many businesses still hang up on the synthetic voice.
Models
The new Gemini 4 Hydrogen model beats the larger Argon on speed, cost and several benchmarks, and is now available to developers and consumers worldwide.
Oossa · Newsletter
The week in AI, explained
Every Monday: the stories worth knowing, in plain language. Free, no spam.
