# Google adds native PyTorch support for vLLM and SGLang on TPU

> Google launches PyTorch backends for the vLLM and SGLang inference engines, letting developers run these models on TPU hardware with familiar PyTorch tools.

Oossa · 2026-10-20 · https://oossa.com/en/google-adds-native-pytorch-support-for-vllm-and-sglang-on-tpu

Google announced new inference engines that run the open‑source vLLM and SGLang projects on TPUs using a native PyTorch backend called TorchTPU. The engines were co‑developed with Inferact (vLLM) and RadixArk (SGLang) and already serve production workloads at companies like Meta. They include TPU‑specific optimizations such as custom Pallas kernels and KV‑cache management, aiming to make large‑scale, low‑latency serving easier for PyTorch users.

## The facts

- vLLM‑TorchTPU can serve a 2.8 T‑parameter Kimi K3 model at over 700 tokens per second on an Ironwood TPU.
- Both engines are running in production with preview customers and have benchmarked the lowest cost per million tokens at $0.0201.

## Why it matters

PyTorch developers can now deploy fast, cost‑efficient inference for large language models on Google’s TPU hardware without switching to JAX.

## Sources & references

1. [Announcing New vLLM and SGLang Inference Engines on TPU](https://developers.googleblog.com/en/announcing-new-vllm-and-sglang-inference-engines-on-tpu/) – Google for Developers
2. [Google has joined Amazon in attacking Nvidia’s stronghold with AI app developers](https://techcrunch.com/2026/10/20/google-has-joined-amazon-in-attacking-nvidias-stronghold-with-ai-app-developers/) – TechCrunch, 2026-10-20

Last updated: 2026-10-20