Amazon announced that SGLang-Neuron is publicly available through an AWS Deep Learning Container. The plugin links the SGLang serving system with AWS Trainium AI chips, letting you launch models and generate RL rollouts using the familiar SGLang API. It runs on native PyTorch and supports features like mixture‑of‑experts and KV‑cache transfer over Elastic Fabric Adapter. A quickstart guide shows how to start the container on a Trainium3 EC2 instance and serve a 120‑billion‑parameter model.
Why it matters
Developers can now run very large language models on AWS’s Trainium chips without changing their SGLang code, lowering cost and simplifying deployment.
