# Microsoft expands Azure Maia to multi‑gigawatt inference in 2027

> Azure Maia 300 will begin production in 2027, promising 40% more tokens per megawatt than top GPU systems, though the claim has drawn technical criticism.

Oossa · 2026-10-16 · https://oossa.com/en/microsoft-expands-azure-maia-to-multi-gigawatt-inference-in-2027

Microsoft announced that its Azure Maia family will scale to multi‑gigawatt deployments starting in 2027. The new Maia 300 accelerator is built for inference work that needs long context and heavy decoding, not for training models. Microsoft says Maia 300 will deliver 40% more tokens per megawatt than leading GPU‑based systems and up to seven times the rack‑scale token throughput of its predecessor, Maia 200.

## What Maia 300 looks like

Each Maia 300 accelerator packs eight HBM4 memory stacks, giving 288 GB of fast memory and 18 TB/s bandwidth inside a 950‑watt power envelope. A rack holds 96 accelerators across 24 trays, consuming 167 kW and offering 27.6 TB of HBM and 1.7 PB/s aggregate bandwidth. The chips are built on TSMC’s N2P process and use a NUMA memory layout so software can keep data close to the compute cores.

The system’s networking uses a custom AI transport on Ethernet. Accelerators can talk directly to any peer in a rack‑level “superpod” without involving a host CPU or separate NIC. Up to 128 racks can be linked, forming a superpod of 12,288 accelerators. Microsoft says this fabric lets a spare tray replace a failed one transparently.

## Reaction to the efficiency claim

The headline metric – tokens per watt – has been called “absurd” by commenters who note that token definitions vary and the metric ignores factors like latency or cost per token. Critics also ask what context lengths were used for the 40% improvement claim, pointing out that long‑context workloads are often limited by memory rather than compute.

## The facts

- Maia 200 is already in production serving Microsoft Copilot and the MAI team, with GitHub Copilot and Microsoft Foundry to follow by the end of 2026.
- Maia 300 will start production deployments in 2027 and is aimed at inference‑first workloads.
- Each Maia 300 accelerator uses 950 W, holds 288 GB HBM4 memory and delivers 18 TB/s bandwidth; a rack of 96 accelerators uses 167 kW and provides 1.7 PB/s aggregate bandwidth.
- Microsoft claims Maia 300 will deliver 40% more tokens per megawatt than leading GPU systems and up to seven times the rack‑scale token throughput of Maia 200.
- The custom AI network fabric can connect up to 12,288 accelerators across 128 racks in a superpod.

## Why it matters

For customers of Microsoft’s own services, the new hardware could mean faster responses and lower energy costs if the claimed efficiency holds. However, Microsoft has not said whether external customers will be able to rent Maia capacity, so the benefit may stay inside Microsoft products for now.

## Sources & references

1. [Microsoft Commits Maia to Multi-Gigawatt Inference, and Its Efficiency Metric Draws Questions](https://www.infoq.com/news/2026/10/azure-maia-300/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=AI%2C+ML+%26+Data+Engineering) – InfoQ, 2026-10-16

Last updated: 2026-10-16