# llama.cpp adds per‑expert token scaling for MoE models

> The open‑source llama.cpp library now scales MoE down‑projections per token, improving tensor‑parallel support.

Oossa · 2026-10-21 · https://oossa.com/en/llama-cpp-adds-per-expert-token-scaling-for-moe-models

The ggml‑org/llama.cpp project released a new version on 2026-10-21. It replaces a single global scaling factor for mixture‑of‑experts (MoE) down‑projections with a per‑expert‑token factor, using a POOL_1D max reduction. The change also adds support for tensor parallelism by handling POOL_1D in the split‑state logic. The update includes code clean‑ups and a refactored scaling return structure.

## The facts

- Release date: 2026-10-21 ("Wed Oct 21 2026 08:33:35 GMT+0200")
- Change: per‑expert‑token scaling for MoE down‑projection

## Why it matters

Developers running MoE models can now get more accurate scaling on each token, which may improve performance on multi‑GPU setups.

## Sources & references

1. [ggml-org/llama.cpp b11785](https://github.com/ggml-org/llama.cpp/releases/tag/b11785) – llama.cpp, 2026-10-21

Last updated: 2026-10-21