# A sparse-attention method speeds up long-context hybrid LLMs

> Researchers report faster decoding in six hybrid models, without changing their weights, while matching baseline accuracy on 21 long-context datasets.

Oossa · 2026-11-05 · https://oossa.com/en/a-sparse-attention-method-speeds-up-long-context-hybrid-llms

Event date: 2026-11-04

Researchers describe a way to speed up long-context text generation in hybrid large language models without retraining them or changing their weights. Their method has each full-attention layer focus on recent tokens, a pinned prefix and a limited set of relevant tokens selected from the wider context.

In tests across six models with 4 billion to 397 billion parameters, the researchers report matching baseline accuracy on 21 datasets. For Qwen3.5-4B, decoding 256,000 tokens was up to 2.22 times faster; these are the paper’s reported results, not an independent evaluation.

## The facts

- The tests covered six hybrid models from three model families, ranging from 4B to 397B parameters.
- The paper was submitted to arXiv on November 4, 2026.

## Why it matters

For teams running long-context hybrid models, the reported speedups could reduce decoding time without retraining, though the paper does not establish how the method performs in their own systems.

## Sources & references

1. [Accelerating Long-Context Decoding in Large-Scale Hybrid LLMs Without Retraining](https://arxiv.org/abs/2611.06133) – arXiv, 2026-11-05

Last updated: 2026-11-05