Researchers led by Shiyu Ji released RoLaQ, a calibration technique for 2‑bit key‑value caches used in multi‑head latent attention. The method jointly adjusts rotation and quantization range while keeping model weights frozen. In tests on three models and six benchmarks, RoLaQ gave the highest average accuracy among 2‑bit methods. On DeepSeek‑V2‑Lite, an SGLang implementation reached 3.96 × the BF16 cache capacity and 2.51 × the decode speed with a 256K token context.
Why it matters
Developers of long‑context language models can run larger prompts faster without extra hardware.
