# NormTune improves cheap fine-tuning for math and reasoning tasks

> Researchers add a norm‑guided step to parameter‑efficient fine‑tuning, closing the gap with full fine‑tuning on several reasoning benchmarks.

Oossa · 2026-10-22 · https://oossa.com/en/normtune-improves-cheap-fine-tuning-for-math-and-reasoning-tasks

The paper NormTune proposes a simple add‑on for parameter‑efficient fine‑tuning (PEFT). It looks at how much each layer’s weights change and boosts layers that change too little. Adding NormTune to existing PEFT methods like LoRA, Bone and MoRA raised average accuracy by up to 24 % on hard reasoning tests such as AIME and MATH. With a larger rank (256), the NormTune‑enhanced methods matched full fine‑tuning performance, sometimes beating it by a few percent.

## The facts

- NormTune added 24.24% average gain to rank‑8 LoRA on AIME 2024, MATH 500, GPQA and AIME 2025.
- Published on 2026-10-22 (arXiv:2610.29134).

## Why it matters

It lets researchers and developers achieve near‑full‑model accuracy on reasoning tasks with far less GPU time and memory.

## Sources & references

1. [NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks](https://arxiv.org/abs/2610.29134) – arXiv, 2026-10-22

Last updated: 2026-10-22