Oossa

RL fine‑tuning boosts Qwen3.8‑27B repeatable success on ThinkingBox benchmark

After 50 steps of offline‑policy‑relative training, Qwen3.8‑27B lifted its always‑correct task count from 38 to 107 on the 507‑task ThinkingBox‑Bench.

NoteBy Published by Oossa: 1 min read

Microsoft and Hugging Face released the ThinkingBox‑Training code on GitHub. They used a reward derived from database‑state checks to fine‑tune the Qwen3.8‑27B model. After 50 GRPO steps, the model’s pass@1 rose from 51.70% to 60.78% and the number of tasks it got right on every of 20 attempts grew from 38 (7.5%) to 107 (21.1%). The improvement came with a modest drop in at‑least‑once coverage.

Why it matters

For companies that need reliable database updates, the fine‑tuned model is about three times more likely to get a task right every time.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.