Oossa

llama.cpp fixes Vulkan NVFP4 scale lookup bug

Release b12073 expands a shared lookup table to prevent out-of-bounds reads for certain scale bytes. Each affected pipeline uses 512 more bytes of shared memory.

NoteBy Published by Oossa: 1 min read

The llama.cpp b12073 release fixes a Vulkan bug in how NVFP4 data looks up UE4M3 scale values. Certain scale bytes could make the code read beyond a shared-memory lookup table, because the table had 128 entries while the raw byte could represent 256 values.

The fix expands the table and its memory budgets to 256 entries. Each NVFP4 pipeline uses 512 more bytes of shared memory. The release notes say models quantized by llama.cpp are unaffected, because its conversion code does not set the bit that triggers the out-of-bounds lookup.

Why it matters

People running NVFP4 workloads on Vulkan get protection from an out-of-bounds lookup, while llama.cpp-quantized models are not affected by the bug, according to the release notes.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.