← All AI terms
What does Quantisation mean?
Quantisation
Storing a model's numbers with fewer bits so it needs less memory and runs faster, at the cost of a little quality. Common for running AI locally.
News that mentions Quantisation
- New technique improves low-bit LLM quantization without extra inference cost
- llama.cpp adds hexagon tensor repacking and new quantization support
- llama.cpp adds Vulkan sparse flash attention for quantized keys and values
- llama.cpp adds new WebGPU quantization formats in b11406 release
- llama.cpp update fixes rare quantization crash
- llama.cpp b11345 adds two Hexagon quantization types