A model optimization technique that compresses deep learning weights from high-precision floating-point values into lower-bit integers to reduce memory and accelerate execution.
Neural network quantization is a mathematical compression technique that reduces the numerical precision of model parameters (weights and activations) from standard 32-bit floating-point (FP32) or 16-bit floating-point (FP16) representations down to compact integers, typically 8-bit (INT8) or 4-bit (INT4).
In the era of massive foundation models, quantization is the single most important technology enabling large language models (LLMs) and speech recognition networks to execute locally on personal computers and laptops without requiring industrial server racks.
The mathematics of uniform affine quantization
To map continuous floating-point weights $W$ in range $[W_{\text{min}}, W_{\text{max}}]$ to discrete $b$-bit integer values $q \in [0, 2^b - 1]$, quantization computes a scale factor $S$ and zero-point offset $Z$:
$$S = \frac{W_{\text{max}} - W_{\text{min}}}{2^b - 1}$$
$$q = \text{round}\left(\frac{W}{S}\right) + Z$$
During inference, matrix multiplications are performed using high-speed integer arithmetic instructions (such as Apple Silicon NEON or AVX-512 vector units), with results dequantized back to floating-point representations when passed to subsequent layers.
Quantization levels compared
| Precision Format | Bits per Weight | Model Size (7B Parameters) | Speedup on M-Series Mac | Accuracy Retention |
|---|---|---|---|---|
| FP32 | 32 bits | ~28.0 GB RAM | 1.0x (Baseline) | 100.0% |
| FP16 | 16 bits | ~14.0 GB RAM | 2.1x faster | 99.9% |
| INT8 (Q8_0) | 8 bits | ~7.2 GB RAM | 3.8x faster | 99.5% |
| INT4 (Q4_K_M) | 4 bits | ~3.8 GB RAM | 5.4x faster | 98.2% |
Why quantization is essential for Apple Silicon
Apple Silicon architecture features a unified memory model where the CPU, GPU, and Apple Neural Engine share a single high-bandwidth memory pool. However, entry-level MacBook Air and Pro computers often ship with 8 GB or 16 GB of RAM.
Without quantization, running a Whisper Large model and a local summarization LLM would consume more than 20 GB of memory, causing severe system paging. With INT4 and INT8 quantization:
- Whisper Small fits into ~500 MB of RAM.
- A 7-billion parameter summary model fits into ~4 GB of RAM.
- The entire transcription and synthesis pipeline runs simultaneously in the background without impacting system responsiveness.
Quantized intelligence in Oyma
Oyma uses state-of-the-art quantized model checkpoints out of the box. By coupling INT8/INT4 weights with Apple Metal and Core ML shaders, Oyma turns everyday MacBook Air laptops into high-throughput AI workstations that run completely cool and silent.
Frequently asked questions about Neural Network Quantization
What is the difference between FP16 and INT4 quantization?
FP16 uses 16 bits per weight (half-precision float), requiring ~2 GB RAM per billion parameters. INT4 uses 4 bits per weight, reducing memory requirements by 75% to ~500 MB per billion parameters with minimal accuracy loss.
Does quantization degrade speech transcription accuracy?
Post-training quantization to INT8 produces virtually zero measurable loss in Word Error Rate. INT4 quantization may introduce slight degradation on rare vocabulary but provides massive speed gains on edge hardware.
How does Oyma leverage quantization on Mac?
Oyma deploys quantized INT8 and INT4 Whisper and LLM models. This allows multi-billion parameter models to fit easily into unified memory on 8 GB and 16 GB Apple Silicon MacBooks.
Master neural network quantization with private Mac notes.
Store your thoughts in standard Markdown, transcribe calls on-device, and retain 100% data sovereignty.