Pre-M5 vs M5 Apple GPU decode behavior for compressed KV
Parent: Mac local LLMs: KV cache sizing and quantization · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
Apple's MLX study reports M5 generation is 19-27% faster than M4, explained by memory bandwidth (120 GB/s M4, 153 GB/s M5, +28%), while TTFT is compute-bound and up to 4x faster with the GPU Neural Accelerators.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- Apple's MLX study reports M5 generation is 19-27% faster than M4, explained by memory bandwidth (120 GB/s M4, 153 GB/s M5, +28%), while TTFT is compute-bound and up to 4x faster with the GPU Neural Accelerators. [source]
- Inference: because M5 accelerators speed matmul but not decode bandwidth, a compressed-KV dequant penalty on pre-M5 ALU-bound GPUs (M1 L2 wall) is not expected to be rescued by the accelerators; no measurement of turbo/q4 KV on M5 vs M1 with identical models was found. [source]
Children
- No children recorded.