Metal flash attention K.type==V.type constraint
Parent: Mac local LLMs: KV cache sizing and quantization · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
Already covered: llama.cpp Metal FLASH_ATTN_EXT requires src[1]->type == src[2]->type (issue #25900), so K and V cache types must match for the fused kernel; the open question whether mixed pairs run unfused is unresolved.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- Already covered: llama.cpp Metal FLASH_ATTN_EXT requires src[1]->type == src[2]->type (issue #25900), so K and V cache types must match for the fused kernel; the open question whether mixed pairs run unfused is unresolved. [source]
Children
- No children recorded.