# 16 GB VRAM residency budgeting > Source-anchored research for 16 GB VRAM residency budgeting. Run status: COMPLETED. Parent: [Blackwell sm_120 local-LLM inference stack on Linux: llama.cpp, Ollama, PyTorch and vLLM over a Thunderbolt eGPU](https://llms-explorer.com/tree/blackwell-sm120-llm-inference-stack/) · 3 facets · 7 facts · page: https://llms-explorer.com/tree/16-gb-vram-residency-budgeting/ ## Structure and components - For Qwen2-14B with 48 layers, 8 KV heads, and 128 head dimension at FP16, KV cache consumes exactly 196.6 KB per token, or 3.14 GB for a 16,384-token context window. (high confidence) — [source](https://arxiv.org/abs/2305.13245) ## Measurements and reference values - On an RTX 5080 with 16 GB GDDR7 VRAM, a 14B Q4 model reserves ~7.62 GB for weights, leaving ~8.38 GB for KV cache, which comfortably accommodates 16,384 tokens with standard FP16 KV cache or 32,768 tokens with FP8 KV cache. (high confidence) — [source](https://github.com/vllm-project/vllm) - For 7B/8B models at 4-bit precision (~4.5 GB weights), remaining VRAM exceeds 11.5 GB, allowing context windows to expand up to 65,536 tokens without host offloading. (high confidence) — [source](https://github.com/vllm-project/vllm) - FP8 KV cache quantization reduces per-token KV memory by 50% down to 98.3 KB per token, permitting up to 32,768 context tokens in 3.14 GB on 16 GB accelerators. (medium confidence) — [source](https://github.com/vllm-project/vllm) - A 14-billion parameter model: FP16 - 28.0 GB, FP8/INT8 - 14.0 GB, 4-bit (Q4_K_M) - 7.62 GB (high confidence) — [source](https://github.com/ggerganov/llama.cpp) - On 16 GB RTX 5080: Q4_K_M leaves 8.38 GB for KV cache and scratch (high confidence) — [source](https://github.com/ggerganov/llama.cpp) ## Facts and statements - In GQA architectures such as Qwen2-14B and Llama-3.1-8B, KV cache memory footprint scales as 2 * num_layers * num_kv_heads * head_dim * precision_bytes per context token. (high confidence) — [source](https://arxiv.org/abs/2305.13245)