Local LLM VRAM Planning 2026: The KV Cache Math Behind the Context You Can Actually Serve
A model card says 40,960 tokens. The Q4_K_M file is 4.68 GiB. The card has 12 GB. You set -c 40960, and one of three things happens: the load fails, the runtime spills layers to CPU and tokens per second collapse, or the server starts and silently serves a shorter context than you asked for. Which of the three you get depends on a number that appears in none of those three places — the key/value cache the runtime allocates at load time from the context you request. This page computes it for four specific models from their own config.json at a pinned revision, turns it into a “largest context that fits this card” table, and ships a planner you can run offline. ...