Computing & Software
Transformer KV Cache Memory Calculator
Estimate key/value cache payload from layer count, KV heads, context length and cache format.
How the calculation works
Key plus value cache payload = 2*layers*kv_heads*head_dim*tokens*batch*bytes_per_value/1073741824 GiB. Cache payload per token per sequence = 2*layers*kv_heads*head_dim*bytes_per_value bytes.
Worked example
Example inputs — Transformer layers: 32; KV heads per layer (not necessarily query heads): 8; Features per KV head: 128; Stored context tokens: 4096; Independent sequences in cache: 1; Effective bytes per cached value: 2. Expected results — Key plus value cache payload: 0.5 GiB; Cache payload per token per sequence: 131072 bytes.
Assumptions and limits
Analytical software/hardware scenario, not a benchmark or a live provider quote. Use the actual architecture, workload and billing definitions; overheads not named in the model are excluded. Counts keys and values only. Grouped-query attention uses KV heads, not query heads. Model weights, quantization metadata, paged-cache fragmentation, activations and engine overhead are excluded.
Reference and further reading
Read the underlying subject reference. Calculith is an independent implementation; the reference does not endorse or certify this tool.
Related calculators
- Password Entropy & Brute-Force Time Estimate
- CPU Instruction Count & CPI Runtime Calculator
- Amdahl & Gustafson Parallel Speedup Calculator
- Roofline Arithmetic Intensity Performance Calculator
- Memory Bus Theoretical Bandwidth Calculator
Example values are editable. Results are estimates where assumptions apply. Calculations run in your browser. Updated 2026-11-16.