LLM VRAM calculator
Estimate GPU memory for a model file, context length and simultaneous requests.
Context and concurrency
Will this model fit on my GPU?
Choose your model and GPU, then adjust context and concurrency.
llama.cpp · Text inference · One GPU · FP16 KV cache
Model coverage: 23 verified profiles
The model menu contains every profile this GPU-fit checker can calculate. The general hardware estimator has a different model list and uses broader planning assumptions.
Missing your model? A result for a similarly sized model is not a substitute. This checker needs the exact GGUF file size, attention configuration and context limit before it can estimate memory.
Gemma, gpt-oss, Qwen3.8, Mistral Small 4 and full DeepSeek R1 have no verified profile here. The general VRAM reference includes broader weight-based estimates; it does not establish context-specific fit.
CPU offload, multiple GPUs, hybrid or sliding-window attention and multimodal inputs are outside this checker. See measured Qwen3 4B memory on a GTX 1660 Ti and NVIDIA T4.
Calculated estimate. No matching memory test is published for these inputs.
Within your chosen memory budget
6.81 GiBestimated demand / 21.60 GiB budget10% reserved from 24 GiB of GPU memory
- Model file / weight allowance
- 4.68 GiB
- FP16 K/V cache
- 1.13 GiB
- Runtime allowance
- 1.00 GiB
- Remaining in budget
- 14.79 GiB
Estimated context ceiling: 40,960 tokens per conversation at 1 concurrent conversation.
Estimated memory fit. Verify the actual allocation in your runtime before relying on this configuration.
Choose hardware for this scenario
RTX 4090 meets this memory estimate with your 10% reserve.
Starting parts: Ryzen 9 9950X, 128 GB RAM, ASUS ROG Crosshair X670E Hero. Estimated PSU target: 1100 W, with 30% power headroom.
Open RTX 4090 parts in Builder →Memory fit does not establish runtime support or speed. Builder checks the parts list; it does not carry out a model test or size host RAM, storage, cooling, or the case for this workload.
Other GPUs within this memory budget
First three catalog matches by memory capacity. This is not a price or performance ranking.
- RTX 507012 GiB · 3.99 GiB remaining in budget
- RTX 4070 SUPER12 GiB · 3.99 GiB remaining in budget
- ASUS Dual RTX 3060 12GB V2 OC12 GiB · 3.99 GiB remaining in budget
Sources, calculation, and limits
All figures use GiB (2³⁰ bytes). Model allowance uses the published GGUF file size as a memory proxy; loaded GPU allocations can differ. Cache = 2 × layers × KV heads × head dimension × tokens × conversations × 2 bytes.
23 supported profiles across Qwen, Llama, Mistral and DeepSeek. Original profiles were checked October 7–8; Qwen2.5 Coder and DeepSeek R1 Distill Llama 8B were checked October 13, 2026. Each model uses its own layer count, KV heads, head dimension and context cap. Llama 3.1 and NeMo include their published native extended context settings; user-added context extension is outside this calculation.
The menu lists every supported model and format. Gemma, full DeepSeek R1, hybrid attention, sliding-window attention, multimodal input, CPU offload and multiple GPUs are not supported by this planner. DeepSeek support here covers the listed dense distilled models only.
Selected Q4_K_M artifact: Published file
Qwen2.5 Coder 7B Instruct: architecture config · model card · quantized files — Architecture and GGUF files published by Qwen. The 32,768-token cap excludes user-added YaRN extension. Checked October 13, 2026.
Qwen2.5 Coder 14B Instruct: architecture config · model card · quantized files — Architecture and GGUF files published by Qwen. The 32,768-token cap excludes user-added YaRN extension. Checked October 13, 2026.
DeepSeek R1 Distill Llama 8B: architecture config · model card · quantized files — Architecture published by DeepSeek; GGUF files published by bartowski. Checked October 13, 2026.
Qwen3 0.6B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Qwen3 1.7B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Qwen3 4B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Qwen3 8B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Qwen3 14B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Qwen3 32B: architecture config · model card · quantized files — Architecture and quantized files published by Qwen.
Llama 3.1 8B Instruct: architecture config · model card · quantized files — Architecture config published by Unsloth for the Meta model; GGUF files published by bartowski.
Llama 3.1 70B Instruct: architecture config · model card · quantized files — Architecture config published by Unsloth for the Meta model; GGUF files published by bartowski.
Mistral 7B Instruct v0.3: architecture config · model card · quantized files — Architecture config published by the model author; GGUF files published by bartowski.
Mistral NeMo 12B Instruct: architecture config · model card · quantized files — Architecture config published by the model author; GGUF files published by bartowski.
DeepSeek R1 Distill Qwen 7B: architecture config · model card · quantized files — Architecture config published by the model author; GGUF files published by bartowski.
DeepSeek R1 Distill Qwen 14B: architecture config · model card · quantized files — Architecture config published by the model author; GGUF files published by bartowski.
DeepSeek R1 Distill Qwen 32B: architecture config · model card · quantized files — Architecture config published by the model author; GGUF files published by bartowski.
Qwen2.5 7B Instruct: architecture config · model card · quantized files — Architecture published by Qwen; GGUF files published by Qwen. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
Qwen2.5 14B Instruct: architecture config · model card · quantized files — Architecture published by Qwen; GGUF files published by Qwen. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
Qwen2.5 32B Instruct: architecture config · model card · quantized files — Architecture published by Qwen; GGUF files published by Qwen. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
Qwen3 4B Instruct 2507: architecture config · model card · quantized files — Architecture published by Qwen; GGUF files published by unsloth. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
Qwen3 30B A3B Instruct 2507: architecture config · model card · quantized files — Architecture published by Qwen; GGUF files published by unsloth. Source revisions and file hashes checked October 26, 2026. All experts remain in the resident-weight estimate; active parameters do not determine stored weights.
Granite 3.3 2b instruct: architecture config · model card · quantized files — Architecture published by ibm-granite; GGUF files published by ibm-granite. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
Granite 3.3 8b instruct: architecture config · model card · quantized files — Architecture published by ibm-granite; GGUF files published by ibm-granite. Source revisions and file hashes checked October 26, 2026. Full-attention FP16 cache estimate; runtime buffers remain an explicit allowance.
llama.cpp context, parallel slots, and cache-type documentation
Compare real file choices before changing hardware
Calculated for llama.cpp text inference on one GPU, with FP16 KV cache, a 1 GiB runtime allowance and 10% of GPU memory reserved. The allowance is adjustable; these are not measured runtime peaks or speed results.
| Model / format | GPU memory | Tokens / conversation | Concurrent conversations | Required / usable GiB | Result and scenario |
|---|---|---|---|---|---|
| Qwen3 4B Q4_K_M | Other 8 GiB GPU (memory budget only) | 8,192 | 1 | 4.45 / 7.20 | Within budget Reproduce these inputs |
| Qwen2.5 7B Instruct Q4_K_M | ASUS Dual RTX 3060 12GB V2 OC | 8,192 | 1 | 5.80 / 10.80 | Within budget Reproduce these inputs |
| Qwen3 8B Q4_K_M | Other 8 GiB GPU (memory budget only) | 8,192 | 1 | 6.81 / 7.20 | Within budget Reproduce these inputs |
| Qwen3 8B Q4_K_M | Other 8 GiB GPU (memory budget only) | 32,768 | 1 | 10.18 / 7.20 | Exceeds memory Reproduce these inputs |
| Qwen3 14B Q4_K_M | RTX 3090 Founders Edition | 8,192 | 1 | 10.63 / 21.60 | Within budget Reproduce these inputs |
| Qwen3 14B Q8_0 | RTX 3090 Founders Edition | 8,192 | 1 | 16.87 / 21.60 | Within budget Reproduce these inputs |
What the result includes
Required memory equals published GGUF file bytes, the full FP16 key/value cache for each conversation, and your chosen runtime allowance. Usable memory is GPU capacity after your reserve. GGUF bytes are a weight-allocation proxy; runtime allocation can differ.
Shortening context or reducing simultaneous requests reduces cache memory. Weight quantization changes file size; it does not automatically change KV-cache precision. Mixture-of-experts estimates include the full published file, not only active parameters.
What needs a runtime test
Prefill, display use, drivers and runtime buffers can exceed the allowance. This tool does not certify speed, CPU offload, multi-GPU splitting, training, multimodal input or unsupported attention architectures.
Read our measured Qwen3 4B context-memory results · Compare broader VRAM planning targets · Estimate a complete hardware configuration