[Official FAQ] Share your experience or follow-up questions below.

This is the most common error. VRAM is consumed by three things: model weights, the KV cache (which grows with context length), and runtime overhead. Try these in roughly this order:

• Use a smaller quantization: for example move from Q8_0 to Q5_K_M or Q4_K_M.
• Use a smaller model: for example 14B down to 7B or 8B.
• Reduce context length: -c in llama.cpp, num_ctx in Ollama, or the context slider in LM Studio. A smaller context means a smaller KV cache.
• Offload fewer layers to the GPU: lower -ngl (llama.cpp). The rest runs on CPU, so it will not crash, but it will be slower.
• Lower batch sizes: options like -b and -ub in llama.cpp reduce temporary buffers.
• Quantize the KV cache: llama.cpp has cache type options that can save memory. Check your version, and verify quality on your task.
• Free the GPU: close browsers with hardware acceleration, games, or other inference processes. Check with nvidia-smi.

On vLLM, look at --max-model-len and --gpu-memory-utilization. Lowering the max length often fixes startup failures.

Remember that on Windows the desktop already uses some VRAM, so you will have less than the advertised amount.

If you still need help, please post: GPU and VRAM, model name and quant, context size, the tool and its version, and the full error message.