A popular r/LocalLLaMA post challenges the old advice that 64GB is a "dead zone" for local LLMs (too small for a 70B with real context, more than a 30B needs). The author has been running Qwen3.8 27B for agentic coding on a 64GB M4 Max for ten days.
Key points from the discussion:
- Setup: Qwen3.8 27B as a roughly 16 GB MLX quant in LM Studio, 256k context, with coding agent sessions sitting at 100k to 200k tokens.
- At about 164k tokens, LM Studio's engine uses around 48 GB: roughly 16 GB for weights, about 11 GB of KV cache, plus a second copy of the context for prompt caching and leftover prefill scratch memory.
- The prompt cache is what makes it usable: with about a 93% hit rate, each turn only processes around 1k new tokens. A full cache miss means a prefill of about 20 minutes.
- KV cache quantization wasn't a good option in that setup, so on 48GB it would only fit with shorter context (around 128k) or by moving to llama.cpp with q8 KV.
- The author's conclusion: with agents, context costs about as much as the model weights, so 64GB fits a capable 27B with long context. Bigger MoE models or running two models at once still want 128GB.
Our practical notes:
1. When sizing hardware for agent workloads, budget memory for weights plus KV cache at your target context length, not weights alone.
2. Prompt caching matters as much as raw speed. Keep the system prompt and tool definitions stable so the cache keeps hitting.
3. If you're on a smaller machine, cap context (for example 64k to 128k) and let the agent summarize or compact older history.
4. Before buying, test your real workflow with a cloud rental or a friend's machine. Agent sessions behave very differently from short chats.
Are you running coding agents locally? How much memory do you have, and what context length works for you in practice? Share your setup below.