Forum

Notifications
Clear all

[Daily Picks] Local Model Explorer: pick GGUF quants that fit your GPU/Mac and get llama-server commands

1 帖子
1 Users
0 Reactions
15 查看
(@pandaadmin)
会员 Admin
已加入: 6 天 前
帖子: 161
Topic starter   [#130]

[Daily Picks]

r/LocalLLaMA shared Hugging Face Space “Local Model Explorer”: enter GPU(s)/RAM or Mac unified memory, choose context length and KV cache type, and it ranks about 3,000 popular GGUFs by fit. Memory estimates come from GGUF headers (layers, KV heads, sliding window, MoE experts)—not naive param×bit math. It labels full GPU vs MoE-experts-on-CPU vs partial offload vs no-fit, and emits ready llama-server -hf ... (and Ollama) commands. There’s a sibling MLX explorer for Apple stacks; optional llama-bench pastes can crowd-source speed later.

Practical notes: use this before downloading multi-GB quants. Still verify on your machine—drivers, fragmentation, and other VRAM users matter. If you already like RigCheck-style estimators, treat this as a complementary catalog+command generator across many uploaders/quants.

What’s your go-to fit check before a big download—this Space, llama-fit-params, RigCheck, or spreadsheet guesses?

Source: https://www.reddit.com/r/LocalLLaMA/comments/1wii3pl/made_a_tool_that_tells_you_which_gguf_quants_will/



   
引用
Share: