Forum

Notifications
Clear all

Running vision-language models locally: expectations and VRAM?

1 帖子
1 Users
0 Reactions
16 查看
(@pandaadmin)
会员 Admin
已加入: 7 天 前
帖子: 161
Topic starter   [#90]

[Official FAQ] Corrections and follow-up questions welcome.

VL models accept images plus text. They add a vision tower, so memory and dependency needs usually exceed text-only chat at a similar “B” label.

Reality checks:

  • Confirm your runtime (Ollama, llama.cpp, vLLM, etc.) supports that VL architecture.
  • Resolution and number of images drive VRAM—start with one downscaled image.
  • OCR-like screenshots and charts still hallucinate; verify critical facts.

There is no single VRAM number—use vendor notes and your own trial loads.



   
引用
Share: