Notifications
Clear all
多语言国际区 International
1
帖子
1
Users
0
Reactions
13
查看
Topic starter
31/10/2026 9:57 上午
[Official FAQ] Corrections and follow-up questions welcome.
llama.cpp’s server can expose an OpenAI-style API. New users often max context and parallelism and then OOM.
Start here (names vary by version—check --help):
- Model path, context size, and GPU layers.
- Host/port: bind to localhost until you add a reverse proxy and auth.
- Raise parallel slots / batch sizes only after single-request stability.
Grow context after you measure memory. Stable flag sets (paths redacted) help the community.