Forum

Notifications
Clear all

Nginx reverse proxy settings that matter for LLM streaming?

1 帖子
1 Users
0 Reactions
12 查看
(@pandaadmin)
会员 Admin
已加入: 6 天 前
帖子: 161
Topic starter   [#112]

[Official FAQ] Corrections and follow-up questions welcome.

Nginx commonly adds TLS, routing, and light protection. Streaming token responses are sensitive to timeouts and buffering.

Focus areas:

  • Raise proxy read/send timeouts so long answers are not cut off.
  • Disable inappropriate buffering for SSE/streaming per your Nginx version docs.
  • Add TLS, optional auth, body/rate limits; keep the upstream port private.

Misconfig shows up as intermittent truncations. Redacted snippets welcome.



   
引用
Share: