mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-07 15:31:30 +02:00
proxy: model-specific deny message for model-allowlist rejections
denyFromManagement returned the generic "LLM policy limit exceeded" for every deny code. Map llm_policy.model_blocked / model_unknown to the same model-specific messages the local guardrail uses, keeping other codes generic so internal quota detail never leaks. Document the fail-open trade-off: the proxy backstop carries the union of restricting policies' models, so per-group narrowing is lost during a CheckLLMPolicyLimits outage.
This commit is contained in:
@@ -143,7 +143,13 @@ sequenceDiagram
|
||||
is genuinely unrestricted. `llm_guardrail` is a per-provider fail-closed
|
||||
backstop: it only carries an allowlist for a provider every authorising
|
||||
policy restricts, and blocks unknown/undetermined models even when
|
||||
management is unreachable.
|
||||
management is unreachable. Because that backstop allowlist is the UNION
|
||||
of every restricting policy's models, per-group narrowing lives only in
|
||||
the authoritative check: during a `CheckLLMPolicyLimits` outage
|
||||
`llm_limit_check` fails open, so a caller can reach any model in the
|
||||
provider's union — a group scoped to model A could reach model B if
|
||||
another group restricts the same provider to B. This is the documented
|
||||
fail-open trade-off; a future flag may switch it to fail-closed.
|
||||
- SSE streaming requires special handling on the response side; the
|
||||
parser must handle partial chunks without buffering the whole
|
||||
stream. See [`modules/32-proxy-llm-parsers.md`](modules/32-proxy-llm-parsers.md).
|
||||
|
||||
Reference in New Issue
Block a user