mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-23 07:09:08 +02:00
[proxy] Exempt non-inference endpoints from the model allowlist gate
GET /v1/models carries no model, and management's per-model allowlist fails closed on an undetermined one, so gateway model discovery denied with model_blocked for every account that enables a model allowlist. The client treats a failed discovery as silent and falls back to its built-in list, so the operator sees an empty picker with no error to chase. The router already classifies these paths and authorises the route against the caller's groups before allowing them, so mark them non-inference there and let the limits gate skip a pre-flight that has no model to evaluate and no tokens to book. The marker comes from the router's own path classification, never from client input, so an inference request cannot set it to escape the allowlist.
This commit is contained in:
@@ -66,6 +66,14 @@ const (
|
||||
// downstream gateways' spend logs.
|
||||
KeyLLMAuthorisingGroups = "llm.authorising_groups"
|
||||
|
||||
// LLM non-inference marker (emitted by llm_router on the allow path
|
||||
// for endpoints that legitimately carry no model, such as model
|
||||
// listing). The router still authorises these against the caller's
|
||||
// groups; the marker only tells the limits gate that a per-model
|
||||
// allowlist has nothing to evaluate, so an empty model must not be
|
||||
// read as an undetermined one. Never derived from client input.
|
||||
KeyLLMNonInference = "llm.non_inference"
|
||||
|
||||
// LLM policy attribution (emitted by llm_limit_check on the allow
|
||||
// path). Names the policy that paid for this request and the
|
||||
// dimension counters the post-flight llm_limit_record middleware
|
||||
|
||||
Reference in New Issue
Block a user