proxy: model-specific deny message for model-allowlist rejections

denyFromManagement returned the generic "LLM policy limit exceeded" for
every deny code. Map llm_policy.model_blocked / model_unknown to the same
model-specific messages the local guardrail uses, keeping other codes
generic so internal quota detail never leaks.

Document the fail-open trade-off: the proxy backstop carries the union of
restricting policies' models, so per-group narrowing is lost during a
CheckLLMPolicyLimits outage.
This commit is contained in:
mlsmaycon
2026-07-27 09:27:01 +00:00
parent a8664aaff3
commit 7cd1cda70e
3 changed files with 51 additions and 2 deletions

View File

@@ -143,7 +143,13 @@ sequenceDiagram
is genuinely unrestricted. `llm_guardrail` is a per-provider fail-closed
backstop: it only carries an allowlist for a provider every authorising
policy restricts, and blocks unknown/undetermined models even when
management is unreachable.
management is unreachable. Because that backstop allowlist is the UNION
of every restricting policy's models, per-group narrowing lives only in
the authoritative check: during a `CheckLLMPolicyLimits` outage
`llm_limit_check` fails open, so a caller can reach any model in the
provider's union — a group scoped to model A could reach model B if
another group restricts the same provider to B. This is the documented
fail-open trade-off; a future flag may switch it to fail-closed.
- SSE streaming requires special handling on the response side; the
parser must handle partial chunks without buffering the whole
stream. See [`modules/32-proxy-llm-parsers.md`](modules/32-proxy-llm-parsers.md).

View File

@@ -175,7 +175,7 @@ func denyFromManagement(resp *proto.CheckLLMPolicyLimitsResponse) *middleware.Ou
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Code: code,
Message: "LLM policy limit exceeded",
Message: denyMessageForCode(code),
},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMPolicyDecision, Value: "deny"},
@@ -184,6 +184,21 @@ func denyFromManagement(resp *proto.CheckLLMPolicyLimitsResponse) *middleware.Ou
}
}
// denyMessageForCode maps a management deny code to a public message.
// Model-allowlist rejections get a model-specific message matching the
// local guardrail; everything else keeps the generic quota wording. The
// message stays generic so it never leaks internal quota detail.
func denyMessageForCode(code string) string {
switch code {
case "llm_policy.model_blocked":
return "model is not in the policy allowlist"
case "llm_policy.model_unknown":
return "request model could not be determined for the policy allowlist"
default:
return "LLM policy limit exceeded"
}
}
// lookupKV returns the value associated with key, or the empty
// string when absent.
func lookupKV(kvs []middleware.KV, key string) string {

View File

@@ -115,6 +115,34 @@ func TestInvoke_DenyConvertsToProxyDeny(t *testing.T) {
assert.NotContains(t, out.DenyReason.Message, "1000", "internal cap numbers must not reach the caller")
}
// TestInvoke_ModelBlockedDenyMessage proves a model-allowlist rejection
// gets a model-specific public message rather than the generic quota
// wording, so a blocked model reads consistently with the local guardrail.
func TestInvoke_ModelBlockedDenyMessage(t *testing.T) {
mgmt := &fakeMgmt{
checkResp: &proto.CheckLLMPolicyLimitsResponse{
Decision: "deny",
DenyCode: "llm_policy.model_blocked",
},
}
m := New(mgmt, nil)
out := runInvoke(t, m, &middleware.Input{
AccountID: "acc-1",
UserGroups: []string{"grp-engineers"},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMResolvedProviderID, Value: "prov-1"},
{Key: middleware.KeyLLMModel, Value: "blocked-model"},
},
})
assert.Equal(t, middleware.DecisionDeny, out.Decision)
require.NotNil(t, out.DenyReason, "deny envelope must carry a reason payload")
assert.Equal(t, "llm_policy.model_blocked", out.DenyReason.Code, "canonical deny code surfaces to the caller")
assert.Equal(t, "model is not in the policy allowlist", out.DenyReason.Message,
"model denials must use a model-specific message, matching the local guardrail")
}
// TestInvoke_NoMgmtClientPassesThrough proves the partial-wiring
// safety: a middleware constructed without a management client
// allows every request without attribution. This makes a half-set-up