[proxy] Strip only a gateway's own namespace before matching a model

Three open review findings.

The discovery filter treated every slash in a listed id as a gateway prefix
and matched the tail against the policy. A self-hosted id carries slashes of
its own, and an upstream may scope ids per tenant, so "tenant-b/claude-sonnet-5"
matched a permitted "claude-sonnet-5" and reached the picker — a model the
policy never named, and one the guardrail denies on sight, since enforcement
compares the id as written. Strip only the namespaces a gateway is known to
prepend, taken from the first slash rather than the last.

The e2e retry loops slept between attempts without watching the context, so a
cancelled run kept retrying calls that fail instantly and spent its remaining
window sleeping between them. They now stop when the context is done.

The streamed provider's setup key outlived its test: deleting the group does
not delete the key that auto-joins it.
This commit is contained in:
mlsmaycon
2026-08-23 06:35:27 +00:00
parent a02aa16cae
commit acfad1a384
5 changed files with 80 additions and 11 deletions

View File

@@ -152,19 +152,43 @@ func entryPermitted(entry map[string]json.RawMessage, permitted map[string]struc
return false
}
// gatewayNamespaces are the provider prefixes a gateway prepends to a model
// it re-exports: LiteLLM lists a Bedrock model the operator registered as
// "anthropic.claude-opus-5" under "bedrock/anthropic.claude-opus-5". Only
// these are stripped before matching.
//
// A slash is not by itself a namespace separator. Self-hosted backends ship
// ids that carry one ("Qwen/Qwen2.5-0.5B-Instruct"), and an upstream is free
// to scope ids per tenant ("tenant-b/claude-sonnet-5"). Treating every slash
// as a prefix let any such id match an allowed model by its tail, so the
// picker offered models the policy never named.
var gatewayNamespaces = map[string]struct{}{
"anthropic": {},
"azure": {},
"bedrock": {},
"mistral": {},
"openai": {},
"vertex_ai": {},
}
// modelIDForms returns the forms a single model id may be written in: the id
// itself, its undated form, and the same two with a gateway's provider
// prefix removed ("vertex_ai/claude-sonnet-5"). The bare id is tried first,
// because a self-hosted id can legitimately contain a slash of its own
// ("Qwen/Qwen2.5-0.5B-Instruct") and must not be cut down to its tail.
// itself, its undated form, and — when the id is namespaced by a gateway we
// recognise — the same two with that namespace removed
// ("vertex_ai/claude-sonnet-5"). The bare id is always tried first.
//
// The namespace is what precedes the FIRST slash: it is a prefix the gateway
// put in front of the whole id, and everything after it is the id the
// operator would have registered, separators included.
func modelIDForms(id string) []string {
if id == "" {
return nil
}
forms := []string{id, sharedllm.NormalizeAnthropicModel(id)}
if slash := strings.LastIndex(id, "/"); slash >= 0 {
tail := id[slash+1:]
forms = append(forms, tail, sharedllm.NormalizeAnthropicModel(tail))
if slash := strings.Index(id, "/"); slash > 0 {
if _, ok := gatewayNamespaces[id[:slash]]; ok {
tail := id[slash+1:]
forms = append(forms, tail, sharedllm.NormalizeAnthropicModel(tail))
}
}
return forms
}

View File

@@ -171,6 +171,24 @@ func TestModelDiscoveryFilter_KeepsSlashBearingIDs(t *testing.T) {
"a slash inside the model id is part of the id, not a provider prefix")
}
// TestModelDiscoveryFilter_RejectsTailMatchOnUnknownNamespace covers the id
// an upstream scopes with a prefix of its own. "tenant-b/claude-sonnet-5"
// ends in a model the policy permits, but it is a different model on a
// different tenant, and the guardrail denies that string outright — so
// offering it hands the picker an entry the next request refuses.
func TestModelDiscoveryFilter_RejectsTailMatchOnUnknownNamespace(t *testing.T) {
ids := listedIDs(t, []string{"claude-sonnet-5"}, `{
"data": [
{"id": "claude-sonnet-5"},
{"id": "tenant-b/claude-sonnet-5"},
{"id": "Qwen/claude-sonnet-5"}
]
}`)
assert.Equal(t, []string{"claude-sonnet-5"}, ids,
"only a namespace a gateway is known to prepend may be stripped before matching")
}
// TestModelDiscoveryFilter_ForwardsOversizedBodyIntact covers a listing past
// the buffering cap. The filter reads one byte beyond the cap to detect the
// size; forwarding only what it read would hand the client a body truncated