[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol (#7154)

[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol

Reviewed the proxy against Claude Code's published gateway contract. The
transport layer already held up; fourteen gaps sat one layer up, in the model
catalog and in the non-inference endpoints clients call.

Two of them cost money. The catalog carried no claude-opus-5 or
claude-sonnet-5, so an operator could not authorise the models coding agents
default to — those requests denied as not-routable, or priced at zero where a
catch-all carried them. And gateway records pin ParserID "openai" while the
same record serves /v1/messages, so Anthropic responses were read with the
OpenAI parser, which never looks at message_start where input tokens live:
input metered as roughly zero on every stream and cost was skipped entirely.

The rest fix requests refused for structural rather than policy reasons: model
discovery denied for every account with a model allowlist, token counting
denied on Bedrock and mis-parsed on Vertex, startup probes refused and written
into the access log at every session start, and denials rendered in a shape no
LLM client parses. Two changes are additive by design — the deny body keeps
every field it had and adds the vendor's error object alongside, and body-level
identity injection is now gated on the request's dialect so it stops sending
OpenAI-shape fields into Anthropic bodies that reject them.

The end-to-end work turned up one more: the discovery filter treated any slash
in a model id as a gateway prefix, which would have dropped every self-hosted
"Qwen/..." model from the picker.
This commit is contained in:
Maycon Santos
2026-08-23 20:02:33 +02:00
committed by GitHub
parent ee253feddf
commit 766fcae3f8
39 changed files with 3094 additions and 154 deletions

View File

@@ -46,6 +46,27 @@ func NormalizeBedrockModel(modelID string) string {
return bedrockVersionSuffix.ReplaceAllString(m, "")
}
// anthropicDatedModel matches a Claude model id carrying the trailing
// "-YYYYMMDD" release-date suffix Anthropic appends to a pinned release,
// capturing the id without it. The "claude" anchor is load-bearing: pricing
// looks every model up through this helper regardless of surface, and an
// operator may register a custom id with any shape at all, so an unanchored
// "-\d{8}$" would let "internal-llm-20250101" silently inherit the rate
// registered for "internal-llm". The anchor also covers the vendor-prefixed
// forms ("anthropic.claude-...", "us.anthropic.claude-...").
var anthropicDatedModel = regexp.MustCompile(`(?i)^(.*claude.*)-\d{8}$`)
// NormalizeAnthropicModel strips the trailing release-date suffix from a
// Claude model id, e.g. "claude-sonnet-4-5-20250929" -> "claude-sonnet-4-5",
// so a dated id a client pins matches the undated one the operator
// registered. Ids that are not Claude-family are returned untouched.
// Callers try the verbatim id first and fall back to this, so two dated
// releases of the same family stay distinct wherever both are registered
// explicitly.
func NormalizeAnthropicModel(modelID string) string {
return anthropicDatedModel.ReplaceAllString(modelID, "$1")
}
// NormalizeVertexModel strips the "@version" suffix from a Vertex AI model id
// (e.g. "claude-sonnet-4-5@20250929" -> "claude-sonnet-4-5") so it matches
// the catalog/pricing key. Vertex publisher models are priced under their

View File

@@ -34,3 +34,29 @@ func TestNormalizeVertexModel(t *testing.T) {
require.Equal(t, want, NormalizeVertexModel(in), "normalize %q", in)
}
}
func TestNormalizeAnthropicModel(t *testing.T) {
cases := map[string]string{
"claude-sonnet-4-5-20250929": "claude-sonnet-4-5",
"claude-3-5-haiku-20241022": "claude-3-5-haiku",
"claude-sonnet-5": "claude-sonnet-5",
"claude-opus-4-8": "claude-opus-4-8",
"anthropic.claude-haiku-4-5": "anthropic.claude-haiku-4-5",
"anthropic.claude-sonnet-4-5-20250929": "anthropic.claude-sonnet-4-5",
"us.anthropic.claude-opus-4-8-20250101": "us.anthropic.claude-opus-4-8",
// Non-Claude ids must survive untouched even when they end in eight
// consecutive digits: an operator can register a custom model under
// any id, and pricing looks every one of them up through this helper.
"gpt-4o": "gpt-4o",
"gpt-4o-2024-08-06": "gpt-4o-2024-08-06",
"gpt-4o-20240806": "gpt-4o-20240806",
"internal-llm-20250101": "internal-llm-20250101",
"deepseek-r1-20250120": "deepseek-r1-20250120",
"Qwen/Qwen2.5-20250101": "Qwen/Qwen2.5-20250101",
"gemini-2-5-pro-20250101": "gemini-2-5-pro-20250101",
"": "",
}
for in, want := range cases {
require.Equal(t, want, NormalizeAnthropicModel(in), "normalize %q", in)
}
}