mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-02 04:51:29 +02:00
[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol (#7154)
[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol Reviewed the proxy against Claude Code's published gateway contract. The transport layer already held up; fourteen gaps sat one layer up, in the model catalog and in the non-inference endpoints clients call. Two of them cost money. The catalog carried no claude-opus-5 or claude-sonnet-5, so an operator could not authorise the models coding agents default to — those requests denied as not-routable, or priced at zero where a catch-all carried them. And gateway records pin ParserID "openai" while the same record serves /v1/messages, so Anthropic responses were read with the OpenAI parser, which never looks at message_start where input tokens live: input metered as roughly zero on every stream and cost was skipped entirely. The rest fix requests refused for structural rather than policy reasons: model discovery denied for every account with a model allowlist, token counting denied on Bedrock and mis-parsed on Vertex, startup probes refused and written into the access log at every session start, and denials rendered in a shape no LLM client parses. Two changes are additive by design — the deny body keeps every field it had and adds the vendor's error object alongside, and body-level identity injection is now gated on the request's dialect so it stops sending OpenAI-shape fields into Anthropic bodies that reject them. The end-to-end work turned up one more: the discovery filter treated any slash in a model id as a gateway prefix, which would have dropped every self-hosted "Qwen/..." model from the picker.
This commit is contained in:
@@ -46,6 +46,27 @@ func NormalizeBedrockModel(modelID string) string {
|
||||
return bedrockVersionSuffix.ReplaceAllString(m, "")
|
||||
}
|
||||
|
||||
// anthropicDatedModel matches a Claude model id carrying the trailing
|
||||
// "-YYYYMMDD" release-date suffix Anthropic appends to a pinned release,
|
||||
// capturing the id without it. The "claude" anchor is load-bearing: pricing
|
||||
// looks every model up through this helper regardless of surface, and an
|
||||
// operator may register a custom id with any shape at all, so an unanchored
|
||||
// "-\d{8}$" would let "internal-llm-20250101" silently inherit the rate
|
||||
// registered for "internal-llm". The anchor also covers the vendor-prefixed
|
||||
// forms ("anthropic.claude-...", "us.anthropic.claude-...").
|
||||
var anthropicDatedModel = regexp.MustCompile(`(?i)^(.*claude.*)-\d{8}$`)
|
||||
|
||||
// NormalizeAnthropicModel strips the trailing release-date suffix from a
|
||||
// Claude model id, e.g. "claude-sonnet-4-5-20250929" -> "claude-sonnet-4-5",
|
||||
// so a dated id a client pins matches the undated one the operator
|
||||
// registered. Ids that are not Claude-family are returned untouched.
|
||||
// Callers try the verbatim id first and fall back to this, so two dated
|
||||
// releases of the same family stay distinct wherever both are registered
|
||||
// explicitly.
|
||||
func NormalizeAnthropicModel(modelID string) string {
|
||||
return anthropicDatedModel.ReplaceAllString(modelID, "$1")
|
||||
}
|
||||
|
||||
// NormalizeVertexModel strips the "@version" suffix from a Vertex AI model id
|
||||
// (e.g. "claude-sonnet-4-5@20250929" -> "claude-sonnet-4-5") so it matches
|
||||
// the catalog/pricing key. Vertex publisher models are priced under their
|
||||
|
||||
@@ -34,3 +34,29 @@ func TestNormalizeVertexModel(t *testing.T) {
|
||||
require.Equal(t, want, NormalizeVertexModel(in), "normalize %q", in)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeAnthropicModel(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"claude-sonnet-4-5-20250929": "claude-sonnet-4-5",
|
||||
"claude-3-5-haiku-20241022": "claude-3-5-haiku",
|
||||
"claude-sonnet-5": "claude-sonnet-5",
|
||||
"claude-opus-4-8": "claude-opus-4-8",
|
||||
"anthropic.claude-haiku-4-5": "anthropic.claude-haiku-4-5",
|
||||
"anthropic.claude-sonnet-4-5-20250929": "anthropic.claude-sonnet-4-5",
|
||||
"us.anthropic.claude-opus-4-8-20250101": "us.anthropic.claude-opus-4-8",
|
||||
// Non-Claude ids must survive untouched even when they end in eight
|
||||
// consecutive digits: an operator can register a custom model under
|
||||
// any id, and pricing looks every one of them up through this helper.
|
||||
"gpt-4o": "gpt-4o",
|
||||
"gpt-4o-2024-08-06": "gpt-4o-2024-08-06",
|
||||
"gpt-4o-20240806": "gpt-4o-20240806",
|
||||
"internal-llm-20250101": "internal-llm-20250101",
|
||||
"deepseek-r1-20250120": "deepseek-r1-20250120",
|
||||
"Qwen/Qwen2.5-20250101": "Qwen/Qwen2.5-20250101",
|
||||
"gemini-2-5-pro-20250101": "gemini-2-5-pro-20250101",
|
||||
"": "",
|
||||
}
|
||||
for in, want := range cases {
|
||||
require.Equal(t, want, NormalizeAnthropicModel(in), "normalize %q", in)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user