[proxy,management] Serve Bedrock model discovery from the control plane (#7250)

[proxy,management] Serve Bedrock model discovery from the control plane

A Bedrock provider could never answer a model-discovery request. The router
sent GET /inference-profiles to the record's upstream, which has to be
bedrock-runtime.<region> for InvokeModel to work, and that host does not
implement the operation. ListInferenceProfiles is a control-plane operation on
bedrock.<region>.amazonaws.com, and one provider record carries one upstream,
so the two hosts genuinely differ.

The route now carries a discovery host, taken from the catalog's declaration
with the region read back out of the configured upstream, and the listing — and
only the listing — goes there. Inference is untouched. A proxied or self-hosted
Bedrock endpoint gets no discovery host at all rather than a guessed one, since
inventing a host would send the operator's credential somewhere they never
configured.

Two things had to follow for the listing to be usable once it arrives. The
response filter only understood OpenAI's {"data":[{"id":…}]}, so a Bedrock
listing fell through it untouched, offering every profile in the account
whatever the policy said. And discoverableModels intersected by exact string,
so a record registering the raw profile id while a guardrail names the catalog
key intersected to nothing — bounding a working provider's listing down to
empty.

Normalisation is the third. The geography in front of a cross-region profile
was matched against a hardcoded list of four, so every profile issued under jp,
au, ca, sa or us-gov carried its prefix into the pricing key, matched no
catalog entry and metered at zero. It is now recognised by either the geography
or the vendor that follows it, so an id has to be new on both axes at once to
slip through — a live eu-central-1 listing returned "global.xai.grok-4.6" days
after the vendor list was first written.

shared/llm/model.go

@@ -10,9 +10,88 @@ import (
 	"strings"
 )
 
-// bedrockRegionPrefixes are the cross-region inference-profile prefixes that
-// front a Bedrock model id (e.g. "eu.anthropic.claude-...").
-var bedrockRegionPrefixes = []string{"us.", "eu.", "apac.", "global."}
+// bedrockVendorNamespaces are the vendor segments a Bedrock model id is
+// published under. They identify the geography in front of a cross-region
+// inference profile without knowing the geography: in
+// "eu.anthropic.claude-...", what makes "eu" a geography is that "anthropic"
+// follows it.
+//
+// A vendor missing from here is not fatal — bedrockGeographies covers the
+// same id from the other side — but it is one of the two ways an id can go
+// unrecognised, and the list needs a new entry whenever AWS onboards a
+// vendor. A live listing found "global.xai.grok-4.6" days after this was
+// first written.
+var bedrockVendorNamespaces = map[string]struct{}{
+	"ai21":       {},
+	"amazon":     {},
+	"anthropic":  {},
+	"cohere":     {},
+	"deepseek":   {},
+	"luma":       {},
+	"meta":       {},
+	"mistral":    {},
+	"openai":     {},
+	"qwen":       {},
+	"stability":  {},
+	"twelvelabs": {},
+	"writer":     {},
+	"xai":        {},
+}
+
+// bedrockGeographies are the geography segments AWS issues cross-region
+// inference profiles under. They recognise a profile whose vendor we have
+// never seen, which is the case bedrockVendorNamespaces alone gets wrong:
+// "global.xai.grok-4.6" is a geography and a model whether or not "xai" is
+// a name we know.
+//
+// Neither list is sufficient alone. A geography list on its own is what this
+// file started with, and it aged badly — it held us, eu, apac and global, so
+// every profile issued under jp, au, ca, sa or us-gov carried its prefix into
+// the pricing key, matched no catalog entry, and reported the model unpriced.
+// A vendor list on its own misses a new vendor under a known geography.
+// Together, an id has to be new on both axes at once to go unrecognised.
+var bedrockGeographies = map[string]struct{}{
+	"apac":   {},
+	"au":     {},
+	"ca":     {},
+	"eu":     {},
+	"global": {},
+	"jp":     {},
+	"sa":     {},
+	"us":     {},
+	"us-gov": {},
+}
+
+// stripBedrockGeography removes the cross-region inference-profile geography
+// from a Bedrock model id, leaving the "<vendor>.<model>" form the catalog and
+// the pricing table key on.
+//
+// A leading segment counts as a geography when it is one we know, or when a
+// known vendor follows it. Either alone is enough: the id has to be new on
+// both axes before its geography survives.
+//
+// The segment has to be followed by two more, so "amazon.nova-pro" stays a
+// vendor and a model rather than becoming a geography and a model — cutting
+// its first segment would strip the vendor away. Over-stripping is the
+// dangerous direction, because the result also decides which route may claim
+// a model.
+func stripBedrockGeography(modelID string) string {
+	geo, rest, found := strings.Cut(modelID, ".")
+	if !found || geo == "" {
+		return modelID
+	}
+	vendor, _, found := strings.Cut(rest, ".")
+	if !found {
+		return modelID
+	}
+	if _, ok := bedrockGeographies[geo]; ok {
+		return rest
+	}
+	if _, ok := bedrockVendorNamespaces[vendor]; ok {
+		return rest
+	}
+	return modelID
+}
 
 // bedrockVersionSuffix matches the trailing "-vN[:N]" or "-YYYYMMDD-vN[:N]"
 // version/throughput suffix of a Bedrock model id.
@@ -37,12 +116,7 @@ func NormalizeBedrockModel(modelID string) string {
 			m = m[i+1:]
 		}
 	}
-	for _, p := range bedrockRegionPrefixes {
-		if strings.HasPrefix(m, p) {
-			m = m[len(p):]
-			break
-		}
-	}
+	m = stripBedrockGeography(m)
 	return bedrockVersionSuffix.ReplaceAllString(m, "")
 }
 

shared/llm/model_test.go

@@ -60,3 +60,61 @@ func TestNormalizeAnthropicModel(t *testing.T) {
 		require.Equal(t, want, NormalizeAnthropicModel(in), "normalize %q", in)
 	}
 }
+
+// TestNormalizeBedrockModel_GeographiesBeyondTheOriginalFour covers the bug
+// that made this vendor-anchored: the geography used to be matched against a
+// list of four, so a profile issued anywhere else kept its prefix, missed the
+// catalog key it was supposed to match, and reported the model unpriced.
+func TestNormalizeBedrockModel_GeographiesBeyondTheOriginalFour(t *testing.T) {
+	for _, geo := range []string{"us", "eu", "apac", "global", "jp", "au", "ca", "sa", "us-gov", "il", "mx"} {
+		t.Run(geo, func(t *testing.T) {
+			got := NormalizeBedrockModel(geo + ".anthropic.claude-sonnet-5-20260514-v1:0")
+			require.Equal(t, "anthropic.claude-sonnet-5", got,
+				"a cross-region profile must reduce to the catalog key whatever geography issued it")
+		})
+	}
+}
+
+// TestNormalizeBedrockModel_KeepsAVendorItCannotMistakeForAGeography pins the
+// direction that must never break: a plain "<vendor>.<model>" id has no
+// geography, and cutting its first segment would strip the vendor away and
+// hand the id to whichever route claims the bare model name.
+func TestNormalizeBedrockModel_KeepsAVendorItCannotMistakeForAGeography(t *testing.T) {
+	cases := map[string]string{
+		"amazon.nova-pro-v1:0":            "amazon.nova-pro",
+		"anthropic.claude-sonnet-5-v1:0":  "anthropic.claude-sonnet-5",
+		"meta.llama3-3-70b-instruct-v1:0": "meta.llama3-3-70b-instruct",
+		"cohere.command-r-plus-v1:0":      "cohere.command-r-plus",
+		// Unknown on both axes: neither the leading segment nor the one
+		// after it is a name we hold, so the id is left exactly as it came.
+		"xx.unknownvendor.some-model-v1:0": "xx.unknownvendor.some-model",
+		"Qwen/Qwen2.5-0.5B-Instruct":       "Qwen/Qwen2.5-0.5B-Instruct",
+	}
+	for in, want := range cases {
+		t.Run(in, func(t *testing.T) {
+			require.Equal(t, want, NormalizeBedrockModel(in))
+		})
+	}
+}
+
+// TestNormalizeBedrockModel_RecognisesAnIdNewOnOneAxis covers what a live
+// eu-central-1 listing returned days after the vendor list was written:
+// "global.xai.grok-4.6", a vendor the list did not hold. Anchoring only on the
+// vendor left the geography in the key, so the id matched no catalog entry and
+// the model metered at zero. Each id below is unfamiliar on one axis and
+// recognised through the other.
+func TestNormalizeBedrockModel_RecognisesAnIdNewOnOneAxis(t *testing.T) {
+	cases := map[string]string{
+		// Known geography, vendor we had never seen (the live case).
+		"global.xai.grok-4.6": "xai.grok-4.6",
+		"eu.xai.grok-4.6":     "xai.grok-4.6",
+		// Known vendor, geography outside the list.
+		"il.anthropic.claude-sonnet-5-20260514-v1:0": "anthropic.claude-sonnet-5",
+		"mx.amazon.nova-2-lite-v1:0":                 "amazon.nova-2-lite",
+	}
+	for in, want := range cases {
+		t.Run(in, func(t *testing.T) {
+			require.Equal(t, want, NormalizeBedrockModel(in))
+		})
+	}
+}