mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-01 12:31:42 +02:00
[proxy,management] Serve Bedrock model discovery from the control plane
A Bedrock provider could never answer a discovery request. The router
routed GET /inference-profiles to the record's upstream, which has to be
bedrock-runtime.<region> for InvokeModel to work, and that host does not
implement the operation — AWS answers <UnknownOperationException/>.
ListInferenceProfiles lives on the control plane at bedrock.<region>.
Give the route a discovery host, taken from the catalog's declaration
with the region read back out of the configured upstream, and send the
listing — and only the listing — there. Inference is untouched, and a
proxied or self-hosted Bedrock endpoint gets no discovery host at all
rather than a guessed one, since inventing a host would send the
operator's credential somewhere they never configured.
Two things had to follow for the listing to be usable once it arrives.
The response filter only understood OpenAI's {data:[{id:…}]}, so a
Bedrock listing was forwarded whole — offering every profile in the
account whatever the policy said. It now recognises the
inferenceProfileSummaries envelope, and matches a listing id against the
record's models after stripping the region prefix and version suffix, so
the two spellings of one model line up.
The policy bound had the same problem from the other side: it intersected
by exact string, so a record registering the raw profile id while a
guardrail names the catalog key intersected to nothing and would have
bounded a working provider's listing down to empty. routeClaimsModel
already normalises the candidate for this reason; the bound now agrees
with it.
The live discovery e2e flips from asserting the 404 to asserting a real
filtered listing. The mock upstream cannot cover any of this: it answers
/inference-profiles on the same listener as everything else, so a
mock-based test passes whichever host the request went to.
This commit is contained in:
@@ -169,17 +169,15 @@ func liveDiscoveryCases() []liveDiscoveryCase {
|
||||
// Bedrock lists inference profiles, not models: matchModelless routes
|
||||
// /inference-profiles to a Bedrock route and refuses /v1/models for one.
|
||||
//
|
||||
// The request reaches AWS and AWS refuses it — bedrock-runtime answers
|
||||
// <UnknownOperationException/>, because ListInferenceProfiles is a CONTROL
|
||||
// PLANE operation served by bedrock.<region>.amazonaws.com, not the runtime
|
||||
// host. A provider record carries one upstream and it has to be the runtime
|
||||
// host for InvokeModel to work, so no Bedrock record can serve a listing as
|
||||
// the model stands today.
|
||||
// The listing is served by the CONTROL PLANE (bedrock.<region>), not the
|
||||
// runtime host a provider record must point at for InvokeModel — the
|
||||
// runtime host answers <UnknownOperationException/>. The router now sends
|
||||
// the listing, and only the listing, to the control plane, so this case
|
||||
// asserts a real filtered listing rather than the 404 it used to get.
|
||||
//
|
||||
// The mock upstream hides this entirely: it answers /inference-profiles on
|
||||
// the same listener as everything else, so the routing test passes there
|
||||
// while the real endpoint 404s. That is the whole reason this file exists,
|
||||
// so the case is kept, asserting what actually happens.
|
||||
// The mock upstream cannot show any of this: it answers
|
||||
// /inference-profiles on the same listener as everything else, so a
|
||||
// mock-based test passes whichever host the request went to.
|
||||
if k := os.Getenv("AWS_BEARER_TOKEN_BEDROCK"); k != "" {
|
||||
region := os.Getenv("AWS_REGION")
|
||||
if region == "" {
|
||||
@@ -192,9 +190,13 @@ func liveDiscoveryCases() []liveDiscoveryCase {
|
||||
cases = append(cases, liveDiscoveryCase{
|
||||
name: "bedrock", catalogID: "bedrock_api",
|
||||
upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k,
|
||||
path: "/inference-profiles",
|
||||
models: []string{sharedllm.NormalizeAnthropicModel(strings.TrimPrefix(model, "global."))},
|
||||
outcome: outcomeUpstreamNoListing,
|
||||
path: "/inference-profiles",
|
||||
// Registered verbatim, as an operator would copy it from AWS: the
|
||||
// region prefix is what makes the id invocable, and the listing
|
||||
// returns ids in exactly this form.
|
||||
models: []string{model},
|
||||
outcome: outcomeFiltered,
|
||||
permitted: []string{model},
|
||||
})
|
||||
}
|
||||
|
||||
@@ -342,8 +344,11 @@ func runLiveDiscoveryCase(t *testing.T, ctx context.Context, tc liveDiscoveryCas
|
||||
}
|
||||
for _, id := range ids {
|
||||
_, direct := permitted[id]
|
||||
_, normalised := permitted[sharedllm.NormalizeAnthropicModel(id)]
|
||||
assert.Truef(t, direct || normalised,
|
||||
_, dated := permitted[sharedllm.NormalizeAnthropicModel(id)]
|
||||
// Bedrock ids carry a region prefix and version suffix the record may
|
||||
// not repeat; the proxy's filter tries the same forms.
|
||||
_, bedrock := permitted[sharedllm.NormalizeBedrockModel(id)]
|
||||
assert.Truef(t, direct || dated || bedrock,
|
||||
"%s offered %q, which no policy on this route permits — every entry the picker shows must be a request the guardrail would allow", tc.name, id)
|
||||
}
|
||||
for _, hidden := range tc.wantHidden {
|
||||
|
||||
Reference in New Issue
Block a user