[management] Fetch a provider's model list from the vendor

The catalog is the only source of models an operator can pick from, and
it cannot know two things that matter. It goes stale — its entries carry
comments recording which models a vendor retired on which date — and it
cannot see an account: which OpenAI models an org is entitled to, which
Bedrock inference profiles a given account and region hold, which Vertex
models a project has enabled.

Add a client that asks the vendor directly, with the endpoint, auth
header and response shape all declared by the catalog rather than
supplied by the caller. The four shapes come from probing the live APIs
(see the discovery e2e); each vendor invented its own envelope and none
can be guessed from the request.

Bedrock is the case that shaped the design. Its listing lives on the
control plane while inference must go to the runtime host, so Discovery
carries its own host. The ids it returns are region-prefixed and are
taken verbatim, because that prefix is what AWS requires and it cannot
be derived from the configured region — an eu-central-1 account holds
global.* profiles alongside its eu.* ones.

The vendor is authoritative for the id; the catalog stays authoritative
for pricing. A discovered model the shipped table cannot price is
reported as such, so it cannot be registered at a silent zero rate.

Management has not made outbound calls on an operator's behalf before,
and it holds a credential for every provider, so the host is checked
before dialing: every resolved address must be public, which covers the
cloud metadata address and NetBird's own overlay range, and redirects
are not followed since they would move the request to a host the check
never saw.

management/internals/modules/agentnetwork/catalog/catalog.go

@@ -113,8 +113,61 @@ type Provider struct {
 	// upstream provider + credentials on Portkey's hosted side).
 	ExtraHeaders []ExtraHeader
 	Models       []Model
+	// Discovery, when non-nil, describes how to ask this vendor which
+	// models the operator's own credential can actually reach, so the
+	// provider form can offer a live list instead of only the hand-curated
+	// Models above. Nil for entries with no listing endpoint (gateways
+	// vary too much) — those keep free-text entry.
+	Discovery *Discovery
 }
 
+// ListingShape names the response envelope a vendor returns its model
+// listing in. Every vendor invented its own, and none of them can be
+// guessed from the request, so the catalog states it.
+type ListingShape string
+
+const (
+	// ShapeOpenAIData is {"data":[{"id":…}]} — OpenAI, and Anthropic, which
+	// adopted the same envelope.
+	ShapeOpenAIData ListingShape = "openai_data"
+	// ShapeBedrockInferenceProfiles is
+	// {"inferenceProfileSummaries":[{"inferenceProfileId":…}]}. The ids carry
+	// the region prefix that makes them invocable, which is exactly what an
+	// operator cannot reconstruct by hand.
+	ShapeBedrockInferenceProfiles ListingShape = "bedrock_inference_profiles"
+	// ShapeVertexPublisherModels is {"publisherModels":[{"name":…}]}, where
+	// name is a resource path and the invocable id is its last segment joined
+	// to a separate versionId field.
+	ShapeVertexPublisherModels ListingShape = "vertex_publisher_models"
+)
+
+// Discovery describes one vendor's model-listing endpoint.
+//
+// Host is deliberately separate from the provider record's upstream URL:
+// Bedrock serves listings from the control plane (bedrock.<region>) while
+// inference must go to the runtime host (bedrock-runtime.<region>), so the
+// two cannot be the same value. Empty Host means "use the record's own
+// upstream", which is right for every vendor that serves both from one host.
+//
+// The regionPlaceholder in Host is substituted from the provider record's
+// region. Deriving the discovery host from the catalog rather than accepting
+// one from the caller is also what keeps this from being an open proxy: the
+// only hosts management will dial are the ones written here.
+type Discovery struct {
+	Host  string
+	Path  string
+	Query string
+	Shape ListingShape
+	// Headers are static headers the vendor requires beyond the credential
+	// (Anthropic versions its API through one and rejects a request without
+	// it). The auth header itself comes from AuthHeaderName/Template.
+	Headers map[string]string
+}
+
+// RegionPlaceholder is replaced in Discovery.Host by the provider record's
+// configured region.
+const RegionPlaceholder = "<region>"
+
 // ExtraHeader names a single optional per-provider routing/config
 // header. Catalog declares N of these per provider type; the operator
 // fills any subset on the provider record (see Provider.ExtraValues).
@@ -245,8 +298,12 @@ var providers = []Provider{
 		AuthHeaderTemplate: "Bearer ${API_KEY}",
 		DefaultContentType: "application/json",
 		BrandColor:         "#10A37F",
-		ParserID:           "openai",
-		PricingSurfaces:    []string{"openai"},
+		Discovery: &Discovery{
+			Path:  "/v1/models",
+			Shape: ShapeOpenAIData,
+		},
+		ParserID:        "openai",
+		PricingSurfaces: []string{"openai"},
 		// Pricing + context windows cross-checked against LiteLLM's
 		// model_prices_and_context_window.json. Notable corrections from
 		// earlier values: o4-mini repriced from $4/$16 to $1.10/$4.40
@@ -284,8 +341,18 @@ var providers = []Provider{
 		AuthHeaderTemplate: "${API_KEY}",
 		DefaultContentType: "application/json",
 		BrandColor:         "#D97757",
-		ParserID:           "anthropic",
-		PricingSurfaces:    []string{"anthropic"},
+		Discovery: &Discovery{
+			Path: "/v1/models",
+			// The default page is short and a picker wants the whole
+			// catalogue in one call.
+			Query: "limit=1000",
+			Shape: ShapeOpenAIData,
+			// Anthropic versions its API through a header and refuses a
+			// request that omits it, listing included.
+			Headers: map[string]string{"anthropic-version": "2023-06-01"},
+		},
+		ParserID:        "anthropic",
+		PricingSurfaces: []string{"anthropic"},
 		// Per Anthropic's current model lineup. Pricing in USD per 1k
 		// tokens. Context windows: 4.6+ family is 1M; Haiku 4.5 stays at
 		// 200K. claude-3-7-sonnet and claude-3-5-haiku retired
@@ -345,6 +412,22 @@ var providers = []Provider{
 		AuthHeaderTemplate: "Bearer ${API_KEY}",
 		DefaultContentType: "application/json",
 		BrandColor:         "#FF9900",
+		// Listings come from the CONTROL PLANE, not the runtime host in
+		// DefaultHost above: ListInferenceProfiles is not an operation
+		// bedrock-runtime implements, and answers <UnknownOperationException/>
+		// there. Inference has to go to the runtime host, so the two hosts
+		// genuinely differ and Discovery.Host carries the difference.
+		//
+		// Inference profiles rather than foundation models because the profile
+		// id is the invocable one: it carries the region prefix (eu., us.,
+		// global.) that AWS requires and that cannot be derived from the
+		// configured region — an eu-central-1 account legitimately holds
+		// global.* profiles.
+		Discovery: &Discovery{
+			Host:  "bedrock." + RegionPlaceholder + ".amazonaws.com",
+			Path:  "/inference-profiles",
+			Shape: ShapeBedrockInferenceProfiles,
+		},
 		// ParserID stays empty (path-style dispatch via IsBedrockPathStyle);
 		// the request parser meters these under the "bedrock" surface.
 		PricingSurfaces: []string{"bedrock"},
@@ -395,6 +478,15 @@ var providers = []Provider{
 		AuthHeaderTemplate: "Bearer ${API_KEY}",
 		DefaultContentType: "application/json",
 		BrandColor:         "#4285F4",
+		// Only the v1beta1 publisher listing answers: the v1 form and the
+		// project-scoped form under BOTH versions return 404. That means the
+		// list is publisher-global — it cannot say which models this project
+		// has enabled — so it is offered as a suggestion beside the catalog
+		// rather than replacing it. See the discovery e2e for the probes.
+		Discovery: &Discovery{
+			Path:  "/v1beta1/publishers/anthropic/models",
+			Shape: ShapeVertexPublisherModels,
+		},
 		// ParserID stays empty (path-style dispatch via IsVertexPathStyle);
 		// Anthropic-on-Vertex requests are metered under the "anthropic"
 		// surface with the bare, unversioned model id.