add model routing and catalog

2026-10-30 07:29:07 +01:00 · 2026-08-20 16:46:43 -04:00
parent b8b04491a1
commit 2ed2b2b079
13 changed files with 219 additions and 15 deletions
@@ -25,7 +25,7 @@ A capability is an API format the gateway will accept and proxy. The incoming re

 Typed providers start with recommended capabilities. Custom providers need at least one selected. You can change capabilities later on the provider's **General** tab. Select every format the upstream actually speaks. Custom providers can mix any combination.

-If two attached providers share a capability for the same model, the gateway picks one using allow lists and catalog ownership. Two providers of the same type both allowing `*` can leave the request ambiguous.
+If two attached providers share a capability for the same model, the gateway picks one at request time. See [Model Routing](/manage/ai/providers/model-routing#provider-selection).

 ## Auth Type

@@ -58,11 +58,7 @@ See [Ollama](/manage/ai/providers/custom/ollama), [vLLM](/manage/ai/providers/cu

 ## Models

-Allow and block lists restrict which model keys this provider may serve. A request must match an allow entry and must not match a block entry. An empty allow list denies all traffic. Patterns support `*` and `?` (`gpt-*`, `claude-3-5-sonnet-?`).
-
-When you attach a provider to a resource you can **inherit** these lists or **select** a subset for that resource only.
-
-OpenRouter, Vercel AI Gateway, and Custom are treated as aggregators: a wildcard allow such as `*` matches at request time but does not expand in `GET /v1/models`. Add exact model keys to make those models show up in discovery.
+Allow and block lists, inherit vs select, and provider selection are covered in [Model Routing](/manage/ai/providers/model-routing). Known Models, ownership scoring, and token pricing for budgets come from the [Model Catalog](/manage/ai/model-catalog).

 ## Headers and TLS

@@ -16,7 +16,7 @@ See [AI Providers](/manage/ai/providers/overview) if you haven't created a provi
 3. Select every **API capability** the upstream speaks. You can add any combination.
 4. Choose **Routing Mode**. For **Upstream URL**, paste the base URL. For **Site Targets**, add targets after save.
 5. Pick **Auth Type** and paste a key if that type needs one.
-6. Set **Allow** and **Block** lists. Add exact model keys if you want them listed in `GET /v1/models`. See [Models](/manage/ai/providers/configuration#models).
+6. Set **Allow** and **Block** lists. Add exact model keys if you want them listed in `GET /v1/models`. See [Model Routing](/manage/ai/providers/model-routing).
 7. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).

 ## Reach a Self-Hosted API
@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
 3. Select **OpenAI Chat Completions** (and any other formats Bifrost should accept).
 4. Set **Routing Mode** to **Site Targets** if Bifrost runs on a site's network. Use **Upstream URL** (`http://<host>:8080`) only when the Pangolin node can reach Bifrost directly.
 5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if Bifrost requires a virtual key or other API key.
-6. Allow the model ids Bifrost should serve, including provider-prefixed keys such as `openai/gpt-4o-mini` if that is how Bifrost names them. See [Models](/manage/ai/providers/configuration#models).
+6. Allow the model ids Bifrost should serve, including provider-prefixed keys such as `openai/gpt-4o-mini` if that is how Bifrost names them. See [Model Routing](/manage/ai/providers/model-routing).
 7. Save. On the **Network** tab, add a target: the Bifrost host, port `8080`, method HTTP.
 8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).

@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
 3. Select **OpenAI Chat Completions**.
 4. Set **Routing Mode** to **Site Targets** if Ollama runs on a site's network. Use **Upstream URL** (`http://<host>:11434`) only when the Pangolin node can reach Ollama directly.
 5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if you set an Ollama API key.
-6. Allow the exact model ids you plan to call, for example `llama3.2`. See [Models](/manage/ai/providers/configuration#models).
+6. Allow the exact model ids you plan to call, for example `llama3.2`. See [Model Routing](/manage/ai/providers/model-routing).
 7. Save. On the **Network** tab, add a target: the Ollama host, port `11434`, method HTTP.
 8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).

@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
 3. Select **OpenAI Chat Completions**.
 4. Set **Routing Mode** to **Site Targets** if vLLM runs on a site's network. Use **Upstream URL** (`http://<host>:8000`) only when the Pangolin node can reach vLLM directly.
 5. Set **Auth Type** to **No Auth** if the server has no key. Use **Bearer** and paste the key if you started vLLM with `--api-key`.
-6. Allow the exact served model id (the `--model` name or `--served-model-name`). See [Models](/manage/ai/providers/configuration#models).
+6. Allow the exact served model id (the `--model` name or `--served-model-name`). See [Model Routing](/manage/ai/providers/model-routing).
 7. Save. On the **Network** tab, add a target: the vLLM host, port `8000`, method HTTP.
 8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).

@@ -0,0 +1,74 @@
+---
+title: "Model Routing"
+description: "Allow and block lists, inherit vs select, and how the gateway picks a provider"
+---
+
+Model routing is how Pangolin decides which attached provider handles a request, and which model keys that provider may serve. Configure lists on the provider, optionally narrow them on the resource, then the gateway picks a provider per request.
+
+See [AI Providers](/manage/ai/providers/overview) for what a provider is, and [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and routing mode. The [Model Catalog](/manage/ai/model-catalog) supplies Known Models, wildcard discovery, ownership scoring, and pricing for budgets.
+
+## Choosing Models on a Provider
+
+On the provider's **Models** tab, every request must match an **allow** entry and must not match a **block** entry. An empty allow list denies all traffic. A key cannot sit on both lists.
+
+You add keys three ways:
+
+| How | What it is | Example |
+|---|---|---|
+| **[Catalog (Known Models)](/manage/ai/model-catalog)** | Pick from the catalog for that provider type | `gpt-4o`, `claude-sonnet-4-6` |
+| **Custom key** | Type an exact model id the upstream accepts | `llama3.2`, `openai/gpt-4o-mini` |
+| **Pattern** | Glob with `*` and `?` | `gpt-*`, `claude-3-5-sonnet-?`, `*` |
+
+`*` matches every model key. Typed providers (OpenAI, Anthropic, Google Gemini, Vertex AI, Amazon Bedrock, Microsoft Foundry) show Known Models from the catalog. [OpenRouter](/manage/ai/providers/open-router), [Vercel AI Gateway](/manage/ai/providers/vercel-ai-gateway), and [Custom](/manage/ai/providers/custom) skip that list; enter custom keys or patterns.
+
+### Allow vs Block
+
+Allow is the positive set. Block subtracts even when allow matched. Allow `*` and block `gpt-4o-mini` to permit every key except that one. Allow `gpt-*` and leave block empty to permit only keys that start with `gpt-`.
+
+### Model Discovery
+
+`GET /v1/models` is answered from these lists plus the catalog. The gateway does not proxy that listing call upstream.
+
+- Exact allow keys are listed as themselves.
+- Patterns expand against the catalog for typed providers (`claude-*` becomes every matching catalog id).
+- On OpenRouter, Vercel AI Gateway, and Custom, add exact allow keys to make those models show up in discovery. A wildcard still matches at request time.
+
+A block pattern hides a model from discovery the same way it would reject the request.
+
+## Inherit vs Select
+
+On an AI Gateway resource, attach providers from the **AI Gateway** settings tab. Each attachment has an access mode:
+
+| Mode | What the resource uses |
+|---|---|
+| **Inherit** | The provider's allow and block lists as configured on the provider |
+| **Select** | A subset of that provider's allow-list models, chosen on this resource |
+
+The same provider can be inherited on one resource and selected on another. You can disable an attachment without removing the provider.
+
+Select only offers models already on the provider's allow list. Use inherit when every resource should see the same keys. Use select when one gateway should expose a narrower set (for example a public resource with `gpt-4o` only, while the provider still allows `gpt-*`).
+
+## Provider Selection
+
+When a resource has more than one attached provider, the gateway picks one per request. Overlapping allows are allowed at save time. Collisions are resolved, or rejected, when the request arrives.
+
+Steps, in order. Later steps run only while more than one candidate remains:
+
+1. **Capability.** The path must match a [capability](/manage/ai/providers/configuration#capabilities) the provider advertises.
+2. **Allow and block.** The effective lists (inherit or select) must pass.
+3. **Specificity.** The most specific matching allow wins: exact keys beat patterns, fewer wildcards win, longer literals win. `gpt-4o` beats `gpt-*` beats `*`.
+4. **Catalog ownership.** A typed provider whose catalog contains the id beats an aggregator that also allows it.
+5. **Class.** Native typed providers beat aggregators (OpenRouter, Vercel AI Gateway), which beat Custom.
+6. **Ambiguous.** If more than one distinct provider remains, the gateway returns `403` with `Model "<id>" is ambiguous across multiple AI providers on this resource`.
+
+Typical ambiguous cases: two OpenAI providers both allowing `*`, or two aggregators both allowing `*` with no native owner on the resource.
+
+`GET /v1/models` skips steps 3-6. There is no requested model to disambiguate, so the gateway returns the union of what every attached provider advertising Anthropic Models would accept.
+
+### Examples
+
+**OpenAI + Anthropic, both `*`.** `POST /v1/chat/completions` with `gpt-4o` goes to OpenAI. Anthropic never reaches scoring; capability alone decides. `POST /v1/messages` with a Claude model goes to Anthropic.
+
+**OpenAI + OpenRouter, both `*`.** `gpt-4o` on Chat Completions matches both. Specificity is a tie (`*` vs `*`). Catalog ownership sends it to OpenAI.
+
+**Two OpenAI providers, both `*`.** Capability, lists, specificity, catalog, and class all leave both candidates. Result is the ambiguous error. Narrow at least one allow list, disable one attachment, or split them across resources.
@@ -27,7 +27,7 @@ OpenRouter's default capability is Chat Completions only. Clients that require R

 ## Models

-OpenRouter is an aggregator. See [Models](/manage/ai/providers/configuration#models) for how allow lists and `GET /v1/models` discovery work. Add exact keys such as `openai/gpt-4o` or `anthropic/claude-sonnet-4`.
+OpenRouter is an aggregator. See [Model Routing](/manage/ai/providers/model-routing) for how allow lists and `GET /v1/models` discovery work. Add exact keys such as `openai/gpt-4o` or `anthropic/claude-sonnet-4`.

 ## Clients

@@ -5,7 +5,7 @@ description: "Connect upstream model APIs once per organization, then attach the

 A provider is an organization-level connection to an upstream model API. It stores the provider type, credentials, base URL, API capabilities, and optional model allow and block lists. Create providers once, then attach them to any AI Gateway resource in the org. Clients call the resource's domain, not the provider directly.

-See [AI Gateway Overview](/manage/ai/overview) for creating a resource and issuing keys. See [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, routing, and model lists.
+See [AI Gateway Overview](/manage/ai/overview) for creating a resource and issuing keys. See [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and routing. See [Model Routing](/manage/ai/providers/model-routing) for allow lists, inherit vs select, and how the gateway picks a provider. See [Model Catalog](/manage/ai/model-catalog) for known models and pricing used by routing and budgets.

 ## What a Provider Is

@@ -30,7 +30,7 @@ The AI Gateway → Providers → Create form, showing the provider type dropdown

 ## Attach to a Resource

-On an AI Gateway resource, attach one or more providers from the **AI Gateway** settings tab. For each attachment you can inherit the provider's model lists or select a subset. Details are in [Provider Configuration](/manage/ai/providers/configuration#models).
+On an AI Gateway resource, attach one or more providers from the **AI Gateway** settings tab. For each attachment you can inherit the provider's model lists or select a subset. Details are in [Model Routing](/manage/ai/providers/model-routing#inherit-vs-select).

 The resource only speaks the API formats its attached providers advertise. Attach whichever providers match the [clients](/manage/ai/overview#4-connect-a-client) you plan to connect.

@@ -25,7 +25,7 @@ See [AI Providers](/manage/ai/providers/overview) if you haven't created a provi

 ## Models

-Vercel AI Gateway is an aggregator. See [Models](/manage/ai/providers/configuration#models) for how allow lists and `GET /v1/models` discovery work. Add exact model keys to the allow list.
+Vercel AI Gateway is an aggregator. See [Model Routing](/manage/ai/providers/model-routing) for how allow lists and `GET /v1/models` discovery work. Add exact model keys to the allow list.

 ## Clients