pangolin_mirror/docs-v2
1
0
Fork 0
mirror of https://github.com/fosrl/docs-v2.git synced 2026-10-12 07:39:08 +02:00
Code Issues Packages Projects Releases Wiki Activity

port mintlify to fumadocs

Browse Source
This commit is contained in:
miloschwartz
2026-09-25 15:31:51 -04:00
parent dc54fb1017
commit 63199a588c
373 changed files with 13533 additions and 4577 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/docs/manage/ai/budgets.mdx
+66
View File
@@ -0,0 +1,66 @@
---
title: "Budgets"
description: "Cap estimated USD spend or token usage on providers, models, resources, roles, and keys"
---
A budget is a cap on estimated USD spend or token count for a rolling window. Pangolin checks every matching enabled budget before it forwards a request. If **any** one is already at or above its maximum, the call is blocked.
Budgets are estimates used for enforcement, not a copy of the vendor invoice. See [Model Catalog](/manage/ai/model-catalog) for where USD rates come from. The same recorded usage appears in [Usage Analytics](/manage/ai/usage-analytics).
## Estimates, Not Invoices
USD cost uses catalog rates (`in`, `out`, `cache`, `reasoning`) times the token buckets Pangolin recorded for the call. Cache-write tokens use the input rate. Missing reasoning rates use the output rate.
If the model ID is not in the catalog, the request has no USD amount and does not count toward USD budgets. Token budgets still sum tokens for that call.
[OpenRouter](/manage/ai/providers/open-router), [Vercel AI Gateway](/manage/ai/providers/vercel-ai-gateway), and [Custom](/manage/ai/providers/custom) may match a catalog ID with approximate pricing.
If the upstream omits usage, Pangolin estimates tokens from the request and response.
The check uses **prior** recorded usage. A request that would cross the cap can still complete, so usage can slightly overshoot. Reset periods are trailing windows from now, not calendar months. Daily means the last 24 hours.
## How Usage Is Calculated
After each call, Pangolin records prompt, cache-read, cache-write, completion, and reasoning tokens.
- **Token** budgets sum those counts.
- **USD** budgets multiply the same counts by catalog rates, then sum dollars.
Enforcement totals that history for the budget's scope and [period](#fields), then compares the total to **Maximum Spend**.
## Where to Set a Budget
Each budget has exactly one scope. You can add more than one budget on the same scope when the unit or period differs, for example a daily USD cap and a monthly token cap on the same provider.
| Scope | Where | Why |
|---|---|---|
| **Provider** | **AI Gateway → Providers → Budget** | Cap everything that uses that upstream key |
| **Model** | Provider **Models** tab, on an allow-list entry | Cap one key or glob, such as `gpt-4o` or `gpt-*` |
| **Public resource** | Resource **Budget** tab | Cap a specific public gateway |
| **Private resource** | Resource **Budget** tab | Cap a client-only gateway |
| **Role** | Role form | Cap everyone in that role. Usage attributed to those users is summed |
| **Virtual API key** | Create or edit the key | Cap a service key or usage attributed to that key |
All matching scopes apply together. A call can hit a provider budget, a resource budget, a role budget, and a key budget at once. Exceeding any of them blocks the request.
## Fields
The editor uses **Spend Type**, **Maximum Spend**, and **Reset Period**.
| Field | Options |
|---|---|
| **Spend Type** | `USD` or `Tokens` |
| **Maximum Spend** | A positive maximum in that unit |
| **Reset Period** | Hourly, Daily (24 hours), Weekly (7 days), Monthly (30 days), Yearly (365 days), or Lifetime |
Lifetime covers all recorded usage for that scope.
## Over Budget
The gateway returns HTTP **429** with the message `AI usage budget exceeded for this request`. The JSON envelope matches the API the client is calling:
| Family | Shape |
|---|---|
| OpenAI | `error.type`: `rate_limit_error`, `error.code`: `rate_limit_exceeded` |
| Anthropic | `error.type`: `rate_limit_error` |
| Gemini | `error.status`: `RESOURCE_EXHAUSTED` |
content/docs/manage/ai/configure-ai-clients/claude-desktop.mdx
+51
View File
@@ -0,0 +1,51 @@
---
title: "Claude Desktop"
description: "Connect Claude Desktop's third-party inference to a Pangolin AI Gateway resource"
---
Claude Desktop can route its requests through a third-party inference gateway instead of Anthropic directly. Point it at an AI Gateway resource that has an Anthropic-compatible provider attached — Anthropic itself, Bedrock, Vertex AI, or a custom Anthropic-compatible endpoint. See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
<Frame>
<img
src="/images/claude_desktop_landing.png"
alt="Claude Desktop third-party inference settings"
/>
</Frame>
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Steps
1. Open Claude Desktop, go to the menu in the top left and select **Help > Troubleshooting**, and turn on **Developer Mode**.
<Frame>
<img
src="/images/claude_desktop_troubleshooting.png"
alt="Claude Desktop third-party inference settings"
/>
</Frame>
2. From the menu bar, open **Developer**, then **Configure Third-Party Inference…**.
<Frame>
<img
src="/images/claude_desktop_developer.png"
alt="Claude Desktop third-party inference settings"
/>
</Frame>
3. Select **Gateway**
4. Select **Credential kind** > **Static API Key**.
5. Fill in:
- **Gateway base URL:** `<endpoint>`
- **Gateway API Key:** `<key>`
<Frame>
<img
src="/images/claude_desktop_developer_settings.png"
alt="Claude Desktop third-party inference settings"
/>
</Frame>
6. Restart Claude Desktop.
Send a message to confirm it's working — check the resource's activity in Pangolin's logs if you want to verify traffic is actually flowing through the gateway rather than straight to Anthropic.
For a private resource, put `none` in the API Key field rather than leaving it blank, and make sure the Pangolin client is installed and connected on the same machine as Claude Desktop.
content/docs/manage/ai/configure-ai-clients/claude.mdx
+116
View File
@@ -0,0 +1,116 @@
---
title: "Claude Code"
description: "Connect Claude Code to a Pangolin AI Gateway resource"
---
Claude Code talks the Anthropic Messages API, so the resource you point it at needs an Anthropic-compatible provider attached - Anthropic itself, Bedrock, Vertex AI, or a custom Anthropic-compatible endpoint like Kimi K2. See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Fastest: Pangolin CLI
[Install the Pangolin CLI](/manage/clients/install-client#quick-install-recommended) if you don't have it, then log in:
```bash
pangolin login
```
Configure Claude Code against a resource:
```bash
pangolin configure claude
```
This prompts you to pick an organization and resource if you have more than one, fetches a key for you if the resource needs one, and writes `~/.claude/settings.json`. To skip the prompts:
```bash
pangolin configure claude --resource <resource-niceid-or-domain>
```
To undo it:
```bash
pangolin configure claude --reset
```
## Manual setup
Pick the preset that matches your provider.
### Default (Anthropic)
Write `~/.claude/settings.json`:
```json
{
"apiKeyHelper": "echo '<key>'",
"env": {
"ANTHROPIC_BASE_URL": "<endpoint>"
}
}
```
Or, for a one-off shell session instead of a permanent settings file:
```bash
export ANTHROPIC_BASE_URL=<endpoint>
export ANTHROPIC_API_KEY=<key>
claude
```
For a private resource, keep `apiKeyHelper`/`ANTHROPIC_API_KEY` in place and set the key to `none`. Claude Code only calls `apiKeyHelper` if it's present, and falls back to whatever account you're already signed into if it isn't - so removing it sends your traffic to Anthropic instead of through the gateway.
### Amazon Bedrock
Use this when the resource has a Bedrock provider attached. Write `~/.claude/settings.json`:
```json
{
"env": {
"ANTHROPIC_MODEL": "claude-sonnet-4-6",
"ANTHROPIC_BEDROCK_BASE_URL": "<endpoint>/bedrock",
"CLAUDE_CODE_USE_BEDROCK": "1",
"CLAUDE_CODE_SKIP_BEDROCK_AUTH": "1"
}
}
```
### Google Vertex AI
Use this when the resource has a Vertex AI provider attached. Write `~/.claude/settings.json`:
```json
{
"env": {
"CLOUD_ML_REGION": "global",
"ANTHROPIC_VERTEX_PROJECT_ID": "<your-gcp-project-id>",
"CLAUDE_CODE_USE_VERTEX": "1",
"CLAUDE_CODE_SKIP_VERTEX_AUTH": "1",
"ANTHROPIC_VERTEX_BASE_URL": "<endpoint>/v1"
}
}
```
Replace `<your-gcp-project-id>` with your actual GCP project ID.
### Kimi K2 (Moonshot AI)
Use this when the resource has a custom provider pointed at Moonshot AI's Anthropic-compatible endpoint. Write `~/.claude/settings.json`:
```json
{
"apiKeyHelper": "echo '<key>'",
"env": {
"ANTHROPIC_BASE_URL": "<endpoint>/anthropic",
"ANTHROPIC_MODEL": "kimi-k2",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "kimi-k2",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "kimi-k2",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "kimi-k2",
"CLAUDE_CODE_SUBAGENT_MODEL": "kimi-k2",
"ENABLE_TOOL_SEARCH": "false"
}
}
```
content/docs/manage/ai/configure-ai-clients/codex.mdx
+59
View File
@@ -0,0 +1,59 @@
---
title: "Codex"
description: "Connect Codex to a Pangolin AI Gateway resource"
---
Codex talks the OpenAI API, so the resource you point it at needs an OpenAI-compatible provider attached — OpenAI, OpenRouter, Vercel AI Gateway, Microsoft Foundry, or a custom OpenAI-compatible endpoint. See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Fastest: Pangolin CLI
[Install the Pangolin CLI](/manage/clients/install-client#quick-install-recommended) if you don't have it, then log in:
```bash
pangolin login
```
Configure Codex against a resource:
```bash
pangolin configure codex
```
This prompts you to pick an organization and resource if you have more than one, fetches a key for you if the resource needs one, and merges a `pangolin` provider into `~/.codex/config.toml` without touching anything else in that file. To skip the prompts:
```bash
pangolin configure codex --resource <resource-niceid-or-domain>
```
To undo it:
```bash
pangolin configure codex --reset
```
## Manual setup
Merge this into `~/.codex/config.toml` (honors `$CODEX_HOME` if you've set it):
```toml
model_provider = "pangolin"
[model_providers.pangolin]
name = "Pangolin AI Gateway"
base_url = "<endpoint>/v1"
wire_api = "responses"
env_key = "PANGOLIN_API_KEY"
```
Codex reads the key from the environment variable named in `env_key`, not from the file. Before running Codex, export it in your shell:
```bash
export PANGOLIN_API_KEY=<key>
```
Codex is the one client here that also tolerates no key at all: for a private resource you can drop the `env_key` line and skip the export entirely. Exporting `PANGOLIN_API_KEY=none` works too, so use whichever you find clearer.
content/docs/manage/ai/configure-ai-clients/gemini.mdx
+58
View File
@@ -0,0 +1,58 @@
---
title: "Gemini CLI"
description: "Connect Gemini CLI to a Pangolin AI Gateway resource"
---
Gemini CLI talks Google's native `generateContent` API, so the resource you point it at needs a Google Gemini (or Vertex AI) provider attached. See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Fastest: Pangolin CLI
[Install the Pangolin CLI](/manage/clients/install-client#quick-install-recommended) if you don't have it, then log in:
```bash
pangolin login
```
Configure Gemini CLI against a resource:
```bash
pangolin configure gemini
```
This prompts you to pick an organization and resource if you have more than one, fetches a key for you if the resource needs one, and writes `~/.gemini/.env` - the file Gemini CLI loads automatically on every run, regardless of which directory you start it from. To skip the prompts:
```bash
pangolin configure gemini --resource <resource-niceid-or-domain>
```
To undo it:
```bash
pangolin configure gemini --reset
```
## Manual setup
Write `~/.gemini/.env`:
```bash
GOOGLE_GEMINI_BASE_URL=<endpoint>
GEMINI_API_KEY=<key>
```
Gemini CLI also checks a project-local `.env` first (in the current directory or a parent, up to `.git` or your home directory) before falling back to `~/.gemini/.env`, so a project-level file with the same two lines overrides this for just that project.
Or, for a one-off shell session instead of a permanent file:
```bash
export GOOGLE_GEMINI_BASE_URL=<endpoint>
export GEMINI_API_KEY=<key>
gemini
```
For a private resource, keep `GEMINI_API_KEY` in place and set it to `none`. Gemini CLI won't start without it set to something.
content/docs/manage/ai/configure-ai-clients/open-webui.mdx
+54
View File
@@ -0,0 +1,54 @@
---
title: "Open WebUI"
description: "Connect Open WebUI to a Pangolin AI Gateway resource"
---
Open WebUI is a self-hosted chat interface. It talks the OpenAI API, so point it at an AI Gateway resource that has an OpenAI-compatible provider attached (OpenAI, OpenRouter, Vercel AI Gateway, Microsoft Foundry, or custom). See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## If you're running Open WebUI for the first time
```bash
docker run -d -p 3000:8080 \
-e OPENAI_API_BASE_URL=<endpoint>/v1 \
-e OPENAI_API_KEY=<key> \
-v open-webui:/app/backend/data \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:main
```
Open `http://localhost:3000`, finish the first-run account setup, and your Pangolin gateway is already the active connection.
## If Open WebUI is already running
Go to **Settings → Connections** and add a connection:
- **URL:** `<endpoint>/v1`
- **Key:** `<key>`
{/*
Screenshot: `/images/ai/openwebui-connections.png`
Open WebUI's Settings → Connections page with the Pangolin gateway URL and key filled in.
<Frame>
<img src="/images/ai/openwebui-connections.png" alt="Open WebUI connection settings" />
</Frame>
*/}
Save, then open a new chat - the models exposed by your resource's attached provider(s) show up in the model picker.
{/*
Screenshot: `/images/ai/openwebui-model-picker.png`
The Open WebUI model picker showing models served through the Pangolin gateway.
<Frame>
<img src="/images/ai/openwebui-model-picker.png" alt="Open WebUI model picker" />
</Frame>
*/}
For a private resource, set the key to `none` rather than leaving it blank. Open WebUI treats a blank key as "no connection configured" and won't list the models.
Note that Open WebUI runs as a server, not on your laptop - so it's the Open WebUI host that needs the Pangolin client installed and connected to reach a private resource, not the browser you're viewing it from.
content/docs/manage/ai/configure-ai-clients/openclaw.mdx
+88
View File
@@ -0,0 +1,88 @@
---
title: "OpenClaw"
description: "Connect OpenClaw to a Pangolin AI Gateway resource"
---
[OpenClaw](https://openclaw.ai) is an open-source agent gateway that can run against any OpenAI- or Anthropic-compatible endpoint. Point it at an AI Gateway resource with a matching provider attached - OpenAI-compatible (OpenAI, OpenRouter, Vercel AI Gateway, Microsoft Foundry, custom) or Anthropic-compatible (Anthropic, Bedrock, Vertex AI, custom). See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`), both on the resource's Keys page, plus the name of a model your attached provider actually serves (e.g. `gpt-4o`, `claude-sonnet-4-6`) - that's `<model-id>` below.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Install
```bash
# macOS/Linux
curl -fsSL https://openclaw.ai/install.sh | bash
# Windows PowerShell
iwr -useb https://openclaw.ai/install.ps1 | iex
```
## Connect it to your resource
```bash
openclaw onboard --non-interactive --accept-risk \
--auth-choice custom-api-key \
--custom-base-url "<endpoint>/v1" \
--custom-model-id "<model-id>" \
--custom-api-key "<key>" \
--custom-compatibility openai \
--install-daemon --skip-channels --skip-skills
```
Set `--custom-compatibility anthropic` instead of `openai` if the resource's provider is Anthropic/Bedrock/Vertex rather than OpenAI-compatible, and drop `--custom-base-url`'s trailing `/v1` in that case (Anthropic-style resources are addressed at the endpoint root).
Prefer the interactive wizard instead:
```bash
openclaw onboard --install-daemon
```
When prompted, choose **QuickStart or Manual mode**, select the custom/OpenAI-compatible provider option, and enter the base URL and key.
## Manual setup
Onboarding writes `~/.openclaw/openclaw.json`. You can edit it directly instead - changes reload automatically:
```json
{
"models": {
"providers": {
"pangolin": {
"baseUrl": "<endpoint>/v1",
"apiKey": "<key>",
"api": "openai-completions",
"models": [
{
"id": "<model-id>",
"name": "<model-id> via Pangolin"
}
]
}
}
},
"agents": {
"defaults": {
"model": { "primary": "pangolin/<model-id>" }
}
}
}
```
To switch models later without re-editing the file:
```bash
openclaw models set pangolin/<model-id>
```
## Verify
```bash
openclaw health
openclaw models status
openclaw agent --agent main -m "Hello, what model are you?"
```
For a private resource, keep `--custom-api-key`/`apiKey` and set it to `none`, and make sure the Pangolin client is connected on whichever machine runs the OpenClaw daemon.
content/docs/manage/ai/configure-ai-clients/opencode.mdx
+80
View File
@@ -0,0 +1,80 @@
---
title: "OpenCode"
description: "Connect OpenCode to a Pangolin AI Gateway resource"
---
OpenCode configures each model provider separately, so it can talk to whichever API formats your resource supports — Anthropic Messages, OpenAI Chat/Responses, or both, depending on which providers are attached. See [AI Gateway Overview](/manage/ai/overview) if you haven't set that up yet.
You'll need the resource's URL (its `<endpoint>`) and its API key (`<key>`). Both are on the resource's Keys page.
<include>../../../../snippets/ai-gateway-key.mdx</include>
## Fastest: Pangolin CLI
[Install the Pangolin CLI](/manage/clients/install-client#quick-install-recommended) if you don't have it, then log in:
```bash
pangolin login
```
Configure OpenCode against a resource:
```bash
pangolin configure opencode
```
This prompts you to pick an organization and resource (if you have more than one), then asks which OpenCode provider ID(s) should point at this gateway — defaults to `anthropic,openai`, but you can enter any comma-separated list OpenCode recognizes (e.g. `openrouter,google`). It fetches a key for you if needed and writes `opencode.json` and `auth.json`. To skip the org/resource prompts:
```bash
pangolin configure opencode --resource <resource-niceid-or-domain>
```
To undo it:
```bash
pangolin configure opencode --reset
```
## Manual setup
Merge this into your global `opencode.json` (`~/.config/opencode/opencode.json`, or `$XDG_CONFIG_HOME/opencode/opencode.json` if set):
```json
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"anthropic": {
"options": {
"baseURL": "<endpoint>/v1"
}
},
"openai": {
"options": {
"baseURL": "<endpoint>/v1"
}
}
}
}
```
Then merge your key into `auth.json` (`~/.local/share/opencode/auth.json`, or `$XDG_DATA_HOME/opencode/auth.json` if set):
```json
{
"anthropic": {
"type": "api",
"key": "<key>"
},
"openai": {
"type": "api",
"key": "<key>"
}
}
```
<Warning>
`auth.json` is required even for a private resource. OpenCode refuses to start a provider with no key at all and fails with `OpenAI API key is missing. Pass it using the 'apiKey' parameter or the OPENAI_API_KEY environment variable.` Set the key to `none` rather than leaving the entry out.
</Warning>
Only add entries for the providers your resource actually supports. To point a different OpenCode provider (`openrouter`, `google`, `groq`, etc.) at this gateway, add a matching block under `provider` in `opencode.json` and a matching key in `auth.json`.
content/docs/manage/ai/model-catalog.mdx
+77
View File
@@ -0,0 +1,77 @@
---
title: "Model Catalog"
description: "Known models, pricing for budgets, and how to load a custom catalog"
---
The model catalog is Pangolin's list of known model ids, which provider type owns them, and token pricing. It is used for:
- The **Known Models** picker on a provider's Models tab
- Expanding wildcard allows in `GET /v1/models`
- [Provider selection](/manage/ai/providers/model-routing#provider-selection) when more than one attached provider allows the same id
- **Budgets and usage cost**, which look up `in` / `out` / `cache` / `reasoning` rates for a model key
See [Model Routing](/manage/ai/providers/model-routing) for allow lists, inherit vs select, and how those lists combine with the catalog at request time.
## Catalog Providers
Default source: `https://api.fossorial.io/api/v1/models`. Typed AI provider types map as follows:
| Provider type | Catalog |
|---|---|
| OpenAI | `openai` |
| Anthropic | `anthropic` |
| Google Gemini | `gemini` |
| Vertex AI | `vertex` |
| Amazon Bedrock | `bedrock` |
| Microsoft Foundry | `azure` |
| OpenRouter, Vercel AI Gateway, Custom | (not mapped) |
Unmapped types skip Known Models and wildcard expansion against the catalog. Enter exact keys if you need them listed in discovery.
## Pricing and Budgets
Each catalog entry can include token rates: `in`, `out`, `cache`, and `reasoning`. [Budget](/manage/ai/budgets) enforcement and usage analytics use those rates when the requested model matches a catalog id. Unknown keys have no catalog price, so they do not count toward USD budgets. Add custom models to the catalog if you need cost tracking for them. Token budgets still sum usage for unknown ids. Cost is per token.
## Loading a Custom Catalog
Self-hosted Pangolin loads and refreshes the catalog from [`config.yml`](/self-host/advanced/config-file#ai-model-catalog):
```yaml
ai:
model_catalog:
upstream_url: "https://api.fossorial.io/api/v1/models"
file: "config/ai-models.json"
merge_file: "config/ai-models-extra.json"
refresh_interval_min_hours: 6
refresh_interval_max_hours: 12
```
| Key | Purpose |
|---|---|
| `upstream_url` | HTTP GET for the catalog JSON. Defaults to the Fossorial API. Point this at your own endpoint to serve a custom catalog. |
| `file` | Local JSON file that **replaces** the HTTP catalog. |
| `merge_file` | Local JSON merged on top of the base catalog. Base entries win on duplicates; the merge file only adds missing models. |
| `refresh_interval_min_hours` / `max_hours` | Background refresh window. The interval is jittered between min and max so many instances do not hit the upstream at the same moment. Defaults are 6 and 12. |
If both `file` and `upstream_url` are set, `file` is used as the base. `merge_file` still applies.
Catalog files and API responses are JSON:
```json
{
"data": [
{
"model": "gpt-4o",
"provider": "openai",
"pricing": {
"in": 0.0000188,
"out": 0.0000188,
"cache": null,
"reasoning": null
}
}
]
}
```
`provider` must be `openai`, `anthropic`, `gemini`, `vertex`, `azure`, or `bedrock`. Prefixes `bedrock*`, `vertex*`, and `azure*` are accepted and normalized to those catalogs. Unknown providers are skipped.
content/docs/manage/ai/multiple-gateway-resources.mdx
+67
View File
@@ -0,0 +1,67 @@
---
title: "Multiple Gateway Resources"
description: "Give different users and roles their own providers, model lists, and budgets with more than one AI Gateway resource"
---
You can create more than one [public](/manage/resources/public/ai-gateway) or [private](/manage/resources/private/ai-gateway) AI Gateway resource. Each one has its own users and roles, attached [providers](/manage/ai/providers/overview), model lists, and [budgets](/manage/ai/budgets). That is how the Employees role gets OpenAI and Anthropic while Contractors get a tighter set, without sharing one allow list.
This is Pangolin's existing access control, applied to AI. A gateway resource is still a resource: you assign [users and roles](/manage/access-control/create-user) to it the same way as [public HTTPS](/manage/resources/public/authentication) or a [private host](/manage/resources/private/authentication). Assign a role when a set of people should share access. Assign individual users when the set is smaller or does not match a role. What differs per resource is which providers, models, and budgets it exposes.
## Users and Roles
Create a resource for the people who should share providers. Assign those users, or a role they belong to, then give the resource its own hostname. Clients point at a URL that already means that resource's providers.
For example:
- Resource **Staff AI**: FQDN `ai-staff.example.com`, role Employees, providers OpenAI and Anthropic.
- Resource **Contractor AI**: FQDN `ai-contractors.example.com`, role Contractors, OpenRouter (or a custom endpoint) with a narrower allow list.
Staff point Claude Code, Codex, or another client at `https://ai-staff.example.com`. Contractors use `https://ai-contractors.example.com`. An [identity key](/manage/ai/virtual-api-keys) is still per user; the hostname is what selects the resource.
The same pattern works across public and private: a public resource for agents on the internet, a private resource for people on the Pangolin client.
## Sharing a Hostname
HTTP and HTTPS resources each need their own fully qualified domain name, because Pangolin would otherwise not know which target to send traffic to. AI Gateway resources can share a FQDN. Every one of them routes to the gateway running inside Pangolin, so they always go to the same place.
The hostname is an entrypoint, not a unique backend. Use a shared name when you want the same split as [users and roles](#users-and-roles), but everyone configures one URL.
You can overlap:
- Several public AI Gateway resources
- Several private AI Gateway resources
- Public and private together
An [HTTP / HTTPS](/manage/resources/public/http-https) resource still cannot use that same name.
### How Pangolin Picks a Resource
On each request, Pangolin lists the enabled AI Gateway resources whose FQDN matches the host, then keeps the ones the caller is allowed to use.
- **Public.** [Users and roles](/manage/resources/public/authentication) on the resource control an [identity key](/manage/ai/virtual-api-keys). A [manual key](/manage/ai/virtual-api-keys#manual-keys) must be scoped to that resource, or to all resources.
- **Private.** [Users, roles, or machines](/manage/resources/private/authentication) granted on the resource, using the connected Pangolin client.
A user or role can be granted on more than one resource in the overlapping set. After access filtering, Pangolin picks among the remaining resources the same way it picks a [provider](/manage/ai/providers/model-routing#provider-selection) on a single resource:
1. **Capability.** The path must match a capability an attached provider advertises.
2. **Allow and block.** The requested model must pass the resource's effective lists.
3. **Specificity, catalog ownership, and class.** Exact allow keys beat patterns. A typed catalog owner beats an aggregator. Native typed providers beat aggregators, which beat Custom.
The chosen resource's attached providers, model lists, and resource-scoped budgets apply. Someone granted on both Staff AI and Contractor AI, both at `ai.example.com`, is routed by the model and API they called.
If more than one resource still matches after those steps, the gateway returns the same ambiguous error as overlapping providers: `Model "<id>" is ambiguous across multiple AI providers on this resource`.
When a public resource and a private resource share a host, calls from the internet use the public path (virtual API key). Calls over the Pangolin client tunnel use the private path (client identity).
The steps above assume [users and roles](/manage/resources/public/authentication) decide who can use each overlapping resource, together with virtual API keys or the connected client. If you also add [access rules](/manage/access-control/rules) (IP, path, geolocation, and similar) on those resources, which overlapping resource a request lands on is undefined. Put access rules on resources that have their own hostname.
```mermaid
flowchart LR
client[Client] --> fqdn["ai.example.com"]
fqdn --> gw[Gateway in Pangolin]
gw --> pick{Access then model}
pick --> eng[Staff AI]
pick --> fin[Contractor AI]
eng --> openai[OpenAI and Anthropic]
fin --> cheap[Restricted providers]
```
content/docs/manage/ai/overview.mdx
+83
View File
@@ -0,0 +1,83 @@
---
title: "AI Gateway Overview"
description: "Route coding agents and AI clients through Pangolin with centralized providers and access control"
---
An AI Gateway resource is a normal Pangolin resource that proxies requests to one or more upstream AI providers (OpenAI, Anthropic, Google Gemini, Bedrock, Vertex AI, or a custom endpoint). Point a client at the resource's URL instead of the provider's, and Pangolin handles auth, access control, and logging in front of it. See [Pangolin vs. Bifrost vs. LiteLLM](/about/pangolin-vs-bifrost-vs-litellm) if you are comparing Pangolin to a traditional LLM gateway.
## How It Fits Together
- **[Providers](/manage/ai/providers/overview)** are configured once per organization - the upstream URL, API key, and capabilities for OpenAI, Anthropic, etc.
- **Resources** (type `AI Gateway`) attach one or more of those providers and get a normal Pangolin domain.
- **[Virtual API Keys](/manage/ai/virtual-api-keys)** are what clients send to a **public** AI Gateway resource. Private resources are reached through the Pangolin network, so the gateway does not check a key.
- The **[Model Catalog](/manage/ai/model-catalog)** is the known-model list and token pricing used by [model routing](/manage/ai/providers/model-routing) and budgets.
- **[Budgets](/manage/ai/budgets)** cap estimated USD spend or token usage on a provider, model, resource, role, or key.
- **[Session Logs](/manage/ai/session-logs)** store prompt and response transcripts for each call.
- **[Usage Analytics](/manage/ai/usage-analytics)** charts cost, tokens, and request volume across providers, resources, roles, users, and keys.
A resource only understands the API format(s) its attached providers support. An Anthropic provider makes the resource speak the Anthropic Messages API; an OpenAI provider makes it speak Chat Completions/Responses; a Gemini provider makes it speak Gemini's `generateContent` API. Attach whichever providers match the clients you plan to connect. See [AI Providers](/manage/ai/providers/overview) for per-type setup and [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and model lists.
## 1. Add a Provider
Create an org-level provider so the gateway has an upstream to call. Sidebar → **AI Gateway** → **Providers** → **Create**, or follow the [AI Providers](/manage/ai/providers/overview) guide. Pick a type, paste in its API key, and save. You can add as many as you need.
## 2. Create a Resource
Resources → **Create** → set **Type** to **AI Gateway**, pick a domain, and attach the provider(s) from step 1. You can also attach providers to an existing resource later from its **AI Gateway** settings tab. How public and private AI Gateway resources reach users is covered on the [public](/manage/resources/public/ai-gateway) and [private](/manage/resources/private/ai-gateway) resource type pages.
<Frame>
<img src="/images/ai/create-ai-gateway-resource.png" alt="Create Public Resource form with Type set to AI Gateway" />
</Frame>
You can create more than one AI Gateway resource so different users and roles get their own providers. Distinct hostnames are the usual approach; they can also share a FQDN. See [Multiple Gateway Resources](/manage/ai/multiple-gateway-resources).
## 3. Authenticate to the Gateway
How a client authenticates depends on whether the resource is public or private.
### Private Resources
Reachable only on devices connected with the [Pangolin client](/manage/clients/install-client). Identity comes from that connection, so you do not issue a [virtual API key](/manage/ai/virtual-api-keys). The desktop client already proved who is calling. Details are on the [private AI Gateway](/manage/resources/private/ai-gateway) resource page.
### Public Resources
Reachable from anywhere. The gateway checks a [virtual API key](/manage/ai/virtual-api-keys) on every call. Details are on the [public AI Gateway](/manage/resources/public/ai-gateway) resource page.
## 4. Connect a Coding Agent, Agent, or AI Client to the Gateway
Every resource has ready-to-copy setup for common clients, and `pangolin configure <client>` can write the config for you. Full instructions per client:
| Client | Needs a provider that speaks | Guide |
|---|---|---|
| Claude Code | Anthropic Messages | [Setup guide](/manage/ai/configure-ai-clients/claude) |
| Codex | OpenAI Chat/Responses | [Setup guide](/manage/ai/configure-ai-clients/codex) |
| OpenCode | Anthropic Messages and/or OpenAI Chat | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| Gemini CLI | Gemini `generateContent` | [Setup guide](/manage/ai/configure-ai-clients/gemini) |
There are also guides for pointing other tools at a Pangolin AI Gateway resource:
- [Open WebUI](/manage/ai/configure-ai-clients/open-webui)
- [Claude Desktop](/manage/ai/configure-ai-clients/claude-desktop)
- [OpenClaw](/manage/ai/configure-ai-clients/openclaw)
## Supported Providers
Each type has its own configuration guide. Capabilities (which API formats a provider speaks) are explained in [Provider Configuration](/manage/ai/providers/configuration#capabilities).
| Provider | Default capabilities | Guide |
|---|---|---|
| OpenAI | Chat Completions, Responses | [Configure](/manage/ai/providers/openai) |
| Anthropic | Messages | [Configure](/manage/ai/providers/anthropic) |
| Google Gemini | `generateContent` | [Configure](/manage/ai/providers/google-gemini) |
| Vertex AI | Google `generateContent`/`rawPredict` | [Configure](/manage/ai/providers/vertex-ai) |
| Amazon Bedrock | Converse | [Configure](/manage/ai/providers/bedrock) |
| Microsoft Foundry | Chat Completions, Responses, Messages | [Configure](/manage/ai/providers/microsoft-foundry) |
| OpenRouter | Chat Completions | [Configure](/manage/ai/providers/open-router) |
| Vercel AI Gateway | Chat Completions, Responses | [Configure](/manage/ai/providers/vercel-ai-gateway) |
| Custom | Whatever you select | [Configure](/manage/ai/providers/custom) |
Use [Custom](/manage/ai/providers/custom) for any upstream that isn't in the typed list. Assign whichever capabilities it speaks - OpenAI, Anthropic, Gemini, Vertex, Bedrock, or a mix. A Moonshot AI (Kimi) endpoint is one example: select Anthropic Messages and point at its Anthropic-compatible URL.
<Card title="Pangolin vs. Bifrost vs. LiteLLM" icon="book" href="/about/pangolin-vs-bifrost-vs-litellm">
How Pangolin's identity-aware AI Gateway compares to traditional LLM gateways.
</Card>
content/docs/manage/ai/providers/anthropic.mdx
+41
View File
@@ -0,0 +1,41 @@
---
title: "Anthropic"
description: "Connect Anthropic as an AI Gateway provider"
---
An Anthropic provider forwards Messages requests to Anthropic's API. Attach it to an AI Gateway resource when you want Claude Code, Claude Desktop, OpenCode, OpenClaw, or any other Anthropic-compatible client to use your org's Anthropic key.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | `https://api.anthropic.com` |
| Auth type | `x-api-key` |
| Capabilities | Anthropic Messages, Anthropic Models |
Anthropic Models (`GET /v1/models`) is answered from this provider's allow and block lists. See [Capabilities](/manage/ai/providers/configuration#capabilities).
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Anthropic**. Name, capabilities, upstream URL, and auth type fill in from the defaults above.
3. Paste your Anthropic API key.
4. Set **Allow** and **Block** lists. `*` allows every model key; `claude-*` is a typical narrower pattern.
5. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
Leave the upstream URL as the default unless you are pointing at a compatible Anthropic-style base URL.
<Tip>
Clients can send `anthropic-version` themselves. You can also set it as a [custom header](/manage/ai/providers/configuration#headers-and-tls) on the provider (`anthropic-version: 2023-06-01`). A header set on the provider is always sent upstream and overrides the same header from the client.
</Tip>
## Clients
| Client | Guide |
|---|---|
| Claude Code | [Setup guide](/manage/ai/configure-ai-clients/claude) |
| Claude Desktop | [Setup guide](/manage/ai/configure-ai-clients/claude-desktop) |
| OpenCode | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| OpenClaw | [Setup guide](/manage/ai/configure-ai-clients/openclaw) |
content/docs/manage/ai/providers/bedrock.mdx
+33
View File
@@ -0,0 +1,33 @@
---
title: "Amazon Bedrock"
description: "Connect Amazon Bedrock as an AI Gateway provider"
---
An Amazon Bedrock provider forwards Converse requests to Bedrock Runtime in the region you configure. Use it when clients call Bedrock's Converse API.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | You enter the regional Bedrock Runtime base URL |
| Auth type | Bearer |
| Capabilities | Bedrock Converse |
Bedrock Model Invoke (`InvokeModel` / `invoke-with-response-stream`) is available as an extra capability if the client uses that API instead of Converse.
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Amazon Bedrock**. Capabilities and auth type fill in from the defaults above.
3. Set **Upstream URL** to your regional Bedrock Runtime host, for example `https://bedrock-runtime.us-east-1.amazonaws.com`.
4. Paste a Bearer credential the upstream accepts. Pangolin sends `Authorization: Bearer <key>` on every request.
5. Set **Allow** and **Block** lists. Use Bedrock model ids (for example `anthropic.claude-sonnet-4-20250514-v1:0` or an inference profile id).
6. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
Add **Bedrock Model Invoke** on the **General** tab if the client calls `/model/{id}/invoke` rather than Converse. Add other capabilities only if this endpoint actually speaks those API formats.
## Clients
Clients that call Bedrock Converse work with the default capability. Claude Code talks Anthropic Messages (`POST /v1/messages`), so it needs a provider that advertises Anthropic Messages, such as [Anthropic](/manage/ai/providers/anthropic), [Microsoft Foundry](/manage/ai/providers/microsoft-foundry), or a [custom](/manage/ai/providers/custom) Anthropic-compatible endpoint.
content/docs/manage/ai/providers/configuration.mdx
+93
View File
@@ -0,0 +1,93 @@
---
title: "Provider Configuration"
description: "Capabilities, auth, routing, and model lists for AI Gateway providers"
---
This page is the field reference for every AI provider type. Type-specific defaults (URL, auth header, capabilities) live on each provider page. See [AI Providers](/manage/ai/providers/overview) for what a provider is and how it attaches to a resource.
Sidebar → **AI Gateway** → **Providers** → **Create**. After you save, the provider has tabs for **General**, **Network**, **Models**, **Authentication**, and **Budget**.
## Capabilities
A capability is an API format the gateway will accept and proxy. The incoming request path selects a capability, and only attached providers that advertise that capability can handle the request. That is why Claude Code needs a provider with Anthropic Messages, and Codex needs a provider with OpenAI Chat Completions or Responses.
| Capability | What the gateway accepts |
|---|---|
| OpenAI Chat Completions | `POST /v1/chat/completions` |
| OpenAI Responses | `POST /v1/responses` |
| Anthropic Messages | `POST /v1/messages` |
| Anthropic Models | `GET /v1/models` (answered from the gateway's allow and block lists, not proxied upstream) |
| Gemini Generate Content | Gemini `generateContent` / `streamGenerateContent` |
| Vertex Generate Content | Vertex AI Gemini format |
| Vertex Raw Predict | Vertex AI `rawPredict` for Anthropic models |
| Bedrock Converse | Amazon Bedrock Converse API |
| Bedrock Model Invoke | Amazon Bedrock InvokeModel |
Typed providers start with recommended capabilities. Custom providers need at least one selected. You can change capabilities later on the provider's **General** tab. Select every format the upstream actually speaks. Custom providers can mix any combination.
If two attached providers share a capability for the same model, the gateway picks one at request time. See [Model Routing](/manage/ai/providers/model-routing#provider-selection).
## Auth Type
How the gateway authenticates to the upstream. Typed providers prefill the header that upstream expects.
| Auth type | What Pangolin sends |
|---|---|
| Bearer | `Authorization: Bearer <key>`. Used by OpenAI and most compatible APIs. |
| `x-api-key` | `x-api-key` header. Used by Anthropic and Anthropic-compatible APIs. |
| `x-goog-api-key` | `x-goog-api-key` header. Used by Google Gemini. |
| Cloudflare AI Gateway | `cf-aig-authorization: Bearer <key>`. |
| Splunk HEC | `Authorization: Splunk <key>`. |
| Passthrough | Forward the caller's remaining API key headers to the upstream (after Pangolin strips its own virtual key). |
| No Auth | Send the request without authentication headers. |
Bearer, `x-api-key`, `x-goog-api-key`, Cloudflare AI Gateway, and Splunk HEC all require an API key on the provider.
## Routing
Where the gateway sends the request after it has selected this provider.
| Mode | When to use |
|---|---|
| **Upstream URL** | Call a public or private API base URL. Typed providers with a public API prefill this. Vertex AI, Amazon Bedrock, and Microsoft Foundry ask you to enter a regional or resource base URL. |
| **Site Targets** | Route through HTTP targets on your Pangolin sites, the same way a public resource reaches an internal service. Available on [Custom](/manage/ai/providers/custom) providers only. |
If you choose **Site Targets**, configure the targets on the provider's **Network** tab after you save.
See [Ollama](/manage/ai/providers/custom/ollama), [vLLM](/manage/ai/providers/custom/vllm), [Bifrost](/manage/ai/providers/custom/bifrost), and [Cloud APIs Through a Site](/manage/ai/providers/custom/cloud-apis-through-a-site) for Site Targets walkthroughs.
## Models
Allow and block lists, inherit vs select, and provider selection are covered in [Model Routing](/manage/ai/providers/model-routing). Known Models, ownership scoring, and token pricing for budgets come from the [Model Catalog](/manage/ai/model-catalog).
## Budget
The provider **Budget** tab caps estimated USD spend or token usage for every call that uses this provider. You can also set a budget on an individual allow-list model from the **Models** tab. See [Budgets](/manage/ai/budgets) for scopes, periods, and the 429 clients receive when a cap is hit.
## Headers and TLS
On **Network** you can also set:
- **Custom headers** sent on every request to this provider (`Header-Name: value`, one per line)
- **Skip TLS verification** for the upstream connection
### Identity Headers
When Pangolin knows which user made the request, it forwards that identity to the upstream on every call:
| Header | Value |
|---|---|
| `Remote-User` | Username |
| `Remote-Email` | Email, when set |
| `Remote-Name` | Display name, when set |
| `Remote-Role` | Org role names, when the user has roles |
The user is known when:
- A public resource is called with an [identity key](/manage/ai/virtual-api-keys#identity-keys)
- A public resource is called with a [manual key](/manage/ai/virtual-api-keys#manual-keys) attributed to a user
- A private AI Gateway resource is called from a connected [Pangolin client](/manage/clients/install-client), and that client maps to a user
An unattributed manual key still authenticates, but these headers are omitted. Empty values are omitted rather than sent blank.
These are the same names as [Forwarded Headers](/manage/access-control/forwarded-headers) on HTTPS resources. A Custom upstream such as [Bifrost](/manage/ai/providers/custom/bifrost) can use them for downstream access control. They are sent in addition to any custom headers you configure on the provider.
content/docs/manage/ai/providers/custom.mdx
+57
View File
@@ -0,0 +1,57 @@
---
title: "Custom"
description: "Connect any upstream endpoint and assign whichever API capabilities it speaks"
---
A custom provider is for an upstream that is not in the typed list: a self-hosted model server, a vendor API, a proxy, or an internal service reached through a Pangolin site. You assign any mix of [capabilities](/manage/ai/providers/configuration#capabilities). The gateway will accept those API formats on resources that attach this provider.
Custom providers start blank. You choose capabilities, [routing](/manage/ai/providers/configuration#routing), and [auth](/manage/ai/providers/configuration#auth-type). At least one capability is required. Site Targets routing is available on Custom only.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before, and [Provider Configuration](/manage/ai/providers/configuration) for the field reference.
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Custom** and give it a name.
3. Select every **API capability** the upstream speaks. You can add any combination.
4. Choose **Routing Mode**. For **Upstream URL**, paste the base URL. For **Site Targets**, add targets after save.
5. Pick **Auth Type** and paste a key if that type needs one.
6. Set **Allow** and **Block** lists. Add exact model keys if you want them listed in `GET /v1/models`. See [Model Routing](/manage/ai/providers/model-routing).
7. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
## Reach a Self-Hosted API
Site Targets send the request over a Pangolin tunnel to a host the site can reach. [Install a site](/manage/sites/install-site) on the same network as the model server so the gateway can route to that API over the tunnel. See [Understanding Sites](/manage/sites/understanding-sites).
After the site is connected, add the process as a target on the provider's **Network** tab (host, port, HTTP or HTTPS). Worked examples: [Ollama](/manage/ai/providers/custom/ollama), [vLLM](/manage/ai/providers/custom/vllm), and [Bifrost](/manage/ai/providers/custom/bifrost).
<Note>
AI Gateway providers are only routed to the Pangolin server nodes. Sites
that are connected to remote nodes will not be accessible to be routed to as
targets on providers. DNS resolution of gateway resources will not point at
remote nodes. Users using gateway resources with custom targets on sites may
wish to create a separate org for these resources so sites are able to
connect to regular nodes.
</Note>
## Examples
Worked Custom setups:
- [Ollama](/manage/ai/providers/custom/ollama): local models on a site, OpenAI Chat Completions
- [vLLM](/manage/ai/providers/custom/vllm): a vLLM OpenAI server on a site
- [Bifrost](/manage/ai/providers/custom/bifrost): downstream model routing with Pangolin identity headers
- [Cloud APIs Through a Site](/manage/ai/providers/custom/cloud-apis-through-a-site): send OpenAI or Anthropic out through a site so the vendor sees that site's IP
The same Custom pattern works for llama.cpp server, LM Studio, LocalAI, and other OpenAI-compatible listeners: pick Chat Completions, point Site Targets at the process, and allow the model ids it serves.
## Example: Kimi K2
This is one custom setup: Moonshot AI's Kimi K2 endpoint speaks the Anthropic API, so you select Anthropic capabilities. A different upstream would get a different capability set.
1. Create a **Custom** provider.
2. Select **Anthropic Messages** and **Anthropic Models**.
3. Set routing to **Upstream URL** and use Moonshot's Anthropic-compatible base URL.
4. Set auth to **`x-api-key`** and paste the Moonshot key.
5. Allow the Kimi model ids you plan to call.
6. Attach the provider to a resource, then point [Claude Code](/manage/ai/configure-ai-clients/claude) or [Claude Desktop](/manage/ai/configure-ai-clients/claude-desktop) at that resource.
content/docs/manage/ai/providers/custom/bifrost.mdx
+27
View File
@@ -0,0 +1,27 @@
---
title: "Bifrost"
description: "Point a Custom provider at Bifrost for downstream model routing and identity-aware access control"
---
[Bifrost](https://www.getmaxim.ai/bifrost) is an OpenAI-compatible LLM gateway that routes to many upstream model providers. Use it as a Custom AI Gateway provider when Pangolin should authenticate clients and Bifrost should pick models, fail over, or apply its own access controls.
Pangolin talks to Bifrost at `/v1/chat/completions` (default port `8080`). Add [Anthropic Messages](/manage/ai/providers/configuration#capabilities) as well if clients will call Bifrost's Anthropic-compatible paths.
See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full capability list.
[Install a site](/manage/sites/install-site) on the same network as Bifrost so Pangolin can reach it over the tunnel. See [Reach a Self-Hosted API](/manage/ai/providers/custom#reach-a-self-hosted-api).
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Custom** and give it a name.
3. Select **OpenAI Chat Completions** (and any other formats Bifrost should accept).
4. Set **Routing Mode** to **Site Targets** if Bifrost runs on a site's network. Use **Upstream URL** (`http://<host>:8080`) only when the Pangolin node can reach Bifrost directly.
5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if Bifrost requires a virtual key or other API key.
6. Allow the model ids Bifrost should serve, including provider-prefixed keys such as `openai/gpt-4o-mini` if that is how Bifrost names them. See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the Bifrost host, port `8080`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
Then point a client at that resource: [Codex](/manage/ai/configure-ai-clients/codex), [OpenCode](/manage/ai/configure-ai-clients/opencode), or [Open WebUI](/manage/ai/configure-ai-clients/open-webui). Clients still call the Pangolin resource. Bifrost only sees the request after Pangolin has authenticated it.
When the caller is a known user, Pangolin forwards [`Remote-User`, `Remote-Email`, `Remote-Name`, and `Remote-Role`](/manage/ai/providers/configuration#identity-headers). Use those in Bifrost (or any Custom upstream) for downstream access control. An unattributed [manual key](/manage/ai/virtual-api-keys#manual-keys) authenticates without sending them.
content/docs/manage/ai/providers/custom/cloud-apis-through-a-site.mdx
+34
View File
@@ -0,0 +1,34 @@
---
title: "Cloud APIs Through a Site"
description: "Send OpenAI or Anthropic traffic out through a Pangolin site so the vendor sees a fixed egress IP"
---
Typed [OpenAI](/manage/ai/providers/openai) and [Anthropic](/manage/ai/providers/anthropic) providers send requests from the Pangolin node's own egress. A [Custom](/manage/ai/providers/custom) provider with **Site Targets** sends the same APIs out through a chosen site, so OpenAI or Anthropic see that site's public IP.
Use this when the vendor allowlists IPs, or all inference must leave from one network. The site is only the egress path. Auth still lives on the provider: paste the real vendor key.
Those typed provider types use **Upstream URL** only. This walkthrough uses Custom plus targets.
See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full capability list.
## Configure
1. Pick a [site](/manage/sites/understanding-sites) that can reach the public internet. That box's public IP is what the vendor will see.
2. Sidebar → **AI Gateway** → **Providers** → **Create**. Set **Provider Type** to **Custom**.
3. Select capabilities that match the cloud API:
- OpenAI: **OpenAI Chat Completions** and **OpenAI Responses**
- Anthropic: **Anthropic Messages** and **Anthropic Models**
4. Set **Routing Mode** to **Site Targets**.
5. Set **Auth Type** to **Bearer** for OpenAI, or **`x-api-key`** for Anthropic, and paste the vendor API key.
6. Allow the model ids you plan to call, then save.
7. On the **Network** tab, add a target on that site. Use the API hostname as the address so TLS Host and SNI match:
| Vendor | Address | Port | Method |
|---|---|---|---|
| OpenAI | `api.openai.com` | `443` | HTTPS |
| Anthropic | `api.anthropic.com` | `443` | HTTPS |
The target address is sent as the Host header.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
The site must resolve and reach that hostname. Clients still point at the resource URL, not at OpenAI or Anthropic directly: [Codex](/manage/ai/configure-ai-clients/codex), [Claude Code](/manage/ai/configure-ai-clients/claude), and the other [client guides](/manage/ai/overview#4-connect-a-client).
content/docs/manage/ai/providers/custom/ollama.mdx
+25
View File
@@ -0,0 +1,25 @@
---
title: "Ollama"
description: "Point a Custom provider at Ollama on a Pangolin site"
---
[Ollama](https://ollama.com) serves local models with an OpenAI-compatible HTTP API. Create a [Custom](/manage/ai/providers/custom) provider and route to that API so Codex, OpenCode, Open WebUI, and other Chat Completions clients can use models running on a site.
Pangolin talks to Ollama at `/v1/chat/completions`. Use that OpenAI-compatible listener, not Ollama's native `/api/chat` path.
See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full capability list.
[Install a site](/manage/sites/install-site) on the same network as Ollama so Pangolin can reach it over the tunnel. See [Reach a Self-Hosted API](/manage/ai/providers/custom#reach-a-self-hosted-api).
## Configure
1. Sidebar > **AI Gateway** > **Providers** > **Create**.
2. Set **Provider Type** to **Custom** and give it a name.
3. Select **OpenAI Chat Completions**.
4. Set **Routing Mode** to **Site Targets** if Ollama runs on a site's network. Use **Upstream URL** (`http://<host>:11434`) only when the Pangolin node can reach Ollama directly.
5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if you set an Ollama API key.
6. Allow the exact model ids you plan to call, for example `llama3.2`. See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the Ollama host, port `11434`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
Then point a client at that resource: [Codex](/manage/ai/configure-ai-clients/codex), [OpenCode](/manage/ai/configure-ai-clients/opencode), or [Open WebUI](/manage/ai/configure-ai-clients/open-webui).
content/docs/manage/ai/providers/custom/vllm.mdx
+25
View File
@@ -0,0 +1,25 @@
---
title: "vLLM"
description: "Point a Custom provider at a vLLM OpenAI server on a Pangolin site"
---
[vLLM](https://docs.vllm.ai) can expose an OpenAI-compatible HTTP server for local or GPU-hosted models. Create a [Custom](/manage/ai/providers/custom) provider and route to that server so Codex, OpenCode, Open WebUI, and other Chat Completions clients can use it.
Pangolin talks to vLLM at `/v1/chat/completions` on the OpenAI server (default port `8000`).
See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full capability list.
[Install a site](/manage/sites/install-site) on the same network as vLLM so Pangolin can reach it over the tunnel. See [Reach a Self-Hosted API](/manage/ai/providers/custom#reach-a-self-hosted-api).
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Custom** and give it a name.
3. Select **OpenAI Chat Completions**.
4. Set **Routing Mode** to **Site Targets** if vLLM runs on a site's network. Use **Upstream URL** (`http://<host>:8000`) only when the Pangolin node can reach vLLM directly.
5. Set **Auth Type** to **No Auth** if the server has no key. Use **Bearer** and paste the key if you started vLLM with `--api-key`.
6. Allow the exact served model id (the `--model` name or `--served-model-name`). See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the vLLM host, port `8000`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
Then point a client at that resource: [Codex](/manage/ai/configure-ai-clients/codex), [OpenCode](/manage/ai/configure-ai-clients/opencode), or [Open WebUI](/manage/ai/configure-ai-clients/open-webui).
content/docs/manage/ai/providers/google-gemini.mdx
+34
View File
@@ -0,0 +1,34 @@
---
title: "Google Gemini"
description: "Connect Google Gemini as an AI Gateway provider"
---
A Google Gemini provider forwards `generateContent` requests to Google's Gemini API. Attach it to an AI Gateway resource when you want Gemini CLI (or another client that speaks the Gemini API) to use your org's Gemini key.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
This type is the Gemini Developer API (`generativelanguage.googleapis.com`). For Vertex AI's `generateContent` / `rawPredict` paths, use [Vertex AI](/manage/ai/providers/vertex-ai) instead.
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | `https://generativelanguage.googleapis.com` |
| Auth type | `x-goog-api-key` |
| Capabilities | Gemini Generate Content |
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Google Gemini**. Name, capabilities, upstream URL, and auth type fill in from the defaults above.
3. Paste your Gemini API key.
4. Set **Allow** and **Block** lists. `*` allows every model key; `gemini-*` is a typical narrower pattern.
5. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
Leave the upstream URL as the default unless you are pointing at a compatible Gemini API base URL.
## Clients
| Client | Guide |
|---|---|
| Gemini CLI | [Setup guide](/manage/ai/configure-ai-clients/gemini) |
content/docs/manage/ai/providers/microsoft-foundry.mdx
+38
View File
@@ -0,0 +1,38 @@
---
title: "Microsoft Foundry"
description: "Connect Microsoft Foundry as an AI Gateway provider"
---
A Microsoft Foundry provider forwards OpenAI-format and Anthropic-format requests to your Azure AI Foundry (or Azure OpenAI) resource. One Foundry provider can serve both Codex-style clients and Claude Code, because the defaults include Chat Completions, Responses, and Anthropic Messages.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | You enter your Foundry or Azure OpenAI resource base URL |
| Auth type | Bearer |
| Capabilities | OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Anthropic Models |
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Microsoft Foundry**. Capabilities and auth type fill in from the defaults above.
3. Set **Upstream URL** to the resource base, for example `https://<resource-name>.services.ai.azure.com` or `https://<resource-name>.openai.azure.com`.
4. Paste a Bearer credential the resource accepts (typically the Azure API key).
5. Set **Allow** and **Block** lists. Use the deployment or model names the Foundry resource exposes.
6. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
Leave capabilities at the defaults if you want both OpenAI-compatible and Anthropic-compatible clients on the same resource. Trim them if this Foundry resource only hosts one API family.
## Clients
| Client | Guide |
|---|---|
| Codex | [Setup guide](/manage/ai/configure-ai-clients/codex) |
| Claude Code | [Setup guide](/manage/ai/configure-ai-clients/claude) |
| OpenCode | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| Open WebUI | [Setup guide](/manage/ai/configure-ai-clients/open-webui) |
| Claude Desktop | [Setup guide](/manage/ai/configure-ai-clients/claude-desktop) |
| OpenClaw | [Setup guide](/manage/ai/configure-ai-clients/openclaw) |
content/docs/manage/ai/providers/model-routing.mdx
+78
View File
@@ -0,0 +1,78 @@
---
title: "Model Routing"
description: "Allow and block lists, inherit vs select, and how the gateway picks a provider"
---
Model routing is how Pangolin decides which attached provider handles a request, and which model keys that provider may serve. Configure lists on the provider, optionally narrow them on the resource, then the gateway picks a provider per request.
See [AI Providers](/manage/ai/providers/overview) for what a provider is, and [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and routing mode. The [Model Catalog](/manage/ai/model-catalog) supplies Known Models, wildcard discovery, ownership scoring, and pricing for budgets.
## Choosing Models on a Provider
On the provider's **Models** tab, every request must match an **allow** entry and must not match a **block** entry. An empty allow list denies all traffic. A key cannot sit on both lists.
<Frame>
<img src="/images/ai/public-resource-models.png" alt="Provider Models tab with allow and block lists for Anthropic" />
</Frame>
You add keys three ways:
| How | What it is | Example |
|---|---|---|
| **[Catalog (Known Models)](/manage/ai/model-catalog)** | Pick from the catalog for that provider type | `gpt-4o`, `claude-sonnet-4-6` |
| **Custom key** | Type an exact model id the upstream accepts | `llama3.2`, `openai/gpt-4o-mini` |
| **Pattern** | Glob with `*` and `?` | `gpt-*`, `claude-3-5-sonnet-?`, `*` |
`*` matches every model key. Typed providers (OpenAI, Anthropic, Google Gemini, Vertex AI, Amazon Bedrock, Microsoft Foundry) show Known Models from the catalog. [OpenRouter](/manage/ai/providers/open-router), [Vercel AI Gateway](/manage/ai/providers/vercel-ai-gateway), and [Custom](/manage/ai/providers/custom) skip that list; enter custom keys or patterns.
### Allow vs Block
Allow is the positive set. Block subtracts even when allow matched. Allow `*` and block `gpt-4o-mini` to permit every key except that one. Allow `gpt-*` and leave block empty to permit only keys that start with `gpt-`.
### Model Discovery
`GET /v1/models` is answered from these lists plus the catalog. The gateway does not proxy that listing call upstream.
- Exact allow keys are listed as themselves.
- Patterns expand against the catalog for typed providers (`claude-*` becomes every matching catalog id).
- On OpenRouter, Vercel AI Gateway, and Custom, add exact allow keys to make those models show up in discovery. A wildcard still matches at request time.
A block pattern hides a model from discovery the same way it would reject the request.
## Inherit vs Select
On an AI Gateway resource, attach providers from the **AI Gateway** settings tab. Each attachment has an access mode:
| Mode | What the resource uses |
|---|---|
| **Inherit** | The provider's allow and block lists as configured on the provider |
| **Select** | A subset of that provider's allow-list models, chosen on this resource |
The same provider can be inherited on one resource and selected on another. You can disable an attachment without removing the provider.
Select only offers models already on the provider's allow list. Use inherit when every resource should see the same keys. Use select when one gateway should expose a narrower set (for example a public resource with `gpt-4o` only, while the provider still allows `gpt-*`).
## Provider Selection
When a resource has more than one attached provider, the gateway picks one per request. Overlapping allows are allowed at save time. Collisions are resolved, or rejected, when the request arrives.
Steps, in order. Later steps run only while more than one candidate remains:
1. **Capability.** The path must match a [capability](/manage/ai/providers/configuration#capabilities) the provider advertises.
2. **Allow and block.** The effective lists (inherit or select) must pass.
3. **Specificity.** The most specific matching allow wins: exact keys beat patterns, fewer wildcards win, longer literals win. `gpt-4o` beats `gpt-*` beats `*`.
4. **Catalog ownership.** A typed provider whose catalog contains the id beats an aggregator that also allows it.
5. **Class.** Native typed providers beat aggregators (OpenRouter, Vercel AI Gateway), which beat Custom.
6. **Ambiguous.** If more than one distinct provider remains, the gateway returns `403` with `Model "<id>" is ambiguous across multiple AI providers on this resource`.
Typical ambiguous cases: two OpenAI providers both allowing `*`, or two aggregators both allowing `*` with no native owner on the resource.
`GET /v1/models` skips steps 3-6. There is no requested model to disambiguate, so the gateway returns the union of what every attached provider advertising Anthropic Models would accept.
### Examples
**OpenAI + Anthropic, both `*`.** `POST /v1/chat/completions` with `gpt-4o` goes to OpenAI. Anthropic never reaches scoring; capability alone decides. `POST /v1/messages` with a Claude model goes to Anthropic.
**OpenAI + OpenRouter, both `*`.** `gpt-4o` on Chat Completions matches both. Specificity is a tie (`*` vs `*`). Catalog ownership sends it to OpenAI.
**Two OpenAI providers, both `*`.** Capability, lists, specificity, catalog, and class all leave both candidates. Result is the ambiguous error. Narrow at least one allow list, disable one attachment, or split them across resources.
content/docs/manage/ai/providers/open-router.mdx
+39
View File
@@ -0,0 +1,39 @@
---
title: "OpenRouter"
description: "Connect OpenRouter as an AI Gateway provider"
---
An OpenRouter provider forwards Chat Completions requests to OpenRouter. Use it as an aggregator when you want one key to reach many upstream models through OpenAI Chat Completions.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | `https://openrouter.ai/api/v1` |
| Auth type | Bearer |
| Capabilities | OpenAI Chat Completions |
OpenRouter's default capability is Chat Completions only. Clients that require Responses or Anthropic Messages need a different provider, extra capabilities this OpenRouter account actually supports, or a [custom](/manage/ai/providers/custom) endpoint.
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **OpenRouter**. Name, capabilities, upstream URL, and auth type fill in from the defaults above.
3. Paste your OpenRouter API key.
4. Set **Allow** and **Block** lists. OpenRouter model keys are often vendor-prefixed, for example `openai/gpt-4o` or `anthropic/claude-sonnet-4`.
5. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
## Models
OpenRouter is an aggregator. See [Model Routing](/manage/ai/providers/model-routing) for how allow lists and `GET /v1/models` discovery work. Add exact keys such as `openai/gpt-4o` or `anthropic/claude-sonnet-4`.
## Clients
| Client | Guide |
|---|---|
| Codex | [Setup guide](/manage/ai/configure-ai-clients/codex) |
| OpenCode | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| Open WebUI | [Setup guide](/manage/ai/configure-ai-clients/open-webui) |
| OpenClaw | [Setup guide](/manage/ai/configure-ai-clients/openclaw) |
content/docs/manage/ai/providers/openai.mdx
+35
View File
@@ -0,0 +1,35 @@
---
title: "OpenAI"
description: "Connect OpenAI as an AI Gateway provider"
---
An OpenAI provider forwards Chat Completions and Responses requests to OpenAI's API. Attach it to an AI Gateway resource when you want Codex, OpenCode, Open WebUI, OpenClaw, or any other OpenAI-compatible client to use your org's OpenAI key.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | `https://api.openai.com/v1` |
| Auth type | Bearer |
| Capabilities | OpenAI Chat Completions, OpenAI Responses |
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **OpenAI**. Name, capabilities, upstream URL, and auth type fill in from the defaults above.
3. Paste your OpenAI API key.
4. Set **Allow** and **Block** lists. `*` allows every model key; `gpt-*` is a typical narrower pattern.
5. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
Leave the upstream URL as the default unless you are pointing at a compatible OpenAI-style base URL.
## Clients
| Client | Guide |
|---|---|
| Codex | [Setup guide](/manage/ai/configure-ai-clients/codex) |
| OpenCode | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| Open WebUI | [Setup guide](/manage/ai/configure-ai-clients/open-webui) |
| OpenClaw | [Setup guide](/manage/ai/configure-ai-clients/openclaw) |
content/docs/manage/ai/providers/overview.mdx
+47
View File
@@ -0,0 +1,47 @@
---
title: "AI Providers"
description: "Connect upstream model APIs once per organization, then attach them to AI Gateway resources"
---
A provider is an organization-level connection to an upstream model API. It stores the provider type, credentials, base URL, API capabilities, and optional model allow and block lists. Create providers once, then attach them to any AI Gateway resource in the org. Clients call the resource's domain, not the provider directly.
See [AI Gateway Overview](/manage/ai/overview) for creating a resource and issuing keys. See [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and routing. See [Model Routing](/manage/ai/providers/model-routing) for allow lists, inherit vs select, and how the gateway picks a provider. See [Model Catalog](/manage/ai/model-catalog) for known models and pricing used by routing and budgets.
## What a Provider Is
Sidebar → **AI Gateway** → **Providers** → **Create**.
Each provider is:
- A **type** (OpenAI, Anthropic, Google Gemini, and others) that fills in recommended defaults
- **Credentials** and a **base URL** the gateway uses when it forwards a request
- **Capabilities** that declare which API formats the provider can handle
- Optional **allow** and **block** lists that restrict which model keys may be used
Providers are reused across resources. The resource still owns the public URL and the keys clients authenticate with.
## Attach to a Resource
On an AI Gateway resource, attach one or more providers from the **AI Gateway** settings tab. For each attachment you can inherit the provider's model lists or select a subset. Details are in [Model Routing](/manage/ai/providers/model-routing#inherit-vs-select).
The resource only speaks the API formats its attached providers advertise. Attach whichever providers match the [clients](/manage/ai/overview#4-connect-a-client) you plan to connect.
<Frame>
<img src="/images/ai/public-resource-providers.png" alt="AI Gateway resource settings with attached providers" />
</Frame>
## Provider Types
| Provider | Default capabilities | Guide |
|---|---|---|
| OpenAI | Chat Completions, Responses | [Configure OpenAI](/manage/ai/providers/openai) |
| Anthropic | Messages, Models | [Configure Anthropic](/manage/ai/providers/anthropic) |
| Google Gemini | Gemini Generate Content | [Configure Google Gemini](/manage/ai/providers/google-gemini) |
| Vertex AI | Vertex Generate Content, Raw Predict | [Configure Vertex AI](/manage/ai/providers/vertex-ai) |
| Amazon Bedrock | Bedrock Converse | [Configure Amazon Bedrock](/manage/ai/providers/bedrock) |
| Microsoft Foundry | Chat Completions, Responses, Messages, Models | [Configure Microsoft Foundry](/manage/ai/providers/microsoft-foundry) |
| OpenRouter | Chat Completions | [Configure OpenRouter](/manage/ai/providers/open-router) |
| Vercel AI Gateway | Chat Completions, Responses | [Configure Vercel AI Gateway](/manage/ai/providers/vercel-ai-gateway) |
| Custom | Whatever you select | [Configure Custom](/manage/ai/providers/custom) |
Custom also covers local engines, downstream routers, and cloud APIs sent through a site. See [Ollama](/manage/ai/providers/custom/ollama), [vLLM](/manage/ai/providers/custom/vllm), [Bifrost](/manage/ai/providers/custom/bifrost), and [Cloud APIs Through a Site](/manage/ai/providers/custom/cloud-apis-through-a-site).
content/docs/manage/ai/providers/vercel-ai-gateway.mdx
+37
View File
@@ -0,0 +1,37 @@
---
title: "Vercel AI Gateway"
description: "Connect Vercel AI Gateway as an AI Gateway provider"
---
A Vercel AI Gateway provider forwards Chat Completions and Responses requests to Vercel's AI Gateway. Use it as an aggregator when you want one key to reach models Vercel exposes through OpenAI-compatible APIs.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | `https://ai-gateway.vercel.sh/v1` |
| Auth type | Bearer |
| Capabilities | OpenAI Chat Completions, OpenAI Responses |
## Configure
1. Sidebar → **AI Gateway** → **Providers** → **Create**.
2. Set **Provider Type** to **Vercel AI Gateway**. Name, capabilities, upstream URL, and auth type fill in from the defaults above.
3. Paste your Vercel AI Gateway API key.
4. Set **Allow** and **Block** lists. Use the model ids Vercel documents for the gateway.
5. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
## Models
Vercel AI Gateway is an aggregator. See [Model Routing](/manage/ai/providers/model-routing) for how allow lists and `GET /v1/models` discovery work. Add exact model keys to the allow list.
## Clients
| Client | Guide |
|---|---|
| Codex | [Setup guide](/manage/ai/configure-ai-clients/codex) |
| OpenCode | [Setup guide](/manage/ai/configure-ai-clients/opencode) |
| Open WebUI | [Setup guide](/manage/ai/configure-ai-clients/open-webui) |
| OpenClaw | [Setup guide](/manage/ai/configure-ai-clients/openclaw) |
content/docs/manage/ai/providers/vertex-ai.mdx
+35
View File
@@ -0,0 +1,35 @@
---
title: "Vertex AI"
description: "Connect Google Vertex AI as an AI Gateway provider"
---
A Vertex AI provider forwards Vertex `generateContent` and `rawPredict` requests to your Google Cloud project. Use it when clients call Vertex AI publisher-model paths rather than the Gemini Developer API.
See [AI Providers](/manage/ai/providers/overview) if you haven't created a provider before. Field options are in [Provider Configuration](/manage/ai/providers/configuration).
For Google's Gemini Developer API, use [Google Gemini](/manage/ai/providers/google-gemini) instead.
## Defaults
| Setting | Value |
|---|---|
| Upstream URL | You enter the regional Vertex AI base URL |
| Auth type | Bearer |
| Capabilities | Vertex Generate Content, Vertex Raw Predict |
Vertex Generate Content covers Gemini-format calls on Vertex. Vertex Raw Predict covers Anthropic models hosted on Vertex (`rawPredict` / `streamRawPredict`).
## Configure
1. Sidebar > **AI Gateway** > **Providers** > **Create**.
2. Set **Provider Type** to **Vertex AI**. Capabilities and auth type fill in from the defaults above.
3. Set **Upstream URL** to your regional Vertex host, for example `https://us-central1-aiplatform.googleapis.com`. The host includes the location.
4. Paste a Bearer credential the Vertex endpoint accepts (typically a Google OAuth access token).
5. Set **Allow** and **Block** lists. Use the model ids Vertex expects, including publisher-qualified keys when that is how the client sends them.
6. Save, then attach the provider to an [AI Gateway resource](/manage/ai/overview).
The gateway authenticates with `Authorization: Bearer <key>`. Add extra capabilities on the **General** tab only if this Vertex endpoint actually speaks those API formats.
## Clients
Clients that call Vertex `generateContent` or `rawPredict` paths work with the default capabilities. Gemini CLI talks the Gemini Developer API (`gemini_generate_content`), so it needs a [Google Gemini](/manage/ai/providers/google-gemini) provider rather than this one.
content/docs/manage/ai/session-logs.mdx
+60
View File
@@ -0,0 +1,60 @@
---
title: "AI Session Logs"
description: "Prompt and response transcripts for AI Gateway requests in an organization"
---
Session logs store the prompt and response for each AI Gateway call. Open **AI Gateway > Session Logs** to browse them. Use them to see what a client sent, what the upstream returned, and which user, key, provider, and resource handled the call.
<Frame>
<img src="/images/ai/session-logs.png" alt="AI Gateway session logs table in the Pangolin dashboard" />
</Frame>
<Note>
Session logs are only available in [Pangolin Cloud](https://app.pangolin.net/auth/signup) or self-hosted [Enterprise Edition](/self-host/enterprise-edition).
</Note>
Token counts and estimated USD live in [Usage Analytics](/manage/ai/usage-analytics). Those records are separate. Changing session log retention does not delete usage data.
<Tip>
Set **AI Gateway Session Log Retention** in org settings before you expect rows. The default is 0 days, which leaves logging off.
</Tip>
## Enable Retention
In org security settings, set **AI Gateway Session Log Retention** to how long Pangolin should keep transcripts. Logging starts only after that value is greater than zero. Older rows are removed when they pass the retention window.
## What Each Row Contains
After a gateway call that belongs to an organization, Pangolin stores:
- Session id (shared with the usage record for that call)
- Time, HTTP status, and whether the response was streamed
- [Capability](/manage/ai/providers/configuration#capabilities), [provider](/manage/ai/providers/overview), and requested model
- Public or private [resource](/manage/ai/multiple-gateway-resources)
- User and [virtual API key](/manage/ai/virtual-api-keys), when they are known
- Raw request and response bodies
- Normalized chat messages used for the transcript view
Expand a row to see session id, status, catalog cost and tokens (joined from usage), and the transcript. Toggle between the chat view and raw JSON.
Very large bodies are truncated at about 200,000 characters. Truncated rows are marked in the UI.
## Filters
Narrow the table by date range, then:
- Provider
- Capability
- Resource
- User
- Virtual API key
- Model
- Streaming or not
## Export
Export the current filters as CSV from the table, the same way as other organization logs.
## Streaming
Session logs can be streamed (like all other logs) to upstream SIEM providers or other data sources. You can set this up following the [streaming](/manage/analytics/streaming) documentation.
content/docs/manage/ai/usage-analytics.mdx
+41
View File
@@ -0,0 +1,41 @@
---
title: "AI Usage Analytics"
description: "Cost, token usage, and request volume across providers, resources, roles, users, and keys"
---
Usage analytics rolls up every AI Gateway call: how many requests ran, how many tokens they used, and estimated USD. Open **AI Gateway → Usage Analytics**. The numbers come from the same recorded usage that [budgets](/manage/ai/budgets) enforce.
<Frame>
<img src="/images/ai/analytics-overview.png" alt="Usage analytics overview with cost, token, and request charts" />
</Frame>
These records are independent of session log retention. For prompt and response text, see [Session Logs](/manage/ai/session-logs). The two share a session id per call.
## How Numbers Are Calculated
USD uses [catalog](/manage/ai/model-catalog) rates (`in`, `out`, `cache`, `reasoning`) when the model id is known. Unknown models have no USD amount. Tokens still sum.
Token buckets are prompt, cache-read, cache-write, completion, and reasoning. If the upstream omits usage, Pangolin estimates tokens from the request and response. Overview shows **Estimated**: the share of requests that used that path.
A non-2xx response from the provider still counts as a request with zero tokens, so volume charts include failed attempts.
These figures are estimates for the dashboard and for budgets, not a copy of the vendor invoice. See [Budgets](/manage/ai/budgets) for the same calculation used at enforcement time.
## Filters
Every tab uses the same filters. The default date range is the last seven days.
- Provider
- Model
- Resource
- Role
- User
- Virtual API key
## Tabs
**Overview** shows totals for requests, tokens, cost, and estimated percent, daily charts for each, and top models by cost.
**Providers**, **Resources**, **Roles**, **Users**, and **Virtual API Keys** each show cost and token trends plus a ranked list for that dimension.
Role is derived from the caller's org roles. Usage rows store the user id, not the role. A [manual key](/manage/ai/virtual-api-keys#manual-keys) with no attributed user appears as an unknown user.
content/docs/manage/ai/virtual-api-keys.mdx
+86
View File
@@ -0,0 +1,86 @@
---
title: "Virtual API Keys"
description: "Authenticate AI clients to public AI Gateway resources without sharing upstream provider keys"
---
A virtual API key is a Pangolin credential that AI clients send to a public AI Gateway resource. Users retrieve their **identity key** by logging in with their existing Pangolin account or [identity provider](/manage/identity-providers/add-an-idp). That key is bound to their Pangolin identity, so calls are attributed to them for auditing, tracking, and access enforcement. When the user is known, Pangolin also forwards that identity upstream as [`Remote-*` headers](/manage/ai/providers/configuration#identity-headers).
The real OpenAI, Anthropic, Gemini, or other provider key stays on the [provider](/manage/ai/providers/overview). Pangolin checks the virtual key, then forwards the request with the upstream secret.
These keys are what coding agents put in their API key field. They are not [Integration API](/manage/integration-api) organization or root keys.
## Why Virtual API Keys
Agents and SDKs expect an API key. Virtual keys let many people and services call the gateway while the org keeps a single upstream provider key.
- Revoke or rotate one person or agent's access without rotating the provider key
- Attribute usage to a user or a named key
- Use the same Pangolin credential in clients that send `Authorization: Bearer`, `x-api-key`, or `x-goog-api-key`
Pangolin accepts the virtual key in those headers and strips it before the upstream call. Keys look like `pangolin-key-<id>.<secret>`.
Logging into Pangolin in a browser is how you **retrieve** a key. Model calls still need the virtual key in the request. A dashboard session cookie cannot proxy through the gateway.
## Public Resources Only
Virtual keys apply to **public** AI Gateway resources. [Private resources](/manage/resources/understanding-resources) are reached through the [Pangolin client](/manage/clients/install-client), so the gateway does not check a key. Clients still need a placeholder in the key field; use the literal string `none`. Deleting the field usually breaks the client.
## Authentication Is Always On
Public AI Gateway resources always require authentication. Unlike [HTTPS public resources](/manage/resources/public/authentication), you cannot turn Platform SSO off.
Assign [users and roles](/manage/access-control/create-user) on the resource the same way you would for a public HTTPS resource. Those grants control who can use an **identity key**.
HTTPS resources can add PIN, passcode, header auth, shareable links, or email OTP for browser access. AI clients authenticate programmatically, so the gateway uses virtual API keys instead of those methods.
## Identity Keys vs Manual Keys
Both kinds are `pangolin-key-…` secrets. They differ in how access is granted.
### Identity Keys
Every organization user already has an identity key. It is unique to their account and authenticates them as that user.
Access follows the resource's **Users** and **Roles**, the same way a public HTTPS resource does. Grant the user or their role on the AI Gateway resource, or their identity key cannot call it.
Admins do not mint identity keys; they already exist. To send them out, see [Email Keys to Users](#email-keys-to-users).
### Manual Keys
Admins create manual keys at **AI Gateway → Virtual API Keys → Virtual Keys**. Scope a key to selected public AI gateways, or to every public AI gateway in the org.
Creating the key immediately grants access to those resources, regardless of the users and roles set on the resource. Use this for services, CI, and shared agents that should not depend on a person's resource grants.
You can optionally associate a user. That association is for usage tracking and so the key appears on that user's keys page. It does not change which resources the key can call.
When you create or edit a key, you can attach a [budget](/manage/ai/budgets) so spend or tokens on that key are capped.
<Frame>
<img src="/images/ai/create-virtual-api-key.png" alt="Create Virtual API Key dialog with name, description, and public AI gateway access" />
</Frame>
## Where to Get a Key
Your identity key and any manual keys attributed to you are available in three places:
1. Visit the public resource URL in a browser and log in with your Pangolin identity. After login, Pangolin shows the key.
2. Open the Resource Launcher, select the resource, and use the more-info side panel.
3. Go to `https://app.pangolin.net/<org-id>/keys` (the org **Your API Keys** page). On self-hosted Pangolin, use the same path on your dashboard URL, for example `https://pangolin.example.com/<org-id>/keys`.
See [Organization ID](/manage/organizations/org-id) if you need to look up `<org-id>`.
Put that key in the AI client. Visiting the resource in a browser only reveals the key; it does not authenticate the agent's requests.
## Email Keys to Users
Admins can email keys so users do not have to find the dashboard page themselves. The message includes the secret and public AI gateway URLs they can call.
### Email Identity Keys
**AI Gateway → Virtual API Keys → Identity Keys → Email Identity Keys.** Send every organization member their identity key, or pick specific users and roles. Recipients without an account email are skipped.
### Email Manual Keys
When you create or edit a manual key, enable **Email key upon generation** (or **Email this key**). Send it to the associated user and to extra addresses you add.
Self-hosted Pangolin needs [SMTP](/self-host/advanced/config-file#email-configuration) configured before either flow can send mail.