pangolin_mirror/docs-v2
1
0
Fork 0
mirror of https://github.com/fosrl/docs-v2.git synced 2026-08-31 03:01:30 +02:00
Code Issues Packages Projects Releases Wiki Activity

add ai gateway docs

Browse Source
This commit is contained in:
miloschwartz
2026-08-21 17:16:13 -04:00
parent 0d56236294
commit 512eca0f86
11 changed files with 257 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs.json
View File

@@ -150,6 +150,8 @@
"manage/analytics/access",
"manage/analytics/connection",
"manage/analytics/action",
"manage/ai/session-logs",
"manage/ai/usage-analytics",
{
"group": "Event Streaming",
"pages": [
@@ -191,6 +193,7 @@
"icon": "robot",
"pages": [
"manage/ai/overview",
"manage/ai/multiple-gateway-resources",
"manage/ai/virtual-api-keys",
{
"group": "Providers",
@@ -230,7 +233,10 @@
"manage/ai/configure-ai-clients/openclaw"
]
},
"manage/ai/model-catalog"
"manage/ai/budgets",
"manage/ai/model-catalog",
"manage/ai/session-logs",
"manage/ai/usage-analytics"
]
},
"manage/endpoints-and-pops",

66
manage/ai/budgets.mdx Normal file
View File

@@ -0,0 +1,66 @@
---
title: "Budgets"
description: "Cap estimated USD spend or token usage on providers, models, resources, roles, and keys"
---
A budget is a cap on estimated USD spend or token count for a rolling window. Pangolin checks every matching enabled budget before it forwards a request. If **any** one is already at or above its maximum, the call is blocked.
Budgets are estimates used for enforcement, not a copy of the vendor invoice. See [Model Catalog](/manage/ai/model-catalog) for where USD rates come from. The same recorded usage appears in [Usage Analytics](/manage/ai/usage-analytics).
## Estimates, Not Invoices
USD cost uses catalog rates (`in`, `out`, `cache`, `reasoning`) times the token buckets Pangolin recorded for the call. Cache-write tokens use the input rate. Missing reasoning rates use the output rate.
If the model ID is not in the catalog, the request has no USD amount and does not count toward USD budgets. Token budgets still sum tokens for that call.
[OpenRouter](/manage/ai/providers/open-router), [Vercel AI Gateway](/manage/ai/providers/vercel-ai-gateway), and [Custom](/manage/ai/providers/custom) may match a catalog ID with approximate pricing.
If the upstream omits usage, Pangolin estimates tokens from the request and response.
The check uses **prior** recorded usage. A request that would cross the cap can still complete, so usage can slightly overshoot. Reset periods are trailing windows from now, not calendar months. Daily means the last 24 hours.
## How Usage Is Calculated
After each call, Pangolin records prompt, cache-read, cache-write, completion, and reasoning tokens.
- **Token** budgets sum those counts.
- **USD** budgets multiply the same counts by catalog rates, then sum dollars.
Enforcement totals that history for the budget's scope and [period](#fields), then compares the total to **Maximum Spend**.
## Where to Set a Budget
Each budget has exactly one scope. You can add more than one budget on the same scope when the unit or period differs, for example a daily USD cap and a monthly token cap on the same provider.
| Scope | Where | Why |
|---|---|---|
| **Provider** | **AI Gateway → Providers → Budget** | Cap everything that uses that upstream key |
| **Model** | Provider **Models** tab, on an allow-list entry | Cap one key or glob, such as `gpt-4o` or `gpt-*` |
| **Public resource** | Resource **Budget** tab | Cap a specific public gateway |
| **Private resource** | Resource **Budget** tab | Cap a client-only gateway |
| **Role** | Role form | Cap everyone in that role. Usage attributed to those users is summed |
| **Virtual API key** | Create or edit the key | Cap a service key or usage attributed to that key |
All matching scopes apply together. A call can hit a provider budget, a resource budget, a role budget, and a key budget at once. Exceeding any of them blocks the request.
## Fields
The editor uses **Spend Type**, **Maximum Spend**, and **Reset Period**.
| Field | Options |
|---|---|
| **Spend Type** | `USD` or `Tokens` |
| **Maximum Spend** | A positive maximum in that unit |
| **Reset Period** | Hourly, Daily (24 hours), Weekly (7 days), Monthly (30 days), Yearly (365 days), or Lifetime |
Lifetime covers all recorded usage for that scope.
## Over Budget
The gateway returns HTTP **429** with the message `AI usage budget exceeded for this request`. The JSON envelope matches the API the client is calling:
| Family | Shape |
|---|---|
| OpenAI | `error.type`: `rate_limit_error`, `error.code`: `rate_limit_exceeded` |
| Anthropic | `error.type`: `rate_limit_error` |
| Gemini | `error.status`: `RESOURCE_EXHAUSTED` |

2
manage/ai/model-catalog.mdx
View File

@@ -30,7 +30,7 @@ Unmapped types skip Known Models and wildcard expansion against the catalog. Ent
## Pricing and Budgets
Each catalog entry can include token rates: `in`, `out`, `cache`, and `reasoning`. Budget enforcement and usage analytics use those rates when the requested model matches a catalog id. Unknown keys have no catalog price, so add custom models to the catalog if you need cost tracking for them.
Each catalog entry can include token rates: `in`, `out`, `cache`, and `reasoning`. [Budget](/manage/ai/budgets) enforcement and usage analytics use those rates when the requested model matches a catalog id. Unknown keys have no catalog price, so they do not count toward USD budgets. Add custom models to the catalog if you need cost tracking for them. Token budgets still sum usage for unknown ids.
## Loading a Custom Catalog

67
manage/ai/multiple-gateway-resources.mdx Normal file
View File

@@ -0,0 +1,67 @@
---
title: "Multiple Gateway Resources"
description: "Give different users and roles their own providers, model lists, and budgets with more than one AI Gateway resource"
---
You can create more than one [public](/manage/resources/public/ai-gateway) or [private](/manage/resources/private/ai-gateway) AI Gateway resource. Each one has its own users and roles, attached [providers](/manage/ai/providers/overview), model lists, and [budgets](/manage/ai/budgets). That is how the Employees role gets OpenAI and Anthropic while Contractors get a tighter set, without sharing one allow list.
This is Pangolin's existing access control, applied to AI. A gateway resource is still a resource: you assign [users and roles](/manage/access-control/create-user) to it the same way as [public HTTPS](/manage/resources/public/authentication) or a [private host](/manage/resources/private/authentication). Assign a role when a set of people should share access. Assign individual users when the set is smaller or does not match a role. What differs per resource is which providers, models, and budgets it exposes.
## Users and Roles
Create a resource for the people who should share providers. Assign those users, or a role they belong to, then give the resource its own hostname. Clients point at a URL that already means that resource's providers.
For example:
- Resource **Staff AI**: FQDN `ai-staff.example.com`, role Employees, providers OpenAI and Anthropic.
- Resource **Contractor AI**: FQDN `ai-contractors.example.com`, role Contractors, OpenRouter (or a custom endpoint) with a narrower allow list.
Staff point Claude Code, Codex, or another client at `https://ai-staff.example.com`. Contractors use `https://ai-contractors.example.com`. An [identity key](/manage/ai/virtual-api-keys) is still per user; the hostname is what selects the resource.
The same pattern works across public and private: a public resource for agents on the internet, a private resource for people on the Pangolin client.
## Sharing a Hostname
HTTP and HTTPS resources each need their own fully qualified domain name, because Pangolin would otherwise not know which target to send traffic to. AI Gateway resources can share a FQDN. Every one of them routes to the gateway running inside Pangolin, so they always go to the same place.
The hostname is an entrypoint, not a unique backend. Use a shared name when you want the same split as [users and roles](#users-and-roles), but everyone configures one URL.
You can overlap:
- Several public AI Gateway resources
- Several private AI Gateway resources
- Public and private together
An [HTTP / HTTPS](/manage/resources/public/http-https) resource still cannot use that same name.
### How Pangolin Picks a Resource
On each request, Pangolin lists the enabled AI Gateway resources whose FQDN matches the host, then keeps the ones the caller is allowed to use.
- **Public.** [Users and roles](/manage/resources/public/authentication) on the resource control an [identity key](/manage/ai/virtual-api-keys). A [manual key](/manage/ai/virtual-api-keys#manual-keys) must be scoped to that resource, or to all resources.
- **Private.** [Users, roles, or machines](/manage/resources/private/authentication) granted on the resource, using the connected Pangolin client.
A user or role can be granted on more than one resource in the overlapping set. After access filtering, Pangolin picks among the remaining resources the same way it picks a [provider](/manage/ai/providers/model-routing#provider-selection) on a single resource:
1. **Capability.** The path must match a capability an attached provider advertises.
2. **Allow and block.** The requested model must pass the resource's effective lists.
3. **Specificity, catalog ownership, and class.** Exact allow keys beat patterns. A typed catalog owner beats an aggregator. Native typed providers beat aggregators, which beat Custom.
The chosen resource's attached providers, model lists, and resource-scoped budgets apply. Someone granted on both Staff AI and Contractor AI, both at `ai.example.com`, is routed by the model and API they called.
If more than one resource still matches after those steps, the gateway returns the same ambiguous error as overlapping providers: `Model "<id>" is ambiguous across multiple AI providers on this resource`.
When a public resource and a private resource share a host, calls from the internet use the public path (virtual API key). Calls over the Pangolin client tunnel use the private path (client identity).
The steps above assume [users and roles](/manage/resources/public/authentication) decide who can use each overlapping resource, together with virtual API keys or the connected client. If you also add [access rules](/manage/access-control/rules) (IP, path, geolocation, and similar) on those resources, which overlapping resource a request lands on is undefined. Put access rules on resources that have their own hostname.
```mermaid
flowchart LR
client[Client] --> fqdn["ai.example.com"]
fqdn --> gw[Gateway in Pangolin]
gw --> pick{Access then model}
pick --> eng[Staff AI]
pick --> fin[Contractor AI]
eng --> openai[OpenAI and Anthropic]
fin --> cheap[Restricted providers]
```

23
manage/ai/overview.mdx
View File

@@ -1,6 +1,6 @@
---
title: "AI Gateway Overview"
description: "Route coding agents and AI clients through Pangolin with centralized providers and keys"
description: "Route coding agents and AI clients through Pangolin with centralized providers and access control"
---
An AI Gateway resource is a normal Pangolin resource that proxies requests to one or more upstream AI providers (OpenAI, Anthropic, Google Gemini, Bedrock, Vertex AI, or a custom endpoint). Point a client at the resource's URL instead of the provider's, and Pangolin handles auth, access control, and logging in front of it.
@@ -11,6 +11,9 @@ An AI Gateway resource is a normal Pangolin resource that proxies requests to on
- **Resources** (type `AI Gateway`) attach one or more of those providers and get a normal Pangolin domain.
- **[Virtual API Keys](/manage/ai/virtual-api-keys)** are what clients send to a **public** AI Gateway resource. Private resources are reached through the Pangolin network, so the gateway does not check a key.
- The **[Model Catalog](/manage/ai/model-catalog)** is the known-model list and token pricing used by [model routing](/manage/ai/providers/model-routing) and budgets.
- **[Budgets](/manage/ai/budgets)** cap estimated USD spend or token usage on a provider, model, resource, role, or key.
- **[Session Logs](/manage/ai/session-logs)** store prompt and response transcripts for each call.
- **[Usage Analytics](/manage/ai/usage-analytics)** charts cost, tokens, and request volume across providers, resources, roles, users, and keys.
A resource only understands the API format(s) its attached providers support. An Anthropic provider makes the resource speak the Anthropic Messages API; an OpenAI provider makes it speak Chat Completions/Responses; a Gemini provider makes it speak Gemini's `generateContent` API. Attach whichever providers match the clients you plan to connect. See [AI Providers](/manage/ai/providers/overview) for per-type setup and [Provider Configuration](/manage/ai/providers/configuration) for capabilities, auth, and model lists.
@@ -22,23 +25,23 @@ Create an org-level provider so the gateway has an upstream to call. Sidebar →
Resources → **Create** → set **Type** to **AI Gateway**, pick a domain, and attach the provider(s) from step 1. You can also attach providers to an existing resource later from its **AI Gateway** settings tab. How public and private AI Gateway resources reach users is covered on the [public](/manage/resources/public/ai-gateway) and [private](/manage/resources/private/ai-gateway) resource type pages.
## 3. Get a Key
You can create more than one AI Gateway resource so different users and roles get their own providers. Distinct hostnames are the usual approach; they can also share a FQDN. See [Multiple Gateway Resources](/manage/ai/multiple-gateway-resources).
How a client authenticates depends on whether the resource is public or private. See [Virtual API Keys](/manage/ai/virtual-api-keys) for identity vs manual keys, always-on auth, and where to copy a key.
## 3. Authenticate to the Gateway
### Public Resources
Reachable from anywhere, so the gateway checks a virtual API key. Copy yours from the resource URL after login, the Resource Launcher more-info panel, or `https://app.pangolin.net/<org-id>/keys` (use your self-hosted dashboard URL in place of `app.pangolin.net` if you self-host).
How a client authenticates depends on whether the resource is public or private.
### Private Resources
Only reachable from devices connected to your Pangolin network, so the gateway does not check a key. Instead, the machine running the client needs the [Pangolin client](/manage/clients/install-client) installed and connected.
Reachable only on devices connected with the [Pangolin client](/manage/clients/install-client). Identity comes from that connection, so you do not issue a [virtual API key](/manage/ai/virtual-api-keys). The desktop client already proved who is calling. Details are on the [private AI Gateway](/manage/resources/private/ai-gateway) resource page.
You still have to put *something* in the client's key field. Most AI clients refuse to start with a missing key, so use the literal string `none`. Deleting the field usually breaks things. Claude Code, for instance, silently falls back to your regular Anthropic account if `apiKeyHelper` is missing, and OpenCode errors out with `OpenAI API key is missing`.
### Public Resources
## 4. Connect a Client
Reachable from anywhere. The gateway checks a [virtual API key](/manage/ai/virtual-api-keys) on every call. Details are on the [public AI Gateway](/manage/resources/public/ai-gateway) resource page.
Every resource's Keys page has ready-to-copy setup for common clients, and `pangolin configure <client>` can write the config for you. Full instructions per client:
## 4. Connect a Coding Agent, Agent, or AI Client to the Gateway
Every resource has ready-to-copy setup for common clients, and `pangolin configure <client>` can write the config for you. Full instructions per client:
| Client | Needs a provider that speaks | Guide |
|---|---|---|

4
manage/ai/providers/configuration.mdx
View File

@@ -60,6 +60,10 @@ See [Ollama](/manage/ai/providers/custom/ollama), [vLLM](/manage/ai/providers/cu
Allow and block lists, inherit vs select, and provider selection are covered in [Model Routing](/manage/ai/providers/model-routing). Known Models, ownership scoring, and token pricing for budgets come from the [Model Catalog](/manage/ai/model-catalog).
## Budget
The provider **Budget** tab caps estimated USD spend or token usage for every call that uses this provider. You can also set a budget on an individual allow-list model from the **Models** tab. See [Budgets](/manage/ai/budgets) for scopes, periods, and the 429 clients receive when a cap is hit.
## Headers and TLS
On **Network** you can also set:

52
manage/ai/session-logs.mdx Normal file
View File

@@ -0,0 +1,52 @@
---
title: "AI Session Logs"
description: "Prompt and response transcripts for AI Gateway requests in an organization"
---
Session logs store the prompt and response for each AI Gateway call. Open **AI Gateway → Session Logs** to browse them. Use them to see what a client sent, what the upstream returned, and which user, key, provider, and resource handled the call.
<Note>
Session logs are only available in [Pangolin Cloud](https://app.pangolin.net/auth/signup) or self-hosted [Enterprise Edition](/self-host/enterprise-edition).
</Note>
Token counts and estimated USD live in [Usage Analytics](/manage/ai/usage-analytics). Those records are separate. Changing session log retention does not delete usage data.
<Tip>
Set **AI Gateway Session Log Retention** in org settings before you expect rows. The default is 0 days, which leaves logging off.
</Tip>
## Enable Retention
In org security settings, set **AI Gateway Session Log Retention** to how long Pangolin should keep transcripts. Logging starts only after that value is greater than zero. Older rows are removed when they pass the retention window.
## What Each Row Contains
After a gateway call that belongs to an organization, Pangolin stores:
- Session id (shared with the usage record for that call)
- Time, HTTP status, and whether the response was streamed
- [Capability](/manage/ai/providers/configuration#capabilities), [provider](/manage/ai/providers/overview), and requested model
- Public or private [resource](/manage/ai/multiple-gateway-resources)
- User and [virtual API key](/manage/ai/virtual-api-keys), when they are known
- Raw request and response bodies
- Normalized chat messages used for the transcript view
Expand a row to see session id, status, catalog cost and tokens (joined from usage), and the transcript. Toggle between the chat view and raw JSON.
Very large bodies are truncated at about 200,000 characters. Truncated rows are marked in the UI.
## Filters
Narrow the table by date range, then:
- Provider
- Capability
- Resource
- User
- Virtual API key
- Model
- Streaming or not
## Export
Export the current filters as CSV from the table, the same way as other organization logs.

37
manage/ai/usage-analytics.mdx Normal file
View File

@@ -0,0 +1,37 @@
---
title: "AI Usage Analytics"
description: "Cost, token usage, and request volume across providers, resources, roles, users, and keys"
---
Usage analytics rolls up every AI Gateway call: how many requests ran, how many tokens they used, and estimated USD. Open **AI Gateway → Usage Analytics**. The numbers come from the same recorded usage that [budgets](/manage/ai/budgets) enforce.
These records are independent of session log retention. For prompt and response text, see [Session Logs](/manage/ai/session-logs). The two share a session id per call.
## How Numbers Are Calculated
USD uses [catalog](/manage/ai/model-catalog) rates (`in`, `out`, `cache`, `reasoning`) when the model id is known. Unknown models have no USD amount. Tokens still sum.
Token buckets are prompt, cache-read, cache-write, completion, and reasoning. If the upstream omits usage, Pangolin estimates tokens from the request and response. Overview shows **Estimated**: the share of requests that used that path.
A non-2xx response from the provider still counts as a request with zero tokens, so volume charts include failed attempts.
These figures are estimates for the dashboard and for budgets, not a copy of the vendor invoice. See [Budgets](/manage/ai/budgets) for the same calculation used at enforcement time.
## Filters
Every tab uses the same filters. The default date range is the last seven days.
- Provider
- Model
- Resource
- Role
- User
- Virtual API key
## Tabs
**Overview** shows totals for requests, tokens, cost, and estimated percent, daily charts for each, and top models by cost.
**Providers**, **Resources**, **Roles**, **Users**, and **Virtual API Keys** each show cost and token trends plus a ranked list for that dimension.
Role is derived from the caller's org roles. Usage rows store the user id, not the role. A [manual key](/manage/ai/virtual-api-keys#manual-keys) with no attributed user appears as an unknown user.

2
manage/ai/virtual-api-keys.mdx
View File

@@ -53,6 +53,8 @@ Creating the key immediately grants access to those resources, regardless of the
You can optionally associate a user. That association is for usage tracking and so the key appears on that user's keys page. It does not change which resources the key can call.
When you create or edit a key, you can attach a [budget](/manage/ai/budgets) so spend or tokens on that key are capped.
## Where to Get a Key
Your identity key and any manual keys attributed to you are available in three places:

4
manage/resources/private/ai-gateway.mdx
View File

@@ -28,6 +28,10 @@ Access is the private resource model: grant users, roles, or machines explicitly
When the connected client maps to a user, Pangolin forwards that identity upstream as [`Remote-*` headers](/manage/ai/providers/configuration#identity-headers).
## More Than One Resource
Give different users and roles their own providers with more than one AI Gateway resource. Distinct hostnames are the usual approach. Unlike other private resource types, they can also share a FQDN because they all route to the gateway inside Pangolin. See [Multiple Gateway Resources](/manage/ai/multiple-gateway-resources).
## Compared to Public AI Gateway
| | Private AI Gateway | [Public AI Gateway](/manage/resources/public/ai-gateway) |

4
manage/resources/public/ai-gateway.mdx
View File

@@ -34,6 +34,10 @@ HTTPS resources can add PIN, passcode, header auth, shareable links, or email OT
You can still attach a [resource policy](/manage/resources/public/resource-policies) for users, roles, and access rules.
## More Than One Resource
Give different users and roles their own providers with more than one AI Gateway resource. Distinct hostnames are the usual approach. Unlike HTTP / HTTPS, they can also share a FQDN because they all route to the gateway inside Pangolin. See [Multiple Gateway Resources](/manage/ai/multiple-gateway-resources).
## Compared to Private AI Gateway
| | Public AI Gateway | [Private AI Gateway](/manage/resources/private/ai-gateway) |