diff --git a/docs.json b/docs.json index 3b4018d..58c7c44 100644 --- a/docs.json +++ b/docs.json @@ -190,7 +190,7 @@ }, { "group": "AI Gateway", - "icon": "robot", + "icon": "sparkles", "pages": [ "manage/ai/overview", "manage/ai/multiple-gateway-resources", diff --git a/images/ai/analytics-overview.png b/images/ai/analytics-overview.png new file mode 100644 index 0000000..c6ccd46 Binary files /dev/null and b/images/ai/analytics-overview.png differ diff --git a/images/ai/create-ai-gateway-resource.png b/images/ai/create-ai-gateway-resource.png new file mode 100644 index 0000000..bd9bd51 Binary files /dev/null and b/images/ai/create-ai-gateway-resource.png differ diff --git a/images/ai/create-virtual-api-key.png b/images/ai/create-virtual-api-key.png new file mode 100644 index 0000000..31c601a Binary files /dev/null and b/images/ai/create-virtual-api-key.png differ diff --git a/images/ai/public-resource-models.png b/images/ai/public-resource-models.png new file mode 100644 index 0000000..b7bc100 Binary files /dev/null and b/images/ai/public-resource-models.png differ diff --git a/images/ai/public-resource-providers.png b/images/ai/public-resource-providers.png new file mode 100644 index 0000000..1b953e1 Binary files /dev/null and b/images/ai/public-resource-providers.png differ diff --git a/images/ai/session-logs.png b/images/ai/session-logs.png new file mode 100644 index 0000000..346267d Binary files /dev/null and b/images/ai/session-logs.png differ diff --git a/manage/ai/overview.mdx b/manage/ai/overview.mdx index 9ea8e6d..23f2a92 100644 --- a/manage/ai/overview.mdx +++ b/manage/ai/overview.mdx @@ -25,6 +25,10 @@ Create an org-level provider so the gateway has an upstream to call. Sidebar → Resources → **Create** → set **Type** to **AI Gateway**, pick a domain, and attach the provider(s) from step 1. You can also attach providers to an existing resource later from its **AI Gateway** settings tab. How public and private AI Gateway resources reach users is covered on the [public](/manage/resources/public/ai-gateway) and [private](/manage/resources/private/ai-gateway) resource type pages. + + Create Public Resource form with Type set to AI Gateway + + You can create more than one AI Gateway resource so different users and roles get their own providers. Distinct hostnames are the usual approach; they can also share a FQDN. See [Multiple Gateway Resources](/manage/ai/multiple-gateway-resources). ## 3. Authenticate to the Gateway diff --git a/manage/ai/providers/model-routing.mdx b/manage/ai/providers/model-routing.mdx index 3f0ac43..304e9de 100644 --- a/manage/ai/providers/model-routing.mdx +++ b/manage/ai/providers/model-routing.mdx @@ -11,6 +11,10 @@ See [AI Providers](/manage/ai/providers/overview) for what a provider is, and [P On the provider's **Models** tab, every request must match an **allow** entry and must not match a **block** entry. An empty allow list denies all traffic. A key cannot sit on both lists. + + Provider Models tab with allow and block lists for Anthropic + + You add keys three ways: | How | What it is | Example | diff --git a/manage/ai/providers/overview.mdx b/manage/ai/providers/overview.mdx index 64f2700..857e8d8 100644 --- a/manage/ai/providers/overview.mdx +++ b/manage/ai/providers/overview.mdx @@ -20,27 +20,15 @@ Each provider is: Providers are reused across resources. The resource still owns the public URL and the keys clients authenticate with. -{/* -Screenshot: `/images/ai/create-provider.png` -The AI Gateway → Providers → Create form, showing the provider type dropdown and API key field filled in for OpenAI or Anthropic. - - Creating an AI provider - -*/} - ## Attach to a Resource On an AI Gateway resource, attach one or more providers from the **AI Gateway** settings tab. For each attachment you can inherit the provider's model lists or select a subset. Details are in [Model Routing](/manage/ai/providers/model-routing#inherit-vs-select). The resource only speaks the API formats its attached providers advertise. Attach whichever providers match the [clients](/manage/ai/overview#4-connect-a-client) you plan to connect. -{/* -Screenshot: `/images/ai/create-resource.png` -The new resource dialog with Type set to "AI Gateway" and a provider selected in the AI Providers section. - Attaching providers to an AI Gateway resource + AI Gateway resource settings with attached providers -*/} ## Provider Types diff --git a/manage/ai/session-logs.mdx b/manage/ai/session-logs.mdx index c57beb3..669be07 100644 --- a/manage/ai/session-logs.mdx +++ b/manage/ai/session-logs.mdx @@ -5,6 +5,10 @@ description: "Prompt and response transcripts for AI Gateway requests in an orga Session logs store the prompt and response for each AI Gateway call. Open **AI Gateway → Session Logs** to browse them. Use them to see what a client sent, what the upstream returned, and which user, key, provider, and resource handled the call. + + AI Gateway session logs table in the Pangolin dashboard + + Session logs are only available in [Pangolin Cloud](https://app.pangolin.net/auth/signup) or self-hosted [Enterprise Edition](/self-host/enterprise-edition). diff --git a/manage/ai/usage-analytics.mdx b/manage/ai/usage-analytics.mdx index 9ca0c0d..d576b00 100644 --- a/manage/ai/usage-analytics.mdx +++ b/manage/ai/usage-analytics.mdx @@ -5,6 +5,10 @@ description: "Cost, token usage, and request volume across providers, resources, Usage analytics rolls up every AI Gateway call: how many requests ran, how many tokens they used, and estimated USD. Open **AI Gateway → Usage Analytics**. The numbers come from the same recorded usage that [budgets](/manage/ai/budgets) enforce. + + Usage analytics overview with cost, token, and request charts + + These records are independent of session log retention. For prompt and response text, see [Session Logs](/manage/ai/session-logs). The two share a session id per call. ## How Numbers Are Calculated diff --git a/manage/ai/virtual-api-keys.mdx b/manage/ai/virtual-api-keys.mdx index aa09012..57d5721 100644 --- a/manage/ai/virtual-api-keys.mdx +++ b/manage/ai/virtual-api-keys.mdx @@ -55,6 +55,10 @@ You can optionally associate a user. That association is for usage tracking and When you create or edit a key, you can attach a [budget](/manage/ai/budgets) so spend or tokens on that key are capped. + + Create Virtual API Key dialog with name, description, and public AI gateway access + + ## Where to Get a Key Your identity key and any manual keys attributed to you are available in three places: