pangolin_mirror/docs-v2
1
0
Fork 0
mirror of https://github.com/fosrl/docs-v2.git synced 2026-10-30 07:29:07 +01:00
Code Issues Packages Projects Releases Wiki Activity

add model routing and catalog

Browse Source
This commit is contained in:
miloschwartz
2026-08-20 16:46:43 -04:00
parent b8b04491a1
commit 2ed2b2b079
13 changed files with 219 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
manage/ai/providers/custom/bifrost.mdx
+1 -1
View File
@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
3. Select **OpenAI Chat Completions** (and any other formats Bifrost should accept).
4. Set **Routing Mode** to **Site Targets** if Bifrost runs on a site's network. Use **Upstream URL** (`http://<host>:8080`) only when the Pangolin node can reach Bifrost directly.
5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if Bifrost requires a virtual key or other API key.
6. Allow the model ids Bifrost should serve, including provider-prefixed keys such as `openai/gpt-4o-mini` if that is how Bifrost names them. See [Models](/manage/ai/providers/configuration#models).
6. Allow the model ids Bifrost should serve, including provider-prefixed keys such as `openai/gpt-4o-mini` if that is how Bifrost names them. See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the Bifrost host, port `8080`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
manage/ai/providers/custom/ollama.mdx
+1 -1
View File
@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
3. Select **OpenAI Chat Completions**.
4. Set **Routing Mode** to **Site Targets** if Ollama runs on a site's network. Use **Upstream URL** (`http://<host>:11434`) only when the Pangolin node can reach Ollama directly.
5. Set **Auth Type** to **No Auth** on a private LAN. Use **Bearer** if you set an Ollama API key.
6. Allow the exact model ids you plan to call, for example `llama3.2`. See [Models](/manage/ai/providers/configuration#models).
6. Allow the exact model ids you plan to call, for example `llama3.2`. See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the Ollama host, port `11434`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).
manage/ai/providers/custom/vllm.mdx
+1 -1
View File
@@ -18,7 +18,7 @@ See [Capabilities](/manage/ai/providers/configuration#capabilities) for the full
3. Select **OpenAI Chat Completions**.
4. Set **Routing Mode** to **Site Targets** if vLLM runs on a site's network. Use **Upstream URL** (`http://<host>:8000`) only when the Pangolin node can reach vLLM directly.
5. Set **Auth Type** to **No Auth** if the server has no key. Use **Bearer** and paste the key if you started vLLM with `--api-key`.
6. Allow the exact served model id (the `--model` name or `--served-model-name`). See [Models](/manage/ai/providers/configuration#models).
6. Allow the exact served model id (the `--model` name or `--served-model-name`). See [Model Routing](/manage/ai/providers/model-routing).
7. Save. On the **Network** tab, add a target: the vLLM host, port `8000`, method HTTP.
8. Attach the provider to an [AI Gateway resource](/manage/ai/overview).