mirror of
https://github.com/netbirdio/docs.git
synced 2026-09-23 15:29:06 +02:00
docs: document adding models not in the catalog (#896)
This commit is contained in:
Binary file not shown.
|
After Width: | Height: | Size: 164 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 145 KiB After Width: | Height: | Size: 166 KiB |
@@ -54,6 +54,19 @@ for details.
|
|||||||
|
|
||||||
See [Providers](/agent-network/providers) for details.
|
See [Providers](/agent-network/providers) for details.
|
||||||
|
|
||||||
|
### Cross-Region Inference Profiles
|
||||||
|
|
||||||
|
Bedrock also exposes cross-region inference profiles, model IDs prefixed with a geography
|
||||||
|
code such as `eu.` or `us.`, that route a request across a set of regions instead of pinning
|
||||||
|
it to one. If the profile you need isn't in the picker, add it the same way as any
|
||||||
|
[model not in the catalog](/agent-network/providers#adding-a-model-not-in-the-catalog): type
|
||||||
|
the full profile ID into the **Model** field on the **Models** tab, for example
|
||||||
|
`eu.anthropic.claude-sonnet-5`, and set its input/output pricing yourself.
|
||||||
|
|
||||||
|
<p>
|
||||||
|
<img src="/docs-static/img/agent-network/providers/agent-network-custom-model.png" alt="Models tab with a geo-specific Bedrock inference profile ID and manually entered input/output pricing" className="imagewrapper" />
|
||||||
|
</p>
|
||||||
|
|
||||||
## Create a Policy
|
## Create a Policy
|
||||||
|
|
||||||
By default nothing is allowed: a policy must connect a source group to the Bedrock provider
|
By default nothing is allowed: a policy must connect a source group to the Bedrock provider
|
||||||
|
|||||||
@@ -114,10 +114,39 @@ to keep the caller's identity out of the upstream request.
|
|||||||
|
|
||||||
Each provider carries a list of models it serves. Leaving the list empty makes the
|
Each provider carries a list of models it serves. Leaving the list empty makes the
|
||||||
provider a catch-all that accepts any model (typical for gateways); listing specific models
|
provider a catch-all that accepts any model (typical for gateways); listing specific models
|
||||||
restricts routing to them. Per-model input/output prices drive the cost figures shown in
|
restricts routing to them. Per-model prices drive the cost figures shown in
|
||||||
[Usage & Logs](/agent-network/usage-and-logs); adjust them if your negotiated rates differ
|
[Usage & Logs](/agent-network/usage-and-logs); adjust them if your negotiated rates differ
|
||||||
from the catalog defaults.
|
from the catalog defaults.
|
||||||
|
|
||||||
|
Each model has an **input** and **output** price per 1k tokens, plus optional
|
||||||
|
**cache** rates that match how the provider bills prompt caching. Which cache
|
||||||
|
fields apply depends on the provider's pricing surface: OpenAI-shape providers
|
||||||
|
use a single **cached input** rate (cached tokens are a subset of the prompt),
|
||||||
|
while Anthropic-shape providers, including Claude on Amazon Bedrock and Google
|
||||||
|
Vertex AI, use separate **cache read** and **cache creation** rates (additive
|
||||||
|
buckets on top of input). Gateways and custom providers expose all cache fields,
|
||||||
|
since NetBird can't know the upstream shape ahead of time.
|
||||||
|
|
||||||
|
Leave a cache rate **blank** to inherit NetBird's default for that model when one
|
||||||
|
exists; set it to **0** to bill that cache bucket at the plain input rate (no
|
||||||
|
discount). See [how caching is metered](/agent-network/usage-and-logs/access-logs)
|
||||||
|
for how these buckets appear in the cost breakdown.
|
||||||
|
|
||||||
|
Self-hosters can seed the catalog defaults these fields prefill from with a
|
||||||
|
pricing file. See
|
||||||
|
[`server.agentNetwork.pricingDefaultsFile`](/selfhosted/maintenance/configuration-files#agent-network-settings).
|
||||||
|
|
||||||
|
### Adding a Model Not in the Catalog
|
||||||
|
|
||||||
|
If the model you need isn't in the picker, type its model ID directly into the **Model** field
|
||||||
|
instead of selecting from the list, for example `eu.anthropic.claude-sonnet-5`. A model NetBird
|
||||||
|
doesn't know has no catalog defaults, so set its **input** and **output** prices (and cache
|
||||||
|
rates, if applicable) yourself for usage and logs to report accurate costs.
|
||||||
|
|
||||||
|
<p>
|
||||||
|
<img src="/docs-static/img/agent-network/providers/agent-network-custom-model.png" alt="Models tab with a custom model ID and manually entered input/output pricing" className="imagewrapper" />
|
||||||
|
</p>
|
||||||
|
|
||||||
## The Keyless Endpoint
|
## The Keyless Endpoint
|
||||||
|
|
||||||
All connected providers share a single account endpoint, generated when you connect your
|
All connected providers share a single account endpoint, generated when you connect your
|
||||||
|
|||||||
@@ -400,6 +400,16 @@ Configures the database backend for the embedded identity provider (Dex). By def
|
|||||||
</Property>
|
</Property>
|
||||||
</Properties>
|
</Properties>
|
||||||
|
|
||||||
|
### Agent Network Settings
|
||||||
|
|
||||||
|
Configures the [Agent Network](/agent-network) (LLM gateway) served by the combined server.
|
||||||
|
|
||||||
|
<Properties>
|
||||||
|
<Property name="server.agentNetwork.pricingDefaultsFile" type="string">
|
||||||
|
Path to a YAML file holding the default LLM pricing table used to seed the model catalog and meter request costs. A relative path is resolved against `dataDir`, so a bare filename such as `pricing.yaml` lands in the data directory. When empty, `{dataDir}/defaults_llm_pricing.yaml` is probed; if no file is present, the compiled-in defaults are used. The schema maps each surface (`openai`, `anthropic`, `bedrock`) to a model to its rates in USD per 1k tokens (`input_per_1k`, `output_per_1k`, and the optional `cached_input_per_1k` / `cache_read_per_1k` / `cache_creation_per_1k`). On the `bedrock` surface, model keys must be the **normalized** id costs are metered under rather than the full id pasted from AWS: drop any `arn:…` wrapper, the cross-region inference-profile prefix (`us.`, `eu.`, `apac.`, `global.`), and the trailing version or throughput suffix (`-v1:0`, `-20250929-v1:0`). For example, requests for `eu.anthropic.claude-sonnet-5` are priced from the key `anthropic.claude-sonnet-5`. File entries replace the compiled-in entry for the same surface and model; everything else keeps the compiled-in rates. The file is re-read periodically (mtime poll). An explicitly configured path that fails to load fails startup; runtime reload errors keep the previous table. Operator price overrides set in the dashboard take precedence over these defaults.
|
||||||
|
</Property>
|
||||||
|
</Properties>
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## dashboard.env
|
## dashboard.env
|
||||||
|
|||||||
Reference in New Issue
Block a user