mirror of
https://github.com/netbirdio/docs.git
synced 2026-08-25 01:01:27 +02:00
docs: document adding models not in the catalog (#896)
This commit is contained in:
@@ -54,6 +54,19 @@ for details.
|
||||
|
||||
See [Providers](/agent-network/providers) for details.
|
||||
|
||||
### Cross-Region Inference Profiles
|
||||
|
||||
Bedrock also exposes cross-region inference profiles, model IDs prefixed with a geography
|
||||
code such as `eu.` or `us.`, that route a request across a set of regions instead of pinning
|
||||
it to one. If the profile you need isn't in the picker, add it the same way as any
|
||||
[model not in the catalog](/agent-network/providers#adding-a-model-not-in-the-catalog): type
|
||||
the full profile ID into the **Model** field on the **Models** tab, for example
|
||||
`eu.anthropic.claude-sonnet-5`, and set its input/output pricing yourself.
|
||||
|
||||
<p>
|
||||
<img src="/docs-static/img/agent-network/providers/agent-network-custom-model.png" alt="Models tab with a geo-specific Bedrock inference profile ID and manually entered input/output pricing" className="imagewrapper" />
|
||||
</p>
|
||||
|
||||
## Create a Policy
|
||||
|
||||
By default nothing is allowed: a policy must connect a source group to the Bedrock provider
|
||||
|
||||
@@ -114,10 +114,39 @@ to keep the caller's identity out of the upstream request.
|
||||
|
||||
Each provider carries a list of models it serves. Leaving the list empty makes the
|
||||
provider a catch-all that accepts any model (typical for gateways); listing specific models
|
||||
restricts routing to them. Per-model input/output prices drive the cost figures shown in
|
||||
restricts routing to them. Per-model prices drive the cost figures shown in
|
||||
[Usage & Logs](/agent-network/usage-and-logs); adjust them if your negotiated rates differ
|
||||
from the catalog defaults.
|
||||
|
||||
Each model has an **input** and **output** price per 1k tokens, plus optional
|
||||
**cache** rates that match how the provider bills prompt caching. Which cache
|
||||
fields apply depends on the provider's pricing surface: OpenAI-shape providers
|
||||
use a single **cached input** rate (cached tokens are a subset of the prompt),
|
||||
while Anthropic-shape providers, including Claude on Amazon Bedrock and Google
|
||||
Vertex AI, use separate **cache read** and **cache creation** rates (additive
|
||||
buckets on top of input). Gateways and custom providers expose all cache fields,
|
||||
since NetBird can't know the upstream shape ahead of time.
|
||||
|
||||
Leave a cache rate **blank** to inherit NetBird's default for that model when one
|
||||
exists; set it to **0** to bill that cache bucket at the plain input rate (no
|
||||
discount). See [how caching is metered](/agent-network/usage-and-logs/access-logs)
|
||||
for how these buckets appear in the cost breakdown.
|
||||
|
||||
Self-hosters can seed the catalog defaults these fields prefill from with a
|
||||
pricing file. See
|
||||
[`server.agentNetwork.pricingDefaultsFile`](/selfhosted/maintenance/configuration-files#agent-network-settings).
|
||||
|
||||
### Adding a Model Not in the Catalog
|
||||
|
||||
If the model you need isn't in the picker, type its model ID directly into the **Model** field
|
||||
instead of selecting from the list, for example `eu.anthropic.claude-sonnet-5`. A model NetBird
|
||||
doesn't know has no catalog defaults, so set its **input** and **output** prices (and cache
|
||||
rates, if applicable) yourself for usage and logs to report accurate costs.
|
||||
|
||||
<p>
|
||||
<img src="/docs-static/img/agent-network/providers/agent-network-custom-model.png" alt="Models tab with a custom model ID and manually entered input/output pricing" className="imagewrapper" />
|
||||
</p>
|
||||
|
||||
## The Keyless Endpoint
|
||||
|
||||
All connected providers share a single account endpoint, generated when you connect your
|
||||
|
||||
@@ -400,6 +400,16 @@ Configures the database backend for the embedded identity provider (Dex). By def
|
||||
</Property>
|
||||
</Properties>
|
||||
|
||||
### Agent Network Settings
|
||||
|
||||
Configures the [Agent Network](/agent-network) (LLM gateway) served by the combined server.
|
||||
|
||||
<Properties>
|
||||
<Property name="server.agentNetwork.pricingDefaultsFile" type="string">
|
||||
Path to a YAML file holding the default LLM pricing table used to seed the model catalog and meter request costs. A relative path is resolved against `dataDir`, so a bare filename such as `pricing.yaml` lands in the data directory. When empty, `{dataDir}/defaults_llm_pricing.yaml` is probed; if no file is present, the compiled-in defaults are used. The schema maps each surface (`openai`, `anthropic`, `bedrock`) to a model to its rates in USD per 1k tokens (`input_per_1k`, `output_per_1k`, and the optional `cached_input_per_1k` / `cache_read_per_1k` / `cache_creation_per_1k`). On the `bedrock` surface, model keys must be the **normalized** id costs are metered under rather than the full id pasted from AWS: drop any `arn:…` wrapper, the cross-region inference-profile prefix (`us.`, `eu.`, `apac.`, `global.`), and the trailing version or throughput suffix (`-v1:0`, `-20250929-v1:0`). For example, requests for `eu.anthropic.claude-sonnet-5` are priced from the key `anthropic.claude-sonnet-5`. File entries replace the compiled-in entry for the same surface and model; everything else keeps the compiled-in rates. The file is re-read periodically (mtime poll). An explicitly configured path that fails to load fails startup; runtime reload errors keep the previous table. Operator price overrides set in the dashboard take precedence over these defaults.
|
||||
</Property>
|
||||
</Properties>
|
||||
|
||||
---
|
||||
|
||||
## dashboard.env
|
||||
|
||||
Reference in New Issue
Block a user