Compare commits

...

42 Commits

Author SHA1 Message Date
mlsmaycon
494b22848b [management] Expose live model discovery on the provider API
Adds POST /api/agent-network/catalog/providers/models, so the provider
form can offer the models an operator's own credential can reach instead
of only the compiled-in catalog.

A caller names a catalog provider and supplies either the key they are
typing (the record does not exist yet) or the id of a saved record whose
stored credential should be reused — which lets the dashboard refresh a
list without ever holding the key. The two are mutually exclusive:
accepting both would run an arbitrary credential under the identity of a
record the caller may only be permitted to read. When a record id is
given, the catalog id and upstream come from the record too, so the
credential cannot be aimed at a different vendor's endpoint.

Gated on Create rather than Read. This spends the operator's credential
against a third party, which is not something a read-only role should be
able to make the server do.

A provider with no listing endpoint answers 422 rather than 500: the
caller falls back to the catalog's own models on that outcome, so it has
to be distinguishable from a failure.

The region is read back out of the configured upstream by matching it
against the catalog's host template, since a provider record has no
region field and the operator already encoded one when they set up
inference. An upstream matching no template is refused rather than
guessed at — a wrong region would dial another account's endpoint.
2026-08-19 11:00:15 +00:00
mlsmaycon
a33fab103d [management] Fetch a provider's model list from the vendor
The catalog is the only source of models an operator can pick from, and
it cannot know two things that matter. It goes stale — its entries carry
comments recording which models a vendor retired on which date — and it
cannot see an account: which OpenAI models an org is entitled to, which
Bedrock inference profiles a given account and region hold, which Vertex
models a project has enabled.

Add a client that asks the vendor directly, with the endpoint, auth
header and response shape all declared by the catalog rather than
supplied by the caller. The four shapes come from probing the live APIs
(see the discovery e2e); each vendor invented its own envelope and none
can be guessed from the request.

Bedrock is the case that shaped the design. Its listing lives on the
control plane while inference must go to the runtime host, so Discovery
carries its own host. The ids it returns are region-prefixed and are
taken verbatim, because that prefix is what AWS requires and it cannot
be derived from the configured region — an eu-central-1 account holds
global.* profiles alongside its eu.* ones.

The vendor is authoritative for the id; the catalog stays authoritative
for pricing. A discovered model the shipped table cannot price is
reported as such, so it cannot be registered at a silent zero rate.

Management has not made outbound calls on an operator's behalf before,
and it holds a credential for every provider, so the host is checked
before dialing: every resolved address must be public, which covers the
cloud metadata address and NetBird's own overlay range, and redirects
are not followed since they would move the request to a host the check
never saw.
2026-08-19 11:00:15 +00:00
mlsmaycon
3439500cc6 [e2e] Retry the endpoint probe when the tunnel is not up yet
ResolveProxyIP exists to wake the lazy proxy peer, and it retried only
curl exit 6 — DNS. The wake-up attempt that arrives before WireGuard has
brought the tunnel up fails with exit 7 instead, and that returned
immediately:

  no HTTP response from vast-azalea.netbird.local: exit status 7
  (curl: (7) Failed to connect ... after 0 ms)

So the one function whose job is to tolerate a not-yet-ready endpoint
failed on the readiness state it was written for, one second after the
client container reported ready. Retry both exit codes within the same
window; anything else would still be failing when the window closed and
still fails immediately.

Raise the access-log ingest window to 60s for the same reason. The proxy
streams each entry with a 10s send timeout of its own, so 30s left
barely three attempts of headroom before a test that had already got its
200 was failed for a row still in flight.
2026-08-18 19:31:52 +00:00
mlsmaycon
aa72df1654 [e2e] Assert what Bedrock discovery actually does against AWS
The first live run answered the question the mock could not. OpenAI and
Anthropic both filter correctly against real catalogues — Anthropic's
dated claude-haiku-4-5-20251001 survives a record registering the
undated id, and OpenAI's listing comes back as the single model the
guardrail permits. Vertex is refused by the proxy, as intended.

Bedrock is the one that was wrong, and wrong about something worth
recording: GET /inference-profiles reaches AWS and AWS answers
<UnknownOperationException/>. ListInferenceProfiles is a control-plane
operation on bedrock.<region>.amazonaws.com; a provider record carries a
single upstream and it must be the runtime host for InvokeModel to work,
so no Bedrock record can serve a listing as the model stands. The mock
serves that path on the same listener as everything else, which is
exactly why this went unnoticed.

Replace the routed/filtered pair with an explicit outcome, since the
three cases are different contracts rather than degrees of success, and
tell apart 'the proxy refused' from 'the vendor refused' by whether the
body names a middleware — no upstream error body does. The two
non-listing outcomes now issue a single request instead of retrying for
the full window waiting on a status that is never coming, which is where
92 of the failing run's 136 seconds went.
2026-08-18 19:08:48 +00:00
mlsmaycon
da155bd60f [e2e] Drive model discovery against the live vendor endpoints
The mock upstream advertises ids we chose, so a listing narrowing to the
ones we authorised is arithmetic we controlled both sides of. It cannot
show the filter surviving a real catalogue: ids we never enumerated,
dated builds whose suffix the vendor picks, surfaces that answer a
listing request with something that is not a listing.

Cover the four surfaces against their real endpoints, each gated on its
own credential so a partial key set still yields partial coverage:

  - OpenAI enumerates two real models and the policy permits one, so
    both bounds are observable at once against a catalogue of dozens.
  - Anthropic returns dated build ids while the record registers the
    undated one, which exercises date-normalisation on ids the vendor
    chose. This is also the surface Claude Code actually calls.
  - Bedrock lists inference profiles rather than models; the request is
    routed but not model-bounded, since filtering keys on /v1/models.
  - Vertex serves no listing at all, so discovery must be refused rather
    than rewritten onto an upstream that would 404 it.

One proxy serves every case, with a group, policy and client per
provider: a model-less request matches exactly one route, so two
providers authorised for the same caller would leave one untested.

Every response is logged before anything is asserted on it. A live
catalogue is the one input the suite does not control, so a failure has
to arrive carrying the response that caused it.
2026-08-18 18:42:15 +00:00
mlsmaycon
52bed3ec26 [e2e] Prove the other team's policy is live in the discovery test
The discovery isolation test drove a single client in the main group.
VLLMUnlistedModel's absence from that client's listing was consistent
with two different worlds: the listing being scoped to the caller's
policy, or the other team's policy never having applied at all. The
test passed either way, so it did not prove what it claimed.

Mint a setup key per group and join a second client on the other
group, reusing the running proxy. The other client must see its own
model before the main client's listing is asserted, and must not see
the main group's model — isolation is checked in both directions.
2026-08-18 16:59:48 +00:00
mlsmaycon
378fe13257 [proxy,management] Bound model discovery to the caller's own policies
The listing was narrowed by the provider record's enumerated models, which is
the right bound only while one policy reaches a provider. Where two teams
share a provider under different allowlists, every caller was offered the
union: each model outside their own policy is a request the guardrail refuses
a moment later, which is the empty-or-wrong picker this endpoint exists to
avoid, moved one level up. A gateway record enumerating nothing was worse
still — it offered the upstream's entire catalogue however narrow the policy.

The synthesiser already knows which policies authorise a provider and which
groups each binds, so the router can answer this at request time where it
knows the caller's groups. Each route now carries one rule per authorising
policy — its source groups and the models it permits — and the listing is
bounded to the union across the rules matching the caller, intersected with
what the provider serves.

This is deliberately finer than the guardrail's own per-provider allowlist,
which stays as it is: that list is a fail-closed backstop and cannot tell who
is asking, so discovery is now narrower than the backstop rather than wider.
A policy setting no allowlist lifts the restriction for the groups it binds,
so nil and empty model lists stay distinct end to end — collapsing them would
let a listing that should offer nothing fall open to everything.
2026-08-18 12:06:41 +00:00
mlsmaycon
520c912c07 [proxy] Cover the gateway-protocol gaps end to end
Six behaviours had unit coverage only, either because they arrived from code
review after the end-to-end tests were written or because no request in the
suite had the shape that reaches them.

Streaming is the important one. Input tokens exist only in a stream's opening
message_start event, and reading a stream with the wrong vendor's parser
misses it — the metering bug this endpoint's protocol work fixed. Nothing in
the suite sent stream: true, so the branch never ran. The mock now serves an
SSE surface on a second listener, reporting counts that differ from its
buffered ones so a passing assertion can only mean the stream accumulator ran,
and one case drives it through a record typed for the wrong surface.

The rest need no new harness capability: the per-model lookup against the
allowlist, the read-method gate on the non-inference paths, dated ids reaching
an undated registration while a pinned build refuses a different one, the
Bedrock inference-profile lookup reaching its upstream rather than a policy
denial, and a custom dated id keeping its own price.

Sub-agent ids stay uncovered: the parser lifts them onto request metadata but
nothing persists them, so there is no queryable surface to assert against
until that half lands. Covered here only to the extent that sending the
headers leaves the request served and metered.
2026-08-16 03:30:22 +00:00
mlsmaycon
9cd7e27027 [misc] Let the reprice e2e retry a request that never got logged
TestPriceChangeUpdatesRecordedCost drives requests in a loop until one is
priced at the new rate, because the price push and the proxy's chain rebuild
are async. The loop could not actually retry: it looked the row up through
findAccessLogBySession, which fails the test outright when no row lands
within 30s, so the first post-update request that produced no row ended the
run instead of yielding to the next attempt.

That is the observed failure — the nightly run has been red on this test
roughly half the time, always with "session id ...-reprice-b-... must be
recorded in an access-log row" after ~41s: container setup, one request, one
30s wait, dead.

A missing row there is expected rather than exceptional. The provider update
rebuilds the middleware chain, and a request served mid-rebuild can complete
without a resolved provider: 200 to the caller, nothing to attribute, so no
row is ever written for it. Split the polling helper into a non-fatal lookup
and keep the fail-fast wrapper for callers whose row must exist, then treat a
miss in the loop like any other not-yet-repriced iteration. Only the outer
deadline is fatal.

Shorten the per-attempt wait to 20s and raise the overall deadline to 180s so
several attempts fit where before the budget allowed barely one.
2026-08-16 02:24:18 +00:00
mlsmaycon
0783605690 Merge remote-tracking branch 'origin/main' into agent-network/gateway-protocol-conformance 2026-08-15 17:27:50 +00:00
Zoltan Papp
16544dbc58 [client] Pass stored email as login hint from UI and keep it on logout (#7199)
* [client] Pass stored email as login hint from UI and keep it on logout

Follow the CLI pattern: the Wails UI now reads the account email from the
user-owned profile state file and passes it as the OIDC login_hint on login
and session extend, since the daemon-side fallback runs as root and cannot
see the user's state file. Logout no longer deletes the stored email, so a
later login preselects the account at the IdP; profile removal remains the
operation that deletes it.

* [client] Log ignored profile lookup errors in extend-session hint fallback
2026-08-15 11:21:57 +02:00
Zoltan Papp
f458c1f265 [client] Skip IPv6 route tests when the default nexthop is unusable (#7212)
* [client] Skip IPv6 route tests when the default nexthop is unusable

ensureIPv6DefaultRoute treated a successful netlink RouteAdd as proof that
a usable IPv6 nexthop exists. Installing ::/0 via loopback can succeed while
the kernel still rejects that nexthop for a concrete prefix, which surfaced
on ubuntu22/20260810.260 runners as:

    add route to table: netlink add route: invalid argument

Probe the resolved nexthop by installing and removing a discard-prefix route
through the same code path the tests use, and skip when it fails. EEXIST
means the nexthop already carries a route, so it counts as usable.

* [client] Probe the IPv6 nexthop through raw netlink

addRoute swallows EAFNOSUPPORT and EOPNOTSUPP via isOpErr, so a nil return
did not prove the probe route was installed. Call netlink directly so an
unsupported operation skips the test instead of passing as usable.
2026-08-15 10:13:06 +02:00
Viktor Liu
ec6f1b8c27 [client] Rank Windows route candidates by combined route and interface metric (#7210) 2026-08-15 09:06:22 +02:00
Zoltan Papp
2cfe14d7ec [client] Keep account email on Android logout, drop it on profile removal (#7200)
Align Android logout semantics with the desktop UI and CLI: logging out no
longer deletes the stored account email, so the next login passes it as the
OIDC login_hint and the IdP preselects the account. Removing the profile is
now the operation that deletes the email; previously RemoveProfile left the
account file behind, which the fixed-name default profile would have
inherited on recreation.
2026-08-14 18:13:52 +02:00
Eduard Gert
85dd335836 [client] Add CI check for translation key parity (#6852)
English (en) is the source of truth for UI translation keys; the other
nine locales rely on runtime English fallback for any missing key, so a
gap never surfaces in CI. Add a dependency-free Node check that fails
when any locale declared in _index.json does not carry the exact same
key set as en (missing or orphaned keys), wired into a dedicated
UI Translations workflow that runs on locale changes.

Also close the one existing gap the check found: ja was missing
daemon.outdated.download ("Download Latest").

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-14 10:57:11 +02:00
Viktor Liu
5544761b47 [client] Add Windows DNS configuration to the debug bundle (#7196) 2026-08-13 20:07:37 +02:00
Kim Harre
1d372bb634 [infrastructure] Support non-interactive installation in getting-started.sh (#7168) 2026-08-13 18:58:03 +03:00
Viktor Liu
e290769df1 [client] Take the graphical session answer from the caller instead of the daemon environment (#7187) 2026-08-13 10:28:34 +02:00
Jack Carter
58c09ead21 [management] Document mutual exclusivity of policy rule ports and port_ranges (#7158) 2026-08-12 20:39:06 +02:00
mlsmaycon
dc91325ac1 [proxy] Keep deliberately pinned dated models distinct
Two changes to routeClaimsModel, both about dated Anthropic ids.

Normalising the configured candidate as well as the requested model made
every dated build of a family interchangeable: a route registered against
claude-sonnet-4-5-20250101 also claimed ...-20250202, so an operator who
pinned a build deliberately would have served a different one, and with
several such routes declaration or path order decided which. Only an undated
registration now absorbs a dated request.

The per-model lookup also stamps the model its path names, so the guardrail's
allowlist — a separate and possibly narrower list than the route's — still
decides GET /v1/models/{id} rather than seeing no model at all.
2026-08-11 15:20:18 +00:00
mlsmaycon
1796b2a1d8 [proxy] Let model discovery past the provider allowlist
The guardrail enforces its own per-provider model allowlist and fails closed
when the request names no model, which is right for a path-routed inference
request whose shape the parser could not read. GET /v1/models names no model
anywhere, so discovery still denied with model_unknown for exactly the
accounts that configured an allowlist — the case skipping the management
pre-flight was meant to fix. Only one of the two gates had been opened, and
a client reads the 403 as an empty model picker.

Exempt requests the router marked non-inference from the unknown-model
branch. A named model is still checked, so the exemption covers only the
endpoints that genuinely name nothing: the listing and the warm probe.
2026-08-11 15:20:18 +00:00
mlsmaycon
dd87760f3d [proxy] Forward an oversized model listing whole
The discovery filter read one byte past its 1 MiB cap to detect a body too
large to rewrite, then closed the upstream body and forwarded the buffer it
had — the response reached the client truncated at exactly the cap, with
Content-Length rewritten to match so nothing looked wrong until the client
tried to parse it.

Splice the bytes already read back in front of the unread remainder and
forward the response as the upstream sent it, headers untouched.
2026-08-11 15:20:06 +00:00
mlsmaycon
63d16a7cd4 [proxy] Name what the warm-probe assertions are pinning
The two assertions in the HEAD subtest carried no failure message, unlike the
rest of the file. Say which half failed.
2026-08-11 13:23:10 +00:00
mlsmaycon
fa691721d0 [proxy] Gate the non-inference mark on a read method
The router classified a non-inference request by path alone, so a POST to
/v1/models/{id} — or to the listing, or the warm probe — was marked
llm.non_inference and skipped the limit check's management pre-flight, even
though such a request can carry an inference body.

Require GET or HEAD, the methods these endpoints actually use. Anything else
falls through to normal per-model routing, which routes on the body's model
under the usual pre-flight, or denies as missing-model when there is none.
2026-08-11 13:16:17 +00:00
mlsmaycon
d46e2574a7 [proxy] Split the router's surface dispatch out of Invoke
Invoke had grown a branch per API surface, each repeating the same
found/unauthorised/unknown switch. Lift the shared denial arms into decide(),
move the model-less endpoints into their own method, and name the two allow
decorations (non-inference marking, Bedrock namespace stripping) so each
surface reads as the one thing it does differently.

No behaviour change.
2026-08-11 13:08:03 +00:00
mlsmaycon
649a867cd3 [proxy] Authorise the per-model lookup against the model table
GET /v1/models/{id} was folded into the model-less endpoints so it would
route rather than deny. Once model-less requests started skipping the
management pre-flight, that also skipped the per-model allowlist: a caller
could confirm the existence and reachability of a model the route does not
list, even though the listing beside it is bounded to that same allowlist.

Resolve the id from the path and route it through the model table like any
other per-model request, keeping model-less treatment for the listing and
the connection-warming probe only. It stays marked non-inference, since the
lookup spends no tokens. A gateway route that enumerates no models still
answers every lookup, as before.

Also record why the Bedrock inference-profile lookup is forwarded rather
than denied: those live on the AWS control plane, and forwarding reproduces
what an unproxied client with the same base URL would see.
2026-08-11 13:04:50 +00:00
mlsmaycon
a39b3c4af4 [proxy] Anchor the Anthropic date strip to Claude ids
The release-date normalizer matched a bare "-YYYYMMDD" suffix on any id.
Pricing looks every model up through it regardless of surface, and an
operator can register a custom model under any id at all, so a custom
"internal-llm-20250101" would silently inherit the rate registered for
"internal-llm".

Anchor the pattern on "claude" so it still covers the vendor-prefixed
Bedrock forms while leaving every other vendor's id untouched.
2026-08-11 13:04:41 +00:00
mlsmaycon
b1337f09d0 [proxy] Add e2e cover for the gateway protocol changes
The routing and parser-selection fixes touch every provider surface, and
the unit tests only prove each side of a seam in isolation. Two suites
close that:

The provider matrix drives one request per wire shape over a single tunnel,
with a record per catalog surface behind it, and asserts the surface each
request was metered under together with the token counts that surface's own
usage block carries. A response read by the wrong provider's parser meters
zero, so a regression fails instead of passing on a coincidental non-zero.
It also covers the Bedrock and Vertex token-counting paths, the warm-up
probe, and the vendor error envelope on a refusal.

The discovery suite covers the configuration that broke: an account with a
model allowlist, where the listing carries no model and the gate failed
closed. It asserts the listing is served, that it is bounded to the
authorised model, and that inference outside the allowlist is still
refused, so the exemption cannot be read as a way around the gate.

The mock upstream grows the Anthropic, Bedrock and token-counting shapes so
one container stands in for every surface, and the client gains GET and
arbitrary-POST helpers for the endpoints that carry no chat body.
2026-08-11 03:26:52 +00:00
mlsmaycon
08e187ccb7 [proxy] Keep slash-bearing model ids in the discovery filter
The filter treated a slash in a listing entry's id as a gateway provider
prefix and matched only the tail. Self-hosted backends serve ids that carry
a slash of their own, so every "Qwen/Qwen2.5-0.5B-Instruct" style model was
dropped from the picker even when the policy named it exactly.

Try the id as written first and fall back to the tail, so both a prefixed
id and a self-hosted one resolve.
2026-08-11 03:20:26 +00:00
mlsmaycon
93cdf64a19 [docs] Document the client checks that bypass the agent network endpoint
A few client-side checks call their vendor directly instead of following
the configured base URL, so they fail on exactly the locked-down networks
Agent Network is built for while inference keeps working. Fast mode reports
a connectivity error, or reports itself disabled by the organization when
the agent holds only a proxy-issued token, and model discovery stays off
until it is turned on explicitly.

Name the variables that settle each case, and say plainly which ones
allowing direct egress does not fix.
2026-08-11 03:07:59 +00:00
mlsmaycon
c04fb1c388 [proxy] Capture sub-agent ids from LLM request headers
A coding agent that spawns helpers stamps the spawned agent's id on every
request it makes, and the spawning agent's id when that helper is nested.
The parser read the session header and ignored both, so parallel agents
inside one session all attributed to the session alone and there was no way
to see which one spent the tokens.

Emit them as metadata alongside the session id. They are opaque grouping
identifiers rather than content, so they are stamped regardless of the
prompt-collection toggle. Persisting them as queryable access-log columns
is a schema change and is deliberately not part of this commit.
2026-08-11 03:07:59 +00:00
mlsmaycon
5353cab54f [proxy] Bound the model-listing response to what policy authorises
Discovery proxies the upstream's full list, so the picker offers every
model the shared provider key can reach and each one outside the policy is
a request the chain denies a moment later. Restricting models is the point
of the product, and the client had no way to see the restriction.

Carry the resolved route's model list on the upstream rewrite and drop the
rest from the listing response. Only a route that enumerates its models
bounds anything: a catch-all claims every model, so its list passes
through. Anything the filter cannot safely rewrite, including a compressed
or oversized body, reaches the client untouched.
2026-08-11 03:07:59 +00:00
mlsmaycon
a64a417ea5 [proxy] Forward the Anthropic connection-warming probe
Clients send HEAD /api/hello before their first inference request to open
the upstream connection early. The path carries no model, so it denied as
not-routable and each session start left a policy rejection in the access
log for a request that was never a policy question.

Treat it as a model-less endpoint. Forwarding it warms the connection the
first real request will use, which is what the probe is for.
2026-08-11 02:58:22 +00:00
mlsmaycon
652c5c8b68 [proxy] Route Bedrock inference-profile lookups
A client resolving a configured inference profile calls
GET /inference-profiles at startup. The path carries no model and was not
recognised as non-inference, so it denied as not-routable and wrote a
policy rejection into the access log on every session start, which is the
log operators read to find real policy problems.

Recognise the path and match it against a Bedrock provider specifically:
sending it to whichever provider happened to be authorised first would
rewrite it to an upstream that 404s it. The optional gateway namespace is
stripped the same way the runtime paths strip it.
2026-08-11 02:57:57 +00:00
mlsmaycon
4d2b8b407b [proxy] Keep the Vertex model id out of the count-tokens method segment
Vertex hangs token counting off the model as its own path segment, and the
parser split the tail on the final colon alone. A count-tokens request
therefore reported its model as "claude-sonnet-5/count-tokens", which no
route claims, so the request denied as not-routable and the access log
recorded a model that does not exist.

Stop at the first "/" after the model id so the method segment stays out of
it, leaving the client free to price its context against the dedicated
endpoint instead of the billable inference one.
2026-08-11 02:56:29 +00:00
mlsmaycon
03e02c86ce [proxy] Route the Bedrock count-tokens action
Both the request parser and the router enumerated Bedrock actions without
count-tokens, so the path carried no model and the request denied as
not-routable. Nothing breaks outright, because the client falls back to
counting context through the inference endpoint, but that fallback is
billable and the dedicated endpoint exists to avoid exactly that.

The action carries a model in the path and returns no usage, so it routes
like any other Bedrock action and meters to zero.
2026-08-11 02:56:00 +00:00
mlsmaycon
789d416215 [proxy] Mirror LLM denials in the caller's provider error shape
A budget stop, a blocked model or an unroutable model all rendered as the
NetBird deny envelope alone. LLM clients only parse their own provider's
error shape, so the reason never reached the user: Claude Code showed an
unexplained API error where it could have shown the policy message.

Carry the resolved surface on the deny reason and add the vendor's error
object next to the existing fields. The body stays a superset of what it
was, so anything reading code, message, details or middleware is
unaffected. Status codes are unchanged here: mapping window caps to 429
needs the window reset plumbed through the limits response before a
correct Retry-After can be sent.
2026-08-11 02:54:54 +00:00
mlsmaycon
6415215126 [proxy] Skip OpenAI-shape identity injection on non-OpenAI bodies
Gateway records enable body-level identity so LiteLLM's tag-budget check
can read it, and the injector wrote "user" and "metadata.tags" into every
JSON object regardless of dialect. Claude Code reaches those same records
on /v1/messages, where "user" is not a permitted top-level field and
metadata accepts only "user_id", so the upstream rejected the request with
a 400 naming a field the client never sent. Rewriting the body also
changed the bytes a gateway-side prompt cache keys on.

Gate the body write on the surface llm_request_parser resolved from the
path. Header stamping is untouched, so spend tracking and per-end-user
budgets keep working on the surfaces that lose the body path.
2026-08-11 02:52:11 +00:00
mlsmaycon
1ae352a08d [proxy] Match dated Anthropic model ids against their undated form
shared/llm normalizes Bedrock and Vertex model ids so both sides of the
routing and pricing contract compare equal, but nothing did the same for a
first-party Anthropic id. A client pinning "claude-sonnet-4-5-20250929"
against a record registered as "claude-sonnet-4-5" denied as not-routable,
and where a catch-all route carried it through, the price lookup missed and
the request recorded no cost.

Add NormalizeAnthropicModel beside the existing two and consult it after an
exact match fails in the router's claim check, the pricing table, and the
per-record price map. Exact matches still win, so an operator who registers
two dated releases of the same family keeps them distinct.
2026-08-11 02:50:43 +00:00
mlsmaycon
d928bcb630 [proxy] Exempt non-inference endpoints from the model allowlist gate
GET /v1/models carries no model, and management's per-model allowlist
fails closed on an undetermined one, so gateway model discovery denied
with model_blocked for every account that enables a model allowlist. The
client treats a failed discovery as silent and falls back to its built-in
list, so the operator sees an empty picker with no error to chase.

The router already classifies these paths and authorises the route against
the caller's groups before allowing them, so mark them non-inference there
and let the limits gate skip a pre-flight that has no model to evaluate
and no tokens to book. The marker comes from the router's own path
classification, never from client input, so an inference request cannot
set it to escape the allowlist.
2026-08-11 02:47:19 +00:00
mlsmaycon
47b2667653 [proxy] Select the LLM parser by request path before provider_id
Gateway catalog entries pin provider_id "openai", and the same record
serves Claude Code on /v1/messages. The parser preferred the pinned id
over the path, so an Anthropic body was read with the OpenAI parser: on a
streaming response the input tokens ride message_start nested under
message, which that parser never reads, so input counted as zero. Both
cache buckets were dropped, and pricing resolved against the openai
surface where no claude-* model exists, skipping cost entirely.

Detect from the path first and keep provider_id as the fallback for
upstreams whose path carries no surface. The Kimi entry already leaves
ParserID empty to work around this; the fallback ordering makes that
unnecessary.
2026-08-11 02:45:23 +00:00
mlsmaycon
875dda1708 [management] Add the Claude 5 lineup to the Agent Network catalog
Claude Code resolves to Opus 5 and Sonnet 5 by default, and neither was
selectable on a provider record. An operator building a record from the
catalog could not authorise the client's own default, so llm_router denied
those requests as model_not_routable. Opus 5 carried a supplemental pricing
row that priced gateway traffic but never reached the dashboard; Sonnet 5
was absent everywhere, so a request that did route through a catch-all
gateway recorded zero cost and under-counted every budget it should have
charged.

Add both to the Anthropic, Bedrock and Vertex lineups at the published
rates, and drop the supplemental rows now that the catalog carries them.
2026-08-11 02:44:02 +00:00
84 changed files with 7108 additions and 290 deletions

42
.github/workflows/ui-translations.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: UI Translations
on:
pull_request:
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
- ".github/workflows/ui-translations.yml"
push:
branches:
- main
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
jobs:
check-translations:
name: Check translation key parity
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "22"
# English (en) is the source of truth for translation keys; every other
# locale declared in _index.json must carry the exact same key set.
- name: Check translation key parity
run: node client/ui/i18n/check-translations.mjs

View File

@@ -40,6 +40,35 @@ You can then use this private endpoint to configure your AI agents, whether that
Full step-by-step setup:
**https://docs.netbird.io/agent-network/quickstart**
## Client settings that don't follow the endpoint
Most of an agent's traffic follows the base URL you hand it, but a few
client-side checks call their vendor directly and never reach the proxy. On a
network that blocks direct egress they fail even though inference works, so
they are worth setting once when you roll the endpoint out.
For Claude Code:
- **Fast mode** checks availability against `api.anthropic.com` rather than the
configured base URL. Set `CLAUDE_CODE_SKIP_FAST_MODE_ORG_CHECK=1` when the
agent authenticates with `ANTHROPIC_AUTH_TOKEN` alone (the usual shape when
the proxy injects the real provider key) or when a TLS-inspecting proxy
answers the check itself. Set
`CLAUDE_CODE_SKIP_FAST_MODE_NETWORK_ERRORS=1` when the network refuses the
connection outright. Fast mode is an Anthropic-API feature, so it is
unavailable on a Bedrock- or Vertex-backed endpoint whatever you set.
- **Model discovery** is off by default. Set
`CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` for the picker to list the
models your policies authorise; the proxy filters the response to that set.
The client gives discovery a three-second budget and treats any redirect as
a failure, so the endpoint must serve `/v1/models` directly.
- **The WebFetch domain safety check** also calls `api.anthropic.com` directly
and is unaffected by the variables above.
Allowing direct egress to `api.anthropic.com` covers the network cases but not
the credential one, where the check reaches Anthropic and is rejected because
the agent presents a proxy-issued key.
## Architecture
Agent Network is built on two existing NetBird capabilities:

View File

@@ -204,8 +204,9 @@ func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}
// An empty hint is deliberate, not a fallback: a fresh or logged-out profile
// leaves the choice to the IdP, which is how accounts get switched.
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
if a.cfgPath != "" {
if hint := readProfileEmail(a.cfgPath); hint != "" {
if setter, ok := oAuthFlow.(loginHintSetter); ok {

View File

@@ -22,7 +22,8 @@ type Profile struct {
ID string
Name string
// Email is the account this profile last logged in with, "" if it never
// completed an SSO login or was logged out. See profile_state.go.
// completed an SSO login. Kept across logouts; cleared when the profile is
// removed. See profile_state.go.
Email string
IsActive bool
}
@@ -200,11 +201,9 @@ func (pm *ProfileManager) LogoutProfile(id string) error {
return fmt.Errorf("failed to save config: %w", err)
}
// Not fatal: a stale hint costs an account switch, not the logout itself.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to clear stored account email for profile %s: %v", id, err)
}
// The stored account email is kept on purpose, matching the desktop and CLI
// logout semantics: the next login passes it as the login_hint so the IdP
// preselects the account. Removing the profile is what deletes it.
log.Infof("logged out from profile: %s", id)
return nil
}
@@ -224,11 +223,24 @@ func (pm *ProfileManager) RenameProfile(id string, newName string) error {
// RemoveProfile deletes a profile
func (pm *ProfileManager) RemoveProfile(id string) error {
configPath, err := pm.getProfileConfigPath(id)
if err != nil {
return err
}
// Use ServiceManager (removes profile from profiles/ directory)
if err := pm.serviceMgr.RemoveProfile(profilemanager.ID(id), androidUsername); err != nil {
return fmt.Errorf("failed to remove profile: %w", err)
}
// The account file is this package's, not the ServiceManager's, so it must
// go here. The default profile has a fixed filename, so a recreated one
// would otherwise inherit the deleted profile's email as its login_hint.
// Not fatal: the profile itself is gone.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to remove stored account email for profile %s: %v", id, err)
}
log.Infof("removed profile: %s", id)
return nil
}

View File

@@ -90,10 +90,10 @@ func writeProfileEmail(configPath string, email string) error {
return nil
}
// removeProfileEmail drops the stored account email. Called on logout: while the
// email is on disk it goes out as a login_hint, which would steer the next login
// straight back into the account just logged out of. Mirrors the desktop UI's
// RemoveProfileState call.
// removeProfileEmail drops the stored account email. Called on profile removal,
// not on logout: a logged-out profile keeps its email so the next login passes
// it as the login_hint, matching the desktop and CLI semantics. Mirrors the
// desktop UI's RemoveProfileState call.
func removeProfileEmail(configPath string) error {
accountPath, err := profileAccountPathFor(configPath)
if err != nil {

View File

@@ -127,10 +127,10 @@ func TestWriteThenReadProfileEmail(t *testing.T) {
t.Fatalf("remove: %v", err)
}
if got := readProfileEmail(configPath); got != "" {
t.Errorf("expected no email after logout, got %q", got)
t.Errorf("expected no email after removal, got %q", got)
}
// Logout may run on a never-logged-in profile, so a second remove must pass.
// Removal may run on a never-logged-in profile, so a second remove must pass.
if err := removeProfileEmail(configPath); err != nil {
t.Fatalf("second remove should be a no-op: %v", err)
}

View File

@@ -305,6 +305,12 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
return domain
}
// A reverse zone names an address prefix, so it follows the address rules,
// which also keeps its digit labels intact.
if zone, ok := a.anonymizeReverseZone(baseDomain); ok {
return withTrailingDot(zone, hasDot)
}
if suffix := protectedSuffix(baseDomain); suffix != "" {
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
return domain
@@ -405,6 +411,10 @@ func (a *Anonymizer) AnonymizeString(str string) string {
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
// Reverse zones go first and are then held out of the passes below: their
// labels are digits, which the address patterns would otherwise consume.
str, restoreZones := a.replaceReverseZones(str)
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
@@ -425,7 +435,7 @@ func (a *Anonymizer) AnonymizeString(str string) string {
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
}
return str
return restoreZones(str)
}
// sortedDomains returns the domain mappings longest-first, so a full-FQDN

View File

@@ -0,0 +1,174 @@
package anonymize
import (
"encoding/hex"
"net/netip"
"regexp"
"strconv"
"strings"
)
const (
reverseZoneSuffixV4 = ".in-addr.arpa"
reverseZoneSuffixV6 = ".ip6.arpa"
v6Nibbles = 32
v4Octets = 4
)
// reverseZoneRegexes match a reverse zone or a full reverse name in free text.
// They are applied before the address passes of AnonymizeString, whose IPv4
// pattern would otherwise consume the digit labels of a zone and replace parts
// of it with unrelated addresses.
var reverseZoneRegexes = []*regexp.Regexp{
regexp.MustCompile(`(?:[0-9]{1,3}\.){1,4}in-addr\.arpa\b`),
regexp.MustCompile(`(?:[0-9a-fA-F]\.){1,32}ip6\.arpa\b`),
}
// anonymizeReverseZone maps a reverse zone to the zone of the anonymized form
// of the prefix it encodes, so it follows the address rules rather than the
// domain ones: the zone of an address that is preserved is preserved too, and
// the zone of one that is replaced names the replacement. This keeps a reverse
// zone recognizable as such, and consistent with the addresses it belongs to
// elsewhere in the same output. It reports false for anything that is not a
// reverse zone.
func (a *Anonymizer) anonymizeReverseZone(domain string) (string, bool) {
prefix, labelCount, suffix, ok := parseReverseZone(domain)
if !ok {
return "", false
}
anonymized := a.AnonymizeIP(prefix)
if anonymized == prefix {
return domain, true
}
return reverseZoneName(anonymized, labelCount) + suffix, true
}
// replaceReverseZones anonymizes every reverse zone in str and swaps each one
// for a placeholder, returning a function that puts the anonymized zones back.
// The placeholders carry no dots, digits or colons, so no later pass matches
// them.
func (a *Anonymizer) replaceReverseZones(str string) (string, func(string) string) {
var zones []string
for _, re := range reverseZoneRegexes {
str = re.ReplaceAllStringFunc(str, func(match string) string {
zone, ok := a.anonymizeReverseZone(match)
if !ok {
return match
}
zones = append(zones, zone)
return reverseZonePlaceholder(len(zones) - 1)
})
}
if len(zones) == 0 {
return str, func(s string) string { return s }
}
return str, func(s string) string {
for i, zone := range zones {
s = strings.ReplaceAll(s, reverseZonePlaceholder(i), zone)
}
return s
}
}
func reverseZonePlaceholder(index int) string {
return "\x00reversezone" + strconv.Itoa(index) + "\x00"
}
// parseReverseZone turns a reverse zone into the address of the prefix its
// labels spell backwards, padding the absent low-order part with zeroes, and
// returns the label count and zone suffix so the name can be rebuilt.
func parseReverseZone(domain string) (netip.Addr, int, string, bool) {
lower := strings.ToLower(domain)
switch {
case strings.HasSuffix(lower, reverseZoneSuffixV4):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV4), ".")
addr, ok := reverseZoneAddrV4(labels)
return addr, len(labels), reverseZoneSuffixV4, ok
case strings.HasSuffix(lower, reverseZoneSuffixV6):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV6), ".")
addr, ok := reverseZoneAddrV6(labels)
return addr, len(labels), reverseZoneSuffixV6, ok
default:
return netip.Addr{}, 0, "", false
}
}
func reverseZoneAddrV4(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v4Octets {
return netip.Addr{}, false
}
var octets [v4Octets]byte
for i, label := range labels {
octet, err := strconv.ParseUint(label, 10, 8)
if err != nil {
return netip.Addr{}, false
}
octets[len(labels)-1-i] = byte(octet)
}
return netip.AddrFrom4(octets), true
}
func reverseZoneAddrV6(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v6Nibbles {
return netip.Addr{}, false
}
nibbles := make([]byte, 0, v6Nibbles)
for i := len(labels) - 1; i >= 0; i-- {
if len(labels[i]) != 1 || !isHexDigit(labels[i][0]) {
return netip.Addr{}, false
}
nibbles = append(nibbles, labels[i][0])
}
for len(nibbles) < v6Nibbles {
nibbles = append(nibbles, '0')
}
var groups []string
for i := 0; i < len(nibbles); i += 4 {
groups = append(groups, string(nibbles[i:i+4]))
}
addr, err := netip.ParseAddr(strings.Join(groups, ":"))
if err != nil {
return netip.Addr{}, false
}
return addr, true
}
// reverseZoneName spells the first labelCount labels of addr backwards, the
// inverse of parseReverseZone, without the zone suffix.
func reverseZoneName(addr netip.Addr, labelCount int) string {
labels := make([]string, 0, labelCount)
if addr.Is4() {
octets := addr.As4()
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, strconv.Itoa(int(octets[i])))
}
return strings.Join(labels, ".")
}
address := addr.As16()
nibbles := hex.EncodeToString(address[:])
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, string(nibbles[i]))
}
return strings.Join(labels, ".")
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}

View File

@@ -0,0 +1,171 @@
package anonymize
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newLeveledAnonymizer(level Level) *Anonymizer {
a := NewAnonymizer(DefaultAddresses())
a.SetLevel(level)
return a
}
// TestAnonymizeDomainReverseZone covers reverse zones going through the address
// rules instead of the domain ones, so a zone stays a zone and an address that
// is preserved keeps the zone that names it.
func TestAnonymizeDomainReverseZone(t *testing.T) {
// 100.64.0.0/10 is the overlay range, which is CGNAT: preserved at the
// default level and replaced from the internal pool at the strict one
const overlayZone = "64.100.in-addr.arpa"
t.Run("overlay zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, overlayZone, a.AnonymizeDomain(overlayZone), "should keep the zone of a preserved address")
})
t.Run("private zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "168.192.in-addr.arpa", a.AnonymizeDomain("168.192.in-addr.arpa"), "should keep the zone of a private address")
})
t.Run("overlay zone replaced at the strict level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelStrict)
got := a.AnonymizeDomain(overlayZone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, overlayZone, got, "should replace the encoded prefix")
assert.Len(t, strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV4), "."), 2,
"should keep the label count, got %q", got)
})
t.Run("public zone replaced at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("113.0.203.in-addr.arpa")
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, "113.0.203.in-addr.arpa", got, "should replace a public prefix")
})
t.Run("zone of an address keeps that address mapping", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
anonymizedAddr := a.AnonymizeIPString("203.0.113.7")
got := a.AnonymizeDomain("7.113.0.203.in-addr.arpa")
octets := strings.Split(anonymizedAddr, ".")
want := octets[3] + "." + octets[2] + "." + octets[1] + "." + octets[0] + reverseZoneSuffixV4
assert.Equal(t, want, got, "should name the same replacement as the address itself")
})
t.Run("ipv6 nibble labels stay single digits", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeDomain(zone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV6), "should stay a reverse zone, got %q", got)
labels := strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV6), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
for _, label := range labels {
assert.Len(t, label, 1, "nibble label %q should stay a single digit", label)
}
})
t.Run("trailing dot is kept", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "64.100.in-addr.arpa.", a.AnonymizeDomain("64.100.in-addr.arpa."), "should keep the trailing dot")
})
t.Run("a domain that only looks like a zone is anonymized as a domain", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("not-a-zone.in-addr.arpa")
assert.NotContains(t, got, "in-addr.arpa", "should fall back to domain anonymization")
})
}
// TestAnonymizeStringReverseZone verifies that a zone inside free text, such as
// a DNS log line, is not chewed up by the address passes. The IPv4 pattern
// matches any run of dotted digits, which a reverse zone is made of.
func TestAnonymizeStringReverseZone(t *testing.T) {
t.Run("ipv6 zone survives the address passes", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeString("question: domain=" + zone + " type=PTR")
assert.Contains(t, got, "type=PTR", "should keep the rest of the line")
assert.NotContains(t, got, "198.51.100", "should not rewrite nibble labels as an address")
labels := strings.Split(strings.TrimSuffix(strings.TrimPrefix(got, "question: domain="), reverseZoneSuffixV6+" type=PTR"), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
})
t.Run("preserved ipv4 zone is untouched", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
line := "reverse zone 64.100.in-addr.arpa registered"
assert.Equal(t, line, a.AnonymizeString(line), "should keep the zone of a preserved address")
})
t.Run("public ipv4 zone is replaced consistently", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeString("zone 113.0.203.in-addr.arpa and address 203.0.113.7")
assert.NotContains(t, got, "113.0.203.in-addr.arpa", "should replace the zone")
assert.NotContains(t, got, "203.0.113.7", "should replace the address")
assert.Contains(t, got, reverseZoneSuffixV4, "should keep the zone suffix")
})
}
func TestParseReverseZone(t *testing.T) {
tests := []struct {
name string
zone string
addr string
labels int
}{
{name: "v4 two labels", zone: "0.100" + reverseZoneSuffixV4, addr: "100.0.0.0", labels: 2},
{name: "v4 three labels", zone: "1.168.192" + reverseZoneSuffixV4, addr: "192.168.1.0", labels: 3},
{name: "v4 full address", zone: "7.113.0.203" + reverseZoneSuffixV4, addr: "203.0.113.7", labels: 4},
{
name: "v6 prefix",
zone: "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6,
addr: "2::",
labels: 28,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
addr, labels, suffix, ok := parseReverseZone(tc.zone)
require.True(t, ok, "should decode the reverse zone")
assert.Equal(t, tc.addr, addr.String(), "should decode to the encoded prefix")
assert.Equal(t, tc.labels, labels, "should count the labels")
assert.Equal(t, tc.zone, reverseZoneName(addr, labels)+suffix, "should re-encode to the original zone")
})
}
}
func TestParseReverseZoneRejectsNonZones(t *testing.T) {
tests := []string{
"example.com",
"in-addr.arpa",
"x.100" + reverseZoneSuffixV4,
"256" + reverseZoneSuffixV4,
"1.2.3.4.5" + reverseZoneSuffixV4,
"ab" + reverseZoneSuffixV6,
"g" + reverseZoneSuffixV6,
}
for _, zone := range tests {
t.Run(zone, func(t *testing.T) {
_, _, _, ok := parseReverseZone(zone)
assert.False(t, ok, "should reject %q", zone)
})
}
}

View File

@@ -5,7 +5,6 @@ import (
"fmt"
"os"
"os/user"
"runtime"
"strings"
log "github.com/sirupsen/logrus"
@@ -121,7 +120,7 @@ func doDaemonLogin(ctx context.Context, cmd *cobra.Command, providedSetupKey str
loginRequest := proto.LoginRequest{
SetupKey: providedSetupKey,
ManagementUrl: managementURL,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
DnsLabels: dnsLabelsReq,
ProfileName: &handle,
@@ -189,7 +188,8 @@ func doExtendSession(ctx context.Context, cmd *cobra.Command) error {
client := proto.NewDaemonServiceClient(conn)
req := &proto.RequestExtendAuthSessionRequest{}
// the CLI runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: util.HasGraphicalSession()}
// Pre-fill the IdP login hint from the active profile so the user
// doesn't have to retype their email. Best-effort: we still proceed
// without a hint if the lookup fails.
@@ -408,7 +408,7 @@ func foregroundGetTokenInfo(ctx context.Context, cmd *cobra.Command, config *pro
hint = profileState.Email
}
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isUnixRunningDesktop(), false, hint)
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, util.HasGraphicalSession(), false, hint)
if err != nil {
return nil, err
}
@@ -458,14 +458,6 @@ func openURL(cmd *cobra.Command, verificationURIComplete, userCode string, noBro
}
}
// isUnixRunningDesktop checks if a Linux OS is running desktop environment
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func setEnvAndFlags(cmd *cobra.Command) error {
SetFlagsFromEnvVars(rootCmd)

View File

@@ -21,8 +21,8 @@ import (
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
@@ -626,7 +626,7 @@ func setupLoginRequest(providedSetupKey string, customDNSAddressConverted []byte
NatExternalIPs: natExternalIPs,
CleanNATExternalIPs: natExternalIPs != nil && len(natExternalIPs) == 0,
CustomDNSAddress: customDNSAddressConverted,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
ExtraIFaceBlacklist: extraIFaceBlackList,
DnsLabels: dnsLabels,

View File

@@ -51,6 +51,7 @@ nftables.txt: Anonymized nftables rules with packet counters across all families
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
config.txt: Anonymized configuration information of the NetBird client.
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
@@ -237,6 +238,13 @@ scutil_dns.txt (macOS only):
- Shows DNS configuration for all network interfaces
- Includes search domains, nameservers, and DNS resolver settings
- All IP addresses and domain names are anonymized
dns_windows.txt (Windows only):
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
- All IP addresses and domain names are anonymized
`
const (

View File

@@ -1,4 +1,4 @@
//go:build !unix
//go:build !unix && !windows
package debug

View File

@@ -0,0 +1,443 @@
//go:build windows
package debug
import (
"encoding/hex"
"errors"
"fmt"
"net/netip"
"strings"
"unsafe"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
nbdns "github.com/netbirdio/netbird/client/internal/dns"
)
const dnsInfoFileName = "dns_windows.txt"
const (
gpoDNSClientRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient`
tcpipParamsPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters`
dnscacheParams = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters`
)
// interfaceDNSValues are the per-interface values that decide how a name is
// resolved and registered. Everything the DNS host manager writes is in here,
// so a bundle shows both what we set and what it replaced.
var interfaceDNSValues = []string{
"NameServer",
"DhcpNameServer",
"Domain",
"DhcpDomain",
"SearchList",
"RegistrationEnabled",
"DisableDynamicUpdate",
"MaxNumberOfAddressesToRegister",
"EnableDHCP",
}
// addDNSInfo collects and adds DNS configuration information to the archive
func (g *BundleGenerator) addDNSInfo() error {
if err := g.addFileToZip(strings.NewReader(g.collectDNSInfo()), dnsInfoFileName); err != nil {
return fmt.Errorf("add DNS info to zip: %w", err)
}
return nil
}
// collectDNSInfo renders the report. Everything below it reaches the platform
// through COM and through lazily resolved procedures, which panic when a
// procedure is missing rather than returning an error, and a debug bundle is not
// allowed to take the daemon down. The panic is contained here, and whatever was
// collected before it is kept and reported with it.
func (g *BundleGenerator) collectDNSInfo() (content string) {
var sb strings.Builder
defer func() {
if r := recover(); r != nil {
log.Errorf("collecting Windows DNS configuration panicked: %v", r)
fmt.Fprintf(&sb, "\nerror: collection stopped: %v\n", r)
}
content = sb.String()
}()
sb.WriteString("Windows DNS configuration\n")
sb.WriteString("=========================\n")
adapters, adaptersErr := adapterAddresses()
g.writeNRPTRules(&sb, "NRPT rules, local policy store", nbdns.DNSPolicyConfigRoot)
g.writeNRPTRules(&sb, "NRPT rules, group policy store", nbdns.GPODNSPolicyConfigRoot)
g.writeEffectiveNRPTPolicies(&sb)
g.writeRegistryKey(&sb, "DNS client group policy", gpoDNSClientRoot)
g.writeRegistryKey(&sb, "Global TCP/IP parameters", tcpipParamsPath)
g.writeRegistryKey(&sb, "Dnscache parameters", dnscacheParams)
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv4", nbdns.InterfaceConfigPath, adapterNames(adapters))
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv6", nbdns.InterfaceConfigPathV6, adapterNames(adapters))
g.writeAdapterDNS(&sb, adapters, adaptersErr)
return sb.String()
}
// writeNRPTRules lists every rule in a policy store, ours and any other
// product's, since a foreign rule for the same namespace decides resolution
// just as ours does. Rules the client wrote are marked.
func (g *BundleGenerator) writeNRPTRules(sb *strings.Builder, title, root string) {
writeSection(sb, title, root)
names, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(names) == 0 {
sb.WriteString("no rules\n")
return
}
for _, name := range names {
owner := ""
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(nbdns.NRPTKeyPrefix)) {
owner = " (netbird)"
}
fmt.Fprintf(sb, "%s%s\n", name, owner)
g.writeValues(sb, root+`\`+name, nil, " ")
}
}
// writeEffectiveNRPTPolicies reports the table the resolver answers from, which
// the registry cannot show: a rule is written before it is loaded, and it keeps
// being enforced after its key is gone until the resolver reloads its policy.
func (g *BundleGenerator) writeEffectiveNRPTPolicies(sb *strings.Builder) {
writeSection(sb, "NRPT policy table in effect", nrptPolicyClass+"."+nrptPolicyMethod+" in "+nrptPolicyNamespace)
entries, err := effectiveNRPTPolicies()
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(entries) == 0 {
sb.WriteString("no policies\n")
return
}
for _, entry := range entries {
fmt.Fprintf(sb, "%s\n", g.anonymizeValue("Namespace", entry.namespace))
for _, value := range entry.values {
fmt.Fprintf(sb, " %s: %s\n", value.name, g.anonymizeValue(value.name, value.value))
}
}
}
// writeInterfaceDNS reports the DNS values of every interface that has any, so
// the netbird interface can be compared against the physical ones. The registry
// keys the values by GUID, so each is named from the adapter list; a GUID with
// no adapter is a leftover key of an interface that no longer exists.
func (g *BundleGenerator) writeInterfaceDNS(sb *strings.Builder, title, root string, names map[string]string) {
writeSection(sb, title, root)
guids, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
var reported int
for _, guid := range guids {
var iface strings.Builder
g.writeValues(&iface, root+`\`+guid, interfaceDNSValues, " ")
if iface.Len() == 0 {
continue
}
name, ok := names[strings.ToLower(guid)]
if !ok {
name = "no adapter with this GUID"
}
reported++
fmt.Fprintf(sb, "%s (%s)\n%s", guid, name, iface.String())
}
if reported == 0 {
sb.WriteString("no interface holds DNS values\n")
}
}
// writeRegistryKey reports the values of a single key, without its subkeys.
func (g *BundleGenerator) writeRegistryKey(sb *strings.Builder, title, path string) {
writeSection(sb, title, path)
var values strings.Builder
g.writeValues(&values, path, nil, "")
if values.Len() == 0 {
sb.WriteString("no values\n")
return
}
sb.WriteString(values.String())
}
// writeValues renders the values of a key. A nil names list reports every
// value, otherwise only those named and present.
func (g *BundleGenerator) writeValues(sb *strings.Builder, path string, names []string, indent string) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, path, registry.QUERY_VALUE)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, windows.ERROR_PATH_NOT_FOUND):
// an absent key is the normal state for the GPO store and for
// interfaces without DNS settings
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", path)
return
case err != nil:
fmt.Fprintf(sb, "%serror: open HKEY_LOCAL_MACHINE\\%s: %v\n", indent, path, err)
return
}
defer closeKey(k)
if names == nil {
names, err = k.ReadValueNames(-1)
if err != nil {
fmt.Fprintf(sb, "%serror: read value names: %v\n", indent, err)
return
}
}
for _, name := range names {
value, err := readRegistryValue(k, name)
switch {
case errors.Is(err, registry.ErrNotExist):
// the caller asks for a fixed set of values, most of which a
// given interface does not carry
continue
case err != nil:
// report rather than omit: a value that is there but cannot be
// read reads as unset otherwise
fmt.Fprintf(sb, "%s%s: error: %v\n", indent, name, err)
continue
}
fmt.Fprintf(sb, "%s%s: %s\n", indent, name, g.anonymizeValue(name, value))
}
}
// anonymizeValue redacts a registry value according to what its name says it
// holds. Domains and addresses are handled per entry rather than by the string
// pass: the pass only replaces domains something else in the bundle already
// seeded, and its address regex would eat the digit labels of a reverse zone.
func (g *BundleGenerator) anonymizeValue(name, value string) string {
if !g.anonymize || value == "" {
return value
}
switch {
case holdsDomains(name):
return joinValueEntries(splitValueEntries(value), g.anonymizeDomain)
case holdsAddresses(name):
return joinValueEntries(splitValueEntries(value), g.anonymizer.AnonymizeIPString)
default:
return g.anonymizer.AnonymizeString(value)
}
}
// holdsDomains reports whether a value name holds domains: the domain list of
// an NRPT rule (Name) or of the policy table (Namespace), a search list, the
// DNS suffix values of the TCP/IP and policy keys, which all end in "Domain"
// (Domain, DhcpDomain, NV Domain, ICSDomain), and a proxy host name.
func holdsDomains(name string) bool {
lower := strings.ToLower(name)
return lower == "name" || lower == "namespace" || lower == "searchlist" ||
strings.HasSuffix(lower, "domain") || strings.HasSuffix(lower, "proxyname")
}
// holdsAddresses reports whether a value name holds DNS server addresses
// (NameServer, DhcpNameServer, GenericDNSServers, NameServers).
func holdsAddresses(name string) bool {
lower := strings.ToLower(name)
return strings.Contains(lower, "nameserver") || strings.Contains(lower, "dnsserver")
}
// adapterNames maps adapter GUIDs, as the registry keys the interfaces, to the
// names an operator sees.
func adapterNames(adapters []*windows.IpAdapterAddresses) map[string]string {
names := make(map[string]string, len(adapters))
for _, adapter := range adapters {
guid := windows.BytePtrToString(adapter.AdapterName)
names[strings.ToLower(guid)] = windows.UTF16PtrToString(adapter.FriendlyName)
}
return names
}
// writeAdapterDNS reports the resolver configuration in effect per adapter,
// which is what the resolver uses for a name no NRPT rule matches.
func (g *BundleGenerator) writeAdapterDNS(sb *strings.Builder, adapters []*windows.IpAdapterAddresses, err error) {
writeSection(sb, "Adapter DNS configuration", "GetAdaptersAddresses")
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
for _, adapter := range adapters {
name := windows.UTF16PtrToString(adapter.FriendlyName)
suffix := g.anonymizeDomain(windows.UTF16PtrToString(adapter.DnsSuffix))
fmt.Fprintf(sb, "%s (index %d, oper status %d)\n", name, adapter.IfIndex, adapter.OperStatus)
fmt.Fprintf(sb, " DNS suffix: %s\n", suffix)
var servers []string
for server := adapter.FirstDnsServerAddress; server != nil; server = server.Next {
addr, ok := netip.AddrFromSlice(server.Address.IP())
if !ok {
continue
}
addr = addr.Unmap()
if g.anonymize {
addr = g.anonymizer.AnonymizeIP(addr)
}
servers = append(servers, addr.String())
}
fmt.Fprintf(sb, " DNS servers: %s\n", strings.Join(servers, ", "))
}
}
// anonymizeDomain anonymizes a single domain, keeping the leading dot an NRPT
// match domain carries.
func (g *BundleGenerator) anonymizeDomain(entry string) string {
if !g.anonymize {
return entry
}
domain, dot := strings.CutPrefix(entry, ".")
if domain == "" {
return entry
}
anonymized := g.anonymizer.AnonymizeDomain(domain)
if dot {
anonymized = "." + anonymized
}
return anonymized
}
// splitValueEntries splits a registry value that holds a list. The separator
// differs per value: a REG_MULTI_SZ arrives joined with ", ", a SearchList is
// comma separated and a NameServer may use commas or spaces.
func splitValueEntries(value string) []string {
return strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == ';' || r == ' ' || r == '\t'
})
}
func joinValueEntries(entries []string, anonymize func(string) string) string {
for i, entry := range entries {
entries[i] = anonymize(entry)
}
return strings.Join(entries, ", ")
}
func writeSection(sb *strings.Builder, title, source string) {
fmt.Fprintf(sb, "\n%s\n%s\n%s\n", title, strings.Repeat("-", len(title)), source)
}
func subKeyNames(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
if err != nil {
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closeKey(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
return names, nil
}
// readRegistryValue renders a value as text regardless of its type, so an
// unexpected type in a policy key still shows up instead of being dropped.
func readRegistryValue(k registry.Key, name string) (string, error) {
_, valueType, err := k.GetValue(name, nil)
if err != nil {
return "", fmt.Errorf("get value %s: %w", name, err)
}
switch valueType {
case registry.SZ, registry.EXPAND_SZ:
value, _, err := k.GetStringValue(name)
if err != nil {
return "", fmt.Errorf("get string value %s: %w", name, err)
}
return value, nil
case registry.MULTI_SZ:
values, _, err := k.GetStringsValue(name)
if err != nil {
return "", fmt.Errorf("get strings value %s: %w", name, err)
}
return strings.Join(values, ", "), nil
case registry.DWORD, registry.QWORD:
value, _, err := k.GetIntegerValue(name)
if err != nil {
return "", fmt.Errorf("get integer value %s: %w", name, err)
}
return fmt.Sprintf("%d (0x%x)", value, value), nil
case registry.BINARY:
value, _, err := k.GetBinaryValue(name)
if err != nil {
return "", fmt.Errorf("get binary value %s: %w", name, err)
}
return hex.EncodeToString(value), nil
default:
return fmt.Sprintf("<unhandled registry type %d>", valueType), nil
}
}
// adapterAddresses returns the adapter list including DNS servers. The call
// reports the size it needs, so grow the buffer and retry until it fits.
func adapterAddresses() (adapters []*windows.IpAdapterAddresses, err error) {
// GetAdaptersAddresses is resolved on first use and panics when it is
// missing, so this reports it as an error and leaves the rest of the
// report intact.
defer func() {
if r := recover(); r != nil {
adapters, err = nil, fmt.Errorf("GetAdaptersAddresses: %v", r)
}
}()
const flags = windows.GAA_FLAG_SKIP_ANYCAST | windows.GAA_FLAG_SKIP_MULTICAST
size := uint32(15000)
for range 3 {
buf := make([]byte, size)
first := (*windows.IpAdapterAddresses)(unsafe.Pointer(&buf[0]))
err := windows.GetAdaptersAddresses(windows.AF_UNSPEC, flags, 0, first, &size)
if errors.Is(err, windows.ERROR_BUFFER_OVERFLOW) {
continue
}
if err != nil {
return nil, fmt.Errorf("GetAdaptersAddresses: %w", err)
}
for adapter := first; adapter != nil; adapter = adapter.Next {
adapters = append(adapters, adapter)
}
return adapters, nil
}
return nil, fmt.Errorf("GetAdaptersAddresses: buffer kept growing")
}
func closeKey(k registry.Key) {
if err := k.Close(); err != nil {
log.Debugf("close registry key: %v", err)
}
}

View File

@@ -0,0 +1,146 @@
//go:build windows
package debug
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/anonymize"
)
func newDNSValueGenerator(level anonymize.Level) *BundleGenerator {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(level)
return &BundleGenerator{
anonymize: true,
anonymizeLevel: level,
anonymizer: anonymizer,
}
}
// TestAnonymizeValueByName covers the value kinds of the DNS registry keys. The
// names decide the treatment, because the string pass alone replaces only
// domains another part of the bundle already seeded.
func TestAnonymizeValueByName(t *testing.T) {
tests := []struct {
name string
valueName string
value string
assert func(t *testing.T, got string)
}{
{
name: "NRPT match domains keep the leading dot",
valueName: "Name",
value: ".internal.example.com, .corp.example.org",
assert: func(t *testing.T, got string) {
t.Helper()
for _, entry := range strings.Split(got, ", ") {
assert.True(t, strings.HasPrefix(entry, "."), "entry %q should keep its leading dot", entry)
assert.NotContains(t, entry, "example", "entry %q should not keep the original domain", entry)
}
},
},
{
name: "any value name ending in Domain is treated as a domain",
valueName: "ICSDomain",
value: "mshome.net",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "mshome", "should anonymize a domain suffix value")
},
},
{
name: "search list is a comma separated domain list",
valueName: "SearchList",
value: "corp.example.com,branch.example.com",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "example", "should anonymize every search domain")
assert.Len(t, strings.Split(got, ", "), 2, "should keep both search domains")
},
},
{
name: "name servers are anonymized as addresses",
valueName: "DhcpNameServer",
value: "203.0.113.10 8.8.8.8",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "203.0.113.10", "should anonymize a public resolver address")
// well-known resolvers stay readable at every level
assert.Contains(t, got, "8.8.8.8", "should keep a well-known resolver address")
},
},
{
name: "opaque values are left to the string pass",
valueName: "DataBasePath",
value: `%SystemRoot%\System32\drivers\etc`,
assert: func(t *testing.T, got string) {
t.Helper()
assert.Equal(t, `%SystemRoot%\System32\drivers\etc`, got, "should not alter a path")
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
g := newDNSValueGenerator(anonymize.LevelDefault)
tc.assert(t, g.anonymizeValue(tc.valueName, tc.value))
})
}
}
// TestParseNRPTPolicyTable parses the MOF text of the policy table out
// parameters, as the provider on a client with one NRPT rule renders it.
func TestParseNRPTPolicyTable(t *testing.T) {
const text = `[abstract]
class __PARAMETERS
{
[Out, EmbeddedInstance("DnsClientPolicyConfiguration"): ToSubClass, ID(2): DisableOverride ToInstance] DnsClientPolicyConfiguration cmdletOutput[] = {
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
DirectAccessQueryIPsecRequired = FALSE;
NameEncoding = "Utf8WithoutMapping";
Namespace = ".0.100.in-addr.arpa";
},
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
NameEncoding = "Utf8WithoutMapping";
NameServers = {"100.0.255.254", "100.0.255.253"};
Namespace = ".nb.internal";
}};
[in] boolean Effective;
[out] uint32 ReturnValue = 0;
};
`
entries := parseNRPTPolicyTable(text)
require.Len(t, entries, 2, "should parse both embedded instances")
assert.Equal(t, ".0.100.in-addr.arpa", entries[0].namespace, "should read the namespace of the first instance")
assert.Equal(t, ".nb.internal", entries[1].namespace, "should read the namespace of the second instance")
assert.Equal(t, []registryValue{
{name: "DirectAccessProxyType", value: "NoProxy"},
{name: "DirectAccessQueryIPsecRequired", value: "FALSE"},
{name: "NameEncoding", value: "Utf8WithoutMapping"},
}, entries[0].values, "should keep the remaining values in order")
assert.Contains(t, entries[1].values, registryValue{name: "NameServers", value: "100.0.255.254, 100.0.255.253"},
"should flatten a MOF array")
for _, value := range entries[1].values {
assert.NotContains(t, value.name, "ReturnValue", "should not read the class level parameters as values")
}
}
func TestParseNRPTPolicyTableEmpty(t *testing.T) {
assert.Empty(t, parseNRPTPolicyTable(""), "should parse no entries from empty text")
assert.Empty(t, parseNRPTPolicyTable("class __PARAMETERS\n{\n};\n"), "should parse no entries from a table with no instances")
}

View File

@@ -0,0 +1,317 @@
//go:build windows
package debug
import (
"errors"
"fmt"
"runtime"
"strings"
"time"
"github.com/go-ole/go-ole"
"github.com/go-ole/go-ole/oleutil"
log "github.com/sirupsen/logrus"
)
const (
// The NRPT policy table is reachable through the CIM class that backs
// Get-DnsClientNrptPolicy. Unlike the rules in the registry, the table is
// what the resolver currently has loaded, which is the only way to tell an
// applied rule from one that is merely written, in either direction.
nrptPolicyNamespace = `root\Microsoft\Windows\DNS`
nrptPolicyClass = "PS_DnsClientNrptPolicy"
nrptPolicyMethod = "Get"
// The class has no instances, so the table comes from the out parameters
// of a static method call, rendered as MOF text: the embedded instances
// arrive as a safe array of objects, which cannot be read back through the
// COM bindings, and the text form carries all of them.
nrptPolicyInstanceKeyword = "instance of DnsClientPolicyConfiguration"
nrptPolicyTimeout = 15 * time.Second
)
// COM initialization results that leave the calling thread usable: S_FALSE for
// a thread this process already initialized, RPC_E_CHANGED_MODE for one that
// belongs to another apartment.
const (
sFalse = 0x00000001
rpcEChangedMode = 0x80010106
)
// nrptQueryInFlight admits one read of the policy table at a time. A provider
// that stops answering keeps its goroutine and the OS thread that goroutine
// pinned, so a later bundle reports that instead of pinning another one.
var nrptQueryInFlight = make(chan struct{}, 1)
// nrptPolicyEntry is one namespace of the effective policy table, holding the
// values of an embedded DnsClientPolicyConfiguration instance in the order the
// provider reported them.
type nrptPolicyEntry struct {
namespace string
values []registryValue
}
// registryValue is a name and its rendered value, shared by the registry and
// policy table readers so both anonymize by value name the same way.
type registryValue struct {
name string
value string
}
// effectiveNRPTPolicies reads the effective NRPT table. The call is bounded
// because a WMI provider can block indefinitely and a debug bundle must not.
func effectiveNRPTPolicies() ([]nrptPolicyEntry, error) {
type result struct {
text string
err error
}
select {
case nrptQueryInFlight <- struct{}{}:
default:
return nil, errors.New("an earlier read of the policy table has not returned")
}
done := make(chan result, 1)
go func() {
// the slot is released here rather than by the caller, so a read that
// outlives the timeout holds it until the provider answers
defer func() { <-nrptQueryInFlight }()
text, err := nrptPolicyTableText()
done <- result{text: text, err: err}
}()
select {
case res := <-done:
if res.err != nil {
return nil, res.err
}
return parseNRPTPolicyTable(res.text), nil
case <-time.After(nrptPolicyTimeout):
return nil, errors.New("read of the policy table timed out")
}
}
// nrptPolicyTableText calls the policy table method and returns the MOF text of
// its out parameters.
func nrptPolicyTableText() (text string, err error) {
// COM is per thread, and the collection is short lived, so the thread is
// pinned for the duration rather than initialized for the process.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
defer func() {
// The COM call chain is dynamically typed, so a provider that answers
// with an unexpected shape must not take the daemon down with it.
if r := recover(); r != nil {
err = fmt.Errorf("read NRPT policy table: %v", r)
}
}()
owns, err := coInitialize()
if err != nil {
return "", err
}
if owns {
defer ole.CoUninitialize()
}
locator, err := oleutil.CreateObject("WbemScripting.SWbemLocator")
if err != nil {
return "", fmt.Errorf("create WMI locator: %w", err)
}
defer locator.Release()
dispatch, err := locator.QueryInterface(ole.IID_IDispatch)
if err != nil {
return "", fmt.Errorf("query WMI locator interface: %w", err)
}
defer dispatch.Release()
service, err := dispatchCall(dispatch, "ConnectServer", nil, nrptPolicyNamespace)
if err != nil {
return "", fmt.Errorf("connect to %s: %w", nrptPolicyNamespace, err)
}
defer service.Release()
inParams, err := spawnMethodInParams(service)
if err != nil {
return "", err
}
defer inParams.Release()
// The effective table is the merge of the local and the group policy
// store, which is what the resolver answers from.
if _, err := oleutil.PutProperty(inParams, "Effective", true); err != nil {
return "", fmt.Errorf("set Effective parameter: %w", err)
}
outParams, err := dispatchCall(service, "ExecMethod", nrptPolicyClass, nrptPolicyMethod, inParams)
if err != nil {
return "", fmt.Errorf("call %s.%s: %w", nrptPolicyClass, nrptPolicyMethod, err)
}
defer outParams.Release()
textVariant, err := oleutil.CallMethod(outParams, "GetObjectText_")
if err != nil {
return "", fmt.Errorf("render policy table: %w", err)
}
defer func() {
if err := textVariant.Clear(); err != nil {
log.Debugf("clear policy table variant: %v", err)
}
}()
return textVariant.ToString(), nil
}
// spawnMethodInParams builds the in parameters instance the method needs. The
// provider rejects the call without one, even when every parameter is optional.
func spawnMethodInParams(service *ole.IDispatch) (*ole.IDispatch, error) {
class, err := dispatchCall(service, "Get", nrptPolicyClass)
if err != nil {
return nil, fmt.Errorf("get class %s: %w", nrptPolicyClass, err)
}
defer class.Release()
methods, err := dispatchProperty(class, "Methods_")
if err != nil {
return nil, fmt.Errorf("get class methods: %w", err)
}
defer methods.Release()
method, err := dispatchCall(methods, "Item", nrptPolicyMethod)
if err != nil {
return nil, fmt.Errorf("get method %s: %w", nrptPolicyMethod, err)
}
defer method.Release()
params, err := dispatchProperty(method, "InParameters")
if err != nil {
return nil, fmt.Errorf("get method parameters: %w", err)
}
defer params.Release()
inParams, err := dispatchCall(params, "SpawnInstance_")
if err != nil {
return nil, fmt.Errorf("spawn parameter instance: %w", err)
}
return inParams, nil
}
// parseNRPTPolicyTable pulls the embedded instances out of the MOF text. Each
// instance is a namespace of the table, with one name and value per line.
func parseNRPTPolicyTable(text string) []nrptPolicyEntry {
var entries []nrptPolicyEntry
var current *nrptPolicyEntry
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(line), ";"))
switch {
case strings.HasPrefix(line, nrptPolicyInstanceKeyword):
entries = append(entries, nrptPolicyEntry{})
current = &entries[len(entries)-1]
continue
case strings.HasPrefix(line, "}"):
// closes an instance, and the array with the last one, so the
// class level parameters that follow are not read as values
current = nil
continue
case current == nil, line == "{":
continue
}
name, value, ok := strings.Cut(line, " = ")
if !ok {
continue
}
value = unquoteMOFValue(value)
if name == "Namespace" {
current.namespace = value
continue
}
current.values = append(current.values, registryValue{name: name, value: value})
}
return entries
}
// unquoteMOFValue renders a MOF scalar or array as plain text: "a" becomes a,
// and {"a", "b"} becomes a, b.
func unquoteMOFValue(value string) string {
value = strings.TrimSpace(value)
if inner, ok := strings.CutPrefix(value, "{"); ok {
value = strings.TrimSuffix(inner, "}")
entries := strings.Split(value, ",")
for i, entry := range entries {
entries[i] = strings.Trim(strings.TrimSpace(entry), `"`)
}
return strings.Join(entries, ", ")
}
return strings.Trim(value, `"`)
}
// coInitialize prepares the calling thread for COM and reports whether this
// call owns the initialization, which decides whether it may be balanced with
// CoUninitialize. S_FALSE took a reference on a thread this process had already
// initialized and so has to be released, while RPC_E_CHANGED_MODE took none:
// the thread belongs to another apartment, which is usable but is not ours to
// uninitialize.
func coInitialize() (bool, error) {
err := ole.CoInitializeEx(0, ole.COINIT_MULTITHREADED)
if err == nil {
return true, nil
}
var oleErr *ole.OleError
if errors.As(err, &oleErr) {
switch oleErr.Code() {
case sFalse:
return true, nil
case rpcEChangedMode:
return false, nil
}
}
return false, fmt.Errorf("initialize COM: %w", err)
}
// dispatchCall calls a COM method that returns an object.
func dispatchCall(dispatch *ole.IDispatch, method string, params ...any) (*ole.IDispatch, error) {
variant, err := oleutil.CallMethod(dispatch, method, params...)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("%s returned no object", method)
}
return object, nil
}
// dispatchProperty reads a COM property that holds an object.
func dispatchProperty(dispatch *ole.IDispatch, property string) (*ole.IDispatch, error) {
variant, err := oleutil.GetProperty(dispatch, property)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("property %s holds no object", property)
}
return object, nil
}

View File

@@ -31,10 +31,28 @@ var (
dnsFlushResolverCacheFn = dnsapi.NewProc("DnsFlushResolverCache")
)
// Registry locations of the host DNS configuration this package programs,
// exported so a diagnostic reader reports the same locations that are written.
const (
dnsPolicyConfigMatchPath = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig\NetBird-Match`
gpoDnsPolicyRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
gpoDnsPolicyConfigMatchPath = gpoDnsPolicyRoot + `\NetBird-Match`
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
NRPTKeyPrefix = "NetBird-Match"
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
DNSPolicyConfigRoot = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig`
// GPODNSPolicyConfigRoot holds the NRPT rules of the group policy store,
// which takes precedence over the local one when it is present.
GPODNSPolicyConfigRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
// InterfaceConfigPath and InterfaceConfigPathV6 hold the per-interface DNS
// settings, keyed by interface GUID, in separate hives per address family.
InterfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
InterfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
)
const (
dnsPolicyConfigMatchPath = DNSPolicyConfigRoot + `\` + NRPTKeyPrefix
gpoDnsPolicyConfigMatchPath = GPODNSPolicyConfigRoot + `\` + NRPTKeyPrefix
dnsPolicyConfigVersionKey = "Version"
dnsPolicyConfigVersionValue = 2
@@ -45,8 +63,6 @@ const (
nrptMaxDomainsPerRule = 50
interfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
interfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
interfaceConfigNameServerKey = "NameServer"
interfaceConfigDhcpNameSrvKey = "DhcpNameServer"
interfaceConfigSearchListKey = "SearchList"
@@ -84,7 +100,7 @@ func newHostManager(wgInterface WGIface) (*registryConfigurator, error) {
}
var useGPO bool
k, err := registry.OpenKey(registry.LOCAL_MACHINE, gpoDnsPolicyRoot, registry.QUERY_VALUE)
k, err := registry.OpenKey(registry.LOCAL_MACHINE, GPODNSPolicyConfigRoot, registry.QUERY_VALUE)
if err != nil {
log.Debugf("failed to open GPO DNS policy root: %v", err)
} else {
@@ -123,7 +139,7 @@ func (r *registryConfigurator) captureOriginalNameservers() ([]netip.Addr, error
seen := make(map[netip.Addr]struct{})
var out []netip.Addr
var merr *multierror.Error
for _, root := range []string{interfaceConfigPath, interfaceConfigPathV6} {
for _, root := range []string{InterfaceConfigPath, InterfaceConfigPathV6} {
addrs, err := r.captureFromTcpipRoot(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("%s: %w", root, err))
@@ -496,7 +512,7 @@ func (r *registryConfigurator) deleteInterfaceRegistryKeyProperty(propertyKey st
}
func (r *registryConfigurator) getInterfaceRegistryKey() (registry.Key, error) {
regKeyPath := interfaceConfigPath + "\\" + r.guid
regKeyPath := InterfaceConfigPath + "\\" + r.guid
regKey, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.SET_VALUE)
if err != nil {
return regKey, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", regKeyPath, err)

View File

@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -0,0 +1,82 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
}
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
return candidates[i].routeMetric < candidates[j].routeMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest route metric
// 3. Lowest interface metric
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -5,6 +5,7 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -5628,9 +5628,13 @@ func (x *GetPeerSSHHostKeyResponse) GetFound() bool {
type RequestJWTAuthRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// hint for OIDC login_hint parameter (typically email address)
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestJWTAuthRequest) Reset() {
@@ -5670,6 +5674,13 @@ func (x *RequestJWTAuthRequest) GetHint() string {
return ""
}
func (x *RequestJWTAuthRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestJWTAuthResponse contains authentication flow information
type RequestJWTAuthResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
@@ -5894,9 +5905,13 @@ type RequestExtendAuthSessionRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestExtendAuthSessionRequest) Reset() {
@@ -5936,6 +5951,13 @@ func (x *RequestExtendAuthSessionRequest) GetHint() string {
return ""
}
func (x *RequestExtendAuthSessionRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestExtendAuthSessionResponse carries the verification URI the UI
// should open in a browser. The daemon retains the flow state and resolves
// it via WaitExtendAuthSession.
@@ -7503,9 +7525,10 @@ const file_daemon_proto_rawDesc = "" +
"sshHostKey\x12\x16\n" +
"\x06peerIP\x18\x02 \x01(\tR\x06peerIP\x12\x1a\n" +
"\bpeerFQDN\x18\x03 \x01(\tR\bpeerFQDN\x12\x14\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"9\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"k\n" +
"\x15RequestJWTAuthRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\x9a\x02\n" +
"\x16RequestJWTAuthResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +
@@ -7525,9 +7548,10 @@ const file_daemon_proto_rawDesc = "" +
"\x14WaitJWTTokenResponse\x12\x14\n" +
"\x05token\x18\x01 \x01(\tR\x05token\x12\x1c\n" +
"\ttokenType\x18\x02 \x01(\tR\ttokenType\x12\x1c\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"C\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"u\n" +
"\x1fRequestExtendAuthSessionRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\xe0\x01\n" +
" RequestExtendAuthSessionResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +

View File

@@ -894,6 +894,10 @@ message GetPeerSSHHostKeyResponse {
message RequestJWTAuthRequest {
// hint for OIDC login_hint parameter (typically email address)
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestJWTAuthResponse contains authentication flow information
@@ -937,6 +941,10 @@ message RequestExtendAuthSessionRequest {
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestExtendAuthSessionResponse carries the verification URI the UI

View File

@@ -1723,8 +1723,8 @@ func (s *Server) RequestJWTAuth(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -1827,8 +1827,8 @@ func (s *Server) RequestExtendAuthSession(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -2000,13 +2000,6 @@ func (s *Server) ExposeService(req *proto.ExposeServiceRequest, srv proto.Daemon
return nil
}
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func (s *Server) runProbes(ctx context.Context, waitForProbeResult bool) {
if s.connectClient == nil {
return

View File

@@ -13,6 +13,7 @@ import (
"golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/util"
)
const (
@@ -92,7 +93,8 @@ func printAuthInstructions(stderr io.Writer, authResponse *proto.RequestJWTAuthR
// RequestJWTToken requests or retrieves a JWT token for SSH authentication
func RequestJWTToken(ctx context.Context, client proto.DaemonServiceClient, stdout, stderr io.Writer, useCache bool, hint string, openBrowser func(string) error) (string, error) {
req := &proto.RequestJWTAuthRequest{}
// the ssh client runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestJWTAuthRequest{HasGraphicalSession: util.HasGraphicalSession()}
if hint != "" {
req.Hint = &hint
}
@@ -193,4 +195,3 @@ func buildAddressList(hostname string, remote net.Addr) []string {
}
return addresses
}

View File

@@ -6,9 +6,11 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -58,10 +60,21 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
return ExtendStartResult{}, err
}
req := &proto.RequestExtendAuthSessionRequest{}
if p.Hint != "" {
h := p.Hint
req.Hint = &h
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -15,7 +15,8 @@
"lint": "eslint \"src/**/*.{ts,tsx}\"",
"lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix",
"check": "pnpm lint && pnpm typecheck && pnpm format:check",
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck"
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck",
"i18n:check": "node ../i18n/check-translations.mjs"
},
"dependencies": {
"@radix-ui/react-dialog": "^1.1.15",

View File

@@ -0,0 +1,104 @@
#!/usr/bin/env node
// Validates that every shipped translation bundle carries exactly the same set
// of keys as the English source of truth. English (en) defines the keys; every
// other locale declared in _index.json must match it 1:1:
//
// - no missing keys — a missing key silently falls back to English at runtime
// (see i18n bundle fallback), so the gap never surfaces to users or CI
// without this check;
// - no orphaned keys — keys left behind after an English key is renamed or
// removed are dead weight and a sign the locale is drifting.
//
// Pure Node, no dependencies, so it runs without installing the frontend
// toolchain.
//
// Local: node client/ui/i18n/check-translations.mjs (or: pnpm i18n:check)
// CI: .github/workflows/ui-translations.yml
import { readdirSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const SOURCE = "en";
const localesDir = join(dirname(fileURLToPath(import.meta.url)), "locales");
const isCI = Boolean(process.env.GITHUB_ACTIONS);
function readJSON(path) {
return JSON.parse(readFileSync(path, "utf8"));
}
function keysOf(langCode) {
return Object.keys(readJSON(join(localesDir, langCode, "common.json")));
}
// Emit a GitHub Actions annotation so failures render inline on the PR diff.
function annotate(file, message) {
if (isCI) console.log(`::error file=${file}::${message}`);
}
const index = readJSON(join(localesDir, "_index.json"));
const declared = index.languages.map((l) => l.code);
if (!declared.includes(SOURCE)) {
console.error(`FATAL: source language "${SOURCE}" is not declared in _index.json`);
process.exit(1);
}
const sourceKeys = keysOf(SOURCE);
const sourceSet = new Set(sourceKeys);
console.log(`Source of truth: ${SOURCE}/common.json — ${sourceKeys.length} keys\n`);
let failed = false;
for (const code of declared) {
if (code === SOURCE) continue;
const file = `client/ui/i18n/locales/${code}/common.json`;
let keys;
try {
keys = keysOf(code);
} catch (e) {
failed = true;
const msg = `bundle is declared in _index.json but common.json is missing or invalid (${e.message})`;
console.error(`${code}: ${msg}`);
annotate("client/ui/i18n/locales/_index.json", `${code}: ${msg}`);
continue;
}
const set = new Set(keys);
const missing = sourceKeys.filter((k) => !set.has(k));
const extra = keys.filter((k) => !sourceSet.has(k));
if (missing.length === 0 && extra.length === 0) {
console.log(`${code}: ${keys.length} keys`);
continue;
}
failed = true;
console.error(`${code}: ${keys.length} keys (expected ${sourceKeys.length})`);
if (missing.length) {
console.error(` missing ${missing.length}: ${missing.join(", ")}`);
annotate(file, `Missing ${missing.length} key(s) present in ${SOURCE}: ${missing.join(", ")}`);
}
if (extra.length) {
console.error(` extra ${extra.length}: ${extra.join(", ")}`);
annotate(file, `Has ${extra.length} key(s) not present in ${SOURCE}: ${extra.join(", ")}`);
}
}
// Locale directories present on disk but not declared in _index.json are never
// loaded by the app — surface them so dead translation files don't rot silently.
const onDisk = readdirSync(localesDir, { withFileTypes: true })
.filter((e) => e.isDirectory())
.map((e) => e.name);
const undeclared = onDisk.filter((d) => !declared.includes(d));
if (undeclared.length) {
console.warn(`\n⚠ locale directories not declared in _index.json (not shipped): ${undeclared.join(", ")}`);
}
console.log();
if (failed) {
console.error("Translation check FAILED — every locale must match the English key set.");
process.exit(1);
}
console.log("Translation check passed — all locales match the English key set.");

View File

@@ -1312,6 +1312,9 @@
"daemon.outdated.description": {
"message": "このアプリを使用するには NetBird サービスを更新してください。"
},
"daemon.outdated.download": {
"message": "最新版をダウンロード"
},
"error.jwt_clock_skew": {
"message": "サインインに失敗しました: このデバイスの時計がサーバーと同期していません。システムの時計を同期してからもう一度お試しください。"
},

View File

@@ -108,10 +108,11 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
}
req := &proto.LoginRequest{
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
IsUnixDesktopClient: runtime.GOOS == "linux",
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
// a login driven by the UI always has a graphical session available
IsUnixDesktopClient: true,
}
if profileName != "" {
req.ProfileName = ptrStr(profileName)
@@ -122,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
}
resp, err := cli.Login(ctx, req)
@@ -227,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -260,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Logout below).
// root and the per-profile state file is user-owned (see Profiles.List).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -23,9 +23,10 @@ import (
// model the client asks for. The proxy prices off the REQUEST model, not the
// upstream response model, so a made-up model id billed at operator rates lets
// these tests assert exact costs without a real vendor key.
// Sourced from the harness so the counts can't drift from the mock's config.
const (
vllmPromptTokens = 11
vllmCompletionTokens = 2
vllmPromptTokens = harness.VLLMChatInputTokens
vllmCompletionTokens = harness.VLLMChatOutputTokens
)
// pricedEnv is a connected single-provider agent-network deployment pointed at
@@ -169,23 +170,48 @@ func chatOnce(t *testing.T, ctx context.Context, env pricedEnv, model, sessionID
return body
}
// findAccessLogBySession polls the access-log page for the row carrying sessionID.
func findAccessLogBySession(t *testing.T, ctx context.Context, sessionID string) api.AgentNetworkAccessLog {
t.Helper()
var row api.AgentNetworkAccessLog
require.Eventually(t, func() bool {
logs, lerr := srv.ListAccessLogs(ctx)
if lerr != nil {
return false
}
for _, r := range logs.Data {
if r.SessionId != nil && *r.SessionId == sessionID {
row = r
return true
// accessLogIngestWindow is how long a single request's access-log row is given
// to appear before the caller gives up on it.
// accessLogIngestWindow bounds how long a row may take to appear after its
// request returned. The proxy streams each entry to management with a 10s send
// timeout of its own, so a request whose send hits one full timeout and is
// retried has not yet missed anything real — 30s left barely three send
// attempts of headroom and lost the race on a loaded runner.
const accessLogIngestWindow = 60 * time.Second
// lookupAccessLogBySession polls the access-log page for the row carrying
// sessionID and reports whether it arrived within the window. It never fails
// the test: callers that can recover — by firing a fresh request under a new
// session — need to see the miss rather than die on it.
func lookupAccessLogBySession(ctx context.Context, sessionID string, within time.Duration) (api.AgentNetworkAccessLog, bool) {
deadline := time.Now().Add(within)
for {
if logs, lerr := srv.ListAccessLogs(ctx); lerr == nil {
for _, r := range logs.Data {
if r.SessionId != nil && *r.SessionId == sessionID {
return r, true
}
}
}
return false
}, 30*time.Second, 2*time.Second, "session id %q must be recorded in an access-log row", sessionID)
if time.Now().After(deadline) {
return api.AgentNetworkAccessLog{}, false
}
select {
case <-ctx.Done():
return api.AgentNetworkAccessLog{}, false
case <-time.After(2 * time.Second):
}
}
}
// findAccessLogBySession polls the access-log page for the row carrying
// sessionID, failing the test if it never lands. Use it for a request whose row
// must exist; where a missing row is a recoverable race, use
// lookupAccessLogBySession and retry.
func findAccessLogBySession(t *testing.T, ctx context.Context, sessionID string) api.AgentNetworkAccessLog {
t.Helper()
row, ok := lookupAccessLogBySession(ctx, sessionID, accessLogIngestWindow)
require.True(t, ok, "session id %q must be recorded in an access-log row", sessionID)
return row
}
@@ -319,6 +345,11 @@ func TestPriceChangeUpdatesRecordedCost(t *testing.T) {
outRateA = 0.020
inRateB = 0.050 // 5x / 4x the original, so a repriced row is unmistakable
outRateB = 0.080
// Per-attempt ingest wait, shorter than the default so a request that
// produces no row costs one retry rather than most of the budget, and an
// overall deadline long enough to hold several attempts.
repriceIngestWindow = 20 * time.Second
repriceDeadline = 180 * time.Second
)
env := provisionPricedProvider(t, ctx, "reprice", []api.AgentNetworkProviderModel{
@@ -353,10 +384,15 @@ func TestPriceChangeUpdatesRecordedCost(t *testing.T) {
// reading its cost, so an un-ingested row is never mistaken for "still rate A".
// The expected new input cost is unmistakably higher than rate A, so a
// lingering old-rate row can't satisfy the check.
//
// Every way an iteration can come up short — the request failing, its row not
// landing, or the row still carrying rate A — is a symptom of the same
// in-flight rebuild, so each one retries under a fresh session rather than
// ending the test. Only the outer deadline is fatal.
wantInputB := float64(vllmPromptTokens) / 1000 * inRateB
var repriced api.AgentNetworkAccessLog
var lastSession string
deadline := time.Now().Add(90 * time.Second)
deadline := time.Now().Add(repriceDeadline)
for time.Now().Before(deadline) {
lastSession = fmt.Sprintf("e2e-session-reprice-b-%d", time.Now().UnixNano())
code, _, cerr := env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireChat, customModel, "Reply with exactly: pong", lastSession)
@@ -364,7 +400,15 @@ func TestPriceChangeUpdatesRecordedCost(t *testing.T) {
time.Sleep(5 * time.Second)
continue
}
row := findAccessLogBySession(t, ctx, lastSession)
row, ok := lookupAccessLogBySession(ctx, lastSession, repriceIngestWindow)
if !ok {
// No row for this request. The provider update rebuilds the proxy's
// middleware chain, and a request served mid-rebuild can complete
// without a resolved provider — 200 to the caller, nothing to
// attribute, so no row is ever written for it. Fire another one.
t.Logf("no access-log row for session %q within %s; retrying under a fresh session", lastSession, repriceIngestWindow)
continue
}
if inDelta(row.InputCostUsd, wantInputB, 1e-6) {
repriced = row
break
@@ -630,3 +674,47 @@ func inDelta(a, b, tol float64) bool {
}
return d <= tol
}
// TestCustomDatedModelKeepsItsOwnPrice covers the review fix that anchored the
// release-date fallback to Claude ids. Pricing looks every model up through
// that helper, so while it matched a bare trailing date any operator id ending
// in eight digits inherited the rate of its undated sibling — a silent
// mis-bill on models NetBird knows nothing about.
func TestCustomDatedModelKeepsItsOwnPrice(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
const (
baseModel = "internal-llm"
datedModel = "internal-llm-20250101"
baseIn = 0.010
baseOut = 0.020
// An order of magnitude apart, so a row billed at the wrong entry is
// unmistakable rather than a rounding argument.
datedIn = 0.100
datedOut = 0.200
)
env := provisionPricedProvider(t, ctx, "customdated", []api.AgentNetworkProviderModel{
{Id: baseModel, InputPer1k: baseIn, OutputPer1k: baseOut},
{Id: datedModel, InputPer1k: datedIn, OutputPer1k: datedOut},
})
t.Run("the undated id bills at its own rate", func(t *testing.T) {
session := fmt.Sprintf("e2e-session-customdated-base-%d", time.Now().UnixNano())
chatOnce(t, ctx, env, baseModel, session)
assertOpenAICostAtRates(t, findAccessLogBySession(t, ctx, session), baseIn, baseOut)
})
t.Run("the dated id keeps its own rate", func(t *testing.T) {
session := fmt.Sprintf("e2e-session-customdated-dated-%d", time.Now().UnixNano())
chatOnce(t, ctx, env, datedModel, session)
row := findAccessLogBySession(t, ctx, session)
assertOpenAICostAtRates(t, row, datedIn, datedOut)
// Spelled out because it is the regression: inheriting the sibling's
// rate would bill this request at a tenth of its price.
assert.Greater(t, row.InputCostUsd, float64(vllmPromptTokens)/1000*baseIn*2,
"a custom dated id must not inherit the undated entry's rate")
})
}

View File

@@ -0,0 +1,400 @@
//go:build e2e
package agentnetwork
import (
"context"
"encoding/json"
"os"
"sort"
"strconv"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
sharedllm "github.com/netbirdio/netbird/shared/llm"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// TestLiveModelDiscovery drives model discovery against the REAL vendor
// endpoints — OpenAI, Anthropic, Bedrock and Vertex — rather than the mock.
//
// The mock upstream proves the filter's mechanics: it advertises ids we chose,
// so a listing narrowing to the ones we authorised is arithmetic we already
// controlled both sides of. What it cannot prove is that the filter survives
// contact with a real catalogue — ids we never enumerated, dated builds whose
// suffix the vendor picks, surfaces that answer a listing request with
// something other than a listing. That is what this covers, and it is the part
// a QA engineer would otherwise have to walk through by hand.
//
// One proxy serves every case. Each provider gets its own group, policy and
// client, because a model-less request matches exactly ONE route
// (matchModelless): with two providers authorised for the same caller, the
// listing would go to whichever won the tiebreak and the other would go
// untested. Group-scoping the caller makes each provider the only candidate
// for its own client.
func TestLiveModelDiscovery(t *testing.T) {
cases := liveDiscoveryCases()
if len(cases) == 0 {
t.Skip("no provider keys set; source ~/.llm-keys to run live model discovery")
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
defer cancel()
t.Logf("[discovery] live matrix: %s", strings.Join(caseNames(cases), ", "))
// Provision every provider, group and policy before the proxy starts: the
// proxy takes a configuration snapshot at connect time and does not
// reconcile provider changes made afterwards.
keys := make(map[string]string, len(cases))
for i := range cases {
keys[cases[i].name] = provisionLiveDiscovery(t, ctx, &cases[i])
}
endpoint, firstIP, firstClient, px := connectClient(t, ctx, "disc-live", keys[cases[0].name])
clients := map[string]*harness.Client{cases[0].name: firstClient}
ips := map[string]string{cases[0].name: firstIP}
for _, tc := range cases[1:] {
cl := joinClient(t, ctx, px, endpoint, keys[tc.name])
ip, err := cl.ResolveProxyIP(ctx, endpoint)
require.NoError(t, err, "resolve endpoint from the %s client", tc.name)
clients[tc.name] = cl
ips[tc.name] = ip
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
runLiveDiscoveryCase(t, ctx, tc, clients[tc.name], endpoint, ips[tc.name])
})
}
}
// discoveryOutcome is what a discovery request must produce end to end. The
// three are genuinely different contracts, not degrees of success: only the
// first puts a bounded listing in front of the caller.
type discoveryOutcome int
const (
// outcomeFiltered: the proxy routes the request and bounds the response to
// what the caller may use.
outcomeFiltered discoveryOutcome = iota
// outcomeDenied: no provider of this shape can serve the surface, so the
// proxy refuses rather than rewriting the request onto an upstream that
// would 404 it. The caller gets a NetBird error, not a vendor one.
outcomeDenied
// outcomeUpstreamNoListing: the proxy routes the request to the configured
// upstream, and the vendor does not implement the endpoint there. Proxy
// side correct, product side a dead end — see the Bedrock case.
outcomeUpstreamNoListing
)
// liveDiscoveryCase is one provider's discovery surface and what the proxy
// must make of it.
type liveDiscoveryCase struct {
name string
catalogID string
upstream string
apiKey string
// path is the discovery endpoint the client calls. Not every surface uses
// /v1/models: Bedrock lists inference profiles instead.
path string
// headers the vendor requires on a bare GET (Anthropic versions its API
// through a header, and rejects a request without one).
headers []string
// models the provider record enumerates. Empty models a gateway record,
// which enumerates nothing and claims everything.
models []string
// allowlist, when non-empty, is a guardrail narrowing the policy below the
// provider's own enumeration — the second of the two bounds discovery
// applies, and the only one a provider record alone cannot demonstrate.
allowlist []string
// outcome is what this surface must produce end to end.
outcome discoveryOutcome
// permitted is every id allowed to survive filtering, in the form the
// provider record registers it. A surviving id counts as permitted when it
// matches one of these outright or after Anthropic date-normalisation.
permitted []string
// wantHidden are ids the upstream is known to advertise and the bound must
// remove. Only set where we enumerate the model ourselves, so the
// expectation cannot rot when a vendor changes its catalogue.
wantHidden []string
}
// liveDiscoveryCases builds the matrix from whichever provider credentials are
// present, mirroring availableProviders' env-var gating so a partial key set
// still yields partial coverage.
func liveDiscoveryCases() []liveDiscoveryCase {
var cases []liveDiscoveryCase
// OpenAI enumerates TWO real models and the policy permits one. That is
// the only case here where both bounds are observable at once: the
// upstream advertises dozens of ids, the provider record cuts them to two,
// and the guardrail cuts those to one.
if k := os.Getenv("OPENAI_TOKEN"); k != "" {
cases = append(cases, liveDiscoveryCase{
name: "openai", catalogID: "openai_api", upstream: "https://api.openai.com", apiKey: k,
path: "/v1/models",
models: []string{"gpt-4o-mini", "gpt-4o"},
allowlist: []string{"gpt-4o-mini"},
outcome: outcomeFiltered,
permitted: []string{"gpt-4o-mini"},
wantHidden: []string{"gpt-4o"},
})
}
// Anthropic is the surface Claude Code actually calls. Its listing returns
// DATED build ids (claude-haiku-4-5-20251001) while the provider record
// registers the undated id, so this is the case that proves the filter's
// date-normalisation against ids the vendor chose rather than ids we wrote.
if k := os.Getenv("ANTHROPIC_TOKEN"); k != "" {
cases = append(cases, liveDiscoveryCase{
name: "anthropic", catalogID: "anthropic_api", upstream: "https://api.anthropic.com", apiKey: k,
path: "/v1/models",
headers: []string{"anthropic-version: 2023-06-01"},
models: []string{"claude-haiku-4-5"},
outcome: outcomeFiltered,
permitted: []string{"claude-haiku-4-5"},
})
}
// Bedrock lists inference profiles, not models: matchModelless routes
// /inference-profiles to a Bedrock route and refuses /v1/models for one.
//
// The request reaches AWS and AWS refuses it — bedrock-runtime answers
// <UnknownOperationException/>, because ListInferenceProfiles is a CONTROL
// PLANE operation served by bedrock.<region>.amazonaws.com, not the runtime
// host. A provider record carries one upstream and it has to be the runtime
// host for InvokeModel to work, so no Bedrock record can serve a listing as
// the model stands today.
//
// The mock upstream hides this entirely: it answers /inference-profiles on
// the same listener as everything else, so the routing test passes there
// while the real endpoint 404s. That is the whole reason this file exists,
// so the case is kept, asserting what actually happens.
if k := os.Getenv("AWS_BEARER_TOKEN_BEDROCK"); k != "" {
region := os.Getenv("AWS_REGION")
if region == "" {
region = "eu-central-1"
}
model := os.Getenv("AWS_BEDROCK_MODEL")
if model == "" {
model = "global.anthropic.claude-sonnet-4-6"
}
cases = append(cases, liveDiscoveryCase{
name: "bedrock", catalogID: "bedrock_api",
upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k,
path: "/inference-profiles",
models: []string{sharedllm.NormalizeAnthropicModel(strings.TrimPrefix(model, "global."))},
outcome: outcomeUpstreamNoListing,
})
}
// Vertex carries the model in the rawPredict path and serves no listing
// endpoint at all, so the proxy must refuse discovery rather than rewrite
// it onto an upstream that would 404.
if sa := os.Getenv("GOOGLE_VERTEX_SA_BASE64"); sa != "" {
if project := os.Getenv("GOOGLE_VERTEX_PROJECT"); project != "" {
region := os.Getenv("GOOGLE_VERTEX_REGION")
if region == "" {
region = "global"
}
host := "aiplatform.googleapis.com"
if region != "global" {
host = region + "-aiplatform.googleapis.com"
}
cases = append(cases, liveDiscoveryCase{
name: "vertex", catalogID: "vertex_ai_api", upstream: "https://" + host,
apiKey: "keyfile::" + sa,
path: "/v1/models",
outcome: outcomeDenied,
})
}
}
return cases
}
// provisionLiveDiscovery creates the group, provider, optional guardrail and
// policy for one case, and returns the setup key a client joins that group
// with. Scoping each provider to its own group is what keeps it the only
// candidate for its own client's model-less request.
func provisionLiveDiscovery(t *testing.T, ctx context.Context, tc *liveDiscoveryCase) string {
t.Helper()
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-disc-live-" + tc.name})
require.NoError(t, err, "create group for %s", tc.name)
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-disc-live-" + tc.name,
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key for %s", tc.name)
require.NotEmpty(t, sk.Key, "setup key plaintext for %s", tc.name)
req := api.AgentNetworkProviderRequest{
Name: "e2e-disc-live-" + tc.name,
ProviderId: tc.catalogID,
UpstreamUrl: tc.upstream,
ApiKey: &tc.apiKey,
Enabled: ptr(true),
}
if len(tc.models) > 0 {
models := make([]api.AgentNetworkProviderModel, 0, len(tc.models))
for _, id := range tc.models {
models = append(models, api.AgentNetworkProviderModel{Id: id, InputPer1k: 0.001, OutputPer1k: 0.002})
}
req.Models = &models
}
prov, err := srv.CreateProvider(ctx, req)
require.NoError(t, err, "create provider %s", tc.name)
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
polReq := api.AgentNetworkPolicyRequest{
Name: "e2e-disc-live-" + tc.name,
Enabled: ptr(true),
SourceGroups: []string{grp.Id},
DestinationProviderIds: []string{prov.Id},
}
if len(tc.allowlist) > 0 {
var gr api.AgentNetworkGuardrailRequest
gr.Name = "e2e-disc-live-" + tc.name
gr.Checks.ModelAllowlist.Enabled = true
gr.Checks.ModelAllowlist.Models = tc.allowlist
g, gerr := srv.CreateGuardrail(ctx, gr)
require.NoError(t, gerr, "create guardrail for %s", tc.name)
t.Cleanup(func() { _ = srv.DeleteGuardrail(context.Background(), g.Id) })
polReq.GuardrailIds = &[]string{g.Id}
}
pol, err := srv.CreatePolicy(ctx, polReq)
require.NoError(t, err, "create policy for %s", tc.name)
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
return sk.Key
}
// runLiveDiscoveryCase issues the discovery request and reports everything the
// vendor said before asserting on any of it. The log is the point on the first
// run: a live catalogue is the one input we do not control, so a failure has to
// arrive with the response that caused it rather than just a count.
func runLiveDiscoveryCase(t *testing.T, ctx context.Context, tc liveDiscoveryCase, cl *harness.Client, endpoint, proxyIP string) {
t.Helper()
// A single request is enough for the two non-listing outcomes, and retrying
// them would burn the retry window waiting for a status that is never
// coming.
if tc.outcome != outcomeFiltered {
code, body, err := cl.Get(ctx, endpoint, proxyIP, tc.path, tc.headers)
require.NoError(t, err, "request must reach the proxy")
t.Logf("[discovery] %s GET %s -> %d; body: %s", tc.name, tc.path, code, truncate(body, 2000))
assert.NotEqual(t, 200, code,
"%s serves no bounded listing, so a 200 here would mean the caller was handed a picker nothing narrows; body: %s",
tc.name, truncate(body, 2000))
// Which side refused is the whole distinction between these two
// outcomes, and a NetBird error is the thing that tells them apart: the
// middleware chain stamps its own name on anything it generates.
if tc.outcome == outcomeDenied {
assert.True(t, isProxyError(body),
"%s serves no listing endpoint at all, so the proxy must refuse the request itself rather than forward it to an upstream that would answer for us; body: %s",
tc.name, truncate(body, 2000))
return
}
assert.False(t, isProxyError(body),
"%s discovery must be routed to the configured upstream and refused by the vendor, not blocked by the proxy; body: %s",
tc.name, truncate(body, 2000))
return
}
code, body := callUntil(t, func() (int, string, error) {
return cl.Get(ctx, endpoint, proxyIP, tc.path, tc.headers)
}, 200)
t.Logf("[discovery] %s GET %s -> %d; body: %s", tc.name, tc.path, code, truncate(body, 4000))
require.Equal(t, 200, code, "%s discovery must be served; body: %s", tc.name, truncate(body, 2000))
ids, ok := listingIDs(body)
require.Truef(t, ok,
"%s answered discovery with something other than a {\"data\":[{\"id\":…}]} listing, which the filter forwards untouched — the caller would get an unbounded picker; body: %s",
tc.name, truncate(body, 2000))
sort.Strings(ids)
t.Logf("[discovery] %s: %d ids after filtering: %s", tc.name, len(ids), strings.Join(ids, ", "))
require.NotEmpty(t, ids, "%s filtered the listing down to nothing; the caller would see an empty picker", tc.name)
permitted := make(map[string]struct{}, len(tc.permitted)*2)
for _, id := range tc.permitted {
permitted[id] = struct{}{}
permitted[sharedllm.NormalizeAnthropicModel(id)] = struct{}{}
}
for _, id := range ids {
_, direct := permitted[id]
_, normalised := permitted[sharedllm.NormalizeAnthropicModel(id)]
assert.Truef(t, direct || normalised,
"%s offered %q, which no policy on this route permits — every entry the picker shows must be a request the guardrail would allow", tc.name, id)
}
for _, hidden := range tc.wantHidden {
assert.NotContainsf(t, ids, hidden,
"%s offered %q, which the provider enumerates but the policy does not permit", tc.name, hidden)
}
}
// isProxyError reports whether a response body was generated by the middleware
// chain rather than forwarded from a vendor. Every chain-generated error names
// the middleware that raised it, which no upstream's error body does — so this
// separates "the proxy refused" from "the proxy routed it and the vendor
// refused", the two failures that otherwise look alike from the client side.
func isProxyError(body string) bool {
return strings.Contains(body, `"middleware":`)
}
// listingIDs pulls the model ids out of a listing response. ok is false when
// the body is not the {"data":[{"id":…}]} shape the filter recognises.
func listingIDs(body string) ([]string, bool) {
var doc struct {
Data []struct {
ID string `json:"id"`
} `json:"data"`
}
if err := json.Unmarshal([]byte(body), &doc); err != nil {
return nil, false
}
if doc.Data == nil {
return nil, false
}
ids := make([]string, 0, len(doc.Data))
for _, entry := range doc.Data {
ids = append(ids, entry.ID)
}
return ids, true
}
func caseNames(cases []liveDiscoveryCase) []string {
names := make([]string, 0, len(cases))
for _, c := range cases {
names = append(names, c.name)
}
return names
}
// truncate bounds a logged response body. A live catalogue can run to tens of
// kilobytes, and the useful part is the front.
func truncate(s string, limit int) string {
if len(s) <= limit {
return s
}
return s[:limit] + "… (" + strconv.Itoa(len(s)-limit) + " more bytes)"
}

View File

@@ -0,0 +1,168 @@
//go:build e2e
package agentnetwork
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// TestDiscoveryBoundToCallersPolicies covers a model listing on a provider two
// teams reach under different allowlists.
//
// Bounding the listing by the provider's enumerated models alone is not enough
// once more than one policy is in play: the caller would be offered every model
// any team may use, and each one outside their own policy is a request the
// guardrail refuses a moment later — the empty-or-wrong picker this endpoint
// exists to avoid, just moved one level up.
//
// The client joins the main group only. Both models are enumerated by the same
// provider and both are advertised by the upstream, so a listing that leaked
// the other team's model would visibly contain it.
func TestDiscoveryBoundToCallersPolicies(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
vllm, err := harness.StartVLLM(ctx, srv)
require.NoError(t, err, "start mock upstream")
t.Cleanup(func() { _ = vllm.Terminate(context.Background()) })
grpMain, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-disc-mp-main"})
require.NoError(t, err, "create main group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grpMain.Id) })
grpOther, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-disc-mp-other"})
require.NoError(t, err, "create other group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grpOther.Id) })
ephemeral := false
mkKey := func(name, groupID string) string {
sk, kerr := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: name,
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{groupID},
Ephemeral: &ephemeral,
})
require.NoError(t, kerr, "mint setup key %s", name)
require.NotEmpty(t, sk.Key, "setup key plaintext")
return sk.Key
}
// One client per group. The second is what makes the first assertion mean
// something: without a client that DOES see the other team's model, its
// absence from the main client's listing could equally be a policy that
// never propagated.
keyMain := mkKey("e2e-disc-mp-main-client", grpMain.Id)
keyOther := mkKey("e2e-disc-mp-other-client", grpOther.Id)
// One provider enumerating both models the upstream advertises, so the
// listing is narrowed by policy rather than by what the provider serves.
staticKey := "static-e2e-token"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "e2e-disc-mp",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
Models: &[]api.AgentNetworkProviderModel{
{Id: harness.VLLMModel, InputPer1k: 0.001, OutputPer1k: 0.001},
{Id: harness.VLLMUnlistedModel, InputPer1k: 0.001, OutputPer1k: 0.001},
},
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
mkGuardrail := func(name, model string) api.AgentNetworkGuardrail {
var gr api.AgentNetworkGuardrailRequest
gr.Name = name
gr.Checks.ModelAllowlist.Enabled = true
gr.Checks.ModelAllowlist.Models = []string{model}
g, gerr := srv.CreateGuardrail(ctx, gr)
require.NoError(t, gerr, "create guardrail %s", name)
t.Cleanup(func() { _ = srv.DeleteGuardrail(context.Background(), g.Id) })
return g
}
gMain := mkGuardrail("e2e-disc-mp-main", harness.VLLMModel)
gOther := mkGuardrail("e2e-disc-mp-other", harness.VLLMUnlistedModel)
enabled := true
polMain, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-disc-mp-main",
Enabled: &enabled,
SourceGroups: []string{grpMain.Id},
DestinationProviderIds: []string{prov.Id},
GuardrailIds: &[]string{gMain.Id},
})
require.NoError(t, err, "create main policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), polMain.Id) })
// The other team's policy, on the same provider, permitting the model the
// client must never be offered.
polOther, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-disc-mp-other",
Enabled: &enabled,
SourceGroups: []string{grpOther.Id},
DestinationProviderIds: []string{prov.Id},
GuardrailIds: &[]string{gOther.Id},
})
require.NoError(t, err, "create other policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), polOther.Id) })
endpoint, proxyIP, clMain, px := connectClient(t, ctx, "disc-mp", keyMain)
clOther := joinClient(t, ctx, px, endpoint, keyOther)
listing := func(t *testing.T, cl *harness.Client, ip string) string {
t.Helper()
code, body := callUntil(t, func() (int, string, error) {
return cl.Get(ctx, endpoint, ip, "/v1/models?limit=1000", nil)
}, 200)
require.Equal(t, 200, code, "discovery must be served; body: %s", body)
return body
}
otherIP, err := clOther.ResolveProxyIP(ctx, endpoint)
require.NoError(t, err, "resolve endpoint from the other client")
// The other team's client first: seeing its own model proves polOther is
// live, so the main client's listing is narrowed by policy scoping rather
// than by the other policy having failed to apply at all.
otherBody := listing(t, clOther, otherIP)
assert.Contains(t, otherBody, harness.VLLMUnlistedModel,
"the other group's policy must be in force, or this test proves nothing")
assert.NotContains(t, otherBody, harness.VLLMModel,
"and it must not be offered the main group's model either — isolation runs both ways")
mainBody := listing(t, clMain, proxyIP)
assert.Contains(t, mainBody, harness.VLLMModel,
"the model the caller's own policy permits must reach the picker")
assert.NotContains(t, mainBody, harness.VLLMUnlistedModel,
"a model only another group's policy permits must not be offered to this caller")
}
// joinClient starts a second tunnel client against an already-running proxy, so
// a test can drive the same endpoint as two different group memberships without
// paying for a second proxy.
func joinClient(t *testing.T, ctx context.Context, px *harness.Proxy, endpoint, setupKey string) *harness.Client {
t.Helper()
cl, err := harness.StartClient(ctx, srv, setupKey)
require.NoError(t, err, "start second client")
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "second client must connect to management")
if _, err := cl.ResolveProxyIP(ctx, endpoint); err != nil {
t.Fatalf("second client could not resolve the endpoint: %v", err)
}
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("second client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
return cl
}

View File

@@ -0,0 +1,455 @@
//go:build e2e
package agentnetwork
import (
"context"
"fmt"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// Models each catalog surface is registered with in the matrix below. They
// differ per provider so the router's choice is unambiguous: a request that
// lands on the wrong provider record fails the surface assertion instead of
// passing by coincidence.
const (
matrixAnthropicModel = "claude-sonnet-5"
matrixBedrockModel = "anthropic.claude-sonnet-5"
// matrixBedrockPathModel is what a Bedrock SDK client puts in the URL: a
// cross-region inference profile with a release date and version suffix.
// The proxy must normalise it back to matrixBedrockModel to route and price.
matrixBedrockPathModel = "us.anthropic.claude-sonnet-5-20250101-v1:0"
// matrixVertexModel differs from the Anthropic record's model on purpose:
// a shared id would leave two routes claiming it and make which one serves
// /v1/messages depend on declaration order.
matrixVertexModel = "claude-haiku-4-5"
matrixVertexProject = "e2e-project"
matrixVertexRegion = "us-east5"
)
// gatewayEnv is a connected client plus a set of provider records, all pointed
// at one mock upstream, so several wire shapes can be driven over a single
// tunnel.
type gatewayEnv struct {
endpoint string
proxyIP string
client *harness.Client
proxy *harness.Proxy
vllm *harness.VLLM
// providerIDs maps the catalog id to the created provider record id.
providerIDs map[string]string
}
// provisionGatewayMatrix brings up one mock upstream and one provider record
// per catalog surface, all authorised for the same group by a single policy.
// Sharing one proxy and client keeps the wire-shape cases to one tunnel setup;
// each case still creates its own session id so its access-log row is findable.
func provisionGatewayMatrix(t *testing.T, ctx context.Context) gatewayEnv {
t.Helper()
vllm, err := harness.StartVLLM(ctx, srv)
require.NoError(t, err, "start mock upstream")
t.Cleanup(func() { _ = vllm.Terminate(context.Background()) })
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-gw-matrix"})
require.NoError(t, err, "create group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-gw-matrix-client",
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key")
require.NotEmpty(t, sk.Key, "setup key plaintext")
// The mock ignores auth, so a dummy credential satisfies each catalog
// entry's auth template. Vertex is the exception: its api_key is a GCP
// service-account keyfile the proxy mints an OAuth token from, and a dummy
// one cannot mint. That is deliberate — the Vertex case below asserts on
// routing, which happens before the token mint.
dummyKey := "sk-gw-e2e"
dummyKeyfile := "keyfile::" + "e2e-not-a-real-service-account-key"
specs := []struct {
name string
catalogID string
apiKey string
models []api.AgentNetworkProviderModel
}{
{
name: "openai", catalogID: "openai_api", apiKey: dummyKey,
models: []api.AgentNetworkProviderModel{{Id: harness.VLLMModel, InputPer1k: 0.001, OutputPer1k: 0.002}},
},
{
name: "anthropic", catalogID: "anthropic_api", apiKey: dummyKey,
models: []api.AgentNetworkProviderModel{{Id: matrixAnthropicModel, InputPer1k: 0.003, OutputPer1k: 0.015}},
},
{
name: "bedrock", catalogID: "bedrock_api", apiKey: dummyKey,
models: []api.AgentNetworkProviderModel{{Id: matrixBedrockModel, InputPer1k: 0.003, OutputPer1k: 0.015}},
},
{
name: "vertex", catalogID: "vertex_ai_api", apiKey: dummyKeyfile,
models: []api.AgentNetworkProviderModel{{Id: matrixVertexModel, InputPer1k: 0.001, OutputPer1k: 0.005}},
},
}
providerIDs := make(map[string]string, len(specs))
ids := make([]string, 0, len(specs))
for _, spec := range specs {
key := spec.apiKey
models := spec.models
prov, perr := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "e2e-gw-" + spec.name,
ProviderId: spec.catalogID,
UpstreamUrl: vllm.URL,
ApiKey: &key,
Enabled: ptr(true),
Models: &models,
})
require.NoError(t, perr, "create %s provider", spec.name)
id := prov.Id
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), id) })
providerIDs[spec.catalogID] = id
ids = append(ids, id)
}
// Uncapped token limit: never blocks the handful of tokens driven here, but
// switches on usage metering so consumption and cost land in the row.
enabled := true
pol, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-gw-matrix",
Enabled: &enabled,
SourceGroups: []string{grp.Id},
DestinationProviderIds: ids,
Limits: &api.AgentNetworkPolicyLimits{
TokenLimit: api.AgentNetworkPolicyTokenLimit{
Enabled: true,
GroupCap: 10_000_000,
UserCap: 10_000_000,
WindowSeconds: 60,
},
},
})
require.NoError(t, err, "create policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
endpoint, proxyIP, cl, px := connectClient(t, ctx, "gw-matrix", sk.Key)
return gatewayEnv{
endpoint: endpoint,
proxyIP: proxyIP,
client: cl,
proxy: px,
vllm: vllm,
providerIDs: providerIDs,
}
}
// connectClient starts a proxy and a tunnel client for the shared account and
// waits until the client can reach the proxy peer, returning the endpoint and
// the proxy's tunnel IP to pin requests to.
func connectClient(t *testing.T, ctx context.Context, name, setupKey string) (string, string, *harness.Client, *harness.Proxy) {
t.Helper()
settings, err := srv.GetSettings(ctx)
require.NoError(t, err, "read settings")
require.NotEmpty(t, settings.Endpoint, "endpoint must be assigned")
proxyToken, err := srv.CreateProxyTokenCLI(ctx, "e2e-"+name+"-proxy")
require.NoError(t, err, "mint proxy token")
px, err := harness.StartProxy(ctx, srv, proxyToken)
require.NoError(t, err, "start proxy")
t.Cleanup(func() { _ = px.Terminate(context.Background()) })
cl, err := harness.StartClient(ctx, srv, setupKey)
require.NoError(t, err, "start client")
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
// The probe resolves the endpoint and its first packet wakes the lazy proxy
// peer, so WaitProxyPeer then observes it connected.
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve endpoint to proxy IP")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
return settings.Endpoint, proxyIP, cl, px
}
// callUntil retries an HTTP call through the tunnel until it returns one of the
// wanted statuses or the deadline passes, absorbing the DNS and tunnel jitter
// the first call through a fresh tunnel can hit. The last status and body are
// returned either way so the caller can assert with real detail.
func callUntil(t *testing.T, call func() (int, string, error), want ...int) (int, string) {
t.Helper()
wanted := make(map[int]struct{}, len(want))
for _, w := range want {
wanted[w] = struct{}{}
}
var code int
var body string
deadline := time.Now().Add(90 * time.Second)
for time.Now().Before(deadline) {
c, b, err := call()
if err == nil {
code, body = c, b
if _, ok := wanted[code]; ok {
return code, body
}
}
time.Sleep(5 * time.Second)
}
return code, body
}
// TestGatewayProtocolProviderMatrix drives one request per wire shape over a
// single tunnel, with a provider record per catalog surface behind it. It is
// the regression net for the routing and parser-selection changes: each case
// asserts the surface the request was metered under and the token counts that
// surface's own usage block carries, so a request parsed by the wrong provider's
// parser meters zero and fails rather than passing on a coincidence.
func TestGatewayProtocolProviderMatrix(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
env := provisionGatewayMatrix(t, ctx)
diag := func() string {
return fmt.Sprintf("\n=== upstream logs ===\n%s\n=== proxy logs ===\n%s",
env.vllm.Logs(context.Background()), env.proxy.Logs(context.Background()))
}
t.Run("openai chat completions", func(t *testing.T) {
session := "e2e-gw-openai"
code, body := callUntil(t, func() (int, string, error) {
return env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireChat, harness.VLLMModel, "ping", session)
}, 200)
require.Equal(t, 200, code, "openai chat must succeed; body: %s%s", body, diag())
require.Contains(t, body, "chat.completion", "body must be an OpenAI completion; got: %s", body)
row := findAccessLogBySession(t, ctx, session)
require.NotNil(t, row.Provider)
assert.Equal(t, "openai", *row.Provider, "the OpenAI chat path must meter under the openai surface")
assert.Equal(t, int64(harness.VLLMChatInputTokens), row.InputTokens, "OpenAI usage block must be read")
assert.Equal(t, int64(harness.VLLMChatOutputTokens), row.OutputTokens)
})
t.Run("anthropic messages", func(t *testing.T) {
session := "e2e-gw-anthropic"
code, body := callUntil(t, func() (int, string, error) {
return env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireMessages, matrixAnthropicModel, "ping", session)
}, 200)
require.Equal(t, 200, code, "anthropic messages must succeed; body: %s%s", body, diag())
row := findAccessLogBySession(t, ctx, session)
require.NotNil(t, row.Provider)
assert.Equal(t, "anthropic", *row.Provider, "the /v1/messages path must meter under the anthropic surface")
// These counts only appear if the Anthropic parser read the response:
// its usage fields are named differently from the OpenAI block.
assert.Equal(t, int64(harness.VLLMMessagesInputTokens), row.InputTokens,
"Anthropic input_tokens must be read; zero here means the wrong parser ran")
assert.Equal(t, int64(harness.VLLMMessagesOutputTokens), row.OutputTokens)
assert.Positive(t, row.CachedInputTokens, "the Anthropic cache-read bucket must be recorded")
assert.Positive(t, row.CostUsd, "a metered request must carry a cost")
require.NotNil(t, row.ResolvedProviderId)
assert.Equal(t, env.providerIDs["anthropic_api"], *row.ResolvedProviderId,
"a vendor-tagged request must not cross to another provider's record")
})
t.Run("bedrock invoke normalises the path model", func(t *testing.T) {
session := "e2e-gw-bedrock"
code, body := callUntil(t, func() (int, string, error) {
return env.client.Bedrock(ctx, env.endpoint, env.proxyIP, matrixBedrockPathModel, "ping", session)
}, 200)
require.Equal(t, 200, code, "bedrock invoke must succeed; body: %s%s", body, diag())
row := findAccessLogBySession(t, ctx, session)
require.NotNil(t, row.Provider)
assert.Equal(t, "bedrock", *row.Provider, "a native Bedrock path must meter under the bedrock surface")
require.NotNil(t, row.Model)
assert.Equal(t, matrixBedrockModel, *row.Model,
"the inference-profile prefix, release date and version suffix must be normalised away")
assert.Equal(t, int64(harness.VLLMMessagesInputTokens), row.InputTokens)
})
t.Run("anthropic token counting", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return env.client.PostJSON(ctx, env.endpoint, env.proxyIP, "/v1/messages/count_tokens",
fmt.Sprintf(`{"model":%q,"messages":[{"role":"user","content":"ping"}]}`, matrixAnthropicModel),
[]string{"anthropic-version: 2023-06-01"})
}, 200)
assert.Equal(t, 200, code, "token counting must route rather than deny; body: %s%s", body, diag())
})
t.Run("bedrock token counting", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return env.client.PostJSON(ctx, env.endpoint, env.proxyIP,
"/model/"+matrixBedrockPathModel+"/count-tokens",
`{"input":{"converse":{"messages":[{"role":"user","content":[{"text":"ping"}]}]}}}`, nil)
}, 200)
assert.Equal(t, 200, code,
"the Bedrock count-tokens action must route; denying it pushes counting onto the billable inference path; body: %s%s",
body, diag())
})
t.Run("vertex token counting reaches its provider", func(t *testing.T) {
// The dummy service-account key cannot mint an OAuth token, so the
// request stops at the upstream credential. Both outcomes render as
// 403, so the deny code is what distinguishes them: upstream_auth_failed
// means the path resolved to the Vertex route and only the credential
// failed, while model_not_routable would mean the method segment was
// swallowed into the model id and no route ever claimed it.
path := fmt.Sprintf("/v1/projects/%s/locations/%s/publishers/anthropic/models/%s/count-tokens:rawPredict",
matrixVertexProject, matrixVertexRegion, matrixVertexModel)
_, body := callUntil(t, func() (int, string, error) {
return env.client.PostJSON(ctx, env.endpoint, env.proxyIP, path,
`{"anthropic_version":"vertex-2023-10-16","messages":[{"role":"user","content":"ping"}]}`, nil)
}, 403)
assert.NotContains(t, body, "model_not_routable",
"the count-tokens method segment must not be parsed as part of the model id; body: %s%s", body, diag())
assert.Contains(t, body, "llm_policy.upstream_auth_failed",
"the request must reach the Vertex route and fail only at the credential; body: %s%s", body, diag())
})
t.Run("connection warming probe", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return env.client.Get(ctx, env.endpoint, env.proxyIP, "/api/hello", nil)
}, 200)
assert.NotEqual(t, 403, code,
"the warm-up probe carries no model and must not be refused as unroutable; body: %s%s", body, diag())
})
t.Run("unknown model denies in the caller's error shape", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireMessages,
"claude-not-a-real-model-9", "ping", "e2e-gw-unknown")
}, 403)
require.Equal(t, 403, code, "a model no provider claims must still be refused; body: %s%s", body, diag())
// The NetBird fields stay where they were for existing consumers.
assert.Contains(t, body, "llm_policy.model_not_routable", "the deny code must be preserved")
// And the vendor's own envelope rides alongside, so the client can show
// the reason instead of an unexplained API error.
assert.Contains(t, body, `"type":"error"`, "an Anthropic caller must get the Anthropic error envelope")
assert.Contains(t, body, "permission_error", "403 must map to the vendor's permission error type")
})
}
// TestModelDiscoveryWithModelAllowlist covers gateway model discovery on an
// account that restricts models, which is the configuration that broke: the
// listing carries no model, and the per-model allowlist fails closed on an
// undetermined one, so discovery denied for exactly the accounts using the
// feature. It also asserts the allowlist still refuses a model outside it, so
// the exemption cannot be read as a way around the gate.
func TestModelDiscoveryWithModelAllowlist(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
vllm, err := harness.StartVLLM(ctx, srv)
require.NoError(t, err, "start mock upstream")
t.Cleanup(func() { _ = vllm.Terminate(context.Background()) })
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-gw-discovery"})
require.NoError(t, err, "create group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-gw-discovery-client",
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key")
require.NotEmpty(t, sk.Key, "setup key plaintext")
// One provider enumerating a single model, while the upstream's own listing
// advertises two. The proxy must serve the shorter list.
dummyKey := "sk-discovery-e2e"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "e2e-gw-discovery",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
Models: &[]api.AgentNetworkProviderModel{
{Id: harness.VLLMModel, InputPer1k: 0.001, OutputPer1k: 0.002},
},
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
// The model allowlist is what makes this a regression test: without a
// guardrail enabled, discovery was never gated in the first place.
var gr api.AgentNetworkGuardrailRequest
gr.Name = "e2e-gw-discovery-allowlist"
gr.Checks.ModelAllowlist.Enabled = true
gr.Checks.ModelAllowlist.Models = []string{harness.VLLMModel}
guard, err := srv.CreateGuardrail(ctx, gr)
require.NoError(t, err, "create guardrail")
t.Cleanup(func() { _ = srv.DeleteGuardrail(context.Background(), guard.Id) })
enabled := true
pol, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-gw-discovery",
Enabled: &enabled,
SourceGroups: []string{grp.Id},
DestinationProviderIds: []string{prov.Id},
GuardrailIds: &[]string{guard.Id},
})
require.NoError(t, err, "create policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
endpoint, proxyIP, cl, px := connectClient(t, ctx, "gw-discovery", sk.Key)
diag := func() string {
return fmt.Sprintf("\n=== upstream logs ===\n%s\n=== proxy logs ===\n%s",
vllm.Logs(context.Background()), px.Logs(context.Background()))
}
t.Run("listing is served and bounded by policy", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return cl.Get(ctx, endpoint, proxyIP, "/v1/models?limit=1000", nil)
}, 200)
require.Equal(t, 200, code,
"discovery must not be refused because the request carries no model; body: %s%s", body, diag())
assert.Contains(t, body, harness.VLLMModel, "the authorised model must reach the picker")
assert.NotContains(t, body, harness.VLLMUnlistedModel,
"a model the policy does not authorise must not be offered; body: %s", body)
})
t.Run("allowlist still refuses a model outside it", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return cl.Chat(ctx, endpoint, proxyIP, harness.WireChat,
harness.VLLMUnlistedModel, "ping", "e2e-gw-discovery-blocked")
}, 403)
require.Equal(t, 403, code,
"exempting model-less endpoints must not exempt inference; body: %s%s", body, diag())
assert.True(t,
strings.Contains(body, "llm_policy.model_blocked") || strings.Contains(body, "llm_policy.model_not_routable"),
"the refusal must name a model policy code; body: %s", body)
})
t.Run("allowlisted model still routes", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return cl.Chat(ctx, endpoint, proxyIP, harness.WireChat,
harness.VLLMModel, "ping", "e2e-gw-discovery-allowed")
}, 200)
require.Equal(t, 200, code, "the allowlisted model must still be served; body: %s%s", body, diag())
})
}

View File

@@ -0,0 +1,242 @@
//go:build e2e
package agentnetwork
import (
"context"
"fmt"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// The cases in this file cover behaviour that arrived from code review, after
// the gateway-protocol end-to-end tests were written. Each had unit coverage
// only; none needed a new harness capability, which is why they belong here
// rather than on a manual checklist.
// TestNonInferenceEndpointsAreAuthorised covers the two review findings on the
// endpoints that carry no body: the per-model lookup must be authorised
// against the same allowlist that bounds the listing beside it, and only a read
// method may claim the non-inference exemption that skips the token pre-flight.
func TestNonInferenceEndpointsAreAuthorised(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
env := provisionDiscoveryProvider(t, ctx)
t.Run("lookup of an authorised model succeeds", func(t *testing.T) {
code, body := callUntil(t, func() (int, string, error) {
return env.client.Get(ctx, env.endpoint, env.proxyIP, "/v1/models/"+harness.VLLMModel, nil)
}, 200)
assert.Equal(t, 200, code, "an allowlisted model must remain reachable; body: %s", body)
})
t.Run("lookup of an unauthorised model is refused", func(t *testing.T) {
code, body, err := env.client.Get(ctx, env.endpoint, env.proxyIP, "/v1/models/"+harness.VLLMUnlistedModel, nil)
require.NoError(t, err, "request must reach the proxy")
assert.Equal(t, 403, code,
"a model the policy does not authorise must not be confirmed by the detail lookup; body: %s", body)
})
// A write must not claim the exemption that lets the listing skip the token
// pre-flight. The body names no model on purpose: that is what a request
// probing for the exemption looks like, and it is the case the method gate
// exists to refuse. (A POST that does name a model is a different thing —
// it routes and meters as the inference request it is.)
for _, path := range []string{"/v1/models", "/v1/models/" + harness.VLLMModel, "/api/hello"} {
t.Run("write to "+path+" is refused", func(t *testing.T) {
code, body, err := env.client.PostJSON(ctx, env.endpoint, env.proxyIP, path,
`{"messages":[{"role":"user","content":"hi"}]}`, nil)
require.NoError(t, err, "request must reach the proxy")
assert.NotEqual(t, 200, code,
"a write to a non-inference path must not be served unmetered; body: %s", body)
})
}
// A request carrying the sub-agent attribution headers must still be served
// and metered normally. Asserting the ids themselves is not possible yet:
// the parser lifts them onto the request's metadata, but nothing persists
// them, so they have no queryable surface to check against.
t.Run("sub-agent headers do not disturb the request", func(t *testing.T) {
sessionID := fmt.Sprintf("e2e-session-agentid-%d", time.Now().UnixNano())
code, body, err := env.client.PostJSON(ctx, env.endpoint, env.proxyIP, "/v1/chat/completions",
fmt.Sprintf(`{"model":%q,"messages":[{"role":"user","content":"Reply with exactly: pong"}]}`, harness.VLLMModel),
[]string{
"x-session-id: " + sessionID,
"x-claude-code-agent-id: agent-child-7",
"x-claude-code-parent-agent-id: agent-root-1",
})
require.NoError(t, err, "request must reach the proxy")
require.Equal(t, 200, code, "the request must succeed; body: %s", body)
row := findAccessLogBySession(t, ctx, sessionID)
assert.Positive(t, row.InputTokens, "the request must still be metered normally")
})
}
// TestDatedModelIdRouting covers both halves of the dated-id rule that review
// tightened: a dated id still reaches an undated registration, but a route
// pinned to one dated build must never serve a different one.
func TestDatedModelIdRouting(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
const (
undated = "claude-sonnet-9"
datedA = "claude-sonnet-9-20250101"
datedB = "claude-sonnet-9-20250202"
)
t.Run("a dated id reaches its undated registration", func(t *testing.T) {
env := provisionModelProvider(t, ctx, "dated-undated", "anthropic_api", undated)
sessionID := fmt.Sprintf("e2e-session-dated-%d", time.Now().UnixNano())
code, body := callUntil(t, func() (int, string, error) {
return env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireMessages, datedA, "Reply with exactly: pong", sessionID)
}, 200)
require.Equal(t, 200, code, "a pinned release of a registered family must route; body: %s", body)
row := findAccessLogBySession(t, ctx, sessionID)
assert.Positive(t, row.InputTokens, "the dated request must price at the registered rate, not zero")
})
t.Run("a route pinned to one dated build refuses another", func(t *testing.T) {
env := provisionModelProvider(t, ctx, "dated-pinned", "anthropic_api", datedA)
code, body := callUntil(t, func() (int, string, error) {
return env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireMessages, datedA, "Reply with exactly: pong", "")
}, 200)
require.Equal(t, 200, code, "the exact dated id must still route; body: %s", body)
code, body, err := env.client.Chat(ctx, env.endpoint, env.proxyIP, harness.WireMessages, datedB, "Reply with exactly: pong", "")
require.NoError(t, err, "request must reach the proxy")
assert.Equal(t, 403, code,
"a provider pinned to one dated build must not serve another; body: %s", body)
})
}
// TestBedrockInferenceProfilesReachTheUpstream covers the startup lookup a
// Bedrock client makes. The proxy forwards it to the configured upstream rather
// than denying it, so what comes back is the upstream's answer — never a
// NetBird policy rejection.
func TestBedrockInferenceProfilesReachTheUpstream(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
env := provisionModelProvider(t, ctx, "infprofiles", "bedrock_api", "anthropic.claude-sonnet-5")
code, body := callUntil(t, func() (int, string, error) {
return env.client.Get(ctx, env.endpoint, env.proxyIP, "/inference-profiles", nil)
}, 200)
assert.Equal(t, 200, code, "the lookup must reach the upstream; body: %s", body)
assert.NotContains(t, body, "llm_policy.",
"the proxy must not answer a control-plane lookup with a policy denial")
assert.Contains(t, body, "inferenceProfileSummaries",
"the upstream's own answer must come back untouched")
}
// provisionDiscoveryProvider brings up one mock-backed provider enumerating a
// single model, with an allowlist guardrail in effect, plus a connected client.
func provisionDiscoveryProvider(t *testing.T, ctx context.Context) pricedEnv {
t.Helper()
env := provisionModelProvider(t, ctx, "noninference", "openai_api", harness.VLLMModel)
var gr api.AgentNetworkGuardrailRequest
gr.Name = "e2e-noninference-allowlist-" + fmt.Sprint(time.Now().UnixNano())
gr.Checks.ModelAllowlist.Enabled = true
gr.Checks.ModelAllowlist.Models = []string{harness.VLLMModel}
guard, err := srv.CreateGuardrail(ctx, gr)
require.NoError(t, err, "create guardrail")
t.Cleanup(func() { _ = srv.DeleteGuardrail(context.Background(), guard.Id) })
enabled := true
_, err = srv.UpdatePolicy(ctx, env.policyID, api.AgentNetworkPolicyRequest{
Name: "e2e-noninference",
Enabled: &enabled,
SourceGroups: []string{env.groupID},
DestinationProviderIds: []string{env.providerID},
GuardrailIds: &[]string{guard.Id},
})
require.NoError(t, err, "attach guardrail to policy")
return env
}
// provisionModelProvider brings up the mock, one provider under the given
// catalog id enumerating exactly one model, an authorising policy, and a
// connected proxy + client.
func provisionModelProvider(t *testing.T, ctx context.Context, name, catalogID, model string) pricedEnv {
t.Helper()
vllm, err := harness.StartVLLM(ctx, srv)
require.NoError(t, err, "start mock upstream")
t.Cleanup(func() { _ = vllm.Terminate(context.Background()) })
suffix := strings.ToLower(name)
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-gwr-" + suffix})
require.NoError(t, err, "create group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-gwr-" + suffix + "-client",
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key")
require.NotEmpty(t, sk.Key, "setup key plaintext")
dummyKey := "sk-gwr-e2e"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "e2e-gwr-" + suffix,
ProviderId: catalogID,
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
Models: &[]api.AgentNetworkProviderModel{
{Id: model, InputPer1k: 0.001, OutputPer1k: 0.002},
},
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
enabled := true
pol, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-gwr-" + suffix,
Enabled: &enabled,
SourceGroups: []string{grp.Id},
DestinationProviderIds: []string{prov.Id},
Limits: &api.AgentNetworkPolicyLimits{
TokenLimit: api.AgentNetworkPolicyTokenLimit{
Enabled: true,
GroupCap: 10_000_000,
UserCap: 10_000_000,
WindowSeconds: 60,
},
},
})
require.NoError(t, err, "create policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
endpoint, proxyIP, cl, px := connectClient(t, ctx, "gwr-"+suffix, sk.Key)
return pricedEnv{
providerID: prov.Id,
groupID: grp.Id,
policyID: pol.Id,
upstream: vllm.URL,
endpoint: endpoint,
proxyIP: proxyIP,
client: cl,
proxy: px,
}
}

View File

@@ -0,0 +1,199 @@
//go:build e2e
package agentnetwork
import (
"context"
"fmt"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// streamedModel is priced high enough that a mis-metered request is obvious in
// the recorded cost, and named so it cannot collide with another test's route.
const streamedModel = "e2e-streamed-model"
const (
streamInRate = 0.010
streamOutRate = 0.020
// The cache-read bucket is priced separately from input, so a run that
// folded the two together fails the per-bucket assertions below.
streamCacheReadRate = 0.001
)
// TestStreamingResponseMetersInputTokens is the end-to-end guard for the
// metering bug this endpoint's gateway-protocol work fixed.
//
// On a streamed answer the input-token count exists only in the opening
// message_start event; every later frame reports output. A response read with
// the wrong vendor's parser — the shape a gateway record produces when it names
// one API surface and serves another — never looks at that event, so input
// metered as zero and the bulk of the bill silently vanished. Nothing in the
// suite sent stream: true before this test, so the whole branch went unrun.
//
// The provider points at the mock's streaming listener, which answers every
// request as SSE with token counts that differ from the buffered surface. That
// difference is the point: passing these assertions is only possible if the
// stream accumulator ran.
func TestStreamingResponseMetersInputTokens(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
env := provisionStreamingProvider(t, ctx, "anthropic_api")
sessionID := fmt.Sprintf("e2e-session-stream-%d", time.Now().UnixNano())
code, body := chatStreamUntil(t, ctx, env, harness.WireMessages, streamedModel, sessionID)
require.Equal(t, 200, code, "streamed chat must succeed; body: %s", body)
assert.Contains(t, body, "message_start",
"the client must receive the event stream itself, not a buffered rewrite of it")
row := findAccessLogBySession(t, ctx, sessionID)
assert.Equal(t, harness.VLLMStreamInputTokens, int(row.InputTokens),
"input tokens live in message_start; zero here is the bug this test exists for")
assert.Equal(t, harness.VLLMStreamOutputTokens, int(row.OutputTokens),
"output tokens ride message_delta and supersede the message_start seed")
assert.Equal(t, harness.VLLMStreamCacheReadTokens, int(row.CachedInputTokens),
"the Anthropic cache bucket rides message_start too, and only its own parser reads it")
// The Anthropic surface bills cache reads additively, so the input bucket
// prices the full input count rather than a remainder.
wantInput := float64(harness.VLLMStreamInputTokens) / 1000 * streamInRate
wantOutput := float64(harness.VLLMStreamOutputTokens) / 1000 * streamOutRate
assert.InDelta(t, wantInput, row.InputCostUsd, 1e-6, "input cost must price the streamed input tokens")
assert.InDelta(t, wantOutput, row.OutputCostUsd, 1e-6, "output cost must price the streamed output tokens")
assert.Greater(t, row.CostUsd, 0.0, "a streamed request must never record as free")
}
// TestStreamingOnGatewayTypedProvider drives the same streamed Anthropic call
// through a provider record whose catalog id names the OpenAI surface — the
// exact misconfiguration that hid the bug, since gateway records commonly pin
// one parser while the upstream serves another shape entirely.
//
// The router must choose the parser from the request path rather than the
// record's provider id, or the Anthropic usage block goes unread and input
// meters at zero all over again.
func TestStreamingOnGatewayTypedProvider(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
env := provisionStreamingProvider(t, ctx, "openai_api")
sessionID := fmt.Sprintf("e2e-session-stream-gw-%d", time.Now().UnixNano())
code, body := chatStreamUntil(t, ctx, env, harness.WireMessages, streamedModel, sessionID)
require.Equal(t, 200, code, "streamed chat through a gateway record must succeed; body: %s", body)
row := findAccessLogBySession(t, ctx, sessionID)
assert.Equal(t, harness.VLLMStreamInputTokens, int(row.InputTokens),
"a record typed openai_api must still read the Anthropic usage block it is actually serving")
assert.Equal(t, harness.VLLMStreamOutputTokens, int(row.OutputTokens),
"output tokens must survive the surface mismatch too")
assert.InDelta(t, float64(harness.VLLMStreamInputTokens)/1000*streamInRate, row.InputCostUsd, 1e-6,
"the request must be priced on the surface it spoke, not the one the record names")
}
// provisionStreamingProvider brings up the mock, one provider pointed at its
// streaming listener under the given catalog id, a policy authorising it, and a
// connected proxy + client.
func provisionStreamingProvider(t *testing.T, ctx context.Context, catalogID string) pricedEnv {
t.Helper()
vllm, err := harness.StartVLLM(ctx, srv)
require.NoError(t, err, "start mock upstream")
t.Cleanup(func() { _ = vllm.Terminate(context.Background()) })
name := "stream-" + catalogID
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-" + name})
require.NoError(t, err, "create group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-" + name + "-client",
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key")
require.NotEmpty(t, sk.Key, "setup key plaintext")
dummyKey := "sk-stream-e2e"
cacheRead := streamCacheReadRate
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: name,
ProviderId: catalogID,
UpstreamUrl: vllm.StreamURL,
ApiKey: &dummyKey,
Enabled: ptr(true),
Models: &[]api.AgentNetworkProviderModel{{
Id: streamedModel,
InputPer1k: streamInRate,
OutputPer1k: streamOutRate,
CacheReadPer1k: &cacheRead,
}},
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
enabled := true
pol, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-" + name,
Enabled: &enabled,
SourceGroups: []string{grp.Id},
DestinationProviderIds: []string{prov.Id},
Limits: &api.AgentNetworkPolicyLimits{
TokenLimit: api.AgentNetworkPolicyTokenLimit{
Enabled: true,
GroupCap: 10_000_000,
UserCap: 10_000_000,
WindowSeconds: 60,
},
},
})
require.NoError(t, err, "create policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
endpoint, proxyIP, cl, px := connectClient(t, ctx, name, sk.Key)
return pricedEnv{
providerID: prov.Id,
groupID: grp.Id,
policyID: pol.Id,
upstream: vllm.StreamURL,
endpoint: endpoint,
proxyIP: proxyIP,
client: cl,
proxy: px,
}
}
// chatStreamUntil drives one streamed chat, retrying to absorb the tunnel and
// DNS jitter a first call through a fresh peer can hit.
func chatStreamUntil(t *testing.T, ctx context.Context, env pricedEnv, kind, model, sessionID string) (int, string) {
t.Helper()
var code int
var body string
deadline := time.Now().Add(90 * time.Second)
for time.Now().Before(deadline) {
c, b, cerr := env.client.ChatStream(ctx, env.endpoint, env.proxyIP, kind, model, "Reply with exactly: pong", sessionID)
if cerr == nil {
code, body = c, b
if code == 200 {
break
}
}
time.Sleep(5 * time.Second)
}
if code != 200 {
t.Logf("=== proxy logs ===\n%s", env.proxy.Logs(context.Background()))
}
return code, body
}

View File

@@ -7,6 +7,7 @@ import (
"errors"
"fmt"
"io"
"net/http"
"os/exec"
"strconv"
"strings"
@@ -199,12 +200,18 @@ func (cl *Client) pollStatus(ctx context.Context, timeout time.Duration, want st
const (
// curlExitCouldNotResolve is curl's exit code for a DNS resolution failure, distinct from connection-level failures.
curlExitCouldNotResolve = 6
// dnsProbeRetryWindow bounds DNS-failure retries: the synthesized zone lands a beat after management connects, so early NXDOMAIN is propagation; a zone still absent after this window is a real failure.
dnsProbeRetryWindow = 30 * time.Second
dnsProbeRetryInterval = 2 * time.Second
// curlExitCouldNotConnect is curl's exit code for a connection that never
// established. The probe exists to WAKE the lazy proxy peer, so the first
// attempt legitimately arrives before WireGuard has brought the tunnel up
// and fails here — which is propagation, exactly like an early NXDOMAIN,
// and belongs inside the retry window rather than failing the test outright.
curlExitCouldNotConnect = 7
// endpointProbeRetryWindow bounds retries of the transient failures above: the synthesized zone and the tunnel both land a beat after management connects. Still failing after this window is a real failure.
endpointProbeRetryWindow = 30 * time.Second
endpointProbeRetryInterval = 2 * time.Second
)
// ResolveProxyIP GETs https://<endpoint>/ from the client's netns: any HTTP status proves DNS + tunnel and wakes the lazy proxy peer; only DNS failures retry, within dnsProbeRetryWindow. Returns the connected IP for --resolve pinning.
// ResolveProxyIP GETs https://<endpoint>/ from the client's netns: any HTTP status proves DNS + tunnel and wakes the lazy proxy peer; DNS and connect failures retry, within endpointProbeRetryWindow. Returns the connected IP for --resolve pinning.
func (cl *Client) ResolveProxyIP(ctx context.Context, endpoint string) (string, error) {
args := []string{
"run", "--rm",
@@ -215,7 +222,7 @@ func (cl *Client) ResolveProxyIP(ctx context.Context, endpoint string) (string,
"-w", "%{remote_ip}",
"https://" + endpoint + "/",
}
deadline := time.Now().Add(dnsProbeRetryWindow)
deadline := time.Now().Add(endpointProbeRetryWindow)
for {
cmd := exec.CommandContext(ctx, "docker", args...)
var stdout, stderr strings.Builder
@@ -231,21 +238,29 @@ func (cl *Client) ResolveProxyIP(ctx context.Context, endpoint string) (string,
}
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) || exitErr.ExitCode() != curlExitCouldNotResolve {
if !errors.As(err, &exitErr) || !isTransientProbeExit(exitErr.ExitCode()) {
return "", fmt.Errorf("no HTTP response from %s: %w (%s)", endpoint, err, strings.TrimSpace(stderr.String()))
}
dnsErr := fmt.Errorf("DNS resolution failed for %s: %s", endpoint, strings.TrimSpace(stderr.String()))
if time.Until(deadline) < dnsProbeRetryInterval {
return "", dnsErr
probeErr := fmt.Errorf("endpoint %s not reachable yet: %s", endpoint, strings.TrimSpace(stderr.String()))
if time.Until(deadline) < endpointProbeRetryInterval {
return "", probeErr
}
select {
case <-ctx.Done():
return "", fmt.Errorf("%w (%w)", dnsErr, ctx.Err())
case <-time.After(dnsProbeRetryInterval):
return "", fmt.Errorf("%w (%w)", probeErr, ctx.Err())
case <-time.After(endpointProbeRetryInterval):
}
}
}
// isTransientProbeExit reports whether a curl exit code describes a state the
// endpoint is expected to pass THROUGH on its way up, rather than a settled
// failure. Anything else — TLS refusal, a protocol error, a bad argument —
// would still be failing after the retry window, so it fails immediately.
func isTransientProbeExit(code int) bool {
return code == curlExitCouldNotResolve || code == curlExitCouldNotConnect
}
// Wire shapes for Chat.
const (
// WireChat is the OpenAI-compatible /v1/chat/completions shape.
@@ -292,6 +307,27 @@ func (cl *Client) ChatPrefixed(ctx context.Context, endpoint, proxyIP, pathPrefi
return cl.post(ctx, endpoint, proxyIP, pathPrefix+path, body, withSessionID(headers, sessionID))
}
// ChatStream is Chat with "stream": true in the request body, so the proxy's
// request parser marks the call as streaming and its response parser takes the
// SSE accumulator rather than the buffered-body path. Pair it with a provider
// pointed at VLLM.StreamURL, which answers every request as an event stream.
func (cl *Client) ChatStream(ctx context.Context, endpoint, proxyIP, kind, model, prompt, sessionID string) (int, string, error) {
var path, body string
var headers []string
switch kind {
case WireMessages:
path = "/v1/messages"
headers = []string{"anthropic-version: 2023-06-01"}
body = fmt.Sprintf(`{"model":%q,"max_tokens":2048,"stream":true,"messages":[{"role":"user","content":%q}]}`, model, prompt)
default:
path = "/v1/chat/completions"
// include_usage is what makes a real OpenAI stream emit its final usage
// frame; without it the last chunk carries no tokens at all.
body = fmt.Sprintf(`{"model":%q,"stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":%q}]}`, model, prompt)
}
return cl.post(ctx, endpoint, proxyIP, path, body, withSessionID(headers, sessionID))
}
// Vertex issues an Anthropic-on-Vertex rawPredict POST over the tunnel. Unlike
// Chat, the model is carried in the request path (project/region/model), so the
// proxy routes by path and mints the service-account OAuth token; the body uses
@@ -322,10 +358,29 @@ func withSessionID(headers []string, sessionID string) []string {
return append(headers, "x-session-id: "+sessionID)
}
// post runs curl in a throwaway container sharing the client's network
// namespace so the request traverses the WireGuard tunnel, pinning the endpoint
// to the proxy IP. It returns the HTTP status and response body.
// Get issues a GET to the agent-network endpoint over the client's tunnel.
// Model discovery and the connection-warming probe are read-only endpoints
// that carry no body, so they can't go through the chat helpers.
func (cl *Client) Get(ctx context.Context, endpoint, proxyIP, path string, extraHeaders []string) (int, string, error) {
return cl.do(ctx, http.MethodGet, endpoint, proxyIP, path, "", extraHeaders)
}
// PostJSON issues an arbitrary JSON POST over the client's tunnel, for wire
// shapes the typed helpers don't cover (token counting, say).
func (cl *Client) PostJSON(ctx context.Context, endpoint, proxyIP, path, body string, extraHeaders []string) (int, string, error) {
return cl.do(ctx, http.MethodPost, endpoint, proxyIP, path, body, extraHeaders)
}
// post issues a JSON POST. Retained as the shorthand the chat helpers use.
func (cl *Client) post(ctx context.Context, endpoint, proxyIP, path, body string, extraHeaders []string) (int, string, error) {
return cl.do(ctx, http.MethodPost, endpoint, proxyIP, path, body, extraHeaders)
}
// do runs curl in a throwaway container sharing the client's network
// namespace so the request traverses the WireGuard tunnel, pinning the endpoint
// to the proxy IP. It returns the HTTP status and response body. An empty body
// sends no payload, which is what a GET needs.
func (cl *Client) do(ctx context.Context, method, endpoint, proxyIP, path, body string, extraHeaders []string) (int, string, error) {
url := "https://" + endpoint + path
args := []string{
"run", "--rm",
@@ -334,13 +389,15 @@ func (cl *Client) post(ctx context.Context, endpoint, proxyIP, path, body string
"-sk", "--connect-timeout", "5", "--max-time", "90",
"--resolve", endpoint + ":443:" + proxyIP,
"-o", "/dev/stderr", "-w", "%{http_code}",
"-X", "POST", url,
"-X", method, url,
"-H", "Content-Type: application/json",
}
for _, h := range extraHeaders {
args = append(args, "-H", h)
}
args = append(args, "--data", body)
if body != "" {
args = append(args, "--data", body)
}
cmd := exec.CommandContext(ctx, "docker", args...)
// -w writes the status code to stdout; -o /dev/stderr writes the body to
// stderr so we can capture both separately.

View File

@@ -18,18 +18,63 @@ const (
vllmImage = "nginx:alpine"
vllmAlias = "vllm"
vllmPort = "8000/tcp"
// vllmStreamPort serves the same wire shapes as an SSE stream. See the
// nginx config for why streaming lives on its own listener.
vllmStreamPort = "8001/tcp"
// VLLMModel is the served model id the mock advertises and echoes back. It
// matches a real small model commonly served by vLLM so the provider's
// enumerated model and the client's request line up.
VLLMModel = "Qwen/Qwen2.5-0.5B-Instruct"
// VLLMUnlistedModel is a second id the mock's model listing advertises but
// no test provider enumerates, so a filtered listing is observably shorter
// than the upstream's own.
VLLMUnlistedModel = "Qwen/Qwen2.5-7B-Instruct"
)
// Token counts the mock reports per wire shape. Tests assert on these rather
// than on "> 0" so a response parsed with the wrong provider's parser (which
// would read a different field, or none) fails loudly instead of passing on
// a coincidental non-zero.
const (
// VLLMChatInputTokens / VLLMChatOutputTokens ride the OpenAI usage block.
VLLMChatInputTokens = 11
VLLMChatOutputTokens = 2
// VLLMMessagesInputTokens / VLLMMessagesOutputTokens ride the Anthropic
// usage block, whose field names the OpenAI parser cannot read.
VLLMMessagesInputTokens = 17
VLLMMessagesOutputTokens = 3
)
// Token counts the streaming surface reports. They differ from the
// non-streaming ones on purpose: a test that asserts these numbers proves the
// SSE accumulator ran, rather than a buffered JSON body having been parsed.
//
// Input and cache-read arrive on message_start; output arrives on
// message_delta and supersedes the seed value message_start carries. Any
// parser that cannot read message_start reports zero input tokens — which is
// exactly the bug these counts exist to catch.
const (
VLLMStreamInputTokens = 29
VLLMStreamOutputTokens = 5
VLLMStreamCacheReadTokens = 7
)
// vllmNginxConf emulates a vLLM OpenAI-compatible server over plain HTTP (vLLM's
// default: no TLS, port 8000). It answers /v1/models with a one-model list and
// any chat/completions path with a canned OpenAI-shaped chat completion carrying
// a non-zero usage block, so the proxy's OpenAI parser records real token
// consumption. Running actual vLLM in CI is infeasible (GPU + multi-GB model
// default: no TLS, port 8000), and additionally answers the wire shapes the
// other catalog surfaces speak so one mock can stand in for every provider the
// proxy routes to. Running actual vLLM in CI is infeasible (GPU + multi-GB model
// download), so this stands in for the wire contract the proxy depends on.
//
// Each shape answers with its own vendor's usage block, so a response parsed
// under the wrong surface meters zero rather than passing by accident:
//
// - /v1/chat/completions (and any unmatched path): OpenAI chat completion.
// - /v1/messages: Anthropic Messages, snake_case usage plus a cache bucket.
// - /model/{id}/invoke: Bedrock InvokeModel, which carries the Anthropic body.
// - the token-counting endpoints: a count, with no usage block at all.
//
// The model listing advertises two models so a policy that authorises one
// produces an observably shorter list than the upstream's own.
const vllmNginxConf = `pid /tmp/nginx.pid;
events {}
http {
@@ -37,13 +82,75 @@ http {
listen 8000;
location = /v1/models {
default_type application/json;
return 200 '{"object":"list","data":[{"id":"Qwen/Qwen2.5-0.5B-Instruct","object":"model","owned_by":"vllm"}]}';
return 200 '{"object":"list","data":[{"id":"Qwen/Qwen2.5-0.5B-Instruct","object":"model","owned_by":"vllm"},{"id":"Qwen/Qwen2.5-7B-Instruct","object":"model","owned_by":"vllm"}]}';
}
location = /v1/messages {
default_type application/json;
return 200 '{"id":"msg_e2e","type":"message","role":"assistant","model":"claude-sonnet-5","content":[{"type":"text","text":"pong"}],"stop_reason":"end_turn","usage":{"input_tokens":17,"output_tokens":3,"cache_read_input_tokens":5}}';
}
location = /v1/messages/count_tokens {
default_type application/json;
return 200 '{"input_tokens":7}';
}
location ~ ^/model/.+/invoke$ {
default_type application/json;
return 200 '{"id":"msg_e2e_bedrock","type":"message","role":"assistant","content":[{"type":"text","text":"pong"}],"stop_reason":"end_turn","usage":{"input_tokens":17,"output_tokens":3,"cache_read_input_tokens":5}}';
}
location ~ ^/model/.+/count-tokens$ {
default_type application/json;
return 200 '{"inputTokens":9}';
}
location = /api/hello {
return 200;
}
location = /inference-profiles {
default_type application/json;
return 200 '{"inferenceProfileSummaries":[{"inferenceProfileId":"us.anthropic.claude-sonnet-5","status":"ACTIVE"}]}';
}
location / {
default_type application/json;
return 200 '{"id":"chatcmpl-e2e-vllm","object":"chat.completion","created":1700000000,"model":"Qwen/Qwen2.5-0.5B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"pong"},"finish_reason":"stop"}],"usage":{"prompt_tokens":11,"completion_tokens":2,"total_tokens":13}}';
}
}
# The streaming surface, on its own port so the response content type is a
# property of the listener rather than of a per-request branch: nginx sets
# Content-Type from default_type, which cannot be varied inside an "if", and
# a second Content-Type via add_header would leave the proxy reading the
# wrong one. A provider record pointed at this port streams every answer.
#
# Input and cache-read tokens ride message_start, output rides message_delta
# — the split that makes a stream different from a buffered body, and the
# reason a parser that ignores message_start meters input as zero.
server {
listen 8001;
location = /v1/messages {
default_type text/event-stream;
return 200 'event: message_start
data: {"type":"message_start","message":{"id":"msg_e2e_stream","type":"message","role":"assistant","model":"claude-sonnet-5","content":[],"usage":{"input_tokens":29,"output_tokens":1,"cache_read_input_tokens":7}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"pong"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":5}}
event: message_stop
data: {"type":"message_stop"}
';
}
location / {
default_type text/event-stream;
return 200 'data: {"choices":[{"delta":{"content":"pong"}}]}
data: {"choices":[],"usage":{"prompt_tokens":29,"completion_tokens":5,"total_tokens":34}}
data: [DONE]
';
}
}
}
`
@@ -55,6 +162,10 @@ type VLLM struct {
workDir string
// URL is the upstream URL the vllm provider points at (http://<alias>:8000).
URL string
// StreamURL is the same mock's streaming listener. A provider pointed here
// answers every request as SSE, so the proxy's streaming accumulator runs
// instead of its buffered-body parser.
StreamURL string
}
// StartVLLM runs the mock vLLM server on the shared network over plain HTTP.
@@ -73,14 +184,17 @@ func StartVLLM(ctx context.Context, c *Combined) (*VLLM, error) {
req := testcontainers.ContainerRequest{
Image: vllmImage,
ExposedPorts: []string{vllmPort},
ExposedPorts: []string{vllmPort, vllmStreamPort},
Networks: []string{c.network.Name},
NetworkAliases: map[string][]string{c.network.Name: {vllmAlias}},
Cmd: []string{"nginx", "-c", "/conf/nginx.conf", "-g", "daemon off;"},
HostConfigModifier: func(hc *container.HostConfig) {
hc.Binds = append(hc.Binds, workDir+":/conf:ro")
},
WaitingFor: wait.ForListeningPort(vllmPort).WithStartupTimeout(60 * time.Second),
WaitingFor: wait.ForAll(
wait.ForListeningPort(vllmPort),
wait.ForListeningPort(vllmStreamPort),
).WithStartupTimeout(60 * time.Second),
}
ctr, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
@@ -92,7 +206,12 @@ func StartVLLM(ctx context.Context, c *Combined) (*VLLM, error) {
return nil, fmt.Errorf("start vllm container: %w", err)
}
return &VLLM{container: ctr, workDir: workDir, URL: "http://" + vllmAlias + ":8000"}, nil
return &VLLM{
container: ctr,
workDir: workDir,
URL: "http://" + vllmAlias + ":8000",
StreamURL: "http://" + vllmAlias + ":8001",
}, nil
}
// Logs returns the vLLM container logs, for diagnostics on failure.

2
go.mod
View File

@@ -57,6 +57,7 @@ require (
github.com/fsnotify/fsnotify v1.9.0
github.com/gliderlabs/ssh v0.3.8
github.com/go-jose/go-jose/v4 v4.1.4
github.com/go-ole/go-ole v1.3.0
github.com/gobwas/ws v1.4.0
github.com/goccy/go-yaml v1.18.0
github.com/godbus/dbus/v5 v5.2.2
@@ -199,7 +200,6 @@ require (
github.com/go-ldap/ldap/v3 v3.4.13 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.2 // indirect
github.com/go-openapi/jsonpointer v0.21.1 // indirect

View File

@@ -111,6 +111,59 @@ check_nb_domain() {
return 0
}
# Non-interactive configuration
# ------------------------------
# Every prompt below can be pre-answered with an environment variable, so the
# script runs unattended (cloud-init, CI, Terraform, curl | bash). resolve()
# is the single place that decides env var vs prompt vs default; the read_*
# helpers stay pure prompts.
#
# Supported env vars:
# NETBIRD_DOMAIN domain/FQDN (required)
# NETBIRD_LETSENCRYPT_EMAIL ACME email (required for built-in Traefik)
# NETBIRD_AGENT_NETWORK true enables the agent-network preset
# NETBIRD_REVERSE_PROXY_TYPE 0-5 (default 0 = built-in Traefik)
# NETBIRD_ENABLE_PROXY true/false (default false)
# NETBIRD_ENABLE_CROWDSEC true/false (default false)
# NETBIRD_TRAEFIK_EXTERNAL_NETWORK external-Traefik network (type 1)
# NETBIRD_TRAEFIK_ENTRYPOINT external-Traefik entrypoint (type 1, default websecure)
# NETBIRD_TRAEFIK_CERTRESOLVER external-Traefik cert resolver (type 1)
# NETBIRD_BIND_LOCALHOST_ONLY true/false (default true, types 2-5)
# NETBIRD_EXTERNAL_PROXY_NETWORK docker network to join (types 2-4)
# NETBIRD_NON_INTERACTIVE true forces unattended mode even with a TTY
# tty_available succeeds only when we may prompt: never when the operator has
# set NETBIRD_NON_INTERACTIVE=true, otherwise only when /dev/tty can actually
# be opened. A PTY can be attached in automation (CI runners, some
# provisioners), so the env override is the authoritative signal and the
# /dev/tty probe is the fallback. /dev/tty is a world-rw device node even with
# no terminal, so a permission test ([ -r ]) is not enough - we must open it.
tty_available() {
[[ "${NETBIRD_NON_INTERACTIVE:-}" == "true" ]] && return 1
{ true < /dev/tty; } 2>/dev/null
}
# resolve ENV_VAR_NAME DEFAULT PROMPT_FN [prompt args...]
# env var set and non-empty -> its value
# interactive -> PROMPT_FN "$@" (prompt behavior unchanged)
# otherwise -> DEFAULT, or abort when DEFAULT is "required"
resolve() {
local env_name="$1" default="$2" prompt_fn="$3"
shift 3
local env_value="${!env_name:-}"
if [[ -n "$env_value" ]]; then
echo "$env_value"
elif tty_available; then
"$prompt_fn" "$@"
elif [[ "$default" == "required" ]]; then
echo "$env_name is required for a non-interactive install." > /dev/stderr
exit 1
else
echo "$default"
fi
return 0
}
read_nb_domain() {
READ_NETBIRD_DOMAIN=""
echo -n "Enter the domain you want to use for NetBird (e.g. netbird.my-domain.com): " > /dev/stderr
@@ -383,7 +436,14 @@ initialize_default_values() {
}
configure_domain() {
# Domain is validated (not a free-form value), so it keeps its own guard
# rather than going through resolve(): a valid NETBIRD_DOMAIN is used as-is,
# otherwise we prompt, or abort when there is no terminal to prompt on.
if ! check_nb_domain "$NETBIRD_DOMAIN"; then
if ! tty_available; then
echo "NETBIRD_DOMAIN is required for a non-interactive install." > /dev/stderr
exit 1
fi
NETBIRD_DOMAIN=$(read_nb_domain)
fi
@@ -411,11 +471,7 @@ apply_agent_network_preset() {
ENABLE_PROXY="true"
ENABLE_CROWDSEC="false"
if [[ -n "${NETBIRD_LETSENCRYPT_EMAIL}" ]]; then
TRAEFIK_ACME_EMAIL="${NETBIRD_LETSENCRYPT_EMAIL}"
else
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
fi
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
echo "" > /dev/stderr
echo "Agent-network preset enabled (NETBIRD_AGENT_NETWORK=true):" > /dev/stderr
@@ -437,35 +493,35 @@ configure_reverse_proxy() {
return 0
fi
# Prompt for reverse proxy type
REVERSE_PROXY_TYPE=$(read_reverse_proxy_type)
# Reverse proxy type (env NETBIRD_REVERSE_PROXY_TYPE, else prompt, else 0)
REVERSE_PROXY_TYPE=$(resolve NETBIRD_REVERSE_PROXY_TYPE 0 read_reverse_proxy_type)
# Handle built-in Traefik prompts (option 0)
if [[ "$REVERSE_PROXY_TYPE" == "0" ]]; then
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
ENABLE_PROXY=$(read_enable_proxy)
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
ENABLE_PROXY=$(resolve NETBIRD_ENABLE_PROXY false read_enable_proxy)
if [[ "$ENABLE_PROXY" == "true" ]]; then
ENABLE_CROWDSEC=$(read_enable_crowdsec)
ENABLE_CROWDSEC=$(resolve NETBIRD_ENABLE_CROWDSEC false read_enable_crowdsec)
fi
fi
# Handle external Traefik-specific prompts (option 1)
if [[ "$REVERSE_PROXY_TYPE" == "1" ]]; then
TRAEFIK_EXTERNAL_NETWORK=$(read_traefik_network)
TRAEFIK_ENTRYPOINT=$(read_traefik_entrypoint)
TRAEFIK_CERTRESOLVER=$(read_traefik_certresolver)
TRAEFIK_EXTERNAL_NETWORK=$(resolve NETBIRD_TRAEFIK_EXTERNAL_NETWORK "" read_traefik_network)
TRAEFIK_ENTRYPOINT=$(resolve NETBIRD_TRAEFIK_ENTRYPOINT websecure read_traefik_entrypoint)
TRAEFIK_CERTRESOLVER=$(resolve NETBIRD_TRAEFIK_CERTRESOLVER "" read_traefik_certresolver)
fi
# Handle port binding for external proxy options (2-5)
if [[ "$REVERSE_PROXY_TYPE" -ge 2 ]]; then
BIND_LOCALHOST_ONLY=$(read_port_binding_preference)
BIND_LOCALHOST_ONLY=$(resolve NETBIRD_BIND_LOCALHOST_ONLY true read_port_binding_preference)
fi
# Handle Docker network prompts for external proxies (options 2-4)
case "$REVERSE_PROXY_TYPE" in
2) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx") ;;
3) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx Proxy Manager") ;;
4) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Caddy") ;;
2) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx") ;;
3) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx Proxy Manager") ;;
4) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Caddy") ;;
*) ;; # No network prompt for other options
esac
return 0
@@ -643,8 +699,13 @@ start_services_and_show_instructions() {
print_post_setup_instructions
echo ""
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
read -r < /dev/tty
if tty_available; then
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
read -r < /dev/tty
else
echo "Non-interactive mode: starting NetBird containers now. Finish configuring"
echo "your reverse proxy using the instructions above so it can reach them."
fi
echo -e "$MSG_STARTING_SERVICES"
$DOCKER_COMPOSE_COMMAND up -d

View File

@@ -113,8 +113,61 @@ type Provider struct {
// upstream provider + credentials on Portkey's hosted side).
ExtraHeaders []ExtraHeader
Models []Model
// Discovery, when non-nil, describes how to ask this vendor which
// models the operator's own credential can actually reach, so the
// provider form can offer a live list instead of only the hand-curated
// Models above. Nil for entries with no listing endpoint (gateways
// vary too much) — those keep free-text entry.
Discovery *Discovery
}
// ListingShape names the response envelope a vendor returns its model
// listing in. Every vendor invented its own, and none of them can be
// guessed from the request, so the catalog states it.
type ListingShape string
const (
// ShapeOpenAIData is {"data":[{"id":…}]} — OpenAI, and Anthropic, which
// adopted the same envelope.
ShapeOpenAIData ListingShape = "openai_data"
// ShapeBedrockInferenceProfiles is
// {"inferenceProfileSummaries":[{"inferenceProfileId":…}]}. The ids carry
// the region prefix that makes them invocable, which is exactly what an
// operator cannot reconstruct by hand.
ShapeBedrockInferenceProfiles ListingShape = "bedrock_inference_profiles"
// ShapeVertexPublisherModels is {"publisherModels":[{"name":…}]}, where
// name is a resource path and the invocable id is its last segment joined
// to a separate versionId field.
ShapeVertexPublisherModels ListingShape = "vertex_publisher_models"
)
// Discovery describes one vendor's model-listing endpoint.
//
// Host is deliberately separate from the provider record's upstream URL:
// Bedrock serves listings from the control plane (bedrock.<region>) while
// inference must go to the runtime host (bedrock-runtime.<region>), so the
// two cannot be the same value. Empty Host means "use the record's own
// upstream", which is right for every vendor that serves both from one host.
//
// The regionPlaceholder in Host is substituted from the provider record's
// region. Deriving the discovery host from the catalog rather than accepting
// one from the caller is also what keeps this from being an open proxy: the
// only hosts management will dial are the ones written here.
type Discovery struct {
Host string
Path string
Query string
Shape ListingShape
// Headers are static headers the vendor requires beyond the credential
// (Anthropic versions its API through one and rejects a request without
// it). The auth header itself comes from AuthHeaderName/Template.
Headers map[string]string
}
// RegionPlaceholder is replaced in Discovery.Host by the provider record's
// configured region.
const RegionPlaceholder = "<region>"
// ExtraHeader names a single optional per-provider routing/config
// header. Catalog declares N of these per provider type; the operator
// fills any subset on the provider record (see Provider.ExtraValues).
@@ -245,8 +298,12 @@ var providers = []Provider{
AuthHeaderTemplate: "Bearer ${API_KEY}",
DefaultContentType: "application/json",
BrandColor: "#10A37F",
ParserID: "openai",
PricingSurfaces: []string{"openai"},
Discovery: &Discovery{
Path: "/v1/models",
Shape: ShapeOpenAIData,
},
ParserID: "openai",
PricingSurfaces: []string{"openai"},
// Pricing + context windows cross-checked against LiteLLM's
// model_prices_and_context_window.json. Notable corrections from
// earlier values: o4-mini repriced from $4/$16 to $1.10/$4.40
@@ -284,8 +341,18 @@ var providers = []Provider{
AuthHeaderTemplate: "${API_KEY}",
DefaultContentType: "application/json",
BrandColor: "#D97757",
ParserID: "anthropic",
PricingSurfaces: []string{"anthropic"},
Discovery: &Discovery{
Path: "/v1/models",
// The default page is short and a picker wants the whole
// catalogue in one call.
Query: "limit=1000",
Shape: ShapeOpenAIData,
// Anthropic versions its API through a header and refuses a
// request that omits it, listing included.
Headers: map[string]string{"anthropic-version": "2023-06-01"},
},
ParserID: "anthropic",
PricingSurfaces: []string{"anthropic"},
// Per Anthropic's current model lineup. Pricing in USD per 1k
// tokens. Context windows: 4.6+ family is 1M; Haiku 4.5 stays at
// 200K. claude-3-7-sonnet and claude-3-5-haiku retired
@@ -296,6 +363,8 @@ var providers = []Provider{
// account to be on >= 30-day data retention or all requests
// 400.
Models: []Model{
{ID: "claude-opus-5", Label: "Claude Opus 5", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "claude-sonnet-5", Label: "Claude Sonnet 5", InputPer1k: 0.003, OutputPer1k: 0.015, CacheReadPer1k: 0.0003, CacheCreationPer1k: 0.00375, ContextWindow: 1000000},
{ID: "claude-fable-5", Label: "Claude Fable 5", InputPer1k: 0.010, OutputPer1k: 0.050, CacheReadPer1k: 0.001, CacheCreationPer1k: 0.0125, ContextWindow: 1000000},
{ID: "claude-opus-4-8", Label: "Claude Opus 4.8", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "claude-opus-4-7", Label: "Claude Opus 4.7", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
@@ -343,6 +412,22 @@ var providers = []Provider{
AuthHeaderTemplate: "Bearer ${API_KEY}",
DefaultContentType: "application/json",
BrandColor: "#FF9900",
// Listings come from the CONTROL PLANE, not the runtime host in
// DefaultHost above: ListInferenceProfiles is not an operation
// bedrock-runtime implements, and answers <UnknownOperationException/>
// there. Inference has to go to the runtime host, so the two hosts
// genuinely differ and Discovery.Host carries the difference.
//
// Inference profiles rather than foundation models because the profile
// id is the invocable one: it carries the region prefix (eu., us.,
// global.) that AWS requires and that cannot be derived from the
// configured region — an eu-central-1 account legitimately holds
// global.* profiles.
Discovery: &Discovery{
Host: "bedrock." + RegionPlaceholder + ".amazonaws.com",
Path: "/inference-profiles",
Shape: ShapeBedrockInferenceProfiles,
},
// ParserID stays empty (path-style dispatch via IsBedrockPathStyle);
// the request parser meters these under the "bedrock" surface.
PricingSurfaces: []string{"bedrock"},
@@ -355,6 +440,8 @@ var providers = []Provider{
// Llama 3.3 70B entry kept unchanged — LiteLLM tracks only
// per-region Llama 3 entries; standalone 3.3 not yet listed.
Models: []Model{
{ID: "anthropic.claude-opus-5", Label: "Claude Opus 5 (Bedrock)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "anthropic.claude-sonnet-5", Label: "Claude Sonnet 5 (Bedrock)", InputPer1k: 0.003, OutputPer1k: 0.015, CacheReadPer1k: 0.0003, CacheCreationPer1k: 0.00375, ContextWindow: 1000000},
{ID: "anthropic.claude-opus-4-8", Label: "Claude Opus 4.8 (Bedrock)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "anthropic.claude-opus-4-7", Label: "Claude Opus 4.7 (Bedrock)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "anthropic.claude-opus-4-6", Label: "Claude Opus 4.6 (Bedrock)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
@@ -391,6 +478,15 @@ var providers = []Provider{
AuthHeaderTemplate: "Bearer ${API_KEY}",
DefaultContentType: "application/json",
BrandColor: "#4285F4",
// Only the v1beta1 publisher listing answers: the v1 form and the
// project-scoped form under BOTH versions return 404. That means the
// list is publisher-global — it cannot say which models this project
// has enabled — so it is offered as a suggestion beside the catalog
// rather than replacing it. See the discovery e2e for the probes.
Discovery: &Discovery{
Path: "/v1beta1/publishers/anthropic/models",
Shape: ShapeVertexPublisherModels,
},
// ParserID stays empty (path-style dispatch via IsVertexPathStyle);
// Anthropic-on-Vertex requests are metered under the "anthropic"
// surface with the bare, unversioned model id.
@@ -406,6 +502,8 @@ var providers = []Provider{
// exists — the router denies unmeterable publishers rather than forward
// them uncounted.
Models: []Model{
{ID: "claude-opus-5", Label: "Claude Opus 5 (Vertex)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "claude-sonnet-5", Label: "Claude Sonnet 5 (Vertex)", InputPer1k: 0.003, OutputPer1k: 0.015, CacheReadPer1k: 0.0003, CacheCreationPer1k: 0.00375, ContextWindow: 1000000},
{ID: "claude-fable-5", Label: "Claude Fable 5 (Vertex)", InputPer1k: 0.010, OutputPer1k: 0.050, CacheReadPer1k: 0.001, CacheCreationPer1k: 0.0125, ContextWindow: 1000000},
{ID: "claude-opus-4-8", Label: "Claude Opus 4.8 (Vertex)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},
{ID: "claude-opus-4-7", Label: "Claude Opus 4.7 (Vertex)", InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625, ContextWindow: 1000000},

View File

@@ -0,0 +1,36 @@
package catalog
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestClaudeLineupSelectable pins the models Claude Code resolves to by
// default. A model absent from the lineup can't be ticked on a provider
// record, so llm_router denies it as not-routable and the operator has no
// way to authorise the client's own default.
func TestClaudeLineupSelectable(t *testing.T) {
for providerID, wanted := range map[string][]string{
"anthropic_api": {"claude-opus-5", "claude-sonnet-5", "claude-haiku-4-5"},
"bedrock_api": {"anthropic.claude-opus-5", "anthropic.claude-sonnet-5", "anthropic.claude-haiku-4-5"},
"vertex_ai_api": {"claude-opus-5", "claude-sonnet-5", "claude-haiku-4-5"},
} {
provider, ok := Lookup(providerID)
require.True(t, ok, "catalog must define %s", providerID)
selectable := make(map[string]Model, len(provider.Models))
for _, m := range provider.Models {
selectable[m.ID] = m
}
for _, id := range wanted {
model, found := selectable[id]
require.True(t, found, "%s must offer %s", providerID, id)
assert.NotEmpty(t, model.Label, "%s/%s needs a label for the picker", providerID, id)
assert.Positive(t, model.InputPer1k, "%s/%s needs an input rate", providerID, id)
assert.Positive(t, model.OutputPer1k, "%s/%s needs an output rate", providerID, id)
assert.Positive(t, model.ContextWindow, "%s/%s needs a context window", providerID, id)
}
}
}

View File

@@ -0,0 +1,137 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/modeldiscovery"
nbcontext "github.com/netbirdio/netbird/management/server/context"
"github.com/netbirdio/netbird/shared/auth"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// discoveryManagerStub records what the handler asked for and returns a canned
// answer. The Manager interface is embedded rather than implemented: only the
// one method is reachable from this handler, and a call to any other should
// fail loudly rather than silently return a zero value.
type discoveryManagerStub struct {
agentnetwork.Manager
gotReq modeldiscovery.Request
gotRecordID string
models []modeldiscovery.Model
err error
}
func (s *discoveryManagerStub) DiscoverProviderModels(
_ context.Context, _, _ string, req modeldiscovery.Request, recordID string,
) ([]modeldiscovery.Model, error) {
s.gotReq = req
s.gotRecordID = recordID
return s.models, s.err
}
// postDiscovery drives the handler with an authenticated request.
func postDiscovery(t *testing.T, stub *discoveryManagerStub, body string) *httptest.ResponseRecorder {
t.Helper()
h := &handler{manager: stub}
req := httptest.NewRequest(http.MethodPost, "/agent-network/catalog/providers/models", strings.NewReader(body))
req = req.WithContext(nbcontext.SetUserAuthInContext(req.Context(), auth.UserAuth{
AccountId: "acc-1",
UserId: "user-1",
}))
rec := httptest.NewRecorder()
h.discoverProviderModels(rec, req)
return rec
}
func TestDiscoverModelsReturnsTheVendorList(t *testing.T) {
stub := &discoveryManagerStub{models: []modeldiscovery.Model{
{ID: "eu.anthropic.claude-haiku-4-5-20251001-v1:0", Label: "EU Claude Haiku 4.5", PricingKnown: true},
{ID: "global.cohere.embed-v4:0", Label: "Global Cohere Embed v4"},
}}
rec := postDiscovery(t, stub, `{
"catalog_provider_id":"bedrock_api",
"upstream_url":"https://bedrock-runtime.eu-central-1.amazonaws.com",
"api_key":"aws-bearer"
}`)
require.Equal(t, http.StatusOK, rec.Code, "body: %s", rec.Body.String())
var out api.AgentNetworkModelDiscoveryResponse
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &out))
require.Len(t, out.Models, 2)
assert.Equal(t, "eu.anthropic.claude-haiku-4-5-20251001-v1:0", out.Models[0].Id)
assert.True(t, out.Models[0].PricingKnown)
// An unpriced model must say so rather than arriving indistinguishable
// from a priced one: registering it silently would meter at zero.
assert.False(t, out.Models[1].PricingKnown)
assert.Equal(t, "bedrock_api", stub.gotReq.CatalogID)
assert.Equal(t, "aws-bearer", stub.gotReq.APIKey)
assert.Empty(t, stub.gotRecordID)
}
func TestDiscoverModelsUsesAStoredRecordWithoutAKey(t *testing.T) {
stub := &discoveryManagerStub{}
rec := postDiscovery(t, stub, `{"catalog_provider_id":"openai_api","provider_id":"prov-42"}`)
require.Equal(t, http.StatusOK, rec.Code, "body: %s", rec.Body.String())
// The dashboard refreshes a saved provider's list without ever holding
// the credential, so the record id has to reach the manager.
assert.Equal(t, "prov-42", stub.gotRecordID)
assert.Empty(t, stub.gotReq.APIKey)
}
// TestDiscoverModelsRefusesMixedCredentials covers the case where a caller
// names a saved provider AND supplies a key. Accepting it would run an
// arbitrary credential under the identity of a record the caller may only be
// permitted to read.
func TestDiscoverModelsRefusesMixedCredentials(t *testing.T) {
stub := &discoveryManagerStub{}
rec := postDiscovery(t, stub, `{
"catalog_provider_id":"openai_api",
"provider_id":"prov-42",
"api_key":"sk-attacker"
}`)
assert.Equal(t, http.StatusBadRequest, rec.Code)
assert.Empty(t, stub.gotRecordID, "the request must be refused before it reaches the manager")
}
// TestDiscoverModelsReportsNoDiscoveryDistinctly matters because the caller
// falls back to the catalog's own model list on this outcome. Collapsing it
// into a generic 500 would turn "this provider has no listing endpoint" into
// "something went wrong", and the form would show an error instead of a list.
func TestDiscoverModelsReportsNoDiscoveryDistinctly(t *testing.T) {
stub := &discoveryManagerStub{err: modeldiscovery.ErrNoDiscovery}
rec := postDiscovery(t, stub, `{"catalog_provider_id":"litellm_proxy","upstream_url":"https://gw.example.com","api_key":"sk"}`)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code)
}
func TestDiscoverModelsRejectsMalformedRequests(t *testing.T) {
for name, body := range map[string]string{
"not json": `{`,
"no catalog provider": `{"api_key":"sk"}`,
"blank catalog provide": `{"catalog_provider_id":" ","api_key":"sk"}`,
} {
t.Run(name, func(t *testing.T) {
stub := &discoveryManagerStub{}
rec := postDiscovery(t, stub, body)
assert.Equal(t, http.StatusBadRequest, rec.Code)
})
}
}

View File

@@ -7,6 +7,7 @@ package handlers
import (
"encoding/json"
"errors"
"math"
"net/http"
"net/url"
@@ -16,6 +17,7 @@ import (
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/catalog"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/modeldiscovery"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/pricing"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
nbcontext "github.com/netbirdio/netbird/management/server/context"
@@ -32,6 +34,7 @@ type handler struct {
func RegisterEndpoints(manager agentnetwork.Manager, router *mux.Router) {
h := &handler{manager: manager}
router.HandleFunc("/agent-network/catalog/providers", h.getCatalogProviders).Methods("GET", "OPTIONS")
router.HandleFunc("/agent-network/catalog/providers/models", h.discoverProviderModels).Methods("POST", "OPTIONS")
router.HandleFunc("/agent-network/providers", h.getAllProviders).Methods("GET", "OPTIONS")
router.HandleFunc("/agent-network/providers", h.createProvider).Methods("POST", "OPTIONS")
router.HandleFunc("/agent-network/providers/{providerId}", h.getProvider).Methods("GET", "OPTIONS")
@@ -61,6 +64,73 @@ func (h *handler) getCatalogProviders(w http.ResponseWriter, r *http.Request) {
util.WriteJSONObject(r.Context(), w, out)
}
// discoverProviderModels asks the vendor which models the operator's own
// credential can reach, so the provider form can offer a live list rather than
// only the static catalog.
func (h *handler) discoverProviderModels(w http.ResponseWriter, r *http.Request) {
userAuth, err := nbcontext.GetUserAuthFromContext(r.Context())
if err != nil {
util.WriteError(r.Context(), err, w)
return
}
var body api.AgentNetworkModelDiscoveryRequest
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
util.WriteErrorResponse("invalid json", http.StatusBadRequest, w)
return
}
if strings.TrimSpace(body.CatalogProviderId) == "" {
util.WriteErrorResponse("catalog_provider_id is required", http.StatusBadRequest, w)
return
}
recordID := strValue(body.ProviderId)
req := modeldiscovery.Request{
CatalogID: body.CatalogProviderId,
UpstreamURL: strValue(body.UpstreamUrl),
APIKey: strValue(body.ApiKey),
}
// One source of credential or the other, never a mix: taking a key from
// the request while addressing a saved record would let a caller run an
// arbitrary credential against a provider they can only read.
if recordID != "" && req.APIKey != "" {
util.WriteErrorResponse("provide either provider_id or api_key, not both", http.StatusBadRequest, w)
return
}
models, err := h.manager.DiscoverProviderModels(r.Context(), userAuth.AccountId, userAuth.UserId, req, recordID)
if err != nil {
// A provider with no listing endpoint is a fact about the catalog
// entry, not a failure: the caller falls back to the catalog's own
// models, so it must be able to tell the two apart.
if errors.Is(err, modeldiscovery.ErrNoDiscovery) {
util.WriteErrorResponse(err.Error(), http.StatusUnprocessableEntity, w)
return
}
util.WriteError(r.Context(), err, w)
return
}
out := api.AgentNetworkModelDiscoveryResponse{Models: make([]api.AgentNetworkDiscoveredModel, 0, len(models))}
for _, m := range models {
entry := api.AgentNetworkDiscoveredModel{Id: m.ID, PricingKnown: m.PricingKnown}
if m.Label != "" {
label := m.Label
entry.Label = &label
}
out.Models = append(out.Models, entry)
}
util.WriteJSONObject(r.Context(), w, out)
}
// strValue reads an optional string field, treating absent as empty.
func strValue(v *string) string {
if v == nil {
return ""
}
return strings.TrimSpace(*v)
}
// applyDefaultPricing overwrites the catalog response's model rates with
// the LIVE default pricing table, which may differ from the compiled-in
// catalog rates when the operator provides a defaults_llm_pricing.yaml.

View File

@@ -13,6 +13,7 @@ import (
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/labelgen"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/modeldiscovery"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
"github.com/netbirdio/netbird/management/internals/modules/reverseproxy/proxy"
"github.com/netbirdio/netbird/management/internals/modules/reverseproxy/sessionkey"
@@ -50,6 +51,7 @@ type Manager interface {
CreateProvider(ctx context.Context, userID string, provider *types.Provider) (*types.Provider, error)
UpdateProvider(ctx context.Context, userID string, provider *types.Provider) (*types.Provider, error)
DeleteProvider(ctx context.Context, accountID, userID, providerID string) error
DiscoverProviderModels(ctx context.Context, accountID, userID string, req modeldiscovery.Request, recordID string) ([]modeldiscovery.Model, error)
GetAllPolicies(ctx context.Context, accountID, userID string) ([]*types.Policy, error)
GetPolicy(ctx context.Context, accountID, userID, policyID string) (*types.Policy, error)
@@ -123,6 +125,11 @@ type managerImpl struct {
permissionsManager permissions.Manager
proxyController proxy.Controller
// modelDiscovery queries vendors for the models a credential can reach.
// A field rather than a package call so tests can drive it without
// reaching the network.
modelDiscovery *modeldiscovery.Client
// reconcileCache holds the last set of synthesised proxy mappings
// per account, each paired with the proxy that served it, so a change
// of serving proxy can be diffed without re-deriving it.
@@ -151,6 +158,7 @@ func NewManager(
accountManager: accountManager,
permissionsManager: permissionsManager,
proxyController: proxyController,
modelDiscovery: &modeldiscovery.Client{},
reconcileCache: make(map[string]map[string]syntheticMapping),
labelRng: rand.New(rand.NewSource(time.Now().UnixNano())),
}
@@ -170,6 +178,37 @@ func (m *managerImpl) GetProvider(ctx context.Context, accountID, userID, provid
return m.store.GetAgentNetworkProviderByID(ctx, store.LockingStrengthNone, accountID, providerID)
}
// DiscoverProviderModels asks the vendor which models a credential can reach.
//
// recordID, when set, names an existing provider whose stored credential and
// upstream are used instead of the ones in req — so the dashboard can refresh
// the list without ever holding the key. Reading a stored credential is a read
// of that provider, and is permission-checked as one.
//
// Gated on Create rather than Read: this spends the operator's credential
// against a third party, which is not something a read-only role should be
// able to make the server do.
func (m *managerImpl) DiscoverProviderModels(ctx context.Context, accountID, userID string, req modeldiscovery.Request, recordID string) ([]modeldiscovery.Model, error) {
if err := m.requirePermission(ctx, accountID, userID, modules.AgentNetworkProviders, operations.Create); err != nil {
return nil, err
}
if recordID != "" {
record, err := m.store.GetAgentNetworkProviderByID(ctx, store.LockingStrengthNone, accountID, recordID)
if err != nil {
return nil, err
}
// The catalog id comes from the stored record too: letting the caller
// name a different one would run a provider's credential against
// whichever vendor endpoint they picked.
req.CatalogID = record.ProviderID
req.UpstreamURL = record.UpstreamURL
req.APIKey = record.APIKey
}
return m.modelDiscovery.Fetch(ctx, req)
}
// CreateProvider persists a new provider for the account. Providers have no
// settings side effects: the account's endpoint is bootstrapped separately and
// explicitly via CreateSettings, and every provider in the account routes
@@ -1017,6 +1056,10 @@ func (*mockManager) GetAllProviders(_ context.Context, _, _ string) ([]*types.Pr
return []*types.Provider{}, nil
}
func (*mockManager) DiscoverProviderModels(_ context.Context, _, _ string, _ modeldiscovery.Request, _ string) ([]modeldiscovery.Model, error) {
return nil, nil
}
func (*mockManager) GetProvider(_ context.Context, _, _, _ string) (*types.Provider, error) {
return &types.Provider{}, nil
}

View File

@@ -0,0 +1,358 @@
// Package modeldiscovery asks a vendor which models an operator's own
// credential can reach, so the provider form can offer a live list instead of
// only the catalog's hand-curated one.
//
// The catalog cannot know two things that matter. It goes stale — its entries
// carry comments tracking which models a vendor retired on which date — and it
// cannot see an account: which OpenAI models an org is entitled to, which
// Bedrock inference profiles a given account and region hold, which Vertex
// models a project has enabled. Those are exactly the facts an operator needs
// when filling in a provider record, and only the vendor has them.
//
// The vendor is authoritative for the model ID. The catalog remains
// authoritative for pricing, and a discovered model the catalog cannot price
// is reported as such rather than silently registered at a rate of zero.
package modeldiscovery
import (
"context"
"encoding/base64"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/netip"
"net/url"
"strings"
"time"
"golang.org/x/oauth2/google"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/catalog"
)
const (
// fetchTimeout bounds one vendor call end to end. A listing is a single
// small GET; anything slower is a vendor problem and the operator is
// waiting on a form.
fetchTimeout = 8 * time.Second
// maxListingBytes bounds the response we will buffer. The largest real
// listing observed is Bedrock's foundation-model catalogue at ~70KB, so
// this is a wide margin over anything legitimate.
maxListingBytes = 2 << 20
// gcpScope matches the scope llm_router mints Vertex tokens under, so a
// credential that works for discovery works for inference too.
gcpScope = "https://www.googleapis.com/auth/cloud-platform"
// vertexKeyfilePrefix marks an api_key that is a base64 service-account
// JSON key rather than a bearer token.
vertexKeyfilePrefix = "keyfile::"
)
// ErrNoDiscovery is returned for a catalog entry that declares no listing
// endpoint. Gateways vary too much to have one, and the caller should fall
// back to the catalog list plus free-text entry rather than treating this as
// a failure.
var ErrNoDiscovery = errors.New("provider has no model-discovery endpoint")
// Model is one discovered model.
type Model struct {
// ID is the identifier to register on the provider record, in the form the
// vendor issues it. For Bedrock that is the region-prefixed inference
// profile id, which is the only form AWS accepts at invoke time.
ID string
// Label is the vendor's display name where it supplies one.
Label string
// PricingKnown reports whether the shipped pricing table can price this
// model. False means the operator must set rates, or the request would
// meter at zero.
PricingKnown bool
}
// Request identifies which vendor to ask and with what credential.
type Request struct {
// CatalogID selects the catalog entry, which supplies the endpoint, the
// auth header and the response shape. The caller never supplies those.
CatalogID string
// UpstreamURL is the provider record's configured upstream. It is used
// only when the catalog entry declares no discovery host of its own.
UpstreamURL string
// Region substitutes the catalog host's <region> placeholder.
Region string
// APIKey is the operator's credential, exactly as stored on the record.
APIKey string
}
// Client fetches model listings. The zero value is usable; Resolver and
// HTTPClient exist so tests can drive it against a local server.
type Client struct {
HTTPClient *http.Client
// Resolver looks up the host for the SSRF check. Nil uses the default.
Resolver *net.Resolver
// AllowPrivateHosts disables the private-address guard. Only tests set it:
// their server is on loopback, which is precisely what the guard blocks.
AllowPrivateHosts bool
}
// Fetch returns the models the credential can reach.
func (c *Client) Fetch(ctx context.Context, req Request) ([]Model, error) {
entry, ok := catalog.Lookup(req.CatalogID)
if !ok {
return nil, fmt.Errorf("unknown catalog provider %q", req.CatalogID)
}
if entry.Discovery == nil {
return nil, ErrNoDiscovery
}
endpoint, err := c.discoveryURL(entry, req)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
defer cancel()
httpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return nil, fmt.Errorf("build discovery request: %w", err)
}
if err := applyAuth(httpReq, entry, req.APIKey); err != nil {
return nil, err
}
for name, value := range entry.Discovery.Headers {
httpReq.Header.Set(name, value)
}
httpReq.Header.Set("Accept", "application/json")
resp, err := c.httpClient().Do(httpReq)
if err != nil {
return nil, fmt.Errorf("reach %s: %w", entry.Name, err)
}
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, maxListingBytes))
if err != nil {
return nil, fmt.Errorf("read %s listing: %w", entry.Name, err)
}
if resp.StatusCode != http.StatusOK {
// Surface the vendor's own status. An operator whose key lacks a scope
// needs to see 403 rather than a generic failure.
return nil, fmt.Errorf("%s returned %d for its model listing", entry.Name, resp.StatusCode)
}
ids, err := parseListing(entry.Discovery.Shape, body)
if err != nil {
return nil, err
}
return decorate(entry, ids), nil
}
// discoveryURL builds the listing URL and refuses one that does not point at a
// public host.
//
// The path, query and (for Bedrock) the host all come from the catalog rather
// than from the caller, so the only operator-controlled part is the host of an
// entry whose listing lives on its own upstream. That still has to be checked:
// management holds credentials for every provider, and an upstream pointed at
// an internal address would turn this endpoint into a probe of the management
// server's own network.
func (c *Client) discoveryURL(entry catalog.Provider, req Request) (string, error) {
host := entry.Discovery.Host
if host == "" {
parsed, err := url.Parse(strings.TrimSpace(req.UpstreamURL))
if err != nil || parsed.Host == "" {
return "", fmt.Errorf("provider upstream %q is not a usable URL", req.UpstreamURL)
}
host = parsed.Host
}
if strings.Contains(host, catalog.RegionPlaceholder) {
region := strings.TrimSpace(req.Region)
if region == "" {
// A provider record carries no region field: the region lives
// inside the upstream host the operator already configured, so
// read it back out rather than asking them for it twice.
region = regionFromUpstream(entry, req.UpstreamURL)
}
if region == "" {
return "", fmt.Errorf("%s discovery needs a region, and none could be read from the provider upstream", entry.Name)
}
host = strings.ReplaceAll(host, catalog.RegionPlaceholder, region)
}
target := &url.URL{Scheme: "https", Host: host, Path: entry.Discovery.Path, RawQuery: entry.Discovery.Query}
if err := c.checkPublicHost(target.Hostname()); err != nil {
return "", err
}
return target.String(), nil
}
// regionFromUpstream recovers the region an operator embedded in the provider
// upstream, by matching it against the catalog's own host template. Bedrock's
// template is "bedrock-runtime.<region>.amazonaws.com" and Vertex's is
// "<region>-aiplatform.googleapis.com", so the region is whatever sits between
// the fixed halves. Returns empty when the upstream does not match the
// template, which is the case for a custom or proxied endpoint.
func regionFromUpstream(entry catalog.Provider, upstreamURL string) string {
prefix, suffix, found := strings.Cut(entry.DefaultHost, catalog.RegionPlaceholder)
if !found {
return ""
}
parsed, err := url.Parse(strings.TrimSpace(upstreamURL))
if err != nil {
return ""
}
host := parsed.Hostname()
if host == "" {
// A bare host with no scheme parses as a path, not a host.
host = strings.TrimSpace(upstreamURL)
}
if !strings.HasPrefix(host, prefix) || !strings.HasSuffix(host, suffix) {
return ""
}
region := host[len(prefix) : len(host)-len(suffix)]
if region == "" || strings.Contains(region, ".") {
return ""
}
return region
}
// checkPublicHost refuses hosts that resolve to an address the management
// server should never be asked to reach on an operator's behalf.
func (c *Client) checkPublicHost(host string) error {
if c.AllowPrivateHosts {
return nil
}
if host == "" {
return errors.New("discovery host is empty")
}
resolver := c.Resolver
if resolver == nil {
resolver = net.DefaultResolver
}
ctx, cancel := context.WithTimeout(context.Background(), fetchTimeout)
defer cancel()
addrs, err := resolver.LookupNetIP(ctx, "ip", host)
if err != nil {
return fmt.Errorf("resolve discovery host %q: %w", host, err)
}
// Every address must be public: a name that resolves to one public and one
// loopback address is still a way to reach loopback.
for _, addr := range addrs {
if !isPublic(addr) {
return fmt.Errorf("discovery host %q resolves to a non-public address", host)
}
}
return nil
}
// isPublic reports whether an address is one we are willing to dial.
func isPublic(addr netip.Addr) bool {
addr = addr.Unmap()
switch {
case !addr.IsValid(),
addr.IsLoopback(),
addr.IsPrivate(),
addr.IsLinkLocalUnicast(),
addr.IsLinkLocalMulticast(),
addr.IsInterfaceLocalMulticast(),
addr.IsMulticast(),
addr.IsUnspecified():
return false
}
// 100.64.0.0/10 (carrier NAT) is where NetBird's own overlay addresses
// live, so it is emphatically not somewhere to send a provider credential.
if addr.Is4() {
b := addr.As4()
if b[0] == 100 && b[1] >= 64 && b[1] <= 127 {
return false
}
}
return true
}
// applyAuth sets the credential header the catalog entry declares. A Vertex
// service-account key is exchanged for an OAuth token first, the same way the
// proxy does at request time.
func applyAuth(req *http.Request, entry catalog.Provider, apiKey string) error {
key := strings.TrimSpace(apiKey)
if key == "" {
return fmt.Errorf("%s discovery needs an API key", entry.Name)
}
if rest, ok := strings.CutPrefix(key, vertexKeyfilePrefix); ok {
token, err := mintGCPToken(req.Context(), rest)
if err != nil {
return err
}
key = token
}
name := entry.AuthHeaderName
if name == "" {
name = "Authorization"
}
template := entry.AuthHeaderTemplate
if template == "" {
template = "${API_KEY}"
}
req.Header.Set(name, strings.ReplaceAll(template, "${API_KEY}", key))
return nil
}
// mintGCPToken exchanges a base64 service-account key for an access token.
func mintGCPToken(ctx context.Context, saKeyB64 string) (string, error) {
jsonKey, err := base64.StdEncoding.DecodeString(strings.TrimSpace(saKeyB64))
if err != nil {
return "", fmt.Errorf("decode service-account key: %w", err)
}
conf, err := google.JWTConfigFromJSON(jsonKey, gcpScope)
if err != nil {
return "", fmt.Errorf("parse service-account key: %w", err)
}
tok, err := conf.TokenSource(ctx).Token()
if err != nil {
return "", fmt.Errorf("mint gcp token: %w", err)
}
return tok.AccessToken, nil
}
// decorate turns raw vendor ids into the models the caller renders, marking
// each with whether the shipped pricing table can price it.
func decorate(entry catalog.Provider, ids []listedModel) []Model {
priced := make(map[string]struct{}, len(entry.Models))
for _, m := range entry.Models {
priced[m.ID] = struct{}{}
}
out := make([]Model, 0, len(ids))
seen := make(map[string]struct{}, len(ids))
for _, listed := range ids {
if listed.id == "" {
continue
}
if _, dup := seen[listed.id]; dup {
continue
}
seen[listed.id] = struct{}{}
// The catalog keys pricing by the normalised id while the vendor
// issues the wire form, so normalise before asking whether we can
// price it — otherwise every Bedrock profile would report unpriced.
_, known := priced[normalizeForPricing(entry.ID, listed.id)]
out = append(out, Model{ID: listed.id, Label: listed.label, PricingKnown: known})
}
return out
}
func (c *Client) httpClient() *http.Client {
if c.HTTPClient != nil {
return c.HTTPClient
}
return &http.Client{
Timeout: fetchTimeout,
// A redirect is a way to move the request to a host the guard above
// never checked, so none are followed.
CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
},
}
}

View File

@@ -0,0 +1,321 @@
package modeldiscovery
import (
"context"
"io"
"net/http"
"net/netip"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/catalog"
)
// stubTransport answers every request with one canned response and records the
// request it was given, so a test can assert on the URL and headers the client
// built without a network round trip.
type stubTransport struct {
status int
body string
got *http.Request
}
func (s *stubTransport) RoundTrip(req *http.Request) (*http.Response, error) {
s.got = req
status := s.status
if status == 0 {
status = http.StatusOK
}
return &http.Response{
StatusCode: status,
Body: io.NopCloser(strings.NewReader(s.body)),
Header: http.Header{"Content-Type": []string{"application/json"}},
Request: req,
}, nil
}
// newStubClient returns a client that never leaves the process. The host guard
// is disabled because it would otherwise resolve the vendor's real name, which
// would make these tests depend on DNS.
func newStubClient(status int, body string) (*Client, *stubTransport) {
tr := &stubTransport{status: status, body: body}
return &Client{
HTTPClient: &http.Client{Transport: tr},
AllowPrivateHosts: true,
}, tr
}
// The payloads below are trimmed from what the vendors actually returned in
// the discovery e2e, rather than invented, so a parser that only works against
// an idealised shape fails here.
const openAIListing = `{"object":"list","data":[
{"id":"gpt-4o-mini","object":"model","created":1721172741,"owned_by":"system"},
{"id":"gpt-4o","object":"model","created":1715367049,"owned_by":"system"}
]}`
const anthropicListing = `{"data":[
{"type":"model","id":"claude-haiku-4-5-20251001","display_name":"Claude Haiku 4.5"},
{"type":"model","id":"claude-sonnet-4-6","display_name":"Claude Sonnet 4.6"}
],"has_more":false}`
const bedrockListing = `{"inferenceProfileSummaries":[
{"inferenceProfileId":"eu.anthropic.claude-haiku-4-5-20251001-v1:0",
"inferenceProfileName":"EU Anthropic Claude Haiku 4.5","status":"ACTIVE","type":"SYSTEM_DEFINED"},
{"inferenceProfileId":"global.cohere.embed-v4:0",
"inferenceProfileName":"Global Cohere Embed v4","status":"ACTIVE","type":"SYSTEM_DEFINED"},
{"inferenceProfileId":"eu.meta.llama3-2-1b-instruct-v1:0",
"inferenceProfileName":"EU Meta Llama 3.2 1B","status":"INACTIVE","type":"SYSTEM_DEFINED"}
]}`
const vertexListing = `{"publisherModels":[
{"name":"publishers/anthropic/models/claude-3-opus","versionId":"20240229","launchStage":"GA"},
{"name":"publishers/anthropic/models/claude-sonnet-4-5","versionId":"20250929","launchStage":"GA"}
]}`
func TestFetchOpenAIListing(t *testing.T) {
cl, tr := newStubClient(http.StatusOK, openAIListing)
models, err := cl.Fetch(context.Background(), Request{
CatalogID: "openai_api",
UpstreamURL: "https://api.openai.com",
APIKey: "sk-test",
})
require.NoError(t, err)
assert.Equal(t, "https://api.openai.com/v1/models", tr.got.URL.String())
assert.Equal(t, "Bearer sk-test", tr.got.Header.Get("Authorization"),
"the credential must be injected through the catalog's auth template")
assert.Equal(t, []string{"gpt-4o-mini", "gpt-4o"}, ids(models))
for _, m := range models {
assert.True(t, m.PricingKnown, "both models are in the shipped catalog: %s", m.ID)
}
}
func TestFetchAnthropicSendsTheVersionHeader(t *testing.T) {
cl, tr := newStubClient(http.StatusOK, anthropicListing)
models, err := cl.Fetch(context.Background(), Request{
CatalogID: "anthropic_api",
UpstreamURL: "https://api.anthropic.com",
APIKey: "sk-ant-test",
})
require.NoError(t, err)
// Anthropic rejects a request without the version header, so a listing
// that reached us at all proves it was sent — but assert it, because the
// failure mode otherwise only shows up against the live API.
assert.Equal(t, "2023-06-01", tr.got.Header.Get("anthropic-version"))
assert.Equal(t, "sk-ant-test", tr.got.Header.Get("x-api-key"),
"Anthropic takes a bare key under its own header, not a Bearer token")
assert.Equal(t, "limit=1000", tr.got.URL.RawQuery)
assert.Equal(t, []string{"claude-haiku-4-5-20251001", "claude-sonnet-4-6"}, ids(models))
assert.Equal(t, "Claude Haiku 4.5", models[0].Label)
}
func TestFetchBedrockUsesTheControlPlaneAndKeepsWireIDs(t *testing.T) {
cl, tr := newStubClient(http.StatusOK, bedrockListing)
models, err := cl.Fetch(context.Background(), Request{
CatalogID: "bedrock_api",
// The record's upstream is the RUNTIME host, which does not serve
// listings. The catalog's own discovery host must win over it.
UpstreamURL: "https://bedrock-runtime.eu-central-1.amazonaws.com",
Region: "eu-central-1",
APIKey: "aws-bearer",
})
require.NoError(t, err)
assert.Equal(t, "https://bedrock.eu-central-1.amazonaws.com/inference-profiles",
tr.got.URL.String(), "listings come from the control plane, not the runtime host")
// Region-prefixed ids verbatim: the prefix is what makes them invocable
// and it cannot be reconstructed — global.* alongside eu.* is exactly the
// case that defeats deriving it from the configured region.
assert.Equal(t, []string{
"eu.anthropic.claude-haiku-4-5-20251001-v1:0",
"global.cohere.embed-v4:0",
}, ids(models), "an INACTIVE profile must not be offered")
assert.True(t, models[0].PricingKnown,
"the catalog prices anthropic.claude-haiku-4-5, which this id normalises to")
assert.False(t, models[1].PricingKnown,
"cohere embed is not in the shipped Bedrock catalog, so the operator must price it")
}
func TestFetchVertexJoinsNameAndVersion(t *testing.T) {
cl, _ := newStubClient(http.StatusOK, vertexListing)
models, err := cl.Fetch(context.Background(), Request{
CatalogID: "vertex_ai_api",
UpstreamURL: "https://us-east5-aiplatform.googleapis.com",
Region: "us-east5",
APIKey: "ya29.test-token",
})
require.NoError(t, err)
// Vertex addresses a model as "<id>@<version>" on rawPredict, and splits
// those across two fields in the listing.
assert.Equal(t, []string{"claude-3-opus@20240229", "claude-sonnet-4-5@20250929"}, ids(models))
assert.Equal(t, "claude-3-opus", models[0].Label)
}
func TestFetchSurfacesTheVendorStatus(t *testing.T) {
cl, _ := newStubClient(http.StatusForbidden, `{"error":{"message":"no access"}}`)
_, err := cl.Fetch(context.Background(), Request{
CatalogID: "openai_api",
UpstreamURL: "https://api.openai.com",
APIKey: "sk-test",
})
require.Error(t, err)
assert.Contains(t, err.Error(), "403",
"an operator whose key lacks access needs to see which status the vendor returned")
}
func TestFetchRejectsAProviderWithoutDiscovery(t *testing.T) {
cl, _ := newStubClient(http.StatusOK, openAIListing)
_, err := cl.Fetch(context.Background(), Request{
CatalogID: "litellm_proxy",
UpstreamURL: "https://gateway.example.com",
APIKey: "sk-test",
})
assert.ErrorIs(t, err, ErrNoDiscovery,
"a gateway with no listing endpoint must be distinguishable from a failure, so the caller can fall back")
}
func TestFetchRequiresACredential(t *testing.T) {
cl, _ := newStubClient(http.StatusOK, openAIListing)
_, err := cl.Fetch(context.Background(), Request{
CatalogID: "openai_api",
UpstreamURL: "https://api.openai.com",
})
require.Error(t, err)
assert.Contains(t, err.Error(), "API key")
}
func TestDiscoveryURLNeedsARegionWhenTheHostTemplatesOne(t *testing.T) {
cl, _ := newStubClient(http.StatusOK, bedrockListing)
// An upstream that matches no catalog template — a proxy in front of
// Bedrock, say — leaves nothing to read the region from. Refusing beats
// guessing: an unsubstituted placeholder would dial a host that does not
// exist, and a guessed region would dial the wrong account's endpoint.
_, err := cl.Fetch(context.Background(), Request{
CatalogID: "bedrock_api",
UpstreamURL: "https://bedrock.internal-proxy.example.com",
APIKey: "aws-bearer",
})
require.Error(t, err)
assert.Contains(t, err.Error(), "region")
}
// TestHostGuardRejectsNonPublicAddresses is the SSRF guard. Management holds a
// credential for every provider, so an upstream pointed at an internal address
// would turn discovery into a way to probe — and hand a token to — the
// management server's own network.
func TestHostGuardRejectsNonPublicAddresses(t *testing.T) {
for _, tc := range []struct {
name string
addr string
want bool
}{
{"loopback v4", "127.0.0.1", false},
{"loopback v6", "::1", false},
{"private 10/8", "10.0.0.5", false},
{"private 172.16/12", "172.16.4.1", false},
{"private 192.168/16", "192.168.1.1", false},
{"link-local", "169.254.169.254", false}, // cloud metadata
{"unspecified", "0.0.0.0", false},
{"multicast", "224.0.0.1", false},
{"netbird overlay 100.64/10", "100.90.1.2", false},
{"v4-mapped loopback", "::ffff:127.0.0.1", false},
{"public v4", "1.1.1.1", true},
{"public v6", "2606:4700:4700::1111", true},
{"just outside CGNAT", "100.128.0.1", true},
} {
t.Run(tc.name, func(t *testing.T) {
addr, err := netip.ParseAddr(tc.addr)
require.NoError(t, err)
assert.Equal(t, tc.want, isPublic(addr))
})
}
}
func TestHostGuardResolvesAndRejectsLocalhost(t *testing.T) {
cl := &Client{}
err := cl.checkPublicHost("localhost")
require.Error(t, err, "a name resolving to loopback must be refused, not just a literal address")
assert.Contains(t, err.Error(), "non-public")
}
// TestEveryDiscoveryEntryHasAParser keeps the catalog and the parser table from
// drifting: adding a Discovery block with a shape nothing parses would fail
// only at runtime, in front of an operator.
func TestEveryDiscoveryEntryHasAParser(t *testing.T) {
for _, entry := range catalog.All() {
if entry.Discovery == nil {
continue
}
t.Run(entry.ID, func(t *testing.T) {
assert.NotEmpty(t, entry.Discovery.Path, "a discovery entry needs a path")
_, err := parseListing(entry.Discovery.Shape, []byte(`{}`))
assert.NoError(t, err, "shape %q has no parser", entry.Discovery.Shape)
})
}
}
func ids(models []Model) []string {
out := make([]string, 0, len(models))
for _, m := range models {
out = append(out, m.ID)
}
return out
}
// TestRegionIsReadBackFromTheUpstream covers the reason the API takes no
// region field: a provider record has none, and the operator already encoded
// it in the upstream host when they configured inference.
func TestRegionIsReadBackFromTheUpstream(t *testing.T) {
cl, tr := newStubClient(http.StatusOK, bedrockListing)
_, err := cl.Fetch(context.Background(), Request{
CatalogID: "bedrock_api",
UpstreamURL: "https://bedrock-runtime.us-west-2.amazonaws.com",
APIKey: "aws-bearer",
})
require.NoError(t, err)
assert.Equal(t, "bedrock.us-west-2.amazonaws.com", tr.got.URL.Host)
}
func TestRegionFromUpstream(t *testing.T) {
bedrock, ok := catalog.Lookup("bedrock_api")
require.True(t, ok)
vertex, ok := catalog.Lookup("vertex_ai_api")
require.True(t, ok)
for _, tc := range []struct {
name string
entry catalog.Provider
upstream string
want string
}{
{"bedrock runtime host", bedrock, "https://bedrock-runtime.eu-central-1.amazonaws.com", "eu-central-1"},
{"bedrock without scheme", bedrock, "bedrock-runtime.ap-south-1.amazonaws.com", "ap-south-1"},
{"vertex regional host", vertex, "https://us-east5-aiplatform.googleapis.com", "us-east5"},
// A proxied or self-hosted upstream matches no template, and guessing
// a region from it would build a URL pointing somewhere arbitrary.
{"unrelated upstream", bedrock, "https://llm.internal.example.com", ""},
{"vertex global host has no region segment", vertex, "https://aiplatform.googleapis.com", ""},
} {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, regionFromUpstream(tc.entry, tc.upstream))
})
}
}

View File

@@ -0,0 +1,134 @@
package modeldiscovery
import (
"encoding/json"
"fmt"
"strings"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/catalog"
sharedllm "github.com/netbirdio/netbird/shared/llm"
)
// listedModel is one entry lifted out of a vendor listing before the catalog
// is consulted about it.
type listedModel struct {
id string
label string
}
// parseListing extracts model ids from a vendor listing. Each vendor invented
// its own envelope, and the shape is declared by the catalog rather than
// sniffed, so a vendor that changes shape fails loudly instead of silently
// returning nothing.
func parseListing(shape catalog.ListingShape, body []byte) ([]listedModel, error) {
switch shape {
case catalog.ShapeOpenAIData:
return parseOpenAIData(body)
case catalog.ShapeBedrockInferenceProfiles:
return parseBedrockInferenceProfiles(body)
case catalog.ShapeVertexPublisherModels:
return parseVertexPublisherModels(body)
default:
return nil, fmt.Errorf("no parser for listing shape %q", shape)
}
}
// parseOpenAIData reads {"data":[{"id":…}]}, which OpenAI defined and
// Anthropic adopted. Anthropic additionally supplies display_name.
func parseOpenAIData(body []byte) ([]listedModel, error) {
var doc struct {
Data []struct {
ID string `json:"id"`
DisplayName string `json:"display_name"`
} `json:"data"`
}
if err := json.Unmarshal(body, &doc); err != nil {
return nil, fmt.Errorf("decode model listing: %w", err)
}
out := make([]listedModel, 0, len(doc.Data))
for _, entry := range doc.Data {
out = append(out, listedModel{id: entry.ID, label: entry.DisplayName})
}
return out, nil
}
// parseBedrockInferenceProfiles reads
// {"inferenceProfileSummaries":[{"inferenceProfileId":…}]}.
//
// The profile id is taken verbatim because its region prefix (eu., us.,
// global.) is what makes it invocable, and it is not derivable from the
// configured region — an account in one region legitimately holds global.*
// profiles alongside its regional ones.
//
// Only ACTIVE profiles are offered: AWS reports others, and registering one
// would produce a model that routes inside NetBird and fails at AWS.
func parseBedrockInferenceProfiles(body []byte) ([]listedModel, error) {
var doc struct {
Summaries []struct {
ID string `json:"inferenceProfileId"`
Name string `json:"inferenceProfileName"`
Status string `json:"status"`
} `json:"inferenceProfileSummaries"`
}
if err := json.Unmarshal(body, &doc); err != nil {
return nil, fmt.Errorf("decode inference-profile listing: %w", err)
}
out := make([]listedModel, 0, len(doc.Summaries))
for _, entry := range doc.Summaries {
if entry.Status != "" && !strings.EqualFold(entry.Status, "ACTIVE") {
continue
}
out = append(out, listedModel{id: entry.ID, label: entry.Name})
}
return out, nil
}
// parseVertexPublisherModels reads {"publisherModels":[{"name":…}]}, where
// name is a resource path ("publishers/anthropic/models/claude-3-opus") and
// the version lives in a separate field.
//
// Vertex addresses a model as "<id>@<version>" on the rawPredict path, so the
// two are joined here: reporting the bare name would hand the operator an id
// that looks usable and is not.
func parseVertexPublisherModels(body []byte) ([]listedModel, error) {
var doc struct {
Models []struct {
Name string `json:"name"`
VersionID string `json:"versionId"`
} `json:"publisherModels"`
}
if err := json.Unmarshal(body, &doc); err != nil {
return nil, fmt.Errorf("decode publisher-model listing: %w", err)
}
out := make([]listedModel, 0, len(doc.Models))
for _, entry := range doc.Models {
id := entry.Name
if slash := strings.LastIndex(id, "/"); slash >= 0 {
id = id[slash+1:]
}
if id == "" {
continue
}
label := id
if entry.VersionID != "" {
id += "@" + entry.VersionID
}
out = append(out, listedModel{id: id, label: label})
}
return out, nil
}
// normalizeForPricing maps a vendor's wire id onto the key the catalog prices
// it under. It mirrors the synthesiser's normalizePricingModelID: the two must
// agree, or a model reported here as priced would meter at the default rate
// instead of the operator's.
func normalizeForPricing(catalogProviderID, modelID string) string {
switch {
case catalog.IsBedrockPathStyle(catalogProviderID):
return sharedllm.NormalizeBedrockModel(modelID)
case catalog.IsVertexPathStyle(catalogProviderID):
return sharedllm.NormalizeVertexModel(modelID)
default:
return modelID
}
}

View File

@@ -47,17 +47,11 @@ var supplementalDefaults = map[string]map[string]Entry{
"gpt-5-nano": {InputPer1k: 0.00005, OutputPer1k: 0.0004, CachedInputPer1k: 0.000005},
},
"anthropic": {
// claude-opus-5 is not yet in the catalog lineup but gateway /
// grandfathered traffic uses it; priced so it isn't skipped.
"claude-opus-5": {InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625},
// "kimi-k3[1m]" is the 1M-context alias some Claude Code guides
// configure against Moonshot's Anthropic-compatible endpoint;
// priced identically to kimi-k3 so those requests aren't skipped.
"kimi-k3[1m]": {InputPer1k: 0.003, OutputPer1k: 0.015, CacheReadPer1k: 0.0003},
},
"bedrock": {
"anthropic.claude-opus-5": {InputPer1k: 0.005, OutputPer1k: 0.025, CacheReadPer1k: 0.0005, CacheCreationPer1k: 0.00625},
},
}
var (

View File

@@ -82,6 +82,11 @@ anthropic:
output_per_1k: 0.015
cache_read_per_1k: 0.0003
cache_creation_per_1k: 0.00375
claude-sonnet-5:
input_per_1k: 0.003
output_per_1k: 0.015
cache_read_per_1k: 0.0003
cache_creation_per_1k: 0.00375
kimi-k3:
input_per_1k: 0.003
output_per_1k: 0.015
@@ -145,6 +150,11 @@ bedrock:
output_per_1k: 0.015
cache_read_per_1k: 0.0003
cache_creation_per_1k: 0.00375
anthropic.claude-sonnet-5:
input_per_1k: 0.003
output_per_1k: 0.015
cache_read_per_1k: 0.0003
cache_creation_per_1k: 0.00375
meta.llama3-3-70b-instruct:
input_per_1k: 0.00072
output_per_1k: 0.00072

View File

@@ -116,11 +116,13 @@ func TestDefaultTable_PinnedRates(t *testing.T) {
assert.InDelta(t, 0.010, fable.InputPer1k, 1e-9, "claude-fable-5 input")
assert.InDelta(t, 0.0125, fable.CacheCreationPer1k, 1e-9, "claude-fable-5 cache creation")
// Supplementals present on their surfaces.
// Every id below must stay priced whichever source provides it: the
// catalog lineup for the current Claude 5 family, supplementalDefaults
// for the ids the dashboard deliberately doesn't offer.
for surface, ids := range map[string][]string{
"openai": {"gpt-5", "gpt-5-mini", "gpt-5-nano"},
"anthropic": {"claude-opus-5", "kimi-k3[1m]", "kimi-k3"},
"bedrock": {"anthropic.claude-opus-5"},
"anthropic": {"claude-opus-5", "claude-sonnet-5", "kimi-k3[1m]", "kimi-k3"},
"bedrock": {"anthropic.claude-opus-5", "anthropic.claude-sonnet-5"},
} {
for _, id := range ids {
_, ok := table[surface][id]

View File

@@ -211,7 +211,19 @@ func SynthesizeServices(ctx context.Context, s store.Store, accountID string) ([
groupIndex := indexProviderGroups(enabledPolicies)
routerCfgJSON, err := buildRouterConfigJSON(enabledProviders, groupIndex)
// The proxy guardrail is a per-provider fail-closed backstop; the
// authoritative per-policy/group decision is management's
// SelectPolicyForRequest. A provider lands in that map only when every
// authorising policy restricts models.
providerAllowlists := buildProviderAllowlists(enabledPolicies, guardrailsByID)
// Discovery gets the finer view: per policy rather than flattened per
// provider, so a listing can be bounded to what the calling groups may
// actually use instead of the union across everyone who reaches the
// provider.
modelPolicies := buildModelPolicies(enabledPolicies, guardrailsByID)
routerCfgJSON, err := buildRouterConfigJSON(enabledProviders, groupIndex, modelPolicies)
if err != nil {
return nil, err
}
@@ -228,11 +240,6 @@ func SynthesizeServices(ctx context.Context, s store.Store, accountID string) ([
mergedGuardrails := mergeGuardrails(enabledPolicies, guardrailsByID)
applyAccountCollectionControls(&mergedGuardrails, settings)
// The proxy guardrail is a per-provider fail-closed backstop; the
// authoritative per-policy/group decision is management's
// SelectPolicyForRequest. A provider lands in this map only when every
// authorising policy restricts models.
providerAllowlists := buildProviderAllowlists(enabledPolicies, guardrailsByID)
guardrailJSON, err := marshalGuardrailConfig(providerAllowlists, mergedGuardrails.PromptCapture)
if err != nil {
return nil, err
@@ -351,6 +358,11 @@ type routerProviderRoute struct {
AuthHeaderName string `json:"auth_header_name"`
AuthHeaderValue string `json:"auth_header_value"`
AllowedGroupIDs []string `json:"allowed_group_ids,omitempty"`
// ModelPolicies is one entry per enabled policy authorising this provider,
// carrying that policy's source groups and the models it permits. The
// router bounds a model listing with it, so a provider two groups reach
// under different allowlists offers each only its own.
ModelPolicies []routerModelPolicy `json:"model_policies,omitempty"`
// Vertex marks a Google Vertex AI provider, whose requests carry the
// model in the URL path. The router selects it by path, bypassing the
// model/vendor table.
@@ -422,7 +434,7 @@ func indexProviderGroups(policies []*types.Policy) map[string][]string {
// path-prefix tiebreak. Providers no enabled policy authorises
// (orphans) are intentionally OMITTED so the router never observes a
// route with an empty ACL.
func buildRouterConfigJSON(providers []*types.Provider, groupIndex map[string][]string) ([]byte, error) {
func buildRouterConfigJSON(providers []*types.Provider, groupIndex map[string][]string, modelPolicies map[string][]routerModelPolicy) ([]byte, error) {
cfg := routerConfig{Providers: make([]routerProviderRoute, 0, len(providers))}
for _, p := range providers {
groups, hasPolicy := groupIndex[p.ID]
@@ -449,6 +461,7 @@ func buildRouterConfigJSON(providers []*types.Provider, groupIndex map[string][]
AuthHeaderName: headerName,
AuthHeaderValue: headerValue,
AllowedGroupIDs: groups,
ModelPolicies: modelPolicies[p.ID],
Vertex: catalog.IsVertexPathStyle(p.ProviderID),
Bedrock: catalog.IsBedrockPathStyle(p.ProviderID),
GCPServiceAccountKeyB64: gcpSAKeyB64,
@@ -1098,3 +1111,46 @@ func mergeGuardrail(g *types.Guardrail, merged *MergedGuardrails) {
}
}
}
// routerModelPolicy mirrors the router's ModelPolicyRule: one authorising
// policy's source groups plus the models it permits. Models is nil for a
// policy that sets no model allowlist, which lifts the restriction for the
// groups it binds — so nil and empty must survive the round trip distinctly.
type routerModelPolicy struct {
GroupIDs []string `json:"group_ids"`
Models []string `json:"models"`
}
// buildModelPolicies indexes, per provider, one rule for each enabled policy
// authorising it: the policy's source groups and the models its guardrail
// permits.
//
// This is deliberately finer than buildProviderAllowlists, which flattens the
// same inputs into one list per provider for the proxy's fail-closed guardrail.
// A flattened list cannot answer "what may THIS caller see", so a provider two
// teams reach under different allowlists would offer each team the other's
// models — a picker full of entries the next request refuses. Keeping the
// source groups alongside the models lets the router answer it at request time,
// where it knows the caller's groups.
func buildModelPolicies(policies []*types.Policy, byID map[string]*types.Guardrail) map[string][]routerModelPolicy {
out := make(map[string][]routerModelPolicy)
for _, p := range policies {
if p == nil || len(p.SourceGroups) == 0 {
continue
}
restricted, models := policyModelAllowlist(p, byID)
rule := routerModelPolicy{GroupIDs: append([]string(nil), p.SourceGroups...)}
if restricted {
// Never nil when restricted: an allowlist permitting nothing must
// stay distinguishable from no allowlist at all.
rule.Models = append([]string{}, models...)
}
for _, providerID := range p.DestinationProviderIDs {
if providerID == "" {
continue
}
out[providerID] = append(out[providerID], rule)
}
}
return out
}

View File

@@ -4,6 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
)
@@ -93,3 +94,75 @@ func TestBuildProviderAllowlists(t *testing.T) {
"an enabled-but-empty allowlist is restricted with an empty set, not unrestricted")
})
}
// policyForGroups builds an enabled policy binding the given source groups to
// the given providers under an optional guardrail.
func policyForGroups(id string, groups []string, guardrailIDs []string, providerIDs ...string) *types.Policy {
return &types.Policy{
ID: id,
Enabled: true,
SourceGroups: groups,
DestinationProviderIDs: providerIDs,
GuardrailIDs: guardrailIDs,
}
}
// TestBuildModelPolicies covers the finer index discovery needs. Where
// buildProviderAllowlists flattens every authorising policy into one list per
// provider — enough for a fail-closed backstop, but blind to who is asking —
// this keeps each policy's source groups beside its models so the router can
// bound a listing to the calling groups.
func TestBuildModelPolicies(t *testing.T) {
byID := map[string]*types.Guardrail{
"g-4o": allowlistGuardrail("g-4o", "acc-1", "gpt-4o"),
"g-opus": allowlistGuardrail("g-opus", "acc-1", "claude-opus-4"),
"g-disabled": {ID: "g-disabled", Checks: types.GuardrailChecks{ModelAllowlist: types.GuardrailModelAllowlist{Enabled: false, Models: []string{"gpt-4o"}}}},
}
t.Run("each policy keeps its own groups and models", func(t *testing.T) {
policies := []*types.Policy{
policyForGroups("p1", []string{"grp-eng"}, []string{"g-4o"}, "prov-x"),
policyForGroups("p2", []string{"grp-sales"}, []string{"g-opus"}, "prov-x"),
}
got := buildModelPolicies(policies, byID)
assert.Equal(t, []routerModelPolicy{
{GroupIDs: []string{"grp-eng"}, Models: []string{"gpt-4o"}},
{GroupIDs: []string{"grp-sales"}, Models: []string{"claude-opus-4"}},
}, got["prov-x"],
"the two policies must stay separable so neither group is offered the other's models")
})
t.Run("an unrestricted policy carries nil models", func(t *testing.T) {
policies := []*types.Policy{
policyForGroups("p1", []string{"grp-eng"}, []string{"g-4o"}, "prov-x"),
policyForGroups("p2", []string{"grp-admin"}, nil, "prov-x"),
}
got := buildModelPolicies(policies, byID)
assert.Nil(t, got["prov-x"][1].Models,
"no allowlist must reach the router as nil, which lifts the restriction for its groups")
})
t.Run("a disabled allowlist is not a restriction", func(t *testing.T) {
policies := []*types.Policy{policyForGroups("p1", []string{"grp-eng"}, []string{"g-disabled"}, "prov-x")}
got := buildModelPolicies(policies, byID)
assert.Nil(t, got["prov-x"][0].Models,
"a guardrail with the allowlist check off restricts nothing")
})
t.Run("an enabled allowlist with no models permits nothing", func(t *testing.T) {
byIDEmpty := map[string]*types.Guardrail{
"g-empty": {ID: "g-empty", Checks: types.GuardrailChecks{ModelAllowlist: types.GuardrailModelAllowlist{Enabled: true}}},
}
policies := []*types.Policy{policyForGroups("p1", []string{"grp-eng"}, []string{"g-empty"}, "prov-x")}
got := buildModelPolicies(policies, byIDEmpty)
require.NotNil(t, got["prov-x"][0].Models,
"an empty allowlist must not arrive as nil — that would read as unrestricted")
assert.Empty(t, got["prov-x"][0].Models)
})
t.Run("a policy binding no groups is skipped", func(t *testing.T) {
policies := []*types.Policy{policyForGroups("p1", nil, []string{"g-4o"}, "prov-x")}
assert.Empty(t, buildModelPolicies(policies, byID),
"a policy with no source groups authorises nobody, so it bounds nobody's listing")
})
}

View File

@@ -13,6 +13,14 @@ func NormalizeBedrockModel(modelID string) string {
return sharedllm.NormalizeBedrockModel(modelID)
}
// NormalizeAnthropicModel strips the trailing "-YYYYMMDD" release-date suffix
// from an Anthropic model id so a dated id a client pins matches the undated
// one the operator registered. Thin delegate to shared/llm for the same
// contract reason as the two below.
func NormalizeAnthropicModel(modelID string) string {
return sharedllm.NormalizeAnthropicModel(modelID)
}
// NormalizeVertexModel strips the "@version" suffix from a Vertex AI model id
// so it matches the catalog/pricing key. Thin delegate to shared/llm, kept
// beside NormalizeBedrockModel for the same contract reason.

View File

@@ -10,6 +10,8 @@ package pricing
import (
"fmt"
"math"
sharedllm "github.com/netbirdio/netbird/shared/llm"
)
// Entry is a single model's input and output pricing, expressed in USD per
@@ -92,7 +94,10 @@ func NewTable(raw map[string]map[string]EntryJSON) (*Table, error) {
return &Table{entries: entries}, nil
}
// Lookup returns the entry for the given provider surface and model.
// Lookup returns the entry for the given provider surface and model. A
// dated Anthropic id falls back to its undated form, so a client pinning
// "claude-sonnet-4-5-20250929" bills at the registered "claude-sonnet-4-5"
// rate instead of recording no cost at all.
func (t *Table) Lookup(provider, model string) (Entry, bool) {
if t == nil {
return Entry{}, false
@@ -101,7 +106,14 @@ func (t *Table) Lookup(provider, model string) (Entry, bool) {
if !ok {
return Entry{}, false
}
e, ok := byModel[model]
if e, found := byModel[model]; found {
return e, true
}
undated := sharedllm.NormalizeAnthropicModel(model)
if undated == model {
return Entry{}, false
}
e, ok := byModel[undated]
return e, ok
}

View File

@@ -175,3 +175,22 @@ func TestNewTable_NilAndEmpty(t *testing.T) {
require.NoError(t, err)
assert.Empty(t, entries, "nil in, empty (never-matching) map out for the per-record map")
}
// TestLookup_DatedAnthropicIDFallsBackToUndated covers a client pinning a
// release date on a model priced under its undated id. Without the
// fallback the request records no cost at all.
func TestLookup_DatedAnthropicIDFallsBackToUndated(t *testing.T) {
table, err := NewTable(map[string]map[string]EntryJSON{
"anthropic": {
"claude-sonnet-4-5": {InputPer1K: 0.003, OutputPer1K: 0.015},
},
})
require.NoError(t, err, "table must build from a valid defaults map")
entry, ok := table.Lookup("anthropic", "claude-sonnet-4-5-20250929")
require.True(t, ok, "a dated id must resolve to the undated entry")
assert.InDelta(t, 0.003, entry.InputPer1K, 1e-9, "dated id must bill at the registered rate")
_, ok = table.Lookup("anthropic", "claude-sonnet-9-9-20250929")
assert.False(t, ok, "an unknown family must stay unpriced")
}

View File

@@ -11,6 +11,7 @@ import (
"fmt"
"strconv"
"github.com/netbirdio/netbird/proxy/internal/llm"
"github.com/netbirdio/netbird/proxy/internal/llm/pricing"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
@@ -175,13 +176,28 @@ func (m *Middleware) Invoke(_ context.Context, in *middleware.Input) (*middlewar
// Anthropic route still bills its cache buckets additively.
func (m *Middleware) lookupCosts(md []middleware.KV, surface, model string, inTokens, outTokens, cachedTokens, cacheCreationTokens int64) (pricing.Costs, bool) {
if recordID := lookupKV(md, middleware.KeyLLMResolvedProviderID); recordID != "" {
if entry, ok := m.perRecord[recordID][model]; ok {
if entry, ok := perRecordEntry(m.perRecord[recordID], model); ok {
return pricing.EntryCosts(entry, surface, inTokens, outTokens, cachedTokens, cacheCreationTokens), true
}
}
return m.defaults.Costs(surface, model, inTokens, outTokens, cachedTokens, cacheCreationTokens)
}
// perRecordEntry resolves the operator's stored price for a model on one
// provider record, falling back to the undated form of a dated Anthropic id
// so a client that pins a release date still bills at the registered rate.
func perRecordEntry(byModel map[string]pricing.Entry, model string) (pricing.Entry, bool) {
if entry, ok := byModel[model]; ok {
return entry, true
}
undated := llm.NormalizeAnthropicModel(model)
if undated == model {
return pricing.Entry{}, false
}
entry, ok := byModel[undated]
return entry, ok
}
// usd renders a cost as the fixed-precision string every cost.usd_* key
// carries, so the per-bucket values and the aggregates round identically.
//

View File

@@ -84,8 +84,10 @@ func (m *Middleware) MutationsSupported() bool { return false }
func (m *Middleware) Invoke(_ context.Context, in *middleware.Input) (*middleware.Output, error) {
model, modelPresent := lookupMetadata(in.Metadata, middleware.KeyLLMModel)
providerID, _ := lookupMetadata(in.Metadata, middleware.KeyLLMResolvedProviderID)
surface, _ := lookupMetadata(in.Metadata, middleware.KeyLLMProvider)
nonInference, _ := lookupMetadata(in.Metadata, middleware.KeyLLMNonInference)
if denial := m.evaluateAllowlist(providerID, model, modelPresent); denial != nil {
if denial := m.evaluateAllowlist(providerID, surface, model, modelPresent, nonInference == "true"); denial != nil {
return denial, nil
}
@@ -114,7 +116,7 @@ func (m *Middleware) Close() error { return nil }
// evaluateAllowlist denies when the resolved provider's allowlist rejects the
// model; nil means proceed. Scoped to the provider llm_router resolved, so an
// unrestricted provider (absent from config) is never caught by another's list.
func (m *Middleware) evaluateAllowlist(providerID, model string, modelPresent bool) *middleware.Output {
func (m *Middleware) evaluateAllowlist(providerID, surface, model string, modelPresent, nonInference bool) *middleware.Output {
if len(m.cfg.ProviderAllowlists) == 0 {
return nil
}
@@ -122,7 +124,7 @@ func (m *Middleware) evaluateAllowlist(providerID, model string, modelPresent bo
// if this request targets a restricted provider — fail closed. llm_router
// normally stamps the provider first, so this is a defensive guard.
if providerID == "" {
return denyModel("", denyCodeModelUnknown, denyMessageModelUnknown, denyReasonModelUnknown)
return denyModel(surface, "", denyCodeModelUnknown, denyMessageModelUnknown, denyReasonModelUnknown)
}
allowlist, restricted := m.cfg.ProviderAllowlists[providerID]
if !restricted {
@@ -133,18 +135,29 @@ func (m *Middleware) evaluateAllowlist(providerID, model string, modelPresent bo
// Fail closed: with an allowlist in effect for this provider, a request whose
// model the parser couldn't extract (absent/empty) is denied. This enforces
// the allowlist for path-routed providers (Bedrock, Vertex) with no body model.
//
// The exception is a non-inference endpoint the router already authorised.
// The model listing and the connection-warming probe name no model
// anywhere — not in a body, not in the path — so failing closed here
// rejected model discovery for exactly the accounts that configured an
// allowlist, which is the outage this endpoint is meant to avoid. The
// per-model lookup does name one (the router stamps it from the path), so
// it still falls through to the allowlist check below.
if !modelPresent || normaliseModel(model) == "" {
return denyModel("", denyCodeModelUnknown, denyMessageModelUnknown, denyReasonModelUnknown)
if nonInference {
return nil
}
return denyModel(surface, "", denyCodeModelUnknown, denyMessageModelUnknown, denyReasonModelUnknown)
}
if modelInAllowlist(allowlist, model) {
return nil
}
return denyModel(model, denyCodeModel, denyMessageModel, denyReasonModel)
return denyModel(surface, model, denyCodeModel, denyMessageModel, denyReasonModel)
}
// denyModel builds a 403 deny Output for a model-allowlist rejection. model is
// included in the details only when non-empty.
func denyModel(model, code, message, reason string) *middleware.Output {
func denyModel(surface, model, code, message, reason string) *middleware.Output {
details := map[string]string{}
if model != "" {
details["model"] = model
@@ -156,6 +169,7 @@ func denyModel(model, code, message, reason string) *middleware.Output {
Code: code,
Message: message,
Details: details,
Surface: surface,
},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMPolicyDecision, Value: "deny"},

View File

@@ -343,3 +343,52 @@ func TestFactoryNormalisesAllowlist(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out2.Decision, "trimmed entry must still match")
}
// TestAllowlistSkipsNonInferenceWithoutModel covers the reported regression:
// GET /v1/models carries no model anywhere, so the fail-closed rule above
// denied model discovery for exactly the accounts that configured a provider
// allowlist — the clients that read a 403 here render an empty model picker.
// The router authorises those endpoints by path before the guardrail sees
// them, so an absent model there is expected rather than undeterminable.
func TestAllowlistSkipsNonInferenceWithoutModel(t *testing.T) {
mw := New(providerCfg("gpt-4o"))
out, err := mw.Invoke(context.Background(), newInputProvider(testProvider,
middleware.KV{Key: middleware.KeyLLMNonInference, Value: "true"},
))
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"model discovery must not be refused because it names no model")
}
// TestAllowlistStillAppliesToNonInferenceWithModel pins that the exemption is
// scoped to requests that genuinely name nothing. The per-model lookup
// (GET /v1/models/{id}) is non-inference too, but the router stamps the model
// from its path, so the allowlist must still decide it — otherwise the
// exemption becomes a way to confirm a model the policy blocks.
func TestAllowlistStillAppliesToNonInferenceWithModel(t *testing.T) {
mw := New(providerCfg("gpt-4o"))
t.Run("model in the allowlist", func(t *testing.T) {
out, err := mw.Invoke(context.Background(), newInputProvider(testProvider,
middleware.KV{Key: middleware.KeyLLMNonInference, Value: "true"},
middleware.KV{Key: middleware.KeyLLMModel, Value: "gpt-4o"},
))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"an allowlisted model must stay reachable")
})
t.Run("model outside the allowlist", func(t *testing.T) {
out, err := mw.Invoke(context.Background(), newInputProvider(testProvider,
middleware.KV{Key: middleware.KeyLLMNonInference, Value: "true"},
middleware.KV{Key: middleware.KeyLLMModel, Value: "claude-opus-5"},
))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"non-inference must not become a way past the allowlist")
require.NotNil(t, out.DenyReason)
assert.Equal(t, "llm_policy.model_blocked", out.DenyReason.Code,
"a named but blocked model is blocked, not unknown")
})
}

View File

@@ -217,6 +217,32 @@ func applyHeaderPair(rule *HeaderPairRule, in *middleware.Input) *middleware.Mut
return mutations
}
// bodyInjectableSurfaces are the request-body dialects that accept the
// OpenAI-standard identity fields this middleware writes. A surface
// outside this set gets header-only stamping: "user" and "metadata.tags"
// are not part of the Anthropic Messages schema, which rejects unknown
// top-level fields and permits only "user_id" under metadata, so writing
// them into an Anthropic-shaped body turns a working request into a 400.
// Claude Code speaks that shape through gateway records pinned to the
// OpenAI parser, so the check keys on the detected surface rather than
// on the provider record.
var bodyInjectableSurfaces = map[string]struct{}{
"openai": {},
// An empty surface means no parser claimed the path (a custom gateway
// base). Those upstreams are OpenAI-compatible by convention, so keep
// the long-standing behaviour rather than silently dropping identity.
"": {},
}
// bodyAcceptsOpenAIIdentity reports whether the request body may carry the
// OpenAI-standard identity fields, read from the surface llm_request_parser
// resolved from the request path.
func bodyAcceptsOpenAIIdentity(in *middleware.Input) bool {
surface, _ := lookupMetadata(in.Metadata, middleware.KeyLLMProvider)
_, ok := bodyInjectableSurfaces[surface]
return ok
}
// injectIntoBody parses the request body and writes the supplied
// identity dimensions into it. Tags land at metadata.tags (creating
// the metadata object when absent); the user identity lands at the
@@ -225,6 +251,8 @@ func applyHeaderPair(rule *HeaderPairRule, in *middleware.Input) *middleware.Mut
// was written. Returns ok=false (no mutation) when:
//
// - both inputs are empty (nothing to write);
// - the body speaks a dialect without these fields (see
// bodyInjectableSurfaces);
// - the body is empty or truncated (we don't have the full document
// to safely round-trip);
// - the body isn't a JSON object (skip silently — this middleware
@@ -245,6 +273,9 @@ func injectIntoBody(in *middleware.Input, tags []string, userID string) ([]byte,
if in == nil || len(in.Body) == 0 || in.BodyTruncated {
return nil, false
}
if !bodyAcceptsOpenAIIdentity(in) {
return nil, false
}
var doc map[string]any
if err := json.Unmarshal(in.Body, &doc); err != nil {
return nil, false

View File

@@ -704,3 +704,57 @@ func TestInject_ExtraHeaders_EmptyValueSkipped(t *testing.T) {
"empty extra value must not be stamped")
}
}
// TestInject_AnthropicBodyIsNotRewritten pins the shape gate. Claude Code
// reaches a LiteLLM record on /v1/messages, where "user" is not a
// permitted top-level field and metadata accepts only "user_id", so
// writing the OpenAI-standard fields would turn a working request into a
// 400 naming a field the client never sent. Header stamping still runs, so
// spend tracking and per-end-user budgets keep working.
func TestInject_AnthropicBodyIsNotRewritten(t *testing.T) {
rule := liteLLMRuleWithBody()
rule.HeaderPair.EndUserIDInBody = true
mw := New(Config{Providers: []ProviderInjection{rule}})
in := newInput(litellmProvider, "alice", []string{"grp-eng"})
in.UserEmail = "alice@example.com"
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "anthropic"})
in.Body = []byte(`{"model":"claude-sonnet-5","messages":[]}`)
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out.Mutations)
assert.Empty(t, out.Mutations.BodyReplace,
"an Anthropic-shaped body must reach the upstream unmodified")
var endUser string
for _, kv := range out.Mutations.HeadersAdd {
if kv.Key == "x-litellm-end-user-id" {
endUser = kv.Value
}
}
assert.Equal(t, "alice@example.com", endUser,
"header stamping must still carry identity when body inject is skipped")
}
// TestInject_OpenAIBodyStillRewritten guards the gate against
// over-reaching: the OpenAI surface must keep its body-level identity,
// which is the only path LiteLLM's tag-budget check reads.
func TestInject_OpenAIBodyStillRewritten(t *testing.T) {
mw := New(Config{Providers: []ProviderInjection{liteLLMRuleWithBody()}})
in := newInput(litellmProvider, "alice", []string{"grp-eng"})
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "openai"})
in.Body = []byte(`{"model":"gpt-4o-mini","messages":[]}`)
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out.Mutations)
require.NotEmpty(t, out.Mutations.BodyReplace, "the OpenAI surface still gets body tags")
var doc map[string]any
require.NoError(t, json.Unmarshal(out.Mutations.BodyReplace, &doc))
meta, ok := doc["metadata"].(map[string]any)
require.True(t, ok, "metadata must be an object")
assert.NotEmpty(t, meta["tags"], "metadata.tags must still be written")
}

View File

@@ -84,6 +84,15 @@ func (m *Middleware) Invoke(ctx context.Context, in *middleware.Input) (*middlew
return allowNoAttribution(), nil
}
// Model-listing and other non-inference endpoints carry no model, and
// management's per-model allowlist fails closed on an empty one. The
// router has already authorised the route against the caller's groups
// and the request consumes no tokens, so gating it on a model that
// cannot exist would only break gateway model discovery.
if lookupKV(in.Metadata, middleware.KeyLLMNonInference) == "true" {
return allowNoAttribution(), nil
}
providerID := lookupKV(in.Metadata, middleware.KeyLLMResolvedProviderID)
if providerID == "" {
// llm_router didn't emit a resolved provider id — usually
@@ -117,7 +126,7 @@ func (m *Middleware) Invoke(ctx context.Context, in *middleware.Input) (*middlew
}
if resp.GetDecision() == "deny" {
return denyFromManagement(resp), nil
return denyFromManagement(resp, lookupKV(in.Metadata, middleware.KeyLLMProvider)), nil
}
return allowFromManagement(resp), nil
}
@@ -161,7 +170,7 @@ func allowFromManagement(resp *proto.CheckLLMPolicyLimitsResponse) *middleware.O
// envelope. The deny code surfaces verbatim through the framework's
// fixed JSON template; arbitrary middleware bytes can't reach the
// wire.
func denyFromManagement(resp *proto.CheckLLMPolicyLimitsResponse) *middleware.Output {
func denyFromManagement(resp *proto.CheckLLMPolicyLimitsResponse, surface string) *middleware.Output {
code := resp.GetDenyCode()
if code == "" {
code = "llm_policy.cap_exceeded"
@@ -176,6 +185,7 @@ func denyFromManagement(resp *proto.CheckLLMPolicyLimitsResponse) *middleware.Ou
DenyReason: &middleware.DenyReason{
Code: code,
Message: denyMessageForCode(code),
Surface: surface,
},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMPolicyDecision, Value: "deny"},

View File

@@ -224,3 +224,35 @@ func TestMetadataKeys_Allowlist(t *testing.T) {
}
assert.ElementsMatch(t, want, keys)
}
// TestInvoke_NonInferenceSkipsPreflight covers gateway model discovery:
// GET /v1/models carries no model, and management's per-model allowlist
// fails closed on an empty one, so a pre-flight would deny discovery for
// exactly the accounts that use the model allowlist. The router marks the
// request non-inference after authorising the route, and the gate must
// then allow without calling management at all.
func TestInvoke_NonInferenceSkipsPreflight(t *testing.T) {
mgmt := &fakeMgmt{
checkResp: &proto.CheckLLMPolicyLimitsResponse{
Decision: "deny",
DenyCode: "llm_policy.model_blocked",
},
}
m := New(mgmt, nil)
out := runInvoke(t, m, &middleware.Input{
AccountID: "acc-1",
UserID: "user-bob",
UserGroups: []string{"grp-engineers"},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMResolvedProviderID, Value: "prov-1"},
{Key: middleware.KeyLLMNonInference, Value: "true"},
},
})
assert.Equal(t, middleware.DecisionAllow, out.Decision, "model-less endpoints must not be gated on a model")
assert.Nil(t, mgmt.checkReq, "no pre-flight may be sent for a non-inference request")
assert.Empty(t, lookupKV(out.Metadata, middleware.KeyLLMSelectedPolicyID),
"no policy is attributed when nothing was metered")
}

View File

@@ -1,9 +1,13 @@
package llm_request_parser
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
func TestParseBedrockPath(t *testing.T) {
@@ -36,3 +40,25 @@ func TestParseBedrockPath(t *testing.T) {
}
}
}
// TestInvoke_BedrockCountTokens covers the dedicated token-counting
// endpoint. Denying it does not break the client, it just pushes context
// counting back onto the inference endpoint, which is billable.
func TestInvoke_BedrockCountTokens(t *testing.T) {
mw := newMiddleware(t)
out, err := mw.Invoke(context.Background(), &middleware.Input{
URL: "/model/us.anthropic.claude-sonnet-4-5-20250929-v1:0/count-tokens",
Body: []byte(`{"input":{"converse":{"messages":[]}}}`),
})
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision)
model, ok := metaValue(t, out.Metadata, middleware.KeyLLMModel)
require.True(t, ok, "count-tokens carries a model in the path and must emit it")
assert.Equal(t, "anthropic.claude-sonnet-4-5", model, "model must be normalized like any other action")
stream, _ := metaValue(t, out.Metadata, middleware.KeyLLMStream)
assert.Equal(t, "false", stream, "count-tokens never streams")
}

View File

@@ -61,6 +61,8 @@ func (middlewareImpl) MetadataKeys() []string {
middleware.KeyLLMRequestPromptRaw,
middleware.KeyLLMCaptureTruncated,
middleware.KeyLLMSessionID,
middleware.KeyLLMAgentID,
middleware.KeyLLMParentAgentID,
}
}
@@ -72,9 +74,9 @@ func (middlewareImpl) Close() error { return nil }
// Invoke detects the LLM provider, parses request facts, and emits
// metadata. Always returns DecisionAllow; never errors. Provider
// selection prefers the configured providerID (synthesiser-stamped on
// agent-network targets) so requests routed to a custom upstream URL
// still resolve. Falls back to URL sniffing when no providerID is set.
// selection prefers the request path, falling back to the configured
// providerID (synthesiser-stamped on agent-network targets) so requests
// routed to a custom upstream URL still resolve.
func (m middlewareImpl) Invoke(_ context.Context, in *middleware.Input) (*middleware.Output, error) {
out := &middleware.Output{Decision: middleware.DecisionAllow}
if in == nil {
@@ -92,9 +94,14 @@ func (m middlewareImpl) Invoke(_ context.Context, in *middleware.Input) (*middle
return m.invokeBedrock(in, br), nil
}
parser, ok := llm.ParserByName(m.providerID)
// A path that names an API surface wins over the configured providerID:
// a gateway record pinned to "openai" still serves Claude Code on
// /v1/messages, and reading that body with the OpenAI parser loses the
// Anthropic usage block and prices the request on the wrong surface.
// providerID stays the fallback for upstreams whose path says nothing.
parser, ok := llm.DetectParser(extractPath(in.URL))
if !ok {
parser, ok = llm.DetectParser(extractPath(in.URL))
parser, ok = llm.ParserByName(m.providerID)
}
if !ok {
return out, nil
@@ -116,9 +123,9 @@ func (m middlewareImpl) Invoke(_ context.Context, in *middleware.Input) (*middle
}
appendSessionID := func(md []middleware.KV) []middleware.KV {
if sessionID != "" {
return append(md, middleware.KV{Key: middleware.KeyLLMSessionID, Value: sessionID})
md = append(md, middleware.KV{Key: middleware.KeyLLMSessionID, Value: sessionID})
}
return md
return appendAgentIDs(md, in.Headers)
}
facts, err := parser.ParseRequest(in.Body)
@@ -160,6 +167,41 @@ func (m middlewareImpl) Invoke(_ context.Context, in *middleware.Input) (*middle
return out, nil
}
// agentIDHeader and parentAgentIDHeader carry sub-agent attribution: a
// coding agent that spawns helpers stamps the spawned agent's id, plus the
// spawning agent's when that helper is itself nested. Both are opaque
// identifiers rather than content, so they're emitted regardless of the
// prompt-collection toggle, the same way the session id is.
const (
agentIDHeader = "x-claude-code-agent-id"
parentAgentIDHeader = "x-claude-code-parent-agent-id"
)
// appendAgentIDs stamps the sub-agent attribution headers onto the metadata
// bag, skipping either one the request doesn't carry.
func appendAgentIDs(md []middleware.KV, headers []middleware.KV) []middleware.KV {
for _, pair := range []struct{ key, header string }{
{middleware.KeyLLMAgentID, agentIDHeader},
{middleware.KeyLLMParentAgentID, parentAgentIDHeader},
} {
if v := headerValue(headers, pair.header); v != "" {
md = append(md, middleware.KV{Key: pair.key, Value: v})
}
}
return md
}
// headerValue returns the first non-empty value for the named header.
// Headers arrive in canonical form, so the match is case-insensitive.
func headerValue(headers []middleware.KV, want string) string {
for _, kv := range headers {
if strings.EqualFold(kv.Key, want) && kv.Value != "" {
return kv.Value
}
}
return ""
}
// sessionIDHeaders are request header names that may carry a client
// session identifier, checked in order, case-insensitively. Matching is
// against Go's canonical header form, so use the hyphenated names the
@@ -173,10 +215,8 @@ var sessionIDHeaders = []string{"x-claude-code-session-id", "session-id", "x-ses
// canonical form, so the match is case-insensitive.
func sessionIDFromHeaders(headers []middleware.KV) string {
for _, want := range sessionIDHeaders {
for _, kv := range headers {
if strings.EqualFold(kv.Key, want) && kv.Value != "" {
return kv.Value
}
if v := headerValue(headers, want); v != "" {
return v
}
}
return ""
@@ -252,6 +292,12 @@ func parseVertexPath(reqPath string) (vertexRequest, bool) {
if c := strings.LastIndex(rest, ":"); c >= 0 {
model, action = rest[:c], rest[c+1:]
}
// Token counting hangs off the model as its own path segment
// (".../models/{model}/count-tokens:rawPredict"), so anything past the
// first "/" belongs to the method rather than the model id.
if slash := strings.Index(model, "/"); slash >= 0 {
model = model[:slash]
}
model = llm.NormalizeVertexModel(model)
if model == "" {
return vertexRequest{}, false
@@ -298,6 +344,7 @@ func (m middlewareImpl) invokeVertex(in *middleware.Input, vx vertexRequest) *mi
if sessionID != "" {
md = append(md, middleware.KV{Key: middleware.KeyLLMSessionID, Value: sessionID})
}
md = appendAgentIDs(md, in.Headers)
promptTruncated := false
if parser != nil && m.capturePrompt {
@@ -345,7 +392,9 @@ func trimBedrockNamespace(reqPath string) string {
//
// /model/{modelId}/{action}
//
// action ∈ {invoke, invoke-with-response-stream, converse, converse-stream}.
// action ∈ {invoke, invoke-with-response-stream, converse, converse-stream,
// count-tokens}. Token counting carries a model and no usage, so it routes
// like any other action and meters to zero.
// The modelId may be URL-encoded and may carry a cross-region inference-profile
// prefix and a version suffix; normalizeBedrockModel strips both so the model
// matches catalog pricing.
@@ -369,7 +418,7 @@ func parseBedrockPath(reqPath string) (bedrockRequest, bool) {
return bedrockRequest{}, false
}
switch action {
case "invoke", "converse":
case "invoke", "converse", "count-tokens":
return bedrockRequest{model: model}, true
case "invoke-with-response-stream", "converse-stream":
return bedrockRequest{model: model, stream: true}, true
@@ -397,6 +446,7 @@ func (m middlewareImpl) invokeBedrock(in *middleware.Input, br bedrockRequest) *
if sessionID != "" {
md = append(md, middleware.KV{Key: middleware.KeyLLMSessionID, Value: sessionID})
}
md = appendAgentIDs(md, in.Headers)
promptTruncated := false
if parser != nil && m.capturePrompt {

View File

@@ -45,6 +45,8 @@ func TestMiddleware_StaticSurface(t *testing.T) {
middleware.KeyLLMRequestPromptRaw,
middleware.KeyLLMCaptureTruncated,
middleware.KeyLLMSessionID,
middleware.KeyLLMAgentID,
middleware.KeyLLMParentAgentID,
}
assert.Equal(t, expected, keys, "metadata key allowlist must match the spec")
}
@@ -230,6 +232,31 @@ func TestInvoke_ProviderIDConfigBypassesURLSniff(t *testing.T) {
assert.Equal(t, "gpt-4o-mini", model)
}
func TestInvoke_PathSurfaceBeatsProviderIDConfig(t *testing.T) {
// Gateway records (LiteLLM, Portkey, OpenRouter) pin provider_id
// "openai", but the same record serves Claude Code on /v1/messages.
// Parsing that body as OpenAI reads no usage off the Anthropic
// response and prices the request on a surface where no claude-*
// model exists, so the path has to win.
mw, err := Factory{}.New([]byte(`{"provider_id":"openai"}`))
require.NoError(t, err, "factory must accept provider_id config")
out, err := mw.Invoke(context.Background(), &middleware.Input{
URL: "/v1/messages",
Body: []byte(`{"model":"claude-sonnet-5","stream":true,"messages":[{"role":"user","content":"Hi"}]}`),
})
require.NoError(t, err)
require.NotNil(t, out)
provider, ok := metaValue(t, out.Metadata, middleware.KeyLLMProvider)
require.True(t, ok, "provider must be emitted")
assert.Equal(t, "anthropic", provider, "the /v1/messages path selects the Anthropic surface")
model, ok := metaValue(t, out.Metadata, middleware.KeyLLMModel)
require.True(t, ok, "model must be extracted")
assert.Equal(t, "claude-sonnet-5", model)
}
func TestInvoke_UnknownProviderIDFallsBackToURL(t *testing.T) {
mw, err := Factory{}.New([]byte(`{"provider_id":"not-a-real-parser"}`))
require.NoError(t, err, "factory must accept any provider_id string")
@@ -416,3 +443,81 @@ func TestInvoke_NilInputAllows(t *testing.T) {
assert.Equal(t, middleware.DecisionAllow, out.Decision, "nil input still allows")
assert.Empty(t, out.Metadata, "nil input emits no metadata")
}
// TestParseVertexPath_CountTokensKeepsModel covers Vertex token counting,
// where the method hangs off the model as its own path segment. Splitting
// only on the final colon swallowed "/count-tokens" into the model id, so
// the router saw a model no route could claim.
func TestParseVertexPath_CountTokensKeepsModel(t *testing.T) {
cases := map[string]struct {
model string
stream bool
}{
"/v1/projects/p/locations/global/publishers/anthropic/models/claude-sonnet-5:rawPredict": {model: "claude-sonnet-5"},
"/v1/projects/p/locations/global/publishers/anthropic/models/claude-sonnet-5:streamRawPredict": {model: "claude-sonnet-5", stream: true},
"/v1/projects/p/locations/global/publishers/anthropic/models/claude-sonnet-5/count-tokens:rawPredict": {model: "claude-sonnet-5"},
"/v1/projects/p/locations/global/publishers/anthropic/models/claude-sonnet-5@20250929/count-tokens:rawPredict": {model: "claude-sonnet-5"},
}
for path, want := range cases {
vx, ok := parseVertexPath(path)
require.True(t, ok, "must parse %q", path)
assert.Equal(t, want.model, vx.model, "model for %q", path)
assert.Equal(t, want.stream, vx.stream, "stream flag for %q", path)
assert.Equal(t, "anthropic", vx.publisher, "publisher for %q", path)
}
}
// TestInvoke_EmitsAgentIDs covers sub-agent attribution: several agents run
// in parallel inside one session, and without their ids every request in
// the session attributes to the session alone.
func TestInvoke_EmitsAgentIDs(t *testing.T) {
mw := newMiddleware(t)
t.Run("spawned agent", func(t *testing.T) {
out, err := mw.Invoke(context.Background(), &middleware.Input{
URL: "/v1/messages",
Body: []byte(`{"model":"claude-sonnet-5","messages":[]}`),
Headers: []middleware.KV{
{Key: "X-Claude-Code-Session-Id", Value: "sess-1"},
{Key: "X-Claude-Code-Agent-Id", Value: "agent-7"},
},
})
require.NoError(t, err)
agent, ok := metaValue(t, out.Metadata, middleware.KeyLLMAgentID)
require.True(t, ok, "the spawned agent's id must be emitted")
assert.Equal(t, "agent-7", agent)
_, ok = metaValue(t, out.Metadata, middleware.KeyLLMParentAgentID)
assert.False(t, ok, "a top-level agent has no parent to emit")
})
t.Run("nested agent", func(t *testing.T) {
out, err := mw.Invoke(context.Background(), &middleware.Input{
URL: "/v1/messages",
Body: []byte(`{"model":"claude-sonnet-5","messages":[]}`),
Headers: []middleware.KV{
{Key: "X-Claude-Code-Agent-Id", Value: "agent-9"},
{Key: "X-Claude-Code-Parent-Agent-Id", Value: "agent-7"},
},
})
require.NoError(t, err)
agent, _ := metaValue(t, out.Metadata, middleware.KeyLLMAgentID)
assert.Equal(t, "agent-9", agent)
parent, ok := metaValue(t, out.Metadata, middleware.KeyLLMParentAgentID)
require.True(t, ok, "a nested agent must carry the spawning agent's id")
assert.Equal(t, "agent-7", parent)
})
t.Run("absent on a plain request", func(t *testing.T) {
out, err := mw.Invoke(context.Background(), &middleware.Input{
URL: "/v1/messages",
Body: []byte(`{"model":"claude-sonnet-5","messages":[]}`),
})
require.NoError(t, err)
_, ok := metaValue(t, out.Metadata, middleware.KeyLLMAgentID)
assert.False(t, ok, "no key is emitted when the client sends no agent id")
})
}

View File

@@ -1,9 +1,13 @@
package llm_router
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
// TestRouteClaimsModel_BedrockNormalizesCandidate guards the fix for the native
@@ -28,3 +32,86 @@ func TestRouteClaimsModel_BedrockNormalizesCandidate(t *testing.T) {
assert.False(t, routeClaimsModel(openai, "us.gpt-4o"),
"non-Bedrock routes must not strip a us. prefix")
}
// TestRouter_BedrockCountTokensRoutes pins that the token-counting action
// reaches the Bedrock route instead of denying as not-routable.
func TestRouter_BedrockCountTokensRoutes(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{{
ID: "bedrock-prod",
Bedrock: true,
Models: []string{"anthropic.claude-sonnet-4-5"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "bedrock-runtime.eu-central-1.amazonaws.com",
}}})
in := newInputWithModelAndURL("anthropic.claude-sonnet-4-5",
"/model/anthropic.claude-sonnet-4-5-20250929-v1:0/count-tokens")
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "bedrock"})
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "count-tokens must route, not deny")
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "bedrock-runtime.eu-central-1.amazonaws.com", out.Mutations.RewriteUpstream.Host)
}
// TestRouter_BedrockInferenceProfilesRoutes covers the startup lookups a
// client makes to resolve a configured inference profile. They carry no
// model, so before they were recognised they denied and wrote a policy
// rejection into the access log on every session start.
func TestRouter_BedrockInferenceProfilesRoutes(t *testing.T) {
bedrock := ProviderRoute{
ID: "bedrock-prod",
Bedrock: true,
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "bedrock-runtime.eu-central-1.amazonaws.com",
}
openai := ProviderRoute{
ID: "openai-prod",
Models: []string{"gpt-4o"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "api.openai.com",
}
mw := New(Config{Providers: []ProviderRoute{openai, bedrock}})
for _, path := range []string{
"/inference-profiles?type=SYSTEM_DEFINED",
"/inference-profiles/us.anthropic.claude-sonnet-5",
} {
out, err := mw.Invoke(context.Background(), newModellessInput(path))
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "%s must route", path)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "bedrock-runtime.eu-central-1.amazonaws.com", out.Mutations.RewriteUpstream.Host,
"%s must reach the Bedrock provider, not the first authorised one", path)
nonInference, _ := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
assert.Equal(t, "true", nonInference, "%s carries no model to gate on", path)
}
}
// TestRouter_BedrockNamespacedInferenceProfilesStripsPrefix pins that the
// optional gateway namespace is removed before the request goes upstream.
func TestRouter_BedrockNamespacedInferenceProfilesStripsPrefix(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{{
ID: "bedrock-prod",
Bedrock: true,
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "bedrock-runtime.eu-central-1.amazonaws.com",
}}})
out, err := mw.Invoke(context.Background(), newModellessInput("/bedrock/inference-profiles"))
require.NoError(t, err)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "/bedrock", out.Mutations.RewriteUpstream.StripPathPrefix,
"the namespace prefix must not reach the real Bedrock endpoint")
}

View File

@@ -44,6 +44,12 @@ type ProviderRoute struct {
AuthHeaderName string `json:"auth_header_name"`
AuthHeaderValue string `json:"auth_header_value"`
AllowedGroupIDs []string `json:"allowed_group_ids"`
// ModelPolicies carries, per authorising policy, the source groups it
// binds and the models it permits. The router uses it to bound a model
// listing to what THIS caller may use: a provider reachable by two groups
// under different allowlists must not offer either group the other's
// models. Empty means no policy restricts models on this route.
ModelPolicies []ModelPolicyRule `json:"model_policies,omitempty"`
// Vertex marks a Google Vertex AI provider. Vertex requests carry the
// model in the URL path, so the router selects this route by path
// (isVertexPath) and bypasses the model/vendor table entirely.
@@ -65,6 +71,18 @@ type ProviderRoute struct {
SkipTLSVerify bool `json:"skip_tls_verify,omitempty"`
}
// ModelPolicyRule is one authorising policy's contribution to what a caller
// may use on a route: the source groups it binds, and the models it permits.
//
// Models is nil when the policy sets no model allowlist — an unrestricted
// policy, which lifts the restriction for the groups it binds. That is why
// nil and empty must stay distinct: an empty list is a guardrail that permits
// nothing, and collapsing the two would let a listing fail open.
type ModelPolicyRule struct {
GroupIDs []string `json:"group_ids"`
Models []string `json:"models"`
}
// Config is the on-wire configuration accepted by the factory. An
// empty Providers slice yields a router that denies every request as
// not-routable; the synthesiser is responsible for stamping the

View File

@@ -109,6 +109,10 @@ func (m *Middleware) MetadataKeys() []string {
middleware.KeyLLMAuthorisingGroups,
middleware.KeyLLMPolicyDecision,
middleware.KeyLLMPolicyReason,
middleware.KeyLLMNonInference,
// Emitted only for the per-model lookup, whose model lives in the path
// rather than a body the parser could read.
middleware.KeyLLMModel,
}
}
@@ -137,29 +141,26 @@ const (
// known to a provider that no policy authorises for the caller deny
// with no_authorised_provider.
func (m *Middleware) Invoke(_ context.Context, in *middleware.Input) (*middleware.Output, error) {
reqPath := requestPath(in.URL)
// The caller's API dialect, used to mirror a denial in the vendor's own
// error shape so the client can explain it to the user.
surface, _ := lookupMetadata(in.Metadata, middleware.KeyLLMProvider)
model, _ := lookupMetadata(in.Metadata, middleware.KeyLLMModel)
// Vertex AI carries the model in the URL path, not the body, and is
// selected by path rather than by the model/vendor table. Route it before
// the model lookup so a model the parser extracted from the path can't be
// claimed by a same-vendor direct provider (e.g. claude-* on api.anthropic.com).
reqPath := requestPath(in.URL)
if isVertexPath(reqPath) {
model, _ := lookupMetadata(in.Metadata, middleware.KeyLLMModel)
// The request parser emits no llm.provider for a Vertex publisher it
// can't parse (e.g. google/gemini). Forwarding such a request would
// bypass token/budget metering, so deny it rather than serve it
// unmetered.
if vendor, _ := lookupMetadata(in.Metadata, middleware.KeyLLMProvider); vendor == "" {
return denyUnmeterable(), nil
if surface == "" {
return denyUnmeterable(surface), nil
}
route, outcome := m.matchVertex(reqPath, model, in.UserGroups)
switch outcome {
case matchOutcomeFound:
return m.allowWithRoute(route, in.UserGroups), nil
case matchOutcomeUnauthorised:
return denyNoAuthorisedRoute(model), nil
default:
return denyUnknownModel(model), nil
}
return m.decide(route, outcome, surface, model, in.UserGroups, nil), nil
}
// Bedrock likewise carries the model in the URL path (/model/{id}/{action}),
@@ -167,52 +168,211 @@ func (m *Middleware) Invoke(_ context.Context, in *middleware.Input) (*middlewar
// before the model lookup; when the prefix is present, strip it from the
// forwarded path so the real Bedrock endpoint receives its native path.
if isBedrockPath(reqPath) {
model, _ := lookupMetadata(in.Metadata, middleware.KeyLLMModel)
native, hadPrefix := splitBedrockNamespace(reqPath)
route, outcome := m.matchBedrock(native, model, in.UserGroups)
switch outcome {
case matchOutcomeFound:
out := m.allowWithRoute(route, in.UserGroups)
if hadPrefix && out.Mutations != nil && out.Mutations.RewriteUpstream != nil {
out.Mutations.RewriteUpstream.StripPathPrefix = bedrockNamespacePrefix
return m.decide(route, outcome, surface, model, in.UserGroups, func(out *middleware.Output) {
if hadPrefix {
stripBedrockNamespace(out)
}
return out, nil
case matchOutcomeUnauthorised:
return denyNoAuthorisedRoute(model), nil
default:
return denyUnknownModel(model), nil
}
}), nil
}
model, ok := lookupMetadata(in.Metadata, middleware.KeyLLMModel)
if !ok || model == "" {
// Non-inference endpoints (model listing) carry no model but still
// need rewriting from the synth placeholder to a real upstream;
// clients such as Codex call GET /v1/models at startup to enumerate
// availability and read a 403 as "model unavailable".
route, outcome := m.matchModelless(requestPath(in.URL), in.UserGroups)
switch outcome {
case matchOutcomeFound:
return m.allowWithRoute(route, in.UserGroups), nil
case matchOutcomeUnauthorised:
// A recognised model-less endpoint exists but no provider
// authorises the caller — deny as an authorisation failure
// rather than masking it as a missing model.
return denyNoAuthorisedRoute(model), nil
default:
return denyMissingModel(), nil
}
// GET /v1/models/{id} carries no body, so no model reaches the router in
// metadata — but the path names one, and answering it confirms a model
// exists and is reachable. Authorise it against the model table like any
// other per-model request, then mark it non-inference so it still skips
// the token pre-flight it would otherwise charge nothing against.
if detail, isDetail := modelDetailID(reqPath); isDetail && isNonInferenceMethod(in.Method) {
route, outcome := m.matchRoute(detail, surface, reqPath, in.UserGroups)
return m.decide(route, outcome, surface, detail, in.UserGroups, func(out *middleware.Output) {
markNonInference(out)
// The parser reads models from JSON bodies only, and this request
// has none, so stamp the one the path names. Without it the
// guardrail's own allowlist — a separate, possibly narrower list
// than the route's — never sees a model to check.
out.Metadata = append(out.Metadata, middleware.KV{Key: middleware.KeyLLMModel, Value: detail})
}), nil
}
vendor, _ := lookupMetadata(in.Metadata, middleware.KeyLLMProvider)
route, outcome := m.matchRoute(model, vendor, requestPath(in.URL), in.UserGroups)
if model == "" {
return m.routeModelless(reqPath, surface, in.Method, in.UserGroups), nil
}
route, outcome := m.matchRoute(model, surface, reqPath, in.UserGroups)
return m.decide(route, outcome, surface, model, in.UserGroups, nil), nil
}
// decide turns a per-model match result into the middleware's decision. Every
// surface that routes by model shares the same two denial arms — a model no
// route claims is not routable, one that some route claims but none authorises
// for this caller is an authorisation failure — so they live here once.
// decorate, when non-nil, adjusts the allow with whatever that surface needs.
func (m *Middleware) decide(
route ProviderRoute,
outcome matchOutcome,
surface, model string,
userGroups []string,
decorate func(*middleware.Output),
) *middleware.Output {
switch outcome {
case matchOutcomeFound:
return m.allowWithRoute(route, in.UserGroups), nil
out := m.allowWithRoute(route, surface, userGroups)
if decorate != nil {
decorate(out)
}
return out
case matchOutcomeUnauthorised:
return denyNoAuthorisedRoute(model), nil
return denyNoAuthorisedRoute(surface, model)
default:
return denyUnknownModel(model), nil
return denyUnknownModel(surface, model)
}
}
// routeModelless serves the endpoints that name no model at all: the model
// listing, the connection-warming probe, and the Bedrock inference-profile
// lookup. They still need rewriting from the synth placeholder to a real
// upstream — clients such as Codex call GET /v1/models at startup to enumerate
// availability and read a 403 as "model unavailable".
func (m *Middleware) routeModelless(reqPath, surface, method string, userGroups []string) *middleware.Output {
route, outcome := m.matchModelless(reqPath, method, userGroups)
switch outcome {
case matchOutcomeFound:
out := m.allowWithRoute(route, surface, userGroups)
markNonInference(out)
if _, hadPrefix := splitBedrockNamespace(reqPath); hadPrefix {
stripBedrockNamespace(out)
}
// What the caller may actually use bounds what the picker may offer:
// every entry outside it is a request the chain will deny a moment
// later.
if reqPath == modelListingPath && out.Mutations != nil && out.Mutations.RewriteUpstream != nil {
if models, bounded := discoverableModels(route, userGroups); bounded {
out.Mutations.RewriteUpstream.DiscoveryModels = models
}
}
return out
case matchOutcomeUnauthorised:
// A recognised model-less endpoint exists but no provider authorises
// the caller — deny as an authorisation failure rather than masking it
// as a missing model.
return denyNoAuthorisedRoute(surface, "")
default:
return denyMissingModel(surface)
}
}
// isNonInferenceMethod reports whether a request method is one the
// non-inference endpoints actually use: the listing and the per-model lookup
// are GET, the connection-warming probe is HEAD or GET. The method is the only
// thing separating "GET /v1/models/{id}" from a POST to the same path carrying
// an inference body, and the non-inference mark exempts a request from the
// token pre-flight — so anything else falls through to normal per-model
// routing, which denies when the request names no model.
func isNonInferenceMethod(method string) bool {
return method == http.MethodGet || method == http.MethodHead
}
// discoverableModels returns the model ids a caller in userGroups may actually
// use on this route, and whether the listing should be bounded to them at all.
//
// Two things narrow a listing, and both must apply or the picker offers models
// the very next request refuses:
//
// - the provider's own enumerated models, when it lists any (a gateway record
// enumerates nothing and claims everything);
// - the model allowlists of the policies that authorise THIS caller. A
// provider reachable by two groups under different allowlists must not
// offer either group the other's models, which is why the rules carry their
// source groups rather than arriving pre-flattened.
//
// A policy that sets no allowlist lifts the restriction for the groups it
// binds, so a caller holding one unrestricted policy sees the provider's full
// list. bounded is false when nothing narrows the listing — an unrestricted
// caller on a route that enumerates nothing — in which case the upstream's own
// answer passes through untouched.
func discoverableModels(route ProviderRoute, userGroups []string) ([]string, bool) {
permitted, restricted := policyPermittedModels(route, userGroups)
switch {
case !restricted && len(route.Models) == 0:
return nil, false
case !restricted:
return append([]string(nil), route.Models...), true
case len(route.Models) == 0:
// A gateway record enumerates nothing, so the allowlist is the whole
// bound — previously such a record offered the upstream's entire
// catalogue however narrow the policy was.
return sortedModels(permitted), true
}
// Both bound: only what the provider serves and the policy permits.
intersection := make(map[string]struct{}, len(route.Models))
for _, m := range route.Models {
if _, ok := permitted[m]; ok {
intersection[m] = struct{}{}
}
}
return sortedModels(intersection), true
}
// policyPermittedModels folds the rules whose groups intersect the caller's
// into the set of models they permit. restricted is false when the caller
// holds at least one authorising policy that sets no allowlist, or when no
// rule binds them at all.
func policyPermittedModels(route ProviderRoute, userGroups []string) (map[string]struct{}, bool) {
permitted := make(map[string]struct{})
restricted := false
for _, rule := range route.ModelPolicies {
if !groupsIntersect(rule.GroupIDs, userGroups) {
continue
}
if rule.Models == nil {
// An unrestricted policy the caller holds lifts the restriction
// entirely, whatever the others say.
return nil, false
}
restricted = true
for _, m := range rule.Models {
permitted[m] = struct{}{}
}
}
return permitted, restricted
}
// groupsIntersect reports whether the two group-id sets share a member.
func groupsIntersect(a, b []string) bool {
for _, x := range a {
for _, y := range b {
if x == y {
return true
}
}
}
return false
}
// sortedModels flattens a model set into a stable slice so the bound the proxy
// applies — and any test asserting on it — does not depend on map order.
func sortedModels(set map[string]struct{}) []string {
out := make([]string, 0, len(set))
for m := range set {
out = append(out, m)
}
sort.Strings(out)
return out
}
// markNonInference tags an allow as a request that spends no tokens, so the
// limit check skips the management pre-flight it would charge nothing against.
func markNonInference(out *middleware.Output) {
out.Metadata = append(out.Metadata, middleware.KV{Key: middleware.KeyLLMNonInference, Value: "true"})
}
// stripBedrockNamespace tells the rewrite to drop the optional "/bedrock"
// gateway namespace so the upstream receives its native Bedrock path.
func stripBedrockNamespace(out *middleware.Output) {
if out.Mutations != nil && out.Mutations.RewriteUpstream != nil {
out.Mutations.RewriteUpstream.StripPathPrefix = bedrockNamespacePrefix
}
}
@@ -300,12 +460,60 @@ func (m *Middleware) matchRoute(model, vendor, reqPath string, userGroups []stri
return best, matchOutcomeFound
}
// isModelLessPath reports whether reqPath is a known OpenAI-shaped
// non-inference endpoint that legitimately carries no model in its
// request (the model-listing endpoints). These must route to an upstream
// rather than deny, so model enumeration works end to end.
// connectionWarmPath is the probe Anthropic clients send before their first
// inference request to open the upstream connection early. Forwarding it
// warms the connection the request will actually use; denying it only fills
// the access log with rejections at every session start.
const connectionWarmPath = "/api/hello"
// modelListingPath is the endpoint clients read at startup to populate
// their model picker. Its response is a list the proxy can bound; the
// per-model "/v1/models/{id}" lookup returns a single object and is left
// alone.
const modelListingPath = "/v1/models"
// isModelLessPath reports whether reqPath is a known non-inference endpoint
// that legitimately carries no model at all: the model listing and the
// connection-warming probe. These must route to an upstream rather than
// deny, so model enumeration works end to end. The per-model
// "/v1/models/{id}" lookup is deliberately excluded — it names a model, so
// it is authorised against the model table instead (see modelDetailID).
func isModelLessPath(reqPath string) bool {
return reqPath == "/v1/models" || strings.HasPrefix(reqPath, "/v1/models/")
return reqPath == modelListingPath || reqPath == connectionWarmPath
}
// modelDetailID returns the model id named by a "/v1/models/{id}" lookup.
// reqPath comes from url.URL.Path, which is already percent-decoded, so an
// id carrying a "/" (a self-hosted "Qwen/Qwen2.5-0.5B-Instruct" sent as
// "Qwen%2FQwen2.5-...") arrives whole and everything after the prefix is the
// id, separators included.
func modelDetailID(reqPath string) (string, bool) {
if !strings.HasPrefix(reqPath, modelListingPath+"/") {
return "", false
}
id := strings.TrimPrefix(reqPath, modelListingPath+"/")
if id == "" {
return "", false
}
return id, true
}
// isBedrockModelLessPath reports whether reqPath is a Bedrock
// inference-profile lookup, optionally behind the "/bedrock" gateway
// namespace. Clients read these at startup to resolve a configured profile
// to its underlying model. They carry no model of their own, so they route
// by path to a Bedrock provider rather than through the model table.
//
// On native AWS these live on the control plane ("bedrock.<region>") while a
// provider's upstream is normally the runtime host ("bedrock-runtime.<region>"),
// so forwarding yields a 404 there. That is deliberate: a client has one base
// URL, so pointing it straight at the runtime host 404s identically, and
// forwarding keeps the proxy transparent instead of inventing a policy denial
// the client would never otherwise see. Operators whose Bedrock upstream is a
// gateway that does serve the lookup get a working answer.
func isBedrockModelLessPath(reqPath string) bool {
native, _ := splitBedrockNamespace(reqPath)
return native == "/inference-profiles" || strings.HasPrefix(native, "/inference-profiles/")
}
// isVertexPath reports whether reqPath is a Google Vertex AI publisher
@@ -332,20 +540,33 @@ func splitBedrockNamespace(reqPath string) (string, bool) {
return reqPath, false
}
// bedrockActions are the runtime actions that follow the model id in a
// Bedrock path. count-tokens is here so a client can price its context
// against the dedicated endpoint; denying it pushes that work back onto
// the inference endpoint, which bills for it.
var bedrockActions = []string{
"/invoke",
"/invoke-with-response-stream",
"/converse",
"/converse-stream",
"/count-tokens",
}
// isBedrockPath reports whether reqPath is an AWS Bedrock runtime model
// endpoint: /model/{modelId}/{action} where action is invoke,
// invoke-with-response-stream, converse, or converse-stream — optionally behind
// a "/bedrock" gateway-namespace prefix. The model lives in the path, so these
// requests are routed by path to the Bedrock provider.
// endpoint: /model/{modelId}/{action} — optionally behind a "/bedrock"
// gateway-namespace prefix. The model lives in the path, so these requests
// are routed by path to the Bedrock provider.
func isBedrockPath(reqPath string) bool {
native, _ := splitBedrockNamespace(reqPath)
if !strings.HasPrefix(native, "/model/") {
return false
}
return strings.HasSuffix(native, "/invoke") ||
strings.HasSuffix(native, "/invoke-with-response-stream") ||
strings.HasSuffix(native, "/converse") ||
strings.HasSuffix(native, "/converse-stream")
for _, action := range bedrockActions {
if strings.HasSuffix(native, action) {
return true
}
}
return false
}
// matchVertex selects the Vertex provider authorised for the caller's groups
@@ -425,19 +646,26 @@ func (m *Middleware) matchPathRoute(reqPath, model string, userGroups []string,
// declaration order), matchOutcomeUnauthorised when no provider authorises
// the caller, or matchOutcomeUnknownModel when the path isn't a recognised
// model-less endpoint.
func (m *Middleware) matchModelless(reqPath string, userGroups []string) (ProviderRoute, matchOutcome) {
if !isModelLessPath(reqPath) {
func (m *Middleware) matchModelless(reqPath, method string, userGroups []string) (ProviderRoute, matchOutcome) {
if !isNonInferenceMethod(method) {
return ProviderRoute{}, matchOutcomeUnknownModel
}
var candidates []ProviderRoute
for _, route := range m.cfg.Providers {
var eligible func(ProviderRoute) bool
switch {
case isBedrockModelLessPath(reqPath):
eligible = func(r ProviderRoute) bool { return r.Bedrock }
case isModelLessPath(reqPath):
// Vertex/Bedrock are path-routed and don't serve OpenAI-style
// model-listing endpoints; including them here could rewrite a
// GET /v1/models to an upstream that 404s it.
if route.Vertex || route.Bedrock {
continue
}
if routeAuthorisesGroups(route, userGroups) {
eligible = func(r ProviderRoute) bool { return !r.Vertex && !r.Bedrock }
default:
return ProviderRoute{}, matchOutcomeUnknownModel
}
var candidates []ProviderRoute
for _, route := range m.cfg.Providers {
if eligible(route) && routeAuthorisesGroups(route, userGroups) {
candidates = append(candidates, route)
}
}
@@ -564,6 +792,16 @@ func routeClaimsModel(route ProviderRoute, model string) bool {
if route.Bedrock && llm.NormalizeBedrockModel(candidate) == model {
return true
}
// A client may pin a dated Anthropic id ("claude-sonnet-4-5-20250929")
// where the operator registered the undated one. Only an undated
// registration absorbs a dated request: normalising both sides would
// let a route pinned to one dated release claim a different one, so an
// operator who deliberately pinned a build would silently serve
// another — and with several such routes, ordering would decide which.
if candidate == llm.NormalizeAnthropicModel(candidate) &&
candidate == llm.NormalizeAnthropicModel(model) {
return true
}
}
return false
}
@@ -612,7 +850,7 @@ func requestPath(raw string) string {
// provider id so identity-stamping middlewares (llm_identity_inject)
// tag the request with ONLY the groups that authorised this specific
// route — not every group the peer happens to be in.
func (m *Middleware) allowWithRoute(route ProviderRoute, userGroups []string) *middleware.Output {
func (m *Middleware) allowWithRoute(route ProviderRoute, surface string, userGroups []string) *middleware.Output {
rewrite := &middleware.UpstreamRewrite{
Scheme: route.UpstreamScheme,
Host: route.UpstreamHost,
@@ -634,7 +872,7 @@ func (m *Middleware) allowWithRoute(route ProviderRoute, userGroups []string) *m
// request time (cached + auto-refreshed) instead of a static value.
bearer, err := m.gcpBearer(route.GCPServiceAccountKeyB64)
if err != nil {
return denyUpstreamAuth()
return denyUpstreamAuth(surface)
}
authValue = bearer
}
@@ -704,11 +942,12 @@ func (m *Middleware) gcpTokenSource(saKeyB64 string) (oauth2.TokenSource, error)
// denyUpstreamAuth is returned when the router cannot obtain the upstream
// credential (e.g. a malformed service-account key or an unreachable token
// endpoint). It surfaces as a 502 — an upstream problem, not a policy denial.
func denyUpstreamAuth() *middleware.Output {
func denyUpstreamAuth(surface string) *middleware.Output {
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 502,
DenyReason: &middleware.DenyReason{
Surface: surface,
Code: denyCodeUpstreamAuth,
Message: "could not obtain upstream credential",
},
@@ -722,11 +961,12 @@ func denyUpstreamAuth() *middleware.Output {
// denyUnmeterable returns the deny envelope for a path-routed request whose
// publisher has no parser surface, so its usage can't be metered. Serving it
// would bypass token/budget caps, so it is rejected with a 403.
func denyUnmeterable() *middleware.Output {
func denyUnmeterable(surface string) *middleware.Output {
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Surface: surface,
Code: denyCodeUnmeterable,
Message: "request publisher is not supported for metering",
},
@@ -739,11 +979,12 @@ func denyUnmeterable() *middleware.Output {
// denyMissingModel returns the deny envelope for a request whose
// envelope has no llm.model metadata.
func denyMissingModel() *middleware.Output {
func denyMissingModel(surface string) *middleware.Output {
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Surface: surface,
Code: denyCodeNotRoutable,
Message: "missing llm.model on request envelope",
},
@@ -756,11 +997,12 @@ func denyMissingModel() *middleware.Output {
// denyUnknownModel returns the deny envelope for a model that no
// configured provider claims.
func denyUnknownModel(model string) *middleware.Output {
func denyUnknownModel(surface, model string) *middleware.Output {
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Surface: surface,
Code: denyCodeNotRoutable,
Message: fmt.Sprintf("no provider configured for model %s", model),
Details: map[string]string{"model": model},
@@ -775,11 +1017,12 @@ func denyUnknownModel(model string) *middleware.Output {
// denyNoAuthorisedRoute returns the deny envelope for a model that one
// or more providers claim, but where no policy authorises the caller's
// groups for any of those providers.
func denyNoAuthorisedRoute(model string) *middleware.Output {
func denyNoAuthorisedRoute(surface, model string) *middleware.Output {
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Surface: surface,
Code: denyCodeNoAuthorisedRoute,
Message: fmt.Sprintf("no policy authorises model %s for the caller's groups", model),
Details: map[string]string{"model": model},

View File

@@ -2,6 +2,7 @@ package llm_router
import (
"context"
"net/http"
"testing"
"github.com/stretchr/testify/assert"
@@ -60,6 +61,8 @@ func TestMiddlewareIdentity(t *testing.T) {
[]string{
middleware.KeyLLMResolvedProviderID,
middleware.KeyLLMAuthorisingGroups,
middleware.KeyLLMNonInference,
middleware.KeyLLMModel,
middleware.KeyLLMPolicyDecision,
middleware.KeyLLMPolicyReason,
},
@@ -171,8 +174,12 @@ func TestRouter_MissingModel(t *testing.T) {
// from which a model could be parsed). UserGroups matches defaultTestGroup.
func newModellessInput(reqURL string) *middleware.Input {
return &middleware.Input{
Slot: middleware.SlotOnRequest,
URL: reqURL,
Slot: middleware.SlotOnRequest,
URL: reqURL,
// The non-inference endpoints are read requests; the method is what
// separates them from an inference body posted to the same path, so
// state it rather than leaning on the zero value.
Method: http.MethodGet,
UserGroups: []string{defaultTestGroup},
}
}
@@ -197,6 +204,12 @@ func TestRouter_ModelLessPath_RoutesToAuthorisedProvider(t *testing.T) {
provider, _ := metaValue(t, out.Metadata, middleware.KeyLLMResolvedProviderID)
assert.Equal(t, "openai-prod", provider, "resolved provider must be the authorised route")
// The limits gate reads this to tell "no model applies here" from
// "the model could not be determined", which fails closed.
nonInference, ok := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
require.True(t, ok, "model-less allow must mark the request non-inference")
assert.Equal(t, "true", nonInference)
}
func TestRouter_ModelLessPath_MultiProviderDeclarationOrder(t *testing.T) {
@@ -873,3 +886,403 @@ func TestRouter_EmptyModelsClaimsAnyModel(t *testing.T) {
resolved, _ := metaValue(t, out.Metadata, middleware.KeyLLMResolvedProviderID)
assert.Equal(t, "litellm", resolved)
}
// TestRouter_DatedAnthropicModelRoutes covers a client pinning a release
// date on a model the operator registered undated. Exact matches still win,
// so an operator who registers both dated releases keeps them distinct.
func TestRouter_DatedAnthropicModelRoutes(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{{
ID: "anthropic-prod",
Vendor: "anthropic",
Models: []string{"claude-sonnet-4-5"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "api.anthropic.com",
}}})
in := newInputWithModelAndURL("claude-sonnet-4-5-20250929", "/v1/messages")
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "anthropic"})
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "a dated id must route to the undated registration")
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "api.anthropic.com", out.Mutations.RewriteUpstream.Host)
}
// TestRouter_ConnectionWarmProbeRoutes covers the HEAD /api/hello probe an
// Anthropic client sends before its first request. Forwarding it warms the
// connection that request will use; denying it only wrote a rejection into
// the access log at every session start.
func TestRouter_ConnectionWarmProbeRoutes(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{{
ID: "anthropic-prod",
Vendor: "anthropic",
Models: []string{"claude-sonnet-5"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "api.anthropic.com",
}}})
in := newModellessInput("/api/hello")
in.Method = http.MethodHead
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "the warm-up probe must reach the upstream")
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "api.anthropic.com", out.Mutations.RewriteUpstream.Host)
nonInference, _ := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
assert.Equal(t, "true", nonInference, "the probe carries no model to gate on")
}
// TestRouter_ModelListingCarriesAuthorisedModels pins the list the proxy
// bounds the discovery response with. A catch-all route enumerates nothing,
// so it must not bound the upstream's list at all.
func TestRouter_ModelListingCarriesAuthorisedModels(t *testing.T) {
enumerated := ProviderRoute{
ID: "anthropic-prod",
Models: []string{"claude-sonnet-5", "claude-haiku-4-5"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "api.anthropic.com",
}
t.Run("enumerated route bounds the listing", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{enumerated}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models"))
require.NoError(t, err)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, []string{"claude-sonnet-5", "claude-haiku-4-5"},
out.Mutations.RewriteUpstream.DiscoveryModels,
"the picker must be bounded by what the route authorises")
})
t.Run("catch-all route leaves the listing alone", func(t *testing.T) {
catchAll := enumerated
catchAll.Models = nil
mw := New(Config{Providers: []ProviderRoute{catchAll}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models"))
require.NoError(t, err)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Empty(t, out.Mutations.RewriteUpstream.DiscoveryModels,
"a route that claims every model cannot bound the upstream's list")
})
t.Run("per-model lookup is not a listing", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{enumerated}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models/claude-sonnet-5"))
require.NoError(t, err)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Empty(t, out.Mutations.RewriteUpstream.DiscoveryModels,
"the single-object lookup has no data array to filter")
})
}
// TestRouter_ModelDetailHonoursAllowlist pins that GET /v1/models/{id} is
// authorised against the model table. It carries no body model, so treating
// it as a model-less endpoint would let a caller confirm a model the route
// does not list — the listing itself is bounded to the allowlist, so the
// detail lookup must be too.
func TestRouter_ModelDetailHonoursAllowlist(t *testing.T) {
enumerated := ProviderRoute{
ID: "anthropic-prod",
Models: []string{"claude-sonnet-5"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "api.anthropic.com",
}
t.Run("allowlisted model routes and skips metering", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{enumerated}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models/claude-sonnet-5"))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "api.anthropic.com", out.Mutations.RewriteUpstream.Host)
nonInference, _ := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
assert.Equal(t, "true", nonInference, "a detail lookup spends no tokens")
})
t.Run("model outside the allowlist denies", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{enumerated}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models/claude-opus-5"))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"a model no route lists must not be confirmed by the detail lookup")
})
t.Run("dated id matches its undated registration", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{enumerated}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models/claude-sonnet-5-20250929"))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"a pinned release of an allowlisted family stays reachable")
})
t.Run("catch-all route still answers every lookup", func(t *testing.T) {
catchAll := enumerated
catchAll.Models = nil
mw := New(Config{Providers: []ProviderRoute{catchAll}})
out, err := mw.Invoke(context.Background(), newModellessInput("/v1/models/anything-at-all"))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"a gateway that enumerates nothing cannot refuse a lookup")
})
}
// TestRouter_NonInferenceRequiresReadMethod pins that the non-inference mark —
// which exempts a request from the token pre-flight — is reachable only by the
// read methods these endpoints actually use. A POST to the same path could
// carry an inference body, so it must not buy the exemption; it falls through
// to normal per-model routing instead, which denies when no model is named.
func TestRouter_NonInferenceRequiresReadMethod(t *testing.T) {
route := ProviderRoute{
ID: "gateway",
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "gateway.example.com",
}
for _, path := range []string{"/v1/models", "/v1/models/claude-sonnet-5", "/api/hello"} {
t.Run("POST "+path, func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(path)
in.Method = http.MethodPost
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"a write to a non-inference path must not route unmetered")
nonInference, _ := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
assert.NotEqual(t, "true", nonInference,
"only a read method may skip the token pre-flight")
})
}
t.Run("HEAD keeps the warm probe working", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(connectionWarmPath)
in.Method = http.MethodHead
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"the HEAD warm probe must still reach the upstream")
nonInference, _ := metaValue(t, out.Metadata, middleware.KeyLLMNonInference)
assert.Equal(t, "true", nonInference,
"the HEAD warm probe carries no model to meter")
})
}
// TestRouter_PinnedDatedModelStaysDistinct pins that a route registered
// against one dated Anthropic release does not claim another. Normalising
// both sides of the comparison made every dated build of a family
// interchangeable, so an operator who deliberately pinned a build would have
// served a different one — and with several such routes, declaration or path
// order would have decided which.
func TestRouter_PinnedDatedModelStaysDistinct(t *testing.T) {
pinned := ProviderRoute{
ID: "anthropic-pinned",
Vendor: "anthropic",
Models: []string{"claude-sonnet-4-5-20250101"},
AllowedGroupIDs: []string{defaultTestGroup},
UpstreamScheme: "https",
UpstreamHost: "pinned.example.com",
}
t.Run("a different dated release is not claimed", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{pinned}})
in := newInputWithModelAndURL("claude-sonnet-4-5-20250202", "/v1/messages")
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "anthropic"})
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"a route pinned to one dated build must not serve another")
})
t.Run("its own dated release still routes", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{pinned}})
in := newInputWithModelAndURL("claude-sonnet-4-5-20250101", "/v1/messages")
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "anthropic"})
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "the exact match must still route")
})
t.Run("two pinned builds each route to their own provider", func(t *testing.T) {
other := pinned
other.ID = "anthropic-pinned-newer"
other.Models = []string{"claude-sonnet-4-5-20250202"}
other.UpstreamHost = "newer.example.com"
mw := New(Config{Providers: []ProviderRoute{pinned, other}})
in := newInputWithModelAndURL("claude-sonnet-4-5-20250202", "/v1/messages")
in.Metadata = append(in.Metadata, middleware.KV{Key: middleware.KeyLLMProvider, Value: "anthropic"})
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.Equal(t, middleware.DecisionAllow, out.Decision)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Equal(t, "newer.example.com", out.Mutations.RewriteUpstream.Host,
"declaration order must not decide between two deliberately pinned builds")
})
}
// TestRouter_DiscoveryBoundToCallersPolicies pins that a model listing is
// bounded by the policies that authorise the caller, not by the union across
// everyone who can reach the provider. Two teams sharing one provider record
// under different allowlists is the case that makes the difference visible: a
// flattened per-provider list would offer each team the other's models, and
// every one of those entries is a request the guardrail then refuses.
func TestRouter_DiscoveryBoundToCallersPolicies(t *testing.T) {
const (
eng = "grp-eng"
sales = "grp-sales"
)
route := ProviderRoute{
ID: "shared-gateway",
Models: []string{"claude-sonnet-5", "claude-haiku-4-5", "gpt-4o"},
AllowedGroupIDs: []string{eng, sales},
UpstreamScheme: "https",
UpstreamHost: "gateway.example.com",
ModelPolicies: []ModelPolicyRule{
{GroupIDs: []string{eng}, Models: []string{"claude-sonnet-5"}},
{GroupIDs: []string{sales}, Models: []string{"gpt-4o"}},
},
}
listingFor := func(t *testing.T, group string) []string {
t.Helper()
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(modelListingPath)
in.UserGroups = []string{group}
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.Equal(t, middleware.DecisionAllow, out.Decision)
require.NotNil(t, out.Mutations)
require.NotNil(t, out.Mutations.RewriteUpstream)
return out.Mutations.RewriteUpstream.DiscoveryModels
}
t.Run("each group sees only its own policy's models", func(t *testing.T) {
assert.Equal(t, []string{"claude-sonnet-5"}, listingFor(t, eng),
"engineering must not be offered the model only sales may use")
assert.Equal(t, []string{"gpt-4o"}, listingFor(t, sales),
"sales must not be offered the model only engineering may use")
})
t.Run("a model no policy allows is offered to nobody", func(t *testing.T) {
for _, group := range []string{eng, sales} {
assert.NotContains(t, listingFor(t, group), "claude-haiku-4-5",
"the provider serves it, but no policy permits it")
}
})
}
// TestRouter_DiscoveryUnrestrictedPolicy covers the lifting rule: a caller
// holding one policy without a model allowlist sees everything the provider
// enumerates, whatever the other policies say.
func TestRouter_DiscoveryUnrestrictedPolicy(t *testing.T) {
const (
eng = "grp-eng"
admin = "grp-admin"
)
route := ProviderRoute{
ID: "shared-gateway",
Models: []string{"claude-sonnet-5", "gpt-4o"},
AllowedGroupIDs: []string{eng, admin},
UpstreamScheme: "https",
UpstreamHost: "gateway.example.com",
ModelPolicies: []ModelPolicyRule{
{GroupIDs: []string{eng}, Models: []string{"claude-sonnet-5"}},
// nil Models: a policy that sets no allowlist at all.
{GroupIDs: []string{admin}},
},
}
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(modelListingPath)
in.UserGroups = []string{eng, admin}
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.ElementsMatch(t, []string{"claude-sonnet-5", "gpt-4o"},
out.Mutations.RewriteUpstream.DiscoveryModels,
"an unrestricted policy the caller holds lifts the restriction")
}
// TestRouter_DiscoveryOnGatewayRecord covers a record that enumerates no
// models. It previously offered the upstream's whole catalogue however narrow
// the policy was, because there was nothing to intersect against; the policy
// allowlist is now the bound on its own.
func TestRouter_DiscoveryOnGatewayRecord(t *testing.T) {
const eng = "grp-eng"
base := ProviderRoute{
ID: "litellm",
AllowedGroupIDs: []string{eng},
UpstreamScheme: "https",
UpstreamHost: "litellm.internal",
}
t.Run("a policy allowlist bounds it", func(t *testing.T) {
route := base
route.ModelPolicies = []ModelPolicyRule{{GroupIDs: []string{eng}, Models: []string{"gpt-4o"}}}
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(modelListingPath)
in.UserGroups = []string{eng}
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, []string{"gpt-4o"}, out.Mutations.RewriteUpstream.DiscoveryModels,
"a catch-all record must still be bounded by what policy permits")
})
t.Run("an allowlist permitting nothing offers nothing", func(t *testing.T) {
route := base
route.ModelPolicies = []ModelPolicyRule{{GroupIDs: []string{eng}, Models: []string{}}}
mw := New(Config{Providers: []ProviderRoute{route}})
in := newModellessInput(modelListingPath)
in.UserGroups = []string{eng}
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Empty(t, out.Mutations.RewriteUpstream.DiscoveryModels,
"an empty allowlist permits nothing, and must not be read as unrestricted")
})
t.Run("no policy restriction leaves the listing alone", func(t *testing.T) {
mw := New(Config{Providers: []ProviderRoute{base}})
in := newModellessInput(modelListingPath)
in.UserGroups = []string{eng}
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
require.NotNil(t, out.Mutations.RewriteUpstream)
assert.Nil(t, out.Mutations.RewriteUpstream.DiscoveryModels,
"nothing narrows the listing, so the upstream's own answer passes through")
})
}

View File

@@ -11,11 +11,78 @@ var codeRegex = regexp.MustCompile(`^[a-z][a-z0-9._-]{0,63}$`)
// denyResponse is the on-wire shape rendered by RenderDenyResponse.
// Keeping this as a typed struct ensures we never leak
// middleware-supplied bytes outside known fields.
//
// Type and Error mirror the denial in the vendor's own error shape when
// the request reached a known LLM surface. LLM clients only parse their
// provider's envelope, so without the mirror a budget stop reaches the
// user as an unexplained API error. The NetBird fields stay where they
// were, so the body is a superset and existing consumers are unaffected.
type denyResponse struct {
Code string `json:"code"`
Message string `json:"message,omitempty"`
Details map[string]string `json:"details,omitempty"`
Middleware string `json:"middleware,omitempty"`
Type string `json:"type,omitempty"`
Error *providerError `json:"error,omitempty"`
}
// providerError is the nested error object both vendor envelopes carry.
type providerError struct {
Type string `json:"type"`
Message string `json:"message,omitempty"`
Code string `json:"code,omitempty"`
}
// Vendor error types keyed by HTTP status, per each provider's published
// error reference.
const (
anthropicErrInvalidRequest = "invalid_request_error"
anthropicErrPermission = "permission_error"
anthropicErrRateLimit = "rate_limit_error"
anthropicErrAPI = "api_error"
openAIErrInvalidRequest = "invalid_request_error"
openAIErrRateLimit = "rate_limit_error"
)
// providerEnvelope returns the vendor-shaped mirror for a denial on the
// given surface, or nil when the surface has no envelope we can speak.
// message is the already-redacted public message.
func providerEnvelope(surface, code, message string, status int) (string, *providerError) {
switch surface {
case "anthropic":
return "error", &providerError{
Type: anthropicErrorType(status),
Message: message,
}
case "openai":
return "", &providerError{
Type: openAIErrorType(status),
Message: message,
Code: code,
}
default:
return "", nil
}
}
func anthropicErrorType(status int) string {
switch status {
case http.StatusForbidden:
return anthropicErrPermission
case http.StatusTooManyRequests:
return anthropicErrRateLimit
case http.StatusBadRequest:
return anthropicErrInvalidRequest
default:
return anthropicErrAPI
}
}
func openAIErrorType(status int) string {
if status == http.StatusTooManyRequests {
return openAIErrRateLimit
}
return openAIErrInvalidRequest
}
// RenderDenyResponse writes a structured JSON deny body. Status is
@@ -36,6 +103,7 @@ func RenderDenyResponse(w http.ResponseWriter, middlewareID string, reason *Deny
Message: truncate(Scan(reason.Message), 256),
Middleware: truncate(Scan(middlewareID), 64),
}
resp.Type, resp.Error = providerEnvelope(reason.Surface, resp.Code, resp.Message, status)
if n := len(reason.Details); n > 0 {
resp.Details = make(map[string]string, min(n, 8))
for k, v := range reason.Details {

View File

@@ -0,0 +1,92 @@
package middleware
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// decodeDeny renders a denial and returns the parsed body plus the status.
func decodeDeny(t *testing.T, reason *DenyReason, status int) (map[string]any, int) {
t.Helper()
rec := httptest.NewRecorder()
RenderDenyResponse(rec, "llm_limit_check", reason, status)
var body map[string]any
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &body), "deny body must be valid JSON")
return body, rec.Code
}
// TestRenderDeny_AnthropicSurfaceMirrorsVendorShape covers a budget stop
// reaching Claude Code. The client only parses the Anthropic envelope, so
// without the mirror the user sees an unexplained API error instead of the
// reason their request was refused.
func TestRenderDeny_AnthropicSurfaceMirrorsVendorShape(t *testing.T) {
body, status := decodeDeny(t, &DenyReason{
Code: "llm_policy.budget_cap_exceeded",
Message: "LLM policy limit exceeded",
Surface: "anthropic",
}, http.StatusForbidden)
assert.Equal(t, http.StatusForbidden, status)
assert.Equal(t, "error", body["type"], "Anthropic errors carry type=error at the top level")
errObj, ok := body["error"].(map[string]any)
require.True(t, ok, "error must be an object")
assert.Equal(t, "permission_error", errObj["type"], "403 maps to permission_error")
assert.Equal(t, "LLM policy limit exceeded", errObj["message"])
// The NetBird fields stay put so existing consumers keep working.
assert.Equal(t, "llm_policy.budget_cap_exceeded", body["code"])
assert.Equal(t, "LLM policy limit exceeded", body["message"])
assert.Equal(t, "llm_limit_check", body["middleware"])
}
// TestRenderDeny_OpenAISurfaceMirrorsVendorShape pins the OpenAI envelope,
// which nests the code and carries no top-level type.
func TestRenderDeny_OpenAISurfaceMirrorsVendorShape(t *testing.T) {
body, _ := decodeDeny(t, &DenyReason{
Code: "llm_policy.model_blocked",
Message: "model is not in the policy allowlist",
Surface: "openai",
}, http.StatusForbidden)
assert.NotContains(t, body, "type", "OpenAI errors have no top-level type")
errObj, ok := body["error"].(map[string]any)
require.True(t, ok, "error must be an object")
assert.Equal(t, "invalid_request_error", errObj["type"])
assert.Equal(t, "llm_policy.model_blocked", errObj["code"], "the NetBird code rides in the vendor code field")
assert.Equal(t, "model is not in the policy allowlist", errObj["message"])
}
// TestRenderDeny_RateLimitStatusMapsToVendorRateLimit pins the mapping a
// client's backoff keys on.
func TestRenderDeny_RateLimitStatusMapsToVendorRateLimit(t *testing.T) {
body, status := decodeDeny(t, &DenyReason{
Code: "llm_policy.token_cap_exceeded",
Message: "LLM policy limit exceeded",
Surface: "anthropic",
}, http.StatusTooManyRequests)
assert.Equal(t, http.StatusTooManyRequests, status, "429 must survive the status clamp")
errObj := body["error"].(map[string]any)
assert.Equal(t, "rate_limit_error", errObj["type"])
}
// TestRenderDeny_NoSurfaceKeepsLegacyShape guards non-LLM middlewares and
// denials raised before a surface is known.
func TestRenderDeny_NoSurfaceKeepsLegacyShape(t *testing.T) {
body, _ := decodeDeny(t, &DenyReason{
Code: "llm_policy.model_not_routable",
Message: "no provider configured for model x",
}, http.StatusForbidden)
assert.NotContains(t, body, "type", "no surface means no vendor mirror")
assert.NotContains(t, body, "error", "no surface means no vendor mirror")
assert.Equal(t, "llm_policy.model_not_routable", body["code"])
}

View File

@@ -22,6 +22,15 @@ const (
// body. Empty for clients that don't send one.
KeyLLMSessionID = "llm.session_id"
// Sub-agent attribution (emitted by llm_request_parser from the
// client's request headers). A coding agent that spawns helpers
// stamps the spawned agent's id, and the spawning agent's id when
// the helper is itself nested, so cost within one session can be
// split across the agents that ran in parallel. These identify an
// agent, not a person or a device: never treat them as a user id.
KeyLLMAgentID = "llm.agent_id"
KeyLLMParentAgentID = "llm.parent_agent_id"
// LLM response-side metadata (emitted by llm_response_parser).
//nolint:gosec // metadata key name, not a credential
KeyLLMInputTokens = "llm.input_tokens"
@@ -66,6 +75,14 @@ const (
// downstream gateways' spend logs.
KeyLLMAuthorisingGroups = "llm.authorising_groups"
// LLM non-inference marker (emitted by llm_router on the allow path
// for endpoints that legitimately carry no model, such as model
// listing). The router still authorises these against the caller's
// groups; the marker only tells the limits gate that a per-model
// allowlist has nothing to evaluate, so an empty model must not be
// read as an undetermined one. Never derived from client input.
KeyLLMNonInference = "llm.non_inference"
// LLM policy attribution (emitted by llm_limit_check on the allow
// path). Names the policy that paid for this request and the
// dimension counters the post-flight llm_limit_record middleware

View File

@@ -179,6 +179,12 @@ type DenyReason struct {
Code string
Message string
Details map[string]string
// Surface names the LLM API dialect the caller speaks (the
// llm.provider value), so the rendered body can mirror the denial in
// that vendor's error shape alongside the NetBird fields. Empty for
// non-LLM middlewares and for denials raised before a surface was
// resolved; the body then carries the NetBird fields alone.
Surface string
}
// Output is the value each middleware returns to the dispatcher. The
@@ -247,6 +253,12 @@ type UpstreamRewrite struct {
// without verifying its TLS certificate. Set by llm_router from the
// provider's skip_tls_verification for self-hosted / internal gateways.
SkipTLSVerify bool
// DiscoveryModels, when non-empty, is the set of model ids the resolved
// route authorises, and the proxy drops everything else from the
// model-listing response. Empty leaves the upstream's list untouched,
// which is what a route that claims every model wants. Set by
// llm_router on a model-listing request only.
DiscoveryModels []string
}
// AuthHeader is a single name/value pair the proxy injects on the

View File

@@ -0,0 +1,191 @@
package proxy
import (
"bytes"
"encoding/json"
"io"
"net/http"
"strconv"
"strings"
sharedllm "github.com/netbirdio/netbird/shared/llm"
)
// maxDiscoveryBodyBytes bounds the model-listing response the filter will
// buffer. A listing is a few kilobytes of ids; anything larger is not a
// listing we recognise, and buffering it to rewrite would cost more than
// the filtering is worth.
const maxDiscoveryBodyBytes = 1 << 20
// modelDiscoveryFilter returns a ModifyResponse hook that drops models the
// caller's policy does not authorise from a model-listing response, then
// delegates to next (which may be nil).
//
// Clients populate their model picker from this endpoint, so an unfiltered
// list offers models the very next request denies. The filter is
// best-effort: a response it cannot safely rewrite passes through
// untouched rather than reaching the client corrupted.
func modelDiscoveryFilter(allowed []string, next func(*http.Response) error) func(*http.Response) error {
permitted := make(map[string]struct{}, len(allowed)*2)
for _, id := range allowed {
permitted[id] = struct{}{}
permitted[sharedllm.NormalizeAnthropicModel(id)] = struct{}{}
}
return func(resp *http.Response) error {
if err := filterModelListing(resp, permitted); err != nil {
return err
}
if next == nil {
return nil
}
return next(resp)
}
}
// filterModelListing rewrites the response body in place, keeping only the
// entries whose id the policy authorises. Responses that are not a plain
// JSON listing are left alone.
func filterModelListing(resp *http.Response, permitted map[string]struct{}) error {
if !isPlainJSONListing(resp) {
return nil
}
// One byte past the cap, so an oversized body is detectable without
// buffering all of it.
body, err := io.ReadAll(io.LimitReader(resp.Body, maxDiscoveryBodyBytes+1))
if err != nil {
_ = resp.Body.Close()
return err
}
if len(body) > maxDiscoveryBodyBytes {
// Too large to filter. Put the bytes already read back in front of the
// unread remainder and forward the response exactly as the upstream
// sent it, headers included. Buffering what was read and closing here
// would truncate the body at the cap and hand the client a short,
// invalid listing — worse than not filtering at all.
resp.Body = spliceBody(body, resp.Body)
return nil
}
if err := resp.Body.Close(); err != nil {
return err
}
filtered, ok := filterListingBody(body, permitted)
if !ok {
restoreBody(resp, body)
return nil
}
restoreBody(resp, filtered)
return nil
}
// isPlainJSONListing reports whether the response is a JSON body the filter
// can parse. A content-encoded body is skipped: the transport only
// transparently decompresses what it negotiated itself, and the client
// negotiates its own encoding on this request.
func isPlainJSONListing(resp *http.Response) bool {
if resp == nil || resp.Body == nil {
return false
}
if resp.StatusCode != http.StatusOK {
return false
}
if enc := resp.Header.Get("Content-Encoding"); enc != "" && !strings.EqualFold(enc, "identity") {
return false
}
return strings.Contains(strings.ToLower(resp.Header.Get("Content-Type")), "application/json")
}
// filterListingBody returns the listing with unauthorised entries removed.
// ok is false when the body is not a listing shape, in which case the
// caller must forward the original bytes.
func filterListingBody(body []byte, permitted map[string]struct{}) ([]byte, bool) {
var doc map[string]json.RawMessage
if err := json.Unmarshal(body, &doc); err != nil {
return nil, false
}
raw, present := doc["data"]
if !present {
return nil, false
}
var entries []map[string]json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return nil, false
}
kept := make([]map[string]json.RawMessage, 0, len(entries))
for _, entry := range entries {
if entryPermitted(entry, permitted) {
kept = append(kept, entry)
}
}
encoded, err := json.Marshal(kept)
if err != nil {
return nil, false
}
doc["data"] = encoded
out, err := json.Marshal(doc)
if err != nil {
return nil, false
}
return out, true
}
// entryPermitted reports whether a listing entry names a model the policy
// authorises, trying every form the same model is written in.
func entryPermitted(entry map[string]json.RawMessage, permitted map[string]struct{}) bool {
raw, ok := entry["id"]
if !ok {
return false
}
var id string
if err := json.Unmarshal(raw, &id); err != nil {
return false
}
for _, candidate := range modelIDForms(id) {
if _, ok := permitted[candidate]; ok {
return true
}
}
return false
}
// modelIDForms returns the forms a single model id may be written in: the id
// itself, its undated form, and the same two with a gateway's provider
// prefix removed ("vertex_ai/claude-sonnet-5"). The bare id is tried first,
// because a self-hosted id can legitimately contain a slash of its own
// ("Qwen/Qwen2.5-0.5B-Instruct") and must not be cut down to its tail.
func modelIDForms(id string) []string {
if id == "" {
return nil
}
forms := []string{id, sharedllm.NormalizeAnthropicModel(id)}
if slash := strings.LastIndex(id, "/"); slash >= 0 {
tail := id[slash+1:]
forms = append(forms, tail, sharedllm.NormalizeAnthropicModel(tail))
}
return forms
}
// restoreBody puts body back on the response and fixes the length headers
// so the client reads exactly what is there.
// spliceBody returns a ReadCloser that yields prefix followed by whatever is
// left in rest, closing rest when closed. It lets the filter put back bytes it
// consumed while deciding, without owning the rest of the stream.
func spliceBody(prefix []byte, rest io.ReadCloser) io.ReadCloser {
return struct {
io.Reader
io.Closer
}{
Reader: io.MultiReader(bytes.NewReader(prefix), rest),
Closer: rest,
}
}
func restoreBody(resp *http.Response, body []byte) {
resp.Body = io.NopCloser(bytes.NewReader(body))
resp.ContentLength = int64(len(body))
resp.Header.Set("Content-Length", strconv.Itoa(len(body)))
}

View File

@@ -0,0 +1,217 @@
package proxy
import (
"bytes"
"encoding/json"
"io"
"net/http"
"strconv"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// jsonListingResponse builds a 200 model-listing response with the given
// body, as an upstream would return it.
func jsonListingResponse(body string) *http.Response {
resp := &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{},
Body: io.NopCloser(strings.NewReader(body)),
ContentLength: int64(len(body)),
}
resp.Header.Set("Content-Type", "application/json")
return resp
}
// listedIDs runs the filter and returns the ids left in the response.
func listedIDs(t *testing.T, allowed []string, body string) []string {
t.Helper()
resp := jsonListingResponse(body) //nolint:bodyclose // in-memory body, replaced by the filter
require.NoError(t, modelDiscoveryFilter(allowed, nil)(resp)) //nolint:bodyclose // in-memory body, replaced by the filter
raw, err := io.ReadAll(resp.Body)
require.NoError(t, err)
var doc struct {
Data []struct {
ID string `json:"id"`
} `json:"data"`
}
require.NoError(t, json.Unmarshal(raw, &doc), "filtered body must stay valid JSON")
ids := make([]string, 0, len(doc.Data))
for _, entry := range doc.Data {
ids = append(ids, entry.ID)
}
return ids
}
// TestModelDiscoveryFilter_KeepsOnlyAuthorisedModels covers the picker a
// developer sees: an unfiltered upstream list offers every model the shared
// key can reach, and each one the policy excludes is a request the chain
// denies a moment later.
func TestModelDiscoveryFilter_KeepsOnlyAuthorisedModels(t *testing.T) {
ids := listedIDs(t, []string{"claude-sonnet-5", "claude-haiku-4-5"}, `{
"data": [
{"id": "claude-opus-5", "display_name": "Claude Opus 5"},
{"id": "claude-sonnet-5", "display_name": "Claude Sonnet 5"},
{"id": "claude-haiku-4-5"}
],
"has_more": false
}`)
assert.Equal(t, []string{"claude-sonnet-5", "claude-haiku-4-5"}, ids,
"only the models the route authorises may reach the picker")
}
// TestModelDiscoveryFilter_MatchesDatedAndPrefixedIDs pins the two id forms
// a gateway returns for a model the operator registered plainly.
func TestModelDiscoveryFilter_MatchesDatedAndPrefixedIDs(t *testing.T) {
ids := listedIDs(t, []string{"claude-sonnet-4-5", "anthropic.claude-opus-5"}, `{
"data": [
{"id": "claude-sonnet-4-5-20250929"},
{"id": "bedrock/anthropic.claude-opus-5"},
{"id": "gpt-4o"}
]
}`)
assert.Equal(t, []string{"claude-sonnet-4-5-20250929", "bedrock/anthropic.claude-opus-5"}, ids,
"a dated or provider-prefixed id must match its registered form")
}
// TestModelDiscoveryFilter_PreservesEnvelopeFields guards the rest of the
// document: clients read paging fields alongside data.
func TestModelDiscoveryFilter_PreservesEnvelopeFields(t *testing.T) {
resp := jsonListingResponse(`{"data":[{"id":"claude-sonnet-5"}],"has_more":true,"first_id":"x"}`) //nolint:bodyclose // in-memory body, replaced by the filter
require.NoError(t, modelDiscoveryFilter([]string{"claude-sonnet-5"}, nil)(resp)) //nolint:bodyclose // in-memory body, replaced by the filter
raw, err := io.ReadAll(resp.Body)
require.NoError(t, err)
var doc map[string]any
require.NoError(t, json.Unmarshal(raw, &doc))
assert.Equal(t, true, doc["has_more"], "paging fields must survive the rewrite")
assert.Equal(t, "x", doc["first_id"])
assert.Equal(t, strconv.Itoa(len(raw)), resp.Header.Get("Content-Length"),
"Content-Length must match the rewritten body")
}
// TestModelDiscoveryFilter_PassesThroughUnfilterable covers the responses
// the filter must not touch: a compressed body it cannot parse, a non-JSON
// body, an error status, and a document with no data array.
func TestModelDiscoveryFilter_PassesThroughUnfilterable(t *testing.T) {
cases := map[string]func() *http.Response{
"compressed": func() *http.Response {
resp := jsonListingResponse(`{"data":[{"id":"gpt-4o"}]}`)
resp.Header.Set("Content-Encoding", "gzip")
return resp
},
"not json": func() *http.Response {
resp := jsonListingResponse(`{"data":[{"id":"gpt-4o"}]}`)
resp.Header.Set("Content-Type", "text/html")
return resp
},
"error status": func() *http.Response {
resp := jsonListingResponse(`{"data":[{"id":"gpt-4o"}]}`)
resp.StatusCode = http.StatusInternalServerError
return resp
},
"no data array": func() *http.Response {
return jsonListingResponse(`{"object":"list"}`)
},
}
for name, build := range cases {
t.Run(name, func(t *testing.T) {
resp := build() //nolint:bodyclose // in-memory body, replaced by the filter
original, err := io.ReadAll(resp.Body)
require.NoError(t, err)
resp.Body = io.NopCloser(bytes.NewReader(original))
require.NoError(t, modelDiscoveryFilter([]string{"claude-sonnet-5"}, nil)(resp)) //nolint:bodyclose // in-memory body, replaced by the filter
got, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, string(original), string(got), "an unfilterable response must reach the client unchanged")
})
}
}
// TestModelDiscoveryFilter_RunsNextHook pins that an existing
// ModifyResponse hook still runs after filtering.
func TestModelDiscoveryFilter_RunsNextHook(t *testing.T) {
called := false
next := func(*http.Response) error {
called = true
return nil
}
resp := jsonListingResponse(`{"data":[{"id":"claude-sonnet-5"}]}`) //nolint:bodyclose // in-memory body, replaced by the filter
require.NoError(t, modelDiscoveryFilter([]string{"claude-sonnet-5"}, next)(resp)) //nolint:bodyclose // in-memory body, replaced by the filter
assert.True(t, called, "the chained hook must still run")
}
// TestModelDiscoveryFilter_KeepsSlashBearingIDs covers self-hosted backends
// whose model ids carry a slash of their own. Treating the slash as a
// gateway prefix and keeping only the tail dropped every such model from
// the picker even though the policy named it exactly.
func TestModelDiscoveryFilter_KeepsSlashBearingIDs(t *testing.T) {
ids := listedIDs(t, []string{"Qwen/Qwen2.5-0.5B-Instruct"}, `{
"object": "list",
"data": [
{"id": "Qwen/Qwen2.5-0.5B-Instruct"},
{"id": "Qwen/Qwen2.5-7B-Instruct"}
]
}`)
assert.Equal(t, []string{"Qwen/Qwen2.5-0.5B-Instruct"}, ids,
"a slash inside the model id is part of the id, not a provider prefix")
}
// TestModelDiscoveryFilter_ForwardsOversizedBodyIntact covers a listing past
// the buffering cap. The filter reads one byte beyond the cap to detect the
// size; forwarding only what it read would hand the client a body truncated
// at exactly 1 MiB — valid-looking, short, and unparseable as JSON. The bytes
// already read must be spliced back in front of the unread remainder so the
// response reaches the client exactly as the upstream sent it.
func TestModelDiscoveryFilter_ForwardsOversizedBodyIntact(t *testing.T) {
// A well-formed listing whose single entry pads the body past the cap.
padding := strings.Repeat("x", maxDiscoveryBodyBytes)
body := `{"object":"list","data":[{"id":"gpt-4o","note":"` + padding + `"}]}`
require.Greater(t, len(body), maxDiscoveryBodyBytes+1,
"the fixture must exceed the cap by more than the one-byte probe")
resp := jsonListingResponse(body) //nolint:bodyclose // in-memory body
require.NoError(t, modelDiscoveryFilter(nil, nil)(resp)) //nolint:bodyclose // in-memory body
got, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, len(body), len(got),
"an oversized listing must reach the client whole, not truncated at the cap")
assert.Equal(t, body, string(got), "the forwarded bytes must be the upstream's own")
var doc map[string]json.RawMessage
assert.NoError(t, json.Unmarshal(got, &doc),
"the forwarded body must still parse as JSON")
}
// TestModelDiscoveryFilter_OversizedBodyKeepsUpstreamHeaders pins that the
// oversized path leaves the response metadata alone. Rewriting Content-Length
// to the truncated prefix is what made the corruption invisible to the client
// until it tried to parse.
func TestModelDiscoveryFilter_OversizedBodyKeepsUpstreamHeaders(t *testing.T) {
padding := strings.Repeat("x", maxDiscoveryBodyBytes)
body := `{"object":"list","data":[{"id":"gpt-4o","note":"` + padding + `"}]}`
resp := jsonListingResponse(body) //nolint:bodyclose // in-memory body
resp.Header.Set("Content-Length", strconv.Itoa(len(body)))
require.NoError(t, modelDiscoveryFilter(nil, nil)(resp)) //nolint:bodyclose // in-memory body
assert.Equal(t, int64(len(body)), resp.ContentLength,
"ContentLength must keep describing the body the client receives")
assert.Equal(t, strconv.Itoa(len(body)), resp.Header.Get("Content-Length"),
"the Content-Length header must not be rewritten to the truncated prefix")
}

View File

@@ -363,6 +363,9 @@ func (p *ReverseProxy) forwardUpstream(respWriter http.ResponseWriter, r *http.R
if result.rewriteRedirects {
rp.ModifyResponse = p.rewriteLocationFunc(effectiveURL, rewriteMatchedPath, r) //nolint:bodyclose
}
if upstreamRewrite != nil && len(upstreamRewrite.DiscoveryModels) > 0 {
rp.ModifyResponse = modelDiscoveryFilter(upstreamRewrite.DiscoveryModels, rp.ModifyResponse) //nolint:bodyclose // the hook replaces the body and closes the original
}
rp.ServeHTTP(respWriter, r.WithContext(ctx))
}

View File

@@ -46,6 +46,27 @@ func NormalizeBedrockModel(modelID string) string {
return bedrockVersionSuffix.ReplaceAllString(m, "")
}
// anthropicDatedModel matches a Claude model id carrying the trailing
// "-YYYYMMDD" release-date suffix Anthropic appends to a pinned release,
// capturing the id without it. The "claude" anchor is load-bearing: pricing
// looks every model up through this helper regardless of surface, and an
// operator may register a custom id with any shape at all, so an unanchored
// "-\d{8}$" would let "internal-llm-20250101" silently inherit the rate
// registered for "internal-llm". The anchor also covers the vendor-prefixed
// forms ("anthropic.claude-...", "us.anthropic.claude-...").
var anthropicDatedModel = regexp.MustCompile(`(?i)^(.*claude.*)-\d{8}$`)
// NormalizeAnthropicModel strips the trailing release-date suffix from a
// Claude model id, e.g. "claude-sonnet-4-5-20250929" -> "claude-sonnet-4-5",
// so a dated id a client pins matches the undated one the operator
// registered. Ids that are not Claude-family are returned untouched.
// Callers try the verbatim id first and fall back to this, so two dated
// releases of the same family stay distinct wherever both are registered
// explicitly.
func NormalizeAnthropicModel(modelID string) string {
return anthropicDatedModel.ReplaceAllString(modelID, "$1")
}
// NormalizeVertexModel strips the "@version" suffix from a Vertex AI model id
// (e.g. "claude-sonnet-4-5@20250929" -> "claude-sonnet-4-5") so it matches
// the catalog/pricing key. Vertex publisher models are priced under their

View File

@@ -34,3 +34,29 @@ func TestNormalizeVertexModel(t *testing.T) {
require.Equal(t, want, NormalizeVertexModel(in), "normalize %q", in)
}
}
func TestNormalizeAnthropicModel(t *testing.T) {
cases := map[string]string{
"claude-sonnet-4-5-20250929": "claude-sonnet-4-5",
"claude-3-5-haiku-20241022": "claude-3-5-haiku",
"claude-sonnet-5": "claude-sonnet-5",
"claude-opus-4-8": "claude-opus-4-8",
"anthropic.claude-haiku-4-5": "anthropic.claude-haiku-4-5",
"anthropic.claude-sonnet-4-5-20250929": "anthropic.claude-sonnet-4-5",
"us.anthropic.claude-opus-4-8-20250101": "us.anthropic.claude-opus-4-8",
// Non-Claude ids must survive untouched even when they end in eight
// consecutive digits: an operator can register a custom model under
// any id, and pricing looks every one of them up through this helper.
"gpt-4o": "gpt-4o",
"gpt-4o-2024-08-06": "gpt-4o-2024-08-06",
"gpt-4o-20240806": "gpt-4o-20240806",
"internal-llm-20250101": "internal-llm-20250101",
"deepseek-r1-20250120": "deepseek-r1-20250120",
"Qwen/Qwen2.5-20250101": "Qwen/Qwen2.5-20250101",
"gemini-2-5-pro-20250101": "gemini-2-5-pro-20250101",
"": "",
}
for in, want := range cases {
require.Equal(t, want, NormalizeAnthropicModel(in), "normalize %q", in)
}
}

View File

@@ -1433,13 +1433,14 @@ components:
enum: [ "all", "tcp", "udp", "icmp", "netbird-ssh" ]
example: "tcp"
ports:
description: Policy rule affected ports
description: Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
x-omit-from-example: true
type: array
items:
type: string
example: "80"
port_ranges:
description: Policy rule affected ports ranges list
description: Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
type: array
items:
$ref: '#/components/schemas/RulePortRange'
@@ -1459,7 +1460,7 @@ components:
- action
RulePortRange:
description: Policy rule affected ports range
description: Policy rule affected ports range. A range with identical start and end values represents a single port.
type: object
properties:
start:
@@ -5334,6 +5335,57 @@ components:
- input_per_1k
- output_per_1k
- context_window
AgentNetworkModelDiscoveryRequest:
type: object
properties:
catalog_provider_id:
type: string
description: Catalog provider to query (AgentNetworkCatalogProvider.id). Determines the listing endpoint, the auth header and the response shape.
example: "bedrock_api"
upstream_url:
type: string
description: |
The upstream being configured. Used to reach vendors that serve their listing from the same host as inference, and to read back the region for those whose host embeds one. Ignored when provider_id is supplied.
example: "https://bedrock-runtime.eu-central-1.amazonaws.com"
api_key:
type: string
description: Credential to query the vendor with, for a provider that has not been saved yet. Mutually exclusive with provider_id.
example: "sk-..."
provider_id:
type: string
description: Existing Agent Network provider record whose stored credential and upstream should be used. Lets the form refresh the list without the client holding the key.
example: "ch8i4ug6lnn4g9hqv7m0"
required:
- catalog_provider_id
AgentNetworkModelDiscoveryResponse:
type: object
properties:
models:
type: array
description: Models the credential can reach, in the order the vendor returned them.
items:
$ref: '#/components/schemas/AgentNetworkDiscoveredModel'
required:
- models
AgentNetworkDiscoveredModel:
type: object
properties:
id:
type: string
description: |
Identifier to register on the provider record, in the form the vendor issues it. For Bedrock this is the region-prefixed inference-profile id, which is the only form AWS accepts at invoke time.
example: "eu.anthropic.claude-haiku-4-5-20251001-v1:0"
label:
type: string
description: Vendor-supplied display name, where the vendor supplies one.
example: "EU Anthropic Claude Haiku 4.5"
pricing_known:
type: boolean
description: Whether NetBird's shipped pricing table can price this model. When false the operator must set input/output rates, or requests to it would record a cost of zero.
example: true
required:
- id
- pricing_known
AgentNetworkCatalogProvider:
type: object
properties:
@@ -14003,6 +14055,42 @@ paths:
"$ref": "#/components/responses/forbidden"
'500':
"$ref": "#/components/responses/internal_error"
/api/agent-network/catalog/providers/models:
post:
summary: Discover the models a provider credential can reach
description: |
Asks the vendor which models the supplied credential can actually use, so the provider form can offer a live list instead of only the static catalog. The endpoint, auth header and response shape are taken from the catalog entry, never from the request.
Supply either an api_key together with the upstream_url being configured (before the provider is saved), or a provider_id of an existing record to reuse its stored credential.
Returns 422 for a catalog provider that has no listing endpoint (most gateways); the caller should fall back to the catalog's own model list. A model whose price the shipped table does not know is returned with pricing_known false, and the operator must set rates for it.
tags: [ Agent Network ]
security:
- BearerAuth: [ ]
- TokenAuth: [ ]
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/AgentNetworkModelDiscoveryRequest'
responses:
'200':
description: The models the credential can reach
content:
application/json:
schema:
$ref: '#/components/schemas/AgentNetworkModelDiscoveryResponse'
'400':
"$ref": "#/components/responses/bad_request"
'401':
"$ref": "#/components/responses/requires_authentication"
'403':
"$ref": "#/components/responses/forbidden"
'422':
"$ref": "#/components/responses/validation_failed_simple"
'500':
"$ref": "#/components/responses/internal_error"
/api/agent-network/providers:
get:
summary: List all Agent Network Providers

View File

@@ -2120,6 +2120,18 @@ type AgentNetworkConsumption struct {
// AgentNetworkConsumptionDimensionKind Whether this row counts a single end user or a single source group across every member.
type AgentNetworkConsumptionDimensionKind string
// AgentNetworkDiscoveredModel defines model for AgentNetworkDiscoveredModel.
type AgentNetworkDiscoveredModel struct {
// Id Identifier to register on the provider record, in the form the vendor issues it. For Bedrock this is the region-prefixed inference-profile id, which is the only form AWS accepts at invoke time.
Id string `json:"id"`
// Label Vendor-supplied display name, where the vendor supplies one.
Label *string `json:"label,omitempty"`
// PricingKnown Whether NetBird's shipped pricing table can price this model. When false the operator must set input/output rates, or requests to it would record a cost of zero.
PricingKnown bool `json:"pricing_known"`
}
// AgentNetworkGuardrail defines model for AgentNetworkGuardrail.
type AgentNetworkGuardrail struct {
// Checks Guardrail check parameters. Each entry has an `enabled` flag plus per-check configuration; disabled entries are inert.
@@ -2167,6 +2179,27 @@ type AgentNetworkGuardrailRequest struct {
Name string `json:"name"`
}
// AgentNetworkModelDiscoveryRequest defines model for AgentNetworkModelDiscoveryRequest.
type AgentNetworkModelDiscoveryRequest struct {
// ApiKey Credential to query the vendor with, for a provider that has not been saved yet. Mutually exclusive with provider_id.
ApiKey *string `json:"api_key,omitempty"`
// CatalogProviderId Catalog provider to query (AgentNetworkCatalogProvider.id). Determines the listing endpoint, the auth header and the response shape.
CatalogProviderId string `json:"catalog_provider_id"`
// ProviderId Existing Agent Network provider record whose stored credential and upstream should be used. Lets the form refresh the list without the client holding the key.
ProviderId *string `json:"provider_id,omitempty"`
// UpstreamUrl The upstream being configured. Used to reach vendors that serve their listing from the same host as inference, and to read back the region for those whose host embeds one. Ignored when provider_id is supplied.
UpstreamUrl *string `json:"upstream_url,omitempty"`
}
// AgentNetworkModelDiscoveryResponse defines model for AgentNetworkModelDiscoveryResponse.
type AgentNetworkModelDiscoveryResponse struct {
// Models Models the credential can reach, in the order the vendor returned them.
Models []AgentNetworkDiscoveredModel `json:"models"`
}
// AgentNetworkPolicy defines model for AgentNetworkPolicy.
type AgentNetworkPolicy struct {
// CreatedAt Timestamp when the policy was created.
@@ -4468,10 +4501,10 @@ type PolicyRule struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4508,10 +4541,10 @@ type PolicyRuleMinimum struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4551,10 +4584,10 @@ type PolicyRuleUpdate struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4962,7 +4995,7 @@ type RouteRequest struct {
SkipAutoApply *bool `json:"skip_auto_apply,omitempty"`
}
// RulePortRange Policy rule affected ports range
// RulePortRange Policy rule affected ports range. A range with identical start and end values represents a single port.
type RulePortRange struct {
// End The ending port of the range
End int `json:"end"`
@@ -6179,6 +6212,9 @@ type PostApiAgentNetworkBudgetRulesJSONRequestBody = AgentNetworkBudgetRuleReque
// PutApiAgentNetworkBudgetRulesRuleIdJSONRequestBody defines body for PutApiAgentNetworkBudgetRulesRuleId for application/json ContentType.
type PutApiAgentNetworkBudgetRulesRuleIdJSONRequestBody = AgentNetworkBudgetRuleRequest
// PostApiAgentNetworkCatalogProvidersModelsJSONRequestBody defines body for PostApiAgentNetworkCatalogProvidersModels for application/json ContentType.
type PostApiAgentNetworkCatalogProvidersModelsJSONRequestBody = AgentNetworkModelDiscoveryRequest
// PostApiAgentNetworkGuardrailsJSONRequestBody defines body for PostApiAgentNetworkGuardrails for application/json ContentType.
type PostApiAgentNetworkGuardrailsJSONRequestBody = AgentNetworkGuardrailRequest

View File

@@ -3,18 +3,69 @@ package util
import (
"os"
"os/exec"
"runtime"
"slices"
"github.com/skratchdot/open-golang/open"
)
const (
// envBrowser overrides the browser OpenBrowser launches
envBrowser = "BROWSER"
// envDesktopSession and envXDGCurrentDesktop are what xdg-open uses to pick a handler
envDesktopSession = "DESKTOP_SESSION"
envXDGCurrentDesktop = "XDG_CURRENT_DESKTOP"
// envDisplay and envWaylandDisplay are what a graphical browser needs to reach a display
envDisplay = "DISPLAY"
envWaylandDisplay = "WAYLAND_DISPLAY"
// envXDGSessionType names the session kind, e.g. tty, x11 or wayland
envXDGSessionType = "XDG_SESSION_TYPE"
)
// OpenBrowser opens the URL in a browser, respecting the BROWSER environment variable.
func OpenBrowser(url string) error {
if browser := os.Getenv("BROWSER"); browser != "" {
if browser := os.Getenv(envBrowser); browser != "" {
return exec.Command(browser, url).Start()
}
return open.Run(url)
}
// browserSessionEnvVars returns the variables that decide whether OpenBrowser can open a URL.
// DISPLAY and WAYLAND_DISPLAY are exactly what xdg-open's own has_display() checks, and without
// them it degrades to terminal browsers. BROWSER is the explicit override both xdg-open and
// OpenBrowser honor first. DESKTOP_SESSION and XDG_CURRENT_DESKTOP only tell xdg-open which
// desktop-specific opener to prefer, so they are weaker evidence, kept because the previous
// detection relied on them alone and dropping them would demote sessions that work today.
func browserSessionEnvVars() []string {
return []string{envDisplay, envWaylandDisplay, envBrowser, envDesktopSession, envXDGCurrentDesktop}
}
// graphicalXDGSessionTypes are the systemd-logind session types that come with a display. The
// other documented values are "tty" and "unspecified"; anything unrecognized is treated as no
// display, so an unknown value picks the device code flow, which works without a browser.
func graphicalXDGSessionTypes() []string {
return []string{"x11", "wayland", "mir"}
}
// HasGraphicalSession reports whether this process can open a browser and serve a loopback
// redirect back to it. Windows and macOS always can. On Linux and FreeBSD the answer is env
// based, so it only holds for a process started from the graphical session itself: a service
// does not inherit those variables and always reports false, which is why callers running in
// the user's session pass their own answer to the daemon.
func HasGraphicalSession() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return true
}
for _, env := range browserSessionEnvVars() {
if os.Getenv(env) != "" {
return true
}
}
return slices.Contains(graphicalXDGSessionTypes(), os.Getenv(envXDGSessionType))
}
// SliceDiff returns the elements in slice `x` that are not in slice `y`
func SliceDiff(x, y []string) []string {
mapY := make(map[string]struct{}, len(y))

50
util/session_test.go Normal file
View File

@@ -0,0 +1,50 @@
package util
import (
"os"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
)
func TestHasGraphicalSession(t *testing.T) {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
assert.True(t, HasGraphicalSession(), "%s always has a graphical session", runtime.GOOS)
return
}
// clear anything inherited from the session running the test, restored on cleanup
for _, env := range append(browserSessionEnvVars(), envXDGSessionType) {
t.Setenv(env, "")
os.Unsetenv(env)
}
assert.False(t, HasGraphicalSession(), "no session variables means no graphical session")
tests := []struct {
env string
value string
expected bool
}{
{env: envDisplay, value: ":0", expected: true},
{env: envWaylandDisplay, value: "wayland-0", expected: true},
{env: envDesktopSession, value: "gnome", expected: true},
{env: envXDGCurrentDesktop, value: "KDE", expected: true},
{env: envBrowser, value: "firefox", expected: true},
{env: envXDGSessionType, value: "wayland", expected: true},
{env: envXDGSessionType, value: "x11", expected: true},
{env: envXDGSessionType, value: "mir", expected: true},
{env: envXDGSessionType, value: "tty", expected: false},
{env: envXDGSessionType, value: "unspecified", expected: false},
// an unrecognized type must not be read as a display: the device code flow works anyway
{env: envXDGSessionType, value: "something-new", expected: false},
}
for _, tt := range tests {
t.Run(tt.env+"="+tt.value, func(t *testing.T) {
t.Setenv(tt.env, tt.value)
assert.Equal(t, tt.expected, HasGraphicalSession(), "%s=%s", tt.env, tt.value)
})
}
}