[proxy,management] Serve Bedrock model discovery from the control plane
A Bedrock provider could never answer a model-discovery request. The router
sent GET /inference-profiles to the record's upstream, which has to be
bedrock-runtime.<region> for InvokeModel to work, and that host does not
implement the operation. ListInferenceProfiles is a control-plane operation on
bedrock.<region>.amazonaws.com, and one provider record carries one upstream,
so the two hosts genuinely differ.
The route now carries a discovery host, taken from the catalog's declaration
with the region read back out of the configured upstream, and the listing — and
only the listing — goes there. Inference is untouched. A proxied or self-hosted
Bedrock endpoint gets no discovery host at all rather than a guessed one, since
inventing a host would send the operator's credential somewhere they never
configured.
Two things had to follow for the listing to be usable once it arrives. The
response filter only understood OpenAI's {"data":[{"id":…}]}, so a Bedrock
listing fell through it untouched, offering every profile in the account
whatever the policy said. And discoverableModels intersected by exact string,
so a record registering the raw profile id while a guardrail names the catalog
key intersected to nothing — bounding a working provider's listing down to
empty.
Normalisation is the third. The geography in front of a cross-region profile
was matched against a hardcoded list of four, so every profile issued under jp,
au, ca, sa or us-gov carried its prefix into the pricing key, matched no
catalog entry and metered at zero. It is now recognised by either the geography
or the vendor that follows it, so an id has to be new on both axes at once to
slip through — a live eu-central-1 listing returned "global.xai.grok-4.6" days
after the vendor list was first written.
[management] Offer a provider's live model list in the config form
Adds POST /api/agent-network/catalog/providers/models, which asks a vendor
which models an operator's own credential can actually reach, so the provider
form can offer a live list instead of only the compiled-in catalog. The catalog
goes stale, and it cannot see an account: which OpenAI models an org is
entitled to, which Bedrock inference profiles an account and region hold, which
Vertex models a project has enabled.
The endpoints, auth headers and response shapes come from probing the live APIs
(#7244); each vendor invented its own envelope and none can be guessed from the
request. Bedrock shaped the design: its listing lives on the control plane
while inference must go to the runtime host, so Discovery carries its own host
rather than reusing the record's upstream, and profile ids are taken verbatim
because the region prefix is what AWS requires at invoke time.
A caller supplies either the key they are typing or the id of a saved record
whose stored credential is reused — never both, since accepting both would run
an arbitrary credential under the identity of a record the caller may only be
permitted to read. Gated on Create rather than Read, because this spends the
operator's credential against a third party.
Management has not made outbound calls on an operator's behalf before and it
holds a credential for every provider, so every resolved address must be public
— covering loopback, RFC1918, the cloud metadata address and NetBird's own
100.64/10 range — and redirects are not followed, since a redirect moves the
request to a host the check never saw.
The vendor is authoritative for the id; the catalog stays authoritative for
pricing. A discovered model the shipped table cannot price returns
pricing_known: false so the operator must set rates rather than being
registered at a silent zero.
[proxy,management] Bound model discovery to the caller's own policies
GET /v1/models was bounded by the provider record's enumerated models, which is
the right bound only while a single policy reaches a provider. Where two teams
share one provider under different allowlists, every caller was offered the
union — each model outside their own policy being a request the guardrail
refuses a moment later. A gateway record enumerating nothing was worse: it
offered the upstream's entire catalogue however narrow the policy was.
Each route now carries one rule per authorising policy — its source groups and
the models it permits — instead of a single flattened list. At request time the
router keeps the rules whose groups intersect the caller's, unions their
models, and intersects that with what the provider serves.
nil and [] stay distinct end to end: a policy setting no allowlist reaches the
router as nil and lifts the restriction for the groups it binds, while an
enabled allowlist with no models arrives as [] and permits nothing. Collapsing
them would let a listing that should offer nothing fall open to everything.
The guardrail's own per-provider allowlist is untouched. It is a fail-closed
backstop that cannot tell who is asking, so discovery is now narrower than the
backstop rather than wider.
[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol
Reviewed the proxy against Claude Code's published gateway contract. The
transport layer already held up; fourteen gaps sat one layer up, in the model
catalog and in the non-inference endpoints clients call.
Two of them cost money. The catalog carried no claude-opus-5 or
claude-sonnet-5, so an operator could not authorise the models coding agents
default to — those requests denied as not-routable, or priced at zero where a
catch-all carried them. And gateway records pin ParserID "openai" while the
same record serves /v1/messages, so Anthropic responses were read with the
OpenAI parser, which never looks at message_start where input tokens live:
input metered as roughly zero on every stream and cost was skipped entirely.
The rest fix requests refused for structural rather than policy reasons: model
discovery denied for every account with a model allowlist, token counting
denied on Bedrock and mis-parsed on Vertex, startup probes refused and written
into the access log at every session start, and denials rendered in a shape no
LLM client parses. Two changes are additive by design — the deny body keeps
every field it had and adds the vendor's error object alongside, and body-level
identity injection is now gated on the request's dialect so it stops sending
OpenAI-shape fields into Anthropic bodies that reject them.
The end-to-end work turned up one more: the discovery filter treated any slash
in a model id as a gateway prefix, which would have dropped every self-hosted
"Qwen/..." model from the picker.
People who only ever reach private services through the reverse proxy were
invisible to activity accounting. Active users are counted from user.LastLogin
or from the LastSeen of a peer they own, and neither column was written on the
proxy paths — so a person signing in via SSO to a proxied service, or a peer
serving one over the mesh, never showed up in the 24 hour numbers.
Both writes now happen where the proxy already authenticates:
- GenerateSessionToken stamps LastLogin after the session token is signed,
the same column and the same way the dashboard and device login paths do.
- ValidateTunnelPeer stamps the calling peer's LastSeen, the column its owner
activates through.
The policy lives in a new reverseproxy/activity manager rather than in the gRPC
service, matching the module layout the other reverse proxy domains use. It
skips what can never count — service users, embedded proxy peers and WASM
clients — and throttles peer writes to once an hour, well inside the window
accounting asks about and far above the proxy's five minute tunnel cache.
The peer write is a single indexed UPDATE that touches only
peer_status_last_seen. Connected and SessionStartedAt are left alone so the
session-ownership fencing MarkPeerConnectedIfNewerSession relies on is never
disturbed, and the timestamp comes from the database clock rather than the
caller, for the same reason the other status writers take it from there. The
caller's cutoff travels into the statement's WHERE, so concurrent requests for
one peer collapse into a single write instead of each acting on its own stale
read, and a peer that was never seen — NULL last seen, since Status is an
embedded pointer — still records its first activity.
Nothing outside the reverse proxy changes behaviour: the only addition
elsewhere is the RefreshPeerLastSeen store method the manager calls.
