[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol (#7154)

[proxy,management] Conform the Agent Network endpoint to the LLM gateway protocol

Reviewed the proxy against Claude Code's published gateway contract. The
transport layer already held up; fourteen gaps sat one layer up, in the model
catalog and in the non-inference endpoints clients call.

Two of them cost money. The catalog carried no claude-opus-5 or
claude-sonnet-5, so an operator could not authorise the models coding agents
default to — those requests denied as not-routable, or priced at zero where a
catch-all carried them. And gateway records pin ParserID "openai" while the
same record serves /v1/messages, so Anthropic responses were read with the
OpenAI parser, which never looks at message_start where input tokens live:
input metered as roughly zero on every stream and cost was skipped entirely.

The rest fix requests refused for structural rather than policy reasons: model
discovery denied for every account with a model allowlist, token counting
denied on Bedrock and mis-parsed on Vertex, startup probes refused and written
into the access log at every session start, and denials rendered in a shape no
LLM client parses. Two changes are additive by design — the deny body keeps
every field it had and adds the vendor's error object alongside, and body-level
identity injection is now gated on the request's dialect so it stops sending
OpenAI-shape fields into Anthropic bodies that reject them.

The end-to-end work turned up one more: the discovery filter treated any slash
in a model id as a gateway prefix, which would have dropped every self-hosted
"Qwen/..." model from the picker.
This commit is contained in:
Maycon Santos
2026-08-23 20:02:33 +02:00
committed by GitHub
parent ee253feddf
commit 766fcae3f8
39 changed files with 3094 additions and 154 deletions
+29
View File
@@ -40,6 +40,35 @@ You can then use this private endpoint to configure your AI agents, whether that
Full step-by-step setup:
**https://docs.netbird.io/agent-network/quickstart**
## Client settings that don't follow the endpoint
Most of an agent's traffic follows the base URL you hand it, but a few
client-side checks call their vendor directly and never reach the proxy. On a
network that blocks direct egress they fail even though inference works, so
they are worth setting once when you roll the endpoint out.
For Claude Code:
- **Fast mode** checks availability against `api.anthropic.com` rather than the
configured base URL. Set `CLAUDE_CODE_SKIP_FAST_MODE_ORG_CHECK=1` when the
agent authenticates with `ANTHROPIC_AUTH_TOKEN` alone (the usual shape when
the proxy injects the real provider key) or when a TLS-inspecting proxy
answers the check itself. Set
`CLAUDE_CODE_SKIP_FAST_MODE_NETWORK_ERRORS=1` when the network refuses the
connection outright. Fast mode is an Anthropic-API feature, so it is
unavailable on a Bedrock- or Vertex-backed endpoint whatever you set.
- **Model discovery** is off by default. Set
`CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` for the picker to list the
models your policies authorise; the proxy filters the response to that set.
The client gives discovery a three-second budget and treats any redirect as
a failure, so the endpoint must serve `/v1/models` directly.
- **The WebFetch domain safety check** also calls `api.anthropic.com` directly
and is unaffected by the variables above.
Allowing direct egress to `api.anthropic.com` covers the network cases but not
the credential one, where the check reaches Anthropic and is rejected because
the agent presents a proxy-issued key.
## Architecture
Agent Network is built on two existing NetBird capabilities: