mirror of https://github.com/netbirdio/netbird.git synced 2026-07-17 03:59:56 +00:00

Files

Maycon Santos b416063bcc [management,proxy] Agent network: per-account LLM gateway (policy, metering, multi-provider) (#6555 )

* [agent-network] Shared proto, OpenAPI schema, and generated types

* [agent-network] Management: store, manager, synthesizer, policy engine, provider catalog, HTTP/gRPC API

Adds the account-scoped agent-network module: provider/policy/budget CRUD and
store, the reverse-proxy service synthesizer, policy selection + limit
enforcement, the provider catalog (incl. Vertex AI and AWS Bedrock entries),
and the management HTTP + proxy gRPC surfaces.

* [management] Fix agent-network proxy-peer fan-out on affected-peer recompute

The affected-peers resolver loaded only persisted reverse-proxy services, but
agent-network services are synthesized on demand and never persisted. As a
result the embedded proxy peer was never folded into the affected set when a
client's group changed, so the proxy received no network-map update for a newly
authorised client and rejected its handshake until a full resync (restart).

loadProxyServices now merges the synthesized agent-network services (injected
via a registration hook to avoid an import cycle), so proxy peers learn newly
authorised clients immediately.

* [proxy] Reverse-proxy middleware framework, chain, and request plumbing

The per-target middleware chain (slots, dispatcher, mutation gate, metadata
merger), body capture, access-log terminal sink, and the proxy wiring that
builds + runs chains for synthesized agent-network services.

* [proxy] LLM parsers, pricing, and builtin middlewares (OpenAI, Anthropic, Vertex AI, AWS Bedrock)

Request/response parsers and SSE/event-stream metering, the embedded pricing
table, and the builtin middleware set: request parser, router, policy
limit-check/record, cost meter, guardrail, identity inject, response parser.
Includes the path-routed providers — Google Vertex AI (keyfile:: service-account
OAuth minting) and AWS Bedrock (bearer auth, invoke/converse/streaming, optional
/bedrock prefix) — plus the Models allowlist and unmeterable-publisher deny.

* [proxy] IPv6 in-place apply and TCP accept-loop hardening on netstack listeners

* [agent-network] End-to-end test suite, module docs, and deployment preset

* [agent-network] Fix codespell typos and exclude false positives

- labelgen word pool: vermillion -> vermilion, racoon -> raccoon.
- codespell ignore list: add flate (Go compress/flate package), recordin
  (a test-local identifier), and unparseable (a valid alternative spelling used
  consistently across identifiers + a metadata-value constant).

* [management] Set LastSeen on injected proxy peer in realstack test (MySQL strict-mode)

The injected embedded proxy peer had a PeerStatus with a zero LastSeen, which
serializes to '0000-00-00' and is rejected by MySQL in strict mode (SQLite
tolerates it). Set LastSeen to a valid time so SaveAccount succeeds on both
engines.

* [agent-network] Remove e2e shell-script suite from this branch

The end-to-end shell scripts under scripts/e2e/ are maintained in a separate
testing suite and are not part of this change set.

* [agent-network] Polish module docs: remove internal review scaffolding, fix links, verify diagrams

Strip PR-review framing, commit references, absolute paths, and stale internal
references from the agent-network module docs; fix broken relative links; verify
all diagrams against the current architecture. Remove the internal AI-reviewer
prompt file.

* [management] Refine session expiration handling to support 3-state encoding for SSO deadlines

* [agent-network] Relocate agentnetwork package to internals/modules

Move management/server/agentnetwork (and its catalog/, labelgen/, types/
subpackages) to management/internals/modules/agentnetwork, alongside the
reverse-proxy module, and rewrite all importers. Pure relocation: package names,
the synthesizer + affectedpeers registration hook, and store access (shared
store.Store) are unchanged, so no import cycle is introduced (affectedpeers
still depends only on the agentnetwork/types leaf).

* [agent-network] Co-locate HTTP handlers in the module (RegisterEndpoints)

Move the agent-network HTTP handlers from server/http/handlers/agentnetwork into
the module at internals/modules/agentnetwork/handlers (package handlers) and
rename the entrypoint AddEndpoints -> RegisterEndpoints, matching the
reverse-proxy module convention. Wiring in http/handler.go updated accordingly.

2026-06-27 13:41:00 +02:00

6.0 KiB

Raw Blame History

Agent Networks — overview

Single-entry point. Feature scope, the module map, and the cross-cutting topics worth keeping in mind, with links into every per-module guide.

TL;DR

Agent Networks introduces an LLM-aware reverse-proxy middleware system plus account-level controls (budget rules, log collection toggles, PII redaction). The management server synthesises a per-peer middleware chain that the proxy executes on every LLM request; the chain enforces quotas, injects identity, redacts PII, parses tokens/cost, and emits access-log entries. The dashboard exposes the surface as a single AI Observability page with four tabs.

Backend lives in this repo, primarily under management/server/agentnetwork, proxy/internal/middleware, and proxy/internal/llm, with wire contracts in shared/management.
Dashboard lives in the dashboard repo under src/modules/agent-network/ and src/app/(dashboard)/agent-network/.

Reading order

#	Doc	Why
1	01-end-to-end-flows.md	Get the three big diagrams in your head first.
2	modules/10-shared-api.md	Wire contracts — every other module either produces or consumes these.
3	modules/21-management-agentnetwork.md	The largest module; everything the proxy executes originates here.
4	modules/30-proxy-middleware-framework.md	The generic plugin system on the proxy side.
5	modules/31-proxy-middleware-builtin.md	The 8 LLM middlewares that ride on the framework.
6	Everything else in any order.

Module map

11 modules. Each is described in detail in its own file under modules/.

#	Module	Risk	BC impact
10	shared/api — proto + OpenAPI	Low	Additive only
20	management/store — SQL persistence	Medium	Auto-migrate (additive)
21	management/agentnetwork — domain layer + synthesizer	High	Additive
22	management/handlers + wiring — HTTP API + gRPC delivery	Medium	Additive
30	proxy/middleware-framework — generic plugin system	High	Additive
31	proxy/middleware-builtin — 8 LLM middlewares	High	Additive
32	proxy/llm-parsers — SDK adapters + pricing	Medium	Additive
33	proxy/runtime — translate + serve + access-log	High	Additive (touches hot path)
40	dashboard — UI for everything above	Medium	Sidebar reshape
50	path-routed-providers — Vertex AI + Bedrock	Medium	Additive (new catalog entries)

The largest and highest-risk module is management/agentnetwork: it is the single writer of the middleware chain the proxy executes.

Cross-cutting topics

These are the items most likely to bite production. Each is fully documented in the linked module guide.

Capture-pointer semantics (*bool for capture_prompt and capture_completion): nil = legacy emit, false = suppress, true = emit. nil-vs-false must be handled at every JSON hop. See 21-management-agentnetwork.md and 31-proxy-middleware-builtin.md.
ProxyMapping.Private preservation on per-proxy live updates. Failure mode: auth skips ValidateTunnelPeer → CapturedData.UserGroups empty → llm_router denies. See 33-proxy-runtime.md.
respInput carrying UserEmail/UserGroups/UserGroupNames onto the response leg in reverseproxy.go. Load-bearing wire that lets llm_limit_record ship non-empty group_ids on RecordLLMUsage. See 33-proxy-runtime.md.
Min-wins all-must-pass budget rule semantics. Every matching rule's remaining quota must be > 0 for the request to proceed; one exhausted rule blocks the whole call. Documented in 21-management-agentnetwork.md and the llm_limit_check middleware in 31-proxy-middleware-builtin.md.
body-tap memory bounds: per-direction 1 MiB cap, shared 256 MiB budget, LimitReader(r.Body, limit+1) for truncation detection with replayReadCloser fallback so upstream still sees the full body. cloneInputFor deep-copies the body up to 16 times per chain — a perf hot-spot. See 30-proxy-middleware-framework.md.
UpstreamRewrite.AuthHeader bypasses the header denylist deliberately. The runtime consumer only unpacks it via the trusted upstream-build path. See 30-proxy-middleware-framework.md.
disable_access_log default-false semantics: the synth target sets it true, all other targets leave it false. See 10-shared-api.md.
String-typed decision / deny_code on CheckLLMPolicyLimitsResponse — would benefit from enum pinning before external consumers integrate. See 10-shared-api.md.

Explicit non-goals

Reaper / GC pass over stale synth services — designed but cut from scope.
URL-sync for tab state on AI Observability — read path is wired (?tab=) but write path isn't. Future work.
CI golden-file regen-and-diff for types.gen.go / proxy_service.pb.go — would catch codegen drift; not yet in place.

Where to read the code

Per-module file scopes are listed in each module guide. Behaviour is covered by Go tests co-located with each package (and an end-to-end chain integration test under proxy/internal/proxy).

6.0 KiB Raw Blame History