sendnrw/netbird
2
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-07-20 07:21:27 +02:00
Code Issues Packages Projects Releases Wiki Activity

[agent-network] End-to-end test suite, module docs, and deployment preset

Browse Source
This commit is contained in:
mlsmaycon
2026-06-27 00:43:35 +02:00
parent 3993fa32e4
commit 29f55d4255
26 changed files with 5176 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

198
docs/agent-networks/00-overview.md Normal file
View File

@@ -0,0 +1,198 @@
# agent-networks PR — overview
Single-entry point for reviewers. Scope, commit roll-up, ownership
matrix, cross-cutting risk hot-list, and links into every per-module
guide.
## TL;DR
This PR pair introduces an **LLM-aware reverse-proxy middleware system**
plus **account-level controls** (budget rules, log collection toggles,
PII redaction) for NetBird agent-networks. The management server
synthesises a per-peer middleware chain that the proxy executes on
every LLM request; the chain enforces quotas, injects identity, redacts
PII, parses tokens/cost, and emits access-log entries. The dashboard
exposes the surface as a single **AI Observability** page with four
tabs.
- **Backend branch:** `feature/agent-networks-backend` on
`/Users/maycon/projects/netbird` — 28 commits vs merge-base
`14af17955`, **~28,000 net LOC added** across 150 files.
- **Dashboard branch:** `feature/agent-networks` on
`/Users/maycon/projects/dashboard` — ~70 commits vs `main`,
**~10,000 net LOC added** across 82 files.
## Reading order
| # | Doc | Why |
|---|-----|-----|
| 1 | [01-end-to-end-flows.md](01-end-to-end-flows.md) | Get the three big diagrams in your head first. |
| 2 | [modules/10-shared-api.md](modules/10-shared-api.md) | Wire contracts — every other module either produces or consumes these. |
| 3 | [modules/21-management-agentnetwork.md](modules/21-management-agentnetwork.md) | The largest module; everything the proxy executes originates here. |
| 4 | [modules/30-proxy-middleware-framework.md](modules/30-proxy-middleware-framework.md) | The generic plugin system on the proxy side. |
| 5 | [modules/31-proxy-middleware-builtin.md](modules/31-proxy-middleware-builtin.md) | The 8 LLM middlewares that ride on the framework. |
| 6 | Everything else in any order. | |
## Ownership matrix
11 review-able modules. Each is described in detail in its own file
under [`modules/`](modules/).
| # | Module | Suggested reviewer | Expertise required | Est. time | Risk | BC impact |
|---|--------|--------------------|--------------------|-----------|------|-----------|
| 10 | [shared/api](modules/10-shared-api.md) — proto + OpenAPI | API/contracts maintainer | protobuf + oapi-codegen + wire-format reasoning | 30 min | Low | Additive only |
| 20 | [management/store](modules/20-management-store.md) — SQL persistence | Storage maintainer | gorm + sqlite/postgres + migrations | 45 min | Medium | Auto-migrate (additive) |
| 21 | [management/agentnetwork](modules/21-management-agentnetwork.md) — domain layer + synthesizer | Management server maintainer | network-map controller + policy engine + Go concurrency | **2–3 h** | **High** | Additive |
| 22 | [management/handlers + wiring](modules/22-management-handlers-wiring.md) — HTTP API + gRPC delivery | Management infra maintainer | REST handlers + RBAC + gRPC streaming + controller wiring | 90 min | Medium | Additive |
| 30 | [proxy/middleware-framework](modules/30-proxy-middleware-framework.md) — generic plugin system | Proxy maintainer | net/http composition + request lifecycle + Go context | 60 min | High | Additive |
| 31 | [proxy/middleware-builtin](modules/31-proxy-middleware-builtin.md) — 8 LLM middlewares | Proxy + LLM owner | OpenAI/Anthropic semantics + the framework above | **2 h** | High | Additive |
| 32 | [proxy/llm-parsers](modules/32-proxy-llm-parsers.md) — SDK adapters + pricing | LLM/proxy owner | OpenAI Responses API + Anthropic Messages API + SSE framing | 75 min | Medium | Additive |
| 33 | [proxy/runtime](modules/33-proxy-runtime.md) — translate + serve + access-log | Proxy maintainer | existing reverseproxy.go lifecycle + access-log emission + synth ingestion | 90 min | High | Additive (touches hot path) |
| 40 | [dashboard](modules/40-dashboard.md) — UI for everything above | Dashboard maintainer | Next.js app router + existing Tabs/Modal/Table patterns + Providers/Permissions/Groups context | **2 h** | Medium | Sidebar reshape |
| 50 | [path-routed-providers](modules/50-path-routed-providers.md) — Vertex AI + Bedrock | Proxy + LLM owner | URL-path routing + GCP OAuth + AWS Bedrock runtime + catalog↔pricing | 45 min | Medium | Additive (new catalog entries) |
| — | E2E scripts under `scripts/e2e/agent-network-policy/` | Whoever runs the e2e suite | bash + tilt | 30 min | Low | New only |
**Total wall-clock review estimate:** ~13–15 hours if reviewers
serialize. With per-module fan-out (one reviewer per row) the critical
path is ~3 hours (the management/agentnetwork module).
## Commit roll-up
### Backend (28 commits, oldest → newest)
| SHA | Module(s) | Subject |
|-----|-----------|---------|
| `06ff17b38` | shared/api, agentnetwork/types, proxy/llm | AN-0: additive proto + OpenAPI schemas + base types |
| `f810a4e35` | management/store | AN-1: store layer for providers/policies/guardrails/settings |
| `77b407632` | management/agentnetwork | AN-2: agentnetwork module (manager, synthesizer, catalog, policyselect) |
| `09e8059b6` | management/handlers+wiring | AN-2b: wire synth services into the network map |
| `9ecb6449d` | management/handlers+wiring | AN-3: HTTP API handlers + route registration |
| `00bf0d328` | proxy/middleware-framework | AN-4: proxy middleware framework |
| `3ed29d855` | proxy/runtime | AN-4b: middleware plumbing in the proxy request path |
| `e64ea4b02` | proxy/middleware-builtin | AN-5: built-in middlewares + registry registration |
| `0f9f56a58` | proxy/runtime | AN-5b: activate the middleware system in the proxy server |
| `9a1547143` | shared/api, proxy/runtime, accesslog | AN-6: access-log agent_network flag end-to-end |
| `8cb9c187d` | shared/api, agentnetwork, runtime | AN-7: enforcement + synth-service delivery to the proxy |
| `9ebe219fd` | proxy/runtime | test: lock the auth→middleware group-propagation wiring |
| `665575932` | agentnetwork tests | test: real-store integration coverage (no MockStore) |
| `263dabd73` | proxy/runtime | fix: preserve ProxyMapping.Private on per-proxy live updates |
| `5adee2cb4` | agentnetwork tests | Add no-mock agent-network provider-CRUD fan-out test |
| `9ae476ea7` | agentnetwork tests | Add no-mock baseline guards for enforcement and guardrail synthesis |
| `a436b5fb3` | agentnetwork, store, shared/api | GC-0: account budget rule type + collection toggles + store CRUD |
| `5b408b0ef` | agentnetwork | GC-1: budget-rule manager CRUD + settings update |
| `b22d5a181` | agentnetwork, builtin | GC-2: enforce account budget rules as a min-wins ceiling |
| `945f17f1a` | agentnetwork, store, builtin | GC-3: account-level prompt-collection master switch |
| `23bdf6871` | handlers, agentnetwork, shared/api | GC-4: HTTP API for budget rules + settings update |
| `468875cb4` | management+proxy | Wire EnableLogCollection to suppress access-log for opted-out requests |
| `7072f8125` | agentnetwork, builtin | Account toggle is sole control for prompt capture + broader PII redaction |
| `b1e66bca2` | builtin, agentnetwork | Broaden phone redactor + fixture-driven Go test + smoke --groups |
| `b438a7194` | agentnetwork, proxy/runtime | Fix budget enforcement + extend PII redaction to all metadata channels |
| `19e03d688` | agentnetwork tests | Self-contained Go tests for the redact-pii wiring across parsers |
| `2a0d4991b` | proxy/runtime | Self-contained full-chain integration test for agent-network requests |
| `4836d5a19` | agentnetwork, builtin | Gate prompt + completion capture on EnablePromptCollection |
### Dashboard (recent → oldest, abbreviated)
- `3fbe44e` Merge Global Controls + Access Log into AI Observability (today)
- `1988e92` agent-network: Global Controls page with account toggles + budget rules
- `2157fec` Consumption page: charts + filters + access-log style
- `62a85b5` add policy Limits tab; slim Guardrails to allowlist + capture
- `2b7f746` Wire Agent Network guardrails to backend
- `5e10a2b` Wire Agent Network policies to backend
- `4d3ac8d` Wire Agent Network providers to backend API
- …plus ~50 other UI iterations
- Full list: `git -C /Users/maycon/projects/dashboard log --oneline main..HEAD`
## Risk hot-list (cross-cutting)
Pulled from the per-module guides; these are the items most likely to
bite production. Each is fully documented in the linked module guide.
1. **Capture-pointer semantics** (`*bool` for `capture_prompt` and
`capture_completion`): nil = legacy emit, false = suppress, true =
emit. Fixed at `4836d5a19`. Reviewers should verify nil-vs-false is
handled at every json hop. See
[21-management-agentnetwork.md](modules/21-management-agentnetwork.md)
and [31-proxy-middleware-builtin.md](modules/31-proxy-middleware-builtin.md).
2. **ProxyMapping.Private preservation** on per-proxy live updates.
Fix at `263dabd73`. Failure mode: `auth` skips
`ValidateTunnelPeer` → `CapturedData.UserGroups` empty → `llm_router`
denies. See [33-proxy-runtime.md](modules/33-proxy-runtime.md).
3. **respInput carrying `UserEmail`/`UserGroups`/`UserGroupNames` onto
the response leg** at `reverseproxy.go:196-223` (fix at `b438a7194`).
Load-bearing wire that lets `llm_limit_record` ship non-empty
`group_ids` on `RecordLLMUsage`. See
[33-proxy-runtime.md](modules/33-proxy-runtime.md).
4. **Min-wins all-must-pass budget rule semantics**. Every matching
rule's remaining quota must be > 0 for the request to proceed; one
exhausted rule blocks the whole call. Documented in
[21-management-agentnetwork.md](modules/21-management-agentnetwork.md)
and the `llm_limit_check` middleware in
[31-proxy-middleware-builtin.md](modules/31-proxy-middleware-builtin.md).
5. **body-tap memory bounds**: per-direction 1 MiB cap, shared 256 MiB
budget, `LimitReader(r.Body, limit+1)` for truncation detection with
`replayReadCloser` fallback so upstream still sees the full body.
`cloneInputFor` deep-copies the body up to 16 times per chain — a
perf hot-spot worth confirming benchmarks for. See
[30-proxy-middleware-framework.md](modules/30-proxy-middleware-framework.md).
6. **UpstreamRewrite.AuthHeader bypasses the header denylist**
deliberately. Confirm the runtime consumer only unpacks it via the
trusted upstream-build path. See
[30-proxy-middleware-framework.md](modules/30-proxy-middleware-framework.md).
7. **Codegen reproducibility gap**:
`shared/management/http/api/generate.sh:14` pulls
`oapi-codegen@latest` while the AN-0 commit message claims v2.7.0 is
pinned. A future regen could produce a different `types.gen.go`. See
[10-shared-api.md](modules/10-shared-api.md).
8. **`disable_access_log` default-false semantics**: synth target sets
it true, all other targets must leave it false. Worth a
synthesizer-side audit. See
[10-shared-api.md](modules/10-shared-api.md).
9. **String-typed `decision` / `deny_code`** on
`CheckLLMPolicyLimitsResponse` — would benefit from enum pinning
before external consumers integrate. See
[10-shared-api.md](modules/10-shared-api.md).
10. **agent_network_chain_realstack_test drift risk**: it inlines the
proto→Spec mapping instead of calling
`proxy/middleware_translate.go`. Test will pass even if translate
regresses. See [33-proxy-runtime.md](modules/33-proxy-runtime.md).
11. **MOCK_GROUPS still in production paths** via
`AgentPoliciesTable.tsx:45,76` as a name-lookup fallback. Other
`MOCK_*` constants are unreferenced. See
[40-dashboard.md](modules/40-dashboard.md).
12. **`updateProvider` / `updatePolicy` / `updateBudgetRule` use `??`
on `enabled`** — toggle paths are safe but an explicit
`enabled:false` from a form would be silently dropped. See
[40-dashboard.md](modules/40-dashboard.md).
13. **Dashboard lint gate is broken** (`next lint` was removed in Next
16). Pre-existing, but this branch effectively ships without a
lint check. See [40-dashboard.md](modules/40-dashboard.md).
14. **Zero Cypress / component test coverage for new agent-network
UI.** See [40-dashboard.md](modules/40-dashboard.md).
15. **Bodytap truncation-replay test missing**; **Dispatcher
timeout/panic tests missing**; **concurrent Budget exhaustion
test missing**; **InvalidateMiddleware + LiveServiceCheck race
test missing**. See
[30-proxy-middleware-framework.md](modules/30-proxy-middleware-framework.md).
## Explicit non-goals
- **Reaper / GC pass over stale synth services** — designed but scope-cut
per AN-0 commit message; not migrated.
- **URL-sync for tab state on AI Observability** — read path is wired
(`?tab=`) but write path isn't. Future work.
- **Lint-gate restoration on the dashboard** — pre-existing,
out-of-scope.
- **CI golden-file regen-and-diff for `types.gen.go` /
`proxy_service.pb.go`** — would catch codegen drift; not in this PR.
## Where to read the code
- Backend: `git -C /Users/maycon/projects/netbird diff 14af17955..HEAD`
- Dashboard: `git -C /Users/maycon/projects/dashboard diff main..HEAD`
- Per-module file scopes are listed in each module guide.
## Where this doc lives
`~/projects/claude-contexts/netbird/agent-networks-pr/`. Not committed
to either repo. Move pieces into `docs/` if/when ready.

220
docs/agent-networks/01-end-to-end-flows.md Normal file
View File

@@ -0,0 +1,220 @@
# End-to-end flows
Three cross-module mermaid diagrams. Each per-module guide repeats the
slice that's relevant to its own scope — these are the canonical
top-down views.
- [Flow A — Config → runtime (synth + deliver)](#flow-a--config--runtime-synth--deliver)
- [Flow B — Request lifecycle through the LLM chain](#flow-b--request-lifecycle-through-the-llm-chain)
- [Flow C — Budget rule feedback loop](#flow-c--budget-rule-feedback-loop)
---
## Flow A — Config → runtime (synth + deliver)
How an operator's change to a Provider, Policy, Guardrail, Budget Rule,
or Settings record ends up as live middleware on a peer's proxy.
```mermaid
sequenceDiagram
autonumber
actor Op as Operator
participant UI as Dashboard
participant HTTP as management/handlers
participant Mgr as agentnetwork.Manager
participant Store as management/store (SQL)
participant Ctl as network_map.Controller
participant Synth as agentnetwork.SynthesizeServices
participant Grpc as management gRPC
participant Proxy as netbird-proxy
participant Xlate as middleware_translate
participant Chain as middleware.Chain
Op->>UI: edit provider/policy/budget/settings
UI->>HTTP: REST PUT/POST /api/agent-network/*
HTTP->>Mgr: SaveProvider / SavePolicy / SaveBudgetRule / SaveSettings
Mgr->>Store: persist (gorm)
Mgr-->>Ctl: account change event (Network-Map dirty)
loop per connected peer
Ctl->>Synth: SynthesizeServices(ctx, store, accountID)
Synth->>Store: load providers, policies, guardrails, budget rules, settings
Synth-->>Synth: build per-peer Service list
Note over Synth: each Service has a middleware<br/>chain with capture_prompt /<br/>capture_completion / redact_pii<br/>baked from account settings
Synth-->>Ctl: []rpservice.Service
Ctl->>Grpc: NetworkMap push (services + middleware configs)
end
Grpc-->>Proxy: NetworkMap stream
Proxy->>Xlate: translate proto MiddlewareConfig → runtime Spec
Xlate->>Chain: register / replace per-service chain
Note over Chain: chain replacement is live<br/>(no proxy restart, in-flight<br/>requests unaffected)
```
**Notes on the diagram**
- The `network_map.Controller` synthesises on every push, not on a
timer. A single config change costs O(connected peers × policies ×
providers) per push. See [`modules/22-management-handlers-wiring.md`](modules/22-management-handlers-wiring.md).
- `SynthesizeServices` is the single source of truth for the wire
format the proxy executes. Anything the proxy does that the
synthesiser didn't request is a bug. See
[`modules/21-management-agentnetwork.md`](modules/21-management-agentnetwork.md).
- The translate step (step 13) is the only place that knows the
middleware-ID strings on the proxy side. It must reject unknown IDs;
silently dropping middlewares would create a security gap (e.g.
missing `llm_limit_check` ⇒ unbounded spend). See
[`modules/33-proxy-runtime.md`](modules/33-proxy-runtime.md).
---
## Flow B — Request lifecycle through the LLM chain
What happens when an agent on the client peer sends a chat-completion /
messages request through the synthesised reverse-proxy.
```mermaid
sequenceDiagram
autonumber
actor Agent as Agent (local)
participant Px as netbird-proxy
participant Auth as auth middleware
participant Map as service-mapping
participant Req as llm_request_parser
participant Rt as llm_router
participant Chk as llm_limit_check
participant Inj as llm_identity_inject
participant Grd as llm_guardrail
participant Up as upstream LLM
participant Resp as llm_response_parser
participant Cost as cost_meter
participant Rec as llm_limit_record
participant Log as access-log
participant MgmtGrpc as management gRPC
Agent->>Px: POST /v1/chat/completions (OpenAI / Anthropic)
Px->>Auth: identify peer (user, groups)
Auth->>Map: resolve service from Host + path
Map-->>Req: dispatch chain in slot order
Req->>Req: parse body → provider, model, prompt, token estimate
Note over Req: capture_prompt gates raw_prompt<br/>capture (nil = legacy emit,<br/>false = drop, true = emit)
Req->>Rt: pass metadata
Rt->>Chk: route to upstream candidate
Chk->>MgmtGrpc: CheckLLMPolicyLimits(provider, model, est_tokens, groups, user)
MgmtGrpc-->>Chk: decision = allow / deny + deny_code
alt decision == deny
Chk-->>Log: emit access-log with deny_code<br/>(if EnableLogCollection)
Chk-->>Agent: 429 (or 403 per deny_code)
else decision == allow
Chk->>Inj: continue
Inj->>Inj: inject NetBird identity headers per provider config
Inj->>Grd: continue
Grd->>Grd: enforce model allowlist
Grd->>Up: forward (over WireGuard)
Up-->>Resp: response (JSON or SSE stream)
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)
Log-->>Agent: 200 + body (streamed if SSE)
end
```
**Notes on the diagram**
- The chain runs in synth-defined order. Re-ordering middlewares
changes invariants — `llm_limit_check` must precede `llm_router` so
a denied request never hits upstream, and `llm_limit_record` must
pair with `llm_limit_check` so a successful check is always recorded
(or the rate-limit semantics break). See
[`modules/31-proxy-middleware-builtin.md`](modules/31-proxy-middleware-builtin.md).
- `llm_guardrail` is also where PII redaction happens
(`redact_pii = settings.RedactPii`). Phones, emails, credit cards,
PII names — see `redact.go` for the full set. See
[`modules/31-proxy-middleware-builtin.md`](modules/31-proxy-middleware-builtin.md).
- SSE streaming requires special handling on the response side; the
parser must handle partial chunks without buffering the whole
stream. See [`modules/32-proxy-llm-parsers.md`](modules/32-proxy-llm-parsers.md).
- Access-log emission is gated on `settings.EnableLogCollection`. With
it OFF, neither the deny nor the allow leg writes an entry — the
chain still runs (budget rules are still enforced) but no audit trail
is kept. See `468875cb4` and
[`modules/33-proxy-runtime.md`](modules/33-proxy-runtime.md).
---
## Flow C — Budget rule feedback loop
How an account's budget rules tighten ceilings on every request and how
consumption flows back into the dashboard.
```mermaid
flowchart LR
subgraph Operator
DashBud[Dashboard Budget Settings tab]
end
subgraph Mgmt[Management]
Save[POST/PUT /api/agent-network/budget-rules]
Store[(SQL store)]
Synth[SynthesizeServices]
Check[CheckLLMPolicyLimits RPC]
Rec[RecordLLMUsage RPC]
Cons[/api/agent-network/consumption]
end
subgraph Proxy[Proxy]
Chk[llm_limit_check]
RecMw[llm_limit_record]
end
subgraph DashView[Dashboard Budget Dashboard tab]
Panel[AgentConsumptionPanel]
end
DashBud -->|create / update rules| Save
Save --> Store
Store --> Synth
Synth -->|push synth-services to peer| Proxy
Chk -->|per request| Check
Check -->|aggregate matching rules<br/>min-wins all-must-pass| Store
Check -->|allow / deny| Chk
RecMw -->|post-response| Rec
Rec -->|tokens + cost + groups + user| Store
Store -->|read counters| Cons
Cons --> Panel
```
**Notes on the diagram**
- **min-wins all-must-pass** is the core semantic. A budget rule binds
to (group set, user set) with a (window, ceiling). At check time,
every rule that matches the caller is evaluated; if ANY rule has
zero remaining quota the request is denied. This is the most
surprising semantic for operators — see GC-2 (`b22d5a181`) and the
invariants section of
[`modules/21-management-agentnetwork.md`](modules/21-management-agentnetwork.md).
- The proxy never makes its own budget decisions. It always asks
management via `CheckLLMPolicyLimits` and reports back via
`RecordLLMUsage`. This keeps account-wide accounting in one place
and avoids per-proxy drift.
- `RecordLLMUsage` must carry `group_ids` and `user_id` so the
decrement hits the right rule(s). The wire that carries those
fields onto the response leg is `respInput` in `reverseproxy.go`,
fixed at `b438a7194`. See
[`modules/33-proxy-runtime.md`](modules/33-proxy-runtime.md).
- The dashboard's Budget Dashboard tab polls
`/api/agent-network/consumption` — not gRPC, not WebSocket. Poll
interval lives in `AgentConsumptionPanel.tsx`. See
[`modules/40-dashboard.md`](modules/40-dashboard.md).
---
## Cross-references
- Per-module guides: [`modules/`](modules/)
- Overview + ownership matrix: [`00-overview.md`](00-overview.md)
- AI reviewer prompt: [`90-agent-review-prompt.md`](90-agent-review-prompt.md)

246
docs/agent-networks/90-agent-review-prompt.md Normal file
View File

@@ -0,0 +1,246 @@
# Prompt for an AI code reviewer
Self-contained prompt to feed to a second AI agent (Claude, GPT, etc.)
so it can perform structured code review of the agent-networks PR in
parallel with the human reviewers. Copy the **Prompt** block below into
the agent's first message. The agent should produce a single markdown
report at a path you specify.
The prompt assumes the agent has filesystem read access to the netbird
and dashboard repos and can run `git`, but does NOT have permission to
modify or commit anything.
---
## Prompt
```
You are reviewing a large PR pair for security, correctness, concurrency,
backward-compatibility, performance, and observability. You are read-only
on both repos: do NOT modify any file, do NOT commit anything, do NOT
stage anything, do NOT run e2e scripts. Only read code and emit a single
markdown report at:
/Users/maycon/projects/claude-contexts/netbird/agent-networks-pr/AGENT_REVIEW_REPORT.md
REPOS
Backend: /Users/maycon/projects/netbird (branch feature/agent-networks-backend, merge-base 14af17955)
Dashboard: /Users/maycon/projects/dashboard (branch feature/agent-networks, merge-base main)
PR CONTEXT
Backend: 28 commits, ~28k LOC added across 150 files.
Dashboard: ~70 commits, ~10k LOC added across 82 files.
The PR introduces an LLM-aware reverse-proxy middleware system for
NetBird agent-networks plus account-level controls (budget rules,
PII redaction, log + prompt capture toggles). The management server
synthesises a per-peer middleware chain that the proxy executes for
every LLM request; the chain enforces quotas, injects identity,
redacts PII, parses tokens / cost, and emits access-log entries.
REQUIRED INPUT — read these BEFORE reading any code
/Users/maycon/projects/claude-contexts/netbird/agent-networks-pr/README.md
/Users/maycon/projects/claude-contexts/netbird/agent-networks-pr/00-overview.md
/Users/maycon/projects/claude-contexts/netbird/agent-networks-pr/01-end-to-end-flows.md
AND the per-module guide for every module you intend to review:
modules/10-shared-api.md
modules/20-management-store.md
modules/21-management-agentnetwork.md
modules/22-management-handlers-wiring.md
modules/30-proxy-middleware-framework.md
modules/31-proxy-middleware-builtin.md
modules/32-proxy-llm-parsers.md
modules/33-proxy-runtime.md
modules/40-dashboard.md
Each guide names its reviewer profile, lists commits in scope, shows
the file map, names invariants, and enumerates "Things to scrutinize".
Use those lists to focus your reading — do not re-derive what the
guides already documented.
YOUR JOB
For each module, validate that the invariants in the guide actually
hold in the code, and find anything the guide missed. The guide is
your CHECKLIST, not the truth — it is itself a draft. Flag any place
where the guide is wrong, stale, or misleading.
FOCUS AREAS (in priority order)
1. Security
- Privilege boundaries: account isolation in store queries; group
resolution can never cross accounts.
- PII handling: every metadata channel out of the proxy must
respect redact_pii when enabled. Confirm at every JSON-marshal
site, every access-log write, every gRPC outbound message.
- Capture-pointer semantics (*bool for capture_prompt /
capture_completion): nil = legacy emit, false = suppress,
true = emit. Verify every json hop preserves nil vs false.
- api_key fields on AgentNetworkProviderRequest must be
write-only — never echoed on responses.
- UpstreamRewrite.AuthHeader deliberately bypasses the header
denylist. Confirm consumers only unpack it via the trusted
upstream-build path; flag if any code path lets a synth-untrusted
caller stage a fake AuthHeader.
- Permission keys on every HTTP handler. Confirm permission?.services?.read
gates every read; permission?.services?.write gates writes.
2. Correctness
- Min-wins all-must-pass budget rule semantics in
management/agentnetwork: every matching rule's remaining quota
must be > 0 for the request to proceed. Verify the
check + record pair stays consistent (no orphan decrements,
no double-count on retries).
- llm_limit_check ↔ llm_limit_record pairing — a successful check
must always result in a corresponding record (or the rate-limit
counters skew). Look for early-returns / panics that could
break the pair.
- ProxyMapping.Private preservation on per-proxy live updates
(263dabd73). Failure mode: auth skips ValidateTunnelPeer →
CapturedData.UserGroups empty → llm_router denies. Verify the
fix covers every gRPC translation path.
- respInput carrying UserEmail/UserGroups/UserGroupNames onto
the response leg (reverseproxy.go:196-223, b438a7194). If this
wire breaks, llm_limit_record ships empty group_ids and
budget decrements miss.
- agent_network_chain_realstack_test.go inlines proto→Spec
mapping instead of calling middleware_translate.go — confirm
this drift risk, suggest fix.
3. Concurrency
- Chain replacement under in-flight requests in proxy/runtime.
Confirm no goroutine sees a half-replaced chain.
- Body-tap memory bounds: 1 MiB per-direction cap, 256 MiB shared
Budget semaphore. Verify the Budget release path runs on every
exit (including panic + timeout).
- cloneInputFor deep-copies the body up to 16 times per chain.
Note allocation cost; flag if benchmarks are missing.
- InvalidateMiddleware + LiveServiceCheck race — flag if the
guide says no test exists.
4. Backward compatibility
- All proto field numbers in proxy_service.proto are in unused
slots; OpenAPI additions are appended; no existing schemas
lose required-ness. Confirm by diffing 14af17955..HEAD on
shared/management/proto/proxy_service.proto and
shared/management/http/api/openapi.yml.
- Pointer fields for capture flags preserve nil-default emission
so existing proxies that don't know about the field keep
working.
- Non-agent-network services on the proxy must still work.
Verify ServiceMapping doesn't gate on agent_network=true.
5. Performance
- SynthesizeServices runs on every NetworkMap push. With N
connected peers × M policies × K providers, what's the worst
case allocation? Flag any O(N²) loop hidden in the synthesiser.
- cloneInputFor allocates per middleware invocation. Estimate
p99 request-time overhead under a 16-middleware chain.
- Streaming response handling — the SSE parser must not buffer
the whole stream.
6. Observability
- Activity codes added for budget-rule decisions. Verify each
has a UI label in the dashboard.
- access-log fields (provider, model, tokens, cost, deny_code,
agent_network flag) match what the dashboard renders.
- Metrics surface: list every metric added in
proxy/internal/metrics + proxy/internal/middleware/metrics.
7. Tests
- For every "what to scrutinize" bullet, find the test that
locks it down. If none exists, flag as a coverage gap.
- Real-store coverage in management/agentnetwork (no mocks)
is a stated goal — confirm and list any remaining mocks.
- Bodytap truncation-replay test missing; Dispatcher
timeout/panic tests missing; concurrent Budget exhaustion test
missing — confirm these gaps from
modules/30-proxy-middleware-framework.md.
OUT OF SCOPE — do not flag these as issues
- Reaper / GC pass over stale synth services (scope-cut per AN-0)
- URL-sync for tab state on AI Observability page
- npm run lint broken on dashboard (pre-existing Next 16 issue)
- Style nitpicks unless they affect correctness
- Anything in MEMORY.md or claude-contexts/ paths
OUTPUT FORMAT (single markdown file)
# Agent review — agent-networks PR
## Summary
- Modules reviewed: <list>
- Findings: <N total — H high, M medium, L low>
- Most concerning: <one-line>
- Disagreements with the human review guides: <list or "none">
## Findings
For each finding:
### F<n>. <Short title> — <severity H/M/L> — module <ID>
- **Where:** file:line (or file:line-range)
- **What:** one sentence on the problem
- **Why it matters:** consequence in production
- **Evidence:** code snippet (≤20 lines) + reasoning
- **Suggested fix:** one paragraph
- **Test that would catch it:** suggest a test name + the assertion shape
## Gaps in the human review guides
For every place the guide is wrong, stale, missing, or misleading:
- **Guide:** modules/<file>.md section
- **Issue:** <what the guide says vs reality>
- **Suggested edit:** <one sentence>
## Confirmations
List the high-stakes invariants you VERIFIED:
- <invariant> — confirmed at file:line
## Methodology
- Files actually read: <N>
- Commits examined individually (git show): <list of SHAs>
- Anything you didn't review and why: <list>
CONSTRAINTS
- Read-only. Do not modify ANY file in either repo.
- Statically-analyzable bash only — no $(), pipes, &&, or grep in
Bash commands. Use Read for file content.
- Use absolute paths (no cd).
- Cite file:line for every concrete claim.
- Severity rubric:
H — could cause data loss, security boundary breach, runaway spend,
or service outage in prod with realistic inputs.
M — correctness bug under non-rare conditions, missing security
control with a defense-in-depth replacement, or a perf issue
that surfaces under typical load.
L — code-smell, missing test, doc drift, minor inefficiency.
- When in doubt about severity, write your reasoning; do not inflate.
- The whole report must fit under 8000 words. If your draft is longer,
cut the LOWEST-severity findings first.
When done, reply with a one-line confirmation including the absolute
report path and the finding counts (H/M/L).
```
---
## How to use this prompt
1. Spawn a second AI session (Claude Code, web Claude, or another agent
with tool access). Make sure it can read both repos and write to
`~/projects/claude-contexts/`.
2. Paste the **Prompt** block above as the agent's first message.
3. When the agent finishes, the report lands at
`~/projects/claude-contexts/netbird/agent-networks-pr/AGENT_REVIEW_REPORT.md`.
4. Diff the agent's findings against your own / your team's review notes
— disagreements are often the most useful signal.
5. Iterate: if the agent missed something, refine this prompt and re-run.
## Notes for the human running the prompt
- The prompt is intentionally bossy about scope and severity. Soft
prompts produce padded reports.
- The "Out of scope" list is the most-edited part: keep it in sync
with what your team has agreed NOT to fix in this PR.
- If you want a more shallow second-opinion run, drop the OUTPUT FORMAT
section and ask for "your top 10 concerns in plain prose."
- If you want a deeper run, add a second pass that asks the agent to
rate every claim in the human review guides as `correct` / `wrong` /
`unclear` with citations.

84
docs/agent-networks/README.md Normal file
View File

@@ -0,0 +1,84 @@
# agent-networks PR — review pack
A self-contained set of review documents for the agent-networks PR pair:
`feature/agent-networks-backend` on this repo (netbird) and
`feature/agent-networks` on the dashboard repo.
## What to read first
1. **[00-overview.md](00-overview.md)** — the single entry point. PR
scope, commit list, ownership matrix, risk hot-list, and links to
every per-module guide.
2. **[01-end-to-end-flows.md](01-end-to-end-flows.md)** — three
high-level mermaid diagrams: config-to-runtime synth/delivery,
per-request lifecycle through the LLM chain, and the budget-rule
feedback loop.
3. **Per-module guides** under `modules/` — one file per
review-able package. Each names its reviewer profile, lists commits
in scope, shows file-level changes, includes its own flow diagrams,
and calls out what to scrutinize.
4. **[90-agent-review-prompt.md](90-agent-review-prompt.md)** — a
self-contained prompt to feed to another AI agent (or a parallel
reviewer) to perform structured code review against the same scope.
## Directory layout
```
docs/agent-networks/
├── README.md # you are here
├── 00-overview.md # PR summary + ownership matrix
├── 01-end-to-end-flows.md # cross-module mermaid diagrams
├── 90-agent-review-prompt.md # prompt for an AI code reviewer
└── modules/
├── 10-shared-api.md # proto + OpenAPI wire contracts
├── 20-management-store.md # SQL persistence layer
├── 21-management-agentnetwork.md # domain layer + synthesizer (largest)
├── 22-management-handlers-wiring.md # HTTP API + gRPC delivery
├── 30-proxy-middleware-framework.md # generic plugin system
├── 31-proxy-middleware-builtin.md # 8 LLM-aware middlewares
├── 32-proxy-llm-parsers.md # OpenAI/Anthropic/Bedrock SDKs + pricing
├── 33-proxy-runtime.md # translate + serve + access-log
├── 40-dashboard.md # UI for everything above (lives in the dashboard repo at feature/agent-networks)
└── 50-path-routed-providers.md # Vertex AI + Bedrock (path-routed, keyfile:: creds, /bedrock prefix)
```
The `40-dashboard.md` module documents code that lives in the **dashboard
repo** (`feature/agent-networks` branch), not in this repo. The guide is
co-located here so backend reviewers see the full picture in one place.
## How the per-module guides are structured
Every `modules/*.md` follows the same template so reviewers can scan a
familiar shape:
- **Reviewer profile / time / risk / backward-compat impact** — fits in
a single quote block at the top so triaging the file is a one-glance
decision.
- **Module boundary** — what this package owns within the PR; where it
sits in the stack.
- **Commits in scope** — pinned to the file scope, not the whole PR.
- **Files changed** — path / status / LOC / role.
- **Architecture & flow** — one or more mermaid diagrams.
- **Public contracts** — function signatures, gRPC messages, JSON
shapes, etc.
- **Invariants** — semantic guarantees the module relies on or
enforces.
- **Things to scrutinize** — split by correctness / security /
concurrency / backward-compat / performance / observability.
- **Test coverage** — every test file that locks down behavior in this
module.
- **Known limitations / non-goals** — what reviewers should NOT flag as
bugs (out of scope on purpose).
- **Cross-references** — upstream/downstream module links + the
end-to-end flow + the overview.
## Repos covered
- **Backend (this repo):** `feature/agent-networks-backend` —
28 commits vs merge-base `14af17955`, ~28k net LOC added.
- **Dashboard:** `feature/agent-networks` on
`netbirdio/netbird-dashboard` — ~70 commits vs `main`, ~10k net LOC
added.
See [00-overview.md](00-overview.md) for the full ownership matrix,
commit roll-up, and cross-cutting risk hot-list.

117
docs/agent-networks/modules/10-shared-api.md Normal file
View File

@@ -0,0 +1,117 @@
# shared/api — wire contracts (proto + OpenAPI)
> **Reviewer profile:** API/contracts maintainer; protobuf field-number discipline + oapi-codegen experience; needs to recognise additive vs. breaking schema changes.
> **Time to review:** 30-45 minutes
> **Risk level:** Medium — wire-format surface that every other module pins against; backward-compat hinges on field-number discipline more than on logic correctness.
> **Backward-compat impact:** Additive only (new proto fields use unallocated numbers, new RPCs default to `Unimplemented`, new OpenAPI schemas/paths are append-only; no existing field/RPC/schema removed or renumbered).
## Module boundary
This module owns the cross-process contract surface between management, proxy, and dashboard. Two artefacts: `shared/management/proto/proxy_service.proto` (management↔proxy gRPC) and `shared/management/http/api/openapi.yml` (dashboard/CLI↔management REST). Both have generated companions checked in (`proxy_service.pb.go`, `proxy_service_grpc.pb.go`, `types.gen.go`) which must travel in lockstep with their sources. `shared/management/status/error.go` is in scope only for the four new typed `NotFound` constructors that the new HTTP handlers return.
Everything downstream — `management/agentnetwork`, `management/server/http/handlers/*`, `proxy/internal/*`, the dashboard SDK — consumes these types verbatim. A review here is about wire stability and codegen reproducibility, not behaviour: the behaviour reviews live in the management and proxy module guides.
## Commits in scope
| SHA | Subject | LOC delta (this scope) |
| --- | ------- | --------- |
| 06ff17b38 | AN-0: additive proto + OpenAPI schemas + base types | +2374 / -774 (yml + types.gen.go regenerated) |
| f810a4e35 | AN-1: store layer for providers/policies/guardrails/settings | +1 status/error.go (NotFound helpers — providers/policies/guardrails) |
| a436b5fb3 | GC-0: account budget rule type + collection toggles + store CRUD | +1 status/error.go (`NewAgentNetworkBudgetRuleNotFoundError`) |
| b22d5a181 | GC-2: enforce account budget rules as a min-wins ceiling | proto: `RecordLLMUsageRequest.group_ids = 8` |
| 23bdf6871 | GC-4: HTTP API for budget rules + settings update | +293/+72 openapi.yml/types.gen.go (BudgetRule schemas, settings PUT) |
| 468875cb4 | Wire EnableLogCollection to suppress access-log | proto: `PathTargetOptions.disable_access_log = 13` |
`management.proto` and `signalexchange.proto` are unchanged. `status/error.go` only receives four additive constructors (lines 208-227); no existing error types are reshaped.
## Files changed
| Path | Status | LOC | Role |
| ---- | ------ | --- | ---- |
| `shared/management/proto/proxy_service.proto` | Modified | +125 / -0 | Source of truth: 2 new RPCs, 1 new message group (`MiddlewareConfig` + slot enum), additive fields on `PathTargetOptions`, `AccessLog`, `RecordLLMUsageRequest` |
| `shared/management/proto/proxy_service.pb.go` | Regenerated | +1932 / -1023 | protoc-gen-go v1.26.0 / protoc v7.34.1 output |
| `shared/management/proto/proxy_service_grpc.pb.go` | Regenerated | +88 / -0 | Adds `CheckLLMPolicyLimits` + `RecordLLMUsage` client/server stubs and `UnimplementedProxyServiceServer` defaults |
| `shared/management/http/api/openapi.yml` | Modified | +2371 / -... | 15 new `AgentNetwork*` schemas, 9 new path groups under `/api/agent-network/*` |
| `shared/management/http/api/types.gen.go` | Regenerated | +504 / -0 | oapi-codegen v2.7.0 output (per commit msg; see codegen note below) |
| `shared/management/status/error.go` | Modified | +20 / -0 | Four `NotFound` constructors for the new resource kinds (lines 208-227) |
## Architecture & flow
```mermaid
sequenceDiagram
participant Dash as Dashboard / CLI
participant Mgmt as management (HTTP+gRPC)
participant Px as proxy
Note over Dash,Mgmt: REST (OpenAPI / types.gen.go)
Dash->>Mgmt: PUT /api/agent-network/providers (AgentNetworkProviderRequest)
Dash->>Mgmt: PUT /api/agent-network/settings (AgentNetworkSettingsRequest)
Dash->>Mgmt: GET /api/agent-network/consumption -> [AgentNetworkConsumption]
Note over Mgmt,Px: gRPC ProxyService (proxy_service.proto)
Mgmt-->>Px: SyncMappingsResponse{ ProxyMapping.path[*].options.middlewares,<br/>agent_network, disable_access_log, capture_* }
Px->>Mgmt: CheckLLMPolicyLimits(account, user, groups, provider, model)
Mgmt-->>Px: decision=allow|deny + selected_policy_id + attribution_group_id + window_seconds
Px->>Mgmt: RecordLLMUsage(account, user, group_id, group_ids, window_seconds, tokens, cost)
Px->>Mgmt: SendAccessLog(AccessLog{ agent_network=true })
```
The proto changes split into three independent slices: (1) **mapping enrichment** — `PathTargetOptions` grows fields 8-13 so management can ship middleware configs, capture limits, and the agent-network / log-suppression flags down to the proxy without a second RPC; (2) **two new request/response RPCs** (`CheckLLMPolicyLimits`, `RecordLLMUsage`) for per-LLM-request budget arbitration; (3) **observability tag** — `AccessLog.agent_network` so management can route logs to the right surface.
The OpenAPI side is a thin CRUD surface — every resource (`Provider`, `Policy`, `Guardrail`, `BudgetRule`, `Settings`) follows the same `GET-list / POST / GET / PUT / DELETE` pattern, plus a read-only `/consumption` listing and a catalog endpoint. The `*Request` variants drop server-controlled fields (id, timestamps). `AgentNetworkBudgetRule` deliberately reuses `AgentNetworkPolicyLimits` to keep wire-shape parity with policies (GC-4 commit msg).
## Public contracts added
- gRPC RPCs (`proxy_service.proto:52-57`): `CheckLLMPolicyLimits(CheckLLMPolicyLimitsRequest) → CheckLLMPolicyLimitsResponse`, `RecordLLMUsage(RecordLLMUsageRequest) → RecordLLMUsageResponse`. Both unary; default `UnimplementedProxyServiceServer` returns `codes.Unimplemented` (`proxy_service_grpc.pb.go:283-289`).
- New messages (`proxy_service.proto:145-175,448-502`): `MiddlewareConfig`, `MiddlewareSlot` enum, `CheckLLMPolicyLimitsRequest`/`Response`, `RecordLLMUsageRequest`/`Response`.
- New `PathTargetOptions` fields 8-13 (`proxy_service.proto:124-140`): `capture_max_request_bytes`, `capture_max_response_bytes`, `capture_content_types`, `middlewares`, `agent_network`, `disable_access_log`. All default-false / zero; pre-existing fields 1-7 byte-for-byte unchanged.
- `AccessLog.agent_network = 18` (`proxy_service.proto:258-261`).
- `RecordLLMUsageRequest.group_ids = 8` (`proxy_service.proto:496-498`) — added in GC-2 specifically so the record path can fan out to every applicable budget rule's window without a re-lookup.
- 15 new OpenAPI component schemas (`openapi.yml:5072-5829`): `AgentNetworkProvider[Request|Model]`, `AgentNetworkCatalog{Model,Provider,IdentityInjection,HeaderPairInjection,JSONMetadataInjection,ExtraHeader}`, `AgentNetworkPolicy[Request|TokenLimit|BudgetLimit|Limits]`, `AgentNetworkGuardrail[Checks|Request]`, `AgentNetworkConsumption`, `AgentNetworkSettings[Request]`, `AgentNetworkBudgetRule[Request]`.
- 9 new path groups (`openapi.yml:12797-13460`): `/api/agent-network/{consumption,settings,budget-rules,budget-rules/{ruleId},catalog/providers,providers,providers/{providerId},policies,policies/{policyId},guardrails,guardrails/{guardrailId}}`.
- Four typed NotFound errors (`shared/management/status/error.go:208-227`).
## Invariants
- **Field-number monotonicity.** Every new proto field uses a previously-unallocated number in its message: `PathTargetOptions` 8-13 (was 1-7), `AccessLog` 18 (was 1-17), `RecordLLMUsageRequest` 8. `SendStatusUpdateRequest.inbound_listener = 50` (pre-existing) reserves 50+ for observability extensions, so 8 on `RecordLLMUsageRequest` doesn't conflict.
- **Old proxies stay compatible.** Old management never sends `disable_access_log`/`middlewares`/`agent_network` (zero value → existing behaviour); old proxies that don't decode these fields just drop them silently (proto3 unknown-field semantics) — log emission stays on. AN-0's commit message explicitly claims "private-service proto/OpenAPI content is byte-for-byte untouched (0 deletions in proxy_service.proto and openapi.yml)" — verify by diff stat that no pre-existing field number changed.
- **Old management stays compatible.** The two new RPCs are registered on the same `management.ProxyService` descriptor; old proxies hitting them get `codes.Unimplemented` from the unimplemented embed (`proxy_service_grpc.pb.go:283-289`), which is the same fallback pattern `SyncMappings` already documents (`proxy_service.proto:20-21`).
- **OpenAPI shapes are append-only.** New schemas are placed at the end of `components.schemas` (line 5072+); new paths at the end of `paths` (line 12797+). No existing schema's `required` list, enum, or property type was changed.
- **`*Request` vs response asymmetry.** Read shapes (`AgentNetworkProvider`, `AgentNetworkPolicy`, `AgentNetworkGuardrail`, `AgentNetworkSettings`, `AgentNetworkBudgetRule`) require `created_at`/`updated_at`; the matching `*Request` shapes do not — server fills them. `AgentNetworkProviderRequest.api_key` is write-only (`openapi.yml:5158-5161` "never returned in responses"); reviewers should confirm the response schema (5072-5138) actually omits `api_key`.
## Things to scrutinize
### Correctness
- `RecordLLMUsageRequest` carries both `group_id` (singular, the attribution group — field 3) and `group_ids` (plural, full membership — field 8). `b22d5a181` adds field 8 to drive account-budget fan-out; double-check that consumers can't accidentally key counters on the wrong one. Field comments at `proxy_service.proto:489-491` and `496-498` distinguish them but it's the kind of subtle thing a follow-up commit might collapse.
- `PathTargetOptions.disable_access_log` is the only field whose default-false meaning **changes semantics** on the proxy side: false → log (status quo), true → suppress. Synthesizer sets `DisableAccessLog = !settings.EnableLogCollection`, so a missing/default settings row yields `EnableLogCollection=false → DisableAccessLog=true → suppressed`. Worth confirming downstream (`agentnetwork.synthesizer`) that operator-defined private services never inherit this flag — the proto field default protects them, but only if synth code is explicit.
- `CheckLLMPolicyLimitsResponse.decision` is a free-form `string` (`proxy_service.proto:471`) rather than an enum. Only documented values are "allow" / "deny". An enum would prevent typo drift; consider before this RPC ships to external consumers.
- `deny_code` (`proxy_service.proto:478-481`) is documented as "a stable label" but is also a free string. Pin the allowed set somewhere observable to the proxy.
### Security
- `AgentNetworkProvider.api_key` MUST be write-only. Schema split (request has it at line 5158; response omits it) looks correct, but a regression here leaks the upstream provider credential to every dashboard reader. Check that the handler explicitly zeros it on the response path.
- `extra_values` / `identity_header_*` headers on `AgentNetworkProvider` get stamped onto upstream requests. Description at `openapi.yml:5099` says "values not declared by the catalog are ignored at synth time" — a contract this module documents but the synthesizer must enforce. Confirm the synth module honours it.
- Cluster + subdomain on `AgentNetworkSettings` are documented immutable (`openapi.yml:5686-5694`) and the `AgentNetworkSettingsRequest` (lines 5733-5752) doesn't accept them. Verify the `PUT /api/agent-network/settings` handler can't be tricked by extra JSON keys (oapi-codegen's `additionalProperties: false` is not declared here; spec defaults to permissive).
### Backward compatibility
- The proto file claims field-number additivity; spot-check by diffing the merge-base proto against HEAD — every numbered field present at merge-base must still have the same name + type. The AN-0 commit message asserts "0 deletions in proxy_service.proto"; `git diff --stat` shows 125 insertions and 0 deletions, so this holds at the source-text level.
- `proxy_service_grpc.pb.go` adds two RPC handlers and registers them in `ProxyService_ServiceDesc.Methods` (lines 543-552). The existing entries are unchanged and order-preserving — gRPC method dispatch is name-keyed, so order doesn't matter, but reviewing the diff (no method renamed/dropped) is still worth a glance.
- OpenAPI 3.0 doesn't have a built-in deprecation flow for paths; if any client tooling iterates `paths.*`, the additive routes shouldn't break it, but generated SDKs (especially the dashboard's) need a regen to gain access to `AgentNetwork*`.
### Codegen pinning
- `generate.sh` (`shared/management/http/api/generate.sh:14`) installs `oapi-codegen@latest`, yet the AN-0 commit message claims v2.7.0 was used. **This is a reproducibility gap** — re-running the script tomorrow may produce a different `types.gen.go`. Either pin the version in `generate.sh` (`@v2.7.0`) or document the pin in a `tools.go`.
- proto codegen has the protoc version stamped in the generated file header (`proxy_service.pb.go:3-4` — protoc-gen-go v1.26.0 / protoc v7.34.1), matching the commit message. Good.
- Reviewers should regenerate locally and confirm zero diff against the committed `types.gen.go` / `proxy_service.pb.go` before merge.
## Test coverage
| Test file | Locks down |
| --------- | ---------- |
| None in this scope | The proto and OpenAPI sources are tested transitively by the handler tests (`shared/management/http/handlers/agentnetwork/...`) and by the synthesizer/manager tests (`management/server/agentnetwork/...`). No round-trip serialisation test exists in the `proto/` or `api/` packages themselves. |
| `shared/management/proto/*_test.go` | (absent) |
| `shared/management/http/api/*_test.go` | (absent) |
Acceptable for codegen artefacts, but a single golden-file test that re-runs `oapi-codegen` and `protoc` in CI and diffs against the checked-in files would close the reproducibility gap noted above.
## Known limitations / explicit non-goals
- **No deprecation surface.** Old fields/RPCs are kept silently; there is no `[deprecated = true]` annotation on anything. Acceptable here because nothing is being removed.
- **No proto-side validation.** Numeric ranges (e.g. `window_seconds >= 60`, `cost_usd >= 0`, capture-byte clamps) are enforced in the OpenAPI schema via `minimum:` and inside Go code by the proxy/management, but `proto3` itself can't express them; downstream is expected to validate every message.
- **`MiddlewareConfig.config_json` is `bytes`** (`proxy_service.proto:163`) — opaque to the proto layer. Schema validity is the middleware factory's problem. This is a deliberate tradeoff (per the comment at 161-162) but worth flagging: a corrupted/malicious config_json can only fail at proxy apply time, not at the wire-decode step.
- **No catalog endpoint schema for the catalog itself** — the catalog data ships as a `GET /api/agent-network/catalog/providers` returning `[AgentNetworkCatalogProvider]` (`openapi.yml:13024`), but the catalog source-of-truth lives in `management/server/agentnetwork/catalog`, not here.
- AN-0's commit message notes "Reaper design dropped per scope decision; not migrated." No reaper-related types appear in this scope — confirmed.
## Cross-references
- Downstream: [management/store](20-management-store.md), [management/agentnetwork](21-management-agentnetwork.md), [management/http handlers](22-management-http-handlers.md), [proxy/runtime](33-proxy-runtime.md)
- End-to-end flow: [../01-end-to-end-flows.md](../01-end-to-end-flows.md)
- Top-level: [../00-overview.md](../00-overview.md)

122
docs/agent-networks/modules/20-management-store.md Normal file
View File

@@ -0,0 +1,122 @@
# management/store — persistence for agent-network entities
> **Reviewer profile:** Storage maintainer; gorm + sqlite/postgres/mysql + AutoMigrate semantics, field-level encryption, upsert/`ON CONFLICT` correctness across engines.
> **Time to review:** 35-45 minutes
> **Risk level:** Medium — six brand-new tables behind AutoMigrate, one upsert-counter table that runs on the request hot path, and one column carrying an encrypted secret.
> **Backward-compat impact:** Additive (six new tables created by AutoMigrate; the `Store` interface gains 23 methods, but no existing column/index is touched).
## Module boundary
This module is the persistence layer for the Agent Network feature. Everything the management server stores about LLM proxying — providers, policies, guardrails, the per-account settings row, a usage-counter table written on every proxied LLM request, and the GC-0 account-budget rules — flows through the methods added to `store.Store`. The module owns six tables, six entity types from `management/server/agentnetwork/types`, and a single hot-path upsert (`IncrementAgentNetworkConsumption`) consumed by the proxy fleet.
Out of scope here: the catalog of provider definitions (compiled-in, no DB), the synthesizer/manager built on top of these CRUDs (covered in [21-management-agentnetwork.md](21-management-agentnetwork.md)), and the HTTP handlers that translate API requests into Save/Delete calls.
## Commits in scope
| SHA | Subject | LOC delta |
| --- | ------- | --------- |
| f810a4e35 | AN-1: store layer for providers/policies/guardrails/settings | +552 |
| 77b407632 | AN-2: agentnetwork module — adds `GetAllAgentNetworkProviders` + consumption methods to the store interface | +161 |
| a436b5fb3 | GC-0: account budget rule type + collection toggles + store CRUD | +241/-1 |
## Files changed
| Path | Status | LOC | Role |
| ---- | ------ | --- | ---- |
| `management/server/store/sql_store_agentnetwork.go` | new | 466 | gorm implementations of all 23 store methods |
| `management/server/store/sql_store_agentnetwork_budgetrule_test.go` | new | 112 | round-trip + account-scoping coverage against a real sqlite store |
| `management/server/store/sql_store.go` | additive | +3 | one import, six entities appended to the `AutoMigrate` slice (sql_store.go:40, sql_store.go:141-142) |
| `management/server/store/store.go` | additive | +26 | 23 methods added to the `Store` interface (store.go:328-354) |
| `management/server/store/store_mock_agentnetwork.go` | new | 346 | mockgen output for the new interface surface |
## Tables added / migrations
All six tables are created by `db.AutoMigrate` invoked from `NewSqlStore` at sql_store.go:133-143. There is no hand-rolled SQL migration script — the schema is whatever GORM derives from the struct tags.
- `agent_network_providers` — `Provider.TableName()` at provider.go:76. PK `id`, index on `account_id`, named index `idx_agent_network_provider` on `provider_id`. Carries an at-rest-encrypted `api_key` and ed25519 `session_private_key` (provider.go:35,56). `extra_values` and `models` are JSON blobs (`serializer:json`).
- `agent_network_policies` — `Policy.TableName()` at policy.go:70. PK `id`, index on `account_id`. JSON columns: `source_groups`, `destination_provider_ids`, `guardrail_ids`, `limits`.
- `agent_network_guardrails` — `Guardrail.TableName()` at guardrail.go:41. PK `id`, index on `account_id`. JSON `checks`.
- `agent_network_settings` — `Settings.TableName()` at settings.go:33. PK `account_id` (one row per account), named index `idx_agent_network_settings_cluster_subdomain` on `subdomain` only — the index name implies a composite, but only one column is tagged.
- `agent_network_consumption` — `Consumption.TableName()` at consumption.go:46. Composite PK across `(account_id, dim_kind, dim_id, window_seconds, window_start_utc)` — the same tuple the upsert keys on.
- `agent_network_budget_rules` — `AccountBudgetRule.TableName()` at budgetrule.go:35. PK `id`, index on `account_id`. JSON `target_groups`, `target_users`, `limits`.
## CRUD surface added
Provider, Policy, Guardrail, BudgetRule follow the same pattern: `Get<Kind>ByID`, `GetAccount<Kind>` (list), `Save<Kind>` (upsert), `Delete<Kind>`, with account-scoping enforced by the existing `accountAndIDQueryCondition` / `accountIDCondition` constants (sql_store.go:59-62). Provider additionally exposes `GetAllAgentNetworkProviders` (cross-account, used by the synthesizer). Settings exposes `Get`/`GetByCluster`/`Save` (no delete — one row per account, created on first save). Consumption exposes the upsert `Increment`, a point `Get`, and a cross-window `List`.
## Architecture & flow
```mermaid
flowchart LR
handlers["HTTP handlers<br/>(management/server/agentnetwork)"] -->|Save/Delete| iface["Store interface<br/>store.go:328-354"]
manager["agentnetwork.Manager"] -->|Get*| iface
synth["synthesizer<br/>(global)"] -->|GetAllAgentNetworkProviders| iface
proxy["proxy fleet<br/>(hot path)"] -->|IncrementAgentNetworkConsumption| iface
iface --> sql["SqlStore methods<br/>sql_store_agentnetwork.go"]
iface -.gomock.-> mock["MockStore<br/>store_mock_agentnetwork.go"]
sql --> gorm["gorm.DB"]
gorm --> tables[("6 tables<br/>agent_network_*")]
sql --> enc["crypt.FieldEncrypt<br/>(provider only)"]
```
Reads decrypt provider secrets in-place; writes do `provider.Copy().EncryptSensitiveData(...)` before `db.Save` so the caller's in-memory object keeps the plaintext `api_key` (sql_store_agentnetwork.go:88-102). Every list/get takes a `LockingStrength` and applies `clause.Locking{Strength: ...}` when non-`None` — matching the rest of the store. The upsert path uses `clause.OnConflict` with `gorm.Expr` server-side increments so concurrent proxy nodes converge without read-modify-write races (sql_store_agentnetwork.go:321-335).
## Invariants enforced at the store layer
- **Account scoping.** Every entity-by-ID method keys on `account_id = ? and id = ?`; no cross-tenant leak path through the API is reachable as long as callers always pass the auth'd `accountID` (sql_store_agentnetwork.go:70,141,201,429).
- **NotFound mapping.** `gorm.ErrRecordNotFound` is translated to typed `status.NewAgentNetwork*NotFoundError`; `Delete*` returns NotFound when `RowsAffected == 0` (sql_store_agentnetwork.go:111-113,171-173,231-233,461-463).
- **Provider secret encryption at rest.** `SaveAgentNetworkProvider` always encrypts before persist; `Get*` always decrypts after read. The plaintext `api_key` never reaches the DB through this layer (sql_store_agentnetwork.go:31,54,80,90).
- **Consumption monotonicity.** The upsert only ever issues `col = col + ?` for the three counter columns — no decrement path exists (sql_store_agentnetwork.go:330-332).
- **Window alignment is the caller's responsibility.** The store stamps `WindowStartUTC` as-passed; alignment to epoch happens in `types.WindowStart` at consumption.go:51-58.
- **Settings has no Delete.** Intentional — one row per account, created on first save; the row sticks around for the account lifetime.
## Things to scrutinize
### Correctness
- `SaveAgentNetworkProvider` saves the copy (sql_store_agentnetwork.go:95). The caller's in-memory pointer therefore keeps plaintext `api_key` and any `CreatedAt`/`UpdatedAt` gorm autofills land on the copy, not the original. Confirm with the AN-2 manager whether callers immediately re-fetch (they should, since timestamps don't sync back).
- `IncrementAgentNetworkConsumption`'s `Create` provides initial counter values (`TokensInput: tokensIn`, etc.) in the row, and on conflict the assignments add the same deltas to the existing values. The insert-vs-update arithmetic is consistent. Cross-check that no engine in use (sqlite, postgres, mysql) silently rejects the `OnConflict` clause — GORM emits engine-specific SQL but `ON DUPLICATE KEY UPDATE` (mysql) vs `ON CONFLICT (...)` (sqlite/postgres) need their unique constraint to match the composite PK on `agent_network_consumption`; it does, by construction.
- `IncrementAgentNetworkConsumption` writes `updated_at: time.Now().UTC()` literally inside the assignments map (sql_store_agentnetwork.go:333) — fine, but it's a Go-side timestamp captured at call time, not a DB-side `now()`. Acceptable for an audit field.
- `GetAgentNetworkConsumption` returns a zero-valued non-nil row on `ErrRecordNotFound` (sql_store_agentnetwork.go:364-371). Document or rename — a typed sentinel error would be more orthodox; callers must know not to error-check.
### Concurrency / transactions
- Hot-path `IncrementAgentNetworkConsumption` runs outside any explicit transaction; concurrency safety relies entirely on the DB serialising the `ON CONFLICT` upsert against the composite PK. This is correct for postgres and mysql; for sqlite it serialises behind the single writer.
- `SaveAgentNetworkSettings` is a blind upsert with no version/etag — concurrent writes from two operators last-write-wins on the collection-toggle flags (settings.go:23-25). Acceptable for admin-curated state but worth flagging.
- `Save*Provider` uses `db.Save` on a struct with a PK already set — GORM emits UPDATE or INSERT based on row existence. No upsert clause is attached, so a race between two creates with the same generated `xid` (vanishingly unlikely) would surface as a PK violation.
### Migration safety
- All six tables ride `AutoMigrate` (sql_store.go:141-142). AutoMigrate is additive: new columns get added, but it never drops columns nor narrows types. The GC-0 commit appends three `bool` columns to `agent_network_settings` (`EnableLogCollection`, `EnablePromptCollection`, `RedactPii`) — for existing rows these default to false at the GORM/DDL layer; the test at sql_store_agentnetwork_budgetrule_test.go:83-112 locks that down on a fresh sqlite. Verify postgres/mysql produce the same default.
- The named index `idx_agent_network_settings_cluster_subdomain` on settings.go:15 is declared on only `subdomain`. Either the cluster column also needs `gorm:"index:idx_agent_network_settings_cluster_subdomain"` to make it composite, or the name is misleading.
- The named index `idx_agent_network_provider` on `Provider.ProviderID` (provider.go:30) is *not* unique and not scoped to account — two providers in the same account with the same `provider_id` are permitted at the DB layer; uniqueness, if any, must live above the store.
### Backward compatibility
- Net additive. No removed methods, no renamed columns, no schema change to existing tables. Existing deployments running a prior binary continue to work; the first boot of the new binary creates the six tables.
- The `Store` interface grows by 23 methods (store.go:330-354); any non-mock external implementer of `store.Store` will fail to compile. The repo only has `SqlStore` + `MockStore`, both updated.
### Performance (indexes, N+1)
- All by-account list queries hit the `idx_account_id` per-table index. No N+1: list methods return the full slice in one query.
- `GetAgentNetworkSettingsByCluster` (sql_store_agentnetwork.go:263-277) does a tablescan on `cluster` — no index. Tolerable for the bootstrap label generator (one-shot at provisioning) but worth noting if the call moves onto a hot path.
- `ListAgentNetworkConsumption` returns every row ever recorded for the account (sql_store_agentnetwork.go:382-400) — unbounded growth, no `LIMIT`, no time filter. With one row per (dim, window) per request burst, this table grows fastest of the six; a retention job + a paginated list method are obvious follow-ups.
## Test coverage
| Test file | Locks down |
| --------- | ---------- |
| `sql_store_agentnetwork_budgetrule_test.go::TestAgentNetworkBudgetRule_RealStore_RoundTrip` | full save → reload of `AccountBudgetRule` including the JSON-serialised `PolicyLimits`, target slices, double-delete returns NotFound (lines 18-59) |
| `sql_store_agentnetwork_budgetrule_test.go::TestAgentNetworkBudgetRule_RealStore_ScopedByAccount` | cross-account isolation for budget rules (lines 63-78) |
| `sql_store_agentnetwork_budgetrule_test.go::TestAgentNetworkSettings_RealStore_CollectionTogglesRoundTrip` | GC-0 toggles default off, survive save/reload at the set values (lines 83-112) |
Gap: there is no store-level test for providers (encryption round-trip), policies, guardrails, or `IncrementAgentNetworkConsumption` (concurrent upsert, window-key uniqueness). The consumption upsert is the most performance-sensitive method in this module and the only one without a real-sqlite test.
## Known limitations / explicit non-goals
- No retention / GC for `agent_network_consumption`.
- No `Delete` for `Settings` (one row per account, cleared with the account).
- No DB-engine-specific tuning — the same struct tags drive sqlite, mysql, postgres.
- Provider `extra_values` and `models` are JSON blobs; querying inside them is not supported by design.
- `GetAgentNetworkConsumption` "not-found = zero row" contract is convenient but unconventional.
## Cross-references
- Upstream: [shared/api](10-shared-api.md), [management/agentnetwork](21-management-agentnetwork.md)
- End-to-end flow: [../01-end-to-end-flows.md](../01-end-to-end-flows.md)
- Top-level: [../00-overview.md](../00-overview.md)

249
docs/agent-networks/modules/21-management-agentnetwork.md Normal file
View File

@@ -0,0 +1,249 @@
# management/agentnetwork — domain layer + synth pipeline
> **Reviewer profile:** Management server maintainer; familiar with the network-map controller, policy engine, gorm/SQLite store, Go concurrency (sync.Mutex + math/rand), and the reverse-proxy `service`/`proxy` packages + `ProxyMapping` proto shape.
> **Time to review:** 180–240 minutes (largest module; budget for diagram cross-check + per-test triage).
> **Risk level:** High — central business logic + budget enforcement + the source of every middleware-chain change the proxy executes.
> **Backward-compat impact:** Additive within agent-network surface; one **behavioural break for opted-out accounts** in parser capture (capture flag now stamped explicitly false instead of being absent — see capture-pointer semantics below). Non-agent-network proxy services untouched (synth chain only ships on `agent-net-svc-*` targets).
## Module boundary
`management/server/agentnetwork` owns every agent-network entity (providers, policies, guardrails, account budget rules, per-account settings, consumption rows) and **translates them into the in-memory `*rpservice.Service` that the reverse-proxy controller turns into `proto.ProxyMapping`s and pushes to clusters**. It is the *only* writer of the agent-network middleware chain.
Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthesizer.go` walks settings + providers + policies + guardrails and emits the per-account service plus every middleware's JSON config; `policyselect.go` runs per-request attribution (min-wins account ceiling, then "drain bigger pool first"); `reconcile.go` diffs successive synth outputs and emits precise Create/Update/Delete proxy-mapping updates plus a peer-map refresh. `labelgen/` mints DNS-safe subdomain labels; `catalog/` is the static provider catalogue; `types/` carries gorm entity structs. The `_realstack_test.go` files at the parent `management/server/` directory exercise the manager + network-map controller end-to-end with no mocks.
## Commits in scope
| SHA | Subject | LOC delta |
| --- | ------- | --------- |
| `06ff17b38` | AN-0: additive base types | +804 |
| `77b407632` | AN-2: agentnetwork module (manager, synth, catalog, policyselect) | +5049 |
| `09e8059b6` | AN-2b: wire synth services into network map | +12/−13 |
| `9ebe219fd` | test: lock auth→middleware group-propagation | +3 |
| `665575932` | test: real-store coverage (no MockStore) | +174 |
| `5adee2cb4` | no-mock provider-CRUD fan-out test | +212 |
| `9ae476ea7` | no-mock baseline guards for enforcement + guardrail synth | +314 |
| `a436b5fb3` | GC-0: budget rule type + collection toggles | +64 |
| `5b408b0ef` | GC-1: budget-rule manager CRUD + settings update | +281 |
| `b22d5a181` | GC-2: account budget rules as min-wins ceiling | +344/−32 |
| `945f17f1a` | GC-3: account prompt-collection master switch | +108/−36 |
| `23bdf6871` | GC-4: HTTP API surface (types-side delta) | +61/−5 |
| `468875cb4` | EnableLogCollection → DisableAccessLog on synth target | +71 |
| `7072f8125` | Account toggle = sole capture control + broader redact | +23/−17 |
| `b438a7194` | Budget enforcement fix + extended PII redaction | +26/−13 |
| `19e03d688` | Tests for redact-pii wiring across parsers | +97 |
| `4836d5a19` | **Live-bug fix: gate `capture_prompt`/`capture_completion` on EnablePromptCollection** | +104/−14 |
## Files changed
| Path | Status | LOC | Role |
| ---- | ------ | --- | ---- |
| `agentnetwork/manager.go` | new | 788 | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | new | 959 | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/policyselect.go` | new | 594 | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | new | 131 | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | new | 685 | Static provider catalogue (auth headers, identity-injection shapes) |
| `agentnetwork/labelgen/{labelgen,words}.go` | new | 202 | DNS-safe subdomain picker + curated wordlist |
| `agentnetwork/types/provider.go` | new | 252 | Provider entity + APIKey + Models + ExtraValues + SessionKeys |
| `agentnetwork/types/policy.go` | new | 192 | Policy entity + `PolicyLimits` (token + budget) |
| `agentnetwork/types/guardrail.go` | new | 120 | Guardrail entity (`ModelAllowlist`, `PromptCapture`) |
| `agentnetwork/types/budgetrule.go` | new | 106 | `AccountBudgetRule` (reuses `PolicyLimits`) |
| `agentnetwork/types/settings.go` | new | 63 | Per-account `Settings` (Cluster, Subdomain, 3 toggles) |
| `agentnetwork/types/consumption.go` | new | 58 | `Consumption` row + `WindowStart` aligner |
| `agentnetwork/{synthesizer,policyselect,reconcile,wire_shape}_*test.go` | new | 2867 | See test coverage table |
| `agentnetwork/types/consumption_test.go` | new | 141 | `WindowStart` alignment proofs |
| `agentnetwork/labelgen/labelgen_test.go` | new | 101 | Deterministic picks + exhaustion + fallback |
| `management/server/agentnetwork_realstack_test.go` | new | 212 | No-mock provider CRUD → network-map fan-out |
| `management/server/agentnetwork_budgetrule_realstack_test.go` | new | 126 | No-mock budget-rule CRUD + settings preserve-immutable |
## Architecture & flow
### Synthesis (settings/policy → wire format)
```mermaid
flowchart TD
A[Mutation: provider/policy/guardrail/settings] --> B[managerImpl.reconcile accountID]
B --> C{proxyController nil?}
C -- yes --> D[accountManager.UpdateAccountPeers only]
C -- no --> E[SynthesizeServices]
E --> F[loadSettings — NotFound returns ok=false, no synth]
F --> G[filterEnabledProviders sorted by CreatedAt]
G --> H[filterEnabledPolicies]
H --> I[backfillProviderSessionKeys if missing]
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
K --> P[buildMiddlewareChain 8 middleware entries]
L --> P
O --> P
P --> Q[buildAccountService: AccessGroups=union source groups, noop.invalid target]
Q --> R[reconcile.diffMappings vs cache]
R --> S[SendServiceUpdateToCluster CREATE/MODIFY/REMOVE]
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### Budget rule resolution (min-wins, group+user bound)
```mermaid
flowchart TD
A[SelectPolicyForRequest in] --> B[checkAccountBudget — runs FIRST, independent of policies]
B --> C[GetAccountAgentNetworkBudgetRules]
C --> D{for each enabled rule}
D --> E{budgetRuleApplies?}
E -- no --> D
E -- yes --> F[attrGroup = lowestIntersect TargetGroups, in.GroupIDs]
F --> G{Token cap enabled?}
G -- yes --> H[evalTokenCap user dim + group dim]
H --> I{exhausted?}
I -- yes --> J[DENY: llm_account.token_cap_exceeded - STOP]
I -- no --> K{Budget cap enabled?}
G -- no --> K
K -- yes --> L[evalBudgetCap user dim + group dim]
L --> M{exhausted?}
M -- yes --> N[DENY: llm_account.budget_cap_exceeded - STOP]
M -- no --> D
K -- no --> D
D --> O[All rules passed -> fall through to per-policy selection]
```
Key invariant: **rules are checked sequentially and ANY exhausted rule denies (all-must-pass / min-wins).** Untargeted rules (`len(TargetGroups)==0 && len(TargetUsers)==0`) apply to every caller (`policyselect.go:393`).
### Policy selection (per-peer, per-request)
```mermaid
flowchart TD
A[Account-budget gate passed] --> B[GetAccountAgentNetworkPolicies]
B --> C[filterApplicablePolicies enabled + provider match + group intersect]
C --> D{candidates empty?}
D -- yes --> E[Allow, empty SelectedPolicyID]
D -- no --> F[scoreCandidates -> scoreOne per policy]
F --> G[scoreOne: attrGroup + window]
G --> H{any cap exhausted?}
H -- yes --> I[Drop policy; record last deny code]
H -- no --> K[Keep as live candidate]
F --> L{live candidates exist?}
L -- no --> M[Deny with last exhaustion code]
L -- yes --> N[Sort: uncapped wins -> larger group token -> group budget -> user token -> user budget -> oldest CreatedAt]
N --> O[winner = scored 0]
O --> P[Allow + SelectedPolicyID + AttributionGroupID + WindowSeconds]
```
End-to-end: a mutation calls `managerImpl.reconcile(ctx, accountID)` (`manager.go:205,239,...`). Reconcile defers an `accountManager.UpdateAccountPeers` so the network-map controller re-runs and `injectAllProxyPolicies` picks up the new access groups; with a `proxyController` wired, it re-synthesizes the service, diffs against `reconcileCache[accountID]` (guarded by `reconcileMu`), and emits proto mappings to the cluster derived from the mapping's domain (`reconcile.go:120`). Synthesis is stateless and idempotent. Sole persistent side effect: `backfillProviderSessionKeys` (`synthesizer.go:249`) mints ed25519 keys on legacy provider rows and writes them back.
At request time the path is independent: the proxy calls `SelectPolicyForRequest` (`policyselect.go:56`); account-budget ceiling first, then per-policy scoring. Token + budget caps share `evalTokenCap` / `evalBudgetCap` — same primitive for account rules and policy limits, `label` differentiates the deny reason. After a served request, `RecordAccountBudgetUsage` (`policyselect.go:415`) fans deltas to every applicable rule's distinct `(dim_kind, dim_id, window)` tuple, deduplicating to prevent double-count when two rules share target+window.
## Public contracts
- **Manager interface** (`manager.go:48-80`): CRUD for `Providers/Policies/Guardrails/BudgetRules`; `GetSettings/UpdateSettings` (cluster + subdomain immutable, only the three toggles mutate); `ListConsumption/RecordConsumption(account, kind, dimID, windowSec, in, out, USD)`; `RecordAccountBudgetUsage(account, user, groups, in, out, USD)`; `SelectPolicyForRequest(ctx, PolicySelectionInput) → *PolicySelectionResult{Allow, SelectedPolicyID, AttributionGroupID, WindowSeconds, DenyCode, DenyReason}`.
- **`PolicySelectionInput`** (`manager.go:85-90`): `{AccountID, UserID, GroupIDs, ProviderID}` — populated by the proxy from CapturedData + `llm_router` resolution.
- **Synthesized middleware chain** (`synthesizer.go:576-657`), order load-bearing — response slot runs reverse-of-slice:
| Slot | Idx | ID | ConfigJSON shape | CanMutate |
| --- | --- | --- | --- | --- |
| on_request | 0 | `llm_request_parser` | `{"capture_prompt": <bool>, "redact_pii"?: true}` | – |
| on_request | 1 | `llm_router` | `{"providers":[{id, models[], upstream_*, auth_header_*, allowed_group_ids[]}]}` | **true** |
| on_request | 2 | `llm_limit_check` | `{}` | – |
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"model_allowlist"?, "prompt_capture":{enabled,redact_pii}}` | – |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | – |
| on_response | 6 | `cost_meter` | `{}` | – |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | – |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
## Invariants
- **Min-wins / all-must-pass for account budget rules** (`checkAccountBudget`, `policyselect.go:353`): every applicable enabled rule is checked; first exhausted cap denies. Untargeted rules bind every caller. GC-2 contract.
- **Account toggle is the SOLE control for capture enablement.** `applyAccountCollectionControls` (`synthesizer.go:701`) sets `merged.PromptCapture.Enabled = settings.EnablePromptCollection` *unconditionally*.
- **Capture-pointer semantics on parser configs** — see "Things to scrutinize" below.
- **`EnableLogCollection` ↔ `DisableAccessLog` is the only access-log toggle** (`synthesizer.go:770`). Default off ⇒ access log suppressed.
- **`RedactPii` flows verbatim to BOTH parsers** (`synthesizer.go:584-585`) and is OR'd into the merged guardrail (`synthesizer.go:706`).
- **Cluster and Subdomain are immutable on Settings.** `UpdateSettings` reloads existing row and overlays only the three toggles (`manager.go:558-561`).
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
## Things to scrutinize
### Correctness
- **Capture-pointer semantics — `*bool` vs `bool` (the live-bug fix at `4836d5a19`).** Three states, owned by separate sides:
- **Wire JSON this module emits:** `buildParserConfigJSON` (`synthesizer.go:678-693`) *always* stamps the capture field. Agent-network targets ship `"capture_prompt": false` or `"capture_prompt": true` — never absent. Same for `"capture_completion"`. The happy-path test pins `{"capture_prompt":false}` (`synthesizer_test.go:174`).
- **Proxy-side parser config (consumer):** parsers decode into `*bool`. Matrix:
- `nil` (field absent) → **legacy default = emit**. Preserved for non-agent-network callers and pre-existing tests (the backward-compat hook).
- `false` (field present, value false) → **suppress emission entirely**. New behaviour for opted-out agent-network accounts. Without this, `enable_log_collection=true` + `enable_prompt_collection=false` leaked raw user input AND raw model output to the access log (the live bug).
- `true` → emit normally.
- **Why the synth always stamps a value:** an agent-network mapping omitting the field would hit legacy "always emit" and re-introduce the leak. The `json.Marshal` error fallback at `synthesizer.go:687` degrades to `{}` — comment-claimed unreachable, but if ever fired re-introduces the leak. Consider fail-closed (return literal `{"capture_prompt":false}`) instead.
- **`scoreCandidates` non-cumulative deny code.** Only the *last* exhausted policy's deny code survives (`policyselect.go:188-190`). Iteration order is store's natural order. Auth signal is `len(scored)==0`, so this is informational only — verify no UI depends on "first exhausted policy" semantics.
- **`effectiveWindowSeconds` token-wins tiebreak.** When both halves are enabled with different windows, token's window wins (`policyselect.go:482`). Verify `RecordLLMUsage` increments against the winning window only.
- **`RecordAccountBudgetUsage` dedup.** Two rules with the same `(kind, dim_id, window)` would double-count without the `tuples` map (`policyselect.go:434-449`). Key includes all three dimensions — correct.
- **Fail-closed on bad provider:** unknown catalog id (`synthesizer.go:794-796`) or empty API key (`synthesizer.go:801-803`) drops the **entire** account's synth, not just the bad provider. Confirm matches operator UX.
### Security
- **Redact OR-merge:** merged `RedactPii` = account OR guardrail (`synthesizer.go:706`). **Parser-side flag is `settings.RedactPii` only, NOT the OR** — a guardrail-only opt-in does not propagate to parsers. Correct because the account toggle gates capture, but flag for the proxy reviewer.
- **Group resolution must not leak across accounts.** Every store call carries `accountID` (`policyselect.go:73, 286, 298, 322, 334, 354`); `lowestIntersect` uses caller's claimed groups only (`policyselect.go:494`). Risk surface is upstream (handler populates `in.GroupIDs`).
- **`UpdateSettings` preserves immutable Cluster + Subdomain** (`manager.go:558`). A client can't rebind the cluster.
- **Provider session keypair backfill writes through `SaveAgentNetworkProvider`** (`synthesizer.go:256`) from a read-shaped call. Idempotent → worst case is a wasted write under concurrent reconcile + snapshot.
### Concurrency
- **`reconcileMu`** guards `reconcileCache`. Lock window is narrow — compute diff inside, send outside (`reconcile.go:56-68`).
- **`labelRngMu`** guards `labelRng` because `math/rand.Source` is unsafe for concurrent use (`manager.go:638-640`).
- **Real-store tests** use `store.NewTestStoreFromSQL` with `t.TempDir()` per test — no shared state, no `t.Parallel()`.
- **`RecordAccountBudgetUsage` dedup `tuples` map is per-call;** concurrent calls fan out fully — correct (each request's tokens book once per applicable rule).
- **Deferred `UpdateAccountPeers` runs inline after the proxy push** (`reconcile.go:28-35`); a slow call stretches CRUD response time.
### Backward compatibility
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
### Observability
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
## Test coverage
| Test file | Locks down |
| --------- | ---------- |
| `synthesizer_test.go` | Mock-store: `HappyPath` (8-mw chain ordering, `{"capture_prompt":false}` baseline); `No{Settings,Providers}`; `Disabled{Provider,Policy}_NoService`; `RouterConfigOrdering`; `PolicyCheckConfig_UnionsSourceGroups`; `OrphanProvider_HasEmptyAllowedGroups`; identity-inject for LiteLLM / Bifrost (overrides + partial disable) / Cloudflare / Portkey / Vercel / OpenRouter / generic non-customizable; `GuardrailMerge_AllowlistUnion_LimitsRestrictive`; `BackfillsMissingSessionKeys`; `HTTPUpstream_KeepsExplicitPort`; `UpstreamURLPath_FlowsToRouter`; `UnknownProviderID_FailsClosed`; `EmptyAPIKey_FailsClosed`. |
| `synthesizer_realstore_test.go` | Real-sqlite: `SurvivesStatusToggle` reproduces the live 403 via disable/re-enable; `Reconcile_RealStore_PushesPrivateAfterStatusToggle` extends through reconcile push. |
| `synthesizer_guardrail_realstore_test.go` | GC-3: `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Live-bug regression suite (4836d5a19):** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. Must pass unchanged after GC-2. |
| `policyselect_account_realstore_test.go` | GC-2: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |
| `reconcile_test.go` | `FirstSynth_EmitsCreate`; `NoChange_EmitsNothingExtra` (re-push as Modified — verify desired); `PolicyRemoved_EmitsDelete`; `NilProxyController_NoOp`; `EmptyAccountID_NoOp`; `ClusterFromMapping`. |
| `wire_shape_test.go` | `TestSynthesizedService_WireShape` — proto-shape lockdown via `ToProtoMapping`. Catches "service not matching" (mapping reaches proxy but no SNI/HTTP route). Asserts ID, Domain, Mode, AuthToken, `Private`, `Auth.Oidc=false`, one path `/` + `https://noop.invalid/`, 8 middlewares with correct slot enums, router config `auth_header_value="Bearer sk-test-key"`. |
| `labelgen/labelgen_test.go` | `PickUnique_{DeterministicWithSeededRng,AvoidsTakenWordsWhenMostAreReserved,FallsBackWhenAllReserved}`; `UniqueWords_DropsDuplicates`. |
| `types/consumption_test.go` | `WindowStart_{AlignedToUnixEpoch,WithinWindowConverges,AcrossWindowsDiverges,DifferentWindowsHaveDifferentBuckets,SubMinuteAndMinuteAlignment,ZeroWindowReturnsInputUTC}`. Bucket alignment so multi-node reads converge. |
| `agentnetwork_realstack_test.go` | `ProviderCRUD_FansOutToProxyAndClientPeers` — no-mock end-to-end through real account manager + network-map + agentnetwork: provider create propagates the updated map to both proxy peer and client peer with the synth DNS surface. |
| `agentnetwork_budgetrule_realstack_test.go` | GC-1: `BudgetRuleCRUD_RealManager`; `UpdateSettings_PreservesImmutableAndTogglesCollection`. |
## Known limitations / explicit non-goals
- **`MergedGuardrails.TokenLimits/Budget/Retention` emit at zero** (`synthesizer.go:940-948`); real enforcement is `Policy.Limits` via `llm_limit_check`. Future cleanup implied.
- **Session keys picked from first enabled provider by created_at** (`pickServiceSessionKeys`, `synthesizer.go:270`). Existing session cookies survive provider edits only while the first-by-CreatedAt provider stays in place. Document for operators.
- **Reconcile failures silently swallowed** (`reconcile.go:42-44`). Persistent failures keep the proxy out of sync until the next reconcile.
- **`scoreCandidates` exposes only the LAST exhaustion's deny code** when multiple policies are exhausted.
- **`bootstrapSettingsIfNeeded` failure is non-fatal to provider create** (`manager.go:200`): provider lands, synth is no-op until the next provider create retries the bootstrap.
- **Budget rules do not trigger a reconcile** (`manager.go:476-477`). Request-time evaluation only; new rules take effect on the next request without a proxy push.
## Cross-references
- **Upstream:** [shared/api](10-shared-api.md), [management/store](20-management-store.md), reverseproxy `service`/`proxy`/`sessionkey` packages, `management/server/permissions` + `activity`.
- **Downstream:** [management/handlers (HTTP wiring)](22-management-handlers-wiring.md), [proxy/middleware-builtin](31-proxy-middleware-builtin.md), network-map controller (`injectAllProxyPolicies` fan-out).
- **End-to-end flow:** [../01-end-to-end-flows.md](../01-end-to-end-flows.md) — "Provider create → reconcile → proxy push → peer map refresh" and "request → policy select → record" diagrams.
- **Top-level:** [../00-overview.md](../00-overview.md)

217
docs/agent-networks/modules/22-management-handlers-wiring.md Normal file
View File

@@ -0,0 +1,217 @@
# management/handlers + wiring — HTTP API + gRPC delivery
> **Reviewer profile:** Management infra maintainer; comfortable with gorilla/mux REST handlers, the existing permissions/operations RBAC, the network_map controller fan-out, gRPC server-side streaming, and the reverse-proxy snapshot/live-update path.
> **Time to review:** 60 minutes.
> **Risk level:** Medium — surface is mostly additive, but two changes are load-bearing: `injectAllProxyPolicies` runs on every per-peer compute, and `shallowCloneMapping` now must round-trip `Private` (a missed field silently breaks every MODIFIED).
> **Backward-compat impact:** Additive on the wire (new routes, new RPCs, new proto fields, new gorm column on `AccessLogEntry`). One management-internal break: `nbhttp.NewAPIHandler` gains a trailing `agentNetworkManager` parameter; `nil` is tolerated and silently skips route registration.
## Module boundary
This module is the seam between the public Agent Network HTTP API and the proxy fleet that serves agent traffic. North side: a `/api/agent-network/*` surface (providers, policies, guardrails, budget rules, settings, consumption) on the existing gorilla router, delegating to `agentnetwork.Manager`. Handlers are thin — they translate `api.*` ↔ `types.*`, validate shape, forward. RBAC and event emission stay inside the manager (`manager.go:680-682`).
South side: `ProxyServiceServer` (`proxy.go`) learns to (a) ship synth services to a proxy on initial snapshot, (b) resolve agent-network domains in `getServiceByDomain` for OIDC/session/tunnel-peer flows, (c) gate LLM requests via `CheckLLMPolicyLimits` + `RecordLLMUsage`, (d) preserve `Private` through `shallowCloneMapping` so per-proxy live updates don't silently flip services public. The network_map controller prepends synth services to `account.Services` on every per-peer compute; `accesslogentry.go` gains an indexed `AgentNetwork` column so the dashboard can filter cheaply.
## Commits in scope
| SHA | Subject | LOC delta |
| --- | ------- | --------- |
| 09e8059b6 | AN-2b: wire synth into network map | controller.go +20 / repository.go +10 |
| 9ecb6449d | AN-3: HTTP API handlers + routes | handlers/*, handler.go, module.go, codes.go |
| 9a154714 | AN-6: access-log `agent_network` flag e2e | accesslogentry.go +5 |
| 8cb9c187d | AN-7: enforcement + synth delivery | proxy.go +185, boot.go +24, service.go +109 |
| 263dabd73 | preserve `Private` in clone | proxy.go +1, proxy_clone_test.go +59 |
| 468875cb4 | wire `EnableLogCollection` suppression | service.go (DisableAccessLog plumbing) |
| 23bdf6871 | GC-4: budget-rule + settings HTTP API | budget_handler.go +172, budget_handler_test.go +127 |
## Files changed
| Path | Status | LOC | Role |
| ---- | ------ | --- | ---- |
| `handlers/agentnetwork/providers_handler.go` | new | 216 | Catalog + provider CRUD + central `AddEndpoints` |
| `handlers/agentnetwork/policies_handler.go` | new | 228 | Policy CRUD + shared `validatePolicy*` |
| `handlers/agentnetwork/guardrails_handler.go` | new | 171 | Guardrail CRUD |
| `handlers/agentnetwork/budget_handler.go` | new | 172 | Account-level budget rule CRUD |
| `handlers/agentnetwork/settings_handler.go` | new | 74 | GET (200+`null` if unbootstrapped) + PUT toggles |
| `handlers/agentnetwork/consumption_handler.go` | new | 53 | Read-only consumption rows |
| `handlers/agentnetwork/handlers_test.go` | new | 239 | Real-store fixture; wire round-trip + validation |
| `handlers/agentnetwork/budget_handler_test.go` | new | 127 | Budget-rule + settings toggles |
| `server/http/handler.go` | edit | +7 | New `agentNetworkManager` arg; conditional `AddEndpoints` |
| `server/permissions/modules/module.go` | edit | +2 | New `AgentNetwork` module key |
| `internals/server/boot.go` | edit | +24 | Wires synthesiser adapter + limits service into proxy server |
| `internals/server/modules.go` | edit | +12 | `AgentNetworkManager()` lazy-create node |
| `internals/controllers/network_map/controller/controller.go` | edit | +20/-4 | `injectAllProxyPolicies` replaces 4 `InjectProxyPolicies` calls |
| `internals/controllers/network_map/controller/repository.go` | edit | +10 | `SynthesizeAgentNetworkServices` repo method |
| `internals/modules/reverseproxy/service/service.go` | edit | +109 | `MiddlewareConfig`, capture limits, `AgentNetwork`, `DisableAccessLog` + proto |
| `internals/modules/reverseproxy/accesslogs/accesslogentry.go` | edit | +5 | Indexed `AgentNetwork bool` from proto |
| `internals/shared/grpc/proxy.go` | edit | +185 | Synth wiring, 2 RPCs, domain fallback, `Private` in clone |
| `internals/shared/grpc/proxy_clone_test.go` | new | 59 | Locks every `ProxyMapping` field minus `AuthToken` |
| `server/activity/codes.go` | edit | +49 | 13 new activity codes (125-137) |
## HTTP routes added
All routes inherit the platform's auth middleware. Perms enforced inside `agentnetwork.Manager.requirePermission` (`manager.go:680-682`) on `modules.AgentNetwork`. Permission column shows the `op` passed to `requirePermission` — read = `Read`, etc.
| Method | Path | Perm | Handler |
| ------ | ---- | ---- | ------- |
| GET | `/agent-network/catalog/providers` | authn only | `providers_handler.go:43` |
| GET | `/agent-network/providers` | read | `providers_handler.go:57` |
| POST | `/agent-network/providers` | create | `providers_handler.go:97` |
| GET | `/agent-network/providers/{providerId}` | read | `providers_handler.go:77` |
| PUT | `/agent-network/providers/{providerId}` | update | `providers_handler.go:132` |
| DELETE | `/agent-network/providers/{providerId}` | delete | `providers_handler.go:172` |
| GET | `/agent-network/policies` | read | `policies_handler.go:32` |
| POST | `/agent-network/policies` | create | `policies_handler.go:72` |
| GET | `/agent-network/policies/{policyId}` | read | `policies_handler.go:52` |
| PUT | `/agent-network/policies/{policyId}` | update | `policies_handler.go:102` |
| DELETE | `/agent-network/policies/{policyId}` | delete | `policies_handler.go:142` |
| GET | `/agent-network/guardrails` | read | `guardrails_handler.go:25` |
| POST | `/agent-network/guardrails` | create | `guardrails_handler.go:65` |
| GET | `/agent-network/guardrails/{guardrailId}` | read | `guardrails_handler.go:45` |
| PUT | `/agent-network/guardrails/{guardrailId}` | update | `guardrails_handler.go:95` |
| DELETE | `/agent-network/guardrails/{guardrailId}` | delete | `guardrails_handler.go:135` |
| GET | `/agent-network/budget-rules` | read | `budget_handler.go:24` |
| POST | `/agent-network/budget-rules` | create | `budget_handler.go:64` |
| GET | `/agent-network/budget-rules/{ruleId}` | read | `budget_handler.go:44` |
| PUT | `/agent-network/budget-rules/{ruleId}` | update | `budget_handler.go:95` |
| DELETE | `/agent-network/budget-rules/{ruleId}` | delete | `budget_handler.go:135` |
| GET | `/agent-network/settings` | read | `settings_handler.go:53` (200+`null` if no row) |
| PUT | `/agent-network/settings` | update | `settings_handler.go:27` |
| GET | `/agent-network/consumption` | read | `consumption_handler.go:21` |
## gRPC RPCs added (or modified)
| RPC | Direction | Trigger |
| --- | --------- | ------- |
| `CheckLLMPolicyLimits` | proxy→mgmt unary | Pre-flight gate; returns allow/deny, selected policy, attribution group, window, deny code+reason (`proxy.go:259-301`). `Unimplemented` when limits service is nil. |
| `RecordLLMUsage` | proxy→mgmt unary | Post-flight write of tokens+cost against policy-window dimensions + every applicable account budget rule (`proxy.go:303-349`). `window_seconds==0` ⇒ no policy cap, only account fan-out runs. |
| `GetMappingUpdate`/`SendServiceUpdate` (stream) | mgmt→proxy | Snapshot (`proxy.go:752-780`) now appends `SynthesizeServicesForCluster`. Live updates use `SendServiceUpdateToCluster` + `shallowCloneMapping`. |
## Architecture & flow
### HTTP request lifecycle
```mermaid
sequenceDiagram
participant DB as Dashboard
participant R as gorilla.Router (/api)
participant H as handler (agentnetwork)
participant M as agentnetwork.Manager
participant S as store.Store
participant AM as accountManager (StoreEvent)
DB->>R: POST /api/agent-network/providers
R->>H: createProvider (auth mw sets UserAuth)
H->>H: GetUserAuthFromContext + validate(req)
H->>M: CreateProvider(userID, provider, bootstrapCluster)
M->>M: requirePermission(AgentNetwork, Create)
M->>S: SaveAgentNetworkProvider
M->>AM: StoreEvent(AgentNetworkProviderCreated)
M-->>H: created provider
H-->>DB: 200 + api.AgentNetworkProvider JSON
```
### Synth-service delivery via gRPC
```mermaid
sequenceDiagram
participant P as Proxy
participant G as ProxyServiceServer
participant SM as service.Manager (persisted)
participant SA as synthesizerAdapter
participant AN as SynthesizeServicesForCluster
participant ST as store.Store
Note over P,G: Initial snapshot
P->>G: GetMappingUpdate (stream open)
G->>SM: GetServicesForCluster(conn.address)
SM-->>G: persisted []*Service
G->>SA: SynthesizeServicesForCluster(conn.address)
SA->>AN: SynthesizeServicesForCluster(store, clusterAddr)
AN->>ST: walk every account; read providers/policies/settings
AN-->>SA: in-memory []*Service
SA-->>G: []*Service
G->>P: response (persisted + synth)
Note over G,P: Per-request live update
G->>G: SendServiceUpdateToCluster(update, clusterAddr)
G->>G: shallowCloneMapping(update) %% Private MUST survive
G->>P: response with single mapping
```
End-to-end: HTTP write persists rows and emits an activity event; the manager then triggers `proxyController.SendServiceUpdate` so proxies re-render. **The snapshot path is the only one that calls into the synthesiser** — on stream open it pulls persisted services then appends synth services for the cluster. Synth services are never persisted. For OIDC/session/tunnel-peer flows, `getServiceByDomain` falls back to `SynthesizeServicesForCluster(clusterFromDomain(domain))` when persisted lookup misses (`proxy.go:1763-1793`). The network_map contribution is orthogonal: per-peer compute prepends the same synth services to `account.Services` before `InjectProxyPolicies`.
## Permissions model added
- `permissions/modules/module.go:22` adds `AgentNetwork Module = "agent_network"`, registered in `All` (`module.go:42`). Standard `operations.{Read,Create,Update,Delete}` matrix.
- Handlers don't call `permissionsManager` directly — they extract `UserAuth` and delegate to `agentnetwork.Manager`, which gates every mutation through `requirePermission` (`manager.go:168, 308, 549`, etc.). Confirm your role-set provider has `agent_network` rows for owner/admin/user/billing-admin before merging.
- `getCatalogProviders` (`providers_handler.go:43`) intentionally skips RBAC — catalog is global static data.
## Activity codes added
`activity/codes.go:244-274` adds Activities 125-137 + string/code mappings (`codes.go:428-444`), following `<domain>.<resource>.<action>` (e.g., `agent_network.provider.create`). Audit-log exporters / SIEM forwarders need to know the new codes.
## Invariants
- **Synth services are never persisted.** Snapshot appends after `serviceManager.GetServicesForCluster` (`proxy.go:761-770`); network_map prepends before `InjectProxyPolicies` (`controller.go:117-126`).
- **`shallowCloneMapping` must round-trip every `ProxyMapping` field except `AuthToken`** — `proxy_clone_test.go:50-58` enforces via `gproto.Equal`. The bug it guards (commit 263dabd73): missing `Private` made every MODIFIED arrive `private=false`, proxy skipped `ValidateTunnelPeer`, `UserGroups` stayed empty, llm_router denied `no_authorised_provider`; restart "fixed" because snapshot uses the original mapping.
- **Limit-window floor is 60s** (`policies_handler.go:189-220`); enabled cap with both per-group and per-user at zero is rejected. Budget rules reuse the same validator (`budget_handler.go:170`).
- **Manager is optional at boot.** `NewAPIHandler` registers routes only when non-nil (`handler.go:129`); `ProxyServiceServer` returns `Unimplemented` from both RPCs when limits service is unwired (`proxy.go:262-265, 306-309`).
- **Settings GET on an unbootstrapped account returns 200 + `null`** (`settings_handler.go:65-72`) — not 404.
## Things to scrutinize
### Correctness
- **`injectAllProxyPolicies` runs on every per-peer compute**: `controller.go:163, 309, 415, 681`. `sendUpdateAccountPeers` is the target of the buffered fan-out — synth runs once per debounced account-update tick **and** once per direct `UpdateAccountPeer`. Cost is O(providers + policies × users-per-group) per account under `LockingStrengthNone`. No per-account synth cache — verify it fits the buffer interval for your largest tenant.
- **`clusterFromDomain` strips at the first `.`** (`proxy.go:1784-1792`). A zero-dot domain returns `""` and the synth call walks every account. Confirm no path reaches this with a malformed/internal domain.
- **Account-budget `RecordConsumption` fans out even when `window_seconds == 0`** (`proxy.go:341-348`) — intentional. Verify the proxy never sends `RecordLLMUsage` for a request that wasn't actually allowed.
### Security
- Every handler extracts `UserAuth` via `nbcontext.GetUserAuthFromContext` before any work. Routes live behind the standard `/api` mux; bypass list is not extended.
- `CheckLLMPolicyLimits` / `RecordLLMUsage` ride the existing **proxy → mgmt** gRPC connection auth. No additional token check inside the RPCs — they trust the connection. Confirm the proxy-side token-verification interceptor in this package gates both.
- `RecordLLMUsage` only validates `account_id != ""` (`proxy.go:317-319`). A compromised proxy can attribute cost to any account in its cluster — was already true for prior RPCs but is louder now that data drives denials.
### Concurrency
- `SetAgentNetworkSynthesizer` / `SetAgentNetworkLimitsService` write under `s.mu.Lock`; read paths copy the interface under read lock (`proxy.go:236-247, 260-263, 304-307`). Same pattern as existing `serviceManager`/`proxyController` setters.
- Manager writes use `LockingStrengthUpdate`; synth reads use `LockingStrengthNone` — read-after-write via the proxy snapshot can observe a stale view by up to one fan-out tick.
- Network_map controller is single-threaded per account; cross-account is parallel.
### Backward compatibility
- `proxy_clone_test.go` is the regression net; any new `ProxyMapping` field must be cloned or explicitly nulled in the test.
- `AccessLogEntry` adds indexed `AgentNetwork bool` — implicit AutoMigrate; deploy story must handle table-rewrite cost on high-volume access-log tables.
- `TargetOptions` gains seven `omitempty` JSON fields (`service.go:69-94`); on-wire shape stays compatible. `targetOptionsToProto` tests all fields when deciding nil (`service.go:551-556`).
- `NewAPIHandler` signature changes — every caller must pass `agentNetworkManager`; `nil` is supported.
### Observability
- 13 new activity codes via `accountManager.StoreEvent` in the manager — confirm dashboard's audit-log UI maps them.
- `AccessLogEntry.AgentNetwork` is indexed for the dashboard's agent-network log filter.
- New RPCs log at error level on store/selector failures (`proxy.go:284, 327, 332, 348`). Snapshot synth failures degrade to warnings — stream is not aborted (`proxy.go:765`).
## Test coverage
| Test | Locks down |
| ---- | ---------- |
| `handlers_test.go::TestPolicyHandler_WindowSecondsRoundTrip` | GET carries `window_seconds`; legacy `window_hours`/`window_days` absent. |
| `handlers_test.go::TestPolicyHandler_RejectsSubMinuteWindow` | POST `<60s` returns 4xx. |
| `handlers_test.go::TestConsumptionHandler_EmptyAccountReturnsArray` | `/consumption` returns `[]` — never null. |
| `handlers_test.go::TestConsumptionHandler_PopulatedAccountListsRows` | RecordConsumption×2 surfaces both with correct tokens/cost/window. |
| `budget_handler_test.go::TestBudgetRuleHandler_RoundTrip` | Targets + PolicyLimits shape round-trip. |
| `budget_handler_test.go::TestBudgetRuleHandler_ListReturnsArray` | Empty-list shape. |
| `budget_handler_test.go::TestBudgetRuleHandler_{RejectsMissingName,RejectsSubMinuteWindow}` | Validation rejections are 4xx. |
| `budget_handler_test.go::TestSettingsHandler_GetExposesCollectionToggles` | All four toggles + computed `Endpoint`. |
| `proxy_clone_test.go::TestShallowCloneMapping_PreservesAllFieldsExceptAuthToken` | Future-proofs clone; every field round-trips, `AuthToken` dropped. |
Handler tests use a real sqlite store + real manager + always-allow permissions mock (`handlers_test.go:53-75`). Create/update/delete success paths flow through `accountManager.StoreEvent` which the fixture doesn't wire — covered by manager-level no-mock tests outside this module.
## Known limitations / explicit non-goals
- No pagination on any list endpoint; no bulk endpoints.
- Synth result is not cached — every snapshot and every per-peer compute repeats the store walk.
- `getSettings` returning `200 + null` is a deliberate dashboard concession.
- No rate-limiting beyond the global `/api` rate limiter.
## Cross-references
- Upstream: [shared/api](10-shared-api.md), [management/agentnetwork](21-management-agentnetwork.md), [management/store](20-management-store.md)
- Downstream: [proxy/runtime](33-proxy-runtime.md)
- End-to-end flow: [../01-end-to-end-flows.md](../01-end-to-end-flows.md)
- Top-level: [../00-overview.md](../00-overview.md)

225
docs/agent-networks/modules/30-proxy-middleware-framework.md Normal file
View File

@@ -0,0 +1,225 @@
# proxy/middleware-framework — generic plugin system
> **Reviewer profile:** Proxy maintainer. Comfort with `net/http` handler composition, request/response interception, Go context propagation, `atomic.Pointer` snapshot patterns, and `sync` primitives. No LLM/agent-network domain knowledge required — every example built into this commit is generic.
> **Time to review:** 60–75 minutes.
> **Risk level:** **High** — every proxied request transits this chain. Budget exhaustion, panic recovery, or chain-close bugs hit the hot path for all targets, not just agent-network ones.
> **Backward-compat impact:** Additive at the proxy. The `middleware` and `bodytap` packages are new (`proxy/internal/middleware/middleware.go:1`, `proxy/internal/middleware/bodytap/request.go:13`); existing proxy targets keep working until a chain is bound to them via `Manager.Rebuild`.
## Module boundary
This module is the **framework only**: slots, chains, registry, dispatcher, accumulator, body-tap, output filters. No middleware *implementation* lives here — those land in `proxy/internal/middleware/builtin/*` (covered in module 31). The package contract is:
1. The proxy hands a `Manager` to its config-apply path. The synth pushes per-path `PathTargetBinding` lists (`proxy/internal/middleware/manager.go:26`) into `Manager.Rebuild`, which resolves each spec via the `Registry`/`Resolver` (`proxy/internal/middleware/registry.go:81-121`) and produces an immutable `Chain` keyed by `serviceID|pathID` (`proxy/internal/middleware/manager.go:410-412`).
2. The reverse-proxy handler captures the request body via `bodytap.CaptureRequest`, calls `Chain.RunRequest`, applies returned mutations (already filtered by `chain.applyMutations`), forwards to the upstream behind a `bodytap.CapturingResponseWriter`, then calls `Chain.RunResponse` and `Chain.RunTerminal`.
3. Middlewares are inert plugins that receive a deep-cloned `Input` and return an `Output` whose decision/mutations are clamped by the dispatcher's `filterOutput` (`proxy/internal/middleware/dispatcher.go:149-172`).
Everything that crosses the framework boundary in either direction is value-typed and deep-copied — middlewares cannot mutate the live request directly, and the framework cannot inadvertently leak middleware-owned slices into the request hot path.
## Commits in scope
| SHA | Subject | LOC delta |
| --- | ------- | --------- |
| `00bf0d328` | [agent-network] AN-4: proxy middleware framework | +2 740 (all 17 files in scope, this is the only commit that touches them) |
`git -C /Users/maycon/projects/netbird log --oneline 14af17955..HEAD -- proxy/internal/middleware/{chain,registry,manager,dispatcher,decision,headerpolicy,bodypolicy,keys,metadata,metrics,middleware,redaction,spec,types}.go proxy/internal/middleware/bodytap` returns exactly one commit. Subsequent AN-5/AN-7/GC-2 commits only add files under `builtin/` or touch wire paths — none modify the framework files in this module's scope. Review can be a single sit-down.
## Files changed
| Path | Status | LOC | Role |
| ---- | ------ | --- | ---- |
| `proxy/internal/middleware/middleware.go` | A | 47 | `Middleware` + `Factory` interfaces. |
| `proxy/internal/middleware/types.go` | A | 242 | `Slot`, `FailMode`, `Decision`, all limit constants, `Input`/`Output`/`Mutations`/`UpstreamRewrite`/`AuthHeader` value types. |
| `proxy/internal/middleware/spec.go` | A | 44 | Apply-time `Spec` (validated wire shape + runtime-injected fields) and `Clone`. |
| `proxy/internal/middleware/registry.go` | A | 121 | `Registry` (factory map, RWMutex) and `Resolver` (Spec → bound `Middleware`). |
| `proxy/internal/middleware/manager.go` | A | 412 | `Manager`, `chainTable` reverse index, `Rebuild`/`Invalidate*`, async chain close. |
| `proxy/internal/middleware/chain.go` | A | 317 | `Chain.RunRequest`/`RunResponse`/`RunTerminal`, mutation gating, `cloneInputFor`. |
| `proxy/internal/middleware/chain_test.go` | A | 330 | Metadata threading, LIFO response order, rewrite gating, UserGroups propagation, terminal accumulation. |
| `proxy/internal/middleware/dispatcher.go` | A | 185 | Timeout/panic recovery, fail-mode, error classification, `filterOutput`. |
| `proxy/internal/middleware/decision.go` | A | 80 | `RenderDenyResponse`, deny-code regex, status clamp. |
| `proxy/internal/middleware/headerpolicy.go` | A | 69 | Compile-in header denylist + `FilterHeaderMutations`. |
| `proxy/internal/middleware/bodypolicy.go` | A | 62 | `ValidateBodyReplace` / `ApplyBodyReplace` smuggling guards. |
| `proxy/internal/middleware/keys.go` | A | 82 | Metadata key namespace constants. |
| `proxy/internal/middleware/metadata.go` | A | 99 | `Accumulator` — allowlist, per-mw/per-request byte caps, redaction. |
| `proxy/internal/middleware/metrics.go` | A | 171 | OTel instrument bundle (`proxy.middleware.*`). |
| `proxy/internal/middleware/redaction.go` | A | 79 | `Scan` — PEM/JWT/AWS/bearer/Luhn-validated CC patterns. |
| `proxy/internal/middleware/bodytap/request.go` | A | 229 | Capture + replay reader, `Budget` semaphore, bypass reason codes. |
| `proxy/internal/middleware/bodytap/response.go` | A | 171 | `CapturingResponseWriter` (tee with `PassthroughWriter` for Flusher/Hijacker preservation). |
The 17 paths the scope listed all land in this single commit; no churn since.
## Slot model
Three slots, declared per-middleware exactly once (`proxy/internal/middleware/types.go:27-41`):
- **`SlotOnRequest`** (`Slot=1`) — runs **before** the upstream call, in registration order. May `DecisionDeny`, may emit `Mutations` (header add/remove, body replace, `UpstreamRewrite`) when both `Spec.CanMutate` and `Middleware.MutationsSupported()` are true. May emit metadata. Each middleware in the slot sees metadata that earlier ones in the same slot just emitted (`proxy/internal/middleware/chain.go:144-178`) — this is how the framework gives middlewares an intra-slot side channel without a global bag.
- **`SlotOnResponse`** (`Slot=2`) — runs **after** the upstream returns, in **reverse** registration order. Cannot deny (clamped in `dispatcher.filterOutput`, `proxy/internal/middleware/dispatcher.go:153-157`). May still mutate response headers in principle, but the current chain only forwards `RewriteUpstream` from on_request, so on_response mutations are observe-only in practice. Threads the same per-slot metadata view as on_request.
- **`SlotTerminal`** (`Slot=3`) — runs **after** every on_response middleware has emitted, in registration order. Sees the full accumulated bag plus prior terminal emissions (`chain.go:221-245`). Cannot deny, cannot mutate (`dispatcher.go:168-170`). Designed for sinks (access log, metrics push, audit emitter).
Splitting a feature across slots (e.g. "parse on the way out, ship on terminal") is the explicit architectural choice — `types.go:7-15` and `types.go:22-25` make it clear no middleware participates in more than one slot.
## Architecture & flow
### Chain dispatch
```mermaid
sequenceDiagram
autonumber
participant H as proxy HTTP handler
participant BT as bodytap.CaptureRequest
participant CH as Chain
participant DI as Dispatcher
participant MW as Middleware (per slot)
participant US as Upstream
participant CW as CapturingResponseWriter
H->>BT: CaptureRequest(r, cfg, budget)
BT-->>H: body[], truncated, release()
H->>CH: RunRequest(ctx, r, Input, Accumulator)
loop on_request, registration order
CH->>CH: cloneInputFor(in, OnRequest)
CH->>DI: Invoke(ctx, spec, mw, call)
DI->>MW: mw.Invoke(callCtx, in)
MW-->>DI: Output{decision, metadata, mutations?}
DI->>DI: filterOutput (clamp deny, gate mutations)
DI-->>CH: filtered Output
CH->>CH: Accumulator.Emit (allowlist + caps + redact)
alt DecisionDeny
CH-->>H: denied, merged, rewrite
else allow
CH->>CH: applyMutations(r, m) and capture rewrite
end
end
CH-->>H: nil, merged, rewrite
H->>US: ProxyRequest (with rewrite/mutations applied)
US-->>CW: bytes (streamed, tee'd into cap-bounded buf)
CW-->>H: passthrough complete
H->>CH: RunResponse(ctx, Input{RespBody:CW.Body(),...}, acc)
loop on_response, REVERSE order (LIFO)
CH->>DI: Invoke (same wrappers)
end
H->>CH: RunTerminal(ctx, Input{Metadata:full bag}, acc)
H->>BT: release() + CW.Release()
```
### Body-tap mechanics (request + response)
```mermaid
flowchart LR
subgraph req[Request capture — bodytap.CaptureRequest]
R0[r.Body] --> R1{cfg.MaxRequestBytes > 0?\nUpgrade absent?\nContent-Type allowed?\nCL <= cap?}
R1 -- no --> R2[bypass = reason\nbody = nil\nr.Body untouched]
R1 -- yes --> R3[Budget.Acquire(cap)]
R3 -- denied --> R4[bypass=BypassBudget]
R3 -- ok --> R5[io.LimitReader(r.Body, cap+1)\nio.ReadAll]
R5 --> R6{len > cap?}
R6 -- truncated --> R7[viewable = buf[:cap]\nr.Body = replayReadCloser{buf, tail}]
R6 -- whole --> R8[r.Body = NopCloser(bytes.Reader(buf))\nclose original]
R7 --> R9[(release captured\nbudget on req end)]
R8 --> R9
end
subgraph resp[Response capture — CapturingResponseWriter]
W0[client] -.-> CW[Write(p)]
CW --> P1[PassthroughWriter.Write(p)\n— bytes leave to client first]
P1 --> P2{!stopped?}
P2 -- yes --> P3{remaining = cap - buf.Len()}
P3 --> P4[buf.Write(p[:take])\nset truncated if take<n]
P2 -- no --> P5[silent drop into the tee\n(client write already done)]
end
```
The body-tap is the highest-leak-risk surface in this module; three details matter:
1. **Request capture is "read-and-replay", not "read-and-forward".** `CaptureRequest` always swaps `r.Body` for either a `bytes.Reader` (whole body fit) or a `replayReadCloser` that replays the captured prefix then drains the remaining stream from the original body (`bodytap/request.go:178-201`). This means the **upstream still sees the full body even when the tap truncates**. The original `r.Body` is **not** closed in the truncated branch — `replayReadCloser.Close()` only closes the tail (`bodytap/request.go:199-201`), which is the same reader, so close once on request end is correct, but reviewers should confirm the upstream proxy always reads to EOF (otherwise the tail is leaked).
2. **Response capture is a write-through tee.** `CapturingResponseWriter.Write` forwards to the underlying writer **first** (`bodytap/response.go:116-117`), then tees into `buf` under its own mutex. Client never blocks on the tee. `Flusher`/`Hijacker` are preserved via the embedded `responsewriter.PassthroughWriter`. SSE/chunked streams flow through untouched; middlewares only see the bounded prefix.
3. **Budget is a single shared semaphore.** `Manager` constructs one `bodytap.Budget` at startup (`manager.go:138-144`, default `256 MiB` from `bodytap/request.go:39`). Every capture pre-acquires its full `MaxRequestBytes` / `MaxResponseBytes` from the budget regardless of actual body size; that prevents a flood of small captures from collectively exceeding the cap, but it also means a misconfigured `MaxRequestBytes = 1 MiB` with 256 concurrent requests already exhausts the default budget. Reviewers should sanity-check the operator-facing defaults that ship with synth-service.
The framework explicitly aborts capture (and increments `proxy.middleware.capture_bypass_total`) before reading the first byte when `Upgrade`/`Connection: upgrade` is set (`bodytap/request.go:120-125`), when the content-type isn't in the allowlist (`bodytap/request.go:126-128`), or when the advertised `Content-Length` already exceeds the cap (`bodytap/request.go:131-133`). This is the right place to make sure WebSocket upgrades and large file uploads never reach the buffer.
## Public contracts
- **`Middleware` interface** (`middleware.go:14-36`): `ID()`, `Version()`, `Slot()`, `AcceptedContentTypes()`, `MetadataKeys()`, `MutationsSupported()`, `Invoke(ctx, *Input) (*Output, error)`, `Close()`. `MetadataKeys()` is the **closed set** the middleware is allowed to emit — the accumulator drops anything outside it (`metadata.go:71-75`). `Close` must be idempotent (called even when `Invoke` was never reached).
- **`Factory` interface** (`middleware.go:44-47`): `ID()`, `New(rawConfig []byte) (Middleware, error)`. `RawConfig` is opaque JSON bytes on the wire (`spec.go:6-12`); each factory owns its own typed config.
- **`Decision` type** (`types.go:59-69`): `Allow=0`, `Deny=1`, `Passthrough=2`. Default-zero is permissive — important because every middleware that omits `Decision` gets `Allow`. Dispatcher clamps `Deny` to `Passthrough` outside `SlotOnRequest` (`dispatcher.go:153-157`).
- **`Mutations`** (`types.go:196-201`): `HeadersAdd`/`HeadersRemove` (filtered through `headerpolicy.go`), `BodyReplace` (gated through `bodypolicy.go`), and `RewriteUpstream`. `RewriteUpstream` is **last-write-wins** within the on_request slot (`chain.go:170-172`, locked down by `TestChain_RunRequest_LatestRewriteWins`).
- **Metadata propagation keys** (`keys.go`): all keys live in a single file and follow `^[a-z][a-z0-9_-]*(\.[a-z0-9_-]*)+$` (`metadata.go:8`). Framework-injected error tagging uses `mw.<id>.error_kind` (`keys.go:81`) so operators can distinguish framework-emitted entries from middleware-emitted ones.
## Invariants
- **Per-request context isolation.** `cloneInputFor` deep-copies every mutable field (`Headers`, `RespHeaders`, `Metadata`, `Body`, `RespBody`, `UserGroups`, `UserGroupNames`) before each invocation (`chain.go:286-308`). A misbehaving middleware that mutates `in.Headers` only corrupts its own copy.
- **Body-tap bounded by capture limit.** Request side uses `io.LimitReader(r.Body, limit+1)` (`bodytap/request.go:152`) — the `+1` is how the code detects truncation (`bodytap/request.go:160`); the surfaced buffer is sliced back down to `limit`. Response side stops teeing once `buf.Len() >= cap` (`bodytap/response.go:121-133`). Neither side can grow the buffer past the configured cap.
- **Headers/body redaction order.** Accumulator runs `Scan(value)` **before** counting cost (`metadata.go:81-82`), so the byte budgets are computed against post-redaction sizes. `Scan` order is PEM → JWT → AWS key → bearer → Luhn-validated CC (`redaction.go:25-51`) — the comment block in `redaction.go:8-13` is explicit that this is best-effort, not DLP.
- **No middleware can starve the chain.** Every invocation runs inside `context.WithTimeout(ctx, clampTimeout(spec.Timeout))` in a separate goroutine (`dispatcher.go:51-94`), with the deadline race-`select`ed against the result channel. A blocked middleware fires the timeout path, gets fail-mode'd, and `IncError(kind=timeout)`. Timeouts are clamped to `[10ms, 5s]` (`types.go:80-86`, `dispatcher.go:174-185`).
- **Panic recovery.** `recover()` captures the panic, logs only the type + a 4 KiB stack prefix (no panic value — avoids leaking secrets the middleware was processing), and produces a `panicError` that flows through fail-mode (`dispatcher.go:64-76`).
- **Chain immutability + atomic swap.** `chainTable` is cloned on every `Rebuild`/`Invalidate*` and swapped via `atomic.Pointer` (`manager.go:44-69`, `manager.go:221-300`). Readers (`ChainFor`) are lock-free; writers serialise on `writeMu`. The retired chain is `Close`-d in a background goroutine bounded by `chainCloseTimeout = 2 * MaxTimeout` (`manager.go:21-22`, `manager.go:326-346`), so in-flight invocations finish on the old chain after the swap.
## Things to scrutinize
### Correctness
- **Chain ordering deterministic from synth output?** `Manager.buildChain` iterates `b.Specs` in slice order and appends to `bound` (`manager.go:366-391`); `NewChain` then partitions by slot but **preserves slice order within each slot** (`chain.go:50-60`). So order on the wire = order observed at runtime. Synth must therefore emit specs in the intended execution order — there is no per-spec `Priority` field. Worth flagging.
- **Decision short-circuit semantics.** `RunRequest` returns immediately on `DecisionDeny` (`chain.go:164-167`) **with the metadata accumulated so far** plus the `denied.Metadata`. Callers that ignore `merged` on deny will lose framework-injected `mw.<id>.error_kind` entries. The proxy runtime is the only caller; confirm it always feeds `merged` into the access log on the deny path as well.
- **`UpstreamRewrite` `AuthHeader` bypass** (`types.go:218-235`). The `AuthHeader`/`StripHeaders` fields *intentionally* bypass the header denylist on the basis that the proxy itself rewrites auth. The denylist still blocks middleware-emitted `HeadersAdd: Authorization=...`. This is a delicate carve-out — review the runtime consumer to confirm only the trusted upstream-build path unpacks `AuthHeader`, never the generic `applyMutations` loop.
- **`replayReadCloser.Close` only closes the tail** (`bodytap/request.go:199-201`). The replay buffer doesn't own a resource, so this is correct, but it conflates "replay finished" with "underlying body closed". If a caller `Close()`s without reading to EOF, the original body is closed but the captured prefix is lost; harmless for the proxy path (upstream always reads to EOF) but worth a doc-comment.
### Security
- **Body-tap memory bounds.** Discussed above — bounded by `MaxBodyCapBytes = 1 MiB` per direction (`types.go:77`) and the shared `Budget` (default 256 MiB). The concerning case is the **deep-copy in `cloneInputFor`** (`chain.go:300-306`): every middleware invocation gets its **own copy** of `Body` and `RespBody`. A chain of N middlewares with a 1 MiB body allocates N MiB of transient bytes per request. With `MaxMiddlewaresPerChain = 16` (`types.go:103`) that's up to 16 MiB extra per in-flight request. Worth pricing into the budget model.
- **Header redaction completeness.** `denyHeaders` (`headerpolicy.go:5-17`) covers the auth/forwarding family and framing (`Content-Length`, `Transfer-Encoding`, `Trailer`). `denyHeaderPrefixes` covers `X-Authenticated-*`, `X-Forwarded-*`, `X-Remote-*`, `X-NetBird-*`. Notably absent: `Range`, `If-Match`/`If-None-Match` (mutation could cause cache poisoning), `Origin`/`Referer`. Not necessarily wrong, but worth a deliberate decision.
- **Metadata key collisions across middlewares.** The accumulator has no cross-middleware uniqueness check; two middlewares with the same key in their allowlist can both emit it, and both copies land in `merged` (`metadata.go:51-99`). Downstream consumers must tolerate duplicates. Worth documenting.
- **Deny rendering.** `RenderDenyResponse` only allows codes matching `^[a-z][a-z0-9._-]{0,63}$` (`decision.go:9`), redacts/truncates message + detail values, caps `Details` at 8 entries (`decision.go:42-50`), clamps status to `[400,499]\{401}` (`decision.go:65-73`). The deny body type is fixed; middlewares cannot inject arbitrary JSON.
### Concurrency
- **Per-request state vs shared state in factories.** Each `Factory.New` is called once per chain build; the returned `Middleware` instance is **shared across all requests** for that chain. `Invoke` must be reentrant. The framework does not enforce this — a buggy middleware that holds per-call state on the struct will silently race. Suggest a `// Invoke must be safe for concurrent use` doc on the interface.
- **`chainTable` clone-on-write** is correct, but `addChain`/`removeChain` mutate the *cloned* table before the swap (`manager.go:71-108`), and they're called under `writeMu`. Readers only ever see the post-swap pointer. Good.
- **`Chain.inflight` WaitGroup**. `Run*` does `Add(1)`/`Done()` (`chain.go:142-143`, `chain.go:194-195`, `chain.go:225-226`); `Close` waits on it bounded by ctx (`chain.go:75-85`). One concern: a *new* `RunRequest` can `Add(1)` *after* `Close` started waiting if the caller still holds a stale chain pointer. `WaitGroup` does not panic on this if the count was already > 0 at `Wait` time, but it does panic if `Add` happens after `Wait` returns and another `Wait` runs. `Close` is documented one-shot, so single-`Wait` is fine, but callers must drop the chain reference before calling `Close`. Worth a code comment near `Close`.
- **Goroutine leaks.** `Dispatcher.Invoke` spawns one goroutine per call and *always* writes to a buffered (cap=1) channel (`dispatcher.go:62-76`), so even if the timeout fires the goroutine completes its send and exits. No leak.
- **`closeChainsAsync`** detaches retired chains into a goroutine (`manager.go:326-346`). If `Manager` is never GC'd this is fine, but there's no shutdown hook to wait on outstanding closes. Reviewers should confirm the proxy shutdown path explicitly drains in-flight requests before tearing down `Manager`, or accept that the last chain-close round may be cut short on exit.
### Performance
- **Allocations per request.** `cloneInputFor` allocates new slices for `Headers`, `RespHeaders`, `Metadata`, `Body`, `RespBody`, `UserGroups`, `UserGroupNames` — once per middleware per request. For a typical 5-middleware chain on a 1 KiB body that's ~10 small slice allocs plus one `Body` copy each. Not a hot-path crisis, but `sync.Pool` for the per-call `Input` would be a natural follow-up.
- **Accumulator allocates a fresh `allowSet` per `Emit` call** (`metadata.go:55-58`). One per middleware per slot pass = up to 48 per request. Cheap, but worth noting.
- **Regex cost.** `Scan` runs five regex passes on every accepted metadata value (`redaction.go:25-51`). Bounded by `MaxMetadataValueBytes = 4 KiB` so worst case is small.
### Observability
- **Per-middleware metrics.** `proxy.middleware.requests_total{middleware,target_id,outcome}` (`metrics.go:34-41`), `duration_ms`, `invocations_total`, `errors_total{kind}`, `metadata_rejected_total{reason}`, `header_mutation_blocked_total{header}`, `capture_bypass_total{reason}`. Comprehensive surface; operators can alert on `errors_total{kind=panic}` and `errors_total{kind=timeout}` separately. **Latency histogram is in milliseconds with default OTel buckets** — for a 10ms–5s timeout range default buckets cover OK, but a custom bucket set centred on 1–500ms would resolve the agent-network response-parser tail better.
- **Decision logs.** Panic logs (`dispatcher.go:69`) include `request_id`, type, and stack but not the panic value (safe). `Chain.Close` logs middleware-close errors at debug (`chain.go:91`). `applyMutations` logs body-replace rejections at warn (`chain.go:278`). No log on the deny path itself — by design, since the access-log terminal middleware is expected to record outcomes.
## Test coverage
| Test file | Locks down |
| --------- | ---------- |
| `proxy/internal/middleware/chain_test.go:77` | `RunRequest` threads metadata across on_request middlewares (regression for the "later mw can't see earlier mw's emissions" bug). |
| `chain_test.go:110` | `RunResponse` reverse-order threading. |
| `chain_test.go:142` | `cost_meter`-shaped scenario: response_parser registered after cost_meter still emits *before* cost_meter sees the bag (the exact `cost.skipped=missing_tokens` regression that prompted this commit). |
| `chain_test.go:178` | `UpstreamRewrite` last-write-wins. |
| `chain_test.go:206` | No middleware emits → nil rewrite. |
| `chain_test.go:224` | Rewrite filtered when `CanMutate=false`. |
| `chain_test.go:245` | `Input.UserGroups` propagates verbatim through `cloneInputFor`. |
| `chain_test.go:304` | Terminal middlewares see the full accumulated bag + prior terminal emissions. |
**Gaps** worth raising with the author:
- No direct test for `Dispatcher.Invoke` timeout / panic / fail-mode behaviour at the framework level (covered indirectly by built-in tests, but a unit test pinning `errors_total{kind=...}` labels would be cheap insurance).
- No test for `bodytap.CaptureRequest` truncated replay (the upstream-sees-full-body invariant is exactly the kind of thing a regression would silently break).
- No test for `Budget` exhaustion behaviour under concurrency.
- No test for `Manager.InvalidateMiddleware` + `LiveServiceCheck` race (the auth-revocation race the comment at `manager.go:33-38` calls out is the load-bearing reason for `LiveServiceCheck`).
## Known limitations / explicit non-goals
- **No middleware-to-middleware RPC.** Side-channel is metadata only.
- **No streaming body inspection.** Middlewares see a bounded prefix; SSE / chunked parsing happens against that prefix in the response middleware.
- **No per-spec priority.** Order is registration order in the spec slice.
- **No retry / circuit-breaker** on middleware errors. Fail-mode is binary (open/closed) and per-spec.
- **Mutations cannot rewrite the request URL path or query** — only `RewriteUpstream` can change scheme/host (+ optional path replacement, see `types.go:218-235`).
- **Redaction is best-effort.** Explicitly documented in `redaction.go:8-13`. Not a DLP solution.
## Cross-references
- Upstream wire shape: [../modules/10-shared-api.md](10-shared-api.md) (Spec/RawConfig encoding from management).
- Built-in middlewares using this framework: [../modules/31-proxy-middleware-builtin.md](31-proxy-middleware-builtin.md).
- Runtime wiring (where `Manager`, `Chain`, and `bodytap` are consumed by the HTTP handler): [../modules/33-proxy-runtime.md](33-proxy-runtime.md).
- End-to-end request flow including capture + chain dispatch: [../01-end-to-end-flows.md](../01-end-to-end-flows.md).
- Top-level architecture: [../00-overview.md](../00-overview.md).

378
docs/agent-networks/modules/31-proxy-middleware-builtin.md Normal file
View File

@@ -0,0 +1,378 @@
# proxy/middleware-builtin — the LLM chain
> Reviewer profile: proxy / LLM. You should be comfortable with Go HTTP middleware
> composition, OpenAI Responses API + Anthropic Messages API request/response shapes,
> SSE framing, and the NetBird management gRPC surface (`proto.ProxyServiceClient`).
> Spend most of your time on the **capture-pointer semantics** and the **limit_check ⇒
> limit_record** record-once invariant — those carry the highest blast radius.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing catalog this chain delegates to.
---
## Module boundary
This module is the registry-mounted middleware set the proxy executes on
every agent-network LLM request. Each sub-package registers itself via
`init()`
([builtin.go:32–34](../../../../netbird/proxy/internal/middleware/builtin/builtin.go));
the proxy server anonymous-imports the set
([all_test.go:11–19](../../../../netbird/proxy/internal/middleware/builtin/all_test.go))
so the registry is populated at boot. The chain is wired by the management
synthesiser and executed by the framework
(`proxy/internal/middleware/{chain,dispatcher,accumulator}.go` — both out
of scope). Everything here reads from / writes to one envelope: the
`middleware.KV` metadata bag plus `middleware.Mutations` for header/body
rewrites.
## The 8 middlewares
| Name | Slot | Inputs (metadata read) | Outputs (metadata written) | Side effects |
|---|---|---|---|---|
| `llm_request_parser` | OnRequest | `Input.{URL,Body,BodyTruncated}` | `llm.{provider,model,stream,request_prompt_raw,capture_truncated}` | none |
| `llm_router` | OnRequest | `llm.model`, `Input.{URL,UserGroups}` | `llm.{resolved_provider_id,authorising_groups}`, `llm_policy.{decision,reason}` | upstream rewrite + auth strip/inject |
| `llm_limit_check` | OnRequest | `llm.{resolved_provider_id,model}`, `Input.{AccountID,UserID,UserGroups}` | `llm.{selected_policy_id,attribution_group_id,attribution_window_seconds}`, `llm_policy.{decision,reason}` | gRPC `CheckLLMPolicyLimits` |
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:26–40](../../../../netbird/proxy/internal/middleware/builtin/all_test.go)
locks the ID set; adding or removing one is a conscious extension.
## Commits in scope
`e64ea4b02` AN-5 built-in middlewares + registry · `a67022ffa` stamp parser
id from catalog · `19d12adc8` openai match bare `/chat/completions` for
Cloudflare AI Gateway · `afefa38ce` cached + cache-creation tokens ·
`8cb9c187d` AN-7 enforcement + synth-service delivery · `b22d5a181` GC-2
account budget rules min-wins · `7072f8125` account toggle sole control for
prompt capture · `b1e66bca2` broader phone redactor · `b438a7194` budget
enforcement fix + PII redaction across channels · `19e03d688` redact-pii
wiring tests · `2a0d4991b` full-chain integration test · **`4836d5a19` gate
prompt + completion capture on EnablePromptCollection (capture-pointer fix)**.
## Files changed
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
| `llm_router/*` | 385 / 84 / 586 | Three-pass route selection (model → groups → path-prefix) |
| `llm_limit_check/*` | 196 / 38 / 182 | Pre-flight `CheckLLMPolicyLimits` (2s, fail-open) |
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
### llm_request_parser
Detects the LLM provider via `llm.DetectParser` (URL sniff) or by name via
`llm.ParserByName` when synthesiser stamps `provider_id`
([middleware.go:96–99](../../../../netbird/proxy/internal/middleware/builtin/llm_request_parser/middleware.go)).
**Path-routed providers short-circuit first:** `parseVertexPath` and
`parseBedrockPath` ([middleware.go:85–94](../../../../netbird/proxy/internal/middleware/builtin/llm_request_parser/middleware.go))
pull the model + vendor out of the URL before parser selection runs — Vertex
from `/v1/projects/.../publishers/{pub}/models/{model}:{action}` (publisher →
vendor via `vertexPublisherVendor`), Bedrock from `/model/{id}/{action}` with
`normalizeBedrockModel` stripping the region prefix + version suffix. See
[50-path-routed-providers.md](./50-path-routed-providers.md) for the full path
grammar. For body-routed providers it decodes the body into `RequestFacts`
(model + stream) and extracts the prompt. On
`capture_prompt=true` (or absent — see capture-pointer semantics below) the
prompt is run through `llm_guardrail.RedactPII` when `redact_pii=true` and
truncated rune-safely to 3500 bytes
([middleware.go:109–122](../../../../netbird/proxy/internal/middleware/builtin/llm_request_parser/middleware.go)).
**Key invariant:** redaction is parser-side, not guardrail-side — access-log
reads `llm.request_prompt_raw` directly.
### llm_router
Three-pass route selection in `matchRoute`
([middleware.go:241–300](../../../../netbird/proxy/internal/middleware/builtin/llm_router/middleware.go)):
filter by `Models` claim → vendor-pin (a vendor-tagged request never crosses to
another vendor's route) → filter by `AllowedGroupIDs` intersection → model
precedence over path → tie-break by longest `UpstreamPath` prefix match.
Model-miss returns `llm_policy.model_not_routable`; known-but-unauthorised
returns `llm_policy.no_authorised_provider`. **Key invariant:** auth-header
strip+inject rides on `UpstreamRewrite.{StripHeaders,AuthHeader}`
([middleware.go:606–646](../../../../netbird/proxy/internal/middleware/builtin/llm_router/middleware.go))
— NOT `HeadersAdd/HeadersRemove` — because the framework's mutation gate
blocks `Authorization` on the generic header path.
**Path-routed providers route before the model table.** `Invoke` checks
`isVertexPath` / `isBedrockPath`
([middleware.go:138–216](../../../../netbird/proxy/internal/middleware/builtin/llm_router/middleware.go))
ahead of the model lookup, so a path-carried model can't be claimed by a
same-vendor body-routed provider. `matchPathRoute` enforces the route's `Models`
allowlist (empty = catch-all) even though the model came from the URL.
Two path-only behaviours:
- **Vertex unmeterable publisher** — when `llm_request_parser` emits no
`llm.provider` (e.g. Gemini/`google`), the router denies with
`llm_policy.unmeterable_publisher` (403) rather than forward it uncounted.
- **GCP token minting** — when the route carries `GCPServiceAccountKeyB64`
(set from a `keyfile::` api_key), `gcpBearer` mints + caches a short-lived
OAuth2 token per request instead of injecting a static value; a bad key or
unreachable token endpoint denies with `llm_policy.upstream_auth_failed`
(502). Bedrock uses its static bearer token directly (no minting).
- **`/bedrock` prefix** — an optional `/bedrock` gateway-namespace prefix is
accepted and stripped via `RewriteUpstream.StripPathPrefix` so the native
`/model/...` path reaches the upstream.
Full treatment in [50-path-routed-providers.md](./50-path-routed-providers.md).
### llm_limit_check
Pre-flight gate. Reads `llm.resolved_provider_id`, calls
`CheckLLMPolicyLimits` with a 2s context timeout
([middleware.go:24, 97–106](../../../../netbird/proxy/internal/middleware/builtin/llm_limit_check/middleware.go)),
on allow stamps `llm.selected_policy_id`, `llm.attribution_group_id`,
`llm.attribution_window_seconds`. **Key invariant:** fail-open. Nil
`MgmtClient`, empty provider id, or RPC error returns `allowNoAttribution()`
— management outage doesn't take down every LLM request. Operators audit via
the access-log; PR3 may switch to fail-closed under a flag.
### llm_identity_inject
Dispatches per-rule between LiteLLM-shaped `HeaderPair`
([middleware.go:169](../../../../netbird/proxy/internal/middleware/builtin/llm_identity_inject/middleware.go))
and Portkey-shaped `JSONMetadata`
([middleware.go:292](../../../../netbird/proxy/internal/middleware/builtin/llm_identity_inject/middleware.go)).
Identity is the peer's email (or `UserID` fallback); tags are the
**authorising-groups intersection** emitted by `llm_router`, not the full
`UserGroups` — a peer in 5 groups authorised under 1 only tags as that 1.
**Anti-spoof:** every `HeadersAdd` is preceded by a `HeadersRemove` of the
same name; the framework runs `Remove` before `Add` so client-supplied
identity never reaches the upstream. Body-level inject (`tags_in_body`,
`end_user_id_in_body`) is skipped on empty / truncated / non-JSON bodies so
header attribution stays intact.
### llm_guardrail
Model allowlist deny + optional prompt-capture-with-redaction. Allowlist
match is case-insensitive via `normaliseModel`; empty allowlist disables the
check. Prompt capture reads `llm.request_prompt_raw` and emits
`llm.request_prompt` only when `prompt_capture.enabled`
([middleware.go:149–165](../../../../netbird/proxy/internal/middleware/builtin/llm_guardrail/middleware.go)).
**Key invariant:** `RedactPII` is the exported function the parsers call —
single PII contract across all three keys.
### llm_response_parser
Buffered and SSE paths share one `Invoke`
([middleware.go:102–127](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/middleware.go)):
content-type sniffing dispatches to `invokeBuffered` (JSON, status<400) or
`invokeStreaming` (text/event-stream, partial bodies tolerated). Streaming
delegates to `accumulateStream`
([streaming.go:21–30](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
using `llm.NewScanner`. A third path, `accumulateBedrockStream`
([streaming_bedrock.go](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/streaming_bedrock.go)),
decodes the AWS binary event-stream (`application/vnd.amazon.eventstream`)
returned by Bedrock's `-stream` actions — InvokeModel `chunk` frames wrap a
base64 Anthropic event, Converse frames carry text + a trailing usage block.
Cached / cache-creation buckets emit only when non-zero, preserving the existing
token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
### llm_limit_record
Post-flight write. Always returns `DecisionAllow`; response has already been
served so RPC errors mustn't surface (logged at `Debugf`). Skip-on-no-signal
at line 81 (zero tokens + zero cost). **Key invariant:** the
skip-on-missing-attribution guard at line 98 is a safety net independent of
the framework's deny short-circuit — if the gate denied and the framework
still runs the recorder, the recorder skips on absent
`UserID`+`groupID`+`UserGroups` and no phantom counter materialises.
## Full-chain diagram (canonical order)
```mermaid
flowchart TD
A[HTTP request] --> B[llm_request_parser<br/>OnRequest]
B -->|llm.provider, llm.model,<br/>llm.stream, llm.request_prompt_raw| C[llm_router<br/>OnRequest]
C -->|llm.resolved_provider_id,<br/>llm.authorising_groups,<br/>upstream rewrite + auth| D[llm_limit_check<br/>OnRequest]
D -->|deny path| Z1[403 llm_policy.*]
D -->|allow + llm.selected_policy_id,<br/>llm.attribution_group_id,<br/>llm.attribution_window_seconds| E[llm_identity_inject<br/>OnRequest]
E -->|header strip+inject<br/>+ optional body rewrite| F[llm_guardrail<br/>OnRequest]
F -->|deny: model_blocked| Z2[403 llm_policy.model_blocked]
F -->|allow + llm.request_prompt| G[upstream LLM call]
G --> H[llm_response_parser<br/>OnResponse]
H -->|llm.{input,output,total,cached_input,cache_creation}_tokens,<br/>llm.response_completion| I[cost_meter<br/>OnResponse]
I -->|cost.usd_total or cost.skipped| J[llm_limit_record<br/>OnResponse]
J --> K[response to client]
```
## limit_check ⇒ limit_record record-once invariant
```mermaid
sequenceDiagram
participant LC as llm_limit_check
participant M as management gRPC
participant U as upstream LLM
participant LR as llm_limit_record
participant DB as sqlite consumption table
LC->>M: CheckLLMPolicyLimits (2s)
alt allow
M-->>LC: selected_policy_id, attribution_group_id, window_s
LC->>U: stamps attribution metadata
U-->>LR: response + tokens (via llm_response_parser + cost_meter)
LR->>M: RecordLLMUsage (5s, debug-on-error)
M->>DB: increment (user, group, window) row
else deny
M-->>LC: llm_policy.token_cap_exceeded
Note over LR: framework short-circuits; even if invoked,<br/>recorder skips on absent UserID+groupID+UserGroups
else mgmt nil / rpc error
LC-->>LC: allowNoAttribution() — fail open
Note over LR: no window_s ⇒ recorder books only account-level<br/>budget rules (which run independently)
end
```
The integration test
[agentnetwork_chain_integration_test.go](../../../../netbird/proxy/internal/middleware/builtin/agentnetwork_chain_integration_test.go)
exercises all three branches against a real sqlite store + bufconn gRPC —
no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
(line 130), `TestChain_DenyPath_GateRejectsAndNoConsumptionWritten` (line
207), `TestChain_CapExhaustTransition` (line 265).
## Public contracts (per-middleware JSON config)
| Middleware | Config shape |
|---|---|
| `llm_request_parser` | `{provider_id?, redact_pii?, capture_prompt?: *bool}` ([factory.go:19–37](../../../../netbird/proxy/internal/middleware/builtin/llm_request_parser/factory.go)) |
| `llm_router` | `{providers: [{id, models, upstream_scheme, upstream_host, upstream_path?, auth_header_name, auth_header_value, allowed_group_ids}]}` |
| `llm_limit_check` | `{}` — pulls `MgmtClient` from `FactoryContext` |
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{model_allowlist: []string, prompt_capture: {enabled, redact_pii}}` |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time.
## Invariants
1. **limit_check ↔ limit_record paired.** They MUST appear together. Gate
stamps attribution metadata on the request leg; recorder reads it on the
response leg. If a chain contains only the recorder, the
skip-on-missing-attribution guard at
[llm_limit_record/middleware.go:81–87, 98–103](../../../../netbird/proxy/internal/middleware/builtin/llm_limit_record/middleware.go)
keeps counters consistent but no enforcement runs. Only-gate means
counters never tick and headroom appears infinite.
2. **`capture_prompt` / `capture_completion` pointer semantics.** Both are
`*bool`. `nil` = "preserve legacy emit" (back-compat default for
non-agent-network callers and pre-toggle tests). `false` = suppress the
key entirely (access-log row carries zero prompt / completion content).
`true` = emit. The synthesiser sets the pointer explicitly to the
account's `EnablePromptCollection` toggle — see commit `4836d5a19` for
the bug where a missing pointer was incorrectly treated as `false`. Fix
in [llm_request_parser/factory.go:55–61](../../../../netbird/proxy/internal/middleware/builtin/llm_request_parser/factory.go)
and the symmetric [llm_response_parser/middleware.go:62–68](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/middleware.go).
`redact_pii` is an orthogonal `bool` controlling **form** of emitted
content, not whether it's emitted.
3. **`redact_pii` is parser-side.** Both parsers import
`llm_guardrail.RedactPII` and run it BEFORE stamping the metadata bag.
Load-bearing because the access-log sink reads `llm.request_prompt_raw`
and `llm.response_completion` directly — by the time `llm_guardrail`
runs its own pass on `llm.request_prompt`, the raw key has already been
stamped. Tests: `TestInvoke_RedactPii_RedactsBeforeEmittingRawPrompt`,
`TestInvoke_RedactPii_RedactsCompletionBeforeEmit`.
4. **Metadata allowlist enforcement.** Every middleware declares
`MetadataKeys()`. The framework accumulator drops any KV outside that
allowlist. When adding a new key, also extend the docstring in
`middleware/keys.go`.
5. **Closed deny-code set.** All deny paths emit one of:
`llm_policy.model_not_routable`, `llm_policy.no_authorised_provider`,
`llm_policy.model_blocked`, `llm_policy.token_cap_exceeded`,
`llm_policy.unmeterable_publisher` (path-routed Vertex publisher with no
parser → 403), `llm_policy.upstream_auth_failed` (GCP token mint failure →
502), or the management-supplied code on `llm_limit_check`. These surface
verbatim; arbitrary middleware text never reaches the wire.
## Things to scrutinise
**Correctness.** `llm_router` model match treats an empty `Models` slice as
"claim every model"
([middleware.go:238–248](../../../../netbird/proxy/internal/middleware/builtin/llm_router/middleware.go))
for gateway-style providers — confirm no real provider record ships with an
empty `Models` by accident. Path-prefix tie-break falls back to declaration
order when no candidate prefix-matches, so the synthesiser must emit a
deterministic order. `llm_limit_record` discards `strconv.ParseInt` errors
([middleware.go:78–80](../../../../netbird/proxy/internal/middleware/builtin/llm_limit_record/middleware.go))
— relies on `llm_response_parser` always emitting parseable values; spot-check
the streaming partial path on truncated bodies.
**Security.** Auth headers must NEVER appear on `Mutations.HeadersAdd/Remove`
for the router — flag any PR that introduces a direct headers path. The
capture-pointer regression (`4836d5a19`) is the kind of bug that ships PII to
logs silently; walk every synthesiser config path and check the pointer is
set explicitly. `llm_identity_inject` body inject silently skips on a
non-object `metadata` field
([middleware.go:262–270](../../../../netbird/proxy/internal/middleware/builtin/llm_identity_inject/middleware.go))
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
**Observability.** Every deny stamps `llm_policy.decision=deny` and a
matching `llm_policy.reason` — access-log can pivot on either.
`llm_limit_record` only logs at `Debugf` on RPC failure
([middleware.go:125–130](../../../../netbird/proxy/internal/middleware/builtin/llm_limit_record/middleware.go));
operators need an alternate signal (metric on `RecordLLMUsage` failures) for
counter accuracy.
## Test coverage
| File | Tests | Notes |
|---|---:|---|
| `all_test.go` | 1 | Registry surface lock |
| `agentnetwork_chain_integration_test.go` | 3 | Allow/deny/cap-exhaust vs live sqlite + bufconn gRPC |
| `llm_request_parser/middleware_test.go` | 18 | `provider_id` bypass, redaction, capture-pointer, rune-safe truncation |
| `llm_router/middleware_test.go` | 19 | Three-pass match, deny codes, path-prefix tie-break, header strip+inject |
| `llm_limit_check/middleware_test.go` | 6 | Allow/deny, fail-open on nil mgmt / RPC error, attribution stamping |
| `llm_identity_inject/middleware_test.go` | 28 | HeaderPair, JSONMetadata, ExtraHeaders, body inject, anti-spoof |
| `llm_guardrail/middleware_test.go` | 15 | Allowlist case-insensitivity, prompt capture toggle, deny shape |
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing loader.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).
- Upstream config: `management/server/agentnetwork/synthesizer` (out of scope).
- Framework: `proxy/internal/middleware/{chain,dispatcher,accumulator,registry}.go`.
- Metadata key registry: `proxy/internal/middleware/keys.go`.
- gRPC surface: `proto.ProxyServiceClient.{CheckLLMPolicyLimits,RecordLLMUsage}`.

401
docs/agent-networks/modules/32-proxy-llm-parsers.md Normal file
View File

@@ -0,0 +1,401 @@
# proxy/llm-parsers — SDK adapters + pricing + SSE
> Reviewer profile: LLM / proxy. You should know the OpenAI Responses API
> (`/v1/responses`) and the older Chat Completions API (`/v1/chat/completions`),
> the Anthropic Messages API (`/v1/messages`), the SSE wire format
> (`event:` / `data:` lines, `\n\n` framing, CRLF tolerance), and the difference
> between OpenAI's cached-prompt **subset** vs Anthropic's cache_read **additive**
> token accounting. The pricing table's per-provider cost formula is the
> highest-leverage place a small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing loader.
---
## Module boundary
`proxy/internal/llm` is the runtime-agnostic LLM library shared by every
middleware that needs to understand provider-specific shapes. Zero
proxy-framework dependencies:
- `parser.go` — `Parser` interface, `Provider` enum, public factories
(`Parsers`, `DetectParser`, `ParserByName`).
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/` — embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `fixtures/` — captured request/response/stream bodies the tests replay.
Lifted out of the legacy plugin runtime by commit `8efbd430d` so the same
parsers can be reused later by a WASM adapter
([parser.go:1–6](../../../../netbird/proxy/internal/llm/parser.go)).
## Commits in scope
`8efbd430d` G4 part 1: lift LLM utils + delete legacy plugin runtime ·
`06ff17b38` AN-0: additive proto + OpenAPI schemas + base types ·
`19d12adc8` openai match bare `/chat/completions` for Cloudflare AI Gateway ·
`afefa38ce` cached + cache-creation tokens for OpenAI and Anthropic ·
`a67022ffa` stamp parser id from catalog into `llm_request_parser`.
Downstream-consumer commits are covered in the sibling module review.
## Files changed
| File | LOC | Notes |
|---|---:|---|
| `parser.go` | 104 | Interface + factories + `Provider{Unknown,OpenAI,Anthropic}` enum |
| `openai.go` | 347 | Chat Completions + Completions + Responses API; cached_tokens subset |
| `openai_test.go` | 222 | 11 tests; fixture replay + cached/Responses-API matrix |
| `anthropic.go` | 172 | Messages + legacy `/v1/complete`; cache_read + cache_creation additive |
| `anthropic_test.go` | 154 | 7 tests including streaming-extraction-skipped contract |
| `bedrock.go` | 190 | AWS Bedrock InvokeModel (snake_case) + Converse (camelCase) response shapes; model lives in URL path |
| `bedrock_test.go` | — | InvokeModel + Converse usage shapes; AWS event-stream content-type → `ErrStreamingUnsupported` on buffered `ParseResponse` |
| `sse.go` | 117 | `bufio`-backed scanner; CRLF normalised; trailing-event handling |
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 21–59 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
## Request body → parser dispatch
```mermaid
flowchart TD
A[HTTP request<br/>URL + JSON body] --> B{ParserByName?<br/>provider_id config set}
B -- yes --> P[matched Parser]
B -- no --> C[DetectParser]
C --> D{loop Parsers<br/>OpenAIParser, AnthropicParser}
D -- DetectFromURL match --> P
D -- no match --> X[ok=false<br/>middleware skips]
P --> E[ParseRequest body]
E -->|err: ErrMalformedRequest| Y[middleware emits provider only]
E --> F[RequestFacts<br/>model + stream]
P --> G[ExtractPrompt body]
G --> H[joinMessages<br/>extractContentParts<br/>decodeStringOrJoin]
H --> I[prompt text<br/>or empty]
F --> J[stamps llm.model + llm.stream]
I --> K[stamps llm.request_prompt_raw<br/>subject to capture_prompt gate]
```
OpenAI's URL hints
([openai.go:27–33](../../../../netbird/proxy/internal/llm/openai.go)) include
both `/v1/chat/completions` and the bare `/chat/completions` — the latter
covers Cloudflare AI Gateway, which rewrites the canonical version segment
(commit `19d12adc8`). Anthropic's hints are `/v1/messages` and `/v1/complete`
([anthropic.go:14–17](../../../../netbird/proxy/internal/llm/anthropic.go)).
Both implementations use case-insensitive substring matching so a proxy prefix
strip / rewrite doesn't defeat detection.
`ParserByName` ([parser.go:93–103](../../../../netbird/proxy/internal/llm/parser.go))
is the **agent-network bypass**: the synthesiser knows which parser to use
because it built the synth service from the catalog, so it stamps
`provider_id` on the parser config and the middleware skips URL sniffing
entirely. This is what makes the same parser set work whether the request
flows to OpenAI direct, to LiteLLM, to Portkey, or to any gateway with a
non-canonical URL shape.
**Path-routed providers (Vertex AI, Bedrock) bypass both `ParserByName` and
`DetectParser`.** The model and the parser surface live in the URL path, so the
request middleware extracts them directly (`parseVertexPath` /
`parseBedrockPath`) before the parser-selection step. For Vertex the publisher
segment picks the parser (`anthropic` → Anthropic parser; `google`/Gemini →
none, request denied as unmeterable). For Bedrock the dedicated `BedrockParser`
handles the response. Full treatment in
[50-path-routed-providers.md](./50-path-routed-providers.md).
## Streaming response → SSE chunker → response parser → completion + token count
```mermaid
sequenceDiagram
participant U as upstream LLM
participant LR as llm_response_parser<br/>(OnResponse)
participant S as llm.NewScanner<br/>(SSE framer)
participant P as Parser-specific accumulator<br/>(accumulateOpenAIStream<br/>or accumulateAnthropicStream)
U-->>LR: text/event-stream<br/>(buffered prefix in RespBody)
LR->>S: NewScanner(bytes.NewReader(body))
loop until EOF or [DONE]
S-->>LR: Event{Type, Data}
LR->>P: dispatch per event.Type<br/>(OpenAI: data-only<br/>Anthropic: named events)
P-->>P: accumulate completion text<br/>track usage from final frame
end
P-->>LR: llm.Usage + completion string
LR->>LR: appendUsage stamps<br/>llm.{input,output,total,cached_input,cache_creation}_tokens
LR->>LR: truncateCompletion(3500 bytes, rune-safe)
LR->>LR: redactPII if redact_pii && captureCompletion
```
`Scanner.Next`
([sse.go:44–87](../../../../netbird/proxy/internal/llm/sse.go)) returns one
event per `\n\n` boundary; multiple `data:` lines join with `\n`; comment lines
(starting with `:`) are skipped per the SSE spec; a trailing event without a
closing blank line is still returned before `io.EOF` so a server that closes
the connection cleanly doesn't lose the last frame
([sse.go:55–58](../../../../netbird/proxy/internal/llm/sse.go)). CRLF is
normalised in `trimEOL` so fixtures captured from live servers replay
unchanged.
## Per-provider
### OpenAI
[openai.go:54–67](../../../../netbird/proxy/internal/llm/openai.go) defines
`openAIRequest` with three prompt fields: `messages` (Chat Completions),
`prompt` (legacy), `input` (Responses API). The decoder uses
`json.RawMessage` so each shape is parsed lazily.
`ParseResponse`
([openai.go:117–146](../../../../netbird/proxy/internal/llm/openai.go))
accepts both naming conventions: Chat Completions returns
`prompt_tokens`/`completion_tokens`, Responses API returns
`input_tokens`/`output_tokens`. `pickInt64` prefers Responses-API names and
falls back — same parser handles both endpoints without per-route config.
`openAICachedTokens` mirrors the fallback for
`input_tokens_details.cached_tokens` vs `prompt_tokens_details.cached_tokens`.
**Key invariant:** `CachedInputTokens` for OpenAI is a SUBSET of
`InputTokens`. The cost meter clamps to guard against malformed upstream
responses where `cached > total`.
### Anthropic
[anthropic.go:37–49](../../../../netbird/proxy/internal/llm/anthropic.go)
defines `anthropicRequest` covering Messages API (`system` + `messages[]`)
and legacy `/v1/complete` (`prompt` string). `ExtractPrompt` emits
`system: <text>` first when present, then per-message `role: content`.
`ParseResponse`
([anthropic.go:82–104](../../../../netbird/proxy/internal/llm/anthropic.go))
fills three independent token buckets: `InputTokens`, `CacheReadInputTokens`,
`CacheCreationInputTokens`. Latter two are **additive** (not subset).
`TotalTokens` sums all four so downstream dashboards render one "tokens"
number without double-counting.
`ExtractCompletion` walks `content[]` `{type, text}` parts and concatenates
non-empty text with newlines, falling back to legacy `completion`.
### Bedrock
[bedrock.go](../../../../netbird/proxy/internal/llm/bedrock.go) implements the
`Parser` interface for the AWS Bedrock runtime. Bedrock is **path-routed**: the
model lives in the URL (`/model/{id}/{action}`), so the request middleware
extracts it (see [50-path-routed-providers.md](./50-path-routed-providers.md))
and `ParseRequest` is a deliberate no-op. The parser's real work is on the
response leg, covering both Bedrock body shapes:
- **InvokeModel** — vendor-native. Anthropic-on-Bedrock returns snake_case usage
(`input_tokens`, `output_tokens`, `cache_read_input_tokens`,
`cache_creation_input_tokens`) with the same additive cache buckets as
first-party Anthropic.
- **Converse** — unified camelCase (`inputTokens`, `outputTokens`,
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
### SSE framing
`Scanner` is `bufio`-backed, 64 KiB read buffer, 1 MiB max line so a
malicious upstream can't blow process memory
([sse.go:33–38, 97–100](../../../../netbird/proxy/internal/llm/sse.go)).
`splitField` strips one space after the `:` per the SSE spec. Documented
`not safe for concurrent use`; every consumer creates a fresh scanner per
response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing catalog
`Table.Cost`
([pricing.go:129–174](../../../../netbird/proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
| Provider | Formula |
|---|---|
| `openai` | `(inTokens − clamped) × InputPer1K + clamped × CachedInputPer1K + outTokens × OutputPer1K` where `clamped = min(cachedInput, inTokens)` |
| `anthropic`, `bedrock` | `inTokens × InputPer1K + cachedInput × CacheReadPer1K + cacheCreation × CacheCreationPer1K + outTokens × OutputPer1K` |
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:172-174](../../../../netbird/proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Loader`
([pricing.go:212–268](../../../../netbird/proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
## Public contracts
**`Parser` interface**
([parser.go:50–66](../../../../netbird/proxy/internal/llm/parser.go)):
```go
type Parser interface {
Provider() Provider
ProviderName() string
DetectFromURL(path string) bool
ParseRequest(body []byte) (RequestFacts, error)
ParseResponse(status int, contentType string, body []byte) (Usage, error)
ExtractPrompt(body []byte) string
ExtractCompletion(status int, contentType string, body []byte) string
}
```
Adding a provider means implementing this interface and appending to the
slice returned by `Parsers()` ([parser.go:78–84](../../../../netbird/proxy/internal/llm/parser.go)).
Order matters: `DetectFromURL` ties resolve by registration order.
`Parsers()` today returns `{OpenAIParser, AnthropicParser, BedrockParser}`.
**`ProviderID` enum**
([parser.go:10–19](../../../../netbird/proxy/internal/llm/parser.go)):
`ProviderUnknown = 0`, `ProviderOpenAI = 1`, `ProviderAnthropic = 2`,
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` lookup**
([pricing.go:129](../../../../netbird/proxy/internal/llm/pricing/pricing.go)):
```go
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
```
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130–132](../../../../netbird/proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
## Invariants
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:14–16](../../../../netbird/proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
([sse.go:55–58](../../../../netbird/proxy/internal/llm/sse.go)). Tests:
`TestSSEScanner_OpenAIFixture`, `TestSSEScanner_AnthropicFixture`,
`TestSSEScanner_MultilineData`, `TestSSEScanner_CRLF`. The streaming
accumulators ride on this: `accumulateAnthropicStream` and
`accumulateOpenAIStream` `break` on any scanner error to return partial
usage rather than aborting
([streaming.go:68–73, 144–150](../../../../netbird/proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:29–30](../../../../netbird/proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:42–49](../../../../netbird/proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370–394](../../../../netbird/proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:25–57](../../../../netbird/proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397–398](../../../../netbird/proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
## Things to scrutinise
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147–149](../../../../netbird/proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
request sending all three reports only `messages` (uncommon but worth
noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
## Test coverage
| File | Tests | Coverage highlights |
|---|---:|---|
| `parser_test.go` | 3 | `Parsers()` shape lock, `DetectParser` URL matrix, provider enum stability |
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../../netbird/proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
`openai_responses.json` (Responses API shape),
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewLoader`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).
- Direct callers: `llm_request_parser/middleware.go:82–94`,
`llm_response_parser/middleware.go:113–123`,
`llm_response_parser/streaming.go:65, 142`, `cost_meter/factory.go:49–57`.
- Out of scope: agent-network synthesiser stamping `provider_id`
(`a67022ffa`) — see management-side module review. Proxy server boot +
`FactoryContext` construction — see proxy-framework module review.

211
docs/agent-networks/modules/33-proxy-runtime.md Normal file
View File

@@ -0,0 +1,211 @@
# proxy/runtime — translate + serve + log
> **Reviewer profile:** Proxy maintainer; familiar with the existing `proxy/internal/proxy/reverseproxy.go` request lifecycle, `proxy/internal/accesslog` emission, `proto.ProxyMapping` ingestion via `SyncMappings`/`GetMappingUpdate`, and the `proxy/internal/middleware` framework introduced in modules 30/31/32.
> **Time to review:** 75 minutes
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
## Module boundary
Turns the synth-service wire format from `ProxyService.SyncMappings`/`GetMappingUpdate` into in-process middleware chains and runs them on top of the existing `httputil.ReverseProxy`. Four concerns: (a) **translate** — `proto.MiddlewareConfig` → validated `middleware.Spec` (proxy/middleware_translate.go) + self-register the eight built-ins (proxy/middleware_register.go); (b) **boot + rebuild** — construct the `middleware.Manager`, share the OTel meter, install the live-service check, rebuild per-path chains on every `addMapping`/`modifyMapping` (proxy/server.go); (c) **serve** — resolve chain at request time, capture bodies under a global budget, invoke `RunRequest`/`RunResponse`/`RunTerminal`, render deny responses, apply `UpstreamRewrite` (proxy/internal/proxy/reverseproxy.go); (d) **log + tag** — emit access-log entries with the new `agent_network` flag, gate emission on `EnableLogCollection` via `DisableAccessLog` (proxy/internal/accesslog).
**Inert for non-agent-network targets**: nil or empty chain → existing fast path (reverseproxy.go:127-139); `SuppressAccessLog` defaults false so the access-log middleware emits unchanged.
## Commits in scope
| SHA | Subject | LOC delta (this module) |
| --- | ------- | --- |
| 3ed29d855 | AN-4b: middleware plumbing in the proxy request path | +711 / −9 |
| e64ea4b02 | AN-5: built-in middlewares + registry registration | +16 |
| 0f9f56a58 | AN-5b: activate the middleware system in the proxy server | +136 / −1 |
| 9a1547143 | AN-6: access-log `agent_network` flag end-to-end | +3 |
| 9ebe219fd | test: lock the auth→middleware group-propagation wiring | +136 |
| 468875cb4 | Wire `EnableLogCollection` to suppress access-log | +222 / −5 |
| b438a7194 | Carry identity onto respInput (PII redaction commit; this module: respInput fix only) | +12 / −4 |
| 2a0d4991b | Self-contained full-chain integration test | +314 |
Total in scope: ~1545 added / 14 deleted across 14 files.
## Files changed
| Path | LOC | Role |
| ---- | --- | ---- |
| proxy/middleware_translate.go | +165 | proto→Spec translation; slot/failmode/timeout mapping; caps |
| proxy/middleware_translate_test.go | +246 | translator unit tests |
| proxy/middleware_register.go | +16 | blank-imports the eight builtins for `init()` registration |
| proxy/server.go | +129 | `initMiddlewareManager`, `rebuildMiddlewareChains`, `isLiveService`, `buildMiddlewareBindings`, new Server fields, `protoToMapping` stamps AgentNetwork/DisableAccessLog/CaptureConfig/Middlewares |
| proxy/internal/proxy/reverseproxy.go | +289 / −9 | `WithMiddlewareManager`, chain dispatch, body capture, `applyUpstreamRewrite`/`Headers`, `buildRequestInput`, response-leg respInput identity fields |
| proxy/internal/proxy/reverseproxy_test.go | +44 | `TestBuildRequestInput_PropagatesIdentityAndGroups` |
| proxy/internal/proxy/context.go | +43 | `agentNetwork`, `suppressAccessLog`, `userGroupNames` on `CapturedData` |
| proxy/internal/proxy/servicemapping.go | +16 | new `PathTarget` fields |
| proxy/internal/proxy/agent_network_chain_realstack_test.go | +314 | end-to-end self-contained chain test |
| proxy/internal/accesslog/logger.go | +2 | `logEntry.AgentNetwork` → `proto.AccessLog` |
| proxy/internal/accesslog/middleware.go | +9 / −1 | reads `GetAgentNetwork()`; gates `l.log` on `!GetSuppressAccessLog()` |
| proxy/internal/accesslog/middleware_test.go | +185 | suppress/default/preserves-usage assertions |
| proxy/internal/auth/middleware_test.go | +92 | tunnel-peer group propagation contract |
| proxy/internal/metrics/metrics.go | +9 | `Meter()` getter for the middleware manager |
## Architecture & flow
### Synth-service ingestion → translate → register → serve
```mermaid
flowchart TD
A[Management SyncMappings/GetMappingUpdate] --> B["processMappings\nserver.go:1492"]
B --> C{Mapping type}
C -->|CREATED| D["addMapping → setupHTTPMapping → updateMapping"]
C -->|MODIFIED| E["modifyMapping → cleanupMappingRoutes → setupHTTPMapping → updateMapping"]
C -->|REMOVED| F["removeMapping → cleanupMappingRoutes → invalidateMiddlewareChains"]
D --> G["protoToMapping\nserver.go:2181"]
E --> G
G --> H["translateMiddlewareConfigs\nmiddleware_translate.go:55"]
G --> I["translateMiddlewareCaptureConfig\nmiddleware_translate.go:18"]
H --> J["[]middleware.Spec on PathTarget"]
I --> K["*bodytap.Config on PathTarget"]
J --> L["proxy.AddMapping\nservicemapping.go:118"]
K --> L
L --> M["rebuildMiddlewareChains\nserver.go:2017 → Manager.Rebuild"]
F --> N["Manager.Invalidate(serviceID)"]
```
### Per-request lifecycle through the chain + accesslog
```mermaid
sequenceDiagram
autonumber
participant C as Client
participant M as accesslog.Middleware
participant A as auth.Middleware (Protect)
participant RP as ReverseProxy.ServeHTTP
participant CH as middleware.Chain
participant U as Upstream
C->>M: HTTP request
M->>M: NewCapturedData(requestID), WithCapturedData(ctx)
M->>A: next.ServeHTTP
A->>A: Private → ValidateTunnelPeer → stamp UserID/Email/Groups/GroupNames/AuthMethod
A->>RP: next.ServeHTTP
RP->>RP: findTargetForRequest → targetResult
RP->>RP: stamp ServiceID/AccountID/AgentNetwork/SuppressAccessLog on CapturedData
RP->>RP: resolveChain via Manager.ChainFor
alt chain == nil or Empty
RP->>U: httputil.ReverseProxy.ServeHTTP (fast path)
else chain non-empty
RP->>RP: bodytap.CaptureRequest (global budget)
RP->>CH: RunRequest
CH-->>RP: denyOutput? requestMeta + upstreamRewrite
alt deny
RP->>C: RenderDenyResponse
else allow
RP->>RP: capturingWriter + applyUpstreamRewrite/Headers
RP->>U: httputil.ReverseProxy.ServeHTTP(respWriter)
U-->>RP: response
RP->>CH: RunResponse (respInput carries UserGroups, b438a7194)
RP->>CH: RunTerminal (merged request+response metadata)
end
end
RP-->>M: handler returns
M->>M: build logEntry incl. AgentNetwork
alt SuppressAccessLog == true
M->>M: skip l.log; still trackUsage
else default
M->>M: l.log → goroutine SendAccessLog
end
```
### EnableLogCollection suppression path
```mermaid
flowchart LR
S["agentnetwork.Settings.EnableLogCollection"] --> B["synthesizer: target.DisableAccessLog = !EnableLogCollection"]
B --> P["proto PathTargetOptions.disable_access_log (field 13)"]
P --> T["protoToMapping reads GetDisableAccessLog()\nserver.go:2211"]
T --> M["PathTarget.DisableAccessLog\nservicemapping.go:47"]
M --> R["ServeHTTP: cd.SetSuppressAccessLog\nreverseproxy.go:106"]
R --> G["accesslog middleware: if !GetSuppressAccessLog l.log\nmiddleware.go:95"]
R --> U["trackUsage unconditional — bandwidth telemetry preserved"]
```
**Ingestion** lands as a `ProxyMapping` batch on `handleSyncMappingsStream`/`handleMappingStream`. `processMappings` dispatches to `addMapping`/`modifyMapping`/`removeMapping`; HTTP goes `setupHTTPMapping → updateMapping → protoToMapping`. `protoToMapping` (server.go:2181) is the single translation surface that materialises `[]middleware.Spec`, `*bodytap.Config`, `AgentNetwork`, `DisableAccessLog` onto each `PathTarget`; `updateMapping` finishes with `s.proxy.AddMapping(m)` (atomic swap under `mappingsMux`) and `s.rebuildMiddlewareChains(svcID, m)`.
At **request time** the access-log middleware stamps `CapturedData`; the auth chain runs (Private services lift `peer_group_ids` from `ValidateTunnelPeer` — auth/middleware_test.go:322). `ReverseProxy.ServeHTTP` resolves the chain; nil or empty → original `httputil.ReverseProxy`, no body capture. When a chain matches, body is captured under the global budget, `RunRequest` produces an `UpstreamRewrite` (`llm_router` selects a provider, rewrites scheme/host/path, injects `Authorization`), and `RunResponse`+`RunTerminal` run after the upstream returns. The terminal slot sees the merged metadata bag — that's how `llm_limit_record` ships the consumption sample. The **access-log** addition: `logEntry.AgentNetwork` from `GetAgentNetwork()` onto `proto.AccessLog.AgentNetwork`; the gate at middleware.go:95 honors `EnableLogCollection`, skipping `l.log` but keeping `trackUsage` so bandwidth telemetry survives.
## Public contracts touched
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).
- `accesslog.logEntry.AgentNetwork` + `proto.AccessLog.AgentNetwork` (logger.go:131, 268).
- `metrics.Metrics.Meter()` exposes the OTel meter for the middleware manager (metrics.go:53-58).
## Invariants
- **Synth-service updates are live (no proxy restart).** Every `MODIFIED` flows through `modifyMapping → cleanupMappingRoutes` (invalidates chains) `→ setupHTTPMapping → updateMapping → rebuildMiddlewareChains`. **Note 263dabd73 (ProxyMapping.Private preservation):** the bug it fixed lives in `management/internals/shared/grpc/proxy.go:shallowCloneMapping`, not this module, but it surfaces here — without the fix, every `MODIFIED` synth service arrived `private=false`, auth skipped `ValidateTunnelPeer`, `CapturedData.UserGroups` stayed empty, `llm_router` denied with `llm_policy.no_authorised_provider`, and only a management restart worked around it. Review this module assuming `mapping.GetPrivate()` is correct on every batch — that assumption is what the cited fix established.
- **`EnableLogCollection=false` suppresses access-log writes but middleware still runs.** Gate is one `if !cd.GetSuppressAccessLog()` immediately around `l.log(entry)` (middleware.go:95); `trackUsage` runs below the gate. Locked by `TestMiddleware_SuppressAccessLog_PreservesUsageTracking` (middleware_test.go:139).
- **`agent_network` flag on access-log entries is set when the chain processed the request.** Source `target.AgentNetwork`, stamped at reverseproxy.go:105, read at accesslog/middleware.go:86.
- **auth → builtin group propagation (9ebe219fd locked this).** `Protect` writes `UserGroups`/`UserGroupNames`; `buildRequestInput` (reverseproxy.go:333) copies them into `middleware.Input`. After b438a7194 the response-leg `respInput` (reverseproxy.go:196-223) also carries `UserEmail`/`UserGroups`/`UserGroupNames` — `llm_limit_record` needs `UserGroups` to ship `group_ids` so management's group-targeted budget rules match (comment at reverseproxy.go:211-215).
- **Empty chains stay on the fast path.** `ServeHTTP` skips body capture and the run sequence when `chain == nil || chain.Empty()` (reverseproxy.go:127).
- **Self-registration is the only way a builtin reaches the registry.** `middleware_register.go` blank-imports each builtin; `init()` adds the factory to `mwbuiltin.DefaultRegistry()`. Missing it → translator drops the entry with a warn (translate.go:97).
## Things to scrutinize
### Correctness
- **Translate edge cases** — drops on nil cfg, empty ID, unknown ID, UNSPECIFIED slot; each logs one warn; volume bounded by `MaxMiddlewaresPerChain`.
- **Re-translate without dropping in-flight requests** — `Manager.Rebuild` is the only call from `rebuildMiddlewareChains`. Reverse proxy reads `ChainFor` once per request (reverseproxy.go:327) and runs the captured `*Chain` for the whole request. Verify in module 30 that `Rebuild` swaps atomically.
- **ProxyMapping.Private preservation** — fixed in 263dabd73 (management). Proxy-side regression catches: `TestProtect_PrivateService_TunnelPeerGroupsPropagate` + the integration test.
- **Body-capture cleanup** — `defer releaseBudget()` (reverseproxy.go:145) and `defer capturingWriter.Release()` (reverseproxy.go:180) must run on every return; confirm no future `return` lands between acquisition and defer.
- **`applyUpstreamRewrite` clones the URL** — `cloned := *orig` value-copies `*url.URL`; safe because overwritten fields are strings, not slices/maps (reverseproxy.go:285-292).
### Security
- **Translate validates every config** — registry membership rejects unknown IDs; UNSPECIFIED slot drops; ID-less drops; raw config copied (not aliased) at translate.go:109.
- **`AuthHeader`/`StripHeaders` only reachable via `UpstreamRewrite`** — regular mutation surface goes through the framework denylist (`Authorization`/`Cookie` blocked); only the router middleware can replace `Authorization` (reverseproxy.go:296-304). Confirm in module 30 nothing outside the proxy-trusted path populates `UpstreamRewrite.AuthHeader`.
- **`stampNetBirdIdentity` strips client-sent values first** (reverseproxy.go:742-743) — anti-spoof for `X-NetBird-User`/`X-NetBird-Groups`; control chars filtered; comma-bearing labels dropped (reverseproxy_test.go:1217/:1243/:1193).
- **Auth → group propagation** — `auth/middleware_test.go:322` and `:366` cover the contract. If auth ever stops calling `ValidateTunnelPeer` for Private services, every agent-network request silently denies.
### Concurrency
- **Chain replacement under in-flight requests** — `findTargetForRequest` takes `mappingsMux.RLock`; `AddMapping` writes. `resolveChain` calls `ChainFor` once; even if `Rebuild` swaps mid-request, in-flight requests keep running on the captured pointer.
- **`CapturedData` mutation across slots** — accessors take `sync.RWMutex`; slices deep-copied on both Set and Get. Verify no caller mutates the returned slice expecting it to land back.
- **`Manager.Invalidate` race** — `removeMapping` invalidates after `cleanupMappingRoutes`; mapping read happens before chain resolution, so requests before invalidate run captured chains; later ones fail `findTargetForRequest`.
- **`Logger.log` goroutine** — `logSem` caps at `maxLogWorkers = 4096`; overflow → `dropped.Add(1)` + debug log. Middleware test uses a buffered channel and 150ms negative-assertion window — review whether 150ms holds on slow CI.
### Backward compatibility
- **Non-agent-network services unaffected** — `protoToMapping` reads new fields only when `opts != nil`; defaults leave `Middlewares`/`CaptureConfig` nil → chain resolves nil → fast path. Existing `reverseproxy_test.go` (non-chain) still passes.
- **`disable_access_log` is proto field 13, default false** — every existing target unset; gate is no-op. Locked by `TestMiddleware_SuppressAccessLog_DefaultEmitsLog` (middleware_test.go:104).
- **`Server` additions optional** — 256 MiB default when `MiddlewareCaptureBudgetBytes ≤ 0` (server.go:1997-2000).
### Performance
- **Translate cost per push** — O(n) with per-entry registry lookup and `config_json` copy; negligible vs. the upstream gRPC unmarshal.
- **Empty-chain hot path** — one `ChainFor` map lookup + one `chain.Empty()` check; no allocation delta vs. pre-PR.
- **Body capture buffer churn** — `bodytap.CaptureRequest` allocates `MaxRequestBytes` per chain-hitting request; `releaseBudget` ties allocation to the 256 MiB proxy-wide budget. Confirm in module 30 the budget is a hard cap.
### Observability
- **Metrics** — `Metrics.Meter()` shared with `middleware.NewMetrics` (server.go:1990-1993) so middleware instruments land in the same prometheus exporter. No new metrics defined here.
- **Access-log accuracy** — every entry carries `AgentNetwork`; terminal-slot metadata merged into `CapturedData.Metadata` (reverseproxy.go:238-241).
- **Deny logs at `Infof`** (reverseproxy.go:170) — review whether `Info` is too noisy at high deny rates; consider Debug or rate-limit.
## Test coverage
| Test file | Locks down |
| --------- | ---------- |
| proxy/middleware_translate_test.go | Empty/nil → nil; field preservation; unknown ID skip; nil registry permissive; timeout clamping; fail-mode + slot incl. UNSPECIFIED-drop; empty-ID drop; truncation above + at `MaxMiddlewaresPerChain` |
| proxy/internal/proxy/reverseproxy_test.go | Rewrite host/headers/cookies/query; trusted proxy; path forwarding; classifyProxyError; X-NetBird-User/Groups anti-spoof + CSV-join + control-char/comma rejection + fallback-to-ID; `TestBuildRequestInput_PropagatesIdentityAndGroups` (UserGroups/Email/GroupNames/AgentNetwork reach `middleware.Input`) |
| proxy/internal/proxy/agent_network_chain_realstack_test.go | **THE end-to-end integration test added by 2a0d4991b.** Drives a real agent-network request through `ReverseProxy.ServeHTTP` with the chain the synthesizer produces, against an in-process management gRPC (bufconn) backed by a real sqlite store + real `agentnetwork.Manager`, plus an `httptest` upstream. No tilt/docker/real LLM. Guarantees: (1) response-leg `respInput` carries `UserGroups` so `llm_limit_record` ships non-empty `group_ids` and the admin-group consumption row increments — the exact regression class b438a7194 fixed; (2) `RedactPii=true` redacts both prompt and completion on captured metadata; (3) full chain runs against a real management stack. **Line 189-211 inlines the proto→Spec mapping** instead of calling the proxy's private `translateMiddlewareConfig` — keep that inline mirror in sync with `proxy/middleware_translate.go` or the test silently diverges from production. |
| proxy/internal/accesslog/middleware_test.go | `SuppressAccessLog=true` skips `SendAccessLog` (150ms negative wait); default emits one send (2s positive); usage tracking runs under suppression |
| proxy/internal/auth/middleware_test.go | `TestProtect_PrivateService_TunnelPeerGroupsPropagate` proves `peer_group_ids` reach `CapturedData.UserGroups`; `TestProtect_PrivateService_TunnelPeerDenied` proves rejected peers 403 without reaching the handler |
The integration test replaces the bash 50/51 e2e legs in ~5s, no docker/tilt — exercising the real synthesizer, `Manager.Rebuild`, `ServeHTTP` dispatch, and `llm_limit_record` writing a real consumption row through the real `agentnetwork.Manager` over real gRPC.
## Known limitations / explicit non-goals
- **Translator does not validate `RawConfig` JSON** — factory's job at `New([]byte)`. Confirm in module 30 that a per-binding factory failure doesn't poison the rest of the chain.
- **No throttle on management push rate** — every `MODIFIED` triggers `Manager.Rebuild`. Mitigation upstream.
- **Streaming responses (SSE)** — body capture is streaming-aware, but response-leg middleware runs only after the response completes; long SSE streams delay `llm_limit_record` until close.
- **OIDC-only path doesn't carry tunnel-peer groups** — agent-network synth services rely on the Private tunnel-peer path; JWT groups claim is the only carrier for non-Private OIDC.
- **`agent_network` flag on L4 entries** not added; HTTP-only in this PR.
- **`mw.capture.bypass_reason` metadata key** documented at reverseproxy.go:151,184; namespace this in module 30/31 to avoid collisions.
## Cross-references
- Upstream: [shared/api](10-shared-api.md), [proxy/middleware-framework](30-proxy-middleware-framework.md), [proxy/middleware-builtin](31-proxy-middleware-builtin.md), [proxy/llm-parsers](32-proxy-llm-parsers.md)
- End-to-end flow: [../01-end-to-end-flows.md](../01-end-to-end-flows.md)
- Top-level: [../00-overview.md](../00-overview.md)

234
docs/agent-networks/modules/40-dashboard.md Normal file
View File

@@ -0,0 +1,234 @@
# dashboard — UI for agent-networks
> **Reviewer profile:** Dashboard maintainer; familiar with the Next.js app-router under `src/app/(dashboard)`, the existing Tabs / Modal / DataTable components, the Providers / Permissions / Groups context pattern, and the React-Flow control-center graph. Light familiarity with the agent-network REST routes is enough.
> **Time to review:** 90 minutes (broad surface; many small components, a few large ones)
> **Risk level:** Medium. New surface is isolated under `src/modules/agent-network/` and `src/app/(dashboard)/agent-network/`, but the PR also reshapes the sidebar, splits `/peers`, renames `reverse-proxy/clusters` → `self-hosted-proxies`, and overlays the Control Center graph. Regressions here would be cross-cutting.
> **Backward-compat impact:** Additive on the API side. Breaking on URL/navigation: `/peers` redirects to `/peers/devices` (src/app/(dashboard)/peers/page.tsx:7-15), `/reverse-proxy/clusters` was renamed to `/reverse-proxy/self-hosted-proxies`, the sidebar lost Access Control / Networks / Reverse Proxy / DNS / standalone Guardrails / Consumption / Activity (Navigation.tsx:165-171 — routes still resolve via URL), and the standalone `/agent-network/{access-log,consumption,global-controls}` routes are gone in favor of `/agent-network/observability` (3fbe44e).
## Module boundary
The dashboard is the only place an operator interacts with agent-networks: provider catalog, configured providers, policies, guardrails, account-level budget rules, account settings (collection / redaction toggles), per-request access log, and consumption rollups all render, paginate, and edit here. Data flows in via SWR (`useFetchApi`) keyed by REST URL. One big context provider (`src/modules/agent-network/AIProvidersProvider.tsx`) aggregates five resources (providers, policies, guardrails, budget rules, settings) plus the proxy access-log stream filtered to `agent_network=true`, and exposes `add* / update* / toggle* / delete*` mutators that call through `useApiCall` and re-`mutate()` SWR. Pages mount the provider once at the top and compose presentational tables and modals beneath. The control-center page additionally fetches `/agent-network/{providers,policies}` directly (control-center/page.tsx:123-130) to overlay graph nodes.
## Commits in scope (newest first)
| SHA | Subject | LOC delta |
| --- | ------- | --------- |
| 3fbe44e | [agent-network] Merge Global Controls + Access Log into AI Observability | +160 / −207 |
| 1988e92 | Global Controls page (account toggles + budget rules) | +1453 / −1 |
| 95b4daa | control center: open agent-policy modal on node click; hide rows-per-page | +51 / −3 |
| b0c4f3e | peers table: compact toolbar + user filter | (peers + UserFilterSelector) |
| a61581b | control center + peers: agent-network overlay + width trims | (control-center + peers) |
| e159d6b / d2ea1fb | control center: drop All Networks dropdown; hide Networks tab | (FlowSelector + page) |
| 7bf59b8 | rename peer sub-views to User Devices and Agents & Servers | (peers + PeersListView) |
| a65a956 | repackage dashboard nav for agent-network focus | (Navigation.tsx) |
| e7bdc5e / 672195a | reverse-proxy: Private toggle + cluster target; drop body-capture knobs | (reverse-proxy) |
| f0b5214 / 4be7d3c / 2875ade / eb54136 / 924537f / c446eec / e042ca3 | Provider modal: OpenRouter, Vercel, Cloudflare, Bifrost (×2), Portkey, LiteLLM Mappings | (AIProviderModal) |
| 1f24392 | window_hours → window_seconds; minute picker on Limits | (PolicyLimitsTab) |
| 2157fec / 59564fd | Consumption page: charts + filters; basic counter view | (ConsumptionPanel + Table) |
| 1af6754 | resolve providers by config-row ID in access log | (AccessLogTable) |
| 6b91b7a | account-level endpoint surface on providers page | (providers/page.tsx) |
| 51221e8 / 6a26ccb | wire access log to /events/proxy?agent_network=true; resolve group IDs | (provider context) |
| b50a689 | Drop Budgets & Alerts; rebuild Access Log in Proxy Events style | (rewrite) |
| 2b7f746 / 5e10a2b / 4d3ac8d / 77a7070 | Wire guardrails / policies / providers / catalog to backend | (provider context) |
| 4cf718f | Add Agent Network dashboard preview | +6036 LOC (initial scaffold) |
Total branch delta vs `main`: ~ +10.9k / −890 LOC across 82 files.
## Surface added
### New pages
| Route | Purpose | Backing module(s) |
| ----- | ------- | ----------------- |
| `/agent-network` | Redirect to `/agent-network/providers` | page.tsx:7-15 |
| `/agent-network/providers` | List + connect providers; header surfaces per-account base URL | providers/page.tsx + AgentProvidersTable + AIProviderModal |
| `/agent-network/policies` | Group → Provider authorization with per-policy Limits + Guardrail attach | policies/page.tsx + AgentPoliciesTable + AgentPolicyModal |
| `/agent-network/guardrails` | Reusable guardrail sets (model allowlist + prompt capture) | guardrails/page.tsx + AgentGuardrailsTable + AgentGuardrailModal |
| `/agent-network/observability` | NEW (3fbe44e). Tabs: Access Logs / Budget Dashboard / Budget Settings / Log Settings | observability/page.tsx |
| `/peers/devices`, `/peers/agents` | Split of `/peers`, shared via `PeersListView` keyed by `kind` | peers/{devices,agents}/page.tsx |
| `/reverse-proxy/self-hosted-proxies` | Renamed from `clusters` | self-hosted-proxies/page.tsx |
Deleted in 3fbe44e: `/agent-network/access-log`, `/agent-network/consumption`, `/agent-network/global-controls`.
### New modules under src/modules/agent-network
| File | Role |
| ---- | ---- |
| AIProvidersProvider.tsx (~1158 LOC) | Aggregates every agent-network resource via SWR; normalises snake↔camel; exposes mutators; holds wizard-open state |
| AIProviderModal.tsx (~1268 LOC) | Connect / edit provider wizard with per-vendor copy (Bifrost, Portkey, LiteLLM, Cloudflare, Vercel, OpenRouter, custom) |
| AIProviderLogo + useProviderCatalog | Catalog-driven brand swatch + SWR hook over `/agent-network/catalog/providers` |
| AgentPoliciesTable + AgentPolicyModal + AgentPolicyGuardrailsTab + AgentPolicyLimitsTab | Policies; modal has 3 tabs (Rule, Limits, Guardrails) |
| AgentGuardrailsTable + AgentGuardrailModal + AgentGuardrailBrowseModal + AgentGuardrailChecksCell | Guardrails CRUD + attach-from-policy |
| AgentBudgetRulesTable + AgentBudgetRuleModal | Account-level budget rules; modal reuses AgentPolicyLimitsTab verbatim |
| AgentAccountControlsCard | Three account-wide toggles (Log Collection / Prompt Collection / Redact PII) |
| AgentAccessLogTable + AgentAccessLogExpandedRow | Access log on `/events/proxy?agent_network=true` |
| AgentConsumptionPanel + AgentConsumptionTable | Token + cost panel: charts + counter table |
| table/AgentProvidersTable + AgentProviderActionCell | Providers table + per-row actions |
| data/mockData.ts | Domain types and a few residual `MOCK_*` constants (see scrutinize) |
### Touched non-agent-network areas
- **control-center**: agent-network overlay (provider + agent-policy nodes); removed All Networks dropdown (a61581b, e159d6b); hid Networks tab in FlowSelector (FlowSelector.tsx:9-14 — enum value kept so `?tab=networks` still type-checks); wrapped `ControlCenterView` in `AIProvidersProvider` (page.tsx:73-83); `agentPolicyNode` clicks routed to a separate state slot (page.tsx:1871-1874). New node renderers: nodes/ProviderNode.tsx, nodes/AgentPolicyNode.tsx (registered at utils/nodes.ts:21-22).
- **peers**: Split into Devices and Agents sub-routes; shared via `PeersListView` keyed by `kind` (PeersListView.tsx:24-95). New compact-toolbar `UserFilterSelector` (users/UserFilterSelector.tsx).
- **reverse-proxy**: Folder rename `clusters/` → `self-hosted-proxies/`; deleted `ClustersFeaturesCell.tsx`, `ClusterTypeIndicator.tsx`; new ReverseProxyClusterTargetSelector for cluster target type (e7bdc5e); Private toggle on target modal; body-capture knobs removed (672195a); new ReverseProxyEventExpandedRow.
- **events**: `ReverseProxyEventsUserCell` rewritten with user + peer fallback (ReverseProxyEventsUserCell.tsx:14-21), shared with the access-log table.
- **navigation**: Full repackaging in Navigation.tsx — Agent Network items flattened (no collapsible parent), distinct icons per item; Access Control, Networks, Reverse Proxy, DNS, standalone Guardrails, Consumption, Activity removed (still URL-reachable, per lines 165-171).
## Architecture & flow
### Page → Provider → Table/Modal hierarchy
```mermaid
graph TD
Nav[Navigation.tsx]
Nav --> ProvidersPage[/agent-network/providers/]
Nav --> PoliciesPage[/agent-network/policies/]
Nav --> GuardrailsPage[/agent-network/guardrails/]
Nav --> ObsPage[/agent-network/observability NEW/]
ProvidersPage --> AIPP1[AIProvidersProvider]
PoliciesPage --> AIPP2[AIProvidersProvider]
GuardrailsPage --> AIPP3[AIProvidersProvider]
ObsPage --> AIPP4[AIProvidersProvider]
ObsPage -.wraps.-> GroupsProvider
ObsPage -.wraps.-> PeersProvider
AIPP1 --> ProvTable[AgentProvidersTable]
ProvTable --> ProvModal[AIProviderModal]
AIPP2 --> PolTable[AgentPoliciesTable]
PolTable --> PolModal[AgentPolicyModal]
PolModal --> PolGuardTab[AgentPolicyGuardrailsTab]
PolModal --> PolLimitsTab[AgentPolicyLimitsTab]
PolGuardTab --> GuardBrowse[AgentGuardrailBrowseModal]
PolGuardTab --> GuardModal[AgentGuardrailModal]
AIPP3 --> GuardTable[AgentGuardrailsTable]
GuardTable --> GuardModal
AIPP4 --> Tabs[Tabs]
Tabs --> AccessLog[AgentAccessLogTable]
Tabs --> Consumption[AgentConsumptionPanel]
Tabs --> BudgetRules[AgentBudgetRulesTable]
Tabs --> AccountCtl[AgentAccountControlsCard]
BudgetRules --> BudgetModal[AgentBudgetRuleModal]
BudgetModal -.reuses.-> PolLimitsTab
```
### AI Observability tab page (today's commit)
```mermaid
graph LR
Page[AIObservabilityPage] --> RA[RestrictedAccess<br/>permission.services.read]
RA --> GP[GroupsProvider]
GP --> PP[PeersProvider]
PP --> AIP[AIProvidersProvider]
AIP --> Tabs[Tabs / TabsList]
Tabs --> T1[Access Logs<br/>AgentAccessLogTable]
Tabs --> T2[Budget Dashboard<br/>AgentConsumptionPanel]
Tabs --> T3[Budget Settings<br/>AgentBudgetRulesTable]
Tabs --> T4[Log Settings<br/>AgentAccountControlsCard]
T1 -.GET.-> EP[/events/proxy?agent_network=true/]
T2 -.GET poll 5s.-> CONS[/agent-network/consumption/]
T3 -.GET/PUT.-> BR[/agent-network/budget-rules/]
T4 -.GET/PUT.-> ST[/agent-network/settings/]
```
### Data fetch path
```mermaid
graph TD
Page[Page component] --> Prov[AIProvidersProvider]
Prov -->|useFetchApi| SWR[(SWR cache<br/>key = URL)]
SWR -.GET.-> P[/agent-network/providers/]
SWR -.GET.-> POL[/agent-network/policies/]
SWR -.GET.-> G[/agent-network/guardrails/]
SWR -.GET.-> BR[/agent-network/budget-rules/]
SWR -.GET ignoreError.-> ST[/agent-network/settings/]
SWR -.GET.-> CAT[/agent-network/catalog/providers/]
SWR -.GET pageSize=100.-> EVT[/events/proxy agent_network=true/]
Prov --> Mut[useApiCall.post/put/del]
Mut -.on success.-> MutateSWR[SWR mutate keys]
Prov --> Children[Tables / Modals via useAIProviders]
```
Every list view reaches management through SWR over `/api/agent-network/*`. The provider context maps snake-case payloads to camelCase domain types (`fromAPI`, `policyFromAPI`, `guardrailFromAPI`, `budgetRuleFromAPI`, `settingsFromAPI`, `accessLogFromAPI` — AIProvidersProvider.tsx:138-562) and back via matching `*ToRequest` adaptors. The access log piggy-backs on `/events/proxy` with `agent_network=true&page_size=100` (line 707-709) and decodes LLM-specific fields from per-event `metadata`. Group IDs on events are resolved to current names through the surrounding GroupsProvider catalog (lines 515-521, 717-731) — no extra round trip. Mutators run `*ToRequest`, await `useApiCall.post/put/del`, call SWR `mutate()`, then `notify`. Errors caught and surfaced via `notify` — no exceptions escape into render. The Connect Provider modal's open state lives in the provider itself (`isWizardOpen` at lines 732-735) so the providers-page empty-state CTA and the table's + button share one modal. Control-center re-fetches `/agent-network/{providers,policies}` directly on top of `AIProvidersProvider` — SWR de-dupes but the code path is harder to reason about.
## Public contracts consumed
- `GET/POST /api/agent-network/providers`, `PUT/DELETE /:id`
- `GET/POST /api/agent-network/policies`, `PUT/DELETE /:id`
- `GET/POST /api/agent-network/guardrails`, `PUT/DELETE /:id`
- `GET/POST /api/agent-network/budget-rules`, `PUT/DELETE /:id`
- `GET/PUT /api/agent-network/settings` (ignoreError-tolerant; 404 = not yet bootstrapped — auto-bootstrap on first provider create via `bootstrap_cluster` field — AIProvidersProvider.tsx:737-760)
- `GET /api/agent-network/catalog/providers` (read-only declarative; backend owns vendor list, IDs, brand colors, models, extra_headers, identity_injection — useProviderCatalog.ts:6-95)
- `GET /api/agent-network/consumption` (polled every 5s on Budget Dashboard — ConsumptionPanel.tsx:53,65-71)
- `GET /api/events/proxy?agent_network=true&page_size=100` (shared with Proxy Events)
- `permission?.services?.read` gates every agent-network route via RestrictedAccess.
`AIProviderId` is a closed union in dashboard types (data/mockData.ts:8-21) but the converter tolerates anything the backend ships — unknown ids fall through to `"custom"` (AIProvidersProvider.tsx:497-506). Catalog values are pure read-through: anything declared in `extra_headers` renders in the modal automatically, copy keyed by header name (`EXTRA_HEADER_UI` in AIProviderModal.tsx:61-89), labeled-fallback for unknown ones.
## Invariants
- Provider context wrap order on user-attribution pages: `GroupsProvider > PeersProvider > AIProvidersProvider` (observability/page.tsx:87-89). Reverse it and access-log group resolution silently drops names.
- Every agent-network route checks `permission?.services?.read` via `RestrictedAccess` (observability/page.tsx:85, providers/page.tsx:184, policies/page.tsx:53, guardrails/page.tsx:55).
- Modal `key={open ? 1 : 0}` pattern is used to force unmount/remount on close so internal `useState` resets between edits (AgentBudgetRuleModal.tsx:60, AgentPolicyModal.tsx:66). Removing this would leak prior-row state into a new-row session.
- `mockData.ts` is the canonical home for ALL agent-network domain types; `MOCK_*` constants must never reach a production code path. One leak remains (below).
## Things to scrutinize
### Correctness
- **Tab-state URL hand-off is one-way.** observability/page.tsx:53-58 reads `?tab=` on mount (despite the file comment at line 28 saying URL hand-off is future) but `setTab` does NOT push back, so reload preserves the chosen tab only if it came in via the link. Inconsistent with control-center (page.tsx:1817-1831).
- **Provider overlay runs only in `applySingleGroupView` / `applyPeerView`** (control-center/page.tsx:557, 1159-1166). User view does NOT show providers — if agent-network is a primary lens, that's a gap.
- **Two useEffects race to invalidate the control-center layout.** page.tsx:1655-1657 drops `layoutInitialized` when `agentPolicies` / `agentProviders` arrive; the main effect (1786-1799) also lists them as deps. Works today but fragile — watch for flash-of-empty-graph.
- **`updateProvider` / `updatePolicy` / `updateBudgetRule` use `??` on `enabled`** (AIProvidersProvider.tsx:784, 859, 1018). Toggle paths are safe; any caller sending `enabled: false` thinking "leave it off" gets `existing.enabled` instead. Audit modal callers.
- **Form validation in modals is minimal.** Window-seconds picker (1f24392) — mockData.ts:209-215 documents "minimum 60 — one minute" but I couldn't find a matching UI guard in PolicyLimitsTab.
### Security
- **No client-side enforcement claims** — every cap, allowlist, and toggle is display + edit; proxy is the source of truth for deny decisions (AccessLogTable.tsx:177-191 renders backend-emitted `denyReason` as-is).
- **Prompt display is gated by what the backend stamps.** When `enable_prompt_collection` is OFF the proxy must not put prompt/completion into event metadata; the dashboard renders whatever it gets verbatim (AccessLogTable lines 532-534, AccessLogExpandedRow.tsx:42-57). No UI filter on top of backend collection switches.
- Account Controls disables `Redact PII` when `Prompt Collection` is off (AgentAccountControlsCard.tsx:122) and clears it on off-transition (line 100), but relies on backend to enforce the same gate at write — confirm PUT handler rejects `redact_pii=true && enable_prompt_collection=false`.
- **Bifrost identity-header overrides**: empty-string vs nil semantics documented in AIProvidersProvider.tsx:772-781 ("omitted = preserve, empty = explicit clear"). Mishandling could leak group attribution to a header the operator thought disabled. Focused read of Bifrost code path in AIProviderModal.tsx recommended.
### Accessibility
- Observability TabsList (observability/page.tsx:96-113) uses the shared Tabs component — should inherit Radix roving-tabindex. All four TabsTriggers carry only icon + text, no `aria-label`; fine because text is visible.
- Modal focus traps are inherited from the shared Modal; agent-network modals don't override them. Quick keyboard pass recommended.
- `EndpointBadge` Copy button (providers/page.tsx:66-76) has an `aria-label`, good.
### Performance
- `AgentConsumptionPanel` polls `/agent-network/consumption` every 5s (ConsumptionPanel.tsx:53,70). Tab switches unmount the panel, so the poll stops — verify in network panel.
- `AgentAccessLogTable` is hard-capped at 100 rows via `page_size=100` (AIProvidersProvider.tsx:707-709). Server-side pagination is future work; high-traffic tenants miss everything past row 100 — known limitation.
- Observability page mounts providers ONCE at page level (observability/page.tsx:87-89); tab switches keep SWR cache hot. Moving the provider mount inside `TabsContent` would re-fetch the access log on every switch.
### Visual consistency
- Observability tab style claims to match peer/page.tsx (3fbe44e commit msg). Outer Tabs `pt-4 pb-0 mb-0`, TabsList `px-8` (observability/page.tsx:94-96) — confirm chrome height matches so the page doesn't visually jump.
- Sidebar: `Boxes` for Providers, `AccessControlIcon` for Policies, `TelescopeIcon` for AI Observability (Navigation.tsx:113,120,133). Reusing `AccessControlIcon` makes Policies look identical to the (now hidden) Access Control item — if Access Control ever comes back, they collide.
- `AgentNetworkIcon` is used in breadcrumbs on every agent-network page but NOT in the sidebar (per-page icons instead). Deliberate departure — record so it doesn't get reverted.
## Test coverage
- **Cypress**: One file (`cypress/e2e/test.cy.ts`) covering only the install-page copy-to-clipboard flow. NOTHING covers agent-network UI.
- **Component / unit tests**: `src/utils/version.test.ts` is the only `.test.*` file in the repo. The agent-network modules ship without component tests.
- Data-cy hooks exist on key controls: `save-account-controls` (AgentAccountControlsCard.tsx:71), `enable-log-collection`, `enable-prompt-collection`, `redact-pii`, plus existing `data-cy={policy.name}` / `data-cy={provider.name}` on ActiveInactiveRow. Sufficient hooks for Cypress flows; none written yet.
- **Tooling gap (not introduced by this PR):** `npm run lint` (`next lint`) is broken in Next 16 — the `lint` subcommand was removed from the Next CLI in 16.x. Same script lives on `main`, but the merge means the dashboard effectively ships without a working lint gate. Worth flagging now; add either a flat-config `eslint .` script or wire ESLint via an explicit `eslint-config-next` invocation.
## Known limitations / explicit non-goals
- **`data/mockData.ts` still contains `MOCK_GROUPS`, `MOCK_PROVIDERS`, `MOCK_PEERS`.** Only `MOCK_GROUPS` is referenced from production — AgentPoliciesTable.tsx:45,76 uses it as a name-lookup fallback when a policy references a group ID the real GroupsProvider doesn't know about. `MOCK_PROVIDERS` / `MOCK_PEERS` are unreferenced; safe to delete. The file is `/* eslint-disable */` so dead-code warnings don't flag them.
- **Tab-state URL hand-off on observability page is one-way** (read-only).
- **Access log hard-capped at 100 rows**; no server-side pagination.
- **No optimistic updates.** All mutations are round-trip; failures rollback via SWR revalidation.
- **`FlowView.NETWORKS` retained but hidden** from FlowSelector (FlowSelector.tsx:9-14). Old `?tab=networks` links still route to the hidden view because `applyNetworksView` still runs.
- **Redirects are not query-preserving** — `router.replace("/peers/devices")` (peers/page.tsx:13) strips any incoming filter params.
- **Control-center cross-fetches** `/agent-network/{providers,policies}` directly on top of `AIProvidersProvider`. Could be collapsed.
- **Sidebar permanently hides Access Control, Networks, Reverse Proxy, standalone Guardrails, DNS, Activity, Consumption.** Routes still resolve via URL (Navigation.tsx:165-171); intentional per a65a956.
## Cross-references
- Upstream API contracts: [shared/api](10-shared-api.md)
- Backend persistence: [management/store](20-management-store.md)
- Backend handler wiring: [management/handlers](22-management-handlers-wiring.md) (planned)
- End-to-end flow narrative: [../01-end-to-end-flows.md](../01-end-to-end-flows.md)
- Top-level overview: [../00-overview.md](../00-overview.md)

251
docs/agent-networks/modules/50-path-routed-providers.md Normal file
View File

@@ -0,0 +1,251 @@
# path-routed providers — Vertex AI + Bedrock
> Reviewer profile: proxy / LLM + management catalog. You should be comfortable
> with the `llm_router` / `llm_request_parser` middlewares
> ([31-proxy-middleware-builtin.md](31-proxy-middleware-builtin.md)), the
> per-provider parser surface ([32-proxy-llm-parsers.md](32-proxy-llm-parsers.md)),
> and the synthesiser's catalog → `ProviderRoute` mapping
> ([21-management-agentnetwork.md](21-management-agentnetwork.md)). This guide
> pulls the **path-routed** provider story together in one place because it
> crosses the catalog, the synthesiser, the request parser, and the router.
Sibling modules: [31-proxy-middleware-builtin.md](31-proxy-middleware-builtin.md)
(router + request parser) and [32-proxy-llm-parsers.md](32-proxy-llm-parsers.md)
(Bedrock parser + pricing).
---
## What "path-routed" means
Most catalog providers carry the model in the request **body** (`{"model": …}`),
so `llm_router` selects an upstream by matching the model name against each
provider's `Models` claim. Two providers instead carry the model in the **URL
path**, so they are routed by path before the model/vendor table is consulted:
| Catalog id | Style flag | Request path shape |
|---|---|---|
| `vertex_ai_api` | `IsVertexPathStyle` → `ProviderRoute.Vertex` | `/v1/projects/{project}/locations/{region}/publishers/{publisher}/models/{model}:{action}` |
| `bedrock_api` | `IsBedrockPathStyle` → `ProviderRoute.Bedrock` | `/model/{modelId}/{action}` (optionally behind `/bedrock`) |
The catalog declares the style with
[`catalog.IsVertexPathStyle` / `catalog.IsBedrockPathStyle`](../../../management/server/agentnetwork/catalog/catalog.go)
and the synthesiser copies the result onto the router route as the `Vertex` /
`Bedrock` booleans
([synthesizer.go:450-451](../../../management/server/agentnetwork/synthesizer.go)).
On the request leg `llm_router.Invoke` dispatches `isVertexPath` / `isBedrockPath`
**before** the model lookup
([llm_router/middleware.go:138-216](../../../proxy/internal/middleware/builtin/llm_router/middleware.go))
so a model the parser extracted from the path can't be claimed by a same-vendor
*body-routed* provider (e.g. `claude-*` on `api.anthropic.com`).
## Google Vertex AI (`vertex_ai_api`)
### Catalog entry
`KindProvider`, parser surface left unset on the catalog entry — the request
parser picks the parser from the URL **publisher** segment, not from
`ParserID`. Upstream host is `<region>-aiplatform.googleapis.com`
(`https://aiplatform.googleapis.com` for the `global` location). The catalog
lists the Claude-on-Vertex lineup (`claude-opus-4-*`, `claude-sonnet-4-*`,
`claude-haiku-4-5`, `claude-fable-5`) at the same per-token rates as the
first-party Anthropic entry
([catalog.go:333-363](../../../management/server/agentnetwork/catalog/catalog.go)).
### Credential — service-account OAuth (`keyfile::`)
Vertex does **not** accept a static API key. The operator sets the provider
`api_key` to:
```
keyfile::<base64 of the GCP service-account JSON key>
```
The synthesiser recognises the `keyfile::` prefix in `providerAuthHeader`
([synthesizer.go:897-903](../../../management/server/agentnetwork/synthesizer.go)),
emits **no** static auth value, and carries the base64 key material on the
route as `GCPServiceAccountKeyB64`
([factory.go:56-61](../../../proxy/internal/middleware/builtin/llm_router/factory.go)).
At request time the router mints a short-lived OAuth2 access token from the key
(cloud-platform scope) and injects `Authorization: Bearer <access-token>` —
never the key itself
([llm_router/middleware.go:621-692](../../../proxy/internal/middleware/builtin/llm_router/middleware.go)):
- One auto-refreshing `oauth2.TokenSource` is cached per key (keyed by a
SHA-256 of the base64 material), so token minting happens once and refreshes
amortise across requests.
- Mint / refresh is bounded by a 10s timeout HTTP client (`gcpTokenTimeout`) so
a slow Google token endpoint can't hang the request.
- A malformed key or an unreachable token endpoint fails the request with
`llm_policy.upstream_auth_failed` at HTTP **502** (an upstream problem, not a
policy denial) — see `denyUpstreamAuth`.
### Metering — Anthropic-on-Vertex only
The request parser extracts `{publisher, model, action}` from the path
(`parseVertexPath`, [llm_request_parser/middleware.go:237-263](../../../proxy/internal/middleware/builtin/llm_request_parser/middleware.go)),
strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic` → `llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
- `openai` → `llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
**Gemini is intentionally denied as unmeterable.** When the parser emits no
`llm.provider` for a Vertex publisher, `llm_router` returns
`llm_policy.unmeterable_publisher` (403) rather than forwarding the request
uncounted — serving it would bypass token / budget metering
([llm_router/middleware.go:144-162, 712-728](../../../proxy/internal/middleware/builtin/llm_router/middleware.go)).
A Gemini parser would lift this restriction; until then the `google` publisher
is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
## AWS Bedrock (`bedrock_api`)
### Catalog entry
`KindProvider`, upstream host `bedrock-runtime.<region>.amazonaws.com`. Metered
models are the Anthropic-on-Bedrock lineup (`anthropic.claude-*`) plus Amazon
Nova and Llama 3.3 entries
([catalog.go:300-332](../../../management/server/agentnetwork/catalog/catalog.go)).
Anthropic-on-Bedrock reuses the first-party Claude prices (with additive cache
buckets); Nova / Llama report no cache, so cost is `input + output`.
### Credential — static bearer token
Bedrock uses the **AWS Bedrock API key** as a static bearer. The operator sets
the provider `api_key` directly (no `keyfile::` prefix); the catalog template
is `Authorization: Bearer ${API_KEY}`
([catalog.go:306-307](../../../management/server/agentnetwork/catalog/catalog.go)).
No token minting — the synthesiser substitutes the key into the template and
the router injects the resulting `Authorization` header after stripping inbound
vendor auth (including client-supplied AWS SigV4 material: `X-Amz-Date`,
`X-Amz-Security-Token`, `X-Amz-Content-Sha256`, see `strippedAuthHeaders`).
### Model id form — cross-region inference profiles
Bedrock model ids in the request path must be the cross-region
**inference-profile** form, e.g.
`eu.anthropic.claude-sonnet-4-5-20250929-v1:0`. The bare
`anthropic.claude-…` id is rejected by AWS. `normalizeBedrockModel`
([llm_request_parser/middleware.go:398-414](../../../proxy/internal/middleware/builtin/llm_request_parser/middleware.go))
strips the region prefix (`us.` / `eu.` / `apac.` / `global.`), an optional ARN
wrapper, and the `-YYYYMMDD-vN[:N]` version/throughput suffix so the normalised
id (`anthropic.claude-sonnet-4-5`) matches the catalog/pricing key.
### Supported endpoints + actions
`/model/{modelId}/{action}` where action ∈ `invoke`,
`invoke-with-response-stream`, `converse`, `converse-stream`
([llm_request_parser/middleware.go:363-390](../../../proxy/internal/middleware/builtin/llm_request_parser/middleware.go)).
`invoke` / `converse` are non-streaming; the `-stream` actions set the streaming
flag.
- **InvokeModel** body uses the vendor-native shape — for Anthropic that means
`"anthropic_version":"bedrock-2023-05-31"` and snake_case usage with additive
cache buckets.
- **Converse** uses the unified camelCase shape with a precomputed `totalTokens`.
- The `BedrockParser` reads both shapes on the response leg
([bedrock.go](../../../proxy/internal/llm/bedrock.go)); the request parser
doesn't need to distinguish them (`ParseRequest` is a no-op — model + stream
come from the path).
### Streaming — AWS binary event-stream
The `-stream` actions return `application/vnd.amazon.eventstream` (the AWS
binary event-stream framing), and streaming **is metered**.
`accumulateBedrockStream`
([llm_response_parser/streaming_bedrock.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming_bedrock.go))
decodes the frames with `aws-sdk-go-v2/aws/protocol/eventstream`:
- InvokeModel `chunk` frames wrap a base64 `{"bytes":…}` payload carrying a
vendor-native (Anthropic) stream event — folded through the shared Anthropic
stream accumulator.
- Converse `contentBlockDelta` frames carry text; the trailing `metadata` frame
carries the final usage block.
- A truncated stream (cut at the body-tap capture cap) decodes best-effort:
frames up to the cut are applied and partial usage is returned.
### Optional `/bedrock` gateway-namespace prefix
Clients may place an optional `/bedrock` prefix before the native path
(`/bedrock/model/{modelId}/{action}`) to disambiguate Bedrock from other
providers that also use `/model/...`. Both the request parser
(`trimBedrockNamespace`) and the router (`splitBedrockNamespace`) accept it.
When the prefix is present, the router sets
`RewriteUpstream.StripPathPrefix = "/bedrock"` so the **native** path
(`/model/...`) is what reaches `bedrock-runtime.<region>.amazonaws.com`
([llm_router/middleware.go:168-184, 320-348](../../../proxy/internal/middleware/builtin/llm_router/middleware.go)).
## Model allowlist on path-routed providers
Because the model lives in the URL rather than the body, a path-routed provider
credential could otherwise be used for any model the upstream supports. The
router still enforces the route's `Models` allowlist via `matchPathRoute`
([llm_router/middleware.go:370-416](../../../proxy/internal/middleware/builtin/llm_router/middleware.go)):
1. Filter to routes of the matching style (`Vertex` / `Bedrock`).
2. Filter to routes whose `AllowedGroupIDs` authorise the caller's groups
(else `no_authorised_provider`).
3. Filter to routes that **claim the requested model**. As with body-routed
providers, an **empty `Models` list = catch-all** (serve any model);
a non-empty list serves only the listed models (else `model_not_routable`).
4. Multiple survivors disambiguate by longest `UpstreamPath` prefix match.
So an operator who lists explicit models on a Vertex/Bedrock provider gets a
hard allowlist; an operator who leaves `Models` empty accepts every model the
upstream serves (still subject to the unmeterable-publisher gate on Vertex).
Model-less OpenAI endpoints (`GET /v1/models`) are **never** routed to a
Vertex/Bedrock provider — `matchModelless` skips path-routed routes
([llm_router/middleware.go:427-462](../../../proxy/internal/middleware/builtin/llm_router/middleware.go))
so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
## Things to scrutinise
**Security.** The Vertex service-account key is never forwarded — only a minted
short-lived bearer. Confirm the key material stays out of access logs (it lives
on `ProviderRoute.GCPServiceAccountKeyB64`, not in any emitted metadata key).
The unmeterable-publisher deny is the only thing standing between an
operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
`vertexPublisherVendor` stays conservative (deny by default for unknown
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
## Cross-references
- Router + request-parser detail: [31-proxy-middleware-builtin.md](31-proxy-middleware-builtin.md)
- Bedrock parser + pricing + SSE / event-stream: [32-proxy-llm-parsers.md](32-proxy-llm-parsers.md)
- Catalog → route synthesis + `keyfile::` handling: [21-management-agentnetwork.md](21-management-agentnetwork.md)
- Overview: [../00-overview.md](../00-overview.md)

55
infrastructure_files/getting-started.sh
View File

@@ -398,7 +398,42 @@ configure_domain() {
return 0
}
apply_agent_network_preset() {
# Agent-network turnkey install: built-in Traefik + NetBird Proxy with
# NB_PROXY_PRIVATE=true, dashboard locked to agent-network-only mode.
# Bypasses every reverse-proxy / proxy / CrowdSec prompt. The only
# inputs we still need from the operator are the domain (handled by
# configure_domain via NETBIRD_DOMAIN env var or interactive prompt)
# and the ACME email — both honor env vars first and fall back to a
# prompt only when unset. CrowdSec is intentionally off.
REVERSE_PROXY_TYPE="0"
ENABLE_PROXY="true"
ENABLE_CROWDSEC="false"
if [[ -n "${NETBIRD_LETSENCRYPT_EMAIL}" ]]; then
TRAEFIK_ACME_EMAIL="${NETBIRD_LETSENCRYPT_EMAIL}"
else
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
fi
echo "" > /dev/stderr
echo "Agent-network preset enabled (NETBIRD_AGENT_NETWORK=true):" > /dev/stderr
echo " - reverse proxy: built-in Traefik" > /dev/stderr
echo " - NetBird Proxy: enabled with NB_PROXY_PRIVATE=true" > /dev/stderr
echo " - dashboard: NETBIRD_AGENT_NETWORK_ONLY=true" > /dev/stderr
echo " - CrowdSec: disabled" > /dev/stderr
echo " - Let's Encrypt email: ${TRAEFIK_ACME_EMAIL}" > /dev/stderr
echo "" > /dev/stderr
}
configure_reverse_proxy() {
# Short-circuit: agent-network preset locks every reverse-proxy /
# proxy / CrowdSec choice and bypasses the interactive prompts.
if [[ "${NETBIRD_AGENT_NETWORK}" == "true" ]]; then
apply_agent_network_preset
return 0
fi
# Prompt for reverse proxy type
REVERSE_PROXY_TYPE=$(read_reverse_proxy_type)
@@ -910,6 +945,15 @@ NGINX_SSL_PORT=443
# Letsencrypt
LETSENCRYPT_DOMAIN=none
EOF
if [[ "${NETBIRD_AGENT_NETWORK}" == "true" ]]; then
cat <<EOF
# Agent-network preset: dashboard hides the standard NetBird surfaces
# and exposes only the AI Observability + agent-network configuration
# pages. Paired with NB_PROXY_PRIVATE=true on the proxy side.
NETBIRD_AGENT_NETWORK_ONLY=true
EOF
fi
return 0
}
@@ -946,6 +990,17 @@ NB_PROXY_PROXY_PROTOCOL=true
NB_PROXY_TRUSTED_PROXIES=$TRAEFIK_IP
EOF
if [[ "${NETBIRD_AGENT_NETWORK}" == "true" ]]; then
cat <<EOF
# Agent-network preset: turn the proxy into the private reverse-proxy
# ingress for agent-network synth services. Disables the public-facing
# surface so the proxy serves only synth-generated routes (the
# llm_router-driven LLM endpoints) and the per-account inbound
# listeners on the embedded netstack.
NB_PROXY_PRIVATE=true
EOF
fi
if [[ "$ENABLE_CROWDSEC" == "true" && -n "$CROWDSEC_BOUNCER_KEY" ]]; then
cat <<EOF
NB_PROXY_CROWDSEC_API_URL=http://crowdsec:8080

956
scripts/e2e/agent-network-full/e2e.sh Normal file
View File

@@ -0,0 +1,956 @@
#!/usr/bin/env bash
# Agent-network full end-to-end driver.
#
# One script, many subcommands, so the operator authorizes a single
# `bash scripts/e2e/agent-network-full/e2e.sh <cmd>` invocation instead of a
# stream of ad-hoc docker/netbird/curl commands.
#
# It joins a Docker NetBird client to the local Tilt management, drives LLM
# chat-completions through the agent-network proxy over the tunnel, and asserts
# token/cost/session capture via the REST API and proxy logs.
#
# Secrets (PAT, provider keys, setup key) are read from files / a sourced
# key file and never echoed.
#
# Usage:
# bash scripts/e2e/agent-network-full/e2e.sh <command>
#
# Commands:
# key mint a reusable setup key bound to the Admins group
# up run the Docker NetBird client and join management
# status show netbird status -d inside the client
# wait wait until the proxy peer is Connected (1/1)
# diag dump client + proxy3 + relay diagnostics
# chat M P chat-completion through the proxy: model M, provider-kind P
# verify consumption + recent access-log rows
# snapshot save current agent-network config to the scratch dir
# clean delete all policies, budget-rules, providers (API teardown)
# providers create the five providers from .llm-keys
# policy create one policy: Admins -> all five providers
# down remove the Docker client container
# restart-proxy tilt trigger proxy3 (re-establish its relay link)
# all clean -> providers -> policy -> up -> wait -> chat-all -> verify
set -uo pipefail
# --- config ------------------------------------------------------------------
NB_API="${NB_API:-http://localhost:8080}"
NB_PAT_FILE="${NB_PAT_FILE:-/Users/maycon/projects/local-dev/nb-pat}"
LLM_KEYS_FILE="${LLM_KEYS_FILE:-/Users/maycon/.llm-keys}"
CLIENT="${CLIENT:-nb-e2e-agent}"
IMAGE="${IMAGE:-netbird:tilt}"
CURL_IMAGE="${CURL_IMAGE:-curlimages/curl:latest}"
MGMT_URL="${MGMT_URL:-http://host.docker.internal:8080}"
PROXY_HOST="${PROXY_HOST:-mitten.proxy.netbird.local}"
ADMINS_GROUP_NAME="${ADMINS_GROUP_NAME:-Admins}"
DASH_DIR="${DASH_DIR:-/Users/maycon/projects/dashboard}"
DASH_URL="${DASH_URL:-http://localhost:3000}"
DASH_USER="${DASH_USER:-netbird@netbird.io}"
DASH_PASS="${DASH_PASS:-netbird@netbird.io}"
STATE_DIR="${STATE_DIR:-/private/tmp/claude-501/-Users-maycon-projects-netbird/a3fe30e4-5777-47d5-b110-ebc228716026/scratchpad/agentnet-snapshot}"
mkdir -p "$STATE_DIR"
[ -r "$NB_PAT_FILE" ] || { echo "FAIL: cannot read PAT at $NB_PAT_FILE" >&2; exit 2; }
PAT="$(tr -d '\n\r ' <"$NB_PAT_FILE")"
AUTH="Authorization: Token $PAT"
B="$NB_API/api/agent-network"
# --- small helpers -----------------------------------------------------------
log() { printf '%s\n' "$*" >&2; }
die() { printf 'FAIL: %s\n' "$*" >&2; exit 1; }
api() { # METHOD PATH [BODY]
local m="$1" p="$2" body="${3-}"
if [ -n "$body" ]; then
curl -fsS -X "$m" -H "$AUTH" -H "Content-Type: application/json" --data "$body" "$NB_API$p"
else
curl -fsS -X "$m" -H "$AUTH" "$NB_API$p"
fi
}
# proxy_ip resolves the proxy host to its NetBird IP from inside the client.
proxy_ip() {
docker exec "$CLIENT" sh -c "getent hosts $PROXY_HOST" 2>/dev/null | awk '{print $1; exit}'
}
admins_group_id() {
api GET /api/groups | jq -r --arg n "$ADMINS_GROUP_NAME" \
'.[] | select((.name//"")|ascii_downcase==($n|ascii_downcase)) | .id' | head -n1
}
require_llm_keys() {
[ -r "$LLM_KEYS_FILE" ] || die "cannot read $LLM_KEYS_FILE"
# shellcheck disable=SC1090
set -a; . "$LLM_KEYS_FILE"; set +a
}
# --- commands ----------------------------------------------------------------
cmd_snapshot() {
api GET /settings_unused 2>/dev/null || true
curl -fsS -H "$AUTH" "$B/settings" >"$STATE_DIR/settings.json"
curl -fsS -H "$AUTH" "$B/providers" >"$STATE_DIR/providers.json"
curl -fsS -H "$AUTH" "$B/policies" >"$STATE_DIR/policies.json"
curl -fsS -H "$AUTH" "$B/budget-rules" >"$STATE_DIR/budget-rules.json"
curl -fsS -H "$AUTH" "$NB_API/api/groups" >"$STATE_DIR/groups.json"
log "snapshot written to $STATE_DIR"
ls -l "$STATE_DIR" >&2
}
cmd_key() {
local gid kjson
gid="$(admins_group_id)"
[ -n "$gid" ] || die "could not resolve Admins group id"
kjson="$(api POST /api/setup-keys "{\"name\":\"e2e-agentnet-docker\",\"type\":\"reusable\",\"expires_in\":86400,\"usage_limit\":0,\"auto_groups\":[\"$gid\"],\"ephemeral\":false}")"
echo "$kjson" | jq -r '.key' >"$STATE_DIR/setup-key.txt"
chmod 600 "$STATE_DIR/setup-key.txt"
echo "$kjson" | jq '{id,name,type,state,valid,auto_groups}' >&2
log "setup key saved (value not printed) to $STATE_DIR/setup-key.txt"
}
cmd_up() {
[ -r "$STATE_DIR/setup-key.txt" ] || cmd_key
local key; key="$(cat "$STATE_DIR/setup-key.txt")"
docker rm -f "$CLIENT" >/dev/null 2>&1 || true
docker run -d --name "$CLIENT" \
--cap-add NET_ADMIN --cap-add SYS_ADMIN --cap-add SYS_RESOURCE \
--add-host host.docker.internal:host-gateway \
-e NB_MANAGEMENT_URL="$MGMT_URL" \
-e NB_SETUP_KEY="$key" \
-e NB_LOG_LEVEL=info \
"$IMAGE" >/dev/null
log "started $CLIENT"
sleep 6
docker exec "$CLIENT" netbird status 2>&1 | sed -n '1,14p' >&2
}
cmd_status() { docker exec "$CLIENT" netbird status -d 2>&1; }
cmd_wait() {
local i=0 line
while [ "$i" -lt 90 ]; do
line="$(docker exec "$CLIENT" netbird status 2>/dev/null | grep '^Peers count' || true)"
log "t=${i}s ${line:-<no status>}"
case "$line" in *"1/1 Connected"*) log "proxy peer connected"; return 0;; esac
sleep 5; i=$((i+5))
done
log "proxy peer did not connect within ${i}s"
return 1
}
cmd_diag() {
log "===== client: peers count ====="
docker exec "$CLIENT" netbird status 2>&1 | grep -E '^(Management|Signal|Relays|Peers)' >&2 || true
log "===== client: relay/handshake (last 15) ====="
docker exec "$CLIENT" sh -c 'tail -n 400 /var/log/netbird/client.log' 2>/dev/null \
| grep -iE 'relay|handshake|offer|answer|error' | tail -15 >&2 || true
log "===== proxy3: relay (last 15) ====="
docker logs local-dev-proxy3-1 2>&1 | grep -iE 'relay|signal|handshake|offer|answer' | tail -15 >&2 || true
log "===== mgmt: proxy peer online ====="
curl -fsS -H "$AUTH" "$NB_API/api/peers" \
| jq -r '.[] | select((.name//"")|test("^proxy-")) | "\(.name) connected=\(.connected) last_seen=\(.last_seen)"' >&2 || true
}
cmd_restart_proxy() {
command -v tilt >/dev/null 2>&1 || die "tilt not on PATH"
tilt trigger proxy3 && log "triggered proxy3"
}
# cmd_chat MODEL KIND — KIND in: chat (POST /v1/chat/completions),
# messages (POST /v1/messages, anthropic body).
CHAT_RESP="$STATE_DIR/last-chat.json"
CHAT_PROMPT_DEFAULT="Reply with exactly: pong"
# _chat MODEL [KIND] [PROMPT] — POST through the proxy from $CHAT_CLIENT
# (defaults to $CLIENT). Echoes the HTTP status code on stdout; writes the
# response body to $CHAT_RESP. Returns 000 if the proxy host won't resolve.
_chat() {
local model="$1" kind="${2:-chat}" prompt="${3:-$CHAT_PROMPT_DEFAULT}"
local client="${CHAT_CLIENT:-$CLIENT}" ip path body out code attempt
local extra_hdr=()
case "$kind" in
messages)
path="/v1/messages"; extra_hdr=(-H "anthropic-version: 2023-06-01")
body="$(jq -n --arg m "$model" --arg p "$prompt" '{model:$m,max_tokens:64,messages:[{role:"user",content:$p}]}')" ;;
*)
path="/v1/chat/completions"
body="$(jq -n --arg m "$model" --arg p "$prompt" '{model:$m,messages:[{role:"user",content:$p}]}')" ;;
esac
# Re-resolve the proxy IP and retry on connection failure (000): a config
# change churns the proxy peer's NetBird IP, so a freshly-joined client may
# need a few seconds for its tunnel + magic DNS to converge.
code=000
for attempt in $(seq 1 "${CHAT_RETRIES:-6}"); do
ip="$(CLIENT="$client" proxy_ip)"
if [ -n "$ip" ]; then
out="$(docker run --rm --network "container:$client" "$CURL_IMAGE" \
-sSk --connect-timeout 5 --max-time 90 --resolve "$PROXY_HOST:443:$ip" \
-w $'\n%{http_code}' -X POST "https://$PROXY_HOST$path" \
-H "Content-Type: application/json" ${extra_hdr[@]+"${extra_hdr[@]}"} --data "$body")"
code="$(printf '%s' "$out" | tail -n1)"
printf '%s' "$out" | sed '$d' >"$CHAT_RESP"
fi
[ "$code" != 000 ] && break
sleep 4
done
echo "$code"
}
cmd_chat() {
local code
log "POST $PROXY_HOST (model=$1 kind=${2:-chat} client=${CHAT_CLIENT:-$CLIENT})"
code="$(_chat "$@")"
cat "$CHAT_RESP" 2>/dev/null
printf '\n[http %s]\n' "$code"
}
cmd_verify() {
log "===== consumption ====="
api GET /api/agent-network/consumption | jq -r '.[] | "\(.dimension_kind)/\(.dimension_id) tokens_in=\(.tokens_input) tokens_out=\(.tokens_output) cost=\(.cost_usd // 0)"' >&2 || true
log "===== last 10 access-log rows ====="
api GET /api/agent-network/access-logs | jq -r '.data[0:10][] | "\(.timestamp) provider=\(.provider) model=\(.model) status=\(.status_code) decision=\(.decision) src=\(.source_ip) session=\(.session_id // "-") in=\(.input_tokens // 0) out=\(.output_tokens // 0) cost=\(.cost_usd // 0)"' >&2 || true
}
cmd_clean() {
cmd_snapshot
local id
for id in $(api GET /api/agent-network/policies | jq -r '.[].id'); do
api DELETE "/api/agent-network/policies/$id" >/dev/null && log "deleted policy $id"
done
for id in $(api GET /api/agent-network/budget-rules | jq -r '.[].id'); do
api DELETE "/api/agent-network/budget-rules/$id" >/dev/null && log "deleted budget-rule $id"
done
for id in $(api GET /api/agent-network/providers | jq -r '.[].id'); do
api DELETE "/api/agent-network/providers/$id" >/dev/null && log "deleted provider $id"
done
log "account cleaned"
}
# create_provider NAME PROVIDER_ID UPSTREAM_URL API_KEY
create_provider() {
local name="$1" pid="$2" url="$3" key="$4" body resp
[ -n "$key" ] || { log "skip $name: empty key"; return 0; }
body="$(jq -n --arg n "$name" --arg p "$pid" --arg u "$url" --arg k "$key" \
'{name:$n,provider_id:$p,upstream_url:$u,api_key:$k,enabled:true}')"
resp="$(api POST /api/agent-network/providers "$body")" || { log "create $name FAILED"; return 1; }
echo "$resp" | jq -r '"created provider \(.name) id=\(.id) provider_id=\(.provider_id)"' >&2
echo "$resp" | jq -r '.id'
}
cmd_providers() {
require_llm_keys
: >"$STATE_DIR/provider-ids.txt"
create_provider "OpenAI API" openai_api "https://api.openai.com" "${OPENAI_TOKEN:-}" >>"$STATE_DIR/provider-ids.txt"
create_provider "Anthropic API" anthropic_api "https://api.anthropic.com" "${ANTHROPIC_TOKEN:-}" >>"$STATE_DIR/provider-ids.txt"
create_provider "Vercel AI Gateway" vercel_ai_gateway "${VERCEL_URL:-}" "${VERCEL_TOKEN:-}" >>"$STATE_DIR/provider-ids.txt"
create_provider "OpenRouter" openrouter "${OPENROUTER_URL:-}" "${OPENROUTER_TOKEN:-}" >>"$STATE_DIR/provider-ids.txt"
create_provider "Cloudflare AI Gateway" cloudflare_ai_gateway "${CLOUDFLARE_URL:-}" "${CLOUDFLARE_TOKEN:-}" >>"$STATE_DIR/provider-ids.txt"
log "provider ids:"; cat "$STATE_DIR/provider-ids.txt" >&2
}
cmd_policy() {
local gid ids body
gid="$(admins_group_id)"; [ -n "$gid" ] || die "no Admins group"
ids="$(api GET /api/agent-network/providers | jq -c '[.[].id]')"
body="$(jq -n --arg n "e2e-all-providers" --arg g "$gid" --argjson dst "$ids" '{
name:$n, description:"e2e: Admins to all providers", enabled:true,
source_groups:[$g], destination_provider_ids:$dst, guardrail_ids:[],
limits:{ budget_limit:{enabled:true,group_cap_usd:1000000,user_cap_usd:1000000,window_seconds:2592000},
token_limit:{enabled:false,group_cap:0,user_cap:0,window_seconds:60} }
}')"
api POST /api/agent-network/policies "$body" | jq -r '"created policy \(.name) id=\(.id) dst=\(.destination_provider_ids|length) providers"' >&2
}
# set_enabled PROVIDER_ID BOOL — PUT the provider back with enabled toggled,
# preserving its required fields and keeping the sealed key (api_key omitted).
set_enabled() {
local pid="$1" en="$2" cur body
cur="$(api GET /api/agent-network/providers | jq -c --arg id "$pid" '.[] | select(.id==$id)')"
[ -n "$cur" ] || { log "no provider $pid"; return 1; }
body="$(echo "$cur" | jq -c --argjson en "$en" '{name,provider_id,upstream_url,enabled:$en} + (if .extra_values then {extra_values} else {} end)')"
api PUT "/api/agent-network/providers/$pid" "$body" >/dev/null
}
# cmd_isolate NAME — leave only the named provider enabled (sole catch-all),
# so a request routes to it without first-party-vendor or first-catch-all
# interference. Matches NAME case-insensitively against provider .name.
cmd_isolate() {
local want="$1" id name en
api GET /api/agent-network/providers | jq -r '.[] | "\(.id)\t\(.name)"' | while IFS=$'\t' read -r id name; do
case "$(echo "$name" | tr '[:upper:]' '[:lower:]')" in
*"$(echo "$want" | tr '[:upper:]' '[:lower:]')"*) en=true ;;
*) en=false ;;
esac
set_enabled "$id" "$en" && log "$name enabled=$en"
done
}
cmd_enable_all() {
local id
for id in $(api GET /api/agent-network/providers | jq -r '.[].id'); do
set_enabled "$id" true && log "enabled $id"
done
}
# cmd_dashboard — drive the live :3000 dashboard (../dashboard repo) with
# Playwright, asserting the API-created providers/policy render in the UI.
cmd_dashboard() {
command -v node >/dev/null 2>&1 || die "node not on PATH"
[ -f "$DASH_DIR/e2e/live-agent-network.mjs" ] || die "dashboard script missing in $DASH_DIR/e2e"
( cd "$DASH_DIR" && BASE_URL="$DASH_URL" DASH_USER="$DASH_USER" DASH_PASS="$DASH_PASS" node e2e/live-agent-network.mjs )
}
cmd_down() { docker rm -f "$CLIENT" >/dev/null 2>&1 && log "removed $CLIENT" || log "no container"; }
cmd_all() {
cmd_clean
cmd_providers
cmd_policy
cmd_up
cmd_wait || { cmd_diag; die "tunnel to proxy not established"; }
log "===== chat: OpenAI =====" ; cmd_chat gpt-5.4 chat
log "===== chat: Anthropic =====" ; cmd_chat claude-haiku-4-5 messages
log "===== chat: Vercel (openai/...) ====="; cmd_chat openai/gpt-4o-mini chat
log "===== chat: OpenRouter (openai/...) =="; cmd_chat openai/gpt-4o-mini chat
sleep 3
cmd_verify
}
# --- scenario helpers --------------------------------------------------------
FAILS=0
ok() { log " PASS: $1"; }
bad() { log " FAIL: $1"; FAILS=$((FAILS+1)); }
expect_code() { # WANT GOT LABEL
if [ "$2" = "$1" ]; then ok "$3 (http $2)"; else bad "$3 (want $1, got $2)"; fi
}
# wait_tunnel [CLIENT] — poll silently until the proxy peer is 1/1 Connected.
wait_tunnel() {
local c="${1:-$CLIENT}" i=0
while [ "$i" -lt 60 ]; do
docker exec "$c" netbird status 2>/dev/null | grep -q '1/1 Connected' && return 0
sleep 3; i=$((i+3))
done
return 1
}
# wait_chat_ready CLIENT KIND MODEL — poll a real request from CLIENT until it
# returns a non-000 status, i.e. the tunnel + magic DNS + WG handshake to the
# proxy peer have all converged. Single-attempt probes pace the outer loop.
# Note: a netbird down/up bounce does NOT help here and is actively harmful —
# the peer is already in the net-map (status shows 0/1, i.e. known-not-connected),
# so the blocker is the WG handshake to a freshly-churned proxy peer, and
# bouncing just resets an in-progress slow handshake. We only wait it out.
wait_chat_ready() {
local c="$1" kind="${2:-chat}" model="${3:-gpt-5.4}" rc i=0
while [ "$i" -lt 30 ]; do
rc="$(CHAT_CLIENT="$c" CHAT_RETRIES=1 _chat "$model" "$kind")"
[ "$rc" != 000 ] && { echo "$rc"; return 0; }
i=$((i + 5))
sleep 5
done
echo 000
return 1
}
# wait_peer_connected CLIENT [TIMEOUT] — poll until the client reports the proxy
# peer as 1/1 Connected, i.e. the proxy peer has re-stabilised after a churn.
# Used to settle the peer before joining a fresh client into it.
wait_peer_connected() {
local c="$1" timeout="${2:-180}" i=0
while [ "$i" -lt "$timeout" ]; do
docker exec "$c" netbird status 2>/dev/null | grep -q '1/1 Connected' && return 0
sleep 5
i=$((i + 5))
done
return 1
}
provider_id_by_name() { api GET /api/agent-network/providers | jq -r --arg n "$1" '.[] | select(.name==$n) | .id' | head -n1; }
access_log_total() { api GET /api/agent-network/access-logs | jq -r '.total_records'; }
access_log_top() { api GET /api/agent-network/access-logs | jq -r ".data[0].$1 // \"\""; }
# ensure_group NAME — return the id of group NAME, creating it if absent.
ensure_group() {
local name="$1" id
id="$(api GET /api/groups | jq -r --arg n "$name" '.[] | select(.name==$n) | .id' | head -n1)"
if [ -z "$id" ] || [ "$id" = null ]; then
id="$(api POST /api/groups "$(jq -n --arg n "$name" '{name:$n}')" | jq -r '.id')"
fi
echo "$id"
}
# policy_put_limits LIMITS_JSON — replace the e2e-all-providers policy limits.
policy_put_limits() {
local pol id body
pol="$(api GET /api/agent-network/policies | jq -c '.[] | select(.name=="e2e-all-providers")')"
id="$(echo "$pol" | jq -r '.id')"
[ -n "$id" ] && [ "$id" != null ] || die "e2e-all-providers policy missing (run 'policy' first)"
body="$(echo "$pol" | jq -c --argjson L "$1" '{name,description,enabled,source_groups,destination_provider_ids,guardrail_ids,limits:$L}')"
api PUT "/api/agent-network/policies/$id" "$body" >/dev/null
}
settings_put() { api PUT /api/agent-network/settings "$1" >/dev/null; }
# --- scenarios ---------------------------------------------------------------
# Scenario 1: policy token-cap enforcement. Seed usage under a high cap, drop
# the cap to 1 token so the next call is denied, then restore and confirm
# recovery. Deterministic regardless of prior window usage.
cmd_scenario_budget() {
log "### scenario: policy token-cap enforcement (deny + recovery) ###"
# Seed AND enforce in the same 3600s token window (counters are per-window).
policy_put_limits '{"budget_limit":{"enabled":false,"group_cap_usd":0,"user_cap_usd":0,"window_seconds":3600},"token_limit":{"enabled":true,"group_cap":100000,"user_cap":100000,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "seed call under high token cap (books usage into 3600s window)"
# Drop the cap to 1 in the SAME window so the seeded usage exhausts it.
policy_put_limits '{"budget_limit":{"enabled":false,"group_cap_usd":0,"user_cap_usd":0,"window_seconds":3600},"token_limit":{"enabled":true,"group_cap":1,"user_cap":1,"window_seconds":3600}}'
wait_tunnel; sleep 2
local code; code="$(_chat gpt-5.4 chat)"
expect_code 403 "$code" "call denied once token cap (1) is exhausted"
log " deny envelope: $(cat "$CHAT_RESP")"
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "call allowed again after cap restored"
}
# Scenario 2: account-level budget rule (separate from policy limits). A rule
# targeting Admins with a 1-token cap must deny via the same gRPC check loop;
# deleting it restores access.
cmd_scenario_budget_rule() {
log "### scenario: account budget-rule enforcement ###"
local gid rid
gid="$(admins_group_id)"
# High cap first so the seed call books usage into the rule's 3600s window.
rid="$(api POST /api/agent-network/budget-rules "$(jq -n --arg g "$gid" '{name:"e2e-tight-rule",enabled:true,target_groups:[$g],target_users:[],limits:{budget_limit:{enabled:false,group_cap_usd:0,user_cap_usd:0,window_seconds:3600},token_limit:{enabled:true,group_cap:100000,user_cap:100000,window_seconds:3600}}}')" | jq -r '.id')"
log " created budget-rule $rid (group token cap=100000)"
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "seed call under high account cap"
# Tighten the same rule/window to 1 so the seeded usage exhausts it.
api PUT "/api/agent-network/budget-rules/$rid" "$(jq -n --arg g "$gid" '{name:"e2e-tight-rule",enabled:true,target_groups:[$g],target_users:[],limits:{budget_limit:{enabled:false,group_cap_usd:0,user_cap_usd:0,window_seconds:3600},token_limit:{enabled:true,group_cap:1,user_cap:1,window_seconds:3600}}}')" >/dev/null
log " tightened budget-rule $rid (group token cap=1)"
wait_tunnel; sleep 2
expect_code 403 "$(_chat gpt-5.4 chat)" "account budget-rule denies when cap exhausted"
api DELETE "/api/agent-network/budget-rules/$rid" >/dev/null && log " deleted budget-rule $rid"
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "allowed again after budget-rule removed"
}
# Scenario 3: multiple groups + policies + destination scoping. A 2nd Docker
# client joins a 2nd group; each group's policy authorises a different
# provider. Cross-provider requests must be denied.
cmd_scenario_multigroup() {
log "### scenario: multi-group / multi-policy destination scoping ###"
local gidA gidB oai ant keyB polAllId
gidA="$(admins_group_id)"; gidB="$(ensure_group e2e-grp-b)"
oai="$(provider_id_by_name 'OpenAI API')"; ant="$(provider_id_by_name 'Anthropic API')"
log " groups: Admins=$gidA grp-b=$gidB ; providers: openai=$oai anthropic=$ant"
# Disable the broad e2e policy; add two narrow ones.
polAllId="$(api GET /api/agent-network/policies | jq -r '.[] | select(.name=="e2e-all-providers") | .id')"
[ -n "$polAllId" ] && api PUT "/api/agent-network/policies/$polAllId" \
"$(api GET /api/agent-network/policies | jq -c --arg id "$polAllId" '.[] | select(.id==$id) | {name,description,enabled:false,source_groups,destination_provider_ids,guardrail_ids,limits}')" >/dev/null
local hi='{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":60}}'
local polA polB
polA="$(api POST /api/agent-network/policies "$(jq -n --arg g "$gidA" --arg p "$oai" --argjson L "$hi" '{name:"e2e-A-admins-openai",enabled:true,source_groups:[$g],destination_provider_ids:[$p],guardrail_ids:[],limits:$L}')" | jq -r '.id')"
polB="$(api POST /api/agent-network/policies "$(jq -n --arg g "$gidB" --arg p "$ant" --argjson L "$hi" '{name:"e2e-B-grpb-anthropic",enabled:true,source_groups:[$g],destination_provider_ids:[$p],guardrail_ids:[],limits:$L}')" | jq -r '.id')"
log " policy A (Admins->OpenAI)=$polA ; policy B (grp-b->Anthropic)=$polB"
# The policy swap above re-synthesises the agent-network service and adds a
# new source group (grp-b), which churns the proxy peer and forces it to
# absorb the new authorisation. Wait for the peer to re-stabilise (client A
# reconnects to the new peer) before joining B, so B handshakes into a stable
# peer instead of racing the churn — otherwise the proxy never answers B's
# offers and it sits at 0/1 Connected.
if ! wait_peer_connected "$CLIENT" 30; then
log " warning: proxy peer did not re-stabilise for client A within 30s"
fi
sleep 5
# 2nd client bound to grp-b.
keyB="$(api POST /api/setup-keys "$(jq -n --arg g "$gidB" '{name:"e2e-agentnet-docker-b",type:"reusable",expires_in:86400,usage_limit:0,auto_groups:[$g],ephemeral:false}')" | jq -r '.key')"
docker rm -f "${CLIENT}-b" >/dev/null 2>&1 || true
docker run -d --name "${CLIENT}-b" --cap-add NET_ADMIN --cap-add SYS_ADMIN --cap-add SYS_RESOURCE \
--add-host host.docker.internal:host-gateway -e NB_MANAGEMENT_URL="$MGMT_URL" -e NB_SETUP_KEY="$keyB" \
-e NB_LOG_LEVEL=info "$IMAGE" >/dev/null
log " started ${CLIENT}-b (grp-b)"
# Gate on real end-to-end connectivity: a freshly-authorised group's client
# takes ~1-2 min for the proxy to absorb and answer its WG handshake.
local ra rb
ra="$(wait_chat_ready "$CLIENT" chat gpt-5.4)"
rb="$(wait_chat_ready "${CLIENT}-b" messages claude-haiku-4-5)"
log " connectivity ready: A(probe=$ra) B(probe=$rb)"
log " -- client A (Admins) --"
CHAT_CLIENT="$CLIENT" expect_code 200 "$(CHAT_CLIENT="$CLIENT" _chat gpt-5.4 chat)" "A->OpenAI authorised"
CHAT_CLIENT="$CLIENT" expect_code 403 "$(CHAT_CLIENT="$CLIENT" _chat claude-haiku-4-5 messages)" "A->Anthropic denied (not in policy A)"
log " -- client B (grp-b) --"
CHAT_CLIENT="${CLIENT}-b" expect_code 200 "$(CHAT_CLIENT="${CLIENT}-b" _chat claude-haiku-4-5 messages)" "B->Anthropic authorised"
CHAT_CLIENT="${CLIENT}-b" expect_code 403 "$(CHAT_CLIENT="${CLIENT}-b" _chat gpt-5.4 chat)" "B->OpenAI denied (not in policy B)"
# Restore: drop A/B, re-enable the broad policy.
api DELETE "/api/agent-network/policies/$polA" >/dev/null 2>&1 || true
api DELETE "/api/agent-network/policies/$polB" >/dev/null 2>&1 || true
[ -n "$polAllId" ] && api PUT "/api/agent-network/policies/$polAllId" \
"$(api GET /api/agent-network/policies | jq -c --arg id "$polAllId" '.[] | select(.id==$id) | {name,description,enabled:true,source_groups,destination_provider_ids,guardrail_ids,limits}')" >/dev/null
docker rm -f "${CLIENT}-b" >/dev/null 2>&1 || true
log " restored broad policy, removed client B"
}
# Scenario 4: log-collection + prompt-collection + redaction settings.
cmd_scenario_logs() {
log "### scenario: log/prompt-collection + redaction settings ###"
settings_put '{"enable_log_collection":true,"enable_prompt_collection":true,"redact_pii":false}'
wait_tunnel; sleep 3
# (a) log collection OFF -> no new access-log row.
local before after
before="$(access_log_total)"
settings_put '{"enable_log_collection":false,"enable_prompt_collection":true,"redact_pii":false}'
wait_tunnel; sleep 3
expect_code 200 "$(_chat gpt-5.4 chat)" "request still served with logging off"
sleep 3; after="$(access_log_total)"
if [ "$after" = "$before" ]; then ok "log collection OFF -> no new row ($before==$after)"; else bad "log collection OFF still wrote a row ($before->$after)"; fi
# (b) log ON, prompt OFF -> row present but request_prompt empty.
settings_put '{"enable_log_collection":true,"enable_prompt_collection":false,"redact_pii":false}'
wait_tunnel; sleep 3
expect_code 200 "$(_chat gpt-5.4 chat "this prompt text must NOT be stored")" "served with prompt collection off"
sleep 3
local p; p="$(access_log_top request_prompt)"
if [ -z "$p" ]; then ok "prompt collection OFF -> request_prompt empty"; else bad "prompt stored despite collection off: [$p]"; fi
# (c) prompt ON + redact ON -> PII scrubbed from stored prompt.
settings_put '{"enable_log_collection":true,"enable_prompt_collection":true,"redact_pii":true}'
wait_tunnel; sleep 3
expect_code 200 "$(_chat gpt-5.4 chat "Contact john.doe@example.com, SSN 123-45-6789, phone 555-123-4567")" "served with redaction on"
sleep 3
local stored; stored="$(access_log_top request_prompt)"
log " stored prompt: [$stored]"
case "$stored" in
*john.doe@example.com*) bad "email leaked into stored prompt" ;;
*) ok "email redacted from stored prompt" ;;
esac
case "$stored" in
*123-45-6789*) bad "SSN leaked into stored prompt" ;;
*) ok "SSN redacted from stored prompt" ;;
esac
settings_put '{"enable_log_collection":true,"enable_prompt_collection":true,"redact_pii":false}'
log " restored settings (log on, prompt on, redact off)"
}
# Scenario 5: per-user cap isolated from the group cap. Seed under a high cap,
# then keep the group cap high but drop the USER cap to 1 — the deny must come
# from the user dimension, proving per-user accounting is independent.
cmd_scenario_user_cap() {
log "### scenario: per-user token cap (isolated from group cap) ###"
policy_put_limits '{"budget_limit":{"enabled":false,"group_cap_usd":0,"user_cap_usd":0,"window_seconds":3600},"token_limit":{"enabled":true,"group_cap":100000,"user_cap":100000,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "seed call (books user+group usage into 3600s window)"
# Group cap stays huge; only the user cap is exhausted.
policy_put_limits '{"budget_limit":{"enabled":false,"group_cap_usd":0,"user_cap_usd":0,"window_seconds":3600},"token_limit":{"enabled":true,"group_cap":100000,"user_cap":1,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 403 "$(_chat gpt-5.4 chat)" "denied by USER cap=1 while group cap=100000 has headroom"
log " deny: $(cat "$CHAT_RESP")"
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 200 "$(_chat gpt-5.4 chat)" "allowed again after user cap lifted"
}
# Scenario 6: guardrail model-allowlist blocking. Attach a guardrail allowing
# only gpt-5.4; a request for any other model must be denied (model_blocked),
# while the allowed model passes.
cmd_scenario_guardrail() {
log "### scenario: guardrail model-allowlist blocking ###"
local gid polId
gid="$(api POST /api/agent-network/guardrails "$(jq -n '{name:"e2e-allowlist",description:"e2e: only gpt-5.4",checks:{model_allowlist:{enabled:true,models:["gpt-5.4"]},prompt_capture:{enabled:true,redact_pii:false}}}')" | jq -r '.id')"
log " created guardrail $gid (allow only gpt-5.4)"
local pol; pol="$(api GET /api/agent-network/policies | jq -c '.[] | select(.name=="e2e-all-providers")')"
polId="$(echo "$pol" | jq -r '.id')"
api PUT "/api/agent-network/policies/$polId" "$(echo "$pol" | jq -c --arg g "$gid" '{name,description,enabled,source_groups,destination_provider_ids,guardrail_ids:[$g],limits}')" >/dev/null
wait_tunnel; sleep 2
expect_code 403 "$(_chat gpt-4o-mini chat)" "model not in allowlist is blocked"
log " deny: $(cat "$CHAT_RESP")"
expect_code 200 "$(_chat gpt-5.4 chat)" "allowlisted model passes"
# Detach + delete.
api PUT "/api/agent-network/policies/$polId" "$(echo "$pol" | jq -c '{name,description,enabled,source_groups,destination_provider_ids,guardrail_ids:[],limits}')" >/dev/null
api DELETE "/api/agent-network/guardrails/$gid" >/dev/null && log " detached + deleted guardrail $gid"
}
# _chat_stream MODEL — streaming chat completion (SSE) with usage included.
# Echoes the HTTP status; writes the raw SSE stream to $CHAT_RESP.
_chat_stream() {
local model="$1" client="${CHAT_CLIENT:-$CLIENT}" ip body out code
ip="$(CLIENT="$client" proxy_ip)"; [ -n "$ip" ] || { echo 000; return; }
body="$(jq -n --arg m "$model" '{model:$m,stream:true,stream_options:{include_usage:true},messages:[{role:"user",content:"Reply with exactly: pong"}]}')"
out="$(docker run --rm --network "container:$client" "$CURL_IMAGE" \
-sSkN --connect-timeout 5 --max-time 90 --resolve "$PROXY_HOST:443:$ip" \
-w $'\n%{http_code}' -X POST "https://$PROXY_HOST/v1/chat/completions" \
-H "Content-Type: application/json" --data "$body")"
code="$(printf '%s' "$out" | tail -n1)"
printf '%s' "$out" | sed '$d' >"$CHAT_RESP"
echo "$code"
}
# Scenario 7: streaming (SSE) token capture. The proxy must accumulate token
# usage from the streamed deltas and persist it on the access-log row.
cmd_scenario_streaming() {
log "### scenario: streaming (SSE) token capture ###"
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
wait_tunnel; sleep 2
expect_code 200 "$(_chat_stream gpt-5.4)" "streaming request served"
if grep -q 'data: \[DONE\]' "$CHAT_RESP"; then ok "SSE stream terminated with [DONE]"; else bad "no [DONE] terminator in SSE stream"; fi
if grep -q '"delta"' "$CHAT_RESP"; then ok "SSE carried incremental deltas"; else bad "no delta chunks in SSE stream"; fi
sleep 3
local st in out
st="$(access_log_top stream)"; in="$(access_log_top input_tokens)"; out="$(access_log_top output_tokens)"
log " access-log: stream=$st input_tokens=$in output_tokens=$out"
if [ "$st" = true ]; then ok "access-log row flagged stream=true"; else bad "stream flag not set (got $st)"; fi
if [ "${in:-0}" -gt 0 ] 2>/dev/null && [ "${out:-0}" -gt 0 ] 2>/dev/null; then ok "streamed token usage captured (in=$in out=$out)"; else bad "streamed token usage not captured (in=$in out=$out)"; fi
}
# wait_for_mgmt — poll the management API until it accepts authed requests.
wait_for_mgmt() {
local i=0
while [ "$i" -lt 90 ]; do
curl -fsS -o /dev/null --max-time 2 -H "$AUTH" "$NB_API/api/users" 2>/dev/null && return 0
sleep 2; i=$((i+2))
done
return 1
}
# Scenario 8: access-log retention pruning. The sweep deletes rows older than
# now - retention_days and runs on management startup. We clone an existing row
# to a synthetic 2020-dated id, set retention to 365d (so only that far-past
# row is eligible — real 2026 rows are safe), restart management to trigger the
# sweep, and assert the synthetic row is gone while real rows survive.
cmd_scenario_retention() {
log "### scenario: access-log retention pruning ###"
local DB="/Users/maycon/projects/wt_testing_data/store.db" acct="d68ag3p31576fp2gmnag" sid="e2e-retention-2020"
command -v sqlite3 >/dev/null 2>&1 || { bad "sqlite3 not on PATH"; return; }
[ -w "$DB" ] || { bad "store.db not writable at $DB"; return; }
local before mid after syn
before="$(sqlite3 "$DB" "SELECT count(*) FROM agent_network_access_log WHERE account_id='$acct'")"
sqlite3 "$DB" "PRAGMA busy_timeout=8000;
DELETE FROM agent_network_access_log WHERE id='$sid';
CREATE TEMP TABLE _t AS SELECT * FROM agent_network_access_log WHERE account_id='$acct' LIMIT 1;
UPDATE _t SET id='$sid', timestamp='2020-01-01 00:00:00.000000000+00:00';
INSERT INTO agent_network_access_log SELECT * FROM _t;" 2>&1 | sed 's/^/ sqlite: /' >&2 || true
mid="$(sqlite3 "$DB" "SELECT count(*) FROM agent_network_access_log WHERE account_id='$acct'")"
if [ "$mid" = "$((before+1))" ]; then ok "synthetic 2020 row inserted ($before -> $mid)"; else bad "insert failed ($before -> $mid)"; return; fi
settings_put '{"enable_log_collection":true,"enable_prompt_collection":true,"redact_pii":false,"access_log_retention_days":365}'
log " retention=365d; restarting management to trigger the startup sweep..."
tilt trigger management >/dev/null 2>&1 || bad "tilt trigger management failed"
wait_for_mgmt || bad "management did not come back up"
sleep 4
syn="$(sqlite3 "$DB" "SELECT count(*) FROM agent_network_access_log WHERE id='$sid'")"
after="$(sqlite3 "$DB" "SELECT count(*) FROM agent_network_access_log WHERE account_id='$acct'")"
if [ "$syn" = 0 ]; then ok "synthetic 2020 row pruned by retention sweep"; else bad "synthetic row survived (sweep didn't prune)"; fi
if [ "$after" = "$before" ]; then ok "real rows preserved (count back to $before)"; else bad "real row count changed ($before -> $after)"; fi
settings_put '{"enable_log_collection":true,"enable_prompt_collection":true,"redact_pii":false,"access_log_retention_days":0}'
sqlite3 "$DB" "DELETE FROM agent_network_access_log WHERE id='$sid'" 2>/dev/null || true
log " restored retention=0 (keep forever); re-establishing tunnel"
wait_tunnel >/dev/null 2>&1 || true
}
# vertex_request IP PATH BODY — POST a Vertex rawPredict through the client
# sidecar; echo the HTTP status and stash the response body in VTX_RESP. Shared
# by the Vertex scenario's initial probe and its live-update propagation checks.
VTX_RESP=""
vertex_request() {
local ip="$1" path="$2" body="$3" out
out="$(docker run --rm --network "container:$CLIENT" "$CURL_IMAGE" \
-sSk --connect-timeout 5 --max-time 90 --resolve "$PROXY_HOST:443:$ip" \
-w $'\n%{http_code}' -X POST "https://$PROXY_HOST$path" \
-H "Content-Type: application/json" --data "$body")"
VTX_RESP="$(printf '%s' "$out" | sed '$d')"
printf '%s' "$out" | tail -n1
}
# bedrock_probe LABEL PATH BODY IP — POST a Bedrock request via the client
# sidecar and assert the pipeline. 200 -> full pass (+ token metering); a
# Bedrock-origin 404 (account use-case gate / model access) -> pipeline OK
# (routing + bearer auth reached the model); 401/403 -> fail.
bedrock_probe() {
local label="$1" path="$2" body="$3" ip="$4" out code resp
out="$(docker run --rm --network "container:$CLIENT" "$CURL_IMAGE" \
-sSk --connect-timeout 5 --max-time 90 --resolve "$PROXY_HOST:443:$ip" \
-w $'\n%{http_code}' -X POST "https://$PROXY_HOST$path" \
-H "Content-Type: application/json" --data "$body")"
code="$(printf '%s' "$out" | tail -n1)"
resp="$(printf '%s' "$out" | sed '$d')"
log " bedrock $label -> http=$code"
case "$code" in
200)
ok "Bedrock $label 200 (path->model parse, routed, bearer accepted)"
sleep 3
local vin vout
vin="$(access_log_top input_tokens)"; vout="$(access_log_top output_tokens)"
if [ "${vin:-0}" -gt 0 ] 2>/dev/null && [ "${vout:-0}" -gt 0 ] 2>/dev/null; then ok "Bedrock $label usage metered (in=$vin out=$vout)"; else bad "Bedrock $label usage not metered (in=$vin out=$vout)"; fi
;;
404)
case "$resp" in
*"use case"*|*NOT_FOUND*|*bedrock*|*"don't have access"*|*"inference profile"*)
ok "reached Bedrock ($label, provider+bearer OK); 404 = account use-case gate / model access" ;;
*) bad "Bedrock $label 404 but not a Bedrock response: $(printf '%s' "$resp" | head -c 160)" ;;
esac ;;
401) bad "Bedrock $label rejected the bearer token (401): $(printf '%s' "$resp" | head -c 160)" ;;
403) bad "Bedrock $label denied before upstream (routing regression): $(printf '%s' "$resp" | head -c 160)" ;;
000) bad "Bedrock $label no connectivity (tunnel not converged)" ;;
*) bad "Bedrock $label unexpected status $code: $(printf '%s' "$resp" | head -c 160)" ;;
esac
}
# Scenario 9: Google Vertex AI (path-routed model + service-account OAuth minting).
# Vertex carries the model in the URL path and authenticates with a short-lived
# OAuth2 access token, not an API key. The operator stores a durable GCP
# service-account key as the provider api_key behind a "keyfile::" prefix; the
# synthesiser ships the base64 SA JSON to the router, which mints + caches a
# Bearer token per request. We assert the whole pipeline: path->model parse,
# route-to-Vertex, token mint + inject, upstream forward. A 200 is a complete
# pass; a 404 whose body is Vertex's own "Publisher model not found / no access"
# also passes the pipeline (the token was accepted) and only signals that the
# test project lacks model access in that region.
cmd_scenario_vertex() {
log "### scenario: Google Vertex AI (path-routed + service-account OAuth minting) ###"
require_llm_keys
if [ -z "${GOOGLE_VERTEX_SA_BASE64:-}" ] || [ -z "${GOOGLE_VERTEXT_PROJECT:-}" ]; then
log " SKIP: GOOGLE_VERTEX_SA_BASE64 / GOOGLE_VERTEXT_PROJECT not set in $LLM_KEYS_FILE"
return 0
fi
local vid pol pid old oldvids region vhost
# The regional endpoint host is "<region>-aiplatform.googleapis.com"; the
# "global" location uses the bare "aiplatform.googleapis.com" host.
region="${GOOGLE_VERTEXT_REGION:-global}"
if [ "$region" = global ]; then vhost="aiplatform.googleapis.com"; else vhost="${region}-aiplatform.googleapis.com"; fi
# Remove any pre-existing Vertex providers, de-referencing them from the broad
# policy first — a provider that is still a policy destination can't be deleted
# (422), and a survivor leaves a duplicate (possibly stale-host) Vertex route.
oldvids="$(api GET /api/agent-network/providers | jq -c '[.[]|select(.provider_id=="vertex_ai_api")|.id]')"
if [ "$oldvids" != "[]" ]; then
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
pid="$(echo "$pol" | jq -r '.id')"
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --argjson rm "$oldvids" '{name,description,enabled,source_groups,destination_provider_ids:(.destination_provider_ids-$rm),guardrail_ids,limits}')" >/dev/null 2>&1 || true
for old in $(echo "$oldvids" | jq -r '.[]'); do
api DELETE "/api/agent-network/providers/$old" >/dev/null 2>&1 || true
done
fi
vid="$(api POST /api/agent-network/providers "$(jq -n --arg k "keyfile::${GOOGLE_VERTEX_SA_BASE64}" --arg u "https://${vhost}" \
'{name:"Vertex e2e",provider_id:"vertex_ai_api",upstream_url:$u,api_key:$k,enabled:true}')" \
| jq -r '.id')"
[ -n "$vid" ] && [ "$vid" != null ] || { bad "create Vertex provider failed"; return; }
log " created Vertex provider $vid (keyfile:: service-account, $vhost)"
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
pid="$(echo "$pol" | jq -r '.id')"
[ -n "$pid" ] && [ "$pid" != null ] || { bad "e2e-all-providers policy missing (run 'policy' first)"; return; }
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --arg v "$vid" '{name,description,enabled,source_groups,destination_provider_ids:((.destination_provider_ids+[$v])|unique),guardrail_ids,limits}')" >/dev/null
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
# Provider create/update propagates to the proxy live via reconcile (the synth
# config is re-pushed and the middleware chain rebuilt on receipt), so just let
# the tunnel settle before probing — no restart needed.
wait_tunnel; sleep 5
local ip path body code resp
ip="$(proxy_ip)"; [ -n "$ip" ] || { bad "could not resolve proxy IP"; return; }
path="/v1/projects/${GOOGLE_VERTEXT_PROJECT}/locations/${region}/publishers/anthropic/models/claude-sonnet-4-5@20250929:rawPredict"
body="$(jq -n '{anthropic_version:"vertex-2023-10-16",max_tokens:64,messages:[{role:"user",content:"Reply with exactly: pong"}]}')"
code="$(vertex_request "$ip" "$path" "$body")"; resp="$VTX_RESP"
log " vertex rawPredict -> http=$code"
case "$code" in
200)
ok "Vertex 200 (path->model parse, routed, token minted + accepted, model available)"
sleep 3
local vin vout
vin="$(access_log_top input_tokens)"; vout="$(access_log_top output_tokens)"
log " access-log: model=$(access_log_top model) input_tokens=$vin output_tokens=$vout"
if [ "${vin:-0}" -gt 0 ] 2>/dev/null && [ "${vout:-0}" -gt 0 ] 2>/dev/null; then ok "Vertex usage metered (in=$vin out=$vout)"; else bad "Vertex usage not metered (in=$vin out=$vout)"; fi
;;
404)
case "$resp" in
*"Publisher model"*|*aiplatform*|*NOT_FOUND*)
ok "reached Vertex with a minted token (pipeline OK); 404 = test project lacks model access" ;;
*) bad "404 but not a Vertex response body: $(printf '%s' "$resp" | head -c 160)" ;;
esac ;;
401) bad "Vertex rejected the minted credential (401) — check the SA key/roles: $(printf '%s' "$resp" | head -c 160)" ;;
403) bad "proxy denied before reaching Vertex (path parse / routing regression): $(printf '%s' "$resp" | head -c 160)" ;;
000) bad "no connectivity to proxy (tunnel not converged)" ;;
*) bad "unexpected status $code: $(printf '%s' "$resp" | head -c 160)" ;;
esac
# Live provider-update propagation: a provider change must reach the proxy node
# without a restart. Disable the provider -> its route disappears and the request
# is denied (403); re-enable -> served again at the baseline status. Proves
# reconcile re-pushes the synth config and the proxy rebuilds its chain live.
case "$code" in
200|404)
log " testing live provider-update propagation (disable/enable, no restart)..."
local dcode ecode
set_enabled "$vid" false; sleep 5
dcode="$(vertex_request "$ip" "$path" "$body")"
if [ "$dcode" = 403 ]; then ok "provider disable reflected on proxy live (route removed, http 403)"; else bad "provider disable not reflected on proxy (want 403, got $dcode)"; fi
set_enabled "$vid" true; sleep 5
ecode="$(vertex_request "$ip" "$path" "$body")"
if [ "$ecode" = "$code" ]; then ok "provider re-enable reflected on proxy live (served again, http $ecode)"; else bad "provider re-enable not reflected on proxy (want $code, got $ecode)"; fi
;;
esac
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --arg v "$vid" '{name,description,enabled,source_groups,destination_provider_ids:(.destination_provider_ids-[$v]),guardrail_ids,limits}')" >/dev/null 2>&1 || true
api DELETE "/api/agent-network/providers/$vid" >/dev/null 2>&1 || true
}
# Scenario 10: real-traffic coverage for the OpenAI-compatible gateway providers
# (Vercel, OpenRouter, Cloudflare). Each is isolated as the sole enabled provider
# so its model routes to it unambiguously, sent a real chat request, and asserted
# to return 200 with metered token usage. A gateway without a configured provider
# is skipped. Restores all providers at the end.
cmd_scenario_providers() {
log "### scenario: gateway providers real-traffic (Vercel / OpenRouter / Cloudflare) ###"
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
local spec iso full model kind r pid code vin vout
for spec in \
"vercel|Vercel AI Gateway|openai/gpt-4o-mini|chat" \
"openrouter|OpenRouter|openai/gpt-4o-mini|chat" \
"cloudflare|Cloudflare AI Gateway|gpt-4o-mini|chat"; do
iso="${spec%%|*}"; r="${spec#*|}"; full="${r%%|*}"; r="${r#*|}"; model="${r%%|*}"; kind="${r##*|}"
pid="$(provider_id_by_name "$full")"
if [ -z "$pid" ] || [ "$pid" = null ]; then log " SKIP $full: not configured"; continue; fi
cmd_isolate "$iso" >/dev/null 2>&1
wait_tunnel; sleep 4
code="$(_chat "$model" "$kind")"
if [ "$code" = 200 ]; then ok "$full served real request ($model, http 200)"; else bad "$full request failed ($model, http $code)"; continue; fi
sleep 3
vin="$(access_log_top input_tokens)"; vout="$(access_log_top output_tokens)"
if [ "${vin:-0}" -gt 0 ] 2>/dev/null && [ "${vout:-0}" -gt 0 ] 2>/dev/null; then ok "$full usage metered (in=$vin out=$vout)"; else bad "$full usage not metered (in=$vin out=$vout)"; fi
done
cmd_enable_all >/dev/null 2>&1
log " re-enabled all providers"
wait_tunnel >/dev/null 2>&1 || true
}
# Scenario 11: AWS Bedrock (path-routed model + bearer auth) across invoke,
# converse, and invoke-with-response-stream. Bedrock carries the model in the URL
# path and authenticates with a static bearer token (Bedrock API key). Asserts
# the pipeline per endpoint: path->model parse + normalize, route-to-Bedrock,
# bearer inject, upstream forward (+ token metering on 200). A Bedrock-origin 404
# (account use-case gate / model access) passes the pipeline; see bedrock_probe.
cmd_scenario_bedrock() {
log "### scenario: AWS Bedrock (path-routed + bearer; invoke/converse/stream) ###"
require_llm_keys
if [ -z "${AWS_BEARER_TOKEN_BEDROCK:-}" ]; then
log " SKIP: AWS_BEARER_TOKEN_BEDROCK not set in $LLM_KEYS_FILE"
return 0
fi
local region model vid pol pid old oldvids
region="${AWS_BEDROCK_REGION:-eu-central-1}"
model="${BEDROCK_MODEL:-eu.anthropic.claude-sonnet-4-5-20250929-v1:0}"
oldvids="$(api GET /api/agent-network/providers | jq -c '[.[]|select(.provider_id=="bedrock_api")|.id]')"
if [ "$oldvids" != "[]" ]; then
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
pid="$(echo "$pol" | jq -r '.id')"
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --argjson rm "$oldvids" '{name,description,enabled,source_groups,destination_provider_ids:(.destination_provider_ids-$rm),guardrail_ids,limits}')" >/dev/null 2>&1 || true
for old in $(echo "$oldvids" | jq -r '.[]'); do
api DELETE "/api/agent-network/providers/$old" >/dev/null 2>&1 || true
done
fi
vid="$(api POST /api/agent-network/providers "$(jq -n --arg k "$AWS_BEARER_TOKEN_BEDROCK" --arg u "https://bedrock-runtime.${region}.amazonaws.com" \
'{name:"Bedrock e2e",provider_id:"bedrock_api",upstream_url:$u,api_key:$k,enabled:true}')" | jq -r '.id')"
[ -n "$vid" ] && [ "$vid" != null ] || { bad "create Bedrock provider failed"; return; }
log " created Bedrock provider $vid (bearer, bedrock-runtime.${region})"
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
pid="$(echo "$pol" | jq -r '.id')"
[ -n "$pid" ] && [ "$pid" != null ] || { bad "e2e-all-providers policy missing (run 'policy' first)"; return; }
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --arg v "$vid" '{name,description,enabled,source_groups,destination_provider_ids:((.destination_provider_ids+[$v])|unique),guardrail_ids,limits}')" >/dev/null
policy_put_limits '{"budget_limit":{"enabled":true,"group_cap_usd":1000000,"user_cap_usd":1000000,"window_seconds":2592000},"token_limit":{"enabled":false,"group_cap":0,"user_cap":0,"window_seconds":3600}}'
wait_tunnel; sleep 5
local ip invokeBody converseBody
ip="$(proxy_ip)"; [ -n "$ip" ] || { bad "could not resolve proxy IP"; return; }
invokeBody='{"anthropic_version":"bedrock-2023-05-31","max_tokens":32,"messages":[{"role":"user","content":"Reply with exactly: pong"}]}'
converseBody='{"messages":[{"role":"user","content":[{"text":"Reply with exactly: pong"}]}],"inferenceConfig":{"maxTokens":32}}'
bedrock_probe "invoke" "/model/${model}/invoke" "$invokeBody" "$ip"
bedrock_probe "converse" "/model/${model}/converse" "$converseBody" "$ip"
bedrock_probe "invoke-stream" "/model/${model}/invoke-with-response-stream" "$invokeBody" "$ip"
# Gateway-namespace prefix: /bedrock/... must route the same and be stripped
# before the upstream call (AWS's native path has no /bedrock prefix).
bedrock_probe "invoke (/bedrock prefix)" "/bedrock/model/${model}/invoke" "$invokeBody" "$ip"
pol="$(api GET /api/agent-network/policies | jq -c '.[]|select(.name=="e2e-all-providers")')"
api PUT "/api/agent-network/policies/$pid" \
"$(echo "$pol" | jq -c --arg v "$vid" '{name,description,enabled,source_groups,destination_provider_ids:(.destination_provider_ids-[$v]),guardrail_ids,limits}')" >/dev/null 2>&1 || true
api DELETE "/api/agent-network/providers/$vid" >/dev/null 2>&1 || true
}
cmd_scenarios() {
FAILS=0
cmd_scenario_budget
cmd_scenario_budget_rule
cmd_scenario_user_cap
cmd_scenario_guardrail
cmd_scenario_streaming
cmd_scenario_multigroup
cmd_scenario_logs
cmd_scenario_retention
cmd_scenario_providers
cmd_scenario_vertex
cmd_scenario_bedrock
log "================================================"
if [ "$FAILS" -eq 0 ]; then log "ALL SCENARIOS PASSED"; else log "SCENARIO FAILURES: $FAILS"; fi
return "$FAILS"
}
# --- dispatch ----------------------------------------------------------------
cmd="${1:-}"; shift || true
case "$cmd" in
snapshot) cmd_snapshot ;;
key) cmd_key ;;
up) cmd_up ;;
status) cmd_status ;;
wait) cmd_wait ;;
diag) cmd_diag ;;
restart-proxy) cmd_restart_proxy ;;
chat) cmd_chat "$@" ;;
verify) cmd_verify ;;
clean) cmd_clean ;;
providers) cmd_providers ;;
policy) cmd_policy ;;
isolate) cmd_isolate "$@" ;;
enable-all) cmd_enable_all ;;
scenario-budget) cmd_scenario_budget ;;
scenario-budget-rule) cmd_scenario_budget_rule ;;
scenario-user-cap) cmd_scenario_user_cap ;;
scenario-guardrail) cmd_scenario_guardrail ;;
scenario-streaming) cmd_scenario_streaming ;;
scenario-multigroup) cmd_scenario_multigroup ;;
scenario-logs) cmd_scenario_logs ;;
scenario-retention) cmd_scenario_retention ;;
scenario-providers) cmd_scenario_providers ;;
scenario-vertex) cmd_scenario_vertex ;;
scenario-bedrock) cmd_scenario_bedrock ;;
dashboard) cmd_dashboard ;;
scenarios) cmd_scenarios ;;
down) cmd_down ;;
all) cmd_all ;;
*) cat >&2 <<'USAGE'
usage: bash scripts/e2e/agent-network-full/e2e.sh <cmd>
setup/flow : snapshot key up status wait diag restart-proxy clean providers policy isolate NAME enable-all down all
traffic : chat MODEL [chat|messages] verify
scenarios : scenarios | scenario-budget | scenario-budget-rule | scenario-user-cap |
scenario-guardrail | scenario-streaming | scenario-multigroup |
scenario-logs | scenario-retention | scenario-providers | scenario-vertex |
scenario-bedrock
dashboard : dashboard (Playwright UI check against the live :3000 dashboard)
USAGE
exit 2 ;;
esac

105
scripts/e2e/agent-network-policy/00-env.sh Executable file
View File

@@ -0,0 +1,105 @@
# shellcheck disable=SC2148
# Sourced helper for the agent-network-policy e2e suite.
# Verifies the agent-network surface: window_seconds (post-rename)
# through the API, /api/agent-network/consumption read endpoint, and
# the CheckLLMPolicyLimits + RecordLLMUsage gRPC RPCs.
#
# Do not run directly.
set -euo pipefail
: "${NB_API:=http://localhost:8080}"
: "${NB_PAT_FILE:=/Users/maycon/projects/local-dev/nb-pat}"
: "${NB_TIMEOUT_SECONDS:=60}"
: "${NB_POLICY_NAME:=e2e-anpol}"
: "${NB_PROVIDER_NAME:=e2e-anpol-provider}"
: "${NB_GROUP_NAME:=e2e-anpol-engineers}"
: "${NB_GRPC_ADDR:=localhost:8080}"
# Proxy token shared with the tilt setup. Keep in sync with the
# NB_PROXY_TOKEN literal in /Users/maycon/projects/local-dev/Tiltfile;
# the management server accepts it as a registered proxy credential
# so the e2e binary can reach the proxy_service gRPC surface.
: "${NB_PROXY_TOKEN:=nbx_MEF9OKRhlLrWkc5TJmM3Eu2rhqigaP2yulHy}"
: "${NB_STATE_DIR:=/tmp/nb-anpol-e2e-state}"
mkdir -p "$NB_STATE_DIR"
if [ ! -r "$NB_PAT_FILE" ]; then
echo "FAIL: cannot read PAT at $NB_PAT_FILE" >&2
exit 2
fi
NB_PAT=$(tr -d '\n\r ' <"$NB_PAT_FILE")
if [ ${#NB_PAT} -lt 16 ]; then
echo "FAIL: PAT at $NB_PAT_FILE is suspiciously short (${#NB_PAT} chars)" >&2
exit 2
fi
if ! command -v jq >/dev/null 2>&1; then
echo "FAIL: jq is required (brew install jq)" >&2
exit 2
fi
# nb_api METHOD PATH [BODY] — wraps curl with PAT auth and JSON.
nb_api() {
local method="$1" path="$2" body="${3-}"
if [ -n "$body" ]; then
curl -fsS -X "$method" \
-H "Authorization: Token $NB_PAT" \
-H "Content-Type: application/json" \
--data "$body" \
"$NB_API$path"
else
curl -fsS -X "$method" \
-H "Authorization: Token $NB_PAT" \
"$NB_API$path"
fi
}
# nb_api_status METHOD PATH [BODY] — returns the HTTP status code
# rather than the body. Use for negative-path tests where a 4xx is
# the expected outcome and curl's default -f would mask the result.
nb_api_status() {
local method="$1" path="$2" body="${3-}"
if [ -n "$body" ]; then
curl -sS -o /dev/null -w '%{http_code}' -X "$method" \
-H "Authorization: Token $NB_PAT" \
-H "Content-Type: application/json" \
--data "$body" \
"$NB_API$path"
else
curl -sS -o /dev/null -w '%{http_code}' -X "$method" \
-H "Authorization: Token $NB_PAT" \
"$NB_API$path"
fi
}
# wait_for COND_CMD TIMEOUT_S — polls every 1s.
wait_for() {
local cmd="$1" timeout="${2:-$NB_TIMEOUT_SECONDS}"
local i=0
while [ "$i" -lt "$timeout" ]; do
if eval "$cmd" >/dev/null 2>&1; then
return 0
fi
sleep 1
i=$((i + 1))
done
return 1
}
# resolve the absolute path of the netbird repo root from the script
# location so the Go smoke binary can be invoked via `go run` against
# the right module regardless of the caller's cwd.
nb_repo_root() {
cd "$(dirname "$0")/../../.." && pwd
}
pass() {
printf 'PASS: %s\n' "$1"
exit 0
}
fail() {
printf 'FAIL: %s — %s\n' "$1" "${2:-}"
exit 1
}

44
scripts/e2e/agent-network-policy/01-tilt-restart.sh Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# 01-tilt-restart: re-trigger management + dashboard before each run so
# we start from a clean process state. The on-disk store survives, so
# the PAT, account, and any residual config persist across restarts.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
command -v tilt >/dev/null 2>&1 || fail "tilt not on PATH" "brew install tilt-dev/tap/tilt"
if ! curl -fsS -o /dev/null --max-time 2 http://localhost:10350/ 2>/dev/null; then
fail "Tilt API not reachable at http://localhost:10350" "is 'tilt up' running in /Users/maycon/projects/local-dev?"
fi
restart_one() {
local name="$1"
if ! tilt trigger "$name" >/dev/null 2>&1; then
fail "tilt trigger $name failed" ""
fi
echo "triggered $name"
}
restart_one management
restart_one dashboard
# proxy3 needs to come back too because PR2 wired the new
# llm_limit_check / llm_limit_record middlewares into the proxy
# binary; without restarting it, e2e keeps exercising whatever
# image was last loaded and silently misses any boot-order or
# wiring regression in the chain. Cheap to restart, expensive to
# silently miss.
restart_one proxy3
echo "waiting for management to accept requests..."
if ! wait_for "curl -fsS -o /dev/null --max-time 2 $NB_API/oauth2/.well-known/openid-configuration" 60; then
fail "management did not come back up within 60s" "check 'tilt logs management'"
fi
echo "management is up"
code=$(curl -fsS -o /dev/null -w '%{http_code}' \
-H "Authorization: Token $NB_PAT" \
"$NB_API/api/users" 2>&1) || true
[ "$code" = "200" ] || fail "PAT auth check failed after restart (HTTP $code)" "the PAT may have been revoked"
pass "Tilt resources restarted: management, dashboard"

132
scripts/e2e/agent-network-policy/10-policy-create.sh Executable file
View File

@@ -0,0 +1,132 @@
#!/usr/bin/env bash
# 10-policy-create: end-to-end round-trip on the new window_seconds
# field. Creates the prerequisites (group + provider with
# bootstrap_cluster), POSTs a policy whose Limits.token_limit and
# budget_limit both carry window_seconds, then re-fetches the policy
# and asserts the field comes back unchanged.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
# 1. Group — sweep first so re-runs are idempotent, then POST.
existing_group=$(nb_api GET /api/groups 2>/dev/null \
| jq -r --arg name "$NB_GROUP_NAME" '.[] | select(.name == $name) | .id // empty' \
| head -1)
if [ -n "$existing_group" ]; then
echo "reusing existing group $existing_group"
group_id="$existing_group"
else
body=$(jq -n --arg name "$NB_GROUP_NAME" '{name:$name}')
group_resp=$(nb_api POST /api/groups "$body")
group_id=$(printf '%s' "$group_resp" | jq -r '.id // ""')
[ -n "$group_id" ] && [ "$group_id" != "null" ] \
|| fail "group create did not return an id" "$group_resp"
echo "created group $group_id"
fi
printf '%s' "$group_id" >"$NB_STATE_DIR/group-id"
# 2. Provider — also idempotent. Bootstrap cluster pinned to the local
# proxy3 cluster so the management settings row resolves and the
# create completes (subsequent provider creates in the same account
# ignore bootstrap_cluster, but the FIRST one needs it).
existing_provider=$(nb_api GET /api/agent-network/providers 2>/dev/null \
| jq -r --arg name "$NB_PROVIDER_NAME" '.[] | select(.name == $name) | .id // empty' \
| head -1)
if [ -n "$existing_provider" ]; then
echo "reusing existing provider $existing_provider"
provider_id="$existing_provider"
else
provider_body=$(jq -n \
--arg name "$NB_PROVIDER_NAME" \
'{
provider_id: "openai_api",
name: $name,
upstream_url: "https://api.openai.com",
api_key: "sk-e2e-placeholder",
bootstrap_cluster: "proxy.netbird.local",
models: []
}')
prov_resp=$(nb_api POST /api/agent-network/providers "$provider_body")
provider_id=$(printf '%s' "$prov_resp" | jq -r '.id // ""')
[ -n "$provider_id" ] && [ "$provider_id" != "null" ] \
|| fail "provider create did not return an id" "$prov_resp"
echo "created provider $provider_id"
fi
printf '%s' "$provider_id" >"$NB_STATE_DIR/provider-id"
# 3. Policy — drop any prior with the same name, then create with the
# NEW window_seconds field on both halves of Limits. 86400s = 24h
# on token, 2_592_000s = 30d on budget so the round-trip is
# unambiguous (no ambiguous unit-conversion artefact when we read
# back).
existing_policy=$(nb_api GET /api/agent-network/policies 2>/dev/null \
| jq -r --arg name "$NB_POLICY_NAME" '.[] | select(.name == $name) | .id // empty' \
| head -1)
if [ -n "$existing_policy" ]; then
echo "deleting existing policy $existing_policy"
nb_api DELETE "/api/agent-network/policies/$existing_policy" >/dev/null 2>&1 || true
fi
policy_body=$(jq -n \
--arg name "$NB_POLICY_NAME" \
--arg group "$group_id" \
--arg provider "$provider_id" \
'{
name: $name,
description: "agent-network e2e: window_seconds round-trip",
enabled: true,
source_groups: [$group],
destination_provider_ids: [$provider],
guardrail_ids: [],
limits: {
token_limit: {
enabled: true,
group_cap: 10000,
user_cap: 5000,
window_seconds: 86400
},
budget_limit: {
enabled: true,
group_cap_usd: 10.0,
user_cap_usd: 2.5,
window_seconds: 2592000
}
}
}')
resp=$(nb_api POST /api/agent-network/policies "$policy_body")
policy_id=$(printf '%s' "$resp" | jq -r '.id // ""')
[ -n "$policy_id" ] && [ "$policy_id" != "null" ] \
|| fail "policy create did not return an id" "$resp"
printf '%s' "$policy_id" >"$NB_STATE_DIR/policy-id"
echo "policy id: $policy_id"
# 4. Round-trip assertions: GET the policy back and verify the
# window_seconds values land on both limit halves. The OLD
# window_hours / window_days fields must be absent.
got=$(nb_api GET "/api/agent-network/policies/$policy_id")
token_window=$(printf '%s' "$got" | jq -r '.limits.token_limit.window_seconds // empty')
budget_window=$(printf '%s' "$got" | jq -r '.limits.budget_limit.window_seconds // empty')
[ "$token_window" = "86400" ] \
|| fail "token_limit.window_seconds did not round-trip" \
"expected=86400 got=$token_window body=$got"
[ "$budget_window" = "2592000" ] \
|| fail "budget_limit.window_seconds did not round-trip" \
"expected=2592000 got=$budget_window body=$got"
# Negative: window_hours / window_days are legacy field names and
# must not be present in the response at all — their presence would
# mean the management server is still emitting the legacy shape.
legacy_h=$(printf '%s' "$got" | jq -r '.limits.token_limit | has("window_hours")')
legacy_d=$(printf '%s' "$got" | jq -r '.limits.token_limit | has("window_days")')
[ "$legacy_h" = "false" ] \
|| fail "legacy window_hours field still present in token_limit response" "$got"
[ "$legacy_d" = "false" ] \
|| fail "legacy window_days field still present in token_limit response" "$got"
echo "token_limit.window_seconds = $token_window"
echo "budget_limit.window_seconds = $budget_window"
pass "policy persisted with window_seconds on both Limits halves"

65
scripts/e2e/agent-network-policy/20-policy-rejects-zero-window.sh Executable file
View File

@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# 20-policy-rejects-zero-window: management must reject a policy whose
# token_limit or budget_limit is enabled but carries window_seconds < 60.
# Anything below the one-minute floor would either be a zero / negative
# window with no reset boundary, or a sub-minute window that produces
# untenable consumption-row volume at scale. The handler's
# validatePolicyLimits guard owes us a 4xx with a useful message.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
[ -r "$NB_STATE_DIR/group-id" ] && [ -r "$NB_STATE_DIR/provider-id" ] \
|| fail "missing group-id / provider-id state" \
"run 10-policy-create.sh first to bootstrap prerequisites"
group_id=$(cat "$NB_STATE_DIR/group-id")
provider_id=$(cat "$NB_STATE_DIR/provider-id")
# Build a payload that is well-formed except for window_seconds=30 on
# token_limit (under the 60s minimum). We mark the field enabled so
# the validation path actually runs — a disabled limit is allowed to
# carry zero.
payload=$(jq -n \
--arg name "$NB_POLICY_NAME-sub-minute" \
--arg group "$group_id" \
--arg provider "$provider_id" \
'{
name: $name,
enabled: true,
source_groups: [$group],
destination_provider_ids: [$provider],
guardrail_ids: [],
limits: {
token_limit: {
enabled: true,
group_cap: 10000,
user_cap: 5000,
window_seconds: 30
},
budget_limit: {
enabled: false,
group_cap_usd: 0,
user_cap_usd: 0,
window_seconds: 0
}
}
}')
code=$(nb_api_status POST /api/agent-network/policies "$payload")
[ "$code" = "400" ] || [ "$code" = "422" ] \
|| fail "expected 400/422 on enabled token_limit with window_seconds<60" \
"got HTTP $code"
# Sweep any policy that may have been mistakenly persisted (defence
# against a future bug; today's handler doesn't get there).
orphan=$(nb_api GET /api/agent-network/policies 2>/dev/null \
| jq -r --arg name "$NB_POLICY_NAME-sub-minute" '.[] | select(.name == $name) | .id // empty' \
| head -1)
if [ -n "$orphan" ]; then
nb_api DELETE "/api/agent-network/policies/$orphan" >/dev/null 2>&1 || true
fail "policy was persisted despite sub-minute window_seconds" \
"id=$orphan — handler validation regression"
fi
echo "POST with token_limit.window_seconds=30 rejected with HTTP $code"
pass "validation rejects sub-minute window_seconds when limit is enabled"

24
scripts/e2e/agent-network-policy/30-consumption-list-empty.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
# 30-consumption-list-empty: GET /api/agent-network/consumption must
# return a JSON array (possibly empty) — never a 404 / 500. The
# endpoint is the read side that backs the dashboard's basic counter
# view and must always be reachable so the page can render an empty
# state.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
resp=$(nb_api GET /api/agent-network/consumption)
# Must be a JSON array.
if ! printf '%s' "$resp" | jq -e 'type == "array"' >/dev/null 2>&1; then
fail "consumption endpoint did not return a JSON array" "$resp"
fi
count=$(printf '%s' "$resp" | jq 'length')
echo "consumption rows: $count"
# Stash the baseline count so 40-grpc-record-and-list can compare.
printf '%s' "$count" >"$NB_STATE_DIR/consumption-baseline"
pass "consumption read endpoint returns a JSON array (count=$count)"

133
scripts/e2e/agent-network-policy/40-grpc-record-and-list.sh Executable file
View File

@@ -0,0 +1,133 @@
#!/usr/bin/env bash
# 40-grpc-record-and-list: drives the new RecordLLMUsage and
# CheckLLMPolicyLimits gRPC RPCs through the e2e usage_smoke helper.
#
# Asserts:
# - CheckLLMPolicyLimits returns decision=allow, picks the lowest
# group id by string sort as attribution, default window of 24h.
# - RecordLLMUsage with both user_id and group_id ticks BOTH the
# user counter and the group counter exactly once each.
# - A second RecordLLMUsage on the same key sums the deltas server
# side (database upsert-increment, no read-modify-write race).
# - The HTTP /api/agent-network/consumption listing reflects the
# post-flight state.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
[ -r "$NB_STATE_DIR/group-id" ] && [ -r "$NB_STATE_DIR/provider-id" ] \
|| fail "missing group-id / provider-id state" \
"run 10-policy-create.sh first to bootstrap prerequisites"
group_id=$(cat "$NB_STATE_DIR/group-id")
provider_id=$(cat "$NB_STATE_DIR/provider-id")
# Resolve the calling account id. /api/accounts returns NetBird xids,
# which is what the gRPC service expects (NOT IDP user UUIDs).
account_id=$(nb_api GET /api/accounts 2>/dev/null | jq -r '.[0].id // empty')
[ -n "$account_id" ] || fail "could not resolve account id" "GET /api/accounts returned nothing"
# Pick a stable user id for the test. The dimension is per-user, so we
# can use any unique value — a synthetic "e2e-..." prefix avoids
# colliding with real user ids in the consumption listing.
user_id="e2e-anpol-user-$$"
window_seconds=86400
echo "account=$account_id user=$user_id group=$group_id window=${window_seconds}s"
# ─── 1. CheckLLMPolicyLimits ────────────────────────────────────────
# Pass 3 group ids out of order; lowest by string sort wins.
groups_csv="grp-zz,$group_id,grp-aa-z"
check_resp=$(cd "$(nb_repo_root)" && go run ./scripts/e2e/agent-network-policy/cmd/usage_smoke check \
--account "$account_id" \
--user "$user_id" \
--groups "$groups_csv" \
--provider "$provider_id" \
--model "gpt-4o" \
--token "$NB_PROXY_TOKEN" \
--addr "$NB_GRPC_ADDR" 2>&1) \
|| fail "CheckLLMPolicyLimits gRPC failed" "$check_resp"
decision=$(printf '%s' "$check_resp" | jq -r '.decision // ""')
attribution=$(printf '%s' "$check_resp" | jq -r '.attribution_group_id // ""')
got_window=$(printf '%s' "$check_resp" | jq -r '.window_seconds // ""')
[ "$decision" = "allow" ] \
|| fail "Check decision must be allow under PR1 stub" "$check_resp"
# Lowest group id of {grp-zz, $group_id, grp-aa-z} by string sort. The
# group_id is an xid (lowercase alnum starting with a digit/letter), so
# the sort answer depends on group_id's prefix. Compute it locally.
expected_low=$(printf '%s\n%s\n%s' "grp-zz" "$group_id" "grp-aa-z" | sort | head -1)
[ "$attribution" = "$expected_low" ] \
|| fail "Check did not pick the lowest-by-sort group" \
"expected=$expected_low got=$attribution"
[ "$got_window" = "86400" ] \
|| fail "Check window_seconds stub default mismatch" "expected=86400 got=$got_window"
echo "Check decision=$decision attribution=$attribution window=${got_window}s"
# ─── 2. RecordLLMUsage — first increment ────────────────────────────
(cd "$(nb_repo_root)" && go run ./scripts/e2e/agent-network-policy/cmd/usage_smoke record \
--account "$account_id" \
--user "$user_id" \
--group "$group_id" \
--window-seconds "$window_seconds" \
--tokens-in 100 \
--tokens-out 50 \
--cost-usd 0.0125 \
--token "$NB_PROXY_TOKEN" \
--addr "$NB_GRPC_ADDR") >/dev/null \
|| fail "RecordLLMUsage (first increment) failed" ""
# ─── 3. RecordLLMUsage — second increment, same key ─────────────────
(cd "$(nb_repo_root)" && go run ./scripts/e2e/agent-network-policy/cmd/usage_smoke record \
--account "$account_id" \
--user "$user_id" \
--group "$group_id" \
--window-seconds "$window_seconds" \
--tokens-in 50 \
--tokens-out 25 \
--cost-usd 0.0025 \
--token "$NB_PROXY_TOKEN" \
--addr "$NB_GRPC_ADDR") >/dev/null \
|| fail "RecordLLMUsage (second increment) failed" ""
# ─── 4. Read-back via HTTP — sums must converge ─────────────────────
listing=$(nb_api GET /api/agent-network/consumption)
user_row=$(printf '%s' "$listing" | jq --arg u "$user_id" \
'map(select(.dimension_kind == "user" and .dimension_id == $u)) | .[0]')
[ "$(printf '%s' "$user_row" | jq -r '. // empty')" != "" ] \
|| fail "user consumption row missing after RecordLLMUsage" "$listing"
user_in=$(printf '%s' "$user_row" | jq '.tokens_input')
user_out=$(printf '%s' "$user_row" | jq '.tokens_output')
user_cost=$(printf '%s' "$user_row" | jq '.cost_usd')
user_window=$(printf '%s' "$user_row" | jq '.window_seconds')
[ "$user_in" = "150" ] && [ "$user_out" = "75" ] \
|| fail "user counter did not sum the two increments" \
"expected tokens_in=150 tokens_out=75; got in=$user_in out=$user_out"
# Floating point compare with awk — drift > 1e-9 is a real bug.
cost_ok=$(awk -v got="$user_cost" 'BEGIN { print (got > 0.0149 && got < 0.0151) ? "y" : "n" }')
[ "$cost_ok" = "y" ] \
|| fail "user cost did not sum to 0.015" "got=$user_cost"
[ "$user_window" = "$window_seconds" ] \
|| fail "user counter window_seconds mismatch" \
"expected=$window_seconds got=$user_window"
# Group row gets the same deltas because RecordLLMUsage ticks both
# dimensions on a single call.
group_row=$(printf '%s' "$listing" | jq --arg g "$group_id" \
'map(select(.dimension_kind == "group" and .dimension_id == $g)) | .[0]')
group_in=$(printf '%s' "$group_row" | jq '.tokens_input')
group_out=$(printf '%s' "$group_row" | jq '.tokens_output')
[ "$group_in" = "150" ] && [ "$group_out" = "75" ] \
|| fail "group counter did not sum the two increments" \
"expected tokens_in=150 tokens_out=75; got in=$group_in out=$group_out"
echo "user counter: $user_in input / $user_out output / \$$user_cost over ${user_window}s"
echo "group counter: $group_in input / $group_out output"
pass "gRPC Check + Record round-trip atomically increments user + group counters"

210
scripts/e2e/agent-network-policy/50-grpc-allow-record-deny.sh Executable file
View File

@@ -0,0 +1,210 @@
#!/usr/bin/env bash
# 50-grpc-allow-record-deny: drives the full lifecycle of the real
# selection algorithm landed in PR2 — initial allow, partial fills
# below the cap, exact-at-cap, then deny once consumption reaches the
# group_cap. Validates that:
#
# - CheckLLMPolicyLimits picks the test policy as the attribution
# when it's the only one authorising the (group, provider) pair.
# - The selected_policy_id + window_seconds round-trip on the wire.
# - Counter-aware headroom math is wired end-to-end (RecordLLMUsage
# ticks counters that the next CheckLLMPolicyLimits call reads).
# - At-cap consumption flips decision from allow to deny with the
# canonical llm_policy.token_cap_exceeded code.
#
# The setup creates a dedicated group + provider + policy with the
# "-tight" suffix so it's isolated from the e2e-anpol resources
# 10-policy-create.sh seeded. 99-cleanup's prefix sweep catches both.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
# Per-run suffix so the consumption counter starts at zero on every
# run. Without this, a successful prior run leaves 200 tokens on the
# (group, 24h) bucket and the next run's stage 1 check sees a deny
# from the start. Consumption rows have no delete endpoint today.
RUN_TAG="$(date +%s)-$$"
NB_TIGHT_GROUP_NAME="$NB_GROUP_NAME-tight-$RUN_TAG"
NB_TIGHT_PROVIDER_NAME="$NB_PROVIDER_NAME-tight-$RUN_TAG"
NB_TIGHT_POLICY_NAME="$NB_POLICY_NAME-tight-$RUN_TAG"
TIGHT_CAP=200
TIGHT_WINDOW_SECONDS=86400
# 1. Group — fresh per-run so the (group, window) consumption counter
# starts at zero. The 99-cleanup sweep catches it via the
# `e2e-anpol-engineers-tight-` prefix.
body=$(jq -n --arg name "$NB_TIGHT_GROUP_NAME" '{name:$name}')
tight_group_id=$(nb_api POST /api/groups "$body" | jq -r '.id // ""')
[ -n "$tight_group_id" ] && [ "$tight_group_id" != "null" ] \
|| fail "tight group create failed" ""
echo "created tight group $tight_group_id"
printf '%s' "$tight_group_id" >"$NB_STATE_DIR/tight-group-id"
# 2. Provider — fresh per-run too so the policy targets a provider
# only this run knows about. Eliminates cross-policy interference if
# a prior run's tight policy targeted the same provider id.
body=$(jq -n \
--arg name "$NB_TIGHT_PROVIDER_NAME" \
'{
provider_id: "openai_api",
name: $name,
upstream_url: "https://api.openai.com",
api_key: "sk-e2e-tight-placeholder",
bootstrap_cluster: "proxy.netbird.local",
models: []
}')
tight_provider_id=$(nb_api POST /api/agent-network/providers "$body" | jq -r '.id // ""')
[ -n "$tight_provider_id" ] && [ "$tight_provider_id" != "null" ] \
|| fail "tight provider create failed" ""
echo "created tight provider $tight_provider_id"
printf '%s' "$tight_provider_id" >"$NB_STATE_DIR/tight-provider-id"
# 3. Policy with a tight token cap so the test can observe the deny
# transition without burning thousands of records.
existing_policy=$(nb_api GET /api/agent-network/policies 2>/dev/null \
| jq -r --arg name "$NB_TIGHT_POLICY_NAME" '.[] | select(.name == $name) | .id // empty' \
| head -1)
if [ -n "$existing_policy" ]; then
echo "deleting existing tight policy $existing_policy so the run starts clean"
nb_api DELETE "/api/agent-network/policies/$existing_policy" >/dev/null 2>&1 || true
fi
policy_body=$(jq -n \
--arg name "$NB_TIGHT_POLICY_NAME" \
--arg group "$tight_group_id" \
--arg provider "$tight_provider_id" \
--argjson cap "$TIGHT_CAP" \
--argjson window "$TIGHT_WINDOW_SECONDS" \
'{
name: $name,
description: "agent-network e2e: cap-exhaust deny test",
enabled: true,
source_groups: [$group],
destination_provider_ids: [$provider],
guardrail_ids: [],
limits: {
token_limit: {
enabled: true,
group_cap: $cap,
user_cap: 0,
window_seconds: $window
},
budget_limit: {
enabled: false,
group_cap_usd: 0,
user_cap_usd: 0,
window_seconds: $window
}
}
}')
tight_policy_id=$(nb_api POST /api/agent-network/policies "$policy_body" | jq -r '.id // ""')
[ -n "$tight_policy_id" ] && [ "$tight_policy_id" != "null" ] \
|| fail "tight policy create failed" ""
printf '%s' "$tight_policy_id" >"$NB_STATE_DIR/tight-policy-id"
echo "created tight policy $tight_policy_id (group_cap=$TIGHT_CAP, window=${TIGHT_WINDOW_SECONDS}s)"
# 4. Resolve the calling account id; the smoke binary stamps it onto
# every gRPC request the way the real proxy does.
account_id=$(nb_api GET /api/accounts 2>/dev/null | jq -r '.[0].id // empty')
[ -n "$account_id" ] || fail "could not resolve account id" ""
# Test user — synthetic prefix avoids colliding with real users.
test_user="e2e-anpol-tight-user-$$"
# Pre-resolve the netbird repo root ONCE so the helpers below can
# pushd / popd into it without `cd "$(dirname "$0")/.."` re-resolving
# from a moved cwd between calls. nb_repo_root walks up from $0; if a
# prior helper left cwd elsewhere the relative walk breaks.
repo_root=$(nb_repo_root)
[ -d "$repo_root" ] || fail "could not resolve netbird repo root" "got=$repo_root"
# Run the smoke binary inside a subshell so its `cd` doesn't pollute
# the caller's cwd between stages.
do_check() {
(
cd "$repo_root" || exit 1
go run ./scripts/e2e/agent-network-policy/cmd/usage_smoke check \
--account "$account_id" \
--user "$test_user" \
--groups "$tight_group_id" \
--provider "$tight_provider_id" \
--model "gpt-4o" \
--token "$NB_PROXY_TOKEN" \
--addr "$NB_GRPC_ADDR"
)
}
do_record() {
local in="$1" out="$2" cost="$3"
(
cd "$repo_root" || exit 1
go run ./scripts/e2e/agent-network-policy/cmd/usage_smoke record \
--account "$account_id" \
--user "$test_user" \
--group "$tight_group_id" \
--window-seconds "$TIGHT_WINDOW_SECONDS" \
--tokens-in "$in" \
--tokens-out "$out" \
--cost-usd "$cost" \
--token "$NB_PROXY_TOKEN" \
--addr "$NB_GRPC_ADDR"
)
}
# Stage 1 — fresh state. Selection must allow and pick our tight
# policy as attribution because it's the ONLY policy authorising the
# (tight_group, tight_provider) tuple. Also confirms the wire-level
# selected_policy_id round-trip from manager → grpc → smoke client.
echo "stage 1: initial check (consumption=0/$TIGHT_CAP)"
resp=$(do_check) || fail "initial check failed" "$resp"
decision=$(printf '%s' "$resp" | jq -r '.decision // ""')
selected=$(printf '%s' "$resp" | jq -r '.selected_policy_id // ""')
window=$(printf '%s' "$resp" | jq -r '.window_seconds // ""')
[ "$decision" = "allow" ] \
|| fail "expected allow on fresh state" "$resp"
[ "$selected" = "$tight_policy_id" ] \
|| fail "selection did not pick the tight policy" \
"expected=$tight_policy_id got=$selected"
[ "$window" = "$TIGHT_WINDOW_SECONDS" ] \
|| fail "window_seconds mismatch on the wire" "expected=$TIGHT_WINDOW_SECONDS got=$window"
echo " → allow / selected=$selected / window=${window}s"
# Stage 2 — book half the cap. The next check must still allow.
echo "stage 2: record 100 input tokens (consumption=100/$TIGHT_CAP)"
do_record 100 0 0 >/dev/null || fail "record (stage 2) failed" ""
resp=$(do_check)
decision=$(printf '%s' "$resp" | jq -r '.decision // ""')
[ "$decision" = "allow" ] \
|| fail "expected allow at half-cap" "$resp"
echo " → allow"
# Stage 3 — push to one token below cap. Headroom shrinks but
# decision stays allow.
echo "stage 3: record 99 more input tokens (consumption=199/$TIGHT_CAP)"
do_record 99 0 0 >/dev/null || fail "record (stage 3) failed" ""
resp=$(do_check)
decision=$(printf '%s' "$resp" | jq -r '.decision // ""')
[ "$decision" = "allow" ] \
|| fail "expected allow at one-below-cap" "$resp"
echo " → allow"
# Stage 4 — exactly at cap. Selector treats consumed >= cap as
# exhausted, so the next check must deny with the canonical token cap
# code. The deny reason names the policy id so operators can debug
# from the access log.
echo "stage 4: record 1 final token (consumption=$TIGHT_CAP/$TIGHT_CAP)"
do_record 1 0 0 >/dev/null || fail "record (stage 4) failed" ""
resp=$(do_check) || fail "check after cap-exhaust failed" "$resp"
decision=$(printf '%s' "$resp" | jq -r '.decision // ""')
deny_code=$(printf '%s' "$resp" | jq -r '.deny_code // ""')
deny_reason=$(printf '%s' "$resp" | jq -r '.deny_reason // ""')
[ "$decision" = "deny" ] \
|| fail "expected DENY at cap" "$resp"
[ "$deny_code" = "llm_policy.token_cap_exceeded" ] \
|| fail "deny_code mismatch" "expected=llm_policy.token_cap_exceeded got=$deny_code resp=$resp"
echo " → DENY / deny_code=$deny_code"
echo " → deny_reason: $deny_reason"
[[ "$deny_reason" == *"$tight_policy_id"* ]] \
|| fail "deny_reason must name the policy id for operator debugging" "$deny_reason"
pass "selection algorithm flips allow → deny at cap-exhaust through the gRPC wire"

58
scripts/e2e/agent-network-policy/99-cleanup.sh Executable file
View File

@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# 99-cleanup: idempotent teardown. Drops the policy, provider, group,
# and any consumption rows our test created so re-runs start fresh.
# shellcheck source=00-env.sh
source "$(dirname "$0")/00-env.sh"
# Policies first (FK / synth chain owns provider references). Sweep
# every policy whose name STARTS with the e2e prefix so per-run
# instances created by 50 (with $RUN_TAG suffixes) get cleaned up.
nb_api GET /api/agent-network/policies 2>/dev/null \
| jq -r --arg pfx "$NB_POLICY_NAME" '.[] | select(.name | startswith($pfx)) | .id' \
| while read -r orphan; do
[ -n "$orphan" ] || continue
code=$(curl -fsS -X DELETE -H "Authorization: Token $NB_PAT" \
-o /dev/null -w '%{http_code}' \
"$NB_API/api/agent-network/policies/$orphan" 2>&1) || true
echo "DELETE policy/$orphan -> $code"
done
# Providers — same prefix sweep. Both the 10-policy-create.sh's main
# provider AND every per-run tight provider 50 minted carry the
# NB_PROVIDER_NAME prefix.
nb_api GET /api/agent-network/providers 2>/dev/null \
| jq -r --arg pfx "$NB_PROVIDER_NAME" '.[] | select(.name | startswith($pfx)) | .id' \
| while read -r orphan; do
[ -n "$orphan" ] || continue
code=$(curl -fsS -X DELETE -H "Authorization: Token $NB_PAT" \
-o /dev/null -w '%{http_code}' \
"$NB_API/api/agent-network/providers/$orphan" 2>&1) || true
echo "DELETE provider/$orphan -> $code"
done
# Groups — prefix sweep. /api/groups doesn't error on a
# referenced-elsewhere group; if so, the delete is a no-op and we
# move on.
nb_api GET /api/groups 2>/dev/null \
| jq -r --arg pfx "$NB_GROUP_NAME" '.[] | select(.name | startswith($pfx)) | .id' \
| while read -r orphan; do
[ -n "$orphan" ] || continue
code=$(curl -fsS -X DELETE -H "Authorization: Token $NB_PAT" \
-o /dev/null -w '%{http_code}' \
"$NB_API/api/groups/$orphan" 2>&1) || true
echo "DELETE group/$orphan -> $code"
done
# We don't expose a consumption-delete endpoint — the rows survive
# until the management store is recycled. Document the residue here
# so anyone debugging "why are there old e2e rows" knows the source.
remaining=$(nb_api GET /api/agent-network/consumption 2>/dev/null \
| jq --arg pfx "e2e-anpol-user-" 'map(select(.dimension_id | startswith($pfx))) | length' \
|| echo "0")
[ "$remaining" = "0" ] || \
echo "(left $remaining e2e consumption rows in the store — there's no delete endpoint yet)"
rm -rf "$NB_STATE_DIR"
pass "tear-down complete (idempotent)"

197
scripts/e2e/agent-network-policy/cmd/usage_smoke/main.go Normal file
View File

@@ -0,0 +1,197 @@
// usage_smoke is the e2e helper that drives the new agent-network
// gRPC RPCs (CheckLLMPolicyLimits, RecordLLMUsage) against a local
// management server. Run from the bash suite as a `go run` so the
// proto types are always in sync with the management binary the
// suite is exercising.
//
// Two subcommands today:
// - record: invokes RecordLLMUsage with the supplied tokens / cost
// and exits 0 on success.
// - check: invokes CheckLLMPolicyLimits and prints the response as
// JSON on stdout so the bash test can assert on it via
// jq.
//
// Auth uses the same proxy bearer-token shape the real proxy uses
// (see proxy/internal/grpc/auth.go); the bash suite reads the token
// from the e2e env (NB_PROXY_TOKEN, defaulted to the Tilt literal).
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"os"
"strings"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"github.com/netbirdio/netbird/shared/management/proto"
)
// proxyTokenCreds mirrors proxy/internal/grpc.WithProxyToken's
// PerRPCCredentials so the e2e binary can reach the gRPC service
// using the same bearer-token shape the real proxy uses. Inlined
// because the production helper lives behind /internal/.
type proxyTokenCreds struct{ token string }
func (c proxyTokenCreds) GetRequestMetadata(context.Context, ...string) (map[string]string, error) {
return map[string]string{"authorization": "Bearer " + c.token}, nil
}
// RequireTransportSecurity is false here because Tilt's management is
// plaintext on localhost — the e2e suite is the *only* caller of this
// binary, never production.
func (proxyTokenCreds) RequireTransportSecurity() bool { return false }
func main() {
if len(os.Args) < 2 {
usage()
}
cmd := os.Args[1]
os.Args = append(os.Args[:1], os.Args[2:]...)
switch cmd {
case "record":
runRecord()
case "check":
runCheck()
default:
usage()
}
}
func usage() {
fmt.Fprintln(os.Stderr, "usage: usage_smoke <record|check> [flags]")
os.Exit(2)
}
func runRecord() {
addr := flag.String("addr", "localhost:8080", "management gRPC address")
token := flag.String("token", os.Getenv("NB_PROXY_TOKEN"), "proxy token")
accountID := flag.String("account", "", "netbird account id")
userID := flag.String("user", "", "netbird user id (optional)")
groupID := flag.String("group", "", "netbird policy attribution group id (optional)")
groupsCSV := flag.String("groups", "", "CSV of caller group ids (for account-rule fan-out)")
windowSeconds := flag.Int64("window-seconds", 86_400, "window length in seconds (0 allowed when only account rules apply)")
tokensIn := flag.Int64("tokens-in", 0, "input tokens to add")
tokensOut := flag.Int64("tokens-out", 0, "output tokens to add")
costUSD := flag.Float64("cost-usd", 0, "USD cost to add")
flag.Parse()
if strings.TrimSpace(*token) == "" {
exitErr("--token is required (or set NB_PROXY_TOKEN)")
}
if strings.TrimSpace(*accountID) == "" {
exitErr("--account is required")
}
var groupIDs []string
for _, g := range strings.Split(*groupsCSV, ",") {
g = strings.TrimSpace(g)
if g != "" {
groupIDs = append(groupIDs, g)
}
}
if *userID == "" && *groupID == "" && len(groupIDs) == 0 {
exitErr("at least one of --user, --group, or --groups must be set")
}
conn := dial(*addr, *token)
defer conn.Close()
client := proto.NewProxyServiceClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_, err := client.RecordLLMUsage(ctx, &proto.RecordLLMUsageRequest{
AccountId: *accountID,
UserId: *userID,
GroupId: *groupID,
GroupIds: groupIDs,
WindowSeconds: *windowSeconds,
TokensInput: *tokensIn,
TokensOutput: *tokensOut,
CostUsd: *costUSD,
})
if err != nil {
exitErr(fmt.Sprintf("RecordLLMUsage: %v", err))
}
//nolint:forbidigo // e2e helper: stdout is the contract with the bash caller
fmt.Println("ok")
}
func runCheck() {
addr := flag.String("addr", "localhost:8080", "management gRPC address")
token := flag.String("token", os.Getenv("NB_PROXY_TOKEN"), "proxy token")
accountID := flag.String("account", "", "netbird account id")
userID := flag.String("user", "", "netbird user id (optional)")
groupsCSV := flag.String("groups", "", "CSV of caller group ids")
providerID := flag.String("provider", "", "agent-network provider id")
model := flag.String("model", "gpt-4o", "upstream model identifier")
flag.Parse()
if strings.TrimSpace(*token) == "" {
exitErr("--token is required (or set NB_PROXY_TOKEN)")
}
if strings.TrimSpace(*accountID) == "" {
exitErr("--account is required")
}
var groupIDs []string
for _, g := range strings.Split(*groupsCSV, ",") {
g = strings.TrimSpace(g)
if g != "" {
groupIDs = append(groupIDs, g)
}
}
conn := dial(*addr, *token)
defer conn.Close()
client := proto.NewProxyServiceClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
resp, err := client.CheckLLMPolicyLimits(ctx, &proto.CheckLLMPolicyLimitsRequest{
AccountId: *accountID,
UserId: *userID,
GroupIds: groupIDs,
ProviderId: *providerID,
Model: *model,
})
if err != nil {
exitErr(fmt.Sprintf("CheckLLMPolicyLimits: %v", err))
}
out, _ := json.Marshal(map[string]any{
"decision": resp.GetDecision(),
"selected_policy_id": resp.GetSelectedPolicyId(),
"attribution_group_id": resp.GetAttributionGroupId(),
"window_seconds": resp.GetWindowSeconds(),
"deny_code": resp.GetDenyCode(),
"deny_reason": resp.GetDenyReason(),
})
//nolint:forbidigo // e2e helper: stdout is the contract with the bash caller
fmt.Println(string(out))
}
// dial connects to the management gRPC over plaintext. The bearer
// token is sent on every RPC via PerRPCCredentials matching the wire
// format the production proxy uses.
func dial(addr, token string) *grpc.ClientConn {
conn, err := grpc.NewClient(addr,
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithPerRPCCredentials(proxyTokenCreds{token: token}),
)
if err != nil {
exitErr(fmt.Sprintf("dial %s: %v", addr, err))
}
return conn
}
func exitErr(msg string) {
fmt.Fprintln(os.Stderr, msg)
os.Exit(1)
}

44
scripts/e2e/agent-network-policy/run-all.sh Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# run-all: every numbered test in order, halts on first FAIL,
# prints a `=== PASS x/y ===` summary. -k keeps state (skips
# 99-cleanup) so you can poke around the persisted policy /
# consumption rows afterwards.
set -uo pipefail
KEEP=0
[ "${1:-}" = "-k" ] && KEEP=1
export NB_KEEP_STATE="$KEEP"
cd "$(dirname "$0")"
scripts=(
01-tilt-restart.sh
10-policy-create.sh
20-policy-rejects-zero-window.sh
30-consumption-list-empty.sh
40-grpc-record-and-list.sh
50-grpc-allow-record-deny.sh
)
[ "$KEEP" -eq 0 ] && scripts+=(99-cleanup.sh)
passed=0
total=${#scripts[@]}
trap '[ "$KEEP" -eq 0 ] && bash ./99-cleanup.sh >/dev/null 2>&1 || true' EXIT
for s in "${scripts[@]}"; do
echo
echo "==================== $s ===================="
if bash "./$s"; then
passed=$((passed + 1))
else
rc=$?
echo
echo "=== FAIL $passed/$total ($s exit=$rc) ==="
exit "$rc"
fi
done
echo
echo "=== PASS $passed/$total ==="