netbird/docs/agent-networks/60-qa-test-cases.md

# Agent Network — QA test cases

A manual/automatable test-case catalogue for the Agent Network feature,
covering the management API (`management/internals/modules/agentnetwork`),
the synthesised reverse-proxy runtime, and the dashboard surface
(`netbirdio/dashboard`, `src/modules/agent-network` + `src/app/(dashboard)/agent-network`).

Companion reading: [00-overview.md](00-overview.md) for the feature map,
[01-end-to-end-flows.md](01-end-to-end-flows.md) for the request lifecycle,
[modules/40-dashboard.md](modules/40-dashboard.md) for the UI.

---

## 1. Scope and conventions

**In scope:** providers, policies, guardrails, account budget rules,
agent-network settings, endpoint bootstrap, request authorisation and routing,
limit enforcement, cost/usage accounting, access logs and sessions, log/prompt
collection, RBAC, the dashboard surfaces and the Agent Network onboarding.

**Out of scope (covered elsewhere):** generic reverse-proxy cluster
provisioning, WireGuard connectivity, IdP/SSO login, billing.

**Priorities**

| Level | Meaning |
|---|---|
| **P0** | Blocks release. Core happy path, data isolation, security, money/limits correctness. |
| **P1** | Important. Validation, secondary flows, most negative cases. |
| **P2** | Nice to have. Cosmetic, rare edge cases, resilience. |

**HTTP status conventions used below** (from `shared/management/http/util/util.go`):

| Condition | Status |
|---|---|
| Validation failure (`status.InvalidArgument`) | **422** Unprocessable Entity |
| Malformed JSON body | **400** Bad Request |
| Missing/insufficient permission | **403** Forbidden |
| Unknown id / not bootstrapped | **404** Not Found |

All endpoints are under `/api/agent-network/...`.

---

## 2. Environment prerequisites

| # | Requirement |
|---|---|
| E1 | A management deployment with at least one **validated FREE reverse-proxy cluster/domain** (the provider wizard only offers `type=FREE && validated=true`). Without one, provider create is blocked in the UI. |
| E2 | At least one proxy instance connected and serving the cluster. |
| E3 | Two NetBird accounts (**Account A**, **Account B**) for isolation tests. |
| E4 | In Account A: users `owner@`, `admin@`, `user@`, plus an `auditor`-role user if available. |
| E5 | Groups: `g-eng`, `g-contractors`, `g-none` (a group with no policy). Peers enrolled and assigned so caller group membership is deterministic. |
| E6 | At least one client machine running the NetBird agent and connected to the tunnel (traffic reaches the endpoint **only** over the tunnel). |
| E7 | Real or mocked upstream credentials for: OpenAI (`openai_api`), Anthropic (`anthropic_api`), one gateway (`litellm_proxy` or `portkey`), and — if testing path-routed providers — Bedrock and/or Vertex (`keyfile::<base64 GCP SA JSON>`). |
| E8 | A tool to issue raw HTTP (curl / Postman) plus at least one real agent (Claude Code or Codex) for the connect-snippet checks. |

**Reference data**

- Catalog ids: `openai_api`, `anthropic_api`, `azure_openai_api`, `bedrock_api`,
  `vertex_ai_api`, `mistral_api`, `kimi_api` (kind=provider); `litellm_proxy`,
  `portkey`, `bifrost`, `cloudflare_ai_gateway`, `vercel_ai_gateway`,
  `openrouter` (kind=gateway); `vllm`, `custom` (kind=custom).
- Deny codes: `llm_policy.token_cap_exceeded`, `llm_policy.budget_cap_exceeded`,
  `llm_account.token_cap_exceeded`, `llm_account.budget_cap_exceeded`,
  `llm_policy.model_blocked`.
- Permission modules: `agent_network.providers|policies|guardrails|budgets|usage|logs|settings`,
  with the `agent_network` parent cascading to all of them.

---

## 3. Bootstrap and account settings

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-BS-01** | Settings read before bootstrap returns defaults | P0 | Fresh account, no provider created | `GET /agent-network/settings` | 200. `cluster`, `subdomain`, `endpoint` all empty; `enable_log_collection=true`; `enable_prompt_collection=false`; `redact_pii=false`; `access_log_retention_days=30`; no `created_at`/`updated_at`. Nothing is persisted (repeat the GET — still no row). |
| **AN-BS-02** | First provider create bootstraps settings | P0 | AN-BS-01 | Create a provider with `bootstrap_cluster=<validated cluster>` | Provider created. `GET /agent-network/settings` now returns the pinned `cluster`, a generated word-based `subdomain`, and `endpoint = "<subdomain>.<cluster>"`. Timestamps present. |
| **AN-BS-03** | Subdomain is unique per cluster | P1 | Several accounts bootstrapped on the same cluster | Bootstrap 5+ accounts on one cluster; collect subdomains | All subdomains distinct; each is a readable word-label with a short account-id suffix. |
| **AN-BS-04** | Bootstrap hint ignored once settings exist | P1 | AN-BS-02 done | Create a second provider passing a *different* `bootstrap_cluster` | Provider created; settings `cluster`/`subdomain`/`endpoint` unchanged. |
| **AN-BS-05** | PUT settings bootstraps when a cluster is supplied | P1 | Fresh account | `PUT /agent-network/settings` with `cluster=<validated cluster>` and the toggles | 200; row created with that cluster and a generated subdomain; toggles applied. |
| **AN-BS-06** | PUT settings without cluster on un-bootstrapped account | P1 | Fresh account | `PUT /agent-network/settings` omitting `cluster` | 404, message names that settings have not been bootstrapped and that `cluster` (or a provider create with `bootstrap_cluster`) is required. |
| **AN-BS-07** | Cluster is immutable | P0 | Bootstrapped account | `PUT /agent-network/settings` with a *different* `cluster` | 422, message `cluster is immutable once assigned (current: <cluster>)`. Stored cluster unchanged. |
| **AN-BS-08** | PUT settings with the same cluster is accepted | P1 | Bootstrapped | `PUT` echoing the current cluster + changed toggles | 200; toggles updated; cluster/subdomain unchanged. |
| **AN-BS-09** | Subdomain can never be set by the client | P1 | Bootstrapped | `PUT` with a `subdomain` field in the body | Server value unchanged (field is server-assigned and ignored). |
| **AN-BS-10** | Collection toggles round-trip | P1 | Bootstrapped | Toggle `enable_log_collection`, `enable_prompt_collection`, `redact_pii`, and each retention option (7/14/30/60/90/0) | Each `PUT` returns the requested value; a subsequent `GET` matches; the dashboard **Configuration → Log Collection** tab reflects it after reload. |
| **AN-BS-11** | Settings change is audited | P2 | Bootstrapped | Change a toggle, then open **Activity/Events** | An `agent network settings updated` event exists with `log_collection`, `prompt_collection`, `redact_pii` in its metadata. |
| **AN-BS-12** | Concurrent settings PUTs don't interleave | P2 | Bootstrapped | Fire 10 concurrent `PUT`s with alternating toggle values | No 500s; final stored state equals one of the submitted payloads (no torn write); cluster unchanged. |

---

## 4. Providers — CRUD and validation

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-PR-01** | Catalog listing | P0 | Authenticated | `GET /agent-network/catalog/providers` | 200 with all catalog entries; each has `id`, `name`, `kind` (`provider`/`gateway`/`custom`), `default_host`, and models with `input_per_1k`/`output_per_1k` (+ cache rates where applicable). Rates match the **live** default pricing table, not stale compiled rates. |
| **AN-PR-02** | Create OpenAI provider (happy path) | P0 | Cluster available | POST with `provider_id=openai_api`, name, `upstream_url=https://api.openai.com`, `api_key`, models, `enabled=true` | 201/200; response contains the provider with `id`; **`api_key` is absent from the response**. |
| **AN-PR-03** | Create Anthropic provider | P0 | — | Same with `provider_id=anthropic_api`, `https://api.anthropic.com` | Created. |
| **AN-PR-04** | Create gateway provider with extra headers | P1 | — | Create `portkey` and fill `extra_values["x-portkey-config"]` | Created; `extra_values` round-trips on GET. |
| **AN-PR-05** | Extra values are restricted to catalog-declared keys | P1 | AN-PR-04 | Send an `extra_values` key the catalog doesn't declare (e.g. `x-bogus`) | Undeclared key is not persisted/returned. Empty-string values are dropped rather than stored. |
| **AN-PR-06** | `provider_id` is required and must be a known catalog id | P0 | — | POST with `provider_id=""`, then `provider_id="not_a_provider"` | 422 `provider_id is required`; 422 `provider_id "not_a_provider" is not a known catalog provider`. |
| **AN-PR-07** | `name` required | P1 | — | POST with blank/whitespace name | 422 `name is required`. |
| **AN-PR-08** | `upstream_url` validation | P0 | — | POST with: empty; `not-a-url`; `ftp://x`; `https://` (no host) | 422 in every case (`upstream_url is required` / `must be a full http(s) URL`). |
| **AN-PR-09** | `api_key` required on create | P0 | — | POST omitting `api_key`, then with `"   "` | 422 `api_key is required`. No provider row is created. |
| **AN-PR-10** | Model rate validation | P0 | — | POST models with: empty `id`; `input_per_1k = -1`; `output_per_1k = NaN`; `cached_input_per_1k = Inf` | 422 each; message names the index, model id and offending field (`models[0] (gpt-4o): input_per_1k must be a finite, non-negative USD rate`). |
| **AN-PR-11** | Empty model list means "all catalog models at catalog prices" | P1 | — | Create with `models: []`, then route a request for any catalog model | Request routes and is billed at the default catalog rate for that model. |
| **AN-PR-12** | Update preserves API key when omitted | P0 | Provider exists | `PUT` the provider without `api_key`, changing only the name | 200; name changed; upstream calls still succeed (stored key intact). |
| **AN-PR-13** | Blank API key on update is rejected | P0 | Provider exists | `PUT` with `api_key: "   "` | 422 `api_key must be non-blank when rotating an agent network provider`. Stored key untouched. |
| **AN-PR-14** | API key rotation | P0 | Provider exists, requests succeeding | `PUT` with a new valid key | 200; subsequent agent requests use the new key (verify with a deliberately wrong key → upstream 401 surfaced; then restore). |
| **AN-PR-15** | Session keypair survives updates | P1 | Provider exists | Update the provider several times | No error; the synthesised service keeps working (existing agent sessions are not broken). |
| **AN-PR-16** | `created_at` is preserved on update | P2 | Provider exists | Update it | `created_at` unchanged; `updated_at` advanced. |
| **AN-PR-17** | Disable a provider | P0 | Provider + policy exist, traffic flowing | Set `enabled=false` | Requests targeting that provider stop being routed. If it was the only enabled provider, the whole endpoint stops serving (see AN-SY-01). |
| **AN-PR-18** | Delete blocked while referenced by a policy | P0 | Policy references the provider | `DELETE /agent-network/providers/{id}` | 422 `provider is in use by N policy/policies (<names>); detach it before deleting`. Provider still present. Message pluralises correctly for 1 vs many. |
| **AN-PR-19** | Delete after detaching | P0 | AN-PR-18, then remove it from the policy | `DELETE` again | 200; provider gone from `GET /agent-network/providers`; the synthesised routing table no longer offers its models. |
| **AN-PR-20** | Unknown provider id | P1 | — | `GET`/`PUT`/`DELETE` with a random id | 404. |
| **AN-PR-21** | `skip_tls_verification` | P1 | A self-hosted upstream with a self-signed cert (`vllm`/`custom`) | Create with the flag off → request; then on → request | Off: upstream dial fails on TLS. On: request succeeds. |
| **AN-PR-22** | `metadata_disabled` | P1 | Gateway provider that receives identity metadata | Create with `metadata_disabled=false`, inspect upstream request; then set `true` | False: user/group identity headers/metadata are stamped. True: they are absent. Catalog `extra_values` routing headers are still stamped in both cases. |
| **AN-PR-23** | Customisable identity header names (Bifrost) | P1 | — | Create `bifrost`, set `identity_header_user_id` / `identity_header_groups`; then clear one | Configured names are stamped upstream; a cleared name disables stamping for that dimension and round-trips as `""` (not omitted) in the API response. |
| **AN-PR-24** | Identity-header anti-spoofing | P0 | Gateway with identity injection | From the client, send a request that *already* sets the identity header with a fake user | The upstream receives NetBird's real caller identity — the client-supplied value is stripped, never forwarded. |
| **AN-PR-25** | Vertex `keyfile::` credential | P1 | GCP SA JSON | Create `vertex_ai_api` with `api_key = keyfile::<base64 SA JSON>` and a `<region>-aiplatform.googleapis.com` upstream | Requests succeed; upstream sees `Authorization: Bearer <short-lived OAuth token>`, never the key material. |
| **AN-PR-26** | Bedrock path-routed provider | P1 | Bedrock creds | Create `bedrock_api`; send a `/model/{modelId}/invoke` (and `/bedrock`-prefixed) request | Routed to the Bedrock provider by path, not by body model; the model recorded in the log is the path model with region/version stripped. |
| **AN-PR-27** | Secrets at rest | P0 | Provider created | Inspect the `agent_network_providers` table | `api_key` and `session_private_key` are ciphertext; `session_public_key` is plaintext. Neither key ever appears in an API response or in logs. |
| **AN-PR-28** | Malformed JSON | P2 | — | POST with a truncated JSON body | 400 `couldn't parse JSON request` (not 422/500). |

---

## 5. Policies — CRUD and validation

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-PO-01** | Create policy (happy path) | P0 | 1 provider, 1 group | POST with name, `source_groups=[g-eng]`, `destination_provider_ids=[p1]`, no limits | Created with an `ainpol_`-prefixed id, `enabled=true` by default. |
| **AN-PO-02** | `name` required | P1 | — | POST with blank name | 422 `name is required`. |
| **AN-PO-03** | `source_groups` must be non-empty and non-blank | P0 | — | POST with `[]`, then `[""]` | 422 `source_groups must contain at least one group id` / `must not contain empty entries`. |
| **AN-PO-04** | `destination_provider_ids` must be non-empty and non-blank | P0 | — | POST with `[]`, then `[""]` | 422 with the matching messages. |
| **AN-PO-05** | Destination provider must exist in the account | P0 | — | POST referencing an id from Account B, and a random id | 422 `destination_provider_ids: provider <id> does not exist` for both. |
| **AN-PO-06** | `guardrail_ids` entries must be non-blank | P1 | — | POST with `guardrail_ids: [""]` | 422 `guardrail_ids must not contain empty entries`. |
| **AN-PO-07** | Token limit window floor | P0 | — | POST with `token_limit.enabled=true`, `window_seconds=59` (and `0`) | 422 `limits.token_limit.window_seconds must be at least 60 (one minute) when enabled`. |
| **AN-PO-08** | Token limit needs a positive cap | P0 | — | POST with `token_limit.enabled=true`, window 3600, both caps `0` | 422 `limits.token_limit requires group_cap or user_cap to be greater than zero when enabled`. |
| **AN-PO-09** | Negative caps rejected | P1 | — | `group_cap=-1`, then `user_cap=-1` | 422 `must not be negative`. |
| **AN-PO-10** | Budget limit mirrors the token rules | P0 | — | Repeat AN-PO-07..09 for `budget_limit` (`group_cap_usd`, `user_cap_usd`) | Same class of 422s with `budget_limit` wording. |
| **AN-PO-11** | Limits omitted → uncapped | P1 | — | POST with `limits` absent | Created; both halves disabled; policy behaves as catch-all-allow (see AN-LM-09). |
| **AN-PO-12** | Disabled limits skip validation | P2 | — | POST with `enabled=false` and `window_seconds=0`, caps 0 | Accepted (validation only applies when the half is enabled). |
| **AN-PO-13** | Update policy | P0 | Policy exists | `PUT` changing name, groups, providers, guardrails, limits | 200; all fields replaced; `created_at` preserved, `updated_at` advanced; changes take effect on the next request without a restart. |
| **AN-PO-14** | Disable a policy | P0 | Policy governs a caller | Set `enabled=false` | Caller loses access through that policy; if no other enabled policy exists, the account's endpoint stops serving (AN-SY-01). |
| **AN-PO-15** | Delete a policy | P0 | Policy exists | `DELETE` | 200; policy gone; a caller authorised only by it can no longer reach the endpoint. |
| **AN-PO-16** | Deleting a group referenced by a policy | P1 | Group used in `source_groups` | Try to delete the group in **Groups** | Deletion is refused with a message naming the agent-network policy (no dangling group reference is left behind). |
| **AN-PO-17** | Policy events are audited | P2 | — | Create/update/delete a policy | Three distinct activity events recorded with the policy name and enabled state. |
| **AN-PO-18** | Same group in two policies | P1 | — | Create two enabled policies with overlapping `source_groups` and the same provider | Both are valid; selection follows the ranking in §7. |

---

## 6. Guardrails

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-GR-01** | Create guardrail with model allowlist | P0 | — | POST name + `checks.model_allowlist = {enabled:true, models:["gpt-4o","gpt-5.4-mini"]}` | Created with an `ainguard_`-prefixed id. |
| **AN-GR-02** | Create guardrail with prompt capture | P0 | — | POST with `checks.prompt_capture = {enabled:true, redact_pii:true}` | Created; both flags round-trip. |
| **AN-GR-03** | Empty allowlist arrays normalise | P2 | — | POST with `models` omitted / `null` | Response returns `models: []`, never `null`. |
| **AN-GR-04** | Update / delete guardrail | P1 | Guardrail exists | `PUT` then `DELETE` | 200 each; `created_at` preserved on update; deletion removes it from the attach picker. |
| **AN-GR-05** | Deleting an attached guardrail | P1 | Guardrail attached to a policy | Delete it | Observe and record the behaviour: either the delete is refused, or the policy's `guardrail_ids` no longer resolves. Follow-up requests must **not** 500, and a stale id must not silently turn a restricted policy into an unrestricted one — verify the effective model gate after deletion. |
| **AN-GR-06** | Model allowlist enforcement — allowed | P0 | Policy → guardrail with `models:["gpt-4o"]` | Agent requests `gpt-4o` | 200; served. |
| **AN-GR-07** | Model allowlist enforcement — blocked | P0 | Same | Agent requests `gpt-4.1` | Denied with `llm_policy.model_blocked`; reason reads `model "gpt-4.1" is not permitted by any applicable policy allowlist`. Access log shows decision `deny` and that reason. |
| **AN-GR-08** | Allowlist matching is case/whitespace tolerant | P1 | Allowlist `["GPT-4o"]` | Request `  gpt-4o ` | Allowed (comparison lowercases and trims on both sides). |
| **AN-GR-09** | Undetermined model fails closed | P0 | Restricted policy | Send a request whose body has no resolvable model | Denied with `llm_policy.model_blocked`; reason `request model could not be determined for the policy allowlist`. |
| **AN-GR-10** | Undetermined model is allowed when unrestricted | P1 | Policy with no allowlist-enabled guardrail | Same request | Not blocked by the allowlist gate. |
| **AN-GR-11** | Union across multiple guardrails on one policy | P1 | Policy with guardrail A (`gpt-4o`) and B (`gpt-4.1`) | Request each model | Both allowed — a policy permits the **union** of its enabled allowlists. |
| **AN-GR-12** | Union across multiple policies | P0 | Policy 1 allows `gpt-4o`, Policy 2 (same caller + provider) allows `gpt-4.1` | Request each | Both allowed; the blocked case only fires when **no** applicable policy permits the model. |
| **AN-GR-13** | Guardrail with allowlist disabled is not restrictive | P1 | Guardrail with `model_allowlist.enabled=false` but a populated `models` array | Request a model not in the array | Allowed. |
| **AN-GR-14** | Prompt capture requires both gates | P0 | Account `enable_prompt_collection=false`, policy guardrail `prompt_capture.enabled=true` | Send a request, then open the access-log row | **No** prompt/completion body captured. Flip the account toggle on → new requests capture bodies. Neither gate alone is sufficient. |
| **AN-GR-15** | PII redaction is account **OR** policy | P0 | Account `redact_pii=false`, policy guardrail `redact_pii=true`; prompt capture on both gates | Send a prompt containing an email/phone/credit-card-shaped string | Captured prompt is redacted. Repeat with the account flag on and the policy flag off → still redacted. |
| **AN-GR-16** | Browse-and-attach modal | P2 | ≥2 guardrails exist, 1 already attached | Open policy → Guardrails tab → **Browse** | Already-attached guardrails are not listed; multi-select works; the Attach button is disabled with nothing selected and shows a correct singular/plural count; when everything is attached, the empty-state copy is shown. |

---

## 7. Policy selection and limit enforcement

These are the money-correctness cases. Use short windows (60–300 s) so caps
can actually be exhausted inside a test run, and record the exact request/response
pairs.

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-LM-01** | Caller with no matching policy | P0 | Caller in `g-none`; no policy names that group | Send a request | Denied — the router is the authorisation gate; no upstream call is made and no upstream cost is incurred. |
| **AN-LM-02** | Policy targets a different provider | P0 | Policy allows provider P1 only | Request a model served by P2 | Not authorised through that policy. |
| **AN-LM-03** | Per-user token cap | P0 | Policy: token limit on, `user_cap=1000`, window 300 s | Drive user U past 1000 total tokens (in+out) in the window, then send one more | The over-cap request is denied with `llm_policy.token_cap_exceeded`; reason reads `user token cap exhausted on policy <id> (used N of 1000)`. |
| **AN-LM-04** | Per-group token cap is shared | P0 | Policy: `group_cap=1000`, window 300 s; users U1 and U2 both in `g-eng` | U1 consumes ~700 tokens; U2 then sends requests | U2 is denied once the **combined** group total reaches 1000 (`group token cap exhausted…`). |
| **AN-LM-05** | Per-user budget cap | P0 | `budget_limit.user_cap_usd=0.01`, window 300 s | Spend past $0.01 | Denied with `llm_policy.budget_cap_exceeded`; reason shows used vs cap to 4 decimals. |
| **AN-LM-06** | Per-group budget cap | P0 | `group_cap_usd` set | Two users in the group spend past it | Denied for whoever crosses the shared total. |
| **AN-LM-07** | Window rollover restores headroom | P0 | AN-LM-03 exhausted | Wait for the aligned window to roll over, then retry | Request is served again; counters restart from zero in the new window. Windows are **aligned** (not sliding) — confirm the reset happens at the window boundary, not exactly N seconds after the first request. |
| **AN-LM-08** | Cap of zero means uncapped | P1 | `token_limit.enabled=true`, `user_cap=0`, `group_cap=5000` | Drive one user past 5000 tokens alone | The user cap never binds; only the group cap does. |
| **AN-LM-09** | Uncapped policy wins over capped | P0 | Policy A (no caps) and Policy B (`group_cap=100`), both matching the caller and provider | Send requests past 100 tokens | All requests are attributed to A and keep succeeding; B's cap never blocks the caller. |
| **AN-LM-10** | Bigger pool drains first | P0 | Policy A `group_cap=10000`, Policy B `group_cap=1000`, both matching | Send requests | A is selected while it has headroom; only after A is exhausted does B pay; when both are exhausted the request is denied. |
| **AN-LM-11** | Ranking tiebreaks | P1 | Two policies with equal group token caps but different group budget caps; then equal on both but different user caps | Send requests | Order: group token cap → group budget cap → user token cap → user budget cap → **older `created_at`**. Repeat the run: the winner is stable and deterministic. |
| **AN-LM-12** | Multi-group attribution is deterministic | P0 | Caller in `g-alpha` and `g-beta`, both in the policy's `source_groups` | Send several requests | Every request attributes to the **lowest by string sort** intersecting group id. Verify the group column in the access log is identical across requests and across management replicas. |
| **AN-LM-13** | Effective window when both halves are enabled | P1 | Policy with token window 300 s and budget window 3600 s, both enabled | Send a request and inspect the recorded consumption window | The **token** window (300 s) is used for the policy-window booking. |
| **AN-LM-14** | Account budget rule — account-wide | P0 | Budget rule with **no** target groups/users, `user_cap_usd` small | Any caller spends past it | Denied with `llm_account.budget_cap_exceeded`, even for a caller on an uncapped policy. |
| **AN-LM-15** | Account rule binds with no matching policy | P0 | Account rule exists; caller matches **no** agent-network policy | Send a request | Ceiling is still evaluated (and denies once exhausted) — account rules are independent of policy selection. |
| **AN-LM-16** | Account rule targeting by user | P0 | Rule with `target_users=[U1]` | U1 and U2 both send traffic past the cap | U1 is denied; U2 is unaffected. |
| **AN-LM-17** | Account rule targeting by group | P0 | Rule with `target_groups=[g-eng]` | Member and non-member both send traffic | Only the member is bound. |
| **AN-LM-18** | Min-wins across rules | P0 | Rule A `user_cap_usd=$10`, Rule B `user_cap_usd=$1`, both applying | Spend past $1 | Denied at $1 — every applicable rule must pass; a looser rule cannot raise the ceiling. |
| **AN-LM-19** | Disabled rule doesn't bind | P1 | Rule set to `enabled=false` | Spend past its cap | Not enforced. |
| **AN-LM-20** | Account and policy caps coexist | P0 | Policy cap 1000 tokens/5 min; account rule 100000 tokens/month | Exhaust the policy cap | `llm_policy.*` deny code (not the account one); the account counter keeps accruing in its own monthly window. |
| **AN-LM-21** | One request never double-counts a cap | P0 | Policy window == an account rule window, same dimension | Send one request and read the consumption rows | The (dimension, window) counter is incremented **once**, not twice. |
| **AN-LM-22** | Deny does not consume upstream quota | P0 | Any exhausted cap | Send the denied request | No upstream call is made; no cost is added for the denied request; the access log records it with decision `deny` and zero/absent tokens. |
| **AN-LM-23** | Concurrency under a near-exhausted cap | P1 | Cap with ~1 request of headroom | Fire 20 concurrent requests | No 500s; counters stay consistent (no negative/duplicated totals); over-cap requests are denied. Record whether any overshoot occurs and by how much — the check is pre-flight, so a small overshoot may be by design; it must be bounded and must not corrupt totals. |
| **AN-LM-24** | Usage deltas are validated | P1 | — | Exercise the usage-recording path with a negative or non-finite delta (unit/API level) | Rejected with `usage deltas must be non-negative and finite`; totals are never decremented. |
| **AN-LM-25** | Deny surfaces usefully to the agent | P1 | Any deny case | Observe the client-side response in Claude Code / curl | A well-formed, upstream-shaped error the SDK can render — not a raw proxy 500 or a hung connection. |

---

## 8. Runtime routing and the tunnel

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-SY-01** | Endpoint requires settings + enabled provider + enabled policy | P0 | Bootstrapped account | Test all four states: (a) all three present; (b) no enabled provider; (c) no enabled policy; (d) no settings row | Only (a) serves. (b)/(c)/(d) produce no synthesised service and the endpoint does not serve LLM traffic. Re-enabling restores service without a restart. |
| **AN-SY-02** | Tunnel-only access | P0 | Endpoint serving | Call `https://<endpoint>/v1/chat/completions` from a machine **not** connected to the NetBird tunnel | Rejected — the synthesised service is private and gated by tunnel-peer access. |
| **AN-SY-03** | Access groups follow policy source groups | P0 | Endpoint serving; policies target `g-eng` only | Connect a peer whose user is only in `g-contractors` and call the endpoint | Rejected at the tunnel-peer gate. Add the user to `g-eng` → allowed (no restart). |
| **AN-SY-04** | No client API key needed | P0 | Endpoint serving | Call with `api_key=none` / no upstream credential | Succeeds; NetBird injects the real key server-side. Also confirm the caller's own bogus `Authorization` header is stripped and replaced. |
| **AN-SY-05** | Model → provider routing | P0 | Two providers with disjoint model lists | Request a model from each | Each routes to the correct upstream; the resolved provider is recorded in the access log. |
| **AN-SY-06** | Streaming (SSE) requests | P0 | — | Send a streaming chat completion (`stream: true`) | Stream is relayed correctly to the client; token usage from the trailing chunk is recorded; cost is non-zero. |
| **AN-SY-07** | Non-streaming requests | P0 | — | Same with `stream: false` | Usage and cost recorded. |
| **AN-SY-08** | Large request body | P1 | — | Send a request whose body exceeds the ~1 MiB capture cap (long context) | Request still routes and succeeds; routing fields (model) are still resolved; capture is truncated, not fatal. |
| **AN-SY-09** | Large / long streaming response | P1 | — | Generate a very long response | Usage is still recorded (the trailing usage event is within the 8 MiB response capture cap). |
| **AN-SY-10** | Config change propagation | P0 | Traffic flowing | Change a policy cap, add a provider, toggle a guardrail | New behaviour is observed on subsequent requests within seconds, with no manual proxy restart. |
| **AN-SY-11** | Upstream failure passthrough | P1 | Provider with a deliberately invalid key | Send a request | The upstream's 401/429/5xx is surfaced to the client intelligibly; the access log records the real status code; NetBird does not mask it as a generic 500. |
| **AN-SY-12** | Unreachable upstream | P1 | Provider pointing at a dead host | Send a request | Clean error to the client within a reasonable timeout; no hang; logged. |
| **AN-SY-13** | Non-LLM path on the endpoint | P2 | — | `GET https://<endpoint>/` and a random path | A sane response/404 — no panic, no stack trace, no credential leakage. |
| **AN-SY-14** | Multiple accounts on one cluster | P0 | Accounts A and B bootstrapped on the same cluster | Call A's endpoint with B's peer, and vice versa | Cross-account calls are rejected; each endpoint only serves its own account's providers and policies. |

---

## 9. Usage, cost and consumption accounting

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-US-01** | Cost matches configured rates | P0 | Provider with known `input_per_1k`/`output_per_1k` | Send a request with known token counts | Recorded `cost_usd` == `(in/1000 × input_rate) + (out/1000 × output_rate)` within rounding. |
| **AN-US-02** | Operator override beats catalog default | P0 | Provider model row priced deliberately differently from the catalog | Send a request | The operator's rate is used. |
| **AN-US-03** | OpenAI-shape cached input tokens | P0 | OpenAI model with `cached_input_per_1k` set; a prompt that hits the cache | Send two identical large prompts | Second call reports cached input tokens; they are billed at the cached rate, and cached tokens are a **subset** of input tokens (not added on top). |
| **AN-US-04** | Anthropic-shape cache read / creation | P0 | Anthropic model with `cache_read_per_1k` and `cache_creation_per_1k` | Run a prompt-caching sequence | Cache-read and cache-creation tokens are billed at their own rates and are **additive** to input tokens. Total tokens in the UI include them; the hover breakdown splits input / output / cache read / cache write. |
| **AN-US-05** | `nil` vs explicit `0` cache rate | P1 | One model with the cache rate omitted, one with it explicitly `0` | Send cache-hitting requests to each | Omitted → NetBird's default rate for that model applies. Explicit `0` → that bucket bills at the **input** rate (no discount), not free. |
| **AN-US-06** | Usage is recorded even with log collection off | P0 | `enable_log_collection=false` | Send several requests | No access-log rows appear, but the **Usage** tab and cost/token totals still update. |
| **AN-US-07** | Usage is recorded with no caps configured | P1 | Policy with no limits | Send requests | Usage/cost rows exist (consumption counters may be skipped, but usage history is not). |
| **AN-US-08** | Usage overview default window | P1 | Usage across >90 days (or seeded data) | `GET /agent-network/usage/overview` with no dates | Defaults to the last 90 days; response doesn't attempt to aggregate all history. |
| **AN-US-09** | Usage overview range clamp | P1 | — | Request a range wider than 366 days | Clamped to 366 days back from the end date; no error, no timeout. |
| **AN-US-10** | Granularity bucketing | P1 | Usage over several days | Request each supported granularity | Buckets are correct and ordered oldest-first; sparse days are represented rather than collapsed. |
| **AN-US-11** | Usage chart vs table agreement | P1 | Any usage | Open **Usage & Logs → Usage** | The token/cost chart, the per-day table and the hover breakdowns agree with each other and with the access-log sum for the same window. |
| **AN-US-12** | Consumption listing | P2 | Caps configured and partly consumed | `GET /agent-network/consumption` | Rows ordered window-newest-first, with dimension kind (user/group), dimension id, window seconds, window start, tokens and cost. |
| **AN-US-13** | Empty state | P2 | Fresh account | Open the Usage tab | "No usage recorded yet" empty state, no chart errors, no NaN/`$NaN` values. |
| **AN-US-14** | Zero-cost models | P2 | Embeddings model with `output_per_1k = 0` | Send an embeddings request | Cost computed from input only; no divide-by-zero or NaN in the UI. |

---

## 10. Access logs and sessions

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-AL-01** | Row content | P0 | Log collection on; a served request | Open **Usage & Logs → Access Logs** and expand the row | Timestamp, user (real identity, not the peer), authorising group, provider, model, path, tokens, cost, status code, duration, decision, session id. Values match the request. |
| **AN-AL-02** | Deny rows | P0 | A capped or model-blocked request | Inspect the row | Decision `deny`; the reason column carries the human-readable deny reason; the deny code is recoverable. |
| **AN-AL-03** | Log collection off suppresses rows | P0 | `enable_log_collection=false` | Send requests, then look | No new access-log rows (existing ones remain). Turn it back on → new requests appear. |
| **AN-AL-04** | Prompt capture content | P0 | Both prompt gates on | Send a request with a distinctive prompt | The expanded row shows the request prompt and the response completion. |
| **AN-AL-05** | Redaction of captured prompts | P0 | AN-GR-15 setup | Send a prompt containing an email address, phone number and card-shaped number | Stored/displayed prompt is redacted. Verify at the **database** level too, not just in the UI. |
| **AN-AL-06** | Pagination | P1 | >100 log rows | Page through; also call the API with `page_size=1000` | Default page size 50 (dashboard uses 25); `page_size` is clamped to 100; page navigation is consistent with no duplicate/missing rows. |
| **AN-AL-07** | Sorting | P1 | Mixed data | Sort by timestamp, model, provider, status_code, duration, cost_usd, total_tokens, user_id, decision, asc and desc | Each sort is correct; an unknown `sort_by` falls back to `timestamp desc` rather than erroring. |
| **AN-AL-08** | Filters | P0 | Mixed data | Filter by date range, user, group, provider (multi), model (multi), decision, path prefix, and free-text search | Each narrows correctly; multi-select is OR-within-field and AND-across-fields; combining filters with sorting and paging stays consistent. |
| **AN-AL-09** | Multi-value query forms | P1 | — | Call the API with `?provider_id=a&provider_id=b` and with `?provider_id=a,b` | Both forms behave identically. |
| **AN-AL-10** | Invalid date is rejected, not ignored | P1 | — | `?start_date=yesterday` and `?end_date=2026-13-45` | 422 `invalid start_date/end_date: …` — the filter must not silently broaden the query. |
| **AN-AL-11** | Default UI window | P2 | — | Open the Access Logs tab fresh | Defaults to the last 14 days; the reset-filters action returns to that window. |
| **AN-AL-12** | Session grouping | P0 | An agent session spanning many requests (e.g. a Claude Code task) | Toggle **group by session** | One row per session with request count, first/last activity, summed tokens, summed cost, and a max status; expanding shows the constituent requests. Toggling back to flat view shows the same data ungrouped. |
| **AN-AL-13** | Session sorting | P1 | — | Sort sessions by last activity, started_at, request_count, cost, tokens, duration, status, user, decision | Correct aggregate ordering; sorting by a flat-only field (model/provider) falls back to the default without error. |
| **AN-AL-14** | Session filter round-trip | P1 | — | From a flat row, filter by its `session_id` | Only that session's requests are listed. |
| **AN-AL-15** | Retention sweep deletes old rows | P0 | Retention set to 7 days; seeded rows older than that | Wait for/trigger the cleanup sweep (runs at startup and on the configured interval, default 24 h) | Rows older than the cutoff are deleted; newer rows and **all usage records** are untouched. |
| **AN-AL-16** | Indefinite retention | P1 | Retention set to `0` (Indefinite) | Run the sweep | No access-log rows are deleted for that account. |
| **AN-AL-17** | Per-account retention isolation | P1 | Account A retention 7 days, Account B indefinite | Run the sweep | Only A's old rows are removed. |
| **AN-AL-18** | Retention change takes effect | P1 | Retention 90 → 7 | Change it and run the sweep | Rows between 7 and 90 days old are removed on the next sweep. |
| **AN-AL-19** | Log volume under load | P2 | — | Drive sustained request volume | Ingest keeps up; no request failures caused by log writes; a log-write failure must not fail the user's LLM request. |

---

## 11. Permissions, roles and tenancy

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-RB-01** | Owner/admin full access | P0 | Owner and admin users | Exercise create/read/update/delete on providers, policies, guardrails, budget rules, settings; read usage and logs | All succeed. |
| **AN-RB-02** | Regular user is denied writes | P0 | `user`-role account member | Attempt each mutating call | 403 on each; the dashboard shows the restricted-access state rather than a broken page. |
| **AN-RB-03** | Read-only role | P1 | Auditor/read-only role, if available | GET providers, policies, guardrails, budgets, usage, logs; then attempt a write | Reads succeed per the role's grants; writes 403. |
| **AN-RB-04** | Submodule grants are independent | P1 | A role granted only `agent_network.logs` read | Read logs; then read providers | Logs succeed; providers 403. |
| **AN-RB-05** | Parent grant cascades | P1 | A role granted `agent_network` | Access every submodule surface | All submodules resolve through the parent grant. |
| **AN-RB-06** | Bootstrap needs the settings permission too | P0 | A user with provider-create but **not** settings-create, on an un-bootstrapped account | Create a provider with `bootstrap_cluster` set | 403 — pinning the cluster/subdomain is a settings write. Without `bootstrap_cluster`, the provider create is allowed. |
| **AN-RB-07** | Cross-account read isolation | P0 | Accounts A and B each with providers/policies/logs | As an A user, `GET` B's provider/policy/guardrail/budget-rule ids directly | 404/403 — never B's data. |
| **AN-RB-08** | Cross-account write isolation | P0 | Same | As an A user, `PUT`/`DELETE` a B resource id | Rejected; B's data unchanged. |
| **AN-RB-09** | Cross-account log isolation | P0 | Both accounts have traffic | List A's access logs and usage | Only A's rows; no B user ids, models or costs. |
| **AN-RB-10** | Unauthenticated access | P0 | — | Call each agent-network endpoint with no token and with an invalid token | 401; no data leaked in the error body. |
| **AN-RB-11** | Service-user / PAT access | P2 | A PAT for an admin | Drive the full provider→policy→request flow via PAT | Works identically to session auth. |

---

## 12. Dashboard — providers page and the provider wizard

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-UI-01** | Empty state before bootstrap | P0 | Fresh account | Open **Agent Network → Providers** | The API-Base-URL card shows the dashed "Connect your first provider to set up your agent network endpoint" prompt and opens the wizard on click. |
| **AN-UI-02** | Endpoint badge after bootstrap | P0 | Bootstrapped | Reload the page | Badge shows `https://<subdomain>.<cluster>`; the tooltip explains base-URL usage; **Copy** copies the full `https://` URL and confirms with a toast. |
| **AN-UI-03** | No validated cluster blocks create | P0 | Account with no validated FREE cluster | Open the wizard | The no-clusters state is shown and the operator cannot complete a first create. Add a validated cluster → the wizard proceeds. |
| **AN-UI-04** | Cluster auto-pick | P1 | ≥1 validated cluster, not bootstrapped | Open the wizard | The first validated cluster is preselected; the operator can change it before the first create. |
| **AN-UI-05** | Cluster picker hidden after bootstrap | P1 | Bootstrapped | Open the wizard for a second provider | No cluster choice is offered (the hint would be ignored anyway). |
| **AN-UI-06** | Provider type switching | P1 | — | Switch between OpenAI / Anthropic / Bedrock / Vertex / LiteLLM / Portkey / Bifrost / OpenRouter / vLLM / Custom | Name and upstream URL prefill from the catalog per type; Vertex shows the clean `https://aiplatform.googleapis.com` placeholder rather than a templated host; provider entries are grouped by kind (provider / gateway / custom). |
| **AN-UI-07** | Continue is gated | P0 | — | Leave name blank, then enter an invalid URL, then a 3-character API key | The Continue/Save action stays disabled until: a type is chosen, the name is non-blank, the URL matches `http(s)://…`, the key is ≥4 characters, and (first create) a cluster is picked. |
| **AN-UI-08** | Mappings tab visibility | P1 | — | Select `litellm_proxy`, `portkey`, `bifrost`, `cloudflare_ai_gateway`, `vercel_ai_gateway`, `openrouter`, `bedrock_api`; then switch to `openai_api` while on the Mappings tab | Mappings appear only for those types; switching to a type without mappings snaps the wizard back to the Provider tab (no blank tab). |
| **AN-UI-09** | Model rows | P1 | — | Add catalog models, add a custom model row, add an empty row, add two rows with the same id | Catalog picks prefill prices; a model already added can't be picked twice; empty rows are dropped on save; duplicate ids collapse to the first row (no ambiguous price is sent). |
| **AN-UI-10** | Price prefill matches live defaults | P1 | An operator pricing override is configured | Add a catalog model | The prefilled price matches what the proxy will actually bill (live default table), not the compiled catalog rate. |
| **AN-UI-11** | Edit shows a masked key | P0 | Provider exists | Open it for edit | The API key field shows a masked placeholder; saving without touching it preserves the stored key (verify a request still succeeds). |
| **AN-UI-12** | Stale extra values are dropped on type switch | P2 | — | Fill Portkey's config header, then switch the type to OpenAI and save | The Portkey-only key is not persisted. |
| **AN-UI-13** | Providers table | P1 | Several providers | Review the table | Name, logo, type, upstream, model count, enabled state; row actions (edit/delete/enable) behave; the delete-blocked error from AN-PR-18 is surfaced as a readable message, not a raw payload. |
| **AN-UI-14** | Agent Config modal — tabs | P0 | Bootstrapped, providers connected | Click **Agent Config** | Tabs: Claude Code, Codex, OpenAI SDK, cURL. Every snippet embeds the account's real endpoint (`https://<endpoint>`, `/v1` where appropriate) and copies to clipboard correctly. |
| **AN-UI-15** | Claude Code backend selector | P1 | — | Switch between Anthropic API / Vertex AI / Bedrock, and toggle JSON ↔ Shell | Each variant renders the right env vars (`ANTHROPIC_BASE_URL`, `ANTHROPIC_VERTEX_BASE_URL` with `/v1`, `ANTHROPIC_BEDROCK_BASE_URL` with `/bedrock`) and JSON/Shell forms are equivalent. |
| **AN-UI-16** | Kimi surfaces are gated on a Kimi provider | P1 | No `kimi_api` provider | Open the modal | No **Kimi CLI** tab and no Kimi option in the Claude Code selector. Add a `kimi_api` provider → both appear, and when Kimi is the only Anthropic-shaped provider the selector opens on Kimi. |
| **AN-UI-17** | Kimi snippets actually work | P1 | `kimi_api` provider with the default `https://api.moonshot.ai` upstream | Follow the Claude Code Kimi snippet (`ANTHROPIC_BASE_URL=<endpoint>/anthropic`) and the Kimi CLI snippet (bare endpoint, `type="anthropic"`) | Both drive real completions through NetBird; usage is logged against the Kimi provider. |
| **AN-UI-18** | cURL snippet is copy-paste runnable | P0 | Tunnel connected, OpenAI provider + policy | Copy the cURL snippet and run it | Returns a completion; the copied text is a single-line command even though it is displayed multi-line. |
| **AN-UI-19** | Codex snippet | P1 | — | Apply the `~/.codex/config.toml` snippet and run Codex | Requests route through NetBird. |
| **AN-UI-20** | OpenAI SDK snippet | P1 | — | Run the Python snippet with `api_key="not-needed"` | Returns a completion. |

---

## 13. Dashboard — policies, guardrails, limits, configuration

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-UI-21** | Policies table | P1 | Several policies | Open **Agent Network → Policies** | Name, enabled toggle, source groups, destination providers, guardrail summary and limits render correctly; the enabled toggle persists. |
| **AN-UI-22** | Policy modal tabs | P1 | — | Walk the General / Guardrails / Limits tabs | Tab state is preserved while the modal is open; validation errors are shown next to the offending field, not only as a toast. |
| **AN-UI-23** | Limits tab window authoring | P1 | — | Author a window in minutes, hours, and days | The wire value is seconds; the default is 30 days (2,592,000 s); Save is disabled unless at least one cap is > 0 and the window is ≥ 60 s. |
| **AN-UI-24** | Limits display formatting | P2 | Policies with several window lengths | Review the limits column | Windows render as human units (e.g. "30 days", "1 hour"), and USD caps are formatted as currency. |
| **AN-UI-25** | Guardrail checks cell | P2 | Guardrails with different check combinations | Review the policy table | The cell summarises model allowlist and prompt capture accurately, including the "no checks" case. |
| **AN-UI-26** | Global Limits tab | P0 | — | **Configuration → Global Limits**: create, edit, disable and delete a budget rule; scope it to groups, to users, to both, and to neither | Each variant saves and reloads correctly; the "applies to everyone" (no targets) case is clearly communicated in the UI. |
| **AN-UI-27** | Budget rule save gating | P1 | — | Leave the name blank; then set a rule with no positive cap | Save stays disabled / the request is rejected with a readable message. |
| **AN-UI-28** | Log Collection tab | P0 | — | Toggle log collection, change retention, toggle prompt collection, click **Save Changes** | Save is disabled until something changes; the retention select is disabled (and visually dimmed) while log collection is off; a failed save keeps the unsaved-changes state; a successful save clears it. |
| **AN-UI-29** | Prompt-collection help copy is accurate | P2 | — | Read the helper text | It states that a policy guardrail must also be enabled (matching AN-GR-14). |
| **AN-UI-30** | Clusters tab | P1 | — | **Configuration → Clusters** | Lists clusters with validation state; the copy differs appropriately between focused (agent-network-only) mode and the full dashboard; the docs link opens. |
| **AN-UI-31** | Deep-linkable tabs | P2 | — | Open `/agent-network/configuration?tab=log-settings`, `?tab=clusters`, `/agent-network/usage?tab=access-logs`; use browser back/forward | The right tab opens; the active tab is reflected in the URL; back/forward restores the previous tab; an invalid `?tab=` value falls back to the default. |
| **AN-UI-32** | Restricted-access states | P1 | A user without `services.read` | Open each Agent Network page | The restricted-access component renders; no underlying data fetches are made (check the network tab); no console errors. |

---

## 14. Focused mode, navigation and onboarding

| ID | Title | P | Preconditions | Steps | Expected |
|---|---|---|---|---|---|
| **AN-NAV-01** | Menu hidden by default | P1 | Account with neither the feature flag nor the account setting | Open the dashboard | No Agent Network menu entry; direct navigation to `/agent-network/providers` behaves per the guard (redirect or restricted state), not a crash. |
| **AN-NAV-02** | Feature flag shows the menu | P0 | `dashboard_features.agent_network = true` | Reload | Agent Network appears in the nav with Providers, Policies, Usage & Logs, Configuration — **alongside** the rest of the dashboard. |
| **AN-NAV-03** | Account-level focused mode | P0 | `settings.agent_network_only = true` | Reload | The dashboard is focused on Agent Network; unrelated sections are hidden. Toggling it back off restores the full dashboard. |
| **AN-NAV-04** | Deployment-level focused mode is a floor | P0 | `NETBIRD_AGENT_NETWORK_ONLY` set on the deployment | Set the account setting to `false` and reload | Focused mode still applies — the env flag cannot be overridden by the per-account setting. |
| **AN-NAV-05** | Signup-source optimism | P1 | New account from the netbird.ai signup source, `signup_form_pending=true`, no explicit `agent_network_only` | Log in | Focused view applies immediately — the regular onboarding must not flash first. |
| **AN-NAV-06** | Explicit choice beats optimism | P1 | Same, but `agent_network_only = false` explicitly | Log in | The user's explicit choice is respected (no focused view). |
| **AN-NAV-07** | Loading guard | P2 | Slow `/accounts` response (throttle the network) | Load the dashboard | No premature redirect or flicker between modes while the account is still loading. |
| **AN-ONB-01** | Full onboarding walkthrough | P0 | Fresh focused-mode account, `signup_form_pending=true` | Walk Signup → Welcome → Device → Provider → Policy → Configure → End | Every step advances; the stepper reflects progress; the modal cannot be dismissed with Esc or an outside click. |
| **AN-ONB-02** | Signup step is skipped when already submitted | P1 | `signup_form_pending=false` | Enter onboarding | Opens on Welcome, never on Signup, and never returns to Signup. |
| **AN-ONB-03** | Device step detects a connection | P0 | No peers | Sit on the Device step and install/connect a client | The step detects the peer within ~5 s (it polls) without a manual refresh, and Continue unblocks. |
| **AN-ONB-04** | First-run seeding | P0 | Fresh account | Reach the Policy step | A "Users" source group exists containing the current user, and the permissive **Default** access-control policy has been removed. Verify both in Groups / Access Control. |
| **AN-ONB-05** | Provider step creates a working provider | P0 | — | Complete the Provider step | A provider is created and settings are bootstrapped (endpoint assigned). |
| **AN-ONB-06** | Policy step creates a working policy | P0 | — | Complete the Policy step | A policy exists linking the seeded group to the new provider. |
| **AN-ONB-07** | Configure step snippets work | P0 | — | Copy the inline snippet on the Configure step and run it from the connected device | A real completion is returned end to end. |
| **AN-ONB-08** | Resume after refresh | P1 | Mid-onboarding (e.g. step 5) | Hard-refresh the browser | Onboarding resumes on the same step, not from the beginning. |
| **AN-ONB-09** | Skip to dashboard | P1 | Any step except Signup and End | Click **Skip to Dashboard** | Onboarding closes and does not reappear on the next load. |
| **AN-ONB-10** | Finish | P1 | End step | Click Finish | Lands on the Agent Network dashboard with the created provider/policy visible and the endpoint badge populated. |
| **AN-ONB-11** | Onboarding is idempotent | P2 | Completed onboarding | Log out and back in; reload several times | Onboarding does not restart; the seeded group is not duplicated. |

---

## 15. Cross-cutting, resilience and security

| ID | Title | P | Steps | Expected |
|---|---|---|---|---|
| **AN-X-01** | Secrets never leak | P0 | Grep every provider/policy/settings API response, the dashboard network tab, and management/proxy logs at debug level for the upstream API key and the session private key | Neither value appears anywhere. |
| **AN-X-02** | Prompt content respects the gates in logs | P0 | With prompt capture **off**, send a distinctive prompt and grep management/proxy logs and the DB | The prompt body appears nowhere. |
| **AN-X-03** | Management restart | P0 | Restart management with traffic in flight | Service resumes; providers/policies/limits/counters survive; the endpoint serves again without reconfiguration. |
| **AN-X-04** | Proxy restart | P0 | Restart the proxy | Config is re-synthesised and pushed; requests resume without a management change. |
| **AN-X-05** | Migration of an existing account | P0 | Upgrade a deployment that already has agent-network rows from the previous release | Existing providers/policies/guardrails/budget rules/settings survive; providers missing a session keypair are backfilled automatically on the next synthesis (no dial failures). |
| **AN-X-06** | Fresh install | P1 | Deploy from scratch and run §3 → §8 | The full flow works with no manual DB steps. |
| **AN-X-07** | SQLite and PostgreSQL parity | P1 | Run §7, §9 and §10 against both store backends | Identical behaviour, especially access-log sorting/grouping and consumption aggregation. |
| **AN-X-08** | Injection safety on filters | P0 | Pass SQL/JS payloads into `search`, `path`, `model`, `user_id`, `session_id` and the group/provider multi-filters | Treated as literal text; no SQL error, no 500, no reflected script execution in the dashboard. |
| **AN-X-09** | Pagination overflow | P2 | Request `?page=99999999999&page_size=100` | No overflow, no 500 — an empty page is fine. |
| **AN-X-10** | Unicode and long strings | P2 | Create providers/policies/guardrails with emoji, RTL text and 500-character names | Stored and rendered correctly (or rejected with a clear length error) — never truncated silently mid-codepoint. |
| **AN-X-11** | Idempotent double-submit | P1 | Double-click Save on the provider, policy and budget-rule modals | Exactly one record is created. |
| **AN-X-12** | Concurrent edits | P2 | Two admins edit the same policy simultaneously | Last write wins without corruption; neither session gets a 500. |
| **AN-X-13** | Activity log coverage | P1 | Create/update/delete one of each entity (provider, policy, guardrail, budget rule) and change settings | Every mutation produces a distinct, correctly attributed activity event. |
| **AN-X-14** | Dark/light and responsive rendering | P2 | View every Agent Network page at 1280 px and 1440 px, and check wide tables | No horizontal page scroll; wide tables scroll within their own container; no clipped controls. |

---

## 16. Suggested execution order

1. **§3 Bootstrap → §4 Providers → §5 Policies** — nothing else can be tested until an endpoint serves.
2. **§8 Runtime routing** — prove one request works end to end over the tunnel.
3. **§7 Limits** and **§9 Usage/cost** — the highest-risk correctness areas; budget the most time here.
4. **§6 Guardrails** and **§10 Access logs** — depend on live traffic from step 2.
5. **§11 Permissions/tenancy** — run against the data created above.
6. **§12–§14 Dashboard, focused mode, onboarding** — needs a second fresh account for the onboarding cases.
7. **§15 Cross-cutting** — restart/migration cases last, since they disturb the environment.

## 17. Reporting template

For each failure, capture: test id, environment (deployment, cluster, account),
the request (method, path, body with secrets redacted), the actual response
(status + body), the access-log row id and session id if one exists, the
management and proxy log excerpts around the timestamp, and the deny code where
one was returned.