[management,proxy] Agent network: per-account LLM gateway (policy, metering, multi-provider) (#6555)

* [agent-network] Shared proto, OpenAPI schema, and generated types

* [agent-network] Management: store, manager, synthesizer, policy engine, provider catalog, HTTP/gRPC API

Adds the account-scoped agent-network module: provider/policy/budget CRUD and
store, the reverse-proxy service synthesizer, policy selection + limit
enforcement, the provider catalog (incl. Vertex AI and AWS Bedrock entries),
and the management HTTP + proxy gRPC surfaces.

* [management] Fix agent-network proxy-peer fan-out on affected-peer recompute

The affected-peers resolver loaded only persisted reverse-proxy services, but
agent-network services are synthesized on demand and never persisted. As a
result the embedded proxy peer was never folded into the affected set when a
client's group changed, so the proxy received no network-map update for a newly
authorised client and rejected its handshake until a full resync (restart).

loadProxyServices now merges the synthesized agent-network services (injected
via a registration hook to avoid an import cycle), so proxy peers learn newly
authorised clients immediately.

* [proxy] Reverse-proxy middleware framework, chain, and request plumbing

The per-target middleware chain (slots, dispatcher, mutation gate, metadata
merger), body capture, access-log terminal sink, and the proxy wiring that
builds + runs chains for synthesized agent-network services.

* [proxy] LLM parsers, pricing, and builtin middlewares (OpenAI, Anthropic, Vertex AI, AWS Bedrock)

Request/response parsers and SSE/event-stream metering, the embedded pricing
table, and the builtin middleware set: request parser, router, policy
limit-check/record, cost meter, guardrail, identity inject, response parser.
Includes the path-routed providers — Google Vertex AI (keyfile:: service-account
OAuth minting) and AWS Bedrock (bearer auth, invoke/converse/streaming, optional
/bedrock prefix) — plus the Models allowlist and unmeterable-publisher deny.

* [proxy] IPv6 in-place apply and TCP accept-loop hardening on netstack listeners

* [agent-network] End-to-end test suite, module docs, and deployment preset

* [agent-network] Fix codespell typos and exclude false positives

- labelgen word pool: vermillion -> vermilion, racoon -> raccoon.
- codespell ignore list: add flate (Go compress/flate package), recordin
  (a test-local identifier), and unparseable (a valid alternative spelling used
  consistently across identifiers + a metadata-value constant).

* [management] Set LastSeen on injected proxy peer in realstack test (MySQL strict-mode)

The injected embedded proxy peer had a PeerStatus with a zero LastSeen, which
serializes to '0000-00-00' and is rejected by MySQL in strict mode (SQLite
tolerates it). Set LastSeen to a valid time so SaveAccount succeeds on both
engines.

* [agent-network] Remove e2e shell-script suite from this branch

The end-to-end shell scripts under scripts/e2e/ are maintained in a separate
testing suite and are not part of this change set.

* [agent-network] Polish module docs: remove internal review scaffolding, fix links, verify diagrams

Strip PR-review framing, commit references, absolute paths, and stale internal
references from the agent-network module docs; fix broken relative links; verify
all diagrams against the current architecture. Remove the internal AI-reviewer
prompt file.

* [management] Refine session expiration handling to support 3-state encoding for SSO deadlines

* [agent-network] Relocate agentnetwork package to internals/modules

Move management/server/agentnetwork (and its catalog/, labelgen/, types/
subpackages) to management/internals/modules/agentnetwork, alongside the
reverse-proxy module, and rewrite all importers. Pure relocation: package names,
the synthesizer + affectedpeers registration hook, and store access (shared
store.Store) are unchanged, so no import cycle is introduced (affectedpeers
still depends only on the agentnetwork/types leaf).

* [agent-network] Co-locate HTTP handlers in the module (RegisterEndpoints)

Move the agent-network HTTP handlers from server/http/handlers/agentnetwork into
the module at internals/modules/agentnetwork/handlers (package handlers) and
rename the entrypoint AddEndpoints -> RegisterEndpoints, matching the
reverse-proxy module convention. Wiring in http/handler.go updated accordingly.

proxy/internal/accesslog/logger.go

@@ -128,6 +128,7 @@ type logEntry struct {
 	BytesDownload int64
 	Protocol      Protocol
 	Metadata      map[string]string
+	AgentNetwork  bool
 }
 
 // Protocol identifies the transport protocol of an access log entry.
@@ -214,6 +215,54 @@ func (l *Logger) allowDenyLog(serviceID types.ServiceID, reason string) bool {
 	return false
 }
 
+// usageMetadataKeys is the allowlist of metadata retained on a stripped,
+// usage-only agent-network entry. Mirrors the llm.* / cost.* keys in
+// proxy/internal/middleware/keys.go — only the dimensions management needs to
+// record a usage row (provider / model / tokens / cost / groups).
+var usageMetadataKeys = map[string]struct{}{
+	"llm.provider":             {},
+	"llm.model":                {},
+	"llm.resolved_provider_id": {},
+	"llm.input_tokens":         {},
+	"llm.output_tokens":        {},
+	"llm.total_tokens":         {},
+	"cost.usd_total":           {},
+	"llm.authorising_groups":   {},
+}
+
+// stripAgentNetworkEntryForUsage returns the entry reduced to what's needed to
+// record usage/cost: it drops request detail (host / path / source IP) and any
+// prompt capture, keeping the LLM usage metadata plus the caller identity
+// (user / auth mechanism) needed for attribution. Shipped when an
+// agent-network account has log collection disabled but usage must still be
+// collected. logEntry is passed by value, so mutating it here is safe; Metadata
+// is replaced with a fresh map rather than mutated in place.
+func stripAgentNetworkEntryForUsage(entry logEntry) logEntry {
+	entry.Host = ""
+	entry.Path = ""
+	entry.SourceIP = netip.Addr{}
+	// Drop the rest of the per-request telemetry too — a usage-only entry
+	// must carry the LLM usage metadata and caller identity, nothing that
+	// describes the individual request.
+	entry.Method = ""
+	entry.ResponseCode = 0
+	entry.DurationMs = 0
+	entry.BytesUpload = 0
+	entry.BytesDownload = 0
+	entry.Protocol = ""
+
+	if len(entry.Metadata) > 0 {
+		stripped := make(map[string]string, len(usageMetadataKeys))
+		for k := range usageMetadataKeys {
+			if v, ok := entry.Metadata[k]; ok {
+				stripped[k] = v
+			}
+		}
+		entry.Metadata = stripped
+	}
+	return entry
+}
+
 func (l *Logger) log(entry logEntry) {
 	// Fire off the log request in a separate routine.
 	// This increases the possibility of losing a log message
@@ -264,6 +313,7 @@ func (l *Logger) log(entry logEntry) {
 				BytesDownload: entry.BytesDownload,
 				Protocol:      string(entry.Protocol),
 				Metadata:      entry.Metadata,
+				AgentNetwork:  entry.AgentNetwork,
 			},
 		}); err != nil {
 			l.logger.WithFields(log.Fields{

proxy/internal/accesslog/middleware.go

@@ -83,11 +83,23 @@ func (l *Logger) Middleware(next http.Handler) http.Handler {
 			BytesDownload: bytesDownload,
 			Protocol:      ProtocolHTTP,
 			Metadata:      capturedData.GetMetadata(),
+			AgentNetwork:  capturedData.GetAgentNetwork(),
 		}
 		l.logger.Debugf("response: request_id=%s method=%s host=%s path=%s status=%d duration=%dms source=%s origin=%s service=%s account=%s",
 			requestID, r.Method, host, r.URL.Path, sw.status, duration.Milliseconds(), sourceIp, capturedData.GetOrigin(), capturedData.GetServiceID(), capturedData.GetAccountID())
 
-		l.log(entry)
+		// Emit the access log unless the matched target opted out
+		// (agent-network synth targets do this when the account's
+		// EnableLogCollection toggle is off). For agent-network entries we
+		// still ship a stripped, usage-only record even when suppressed, so
+		// usage/cost is collected regardless of the log-collection toggle;
+		// request detail and prompt capture are dropped before sending.
+		switch {
+		case !capturedData.GetSuppressAccessLog():
+			l.log(entry)
+		case entry.AgentNetwork:
+			l.log(stripAgentNetworkEntryForUsage(entry))
+		}
 
 		// Track usage for cost monitoring (upload + download) by domain
 		l.trackUsage(host, bytesUpload+bytesDownload)

proxy/internal/accesslog/middleware_test.go

@@ -0,0 +1,185 @@
+package accesslog
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"google.golang.org/grpc"
+
+	"github.com/netbirdio/netbird/proxy/internal/proxy"
+	"github.com/netbirdio/netbird/shared/management/proto"
+)
+
+// recorderClient is a minimal stub for the access-log gRPCClient interface. It
+// counts SendAccessLog invocations and signals on every call so tests can
+// deterministically wait for the goroutine inside Logger.log without sleeping.
+type recorderClient struct {
+	mu        sync.Mutex
+	calls     int64
+	lastEntry *proto.AccessLog
+	called    chan struct{}
+}
+
+func newRecorderClient() *recorderClient {
+	return &recorderClient{called: make(chan struct{}, 16)}
+}
+
+func (r *recorderClient) SendAccessLog(_ context.Context, in *proto.SendAccessLogRequest, _ ...grpc.CallOption) (*proto.SendAccessLogResponse, error) {
+	r.mu.Lock()
+	r.calls++
+	r.lastEntry = in.GetLog()
+	r.mu.Unlock()
+	select {
+	case r.called <- struct{}{}:
+	default:
+	}
+	return &proto.SendAccessLogResponse{}, nil
+}
+
+func (r *recorderClient) callCount() int64 {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return r.calls
+}
+
+// newTestLogger builds a Logger backed by the supplied recorderClient. It is
+// the same constructor production uses, just with a stub gRPC client — no
+// mocks, no interface re-implementations.
+func newTestLogger(t *testing.T, client *recorderClient) *Logger {
+	t.Helper()
+	logger := NewLogger(client, nil, nil)
+	t.Cleanup(logger.Close)
+	return logger
+}
+
+// TestMiddleware_SuppressAccessLog_SkipsLogSink asserts the suppression gate.
+// When the inner handler stamps SuppressAccessLog=true on CapturedData (mirrors
+// what reverseproxy does when the matched target's DisableAccessLog flag is
+// set), the middleware must NOT invoke the access-log sink. Bandwidth telemetry
+// (trackUsage) keeps running — it's the call to SendAccessLog that we gate.
+func TestMiddleware_SuppressAccessLog_SkipsLogSink(t *testing.T) {
+	client := newRecorderClient()
+	l := newTestLogger(t, client)
+
+	inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		cd := proxy.CapturedDataFromContext(r.Context())
+		require.NotNil(t, cd, "middleware must inject CapturedData into the request context")
+		cd.SetSuppressAccessLog(true)
+		w.WriteHeader(http.StatusOK)
+	})
+
+	srv := httptest.NewServer(l.Middleware(inner))
+	t.Cleanup(srv.Close)
+
+	resp, err := http.Get(srv.URL + "/agent-network/v1/chat/completions")
+	require.NoError(t, err, "GET against suppressed target must succeed")
+	require.NoError(t, resp.Body.Close())
+	require.Equal(t, http.StatusOK, resp.StatusCode, "inner handler must run normally")
+
+	// Give the goroutine fence a beat (Logger.log dispatches in a goroutine).
+	// The negative assertion needs a small window: if a send is going to
+	// happen, it happens promptly.
+	select {
+	case <-client.called:
+		t.Fatalf("access-log sink must not be invoked when SuppressAccessLog=true (got %d call(s))", client.callCount())
+	case <-time.After(150 * time.Millisecond):
+	}
+
+	assert.Equal(t, int64(0), client.callCount(),
+		"SendAccessLog must not be called for suppressed requests")
+}
+
+// TestMiddleware_SuppressAccessLog_DefaultEmitsLog is the regression sanity:
+// when nothing sets SuppressAccessLog (the universal default for every
+// non-agent-network target), the middleware MUST still emit the access-log
+// entry. This is the guarantee that wires-through to the EnableLogCollection
+// gate without breaking anyone who isn't opted in.
+func TestMiddleware_SuppressAccessLog_DefaultEmitsLog(t *testing.T) {
+	client := newRecorderClient()
+	l := newTestLogger(t, client)
+
+	var innerRan atomic.Bool
+	inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		innerRan.Store(true)
+		// Intentionally DO NOT touch SuppressAccessLog — mirrors every
+		// non-agent-network target.
+		w.WriteHeader(http.StatusOK)
+	})
+
+	srv := httptest.NewServer(l.Middleware(inner))
+	t.Cleanup(srv.Close)
+
+	resp, err := http.Get(srv.URL + "/service/healthz")
+	require.NoError(t, err, "GET against default target must succeed")
+	require.NoError(t, resp.Body.Close())
+	require.Equal(t, http.StatusOK, resp.StatusCode, "inner handler must run normally")
+	require.True(t, innerRan.Load(), "inner handler must have run")
+
+	select {
+	case <-client.called:
+	case <-time.After(2 * time.Second):
+		t.Fatalf("SendAccessLog must be invoked for non-suppressed requests, none observed (calls=%d)", client.callCount())
+	}
+
+	assert.Equal(t, int64(1), client.callCount(),
+		"non-suppressed request must produce exactly one access-log send")
+}
+
+// TestMiddleware_SuppressAccessLog_PreservesUsageTracking proves the gate is
+// surgical: with SuppressAccessLog=true the access-log send is skipped, but
+// the per-domain usage tracker still records the bytes transferred. This is
+// the cost-monitoring guarantee called out in the gate's comment.
+func TestMiddleware_SuppressAccessLog_PreservesUsageTracking(t *testing.T) {
+	client := newRecorderClient()
+	l := newTestLogger(t, client)
+
+	payload := []byte("ok")
+	inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		cd := proxy.CapturedDataFromContext(r.Context())
+		require.NotNil(t, cd, "middleware must inject CapturedData")
+		cd.SetSuppressAccessLog(true)
+		w.WriteHeader(http.StatusOK)
+		_, _ = w.Write(payload)
+	})
+
+	srv := httptest.NewServer(l.Middleware(inner))
+	t.Cleanup(srv.Close)
+
+	resp, err := http.Get(srv.URL + "/agent-network/v1/chat/completions")
+	require.NoError(t, err)
+	require.NoError(t, resp.Body.Close())
+
+	// Allow trackUsage to land — it runs synchronously after l.log(entry) is
+	// (would have been) called.
+	time.Sleep(50 * time.Millisecond)
+
+	l.usageMux.Lock()
+	usage, present := l.domainUsage[hostNoPort(srv.URL)]
+	l.usageMux.Unlock()
+	require.True(t, present, "domain usage must be tracked even when the access-log is suppressed")
+	assert.Greater(t, usage.bytesTransferred, int64(0), "bytesTransferred must include the response payload")
+	assert.Equal(t, int64(0), client.callCount(),
+		"SendAccessLog must remain suppressed across the response write")
+}
+
+// hostNoPort extracts the host name from an httptest server URL. The
+// middleware strips the port before keying domain usage, so the test mirrors
+// that to look the entry up.
+func hostNoPort(url string) string {
+	// httptest URLs are always "http://127.0.0.1:PORT".
+	const prefix = "http://"
+	host := url[len(prefix):]
+	for i := 0; i < len(host); i++ {
+		if host[i] == ':' || host[i] == '/' {
+			return host[:i]
+		}
+	}
+	return host
+}