Files
netbird/proxy/internal/accesslog/middleware_test.go
Maycon Santos b416063bcc [management,proxy] Agent network: per-account LLM gateway (policy, metering, multi-provider) (#6555)
* [agent-network] Shared proto, OpenAPI schema, and generated types

* [agent-network] Management: store, manager, synthesizer, policy engine, provider catalog, HTTP/gRPC API

Adds the account-scoped agent-network module: provider/policy/budget CRUD and
store, the reverse-proxy service synthesizer, policy selection + limit
enforcement, the provider catalog (incl. Vertex AI and AWS Bedrock entries),
and the management HTTP + proxy gRPC surfaces.

* [management] Fix agent-network proxy-peer fan-out on affected-peer recompute

The affected-peers resolver loaded only persisted reverse-proxy services, but
agent-network services are synthesized on demand and never persisted. As a
result the embedded proxy peer was never folded into the affected set when a
client's group changed, so the proxy received no network-map update for a newly
authorised client and rejected its handshake until a full resync (restart).

loadProxyServices now merges the synthesized agent-network services (injected
via a registration hook to avoid an import cycle), so proxy peers learn newly
authorised clients immediately.

* [proxy] Reverse-proxy middleware framework, chain, and request plumbing

The per-target middleware chain (slots, dispatcher, mutation gate, metadata
merger), body capture, access-log terminal sink, and the proxy wiring that
builds + runs chains for synthesized agent-network services.

* [proxy] LLM parsers, pricing, and builtin middlewares (OpenAI, Anthropic, Vertex AI, AWS Bedrock)

Request/response parsers and SSE/event-stream metering, the embedded pricing
table, and the builtin middleware set: request parser, router, policy
limit-check/record, cost meter, guardrail, identity inject, response parser.
Includes the path-routed providers — Google Vertex AI (keyfile:: service-account
OAuth minting) and AWS Bedrock (bearer auth, invoke/converse/streaming, optional
/bedrock prefix) — plus the Models allowlist and unmeterable-publisher deny.

* [proxy] IPv6 in-place apply and TCP accept-loop hardening on netstack listeners

* [agent-network] End-to-end test suite, module docs, and deployment preset

* [agent-network] Fix codespell typos and exclude false positives

- labelgen word pool: vermillion -> vermilion, racoon -> raccoon.
- codespell ignore list: add flate (Go compress/flate package), recordin
  (a test-local identifier), and unparseable (a valid alternative spelling used
  consistently across identifiers + a metadata-value constant).

* [management] Set LastSeen on injected proxy peer in realstack test (MySQL strict-mode)

The injected embedded proxy peer had a PeerStatus with a zero LastSeen, which
serializes to '0000-00-00' and is rejected by MySQL in strict mode (SQLite
tolerates it). Set LastSeen to a valid time so SaveAccount succeeds on both
engines.

* [agent-network] Remove e2e shell-script suite from this branch

The end-to-end shell scripts under scripts/e2e/ are maintained in a separate
testing suite and are not part of this change set.

* [agent-network] Polish module docs: remove internal review scaffolding, fix links, verify diagrams

Strip PR-review framing, commit references, absolute paths, and stale internal
references from the agent-network module docs; fix broken relative links; verify
all diagrams against the current architecture. Remove the internal AI-reviewer
prompt file.

* [management] Refine session expiration handling to support 3-state encoding for SSO deadlines

* [agent-network] Relocate agentnetwork package to internals/modules

Move management/server/agentnetwork (and its catalog/, labelgen/, types/
subpackages) to management/internals/modules/agentnetwork, alongside the
reverse-proxy module, and rewrite all importers. Pure relocation: package names,
the synthesizer + affectedpeers registration hook, and store access (shared
store.Store) are unchanged, so no import cycle is introduced (affectedpeers
still depends only on the agentnetwork/types leaf).

* [agent-network] Co-locate HTTP handlers in the module (RegisterEndpoints)

Move the agent-network HTTP handlers from server/http/handlers/agentnetwork into
the module at internals/modules/agentnetwork/handlers (package handlers) and
rename the entrypoint AddEndpoints -> RegisterEndpoints, matching the
reverse-proxy module convention. Wiring in http/handler.go updated accordingly.
2026-06-27 13:41:00 +02:00

186 lines
6.4 KiB
Go

package accesslog
import (
"context"
"net/http"
"net/http/httptest"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"github.com/netbirdio/netbird/proxy/internal/proxy"
"github.com/netbirdio/netbird/shared/management/proto"
)
// recorderClient is a minimal stub for the access-log gRPCClient interface. It
// counts SendAccessLog invocations and signals on every call so tests can
// deterministically wait for the goroutine inside Logger.log without sleeping.
type recorderClient struct {
mu sync.Mutex
calls int64
lastEntry *proto.AccessLog
called chan struct{}
}
func newRecorderClient() *recorderClient {
return &recorderClient{called: make(chan struct{}, 16)}
}
func (r *recorderClient) SendAccessLog(_ context.Context, in *proto.SendAccessLogRequest, _ ...grpc.CallOption) (*proto.SendAccessLogResponse, error) {
r.mu.Lock()
r.calls++
r.lastEntry = in.GetLog()
r.mu.Unlock()
select {
case r.called <- struct{}{}:
default:
}
return &proto.SendAccessLogResponse{}, nil
}
func (r *recorderClient) callCount() int64 {
r.mu.Lock()
defer r.mu.Unlock()
return r.calls
}
// newTestLogger builds a Logger backed by the supplied recorderClient. It is
// the same constructor production uses, just with a stub gRPC client — no
// mocks, no interface re-implementations.
func newTestLogger(t *testing.T, client *recorderClient) *Logger {
t.Helper()
logger := NewLogger(client, nil, nil)
t.Cleanup(logger.Close)
return logger
}
// TestMiddleware_SuppressAccessLog_SkipsLogSink asserts the suppression gate.
// When the inner handler stamps SuppressAccessLog=true on CapturedData (mirrors
// what reverseproxy does when the matched target's DisableAccessLog flag is
// set), the middleware must NOT invoke the access-log sink. Bandwidth telemetry
// (trackUsage) keeps running — it's the call to SendAccessLog that we gate.
func TestMiddleware_SuppressAccessLog_SkipsLogSink(t *testing.T) {
client := newRecorderClient()
l := newTestLogger(t, client)
inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
cd := proxy.CapturedDataFromContext(r.Context())
require.NotNil(t, cd, "middleware must inject CapturedData into the request context")
cd.SetSuppressAccessLog(true)
w.WriteHeader(http.StatusOK)
})
srv := httptest.NewServer(l.Middleware(inner))
t.Cleanup(srv.Close)
resp, err := http.Get(srv.URL + "/agent-network/v1/chat/completions")
require.NoError(t, err, "GET against suppressed target must succeed")
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, "inner handler must run normally")
// Give the goroutine fence a beat (Logger.log dispatches in a goroutine).
// The negative assertion needs a small window: if a send is going to
// happen, it happens promptly.
select {
case <-client.called:
t.Fatalf("access-log sink must not be invoked when SuppressAccessLog=true (got %d call(s))", client.callCount())
case <-time.After(150 * time.Millisecond):
}
assert.Equal(t, int64(0), client.callCount(),
"SendAccessLog must not be called for suppressed requests")
}
// TestMiddleware_SuppressAccessLog_DefaultEmitsLog is the regression sanity:
// when nothing sets SuppressAccessLog (the universal default for every
// non-agent-network target), the middleware MUST still emit the access-log
// entry. This is the guarantee that wires-through to the EnableLogCollection
// gate without breaking anyone who isn't opted in.
func TestMiddleware_SuppressAccessLog_DefaultEmitsLog(t *testing.T) {
client := newRecorderClient()
l := newTestLogger(t, client)
var innerRan atomic.Bool
inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
innerRan.Store(true)
// Intentionally DO NOT touch SuppressAccessLog — mirrors every
// non-agent-network target.
w.WriteHeader(http.StatusOK)
})
srv := httptest.NewServer(l.Middleware(inner))
t.Cleanup(srv.Close)
resp, err := http.Get(srv.URL + "/service/healthz")
require.NoError(t, err, "GET against default target must succeed")
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, "inner handler must run normally")
require.True(t, innerRan.Load(), "inner handler must have run")
select {
case <-client.called:
case <-time.After(2 * time.Second):
t.Fatalf("SendAccessLog must be invoked for non-suppressed requests, none observed (calls=%d)", client.callCount())
}
assert.Equal(t, int64(1), client.callCount(),
"non-suppressed request must produce exactly one access-log send")
}
// TestMiddleware_SuppressAccessLog_PreservesUsageTracking proves the gate is
// surgical: with SuppressAccessLog=true the access-log send is skipped, but
// the per-domain usage tracker still records the bytes transferred. This is
// the cost-monitoring guarantee called out in the gate's comment.
func TestMiddleware_SuppressAccessLog_PreservesUsageTracking(t *testing.T) {
client := newRecorderClient()
l := newTestLogger(t, client)
payload := []byte("ok")
inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
cd := proxy.CapturedDataFromContext(r.Context())
require.NotNil(t, cd, "middleware must inject CapturedData")
cd.SetSuppressAccessLog(true)
w.WriteHeader(http.StatusOK)
_, _ = w.Write(payload)
})
srv := httptest.NewServer(l.Middleware(inner))
t.Cleanup(srv.Close)
resp, err := http.Get(srv.URL + "/agent-network/v1/chat/completions")
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
// Allow trackUsage to land — it runs synchronously after l.log(entry) is
// (would have been) called.
time.Sleep(50 * time.Millisecond)
l.usageMux.Lock()
usage, present := l.domainUsage[hostNoPort(srv.URL)]
l.usageMux.Unlock()
require.True(t, present, "domain usage must be tracked even when the access-log is suppressed")
assert.Greater(t, usage.bytesTransferred, int64(0), "bytesTransferred must include the response payload")
assert.Equal(t, int64(0), client.callCount(),
"SendAccessLog must remain suppressed across the response write")
}
// hostNoPort extracts the host name from an httptest server URL. The
// middleware strips the port before keying domain usage, so the test mirrors
// that to look the entry up.
func hostNoPort(url string) string {
// httptest URLs are always "http://127.0.0.1:PORT".
const prefix = "http://"
host := url[len(prefix):]
for i := 0; i < len(host); i++ {
if host[i] == ':' || host[i] == '/' {
return host[:i]
}
}
return host
}