Compare commits

...

5 Commits

Author SHA1 Message Date
Maycon Santos
16b65729ae [proxy] full-chain regression test: Vertex model allowlist is enforced
Add a realstack test that drives Anthropic-on-Vertex requests (model in
the URL path, no model in the body — the shape Claude Code sends) through
the full synthesized middleware chain against an in-process management
stack and a fake Vertex upstream, reproducing the customer scenario end
to end.

With a guardrail allowlisting only Sonnet, an Opus rawPredict must be
denied by the guardrail (llm_policy.model_blocked) before the upstream,
and Sonnet must be served. Covers two provider shapes:

  - catch_all: provider registered with no models (historical Vertex
    default) — the router waves every model through, so this is the
    direct guard for "the disallowed model is never blocked".
  - versioned_models: provider registered with the raw "@version" ids —
    the router must normalize the registered id to route the
    version-stripped request model while the guardrail still blocks Opus.

A static bearer api key is used so the router injects a static auth
header instead of minting a GCP OAuth token (which needs network egress
and would deny before the guardrail runs, masking the decision under
test). The upstream-hit counter proves a denied request never leaks to
Google.
2026-07-22 17:56:07 +00:00
Maycon Santos
1b7d23ee9f [e2e] cover unversioned Vertex models, versioned config ids, and count-tokens
Extend the agent-network e2e to the Vertex request shapes a customer
actually sends through Claude Code:

- Register the Vertex provider's model under the raw "@version" id the
  Google console documents, so the router's candidate normalization is
  exercised end to end (previously the provider was created with no
  models array, leaving Vertex per-provider matching untested).
- Allowlist the raw "@version" id in the guardrail, so entry
  normalization is exercised against version-stripped request models.
- Drive the unversioned model id (…/models/claude-sonnet-4-5:rawPredict)
  through the tunnel and require 200 — the customer-reported shape.
- Drive the token-count endpoint (…/models/count-tokens:rawPredict, real
  model in the body) via the new harness VertexCountTokens: an
  allowlisted body model must serve 200, a disallowed one must deny 403
  before the upstream.
2026-07-22 17:56:07 +00:00
Maycon Santos
a69fe9ba8c [proxy] normalize Vertex model ids in routing, guardrail, and token counting
Vertex requests reach the router/guardrail with the "@version" suffix
already stripped from the URL model by the request parser, but the
operator-facing config may carry the raw versioned id the Google console
documents (e.g. "claude-opus-4-6@20250514"):

- a Vertex provider registered with versioned model ids denied every
  request as model_not_routable (the Vertex analog of the Bedrock gap
  fixed in #6773), and
- a guardrail allowlist entry with a versioned id never matched, denying
  the model as model_blocked.

Both bit a customer driving Anthropic-on-Vertex through the agent
network with unversioned model ids (…/models/claude-opus-4-6:rawPredict
at the global location).

Introduce a shared llm.NormalizeVertexModel (the same "@" stripping the
parser already does) and apply it to a Vertex route's candidate models in
routeClaimsModel and to guardrail allowlist entries, so either spelling
of the same model matches. Non-Vertex routes and entries without "@"
keep exact matching.

Also resolve the real model for the Vertex token-count endpoint:
…/models/count-tokens:rawPredict carries the literal "count-tokens"
pseudo-model in the URL and the actual model in the body (Claude Code
sends one such call per request), so any configured allowlist denied all
token counting. The parser now reads the body model for that endpoint;
an unreadable body keeps the pseudo-model and fails closed.
2026-07-22 17:56:06 +00:00
Maycon Santos
9770814f39 [client] warm lazy connections from the DNS resolver (#6854)
## Describe your changes

Supersedes #6767 (same change, moved to an unprefixed branch; review
feedback from there is addressed here).

With lazy connections enabled, a peer is not dialed until on-demand
traffic arrives, so the first request to a peer resolved by name (e.g.
the agent-network reverse-proxy) races — or loses to — the WireGuard
handshake. Activation was previously reactive only: a data-path packet
or an inbound signal.

This adds a proactive, DNS-time trigger. When the local resolver answers
for an overlay name, it now warms the lazy connection to the peer(s) the
answer points at and waits briefly for one to connect before returning
the response — so by the time the client sends its first packet the
tunnel is already up.

- `dns/local`: new `PeerActivator` capability + `SetPeerActivator`
setter (mirrors the existing `PeerConnectivity`/`SetPeerConnectivity`
injection). `ServeDNS` warms on the pre-filter answer, so activating a
lazily-idle peer also lets it survive the disconnected-peer filter.
Warm-up is scoped to match-only (non-authoritative) zones — the
synthesized private-service zones and user-created zones — so plain
peer-name lookups in the account's authoritative peer zone never wake
idle peers. No-op when no activator is wired (lazy off) or the answer
carries no peer IPs. Budget is `NB_DNS_LAZY_WARMUP_TIMEOUT` (default 2s,
parsed once at construction, invalid values logged); on timeout the
answer is returned anyway (never SERVFAIL).
- The resolver-facing interfaces (`PeerActivator`, `PeerConnectivity`)
take `netip.Addr` instead of string IPs; record addresses are extracted
as `netip.Addr` (v4-mapped forms unmapped) and converted to string only
at the `peer.Status` boundary.
- `SetPeerActivator` is part of the `dns.Server` interface (no-op on the
mock), so the engine wires it without a type assertion.
- `client/internal`: a small engine-side adapter (`dnsPeerActivator`)
resolves answer IPs to peers via `Status.PeerStateByIP`, activates them
through `ConnMgr.ActivatePeer` (HA fan-out included), and polls
`PeerStateByIP` until one is connected. The activation dial is tied to
the engine's long-lived context so a handshake that outlasts the
per-query wait still completes in the background. `ConnMgr.ActivatePeer`
is safe for concurrent use (the lazy manager pointer is guarded by a
dedicated RWMutex and the manager is internally synchronized), so the
DNS path never contends with network-map processing on `syncMsgMux`.

Scope is overlay-only for free: the trigger lives in the local resolver,
which only answers for NetBird-managed names; public/upstream DNS is
unaffected. Already-connected peers short-circuit, so steady-state DNS
latency is unchanged.

## Issue ticket number and link

N/A — follow-up to the lazy-connection rollout; fixes the agent-network
cold-start observed in the e2e (proxy peer stuck disconnected until
traffic).

### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [x] Created tests that fail without the change (if possible)
- [x] This change does **not** modify the public API, gRPC protocols,
functionality behavior, CLI / service flags, or introduce a new feature
— **OR** I have discussed it with the NetBird team beforehand (link the
issue / Slack thread in the description). See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

Internal client behavior; the only knob is the optional
`NB_DNS_LAZY_WARMUP_TIMEOUT` tuning env var with a safe default.

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

## Tests

- `dns/local`: warm-up invokes the activator with the answer's peer
address in match-only zones; authoritative-zone answers never trigger
warm-up; no-activator path unchanged; no-answer queries don't invoke the
activator; `NB_DNS_LAZY_WARMUP_TIMEOUT` parsing
(valid/invalid/non-positive); `extractRecordAddr` unmaps v4-mapped
record data.
- `client/internal`: `dnsPeerActivator` skips
connected/unknown/conn-less peers with no wait, returns as soon as a
pending peer connects, and releases the DNS response at the budget when
the peer stays idle; `ConnMgr.ActivatePeer` races the manager lifecycle
cleanly under `-race`.
- Agent-network e2e green on this change (with lazy connections
enabled): https://github.com/netbirdio/netbird/actions/runs/29891467544
2026-07-22 18:47:48 +02:00
Nicolas Frati
b0c1ed31b8 [infrastructure] Add unified admin CLI for self-hosted helpers (#6507)
## Summary

Adds a unified `admin` CLI for self-hosted instance administrators in
both the management and combined binaries.

## User Management

### `admin user change-password`
- Changes a local embedded IdP user's password.
- Selects the user with `--email` or `--user-id`.
- Reads the new password from `--password` or `--password-file`.
- Clears the user's local authentication session so the new password is
required on the next login.
- **Alias:** `admin user set-password`.

### `admin user reset-mfa`
- Resets a local embedded IdP user's MFA enrollment.
- Selects the user with `--email` or `--user-id`.
- Clears TOTP/WebAuthn enrollment data and removes the local
authentication session.
- The user will re-enroll MFA on the next login.

## MFA Management

### `admin mfa status`
- Shows whether local MFA is enabled in the account settings.
- Checks the embedded IdP client configuration and reports whether MFA
is enabled there.

### `admin mfa enable`
- Enables local MFA for embedded IdP users.
- Updates embedded IdP clients and saves the account MFA setting.
- Records an audit event on a best-effort basis.

### `admin mfa disable`
- Disables local MFA for embedded IdP users.
- Updates embedded IdP clients and saves the account MFA setting.
- Records an audit event on a best-effort basis.

## Reverse Proxy Tokens

### `admin token create --name <name> [--expires-in <duration>]`
- Creates a reverse proxy access token.
- Prints the plaintext token once, along with the token ID.
- `--expires-in` supports values such as `24h`, `30d`, or `365d`. If
omitted, the token never expires.

### `admin token list`
- Lists reverse proxy access tokens.
- Shows the token ID, name, creation date, expiration, last-used time,
and revocation status.
- **Alias:** `admin token ls`.

### `admin token revoke <token-id>`
- Revokes a reverse proxy access token.
- Revoked tokens can no longer authenticate reverse proxy instances.

## Reverse Proxy Management

### `admin proxy disconnect-all`
- Lists registered reverse proxy instances and force-marks all connected
instances as disconnected.
- Useful for repairing stale proxy state after an unclean management
server shutdown.
- Prompts for confirmation by default.
- `--dry-run` previews the changes without applying them.
- `--force` skips the confirmation prompt.
- Live proxies may appear again after their next heartbeat, reconnect,
or re-registration.

## Compatibility Commands

### `token ...`
- Deprecated top-level compatibility path.
- Behaves the same as `admin token ...`.
- Retained so existing scripts using `token create`, `token list`, or
`token revoke` continue to work.

## Changes

- Adds reusable `management/cmd/admin` command package.
- Wires `admin` into `netbird-mgmt` and `combined`.
- Adds local user password reset with existing password strength
validation.
- Adds local MFA enrollment reset by clearing Dex TOTP/WebAuthn
credentials and local auth sessions.
- Adds local MFA enable/disable/status helpers for embedded IdP
deployments.
- Moves proxy access token commands under `admin token` for a single
admin-focused CLI entry point.
- Exports `server.ValidatePassword` for reuse by CLI helpers.

## Tests

```bash
go test ./management/cmd/...
go test ./management/cmd/admin ./management/cmd ./combined/cmd
go test ./management/server -run TestValidatePassword
```

Pre-push lint also passed.


### Checklist
- [ ] Is it a bug fix
- [ ] Is a typo/documentation fix
- [x] Is a feature enhancement
- [ ] It is a refactor
- [ ] Created tests that fail without the change (if possible)
- [ ] This change does **not** modify the public API, gRPC protocols,
functionality behavior, CLI / service flags, or introduce a new feature
— **OR** I have discussed it with the NetBird team beforehand (link the
issue / Slack thread in the description). See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [x] I added/updated documentation for this change
- [ ] Documentation is **not needed** for this change (explain why)

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/832

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added self-hosted admin CLI commands for changing passwords, resetting
MFA (including WebAuthn), and managing embedded IdP client MFA
(enable/disable/status).
* Introduced a unified admin command entry point and improved
data-directory handling for embedded IdP storage.

* **Refactor**
* Centralized password strength validation into a shared exported
validator.

* **Tests**
* Added a comprehensive admin command test suite covering password
input, selectors, MFA reset, and client MFA state handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-22 18:47:19 +02:00
49 changed files with 3234 additions and 242 deletions

View File

@@ -34,6 +34,8 @@ const (
// - Handling connection establishment based on peer signaling
//
// The implementation is not thread-safe; it is protected by engine.syncMsgMux.
// The only exception is ActivatePeer, which is safe for concurrent use so the
// DNS warm-up path can call it without contending on the engine mutex.
type ConnMgr struct {
peerStore *peerstore.Store
statusRecorder *peer.Status
@@ -42,6 +44,10 @@ type ConnMgr struct {
rosenpassEnabled bool
lazyConnMgr *manager.Manager
// lazyConnMgrMu guards the lazyConnMgr pointer for readers outside the
// engine loop (ActivatePeer). Writers hold it in addition to
// engine.syncMsgMux; all other reads stay under engine.syncMsgMux only.
lazyConnMgrMu sync.RWMutex
wg sync.WaitGroup
lazyCtx context.Context
@@ -238,12 +244,20 @@ func (e *ConnMgr) RemovePeerConn(peerKey string) {
conn.Log.Infof("removed peer from lazy conn manager")
}
// ActivatePeer wakes an idle lazy connection. Unlike the rest of ConnMgr it is
// safe for concurrent use: the lazy manager pointer is read under lazyConnMgrMu
// and the manager itself is internally synchronized, so callers outside the
// engine loop (DNS warm-up) do not need engine.syncMsgMux.
func (e *ConnMgr) ActivatePeer(ctx context.Context, conn *peer.Conn) {
if !e.isStartedWithLazyMgr() {
e.lazyConnMgrMu.RLock()
lazyConnMgr := e.lazyConnMgr
started := lazyConnMgr != nil && e.lazyCtxCancel != nil
e.lazyConnMgrMu.RUnlock()
if !started {
return
}
if found := e.lazyConnMgr.ActivatePeer(conn.GetKey()); found {
if found := lazyConnMgr.ActivatePeer(conn.GetKey()); found {
if err := conn.Open(ctx); err != nil {
conn.Log.Errorf("failed to open connection: %v", err)
}
@@ -268,16 +282,21 @@ func (e *ConnMgr) Close() {
e.lazyCtxCancel()
e.wg.Wait()
e.lazyConnMgrMu.Lock()
e.lazyConnMgr = nil
e.lazyConnMgrMu.Unlock()
}
func (e *ConnMgr) initLazyManager(engineCtx context.Context) {
cfg := manager.Config{
InactivityThreshold: inactivityThresholdEnv(),
}
e.lazyConnMgr = manager.NewManager(cfg, engineCtx, e.peerStore, e.iface)
e.lazyConnMgrMu.Lock()
e.lazyConnMgr = manager.NewManager(cfg, engineCtx, e.peerStore, e.iface)
e.lazyCtx, e.lazyCtxCancel = context.WithCancel(engineCtx)
e.lazyConnMgrMu.Unlock()
e.wg.Add(1)
go func() {
@@ -316,7 +335,10 @@ func (e *ConnMgr) closeManager(ctx context.Context) {
e.lazyCtxCancel()
e.wg.Wait()
e.lazyConnMgrMu.Lock()
e.lazyConnMgr = nil
e.lazyConnMgrMu.Unlock()
for _, peerID := range e.peerStore.PeersPubKey() {
e.peerStore.PeerConnOpen(ctx, peerID)

View File

@@ -1,10 +1,21 @@
package internal
import (
"context"
"net"
"net/netip"
"os"
"sync"
"testing"
"time"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"github.com/netbirdio/netbird/client/iface/wgaddr"
"github.com/netbirdio/netbird/client/internal/lazyconn"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/peerstore"
"github.com/netbirdio/netbird/monotime"
)
func TestResolveLazyForce(t *testing.T) {
@@ -38,3 +49,58 @@ func TestResolveLazyForce(t *testing.T) {
})
}
}
type mockLazyWGIface struct{}
func (mockLazyWGIface) RemovePeer(string) error { return nil }
func (mockLazyWGIface) UpdatePeer(string, []netip.Prefix, time.Duration, *net.UDPAddr, *wgtypes.Key) error {
return nil
}
func (mockLazyWGIface) IsUserspaceBind() bool { return false }
func (mockLazyWGIface) Address() wgaddr.Address { return wgaddr.Address{} }
func (mockLazyWGIface) LastActivities() map[string]monotime.Time { return nil }
func (mockLazyWGIface) MTU() uint16 { return 1280 }
// TestConnMgr_ActivatePeerConcurrentWithLifecycle exercises ActivatePeer from
// non-engine goroutines (the DNS warm-up path) racing the manager lifecycle,
// which stays on the engine loop. Run with -race: it fails if ActivatePeer
// still requires engine.syncMsgMux for safety.
func TestConnMgr_ActivatePeerConcurrentWithLifecycle(t *testing.T) {
t.Setenv(lazyconn.EnvLazyConn, "on")
status := peer.NewRecorder("https://mgm")
store := peerstore.NewConnStore()
connMgr := NewConnMgr(&EngineConfig{}, status, store, mockLazyWGIface{})
conn := newTestPeerConn(t, "peerA")
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
connMgr.Start(ctx)
done := make(chan struct{})
var wg sync.WaitGroup
for range 4 {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-done:
return
default:
connMgr.ActivatePeer(ctx, conn)
}
}
}()
}
// Let the activators spin against the started manager, then tear it down
// underneath them and let them spin against the stopped manager.
time.Sleep(100 * time.Millisecond)
connMgr.Close()
time.Sleep(50 * time.Millisecond)
close(done)
wg.Wait()
}

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"net"
"net/netip"
"os"
"slices"
"strings"
"sync"
@@ -36,7 +37,43 @@ type resolver interface {
// record is left alone (it points at something outside our mesh, e.g.
// a non-peer upstream).
type PeerConnectivity interface {
IsConnectedByIP(ip string) (known, connected bool)
IsConnectedByIP(ip netip.Addr) (known, connected bool)
}
// PeerActivator wakes lazy-connection peers on demand. The local resolver calls
// it with the tunnel IPs an answer points at, so a peer that is idle (lazily
// disconnected) starts connecting at DNS-resolution time rather than racing the
// client's first request packet. nil disables warm-up.
type PeerActivator interface {
// ActivatePeersByIP triggers wake-up for the peer(s) owning addrs and blocks
// until one is connected or ctx (a short per-query budget) expires. It is a
// fast no-op for unknown or already-connected addresses.
ActivatePeersByIP(ctx context.Context, addrs []netip.Addr)
}
const (
defaultLazyWarmupTimeout = 2 * time.Second
envLazyWarmupTimeout = "NB_DNS_LAZY_WARMUP_TIMEOUT"
)
// lazyWarmupTimeoutFromEnv returns the per-query budget for waking a
// lazy-connection peer a DNS answer points at. Tunable via
// NB_DNS_LAZY_WARMUP_TIMEOUT (a Go duration). Parsed once at construction time.
func lazyWarmupTimeoutFromEnv() time.Duration {
v := os.Getenv(envLazyWarmupTimeout)
if v == "" {
return defaultLazyWarmupTimeout
}
d, err := time.ParseDuration(v)
if err != nil {
log.Warnf("invalid %s value %q, using default %s: %v", envLazyWarmupTimeout, v, defaultLazyWarmupTimeout, err)
return defaultLazyWarmupTimeout
}
if d <= 0 {
log.Warnf("non-positive %s value %q, using default %s", envLazyWarmupTimeout, v, defaultLazyWarmupTimeout)
return defaultLazyWarmupTimeout
}
return d
}
type Resolver struct {
@@ -51,6 +88,12 @@ type Resolver struct {
// filter and preserves the legacy "return whatever is registered"
// behaviour for callers that never wire a status source.
peerConn PeerConnectivity
// peerActivator, when non-nil, is called at resolution time to warm the
// lazy connection to the peer(s) an answer points at. nil disables warm-up.
peerActivator PeerActivator
// warmupTimeout is the per-query budget for the lazy-connection warm-up
// wait, resolved from the environment once at construction time.
warmupTimeout time.Duration
ctx context.Context
cancel context.CancelFunc
@@ -59,11 +102,12 @@ type Resolver struct {
func NewResolver() *Resolver {
ctx, cancel := context.WithCancel(context.Background())
return &Resolver{
records: make(map[dns.Question][]dns.RR),
domains: make(map[domain.Domain]struct{}),
zones: make(map[domain.Domain]bool),
ctx: ctx,
cancel: cancel,
records: make(map[dns.Question][]dns.RR),
domains: make(map[domain.Domain]struct{}),
zones: make(map[domain.Domain]bool),
warmupTimeout: lazyWarmupTimeoutFromEnv(),
ctx: ctx,
cancel: cancel,
}
}
@@ -76,6 +120,14 @@ func (d *Resolver) SetPeerConnectivity(p PeerConnectivity) {
d.peerConn = p
}
// SetPeerActivator wires the DNS-time lazy-connection warm-up. Pass nil to
// disable. Safe to call multiple times; the latest value wins.
func (d *Resolver) SetPeerActivator(a PeerActivator) {
d.mu.Lock()
defer d.mu.Unlock()
d.peerActivator = a
}
func (d *Resolver) MatchSubdomains() bool {
return true
}
@@ -122,6 +174,9 @@ func (d *Resolver) ServeDNS(w dns.ResponseWriter, r *dns.Msg) {
replyMessage.RecursionAvailable = true
result := d.lookupRecords(logger, question)
// Warm before filtering: activation flips a lazily-idle target to connected,
// which then lets it survive the disconnected-peer filter below.
d.warmLazyPeers(question, result.records)
result.records = d.filterDisconnectedPeerAnswers(logger, question, result.records)
replyMessage.Authoritative = !result.hasExternalData
replyMessage.Answer = result.records
@@ -495,8 +550,8 @@ func (d *Resolver) filterDisconnectedPeerAnswers(logger *log.Entry, question dns
kept := make([]dns.RR, 0, len(records))
var dropped int
for _, rr := range records {
ip := extractRecordIP(rr)
if ip == "" {
ip, ok := extractRecordAddr(rr)
if !ok {
kept = append(kept, rr)
continue
}
@@ -518,22 +573,57 @@ func (d *Resolver) filterDisconnectedPeerAnswers(logger *log.Entry, question dns
return kept
}
// extractRecordIP returns the dotted-decimal / colon-hex IP carried by
// an A or AAAA record, or "" for any other record type.
func extractRecordIP(rr dns.RR) string {
// warmLazyPeers triggers lazy-connection wake-up for the peers a resolved
// answer points at and waits briefly for one to connect, so the caller's first
// request doesn't race the connection establishment. Warm-up is scoped to
// match-only (non-authoritative) zones — the synthesized private-service zones
// and user-created zones whose records point at specific peers. The account's
// peer zone is authoritative, so plain peer-name lookups never trigger warm-up;
// otherwise resolving any peer's name would wake its idle connection, defeating
// laziness mesh-wide. No-op when no activator is wired (lazy connections
// disabled) or the answer carries no peer IPs.
func (d *Resolver) warmLazyPeers(question dns.Question, records []dns.RR) {
if len(records) < 2 {
return
}
d.mu.RLock()
activator := d.peerActivator
var nonAuth, found bool
if activator != nil {
nonAuth, found = d.findZone(question.Name)
}
d.mu.RUnlock()
if activator == nil || !found || !nonAuth {
return
}
var addrs []netip.Addr
for _, rr := range records {
if addr, ok := extractRecordAddr(rr); ok {
addrs = append(addrs, addr)
}
}
if len(addrs) == 0 {
return
}
ctx, cancel := context.WithTimeout(d.ctx, d.warmupTimeout)
defer cancel()
activator.ActivatePeersByIP(ctx, addrs)
}
// extractRecordAddr returns the IP address carried by an A or AAAA record.
// ok is false for any other record type or a record with no address.
func extractRecordAddr(rr dns.RR) (netip.Addr, bool) {
switch r := rr.(type) {
case *dns.A:
if r.A == nil {
return ""
}
return r.A.String()
addr, ok := netip.AddrFromSlice(r.A)
return addr.Unmap(), ok
case *dns.AAAA:
if r.AAAA == nil {
return ""
}
return r.AAAA.String()
addr, ok := netip.AddrFromSlice(r.AAAA)
return addr.Unmap(), ok
}
return ""
return netip.Addr{}, false
}
// Update replaces all zones and their records

View File

@@ -37,8 +37,8 @@ type mockPeerConnectivity struct {
byIP map[string]struct{ known, connected bool }
}
func (m mockPeerConnectivity) IsConnectedByIP(ip string) (known, connected bool) {
v, ok := m.byIP[ip]
func (m mockPeerConnectivity) IsConnectedByIP(ip netip.Addr) (known, connected bool) {
v, ok := m.byIP[ip.String()]
if !ok {
return false, false
}

View File

@@ -0,0 +1,204 @@
package local
import (
"context"
"net"
"net/netip"
"sync"
"testing"
"time"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/internal/dns/test"
nbdns "github.com/netbirdio/netbird/dns"
)
// recordingActivator records the addresses it was asked to warm and returns
// immediately, so ServeDNS is not blocked by the test.
type recordingActivator struct {
mu sync.Mutex
called bool
addrs []netip.Addr
}
func (r *recordingActivator) ActivatePeersByIP(_ context.Context, addrs []netip.Addr) {
r.mu.Lock()
defer r.mu.Unlock()
r.called = true
r.addrs = append(r.addrs, addrs...)
}
func serveA(t *testing.T, resolver *Resolver, name string) *dns.Msg {
t.Helper()
var resp *dns.Msg
w := &test.MockResponseWriter{WriteMsgFunc: func(m *dns.Msg) error { resp = m; return nil }}
resolver.ServeDNS(w, new(dns.Msg).SetQuestion(name, dns.TypeA))
return resp
}
// serviceZone registers rec in a match-only (non-authoritative) zone, the shape
// the synthesized private-service zones arrive in.
func serviceZone(t *testing.T, resolver *Resolver, zone string, records ...nbdns.SimpleRecord) {
t.Helper()
resolver.Update([]nbdns.CustomZone{{
Domain: zone,
Records: records,
NonAuthoritative: true,
}})
}
func TestLocalResolver_WarmsLazyPeerOnResolve(t *testing.T) {
// Warm-up fires only for multi-record answers (the HA / round-robin shape of
// the synthesized private-service zones), so register two peer targets.
const name = "svc.proxy.netbird.cloud."
recs := []nbdns.SimpleRecord{
{Name: name, Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.7"},
{Name: name, Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.8"},
}
resolver := NewResolver()
serviceZone(t, resolver, "proxy.netbird.cloud", recs...)
act := &recordingActivator{}
resolver.SetPeerActivator(act)
resp := serveA(t, resolver, name)
require.NotNil(t, resp, "resolver must answer")
require.NotEmpty(t, resp.Answer, "answer must carry the A records")
act.mu.Lock()
defer act.mu.Unlock()
assert.True(t, act.called, "activator must be invoked for a multi-record service-zone answer")
assert.Contains(t, act.addrs, netip.MustParseAddr("100.64.0.7"), "activator must receive the first peer IP")
assert.Contains(t, act.addrs, netip.MustParseAddr("100.64.0.8"), "activator must receive the second peer IP")
}
func TestLocalResolver_NoWarmupForSingleRecord(t *testing.T) {
// A single-record answer does not trigger warm-up; the resolver only warms
// multi-record answers.
rec := nbdns.SimpleRecord{Name: "svc.proxy.netbird.cloud.", Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.7"}
resolver := NewResolver()
serviceZone(t, resolver, "proxy.netbird.cloud", rec)
act := &recordingActivator{}
resolver.SetPeerActivator(act)
resp := serveA(t, resolver, rec.Name)
require.NotNil(t, resp, "resolver must answer")
require.NotEmpty(t, resp.Answer, "answer must carry the A record")
act.mu.Lock()
defer act.mu.Unlock()
assert.False(t, act.called, "activator must not be invoked for a single-record answer")
}
func TestLocalResolver_NoActivatorNoWarmup(t *testing.T) {
// With no activator wired the resolver behaves exactly as before.
rec := nbdns.SimpleRecord{Name: "svc.proxy.netbird.cloud.", Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.7"}
resolver := NewResolver()
serviceZone(t, resolver, "proxy.netbird.cloud", rec)
resp := serveA(t, resolver, rec.Name)
require.NotNil(t, resp, "resolver must still answer without an activator")
require.NotEmpty(t, resp.Answer, "answer must carry the A record")
}
func TestLocalResolver_NoWarmupForMissingRecord(t *testing.T) {
// A query that resolves to nothing must not invoke the activator (no IPs).
resolver := NewResolver()
serviceZone(t, resolver, "proxy.netbird.cloud",
nbdns.SimpleRecord{Name: "svc.proxy.netbird.cloud.", Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.7"})
act := &recordingActivator{}
resolver.SetPeerActivator(act)
serveA(t, resolver, "absent.proxy.netbird.cloud.")
act.mu.Lock()
defer act.mu.Unlock()
assert.False(t, act.called, "activator must not be invoked when there is no answer")
}
func TestLocalResolver_NoWarmupInAuthoritativeZone(t *testing.T) {
// The account's peer zone is authoritative; resolving a peer's name there
// must not wake its lazy connection — warm-up is scoped to match-only
// (non-authoritative) zones such as the synthesized private-service zones.
// Use a multi-record answer so the authoritative-zone scoping is the only
// reason warm-up is skipped, not the single-record guard.
const name = "peer.netbird.cloud."
recs := []nbdns.SimpleRecord{
{Name: name, Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.9"},
{Name: name, Type: 1, Class: nbdns.DefaultClass, TTL: 300, RData: "100.64.0.10"},
}
resolver := NewResolver()
resolver.Update([]nbdns.CustomZone{{
Domain: "netbird.cloud",
Records: recs,
}})
act := &recordingActivator{}
resolver.SetPeerActivator(act)
resp := serveA(t, resolver, name)
require.NotNil(t, resp, "resolver must answer")
require.NotEmpty(t, resp.Answer, "answer must carry the A records")
act.mu.Lock()
defer act.mu.Unlock()
assert.False(t, act.called, "activator must not be invoked for authoritative-zone answers")
}
func TestLazyWarmupTimeoutFromEnv(t *testing.T) {
tests := []struct {
name string
value string
envSet bool
want time.Duration
}{
{name: "unset uses default", want: defaultLazyWarmupTimeout},
{name: "valid overrides", value: "5s", envSet: true, want: 5 * time.Second},
{name: "invalid falls back", value: "not-a-duration", envSet: true, want: defaultLazyWarmupTimeout},
{name: "negative falls back", value: "-1s", envSet: true, want: defaultLazyWarmupTimeout},
{name: "zero falls back", value: "0s", envSet: true, want: defaultLazyWarmupTimeout},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.envSet {
t.Setenv(envLazyWarmupTimeout, tt.value)
}
assert.Equal(t, tt.want, lazyWarmupTimeoutFromEnv())
assert.Equal(t, tt.want, NewResolver().warmupTimeout, "constructor must resolve the timeout once")
})
}
}
func TestExtractRecordAddr(t *testing.T) {
t.Run("A record yields unmapped v4", func(t *testing.T) {
// net.ParseIP returns the 16-byte v4-in-v6 form, the same shape
// miekg/dns stores after parsing an A record; the extracted address
// must compare equal to a plain v4 netip.Addr.
addr, ok := extractRecordAddr(&dns.A{A: net.ParseIP("100.64.0.7")})
require.True(t, ok)
assert.True(t, addr.Is4())
assert.Equal(t, netip.MustParseAddr("100.64.0.7"), addr)
})
t.Run("AAAA record yields v6", func(t *testing.T) {
addr, ok := extractRecordAddr(&dns.AAAA{AAAA: net.ParseIP("fd00::1")})
require.True(t, ok)
assert.Equal(t, netip.MustParseAddr("fd00::1"), addr)
})
t.Run("A record without address", func(t *testing.T) {
_, ok := extractRecordAddr(&dns.A{})
assert.False(t, ok)
})
t.Run("non-address record", func(t *testing.T) {
_, ok := extractRecordAddr(&dns.CNAME{Target: "target.netbird.cloud."})
assert.False(t, ok)
})
}

View File

@@ -8,6 +8,7 @@ import (
"github.com/miekg/dns"
dnsconfig "github.com/netbirdio/netbird/client/internal/dns/config"
"github.com/netbirdio/netbird/client/internal/dns/local"
nbdns "github.com/netbirdio/netbird/dns"
"github.com/netbirdio/netbird/route"
"github.com/netbirdio/netbird/shared/management/domain"
@@ -92,6 +93,11 @@ func (m *MockServer) SetFirewall(Firewall) {
// Mock implementation - no-op
}
// SetPeerActivator mock implementation of SetPeerActivator from Server interface
func (m *MockServer) SetPeerActivator(local.PeerActivator) {
// Mock implementation - no-op
}
// BeginBatch mock implementation of BeginBatch from Server interface
func (m *MockServer) BeginBatch() {
// Mock implementation - no-op

View File

@@ -82,6 +82,7 @@ type Server interface {
PopulateManagementDomain(mgmtURL *url.URL) error
SetRouteSources(selected, active func() route.HAMap)
SetFirewall(Firewall)
SetPeerActivator(local.PeerActivator)
}
type nsGroupsByDomain struct {
@@ -491,6 +492,13 @@ func (s *DefaultServer) SetFirewall(fw Firewall) {
}
}
// SetPeerActivator wires the DNS-time lazy-connection warm-up on the local
// resolver. Injected after the connection manager exists (it does not at
// DNS-server construction time). Pass nil to disable.
func (s *DefaultServer) SetPeerActivator(a local.PeerActivator) {
s.localResolver.SetPeerActivator(a)
}
// Stop stops the server
func (s *DefaultServer) Stop() {
s.ctxCancel()
@@ -1435,11 +1443,11 @@ type localPeerConnectivity struct {
// IsConnectedByIP looks the IP up in the peerstore and surfaces both
// the known and connected bits. Used by Resolver.filterDisconnectedPeerAnswers.
func (l localPeerConnectivity) IsConnectedByIP(ip string) (known, connected bool) {
func (l localPeerConnectivity) IsConnectedByIP(ip netip.Addr) (known, connected bool) {
if l.status == nil {
return false, false
}
state, ok := l.status.PeerStateByIP(ip)
state, ok := l.status.PeerStateByIP(ip.String())
if !ok {
return false, false
}

View File

@@ -0,0 +1,76 @@
package internal
import (
"context"
"net/netip"
"time"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/peerstore"
)
const dnsActivationPollInterval = 50 * time.Millisecond
// dnsPeerActivator wakes lazy-connection peers from the DNS resolution path. It
// implements dns/local.PeerActivator. DNS queries run on their own goroutines,
// so it only touches state that is safe for concurrent use — ConnMgr.ActivatePeer,
// peerstore.Store and peer.Status — and never takes the engine's syncMsgMux,
// keeping DNS resolution from contending with network-map processing.
type dnsPeerActivator struct {
connMgr *ConnMgr
peerStore *peerstore.Store
status *peer.Status
// ctx is the engine's long-lived context. The connection dial is tied to it
// (not the per-query DNS wait budget) so a handshake that outlasts the wait
// still completes in the background rather than being cancelled at the deadline.
ctx context.Context
}
// ActivatePeersByIP triggers wake-up for the peer(s) owning addrs and waits
// until one is connected or ctx (the per-query DNS wait budget) expires.
// Activation itself is tied to the engine's long-lived context so the dial
// survives a wait that times out. Unknown or already-connected addresses are
// skipped, so the steady-state (warm) path adds no latency.
func (a *dnsPeerActivator) ActivatePeersByIP(ctx context.Context, addrs []netip.Addr) {
if a == nil || a.connMgr == nil {
return
}
var pending []string
for _, addr := range addrs {
ip := addr.String()
st, ok := a.status.PeerStateByIP(ip)
if !ok || st.ConnStatus == peer.StatusConnected {
continue
}
conn, ok := a.peerStore.PeerConn(st.PubKey)
if !ok {
continue
}
a.connMgr.ActivatePeer(a.ctx, conn)
pending = append(pending, ip)
}
if len(pending) == 0 {
return
}
a.waitConnected(ctx, pending)
}
// waitConnected blocks until any of ips reports a connected peer or ctx expires.
func (a *dnsPeerActivator) waitConnected(ctx context.Context, ips []string) {
ticker := time.NewTicker(dnsActivationPollInterval)
defer ticker.Stop()
for {
for _, ip := range ips {
if st, ok := a.status.PeerStateByIP(ip); ok && st.ConnStatus == peer.StatusConnected {
return
}
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}

View File

@@ -0,0 +1,129 @@
package internal
import (
"context"
"net/netip"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/peerstore"
)
func newTestPeerConn(t *testing.T, key string) *peer.Conn {
t.Helper()
conn, err := peer.NewConn(peer.ConnConfig{
Key: key,
LocalKey: "local",
WgConfig: peer.WgConfig{
AllowedIps: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
},
}, peer.ServiceDependencies{})
require.NoError(t, err)
return conn
}
func newTestDNSPeerActivator(t *testing.T) (*dnsPeerActivator, *peer.Status, *peerstore.Store) {
t.Helper()
status := peer.NewRecorder("https://mgm")
store := peerstore.NewConnStore()
// ConnMgr without Start: the lazy manager is nil, so ActivatePeer is a
// no-op — these tests exercise the activator's skip/wait logic.
connMgr := NewConnMgr(&EngineConfig{}, status, store, nil)
return &dnsPeerActivator{
connMgr: connMgr,
peerStore: store,
status: status,
ctx: context.Background(),
}, status, store
}
func TestDNSPeerActivator_NilSafe(t *testing.T) {
var a *dnsPeerActivator
a.ActivatePeersByIP(context.Background(), []netip.Addr{netip.MustParseAddr("100.64.0.1")})
}
// TestDNSPeerActivator_SkipsUnknownAndConnectedPeers verifies the steady-state
// (warm) path adds no latency: already-connected and unknown addresses never
// enter the wait loop.
func TestDNSPeerActivator_SkipsUnknownAndConnectedPeers(t *testing.T) {
a, status, store := newTestDNSPeerActivator(t)
require.NoError(t, status.AddPeer("peerA", "a.netbird.cloud", "100.64.0.1", "fd00::1"))
require.NoError(t, status.UpdatePeerState(peer.State{PubKey: "peerA", ConnStatus: peer.StatusConnected}))
store.AddPeerConn("peerA", newTestPeerConn(t, "peerA"))
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
start := time.Now()
a.ActivatePeersByIP(ctx, []netip.Addr{
netip.MustParseAddr("100.64.0.1"), // known, connected -> skipped
netip.MustParseAddr("fd00::1"), // known via IPv6, connected -> skipped
netip.MustParseAddr("100.64.0.99"), // unknown -> skipped
})
require.Less(t, time.Since(start), time.Second, "no pending peer must mean no wait")
}
// TestDNSPeerActivator_WaitsForPendingPeerToConnect verifies the wait loop
// returns as soon as a pending peer reports connected, well before the
// per-query budget expires.
func TestDNSPeerActivator_WaitsForPendingPeerToConnect(t *testing.T) {
a, status, store := newTestDNSPeerActivator(t)
require.NoError(t, status.AddPeer("peerA", "a.netbird.cloud", "100.64.0.1", ""))
store.AddPeerConn("peerA", newTestPeerConn(t, "peerA"))
go func() {
time.Sleep(150 * time.Millisecond)
_ = status.UpdatePeerState(peer.State{PubKey: "peerA", ConnStatus: peer.StatusConnected})
}()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
start := time.Now()
a.ActivatePeersByIP(ctx, []netip.Addr{netip.MustParseAddr("100.64.0.1")})
elapsed := time.Since(start)
require.GreaterOrEqual(t, elapsed, 100*time.Millisecond, "must wait for the pending peer")
require.Less(t, elapsed, 5*time.Second, "must return on connect, not at the deadline")
}
// TestDNSPeerActivator_ReturnsAtBudgetWhenPeerStaysIdle verifies a peer that
// never connects releases the DNS response at the per-query budget instead of
// blocking it indefinitely.
func TestDNSPeerActivator_ReturnsAtBudgetWhenPeerStaysIdle(t *testing.T) {
a, status, store := newTestDNSPeerActivator(t)
require.NoError(t, status.AddPeer("peerA", "a.netbird.cloud", "100.64.0.1", ""))
store.AddPeerConn("peerA", newTestPeerConn(t, "peerA"))
ctx, cancel := context.WithTimeout(context.Background(), 300*time.Millisecond)
defer cancel()
start := time.Now()
a.ActivatePeersByIP(ctx, []netip.Addr{netip.MustParseAddr("100.64.0.1")})
elapsed := time.Since(start)
require.GreaterOrEqual(t, elapsed, 250*time.Millisecond, "must wait out the budget for a pending peer")
require.Less(t, elapsed, 5*time.Second, "must not block past the budget")
}
// TestDNSPeerActivator_NoWaitWithoutPeerConn verifies a known-but-idle peer
// with no connection object in the store is not waited on: there is nothing to
// activate, so waiting could only ever time out.
func TestDNSPeerActivator_NoWaitWithoutPeerConn(t *testing.T) {
a, status, _ := newTestDNSPeerActivator(t)
require.NoError(t, status.AddPeer("peerA", "a.netbird.cloud", "100.64.0.1", ""))
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
start := time.Now()
a.ActivatePeersByIP(ctx, []netip.Addr{netip.MustParseAddr("100.64.0.1")})
require.Less(t, time.Since(start), time.Second, "peer without a conn must not be waited on")
}

View File

@@ -665,6 +665,16 @@ func (e *Engine) Start(netbirdConfig *mgmProto.NetbirdConfig, mgmtURL *url.URL)
e.connMgr = NewConnMgr(e.config, e.statusRecorder, e.peerStore, wgIface)
e.connMgr.Start(e.ctx)
// Wire DNS-time lazy-connection warm-up now that the connection manager
// exists (it does not at DNS-server construction time). A DNS answer that
// points at an idle peer then wakes it before the client's first request.
e.dnsServer.SetPeerActivator(&dnsPeerActivator{
connMgr: e.connMgr,
peerStore: e.peerStore,
status: e.statusRecorder,
ctx: e.ctx,
})
e.srWatcher = guard.NewSRWatcher(e.signal, e.relayManager, e.mobileDep.IFaceDiscover, iceCfg)
e.srWatcher.Start(peer.IsForceRelayed())

151
combined/cmd/admin.go Normal file
View File

@@ -0,0 +1,151 @@
package cmd
import (
"context"
"fmt"
"github.com/dexidp/dex/storage"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/netbirdio/netbird/formatter/hook"
admincmd "github.com/netbirdio/netbird/management/cmd/admin"
tokencmd "github.com/netbirdio/netbird/management/cmd/token"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/server/activity"
activitystore "github.com/netbirdio/netbird/management/server/activity/store"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/util"
)
// newAdminCommands creates the admin command tree with combined-specific resource openers.
func newAdminCommands() *cobra.Command {
return admincmd.NewCommands(admincmd.Openers{
Resources: withAdminResources,
Store: withAdminStoreOnly,
IDP: withAdminIDPOnly,
})
}
func newLegacyTokenCommand() *cobra.Command {
cmd := tokencmd.NewCommands(tokencmd.StoreOpener(withAdminStoreOnly))
cmd.Deprecated = "use 'admin token' instead"
return cmd
}
// withAdminResources loads the combined YAML config, initializes stores, and calls fn.
func withAdminResources(cmd *cobra.Command, fn func(ctx context.Context, resources admincmd.Resources) error) error {
return withAdminConfig(cmd, func(ctx context.Context, cfg *CombinedConfig) error {
mgmtConfig, err := adminManagementConfig(cfg)
if err != nil {
return err
}
managementStore, err := openAdminStore(ctx, cfg)
if err != nil {
return err
}
defer admincmd.CloseStore(ctx, managementStore)
idpStorage, idpStorageFile, err := admincmd.OpenIDPStorage(mgmtConfig)
if err != nil {
return err
}
defer admincmd.CloseIDPStorage(idpStorage)
eventStore, esErr := openAdminEventStore(ctx, cfg, mgmtConfig)
if esErr != nil {
_, _ = fmt.Fprintf(cmd.ErrOrStderr(), "Warning: audit events will not be recorded: %v\n", esErr)
}
if eventStore != nil {
defer func() {
if err := eventStore.Close(ctx); err != nil {
log.Debugf("close activity event store: %v", err)
}
}()
}
return fn(ctx, admincmd.Resources{Store: managementStore, IDPStorage: idpStorage, IDPStorageFile: idpStorageFile, EventStore: eventStore})
})
}
// withAdminStoreOnly opens only the management store for admin subcommands that do not
// need embedded IdP storage.
func withAdminStoreOnly(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error {
return withAdminConfig(cmd, func(ctx context.Context, cfg *CombinedConfig) error {
managementStore, err := openAdminStore(ctx, cfg)
if err != nil {
return err
}
defer admincmd.CloseStore(ctx, managementStore)
return fn(ctx, managementStore)
})
}
func withAdminIDPOnly(cmd *cobra.Command, fn func(ctx context.Context, idpStorage storage.Storage, storageFile string) error) error {
return withAdminConfig(cmd, func(ctx context.Context, cfg *CombinedConfig) error {
mgmtConfig, err := adminManagementConfig(cfg)
if err != nil {
return err
}
idpStorage, idpStorageFile, err := admincmd.OpenIDPStorage(mgmtConfig)
if err != nil {
return err
}
defer admincmd.CloseIDPStorage(idpStorage)
return fn(ctx, idpStorage, idpStorageFile)
})
}
func withAdminConfig(cmd *cobra.Command, fn func(ctx context.Context, cfg *CombinedConfig) error) error {
if err := util.InitLog("error", "console"); err != nil {
return fmt.Errorf("init log: %w", err)
}
ctx := context.WithValue(cmd.Context(), hook.ExecutionContextKey, hook.SystemSource) //nolint:staticcheck
cfg, err := LoadConfig(configPath)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
cfg.ApplyAdminDefaults()
applyServerStoreEnv(cfg.Server.Store)
return fn(ctx, cfg)
}
func adminManagementConfig(cfg *CombinedConfig) (*nbconfig.Config, error) {
mgmtConfig, err := cfg.ToManagementConfig()
if err != nil {
return nil, fmt.Errorf("create management config: %w", err)
}
return mgmtConfig, nil
}
func openAdminStore(ctx context.Context, cfg *CombinedConfig) (store.Store, error) {
managementStore, err := store.NewStore(ctx, types.Engine(cfg.Management.Store.Engine), cfg.Management.DataDir, nil, true)
if err != nil {
return nil, fmt.Errorf("create store: %w", err)
}
return managementStore, nil
}
func openAdminEventStore(ctx context.Context, cfg *CombinedConfig, config *nbconfig.Config) (activity.Store, error) {
if config.DataStoreEncryptionKey == "" {
return nil, fmt.Errorf("data store encryption key is not configured")
}
if err := applyActivityStoreEnv(cfg.Server.ActivityStore); err != nil {
return nil, fmt.Errorf("configure activity event store: %w", err)
}
eventStore, err := activitystore.NewSqlStore(ctx, config.Datadir, config.DataStoreEncryptionKey)
if err != nil {
return nil, fmt.Errorf("open activity event store: %w", err)
}
if eventStore == nil {
return nil, fmt.Errorf("open activity event store: returned nil store")
}
return eventStore, nil
}

View File

@@ -0,0 +1,47 @@
package cmd
import (
"context"
"os"
"testing"
"github.com/stretchr/testify/require"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
)
func TestApplyAdminDefaultsCopiesServerStoreWithoutExposedAddress(t *testing.T) {
cfg := DefaultConfig()
cfg.Server.ExposedAddress = ""
cfg.Server.DataDir = "/srv/netbird"
cfg.Server.Store = StoreConfig{
Engine: "postgres",
DSN: "postgres://user:pass@example.com/netbird",
}
cfg.ApplyAdminDefaults()
require.Equal(t, "/srv/netbird", cfg.Management.DataDir)
require.Equal(t, "postgres", cfg.Management.Store.Engine)
require.Equal(t, cfg.Server.Store.DSN, cfg.Management.Store.DSN)
}
func TestOpenAdminEventStoreMissingEncryptionKeyReturnsNilInterface(t *testing.T) {
eventStore, err := openAdminEventStore(context.Background(), &CombinedConfig{}, &nbconfig.Config{})
require.Error(t, err)
require.Contains(t, err.Error(), "encryption key")
require.Nil(t, eventStore)
}
func TestApplyServerStoreEnv(t *testing.T) {
t.Setenv("NB_STORE_ENGINE_POSTGRES_DSN", "")
t.Setenv("NB_STORE_ENGINE_MYSQL_DSN", "")
t.Setenv("NB_STORE_ENGINE_SQLITE_FILE", "")
applyServerStoreEnv(StoreConfig{Engine: "postgres", DSN: "postgres-dsn", File: "store.db"})
require.Equal(t, "postgres-dsn", os.Getenv("NB_STORE_ENGINE_POSTGRES_DSN"))
require.Equal(t, "store.db", os.Getenv("NB_STORE_ENGINE_SQLITE_FILE"))
applyServerStoreEnv(StoreConfig{Engine: "mysql", DSN: "mysql-dsn"})
require.Equal(t, "mysql-dsn", os.Getenv("NB_STORE_ENGINE_MYSQL_DSN"))
}

View File

@@ -6,8 +6,7 @@ import (
"net"
"net/netip"
"os"
"path"
"path/filepath"
filePath "path/filepath"
"strings"
"time"
@@ -303,6 +302,19 @@ func (c *CombinedConfig) ApplySimplifiedDefaults() {
c.autoConfigureClientSettings(exposedProto, exposedHost, exposedHostPort, hasExternalStuns, hasExternalRelay, hasExternalSignal)
}
// ApplyAdminDefaults applies the management settings needed by admin commands even
// when the full server config is invalid and ApplySimplifiedDefaults cannot run.
func (c *CombinedConfig) ApplyAdminDefaults() {
if c.Management.DataDir == "" || c.Management.DataDir == "/var/lib/netbird/" {
c.Management.DataDir = c.Server.DataDir
}
if c.Management.Store.Engine == "" || c.Management.Store.Engine == "sqlite" {
if c.Server.Store.Engine != "" || c.Server.Store.File != "" || c.Server.Store.DSN != "" {
c.Management.Store = c.Server.Store
}
}
}
// applyRelayDefaults configures the relay service if no external relay is configured.
func (c *CombinedConfig) applyRelayDefaults(exposedProto, exposedHostPort string, hasExternalRelay, hasExternalStuns bool) {
if hasExternalRelay {
@@ -580,11 +592,11 @@ func (c *CombinedConfig) buildEmbeddedIdPConfig(mgmt ManagementConfig) (*idp.Emb
return nil, fmt.Errorf("authStore.dsn is required when authStore.engine is postgres")
}
} else {
authStorageFile = path.Join(mgmt.DataDir, "idp.db")
authStorageFile = filePath.Join(mgmt.DataDir, "idp.db")
if c.Server.AuthStore.File != "" {
authStorageFile = c.Server.AuthStore.File
if !filepath.IsAbs(authStorageFile) {
authStorageFile = filepath.Join(mgmt.DataDir, authStorageFile)
if !filePath.IsAbs(authStorageFile) {
authStorageFile = filePath.Join(mgmt.DataDir, authStorageFile)
}
}
}
@@ -734,7 +746,7 @@ func ApplyEmbeddedIdPConfig(ctx context.Context, cfg *nbconfig.Config, mgmtPort
cfg.EmbeddedIdP.Storage.Type = "sqlite3"
}
if cfg.EmbeddedIdP.Storage.Config.File == "" && cfg.Datadir != "" {
cfg.EmbeddedIdP.Storage.Config.File = path.Join(cfg.Datadir, "idp.db")
cfg.EmbeddedIdP.Storage.Config.File = filePath.Join(cfg.Datadir, "idp.db")
}
issuer := cfg.EmbeddedIdP.Issuer

View File

@@ -65,7 +65,8 @@ func init() {
rootCmd.PersistentFlags().StringVarP(&configPath, "config", "c", "", "path to YAML configuration file (required)")
_ = rootCmd.MarkPersistentFlagRequired("config")
rootCmd.AddCommand(newTokenCommands())
rootCmd.AddCommand(newAdminCommands())
rootCmd.AddCommand(newLegacyTokenCommand())
}
func RootCmd() *cobra.Command {
@@ -123,6 +124,37 @@ func execute(cmd *cobra.Command, _ []string) error {
}
// initializeConfig loads and validates the configuration, then initializes logging.
func applyServerStoreEnv(storeConfig StoreConfig) {
if dsn := storeConfig.DSN; dsn != "" {
switch strings.ToLower(storeConfig.Engine) {
case "postgres":
os.Setenv("NB_STORE_ENGINE_POSTGRES_DSN", dsn)
case "mysql":
os.Setenv("NB_STORE_ENGINE_MYSQL_DSN", dsn)
}
}
if file := storeConfig.File; file != "" {
os.Setenv("NB_STORE_ENGINE_SQLITE_FILE", file)
}
}
func applyActivityStoreEnv(storeConfig StoreConfig) error {
if engine := storeConfig.Engine; engine != "" {
engineLower := strings.ToLower(engine)
if engineLower == "postgres" && storeConfig.DSN == "" {
return fmt.Errorf("activityStore.dsn is required when activityStore.engine is postgres")
}
os.Setenv("NB_ACTIVITY_EVENT_STORE_ENGINE", engineLower)
if dsn := storeConfig.DSN; dsn != "" {
os.Setenv("NB_ACTIVITY_EVENT_POSTGRES_DSN", dsn)
}
}
if file := storeConfig.File; file != "" {
os.Setenv("NB_ACTIVITY_EVENT_SQLITE_FILE", file)
}
return nil
}
func initializeConfig() error {
var err error
config, err = LoadConfig(configPath)
@@ -138,30 +170,10 @@ func initializeConfig() error {
return fmt.Errorf("failed to initialize log: %w", err)
}
if dsn := config.Server.Store.DSN; dsn != "" {
switch strings.ToLower(config.Server.Store.Engine) {
case "postgres":
os.Setenv("NB_STORE_ENGINE_POSTGRES_DSN", dsn)
case "mysql":
os.Setenv("NB_STORE_ENGINE_MYSQL_DSN", dsn)
}
}
if file := config.Server.Store.File; file != "" {
os.Setenv("NB_STORE_ENGINE_SQLITE_FILE", file)
}
applyServerStoreEnv(config.Server.Store)
if engine := config.Server.ActivityStore.Engine; engine != "" {
engineLower := strings.ToLower(engine)
if engineLower == "postgres" && config.Server.ActivityStore.DSN == "" {
return fmt.Errorf("activityStore.dsn is required when activityStore.engine is postgres")
}
os.Setenv("NB_ACTIVITY_EVENT_STORE_ENGINE", engineLower)
if dsn := config.Server.ActivityStore.DSN; dsn != "" {
os.Setenv("NB_ACTIVITY_EVENT_POSTGRES_DSN", dsn)
}
}
if file := config.Server.ActivityStore.File; file != "" {
os.Setenv("NB_ACTIVITY_EVENT_SQLITE_FILE", file)
if err := applyActivityStoreEnv(config.Server.ActivityStore); err != nil {
return err
}
log.Infof("Starting combined NetBird server")

View File

@@ -1,63 +0,0 @@
package cmd
import (
"context"
"fmt"
"os"
"strings"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/netbirdio/netbird/formatter/hook"
tokencmd "github.com/netbirdio/netbird/management/cmd/token"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/util"
)
// newTokenCommands creates the token command tree with combined-specific store opener.
func newTokenCommands() *cobra.Command {
return tokencmd.NewCommands(withTokenStore)
}
// withTokenStore loads the combined YAML config, initializes the store, and calls fn.
func withTokenStore(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error {
if err := util.InitLog("error", "console"); err != nil {
return fmt.Errorf("init log: %w", err)
}
ctx := context.WithValue(cmd.Context(), hook.ExecutionContextKey, hook.SystemSource) //nolint:staticcheck
cfg, err := LoadConfig(configPath)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
if dsn := cfg.Server.Store.DSN; dsn != "" {
switch strings.ToLower(cfg.Server.Store.Engine) {
case "postgres":
os.Setenv("NB_STORE_ENGINE_POSTGRES_DSN", dsn)
case "mysql":
os.Setenv("NB_STORE_ENGINE_MYSQL_DSN", dsn)
}
}
if file := cfg.Server.Store.File; file != "" {
os.Setenv("NB_STORE_ENGINE_SQLITE_FILE", file)
}
datadir := cfg.Management.DataDir
engine := types.Engine(cfg.Management.Store.Engine)
s, err := store.NewStore(ctx, engine, datadir, nil, true)
if err != nil {
return fmt.Errorf("create store: %w", err)
}
defer func() {
if err := s.Close(ctx); err != nil {
log.Debugf("close store: %v", err)
}
}()
return fn(ctx, s)
}

View File

@@ -91,14 +91,22 @@ func availableProviders() []providerCase {
if region == "" {
region = "us-east-1"
}
ps = append(ps, providerCase{name: "bedrock", catalogID: "bedrock_api", upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k, model: "us.anthropic.claude-haiku-4-5", kind: harness.WireBedrock})
// A valid Bedrock inference-profile id (region prefix + date + version),
// overridable per account. `global.` profiles can be invoked from any
// region; set AWS_BEDROCK_MODEL to match the enabled profile for the token.
model := os.Getenv("AWS_BEDROCK_MODEL")
if model == "" {
model = "global.anthropic.claude-haiku-4-5-20251001-v1:0"
}
ps = append(ps, providerCase{name: "bedrock", catalogID: "bedrock_api", upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k, model: model, kind: harness.WireBedrock})
}
return ps
}
// providerRequest builds a create request for a matrix provider: enabled, with
// a uniquely-priced model for body-routed providers and none for the
// path-routed Vertex (whose model lives in the request path).
// a uniquely-priced model registered under the id an operator would paste —
// the normalized catalog id for Bedrock, the raw form elsewhere (including the
// "@version" Vertex id, which the router must normalize to route).
func providerRequest(pc providerCase) api.AgentNetworkProviderRequest {
req := api.AgentNetworkProviderRequest{
Name: pc.name,
@@ -107,10 +115,15 @@ func providerRequest(pc providerCase) api.AgentNetworkProviderRequest {
ApiKey: &pc.apiKey,
Enabled: ptr(true),
}
if pc.kind != harness.WireVertex {
req.Models = &[]api.AgentNetworkProviderModel{
{Id: pc.model, InputPer1k: 0.001, OutputPer1k: 0.002},
}
// Register Bedrock under its normalized catalog id (the router strips the
// region prefix before matching); other providers, incl. the raw "@version"
// Vertex id, register as-is so the router's normalization is exercised.
modelID := pc.model
if pc.kind == harness.WireBedrock {
modelID = catalogModel(pc)
}
req.Models = &[]api.AgentNetworkProviderModel{
{Id: modelID, InputPer1k: 0.001, OutputPer1k: 0.002},
}
return req
}
@@ -201,11 +214,13 @@ func TestProvidersMatrix(t *testing.T) {
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
// Resolve first: the DNS lookup triggers the lazy-connection warm-up, waking
// the proxy peer so WaitProxyPeer then observes it connected.
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve agent-network endpoint to proxy IP")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve agent-network endpoint to proxy IP")
for _, pc := range matrix {
pc := pc

View File

@@ -4,6 +4,7 @@ package agentnetwork
import (
"context"
"regexp"
"strings"
"testing"
"time"
@@ -15,13 +16,29 @@ import (
"github.com/netbirdio/netbird/shared/management/http/api"
)
// bedrockRegionPrefixes and bedrockVersionSuffix mirror the proxy's Bedrock
// model normalization (region/inference-profile prefix + version suffix) so the
// provider is registered under the same catalog key the router matches against.
var (
bedrockRegionPrefixes = []string{"us.", "eu.", "apac.", "global."}
bedrockVersionSuffix = regexp.MustCompile(`-(\d{8}-)?v\d+(:\d+)?$`)
)
// catalogModel returns the normalized catalog id the proxy stamps for a
// path-routed provider's configured model — the form the guardrail allowlist is
// compared against (region prefix / @version stripped).
// path-routed provider's configured model — the form the router and guardrail
// allowlist compare against (Bedrock region prefix + version stripped, Vertex
// @version stripped).
func catalogModel(pc providerCase) string {
switch pc.kind {
case harness.WireBedrock:
return strings.TrimPrefix(pc.model, "us.")
m := pc.model
for _, p := range bedrockRegionPrefixes {
if strings.HasPrefix(m, p) {
m = m[len(p):]
break
}
}
return bedrockVersionSuffix.ReplaceAllString(m, "")
case harness.WireVertex:
return strings.SplitN(pc.model, "@", 2)[0]
default:
@@ -108,7 +125,13 @@ func TestModelAllowlistEnforced(t *testing.T) {
require.NoError(t, perr, "create provider %s", pc.name)
id := prov.Id
ids = append(ids, id)
allowed = append(allowed, catalogModel(pc))
// Vertex allowlists the raw "@version" id (the guardrail normalizes it);
// other providers allowlist the normalized catalog id.
if pc.kind == harness.WireVertex {
allowed = append(allowed, pc.model)
} else {
allowed = append(allowed, catalogModel(pc))
}
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), id) })
}
@@ -147,11 +170,13 @@ func TestModelAllowlistEnforced(t *testing.T) {
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
// Resolve first: the DNS lookup triggers the lazy-connection warm-up, waking
// the proxy peer so WaitProxyPeer then observes it connected.
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve agent-network endpoint to proxy IP")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve agent-network endpoint to proxy IP")
for _, pc := range providers {
pc := pc
@@ -163,6 +188,22 @@ func TestModelAllowlistEnforced(t *testing.T) {
// the upstream), regardless of whether it is a real catalog model.
assert.Equal(t, 403, sendModel(ctx, t, cl, settings.Endpoint, proxyIP, pc, disallowedModel(pc)),
"model outside the allowlist must be denied for %s", pc.name)
if pc.kind != harness.WireVertex {
return
}
// Unversioned model id (the customer-reported shape) must pass the
// same allowlist entry.
assert.Equal(t, 200, sendModel(ctx, t, cl, settings.Endpoint, proxyIP, pc, catalogModel(pc)),
"unversioned model id must be permitted for %s", pc.name)
// count-tokens carries the real model in the body: allowed → served.
code, _, err := cl.VertexCountTokens(ctx, settings.Endpoint, proxyIP, pc.project, pc.region, pc.model)
require.NoError(t, err, "count-tokens request must reach the proxy for %s", pc.name)
assert.Equal(t, 200, code, "count-tokens with an allowlisted body model must be permitted for %s", pc.name)
// Disallowed body model → denied before the upstream.
code, _, err = cl.VertexCountTokens(ctx, settings.Endpoint, proxyIP, pc.project, pc.region, disallowedModel(pc))
require.NoError(t, err, "count-tokens request must reach the proxy for %s", pc.name)
assert.Equal(t, 403, code, "count-tokens with a body model outside the allowlist must be denied for %s", pc.name)
})
}
}

View File

@@ -104,11 +104,13 @@ func TestProviderSkipTLSVerification(t *testing.T) {
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
// Resolve first: the DNS lookup triggers the lazy-connection warm-up, waking
// the proxy peer so WaitProxyPeer then observes it connected.
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve endpoint to proxy IP")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve endpoint to proxy IP")
// Positive: skip=true reaches the self-signed upstream. Retry to absorb
// tunnel/DNS jitter on the first call; success also proves the path works.

View File

@@ -106,11 +106,13 @@ func TestVLLMProvider(t *testing.T) {
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
// Resolve first: the DNS lookup triggers the lazy-connection warm-up, waking
// the proxy peer so WaitProxyPeer then observes it connected.
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve endpoint to proxy IP")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve endpoint to proxy IP")
before, _ := srv.ListAccessLogs(ctx)
sessionID := "e2e-session-vllm"

View File

@@ -231,6 +231,15 @@ func (cl *Client) Vertex(ctx context.Context, endpoint, proxyIP, project, region
return cl.post(ctx, endpoint, proxyIP, path, body, withSessionID(nil, sessionID))
}
// VertexCountTokens issues the Anthropic-on-Vertex token-count POST: the URL
// carries the "count-tokens" pseudo-model and the real model travels in the
// body, so the proxy must resolve the body model for the allowlist and routing.
func (cl *Client) VertexCountTokens(ctx context.Context, endpoint, proxyIP, project, region, model string) (int, string, error) {
path := fmt.Sprintf("/v1/projects/%s/locations/%s/publishers/anthropic/models/count-tokens:rawPredict", project, region)
body := fmt.Sprintf(`{"model":%q,"messages":[{"role":"user","content":"hi"}]}`, model)
return cl.post(ctx, endpoint, proxyIP, path, body, nil)
}
// Bedrock issues a native AWS Bedrock InvokeModel POST over the tunnel. The
// model id is carried in the request path (/model/{id}/invoke), so the proxy
// routes by path; the body uses the bedrock anthropic_version rather than a

View File

@@ -40,7 +40,7 @@ type Config struct {
GRPCAddr string
}
const localConnectorID = "local"
const LocalConnectorID = "local"
// Provider wraps a Dex server
type Provider struct {
@@ -494,18 +494,60 @@ func (p *Provider) Storage() storage.Storage {
return p.storage
}
// SetClientsMFAChain updates the MFAChain field on OAuth2 clients in Dex storage.
// Pass a non-empty slice (e.g. []string{"default-totp"}) to enable MFA, or nil to disable it.
func SetClientsMFAChain(ctx context.Context, st storage.Storage, clientIDs []string, mfaChain []string) error {
previousChains := make(map[string][]string, len(clientIDs))
for _, clientID := range clientIDs {
client, err := st.GetClient(ctx, clientID)
if err != nil {
return fmt.Errorf("failed to get client %s before MFA chain update: %w", clientID, err)
}
previousChains[clientID] = cloneMFAChain(client.MFAChain)
}
updatedClientIDs := make([]string, 0, len(clientIDs))
for _, clientID := range clientIDs {
if err := st.UpdateClient(ctx, clientID, func(old storage.Client) (storage.Client, error) {
old.MFAChain = cloneMFAChain(mfaChain)
return old, nil
}); err != nil {
if rollbackErr := rollbackClientsMFAChain(ctx, st, updatedClientIDs, previousChains); rollbackErr != nil {
return fmt.Errorf("failed to update MFA chain on client %s: %w (also failed to roll back previous MFA chains: %v)", clientID, err, rollbackErr)
}
return fmt.Errorf("failed to update MFA chain on client %s: %w", clientID, err)
}
updatedClientIDs = append(updatedClientIDs, clientID)
}
return nil
}
func rollbackClientsMFAChain(ctx context.Context, st storage.Storage, clientIDs []string, previousChains map[string][]string) error {
var rollbackErrs []error
for i := len(clientIDs) - 1; i >= 0; i-- {
clientID := clientIDs[i]
previousChain := cloneMFAChain(previousChains[clientID])
if err := st.UpdateClient(ctx, clientID, func(old storage.Client) (storage.Client, error) {
old.MFAChain = previousChain
return old, nil
}); err != nil {
rollbackErrs = append(rollbackErrs, fmt.Errorf("client %s: %w", clientID, err))
}
}
return errors.Join(rollbackErrs...)
}
func cloneMFAChain(chain []string) []string {
if chain == nil {
return nil
}
return append([]string(nil), chain...)
}
// SetClientsMFAChain updates the MFAChain field on the dashboard and CLI OAuth2 clients.
// Pass a non-empty slice (e.g. []string{"default-totp"}) to enable MFA, or nil to disable it.
func (p *Provider) SetClientsMFAChain(ctx context.Context, clientIDs []string, mfaChain []string) error {
for _, clientID := range clientIDs {
if err := p.storage.UpdateClient(ctx, clientID, func(old storage.Client) (storage.Client, error) {
old.MFAChain = mfaChain
return old, nil
}); err != nil {
return fmt.Errorf("failed to update MFA chain on client %s: %w", clientID, err)
}
}
return nil
return SetClientsMFAChain(ctx, p.storage, clientIDs, mfaChain)
}
// Handler returns the Dex server as an http.Handler for embedding in another server.
@@ -545,7 +587,7 @@ func (p *Provider) CreateUser(ctx context.Context, email, username, password str
// Encode the user ID in Dex's format: base64(protobuf{user_id, connector_id})
// This matches the format Dex uses in JWT tokens
encodedID := EncodeDexUserID(userID, localConnectorID)
encodedID := EncodeDexUserID(userID, LocalConnectorID)
return encodedID, nil
}
@@ -624,7 +666,7 @@ func DecodeDexUserID(encodedID string) (userID, connectorID string, err error) {
// local password connector.
func IsLocalUserID(encodedID string) bool {
_, connectorID, err := DecodeDexUserID(encodedID)
return err == nil && connectorID == localConnectorID
return err == nil && connectorID == LocalConnectorID
}
// GetUser returns a user by email

View File

@@ -3,6 +3,8 @@ package dex
import (
"context"
"encoding/json"
"errors"
"io"
"log/slog"
"net/http"
"net/http/httptest"
@@ -11,11 +13,44 @@ import (
"testing"
"github.com/dexidp/dex/storage"
"github.com/dexidp/dex/storage/memory"
sqllib "github.com/dexidp/dex/storage/sql"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type updateFailingStorage struct {
storage.Storage
failClientID string
}
func (s *updateFailingStorage) UpdateClient(ctx context.Context, id string, updater func(storage.Client) (storage.Client, error)) error {
if id == s.failClientID {
return errors.New("forced update failure")
}
return s.Storage.UpdateClient(ctx, id, updater)
}
func TestSetClientsMFAChainRollsBackUpdatedClients(t *testing.T) {
ctx := context.Background()
st := memory.New(slog.New(slog.NewTextHandler(io.Discard, nil)))
require.NoError(t, st.CreateClient(ctx, storage.Client{ID: "client-1", MFAChain: []string{"old-1"}}))
require.NoError(t, st.CreateClient(ctx, storage.Client{ID: "client-2", MFAChain: []string{"old-2"}}))
err := SetClientsMFAChain(ctx, &updateFailingStorage{Storage: st, failClientID: "client-2"}, []string{"client-1", "client-2"}, []string{"new"})
require.Error(t, err)
require.Contains(t, err.Error(), "failed to update MFA chain on client client-2")
client1, err := st.GetClient(ctx, "client-1")
require.NoError(t, err)
require.Equal(t, []string{"old-1"}, client1.MFAChain)
client2, err := st.GetClient(ctx, "client-2")
require.NoError(t, err)
require.Equal(t, []string{"old-2"}, client2.MFAChain)
}
func TestUserCreationFlow(t *testing.T) {
ctx := context.Background()

View File

@@ -556,7 +556,7 @@ start_services_and_show_instructions() {
echo "Creating proxy access token..."
# Use docker exec with bash to run the token command directly
PROXY_TOKEN=$($DOCKER_COMPOSE_COMMAND exec -T netbird-server \
/go/bin/netbird-server token create --name "default-proxy" --config /etc/netbird/config.yaml 2>/dev/null | grep "^Token:" | awk '{print $2}')
/go/bin/netbird-server admin token create --name "default-proxy" --config /etc/netbird/config.yaml 2>/dev/null | grep "^Token:" | awk '{print $2}')
if [[ -z "$PROXY_TOKEN" ]]; then
echo "ERROR: Failed to create proxy token. Check netbird-server logs." > /dev/stderr

177
management/cmd/admin.go Normal file
View File

@@ -0,0 +1,177 @@
package cmd
import (
"context"
"fmt"
"path"
"path/filepath"
"github.com/dexidp/dex/storage"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/netbirdio/netbird/formatter/hook"
admincmd "github.com/netbirdio/netbird/management/cmd/admin"
tokencmd "github.com/netbirdio/netbird/management/cmd/token"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/server/activity"
activitystore "github.com/netbirdio/netbird/management/server/activity/store"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/util"
)
var adminDatadir string
// newAdminCommands creates the admin command tree with management-specific resource openers.
func newAdminCommands() *cobra.Command {
cmd := admincmd.NewCommands(admincmd.Openers{
Resources: withAdminResources,
Store: withAdminStoreOnly,
IDP: withAdminIDPOnly,
})
cmd.PersistentFlags().StringVar(&adminDatadir, "datadir", "", "Override the data directory from config (used for store.db and the default idp.db)")
return cmd
}
func newLegacyTokenCommand() *cobra.Command {
cmd := tokencmd.NewCommands(tokencmd.StoreOpener(withAdminStoreOnly))
cmd.Deprecated = "use 'admin token' instead"
cmd.PersistentFlags().StringVar(&nbconfig.MgmtConfigPath, "config", defaultMgmtConfig, "Netbird config file location")
return cmd
}
// withAdminResources initializes logging, loads config, opens the management store
// and embedded IdP storage, and calls fn.
func withAdminResources(cmd *cobra.Command, fn func(ctx context.Context, resources admincmd.Resources) error) error {
return withAdminConfig(cmd, true, func(ctx context.Context, config *nbconfig.Config, datadir string) error {
managementStore, err := openAdminStore(ctx, config, datadir)
if err != nil {
return err
}
defer admincmd.CloseStore(ctx, managementStore)
idpStorage, idpStorageFile, err := admincmd.OpenIDPStorage(config)
if err != nil {
return err
}
defer admincmd.CloseIDPStorage(idpStorage)
eventStore, esErr := openAdminEventStore(ctx, config, datadir)
if esErr != nil {
_, _ = fmt.Fprintf(cmd.ErrOrStderr(), "Warning: audit events will not be recorded: %v\n", esErr)
}
if eventStore != nil {
defer func() {
if err := eventStore.Close(ctx); err != nil {
log.Debugf("close activity event store: %v", err)
}
}()
}
return fn(ctx, admincmd.Resources{Store: managementStore, IDPStorage: idpStorage, IDPStorageFile: idpStorageFile, EventStore: eventStore})
})
}
// withAdminStoreOnly opens only the management store for admin subcommands that do not
// need embedded IdP storage.
func withAdminStoreOnly(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error {
return withAdminConfig(cmd, false, func(ctx context.Context, config *nbconfig.Config, datadir string) error {
managementStore, err := openAdminStore(ctx, config, datadir)
if err != nil {
return err
}
defer admincmd.CloseStore(ctx, managementStore)
return fn(ctx, managementStore)
})
}
func withAdminIDPOnly(cmd *cobra.Command, fn func(ctx context.Context, idpStorage storage.Storage, storageFile string) error) error {
return withAdminConfig(cmd, true, func(ctx context.Context, config *nbconfig.Config, _ string) error {
idpStorage, idpStorageFile, err := admincmd.OpenIDPStorage(config)
if err != nil {
return err
}
defer admincmd.CloseIDPStorage(idpStorage)
return fn(ctx, idpStorage, idpStorageFile)
})
}
func withAdminConfig(cmd *cobra.Command, applyIDPDefaults bool, fn func(ctx context.Context, config *nbconfig.Config, datadir string) error) error {
if err := util.InitLog("error", "console"); err != nil {
return fmt.Errorf("init log: %w", err)
}
ctx := context.WithValue(cmd.Context(), hook.ExecutionContextKey, hook.SystemSource) //nolint:staticcheck
config, datadir, err := loadAdminMgmtConfig(ctx, applyIDPDefaults)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
return fn(ctx, config, datadir)
}
func loadAdminMgmtConfig(ctx context.Context, applyIDPDefaults bool) (*nbconfig.Config, string, error) {
config := &nbconfig.Config{}
if _, err := util.ReadJsonWithEnvSub(nbconfig.MgmtConfigPath, config); err != nil {
return nil, "", err
}
if applyIDPDefaults {
if err := ApplyEmbeddedIdPConfig(ctx, config); err != nil {
return nil, "", err
}
}
datadir := config.Datadir
applyAdminDatadirOverride(config, &datadir)
return config, datadir, nil
}
func applyAdminDatadirOverride(config *nbconfig.Config, datadir *string) {
if adminDatadir == "" {
return
}
oldDatadir := *datadir
*datadir = adminDatadir
if config.EmbeddedIdP != nil && config.EmbeddedIdP.Storage.Type == "sqlite3" && isDefaultIDPStorageFile(config.EmbeddedIdP.Storage.Config.File, oldDatadir) {
config.EmbeddedIdP.Storage.Config.File = filepath.Join(*datadir, "idp.db")
}
}
func isDefaultIDPStorageFile(file, datadir string) bool {
if file == "" {
return true
}
defaultFile := filepath.Join(datadir, "idp.db")
legacyDefaultFile := path.Join(datadir, "idp.db")
legacySlashDefaultFile := path.Join(filepath.ToSlash(datadir), "idp.db")
return filepath.Clean(file) == filepath.Clean(defaultFile) ||
file == legacyDefaultFile ||
filepath.ToSlash(file) == legacySlashDefaultFile
}
func openAdminStore(ctx context.Context, config *nbconfig.Config, datadir string) (store.Store, error) {
managementStore, err := store.NewStore(ctx, config.StoreConfig.Engine, datadir, nil, true)
if err != nil {
return nil, fmt.Errorf("create store: %w", err)
}
return managementStore, nil
}
func openAdminEventStore(ctx context.Context, config *nbconfig.Config, datadir string) (activity.Store, error) {
if config.DataStoreEncryptionKey == "" {
return nil, fmt.Errorf("data store encryption key is not configured")
}
eventStore, err := activitystore.NewSqlStore(ctx, datadir, config.DataStoreEncryptionKey)
if err != nil {
return nil, fmt.Errorf("open activity event store: %w", err)
}
if eventStore == nil {
return nil, fmt.Errorf("open activity event store: returned nil store")
}
return eventStore, nil
}

View File

@@ -0,0 +1,577 @@
// Package admincmd provides reusable cobra commands for self-hosted administrator helpers.
// Both the management and combined binaries use these commands, each providing
// their own opener to handle config loading and storage initialization.
package admincmd
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"os"
"strings"
"time"
"github.com/dexidp/dex/storage"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"golang.org/x/crypto/bcrypt"
"github.com/netbirdio/netbird/formatter/hook"
nbdex "github.com/netbirdio/netbird/idp/dex"
"github.com/netbirdio/netbird/management/cmd/proxy"
"github.com/netbirdio/netbird/management/cmd/token"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/server"
"github.com/netbirdio/netbird/management/server/activity"
"github.com/netbirdio/netbird/management/server/idp"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
)
// Resources contains the storages required by the admin commands.
type Resources struct {
Store store.Store
IDPStorage storage.Storage
IDPStorageFile string
EventStore activity.Store
}
// Opener initializes command resources from the command context and calls fn.
type Opener func(cmd *cobra.Command, fn func(ctx context.Context, resources Resources) error) error
// StoreOpener initializes only the management store from the command context and calls fn.
type StoreOpener func(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error
// IDPOpener initializes only the embedded IdP storage from the command context and calls fn.
type IDPOpener func(cmd *cobra.Command, fn func(ctx context.Context, idpStorage storage.Storage, storageFile string) error) error
// Openers contains the resource openers needed by the admin command tree.
type Openers struct {
Resources Opener
Store StoreOpener
IDP IDPOpener
}
type userSelector struct {
email string
userID string
}
func (s userSelector) normalized() userSelector {
return userSelector{
email: strings.TrimSpace(s.email),
userID: strings.TrimSpace(s.userID),
}
}
func (s userSelector) validate() error {
s = s.normalized()
if (s.email == "") == (s.userID == "") {
return fmt.Errorf("provide exactly one of --email or --user-id")
}
return nil
}
// NewCommands creates the admin command tree with the given resource openers.
func NewCommands(openers Openers) *cobra.Command {
adminCmd := &cobra.Command{
Use: "admin",
Short: "Self-hosted administrator helpers",
Long: "Administrative helpers for self-hosted deployments using the embedded identity provider.",
}
userCmd := &cobra.Command{
Use: "user",
Short: "Manage local embedded IdP users",
}
var passwordSelector userSelector
var password string
var passwordFile string
passwordCmd := &cobra.Command{
Use: "change-password (--email email | --user-id id) (--password password | --password-file path)",
Aliases: []string{"set-password"},
Short: "Change a local user's password",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
if err := passwordSelector.validate(); err != nil {
return err
}
newPassword, err := resolvePasswordInput(cmd, password, passwordFile)
if err != nil {
return err
}
return openers.IDP(cmd, func(ctx context.Context, idpStorage storage.Storage, storageFile string) error {
return runChangePassword(ctx, idpStorage, cmd.OutOrStdout(), passwordSelector, newPassword, storageFile)
})
},
}
addUserSelectorFlags(passwordCmd, &passwordSelector)
passwordCmd.Flags().StringVar(&password, "password", "", "New password for the user")
passwordCmd.Flags().StringVar(&passwordFile, "password-file", "", "Read new password from file ('-' for stdin)")
var resetSelector userSelector
resetMFACmd := &cobra.Command{
Use: "reset-mfa (--email email | --user-id id)",
Short: "Reset a local user's MFA enrollment",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
if err := resetSelector.validate(); err != nil {
return err
}
return openers.IDP(cmd, func(ctx context.Context, idpStorage storage.Storage, storageFile string) error {
return runResetMFA(ctx, idpStorage, cmd.OutOrStdout(), resetSelector, storageFile)
})
},
}
addUserSelectorFlags(resetMFACmd, &resetSelector)
userCmd.AddCommand(passwordCmd, resetMFACmd)
mfaCmd := &cobra.Command{
Use: "mfa",
Short: "Manage local MFA for embedded IdP users",
}
enableCmd := &cobra.Command{
Use: "enable",
Short: "Enable MFA for local embedded IdP users",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return openers.Resources(cmd, func(ctx context.Context, resources Resources) error {
return runSetMFAEnabled(ctx, resources, cmd.OutOrStdout(), true)
})
},
}
disableCmd := &cobra.Command{
Use: "disable",
Short: "Disable MFA for local embedded IdP users",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return openers.Resources(cmd, func(ctx context.Context, resources Resources) error {
return runSetMFAEnabled(ctx, resources, cmd.OutOrStdout(), false)
})
},
}
statusCmd := &cobra.Command{
Use: "status",
Short: "Show local MFA status",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return openers.Resources(cmd, func(ctx context.Context, resources Resources) error {
return runMFAStatus(ctx, resources, cmd.OutOrStdout())
})
},
}
mfaCmd.AddCommand(enableCmd, disableCmd, statusCmd)
adminCmd.AddCommand(userCmd, mfaCmd)
if openers.Store != nil {
adminCmd.AddCommand(tokencmd.NewCommands(tokencmd.StoreOpener(openers.Store)))
adminCmd.AddCommand(proxycmd.NewCommands(proxycmd.StoreOpener(openers.Store)))
}
return adminCmd
}
// OpenEmbeddedIDPStorage opens the Dex storage configured for the embedded IdP.
func OpenEmbeddedIDPStorage(cfg *idp.EmbeddedIdPConfig) (storage.Storage, error) {
if cfg == nil || !cfg.Enabled {
return nil, fmt.Errorf("admin commands require the embedded IdP to be enabled")
}
yamlConfig, err := cfg.ToYAMLConfig()
if err != nil {
return nil, fmt.Errorf("build embedded IdP config: %w", err)
}
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
st, err := yamlConfig.Storage.OpenStorage(logger)
if err != nil {
return nil, fmt.Errorf("open embedded IdP storage: %w", err)
}
return st, nil
}
// CloseStore closes the management store and logs cleanup errors at debug level.
func CloseStore(ctx context.Context, s store.Store) {
if s == nil {
return
}
if err := s.Close(ctx); err != nil {
log.Debugf("close store: %v", err)
}
}
// OpenIDPStorage opens embedded IdP storage and returns its sqlite file path when applicable.
func OpenIDPStorage(config *nbconfig.Config) (storage.Storage, string, error) {
if config == nil {
return nil, "", fmt.Errorf("management config is required")
}
idpStorage, err := OpenEmbeddedIDPStorage(config.EmbeddedIdP)
if err != nil {
return nil, "", err
}
return idpStorage, embeddedIDPStorageFile(config), nil
}
func embeddedIDPStorageFile(config *nbconfig.Config) string {
if config.EmbeddedIdP == nil || config.EmbeddedIdP.Storage.Type != "sqlite3" {
return ""
}
return config.EmbeddedIdP.Storage.Config.File
}
// CloseIDPStorage closes embedded IdP storage and logs cleanup errors at debug level.
func CloseIDPStorage(s storage.Storage) {
if s == nil {
return
}
if err := s.Close(); err != nil {
log.Debugf("close embedded IdP storage: %v", err)
}
}
func addUserSelectorFlags(cmd *cobra.Command, selector *userSelector) {
cmd.Flags().StringVar(&selector.email, "email", "", "User email")
cmd.Flags().StringVar(&selector.userID, "user-id", "", "User ID")
}
func resolvePasswordInput(cmd *cobra.Command, password, passwordFile string) (string, error) {
if password != "" && passwordFile != "" {
return "", fmt.Errorf("provide only one of --password or --password-file")
}
if passwordFile == "" {
return password, nil
}
var data []byte
var err error
if passwordFile == "-" {
data, err = io.ReadAll(cmd.InOrStdin())
} else {
data, err = os.ReadFile(passwordFile)
}
if err != nil {
return "", fmt.Errorf("read password: %w", err)
}
return strings.TrimRight(string(data), "\r\n"), nil
}
func runChangePassword(ctx context.Context, idpStorage storage.Storage, w io.Writer, selector userSelector, password string, idpStorageFile string) error {
if idpStorage == nil {
return fmt.Errorf("embedded IdP storage is required")
}
selector = selector.normalized()
if err := selector.validate(); err != nil {
return err
}
if password == "" {
return fmt.Errorf("password is required")
}
if err := server.ValidatePassword(password); err != nil {
return fmt.Errorf("invalid password: %w", err)
}
user, err := findLocalUser(ctx, idpStorage, selector, idpStorageFile)
if err != nil {
return err
}
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return fmt.Errorf("hash password: %w", err)
}
if err := idpStorage.UpdatePassword(ctx, user.Email, func(old storage.Password) (storage.Password, error) {
old.Hash = hash
return old, nil
}); err != nil {
return fmt.Errorf("update password for %s: %w", user.Email, err)
}
if err := deleteLocalAuthSession(ctx, idpStorage, user.UserID); err != nil {
return err
}
_, _ = fmt.Fprintf(w, "Password updated for %s.\n", user.Email)
return nil
}
func runResetMFA(ctx context.Context, idpStorage storage.Storage, w io.Writer, selector userSelector, idpStorageFile string) error {
if idpStorage == nil {
return fmt.Errorf("embedded IdP storage is required")
}
selector = selector.normalized()
if err := selector.validate(); err != nil {
return err
}
user, err := findLocalUser(ctx, idpStorage, selector, idpStorageFile)
if err != nil {
return err
}
reset := false
err = idpStorage.UpdateUserIdentity(ctx, user.UserID, idp.LocalConnectorID, func(old storage.UserIdentity) (storage.UserIdentity, error) {
reset = reset || len(old.MFASecrets) > 0 || len(old.WebAuthnCredentials) > 0
old.MFASecrets = map[string]*storage.MFASecret{}
old.WebAuthnCredentials = map[string][]storage.WebAuthnCredential{}
return old, nil
})
if errors.Is(err, storage.ErrNotFound) {
if err := deleteLocalAuthSession(ctx, idpStorage, user.UserID); err != nil {
return err
}
_, _ = fmt.Fprintf(w, "No MFA enrollment found for %s.\n", user.Email)
return nil
}
if err != nil {
return fmt.Errorf("reset MFA for %s: %w", user.Email, err)
}
if err := deleteLocalAuthSession(ctx, idpStorage, user.UserID); err != nil {
return err
}
if reset {
_, _ = fmt.Fprintf(w, "MFA reset for %s. The user will re-enroll at next login.\n", user.Email)
} else {
_, _ = fmt.Fprintf(w, "No MFA enrollment found for %s.\n", user.Email)
}
return nil
}
func runSetMFAEnabled(ctx context.Context, resources Resources, w io.Writer, enabled bool) error {
if resources.Store == nil {
return fmt.Errorf("management store is required")
}
if resources.IDPStorage == nil {
return fmt.Errorf("embedded IdP storage is required")
}
accountID, settings, err := getSingleAccountSettings(ctx, resources.Store)
if err != nil {
return err
}
oldEnabled := settings.LocalMfaEnabled
newSettings := settings.Copy()
newSettings.LocalMfaEnabled = enabled
if err := setIDPClientsMFA(ctx, resources.IDPStorage, enabled); err != nil {
return err
}
if err := resources.Store.SaveAccountSettings(ctx, accountID, newSettings); err != nil {
if rollbackErr := setIDPClientsMFA(ctx, resources.IDPStorage, oldEnabled); rollbackErr != nil {
return fmt.Errorf("save local MFA account setting: %w (also failed to roll back embedded IdP MFA state: %v)", err, rollbackErr)
}
return fmt.Errorf("save local MFA account setting: %w", err)
}
if err := storeMFAActivity(ctx, resources.EventStore, accountID, enabled); err != nil {
_, _ = fmt.Fprintf(w, "Warning: failed to record audit event: %v\n", err)
}
state := "disabled"
if enabled {
state = "enabled"
}
_, _ = fmt.Fprintf(w, "Local MFA %s.\n", state)
return nil
}
func runMFAStatus(ctx context.Context, resources Resources, w io.Writer) error {
if resources.Store == nil {
return fmt.Errorf("management store is required")
}
if resources.IDPStorage == nil {
return fmt.Errorf("embedded IdP storage is required")
}
_, settings, err := getSingleAccountSettings(ctx, resources.Store)
if err != nil {
return err
}
accountStatus := "disabled"
if settings.LocalMfaEnabled {
accountStatus = "enabled"
}
clientStatus, err := idpClientsMFAStatus(ctx, resources.IDPStorage)
if err != nil {
return err
}
_, _ = fmt.Fprintf(w, "Account setting: %s\n", accountStatus)
_, _ = fmt.Fprintf(w, "Embedded IdP clients: %s\n", clientStatus)
return nil
}
func getSingleAccountSettings(ctx context.Context, s store.Store) (string, *types.Settings, error) {
count, err := s.GetAccountsCounter(ctx)
if err != nil {
return "", nil, fmt.Errorf("count accounts: %w", err)
}
if count != 1 {
return "", nil, fmt.Errorf("expected exactly one account, got %d; local MFA is supported only in single-account embedded IdP deployments", count)
}
accountID, err := s.GetAnyAccountID(ctx)
if err != nil {
return "", nil, fmt.Errorf("get account ID: %w", err)
}
settings, err := s.GetAccountSettings(ctx, store.LockingStrengthNone, accountID)
if err != nil {
return "", nil, fmt.Errorf("get account settings: %w", err)
}
if settings == nil {
settings = &types.Settings{}
}
return accountID, settings, nil
}
func storeMFAActivity(ctx context.Context, eventStore activity.Store, accountID string, enabled bool) error {
if eventStore == nil {
return nil
}
event := activity.AccountLocalMfaDisabled
if enabled {
event = activity.AccountLocalMfaEnabled
}
_, err := eventStore.Save(ctx, &activity.Event{
Timestamp: time.Now().UTC(),
Activity: event,
InitiatorID: string(hook.SystemSource),
TargetID: accountID,
AccountID: accountID,
})
if err != nil {
return fmt.Errorf("save local MFA audit event: %w", err)
}
return nil
}
func findLocalUser(ctx context.Context, idpStorage storage.Storage, selector userSelector, idpStorageFile string) (storage.Password, error) {
selector = selector.normalized()
if err := selector.validate(); err != nil {
return storage.Password{}, err
}
if selector.email != "" {
user, err := idpStorage.GetPassword(ctx, selector.email)
if errors.Is(err, storage.ErrNotFound) {
if empty, listErr := localUsersEmpty(ctx, idpStorage); listErr != nil {
return storage.Password{}, listErr
} else if empty {
return storage.Password{}, noLocalUsersError(idpStorageFile)
}
return storage.Password{}, fmt.Errorf("local user with email %q not found", selector.email)
}
if err != nil {
return storage.Password{}, fmt.Errorf("get local user by email %q: %w", selector.email, err)
}
return user, nil
}
rawUserID := selector.userID
if decodedUserID, _, err := nbdex.DecodeDexUserID(selector.userID); err == nil && decodedUserID != "" {
rawUserID = decodedUserID
}
users, err := idpStorage.ListPasswords(ctx)
if err != nil {
return storage.Password{}, fmt.Errorf("list local users: %w", err)
}
for _, user := range users {
if user.UserID == rawUserID || user.UserID == selector.userID {
return user, nil
}
}
if len(users) == 0 {
return storage.Password{}, noLocalUsersError(idpStorageFile)
}
return storage.Password{}, fmt.Errorf("local user with ID %q not found", selector.userID)
}
func localUsersEmpty(ctx context.Context, idpStorage storage.Storage) (bool, error) {
users, err := idpStorage.ListPasswords(ctx)
if err != nil {
return false, fmt.Errorf("list local users: %w", err)
}
return len(users) == 0, nil
}
func noLocalUsersError(idpStorageFile string) error {
location := ""
if idpStorageFile != "" {
location = fmt.Sprintf(" (%s)", idpStorageFile)
}
return fmt.Errorf("no local users exist in the embedded IdP storage%s; the management server may never have started with this config, or --datadir points at the wrong location", location)
}
func deleteLocalAuthSession(ctx context.Context, idpStorage storage.Storage, userID string) error {
err := idpStorage.DeleteAuthSession(ctx, userID, idp.LocalConnectorID)
if err == nil || errors.Is(err, storage.ErrNotFound) {
return nil
}
return fmt.Errorf("delete local auth session for user %s: %w", userID, err)
}
func setIDPClientsMFA(ctx context.Context, idpStorage storage.Storage, enabled bool) error {
var mfaChain []string
if enabled {
mfaChain = []string{idp.DefaultTOTPAuthenticatorID}
}
clientIDs := []string{idp.StaticClientCLI, idp.StaticClientDashboard}
if err := nbdex.SetClientsMFAChain(ctx, idpStorage, clientIDs, mfaChain); err != nil {
if errors.Is(err, storage.ErrNotFound) {
return fmt.Errorf("embedded IdP client not found; start the management server once before toggling MFA: %w", err)
}
return fmt.Errorf("update MFA chain on embedded IdP clients: %w", err)
}
return nil
}
func idpClientsMFAStatus(ctx context.Context, idpStorage storage.Storage) (string, error) {
clientIDs := []string{idp.StaticClientCLI, idp.StaticClientDashboard}
enabledCount := 0
for _, clientID := range clientIDs {
client, err := idpStorage.GetClient(ctx, clientID)
if errors.Is(err, storage.ErrNotFound) {
return "unknown", fmt.Errorf("embedded IdP client %q not found", clientID)
}
if err != nil {
return "unknown", fmt.Errorf("get embedded IdP client %q: %w", clientID, err)
}
if hasAuthenticator(client.MFAChain, idp.DefaultTOTPAuthenticatorID) {
enabledCount++
}
}
switch enabledCount {
case 0:
return "disabled", nil
case len(clientIDs):
return "enabled", nil
default:
return "partially enabled", nil
}
}
func hasAuthenticator(chain []string, authenticatorID string) bool {
for _, id := range chain {
if id == authenticatorID {
return true
}
}
return false
}

View File

@@ -0,0 +1,250 @@
package admincmd
import (
"bytes"
"context"
"io"
"log/slog"
"strings"
"testing"
"time"
"github.com/dexidp/dex/storage"
"github.com/dexidp/dex/storage/memory"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"golang.org/x/crypto/bcrypt"
nbdex "github.com/netbirdio/netbird/idp/dex"
"github.com/netbirdio/netbird/management/server/idp"
mgmtstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
)
func newTestIDPStorage(t *testing.T) storage.Storage {
t.Helper()
st := memory.New(slog.New(slog.NewTextHandler(io.Discard, nil)))
hash, err := bcrypt.GenerateFromPassword([]byte("OldPass1!"), bcrypt.DefaultCost)
require.NoError(t, err)
require.NoError(t, st.CreatePassword(context.Background(), storage.Password{
Email: "user@example.com",
Username: "User",
UserID: "user-1",
Hash: hash,
}))
require.NoError(t, st.CreateUserIdentity(context.Background(), storage.UserIdentity{
UserID: "user-1",
ConnectorID: idp.LocalConnectorID,
MFASecrets: map[string]*storage.MFASecret{
idp.DefaultTOTPAuthenticatorID: {
AuthenticatorID: idp.DefaultTOTPAuthenticatorID,
Type: "TOTP",
Secret: "otpauth://totp/NetBird:user@example.com?secret=ABC",
Confirmed: true,
CreatedAt: time.Now(),
},
},
WebAuthnCredentials: map[string][]storage.WebAuthnCredential{
"webauthn": {{CredentialID: []byte("credential")}},
},
}))
require.NoError(t, st.CreateAuthSession(context.Background(), storage.AuthSession{
UserID: "user-1",
ConnectorID: idp.LocalConnectorID,
Nonce: "nonce",
}))
require.NoError(t, st.CreateClient(context.Background(), storage.Client{ID: idp.StaticClientCLI, Name: "CLI"}))
require.NoError(t, st.CreateClient(context.Background(), storage.Client{ID: idp.StaticClientDashboard, Name: "Dashboard"}))
return st
}
func TestRunChangePassword(t *testing.T) {
ctx := context.Background()
st := newTestIDPStorage(t)
var out bytes.Buffer
err := runChangePassword(ctx, st, &out, userSelector{email: "user@example.com"}, "NewPass1!", "")
require.NoError(t, err)
require.Contains(t, out.String(), "Password updated")
user, err := st.GetPassword(ctx, "user@example.com")
require.NoError(t, err)
require.NoError(t, bcrypt.CompareHashAndPassword(user.Hash, []byte("NewPass1!")))
_, err = st.GetAuthSession(ctx, "user-1", idp.LocalConnectorID)
require.ErrorIs(t, err, storage.ErrNotFound)
}
func TestRunChangePasswordValidatesPassword(t *testing.T) {
st := newTestIDPStorage(t)
err := runChangePassword(context.Background(), st, io.Discard, userSelector{email: "user@example.com"}, "short", "")
require.Error(t, err)
require.Contains(t, err.Error(), "invalid password")
}
func TestRunResetMFA(t *testing.T) {
ctx := context.Background()
st := newTestIDPStorage(t)
var out bytes.Buffer
encodedUserID := nbdex.EncodeDexUserID("user-1", idp.LocalConnectorID)
err := runResetMFA(ctx, st, &out, userSelector{userID: encodedUserID}, "")
require.NoError(t, err)
require.Contains(t, out.String(), "MFA reset")
identity, err := st.GetUserIdentity(ctx, "user-1", idp.LocalConnectorID)
require.NoError(t, err)
require.Empty(t, identity.MFASecrets)
require.Empty(t, identity.WebAuthnCredentials)
_, err = st.GetAuthSession(ctx, "user-1", idp.LocalConnectorID)
require.ErrorIs(t, err, storage.ErrNotFound)
}
func TestRunResetMFAWithoutEnrollment(t *testing.T) {
ctx := context.Background()
st := newTestIDPStorage(t)
require.NoError(t, st.UpdateUserIdentity(ctx, "user-1", idp.LocalConnectorID, func(old storage.UserIdentity) (storage.UserIdentity, error) {
old.MFASecrets = nil
old.WebAuthnCredentials = nil
return old, nil
}))
var out bytes.Buffer
err := runResetMFA(ctx, st, &out, userSelector{email: "user@example.com"}, "")
require.NoError(t, err)
require.Contains(t, out.String(), "No MFA enrollment found")
}
func TestSetIDPClientsMFA(t *testing.T) {
ctx := context.Background()
st := newTestIDPStorage(t)
require.NoError(t, setIDPClientsMFA(ctx, st, true))
status, err := idpClientsMFAStatus(ctx, st)
require.NoError(t, err)
require.Equal(t, "enabled", status)
require.NoError(t, setIDPClientsMFA(ctx, st, false))
status, err = idpClientsMFAStatus(ctx, st)
require.NoError(t, err)
require.Equal(t, "disabled", status)
}
func newTestManagementStore(t *testing.T, localMFAEnabled bool) mgmtstore.Store {
t.Helper()
ctx := context.Background()
st, err := mgmtstore.NewStore(ctx, types.SqliteStoreEngine, t.TempDir(), nil, false)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, st.Close(ctx)) })
require.NoError(t, st.SaveAccount(ctx, &types.Account{
Id: "account-1",
Settings: &types.Settings{LocalMfaEnabled: localMFAEnabled},
}))
return st
}
func TestRunSetMFAEnabledDoesNotSaveWhenIDPUpdateFails(t *testing.T) {
ctx := context.Background()
managementStore := newTestManagementStore(t, false)
idpStorage := memory.New(slog.New(slog.NewTextHandler(io.Discard, nil)))
err := runSetMFAEnabled(ctx, Resources{Store: managementStore, IDPStorage: idpStorage}, io.Discard, true)
require.Error(t, err)
require.Contains(t, err.Error(), "embedded IdP client")
settings, err := managementStore.GetAccountSettings(ctx, mgmtstore.LockingStrengthNone, "account-1")
require.NoError(t, err)
require.False(t, settings.LocalMfaEnabled)
}
func TestRunSetMFAEnabledUpdatesSettingsAfterIDP(t *testing.T) {
ctx := context.Background()
managementStore := newTestManagementStore(t, false)
idpStorage := newTestIDPStorage(t)
err := runSetMFAEnabled(ctx, Resources{Store: managementStore, IDPStorage: idpStorage}, io.Discard, true)
require.NoError(t, err)
settings, err := managementStore.GetAccountSettings(ctx, mgmtstore.LockingStrengthNone, "account-1")
require.NoError(t, err)
require.True(t, settings.LocalMfaEnabled)
clientStatus, err := idpClientsMFAStatus(ctx, idpStorage)
require.NoError(t, err)
require.Equal(t, "enabled", clientStatus)
}
func TestRunSetMFAEnabledSucceedsWithNilEventStore(t *testing.T) {
ctx := context.Background()
managementStore := newTestManagementStore(t, false)
idpStorage := newTestIDPStorage(t)
var out bytes.Buffer
var err error
require.NotPanics(t, func() {
err = runSetMFAEnabled(ctx, Resources{Store: managementStore, IDPStorage: idpStorage, EventStore: nil}, &out, true)
})
require.NoError(t, err)
require.Contains(t, out.String(), "Local MFA enabled")
settings, err := managementStore.GetAccountSettings(ctx, mgmtstore.LockingStrengthNone, "account-1")
require.NoError(t, err)
require.True(t, settings.LocalMfaEnabled)
}
func TestUserSelectorValidate(t *testing.T) {
require.NoError(t, userSelector{email: " user@example.com "}.validate())
require.NoError(t, userSelector{userID: "user-1"}.validate())
require.Error(t, userSelector{}.validate())
require.Error(t, userSelector{email: "user@example.com", userID: "user-1"}.validate())
}
func TestFindLocalUserNotFound(t *testing.T) {
st := newTestIDPStorage(t)
_, err := findLocalUser(context.Background(), st, userSelector{email: "missing@example.com"}, "")
require.Error(t, err)
require.True(t, strings.Contains(err.Error(), "not found"))
}
func TestFindLocalUserZeroUsersIncludesStoragePath(t *testing.T) {
st := memory.New(slog.New(slog.NewTextHandler(io.Discard, nil)))
_, err := findLocalUser(context.Background(), st, userSelector{email: "missing@example.com"}, "/var/lib/netbird/idp.db")
require.Error(t, err)
require.Contains(t, err.Error(), "no local users exist")
require.Contains(t, err.Error(), "/var/lib/netbird/idp.db")
}
func TestUserCommandValidatesSelectorBeforeOpeningStorage(t *testing.T) {
opened := false
cmd := NewCommands(Openers{
IDP: func(cmd *cobra.Command, fn func(ctx context.Context, idpStorage storage.Storage, storageFile string) error) error {
opened = true
return nil
},
})
cmd.SetArgs([]string{"user", "change-password", "--password", "NewPass1!"})
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
err := cmd.Execute()
require.Error(t, err)
require.Contains(t, err.Error(), "provide exactly one")
require.False(t, opened)
}
func TestResolvePasswordInputFromStdin(t *testing.T) {
cmd := &cobra.Command{}
cmd.SetIn(strings.NewReader("NewPass1!\n"))
password, err := resolvePasswordInput(cmd, "", "-")
require.NoError(t, err)
require.Equal(t, "NewPass1!", password)
}
func TestResolvePasswordInputRejectsMultipleSources(t *testing.T) {
_, err := resolvePasswordInput(&cobra.Command{}, "NewPass1!", "-")
require.Error(t, err)
}

View File

@@ -0,0 +1,80 @@
package cmd
import (
"context"
"path"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/server/idp"
)
func TestApplyAdminDatadirOverrideRelocatesDefaultIDPStorage(t *testing.T) {
oldDatadir := filepath.Join(t.TempDir(), "old")
newDatadir := filepath.Join(t.TempDir(), "new")
for _, defaultFile := range []string{
"",
filepath.Join(oldDatadir, "idp.db"),
path.Join(oldDatadir, "idp.db"),
} {
t.Run(defaultFile, func(t *testing.T) {
cfg := &nbconfig.Config{
EmbeddedIdP: &idp.EmbeddedIdPConfig{
Enabled: true,
Storage: idp.EmbeddedStorageConfig{
Type: "sqlite3",
Config: idp.EmbeddedStorageTypeConfig{
File: defaultFile,
},
},
},
}
datadir := oldDatadir
oldAdminDatadir := adminDatadir
adminDatadir = newDatadir
t.Cleanup(func() { adminDatadir = oldAdminDatadir })
applyAdminDatadirOverride(cfg, &datadir)
require.Equal(t, newDatadir, datadir)
require.Equal(t, filepath.Join(newDatadir, "idp.db"), cfg.EmbeddedIdP.Storage.Config.File)
})
}
}
func TestOpenAdminEventStoreMissingEncryptionKeyReturnsNilInterface(t *testing.T) {
eventStore, err := openAdminEventStore(context.Background(), &nbconfig.Config{}, t.TempDir())
require.Error(t, err)
require.Contains(t, err.Error(), "encryption key")
require.Nil(t, eventStore)
}
func TestApplyAdminDatadirOverrideKeepsExplicitIDPStorage(t *testing.T) {
oldDatadir := filepath.Join(t.TempDir(), "old")
newDatadir := filepath.Join(t.TempDir(), "new")
explicitFile := filepath.Join(t.TempDir(), "custom-idp.db")
cfg := &nbconfig.Config{
EmbeddedIdP: &idp.EmbeddedIdPConfig{
Enabled: true,
Storage: idp.EmbeddedStorageConfig{
Type: "sqlite3",
Config: idp.EmbeddedStorageTypeConfig{
File: explicitFile,
},
},
},
}
datadir := oldDatadir
oldAdminDatadir := adminDatadir
adminDatadir = newDatadir
t.Cleanup(func() { adminDatadir = oldAdminDatadir })
applyAdminDatadirOverride(cfg, &datadir)
require.Equal(t, newDatadir, datadir)
require.Equal(t, explicitFile, cfg.EmbeddedIdP.Storage.Config.File)
}

View File

@@ -13,6 +13,7 @@ import (
"os"
"os/signal"
"path"
"path/filepath"
"strings"
"syscall"
@@ -222,7 +223,7 @@ func ApplyEmbeddedIdPConfig(ctx context.Context, cfg *nbconfig.Config) error {
cfg.EmbeddedIdP.Storage.Type = "sqlite3"
}
if cfg.EmbeddedIdP.Storage.Config.File == "" && cfg.Datadir != "" {
cfg.EmbeddedIdP.Storage.Config.File = path.Join(cfg.Datadir, "idp.db")
cfg.EmbeddedIdP.Storage.Config.File = filepath.Join(cfg.Datadir, "idp.db")
}
issuer := cfg.EmbeddedIdP.Issuer

View File

@@ -0,0 +1,141 @@
// Package proxycmd provides reusable cobra commands for managing reverse proxy instances.
// Both the management and combined binaries use these commands, each providing
// their own StoreOpener to handle config loading and store initialization.
package proxycmd
import (
"bufio"
"context"
"fmt"
"io"
"strings"
"text/tabwriter"
"github.com/spf13/cobra"
rpproxy "github.com/netbirdio/netbird/management/internals/modules/reverseproxy/proxy"
"github.com/netbirdio/netbird/management/server/store"
)
// StoreOpener initializes a store from the command context and calls fn.
type StoreOpener func(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error
const disconnectAllConfirmation = "disconnect all proxies"
// NewCommands creates the proxy command tree with the given store opener.
// Returns the parent "proxy" command with the disconnect-all subcommand.
func NewCommands(opener StoreOpener) *cobra.Command {
var dryRun bool
var force bool
proxyCmd := &cobra.Command{
Use: "proxy",
Short: "Manage reverse proxy instances",
Long: "Commands for inspecting and repairing the reverse proxy instances registered with the management server.",
}
disconnectAllCmd := &cobra.Command{
Use: "disconnect-all",
Short: "Force-mark all reverse proxy instances as disconnected",
Long: "Lists all reverse proxy instances and force-marks them as disconnected, regardless of their session state. " +
"Use this to repair stale connection state, e.g. after an unclean management server shutdown. " +
"By default, it asks for manual confirmation before changing state. Use --dry-run to preview without changing state, or --force to skip confirmation. " +
"Run during a maintenance window; affected live proxies may stay hidden until their next heartbeat or reconnect/re-register.",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return opener(cmd, func(ctx context.Context, s store.Store) error {
return runDisconnectAll(ctx, s, cmd.OutOrStdout(), cmd.InOrStdin(), dryRun, force)
})
},
}
disconnectAllCmd.Flags().BoolVar(&dryRun, "dry-run", false, "List reverse proxy instances that would be disconnected without changing state")
disconnectAllCmd.Flags().BoolVar(&force, "force", false, "Skip the confirmation prompt and apply the repair")
proxyCmd.AddCommand(disconnectAllCmd)
return proxyCmd
}
func runDisconnectAll(ctx context.Context, s store.Store, out io.Writer, in io.Reader, dryRun, force bool) error {
proxies, err := s.GetAllProxies(ctx)
if err != nil {
return fmt.Errorf("list proxies: %w", err)
}
if len(proxies) == 0 {
_, _ = fmt.Fprintln(out, "No reverse proxy instances found.")
return nil
}
toDisconnect := 0
w := tabwriter.NewWriter(out, 0, 0, 2, ' ', 0)
_, _ = fmt.Fprintln(w, "ID\tCLUSTER\tIP\tACCOUNT\tSTATUS\tLAST SEEN")
_, _ = fmt.Fprintln(w, "--\t-------\t--\t-------\t------\t---------")
for _, p := range proxies {
if p.Status != rpproxy.StatusDisconnected {
toDisconnect++
}
account := "-"
if p.AccountID != nil {
account = *p.AccountID
}
_, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n",
p.ID,
p.ClusterAddress,
p.IPAddress,
account,
p.Status,
p.LastSeen.Format("2006-01-02 15:04:05"),
)
}
if err := w.Flush(); err != nil {
return fmt.Errorf("write proxy list: %w", err)
}
if dryRun {
_, _ = fmt.Fprintf(out, "\nDry run: would force-mark %d of %d reverse proxy instance(s) as disconnected.\n", toDisconnect, len(proxies))
return nil
}
if !force {
confirmed, err := confirmDisconnectAll(out, in)
if err != nil {
return err
}
if !confirmed {
_, _ = fmt.Fprintln(out, "Aborted. No reverse proxy instances were changed.")
return nil
}
}
disconnected, err := s.DisconnectAllProxies(ctx)
if err != nil {
return fmt.Errorf("disconnect proxies: %w", err)
}
_, _ = fmt.Fprintf(out, "\nForce-marked %d of %d reverse proxy instance(s) as disconnected.\n", disconnected, len(proxies))
return nil
}
func confirmDisconnectAll(out io.Writer, in io.Reader) (bool, error) {
if in == nil {
in = strings.NewReader("")
}
_, _ = fmt.Fprintln(out, "\nWARNING: This command changes stored reverse proxy state for every non-disconnected instance.")
_, _ = fmt.Fprintln(out, "Run it during a maintenance window; affected live proxies may stay hidden until "+
"their next heartbeat or reconnect/re-register.")
_, _ = fmt.Fprintf(out, "Type %q to continue: ", disconnectAllConfirmation)
scanner := bufio.NewScanner(in)
if !scanner.Scan() {
if err := scanner.Err(); err != nil {
return false, fmt.Errorf("read confirmation: %w", err)
}
return false, nil
}
return strings.EqualFold(strings.TrimSpace(scanner.Text()), disconnectAllConfirmation), nil
}

View File

@@ -0,0 +1,180 @@
package proxycmd
import (
"bytes"
"context"
"strings"
"testing"
"time"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
rpproxy "github.com/netbirdio/netbird/management/internals/modules/reverseproxy/proxy"
"github.com/netbirdio/netbird/management/server/store"
)
func newTestStore(t *testing.T) store.Store {
t.Helper()
s, cleanup, err := store.NewTestStoreFromSQL(context.Background(), "", t.TempDir())
require.NoError(t, err)
t.Cleanup(cleanup)
return s
}
func seedProxies(t *testing.T, ctx context.Context, s store.Store) {
t.Helper()
accountID := "account-1"
alreadyDisconnectedAt := time.Now().Add(-time.Hour)
seed := []*rpproxy.Proxy{
{
ID: "proxy-1",
SessionID: "session-1",
ClusterAddress: "cluster-a.example.com",
IPAddress: "10.0.0.1",
LastSeen: time.Now(),
Status: rpproxy.StatusConnected,
},
{
ID: "proxy-2",
SessionID: "session-2",
ClusterAddress: "cluster-b.example.com",
IPAddress: "10.0.0.2",
AccountID: &accountID,
LastSeen: time.Now(),
Status: rpproxy.StatusConnected,
},
{
ID: "proxy-3",
SessionID: "session-3",
ClusterAddress: "cluster-a.example.com",
IPAddress: "10.0.0.3",
LastSeen: time.Now().Add(-time.Hour),
Status: rpproxy.StatusDisconnected,
DisconnectedAt: &alreadyDisconnectedAt,
},
}
for _, p := range seed {
require.NoError(t, s.SaveProxy(ctx, p))
}
}
func proxiesByID(t *testing.T, ctx context.Context, s store.Store) map[string]*rpproxy.Proxy {
t.Helper()
proxies, err := s.GetAllProxies(ctx)
require.NoError(t, err)
require.Len(t, proxies, 3)
byID := make(map[string]*rpproxy.Proxy, len(proxies))
for _, p := range proxies {
byID[p.ID] = p
}
return byID
}
func TestRunDisconnectAllWithConfirmation(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
seedProxies(t, ctx, s)
var out bytes.Buffer
require.NoError(t, runDisconnectAll(ctx, s, &out, strings.NewReader(disconnectAllConfirmation+"\n"), false, false))
output := out.String()
require.Contains(t, output, "proxy-1")
require.Contains(t, output, "proxy-2")
require.Contains(t, output, "proxy-3")
require.Contains(t, output, "cluster-a.example.com")
require.Contains(t, output, "account-1")
require.Contains(t, output, "Type \"disconnect all proxies\" to continue")
require.Contains(t, output, "Force-marked 2 of 3 reverse proxy instance(s) as disconnected.")
for _, p := range proxiesByID(t, ctx, s) {
require.Equal(t, rpproxy.StatusDisconnected, p.Status, "proxy %s should be disconnected", p.ID)
require.NotNil(t, p.DisconnectedAt, "proxy %s should have a disconnected timestamp", p.ID)
}
}
func TestRunDisconnectAllForceSkipsConfirmation(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
seedProxies(t, ctx, s)
var out bytes.Buffer
require.NoError(t, runDisconnectAll(ctx, s, &out, strings.NewReader(""), false, true))
output := out.String()
require.NotContains(t, output, "Type \"disconnect all proxies\" to continue")
require.Contains(t, output, "Force-marked 2 of 3 reverse proxy instance(s) as disconnected.")
}
func TestRunDisconnectAllAbortLeavesProxiesUnchanged(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
seedProxies(t, ctx, s)
var out bytes.Buffer
require.NoError(t, runDisconnectAll(ctx, s, &out, strings.NewReader("no\n"), false, false))
output := out.String()
require.Contains(t, output, "Type \"disconnect all proxies\" to continue")
require.Contains(t, output, "Aborted. No reverse proxy instances were changed.")
byID := proxiesByID(t, ctx, s)
require.Equal(t, rpproxy.StatusConnected, byID["proxy-1"].Status)
require.Equal(t, rpproxy.StatusConnected, byID["proxy-2"].Status)
require.Equal(t, rpproxy.StatusDisconnected, byID["proxy-3"].Status)
}
func TestRunDisconnectAllDryRunLeavesProxiesUnchanged(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
seedProxies(t, ctx, s)
var out bytes.Buffer
require.NoError(t, runDisconnectAll(ctx, s, &out, strings.NewReader(""), true, false))
output := out.String()
require.Contains(t, output, "Dry run: would force-mark 2 of 3 reverse proxy instance(s) as disconnected.")
require.NotContains(t, output, "Type \"disconnect all proxies\" to continue")
byID := proxiesByID(t, ctx, s)
require.Equal(t, rpproxy.StatusConnected, byID["proxy-1"].Status)
require.Equal(t, rpproxy.StatusConnected, byID["proxy-2"].Status)
require.Equal(t, rpproxy.StatusDisconnected, byID["proxy-3"].Status)
}
func TestNewCommandsDisconnectAllDryRun(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
seedProxies(t, ctx, s)
opened := false
cmd := NewCommands(func(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error {
opened = true
return fn(cmd.Context(), s)
})
var out bytes.Buffer
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetIn(strings.NewReader(""))
cmd.SetArgs([]string{"disconnect-all", "--dry-run"})
require.NoError(t, cmd.ExecuteContext(ctx))
require.True(t, opened)
require.Contains(t, out.String(), "Dry run: would force-mark 2 of 3 reverse proxy instance(s) as disconnected.")
}
func TestRunDisconnectAllEmpty(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
var out bytes.Buffer
require.NoError(t, runDisconnectAll(ctx, s, &out, strings.NewReader(""), false, false))
require.Contains(t, out.String(), "No reverse proxy instances found.")
}

View File

@@ -83,7 +83,8 @@ func init() {
rootCmd.AddCommand(migrationCmd)
tc := newTokenCommands()
tc.PersistentFlags().StringVar(&nbconfig.MgmtConfigPath, "config", defaultMgmtConfig, "Netbird config file location")
rootCmd.AddCommand(tc)
ac := newAdminCommands()
ac.PersistentFlags().StringVar(&nbconfig.MgmtConfigPath, "config", defaultMgmtConfig, "Netbird config file location")
rootCmd.AddCommand(ac)
rootCmd.AddCommand(newLegacyTokenCommand())
}

View File

@@ -1,55 +0,0 @@
package cmd
import (
"context"
"fmt"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/netbirdio/netbird/formatter/hook"
tokencmd "github.com/netbirdio/netbird/management/cmd/token"
nbconfig "github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/util"
)
var tokenDatadir string
// newTokenCommands creates the token command tree with management-specific store opener.
func newTokenCommands() *cobra.Command {
cmd := tokencmd.NewCommands(withTokenStore)
cmd.PersistentFlags().StringVar(&tokenDatadir, "datadir", "", "Override the data directory from config (where store.db is located)")
return cmd
}
// withTokenStore initializes logging, loads config, opens the store, and calls fn.
func withTokenStore(cmd *cobra.Command, fn func(ctx context.Context, s store.Store) error) error {
if err := util.InitLog("error", "console"); err != nil {
return fmt.Errorf("init log: %w", err)
}
ctx := context.WithValue(cmd.Context(), hook.ExecutionContextKey, hook.SystemSource) //nolint:staticcheck
config, err := LoadMgmtConfig(ctx, nbconfig.MgmtConfigPath)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
datadir := config.Datadir
if tokenDatadir != "" {
datadir = tokenDatadir
}
s, err := store.NewStore(ctx, config.StoreConfig.Engine, datadir, nil, true)
if err != nil {
return fmt.Errorf("create store: %w", err)
}
defer func() {
if err := s.Close(ctx); err != nil {
log.Debugf("close store: %v", err)
}
}()
return fn(ctx, s)
}

View File

@@ -608,11 +608,11 @@ func (s *ProxyServiceServer) disconnectProxy(conn *proxyConnection) {
if err := s.proxyController.UnregisterProxyFromCluster(context.Background(), conn.address, conn.proxyID); err != nil {
log.Warnf("Failed to unregister proxy %s from cluster: %v", conn.proxyID, err)
}
conn.cancel()
if err := s.proxyManager.Disconnect(context.Background(), conn.proxyID, conn.sessionID); err != nil {
log.Warnf("Failed to mark proxy %s as disconnected: %v", conn.proxyID, err)
}
conn.cancel()
log.Infof("Proxy %s session %s disconnected", conn.proxyID, conn.sessionID)
}

View File

@@ -21,8 +21,11 @@ import (
)
const (
staticClientDashboard = "netbird-dashboard"
staticClientCLI = "netbird-cli"
StaticClientDashboard = "netbird-dashboard"
StaticClientCLI = "netbird-cli"
DefaultTOTPAuthenticatorID = "default-totp"
LocalConnectorID = dex.LocalConnectorID
defaultCLIRedirectURL1 = "http://localhost:53000/"
defaultCLIRedirectURL2 = "http://localhost:54000/"
defaultScopes = "openid profile email groups"
@@ -189,14 +192,14 @@ func (c *EmbeddedIdPConfig) ToYAMLConfig() (*dex.YAMLConfig, error) {
EnablePasswordDB: true,
StaticClients: []storage.Client{
{
ID: staticClientDashboard,
ID: StaticClientDashboard,
Name: "NetBird Dashboard",
Public: true,
RedirectURIs: redirectURIs,
PostLogoutRedirectURIs: sanitizePostLogoutRedirectURIs(dashboardPostLogoutRedirectURIs),
},
{
ID: staticClientCLI,
ID: StaticClientCLI,
Name: "NetBird CLI",
Public: true,
RedirectURIs: redirectURIs,
@@ -258,13 +261,13 @@ func sanitizePostLogoutRedirectURIs(uris []string) []string {
func configureMFA(cfg *dex.YAMLConfig, sessionMaxLifetime, sessionIdleTimeout string, rememberMe bool, sessionCookieEncryptionKey string) error {
cfg.MFA.Authenticators = []dex.MFAAuthenticator{{
ID: "default-totp",
ID: DefaultTOTPAuthenticatorID,
// Has to be caps otherwise it will fail
Type: "TOTP",
Config: map[string]interface{}{
"issuer": "NetBird",
},
ConnectorTypes: []string{"local"},
ConnectorTypes: []string{LocalConnectorID},
}}
if sessionMaxLifetime == "" {
@@ -740,7 +743,7 @@ func (m *EmbeddedIdPManager) GetDefaultScopes() string {
// GetCLIClientID returns the client ID for CLI authentication.
func (m *EmbeddedIdPManager) GetCLIClientID() string {
return staticClientCLI
return StaticClientCLI
}
// GetCLIRedirectURLs returns the redirect URLs configured for the CLI client.
@@ -779,7 +782,7 @@ func (m *EmbeddedIdPManager) GetLocalKeysLocation() string {
// GetClientIDs returns the OAuth2 client IDs configured for this provider.
func (m *EmbeddedIdPManager) GetClientIDs() []string {
return []string{staticClientDashboard, staticClientCLI}
return []string{StaticClientDashboard, StaticClientCLI}
}
// GetUserIDClaim returns the JWT claim name used for user identification.
@@ -796,11 +799,11 @@ func (m *EmbeddedIdPManager) IsLocalAuthDisabled() bool {
func (m *EmbeddedIdPManager) SetMFAEnabled(ctx context.Context, enabled bool) error {
var mfaChain []string
if enabled {
mfaChain = []string{"default-totp"}
mfaChain = []string{DefaultTOTPAuthenticatorID}
}
if err := m.provider.SetClientsMFAChain(ctx, []string{
staticClientCLI,
staticClientDashboard,
StaticClientCLI,
StaticClientDashboard,
}, mfaChain); err != nil {
return fmt.Errorf("failed to set MFA enabled=%v: %w", enabled, err)
}

View File

@@ -331,7 +331,7 @@ func TestEmbeddedIdPConfig_ToYAMLConfig_IncludesDeviceCallbackRedirectURI(t *tes
var cliRedirectURIs []string
for _, client := range yamlConfig.StaticClients {
if client.ID == staticClientCLI {
if client.ID == StaticClientCLI {
cliRedirectURIs = client.RedirectURIs
break
}

View File

@@ -6140,6 +6140,37 @@ func (s *SqlStore) DisconnectProxy(ctx context.Context, proxyID, sessionID strin
return nil
}
// GetAllProxies returns all reverse proxy instance rows.
func (s *SqlStore) GetAllProxies(ctx context.Context) ([]*proxy.Proxy, error) {
var proxies []*proxy.Proxy
result := s.db.Order("cluster_address, id").Find(&proxies)
if result.Error != nil {
log.WithContext(ctx).Errorf("failed to get proxies: %v", result.Error)
return nil, status.Errorf(status.Internal, "failed to get proxies")
}
return proxies, nil
}
// DisconnectAllProxies force-marks every proxy that is not already disconnected
// as disconnected, regardless of session ID. Unlike DisconnectProxy it is not
// session-guarded: it is an administrative repair helper, not part of the
// connection lifecycle. last_seen is left untouched so the stale-proxy reaper
// keeps working off the real last heartbeat. Returns the number of proxies updated.
func (s *SqlStore) DisconnectAllProxies(ctx context.Context) (int64, error) {
result := s.db.
Model(&proxy.Proxy{}).
Where("status != ?", proxy.StatusDisconnected).
Updates(map[string]any{
"status": proxy.StatusDisconnected,
"disconnected_at": time.Now(),
})
if result.Error != nil {
log.WithContext(ctx).Errorf("failed to disconnect all proxies: %v", result.Error)
return 0, status.Errorf(status.Internal, "failed to disconnect all proxies")
}
return result.RowsAffected, nil
}
// UpdateProxyHeartbeat updates the last_seen timestamp for the proxy's current session.
func (s *SqlStore) UpdateProxyHeartbeat(ctx context.Context, p *proxy.Proxy) error {
now := time.Now()
@@ -6147,7 +6178,11 @@ func (s *SqlStore) UpdateProxyHeartbeat(ctx context.Context, p *proxy.Proxy) err
result := s.db.
Model(&proxy.Proxy{}).
Where("id = ? AND session_id = ?", p.ID, p.SessionID).
Update("last_seen", now)
Updates(map[string]any{
"last_seen": now,
"status": proxy.StatusConnected,
"disconnected_at": nil,
})
if result.Error != nil {
log.WithContext(ctx).Errorf("failed to update proxy heartbeat: %v", result.Error)

View File

@@ -0,0 +1,156 @@
package store
import (
"context"
"os"
"runtime"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
rpproxy "github.com/netbirdio/netbird/management/internals/modules/reverseproxy/proxy"
)
// TestSqlStore_DisconnectAllProxies guards the administrative
// force-disconnect helper:
//
// 1. Every proxy that is not already disconnected is marked
// disconnected regardless of its session ID (unlike
// DisconnectProxy, which is session-guarded).
// 2. Rows that are already disconnected are left untouched, so their
// original disconnected_at is preserved and the returned count
// reflects only the rows that actually changed.
// 3. last_seen is not modified — the stale-proxy reaper keeps working
// off the real last heartbeat.
func TestSqlStore_DisconnectAllProxies(t *testing.T) {
if (os.Getenv("CI") == "true" && runtime.GOOS == "darwin") || runtime.GOOS == "windows" {
t.Skip("skip CI tests on darwin and windows")
}
runTestForAllEngines(t, "", func(t *testing.T, store Store) {
ctx := context.Background()
lastSeenFresh := time.Now().Add(-30 * time.Second)
lastSeenStale := time.Now().Add(-30 * time.Minute)
oldDisconnectedAt := time.Now().Add(-time.Hour)
accountID := "acct-disconnect"
proxies := []*rpproxy.Proxy{
{
ID: "p-connected-fresh",
SessionID: "sess-1",
ClusterAddress: "cluster-a.example.com",
IPAddress: "10.0.0.1",
LastSeen: lastSeenFresh,
Status: rpproxy.StatusConnected,
},
{
ID: "p-connected-stale",
SessionID: "sess-2",
ClusterAddress: "cluster-b.example.com",
IPAddress: "10.0.0.2",
AccountID: &accountID,
LastSeen: lastSeenStale,
Status: rpproxy.StatusConnected,
},
{
ID: "p-already-disconnected",
SessionID: "sess-3",
ClusterAddress: "cluster-a.example.com",
IPAddress: "10.0.0.3",
LastSeen: lastSeenStale,
Status: rpproxy.StatusDisconnected,
DisconnectedAt: &oldDisconnectedAt,
},
}
for _, p := range proxies {
require.NoError(t, store.SaveProxy(ctx, p))
}
all, err := store.GetAllProxies(ctx)
require.NoError(t, err)
require.Len(t, all, 3)
disconnected, err := store.DisconnectAllProxies(ctx)
require.NoError(t, err)
assert.Equal(t, int64(2), disconnected)
all, err = store.GetAllProxies(ctx)
require.NoError(t, err)
require.Len(t, all, 3)
byID := make(map[string]*rpproxy.Proxy, len(all))
for _, p := range all {
byID[p.ID] = p
}
for id, p := range byID {
assert.Equal(t, rpproxy.StatusDisconnected, p.Status, "proxy %s should be disconnected", id)
require.NotNil(t, p.DisconnectedAt, "proxy %s should have disconnected_at set", id)
}
// force-marked rows carry a fresh disconnected_at; the untouched row keeps its original one
assert.WithinDuration(t, time.Now(), *byID["p-connected-fresh"].DisconnectedAt, 10*time.Second)
assert.WithinDuration(t, time.Now(), *byID["p-connected-stale"].DisconnectedAt, 10*time.Second)
assert.WithinDuration(t, oldDisconnectedAt, *byID["p-already-disconnected"].DisconnectedAt, time.Second)
// last_seen is preserved so the stale reaper schedule is unaffected
assert.WithinDuration(t, lastSeenFresh, byID["p-connected-fresh"].LastSeen, time.Second)
assert.WithinDuration(t, lastSeenStale, byID["p-connected-stale"].LastSeen, time.Second)
// idempotent: a second run has nothing left to update
disconnected, err = store.DisconnectAllProxies(ctx)
require.NoError(t, err)
assert.Equal(t, int64(0), disconnected)
})
}
func TestSqlStore_UpdateProxyHeartbeatRestoresDisconnectedCurrentSession(t *testing.T) {
if (os.Getenv("CI") == "true" && runtime.GOOS == "darwin") || runtime.GOOS == "windows" {
t.Skip("skip CI tests on darwin and windows")
}
runTestForAllEngines(t, "", func(t *testing.T, store Store) {
ctx := context.Background()
proxy := &rpproxy.Proxy{
ID: "p-heartbeat",
SessionID: "sess-heartbeat",
ClusterAddress: "cluster-heartbeat.example.com",
IPAddress: "10.0.0.10",
LastSeen: time.Now().Add(-30 * time.Second),
Status: rpproxy.StatusConnected,
}
require.NoError(t, store.SaveProxy(ctx, proxy))
disconnected, err := store.DisconnectAllProxies(ctx)
require.NoError(t, err)
require.Equal(t, int64(1), disconnected)
require.NoError(t, store.UpdateProxyHeartbeat(ctx, &rpproxy.Proxy{ID: proxy.ID, SessionID: proxy.SessionID}))
all, err := store.GetAllProxies(ctx)
require.NoError(t, err)
require.Len(t, all, 1)
assert.Equal(t, rpproxy.StatusConnected, all[0].Status)
assert.Nil(t, all[0].DisconnectedAt)
assert.WithinDuration(t, time.Now(), all[0].LastSeen, 10*time.Second)
addresses, err := store.GetActiveProxyClusterAddresses(ctx)
require.NoError(t, err)
assert.Contains(t, addresses, proxy.ClusterAddress)
})
}
func TestSqlStore_GetAllProxies_Empty(t *testing.T) {
if (os.Getenv("CI") == "true" && runtime.GOOS == "darwin") || runtime.GOOS == "windows" {
t.Skip("skip CI tests on darwin and windows")
}
runTestForAllEngines(t, "", func(t *testing.T, store Store) {
all, err := store.GetAllProxies(context.Background())
require.NoError(t, err)
assert.Empty(t, all)
})
}

View File

@@ -323,6 +323,8 @@ type Store interface {
GetClusterSupportsCrowdSec(ctx context.Context, clusterAddr string) *bool
GetClusterSupportsPrivate(ctx context.Context, clusterAddr string) *bool
CleanupStaleProxies(ctx context.Context, inactivityDuration time.Duration) error
GetAllProxies(ctx context.Context) ([]*proxy.Proxy, error)
DisconnectAllProxies(ctx context.Context) (int64, error)
GetProxyByAccountID(ctx context.Context, accountID string) (*proxy.Proxy, error)
CountProxiesByAccountID(ctx context.Context, accountID string) (int64, error)
IsClusterAddressConflicting(ctx context.Context, clusterAddress, accountID string) (bool, error)

View File

@@ -845,6 +845,21 @@ func (mr *MockStoreMockRecorder) DeleteZoneDNSRecords(ctx, accountID, zoneID int
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DeleteZoneDNSRecords", reflect.TypeOf((*MockStore)(nil).DeleteZoneDNSRecords), ctx, accountID, zoneID)
}
// DisconnectAllProxies mocks base method.
func (m *MockStore) DisconnectAllProxies(ctx context.Context) (int64, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "DisconnectAllProxies", ctx)
ret0, _ := ret[0].(int64)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// DisconnectAllProxies indicates an expected call of DisconnectAllProxies.
func (mr *MockStoreMockRecorder) DisconnectAllProxies(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DisconnectAllProxies", reflect.TypeOf((*MockStore)(nil).DisconnectAllProxies), ctx)
}
// DisconnectProxy mocks base method.
func (m *MockStore) DisconnectProxy(ctx context.Context, proxyID, sessionID string) error {
m.ctrl.T.Helper()
@@ -1761,6 +1776,21 @@ func (mr *MockStoreMockRecorder) GetAllEphemeralPeers(ctx, lockStrength interfac
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetAllEphemeralPeers", reflect.TypeOf((*MockStore)(nil).GetAllEphemeralPeers), ctx, lockStrength)
}
// GetAllProxies mocks base method.
func (m *MockStore) GetAllProxies(ctx context.Context) ([]*proxy.Proxy, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetAllProxies", ctx)
ret0, _ := ret[0].([]*proxy.Proxy)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetAllProxies indicates an expected call of GetAllProxies.
func (mr *MockStoreMockRecorder) GetAllProxies(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetAllProxies", reflect.TypeOf((*MockStore)(nil).GetAllProxies), ctx)
}
// GetAllProxyAccessTokens mocks base method.
func (m *MockStore) GetAllProxyAccessTokens(ctx context.Context, lockStrength LockingStrength) ([]*types3.ProxyAccessToken, error) {
m.ctrl.T.Helper()

View File

@@ -1849,12 +1849,17 @@ func (am *DefaultAccountManager) DeleteUserInvite(ctx context.Context, accountID
const minPasswordLength = 8
// validatePassword checks password strength requirements:
// validatePassword checks password strength requirements.
func validatePassword(password string) error {
return ValidatePassword(password)
}
// ValidatePassword checks password strength requirements:
// - Minimum 8 characters
// - At least 1 digit
// - At least 1 uppercase letter
// - At least 1 special character
func validatePassword(password string) error {
func ValidatePassword(password string) error {
if len(password) < minPasswordLength {
return errors.New("password must be at least 8 characters long")
}

View File

@@ -0,0 +1,14 @@
package llm
import "strings"
// NormalizeVertexModel strips the "@version" suffix from a Vertex AI model id
// so it matches the catalog key, e.g. "claude-sonnet-4-5@20250929" ->
// "claude-sonnet-4-5". Shared by the parser, router, and guardrail so both
// spellings compare equal.
func NormalizeVertexModel(modelID string) string {
if at := strings.Index(modelID, "@"); at >= 0 {
return modelID[:at]
}
return modelID
}

View File

@@ -0,0 +1,20 @@
package llm
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestNormalizeVertexModel(t *testing.T) {
cases := map[string]string{
"claude-sonnet-4-5@20250929": "claude-sonnet-4-5",
"claude-opus-4-6@20250514": "claude-opus-4-6",
"claude-opus-4-6": "claude-opus-4-6", // bare id passes through
"text-embedding-005@001": "text-embedding-005",
"@cf/meta/llama-3-8b": "", // leading "@" -> empty (treated as no model)
}
for in, want := range cases {
require.Equal(t, want, NormalizeVertexModel(in), "normalize %q", in)
}
}

View File

@@ -13,6 +13,7 @@ import (
"context"
"unicode/utf8"
"github.com/netbirdio/netbird/proxy/internal/llm"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
@@ -162,6 +163,11 @@ func (m *Middleware) modelInAllowlist(model string) bool {
if allowed == normalised {
return true
}
// Accept a Vertex entry stored with its "@version" suffix against the
// suffix-stripped request model. Entries without "@" stay exact.
if v := llm.NormalizeVertexModel(allowed); v != "" && v != allowed && v == normalised {
return true
}
}
return false
}

View File

@@ -70,6 +70,25 @@ func TestAllowlistMatchAllows(t *testing.T) {
assert.Equal(t, middleware.DecisionAllow, out.Decision, "model in allowlist must be allowed")
}
// A Vertex "@version" allowlist entry must match the version-stripped request
// model the parser emits.
func TestAllowlistVertexVersionedEntryMatchesStrippedModel(t *testing.T) {
mw := New(Config{ModelAllowlist: []string{"claude-opus-4-6@20250514"}})
out, err := mw.Invoke(context.Background(), newInput(
middleware.KV{Key: middleware.KeyLLMModel, Value: "claude-opus-4-6"},
))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"@version allowlist entry must match the version-stripped request model")
out, err = mw.Invoke(context.Background(), newInput(
middleware.KV{Key: middleware.KeyLLMModel, Value: "claude-opus-4-8"},
))
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"a different model must stay denied")
}
func TestAllowlistMissDenies(t *testing.T) {
mw := New(Config{ModelAllowlist: []string{"gpt-4o"}})
out, err := mw.Invoke(context.Background(), newInput(

View File

@@ -104,3 +104,93 @@ func TestModelAllowlist_URLRoutedProviders(t *testing.T) {
})
}
}
// TestModelAllowlist_VertexRequestShapes replays the Vertex request shapes an
// Anthropic SDK client sends (model in the URL path, optionally unversioned,
// plus the count-tokens body-model endpoint) against bare and "@version"
// allowlists. URLs mirror a customer-reported request.
func TestModelAllowlist_VertexRequestShapes(t *testing.T) {
const (
opusBare = "/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/claude-opus-4-6:rawPredict"
opusBareSSE = "/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/claude-opus-4-6:streamRawPredict"
countTokens = "/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/count-tokens:rawPredict"
messagesBody = `{"anthropic_version":"vertex-2023-10-16","messages":[{"role":"user","content":"hi"}]}`
countOpusBody = `{"model":"claude-opus-4-6","messages":[{"role":"user","content":"hi"}]}`
)
tests := []struct {
name string
url string
body string
allowlist []string
decision middleware.Decision
denyCode string
}{
{
name: "unversioned model allowed by bare catalog entry",
url: opusBare,
body: messagesBody,
allowlist: []string{"claude-opus-4-6"},
decision: middleware.DecisionAllow,
},
{
name: "unversioned model allowed by @version allowlist entry",
url: opusBare,
body: messagesBody,
allowlist: []string{"claude-opus-4-6@20250514"},
decision: middleware.DecisionAllow,
},
{
name: "streaming action allowed the same as rawPredict",
url: opusBareSSE,
body: messagesBody,
allowlist: []string{"claude-opus-4-6"},
decision: middleware.DecisionAllow,
},
{
// The customer report: a Sonnet-only allowlist must block Opus.
name: "unversioned model outside the allowlist denied",
url: opusBare,
body: messagesBody,
allowlist: []string{"claude-sonnet-4-5"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_blocked",
},
{
name: "count-tokens resolves the body model and passes when allowed",
url: countTokens,
body: countOpusBody,
allowlist: []string{"claude-opus-4-6"},
decision: middleware.DecisionAllow,
},
{
name: "count-tokens with a disallowed body model denied",
url: countTokens,
body: countOpusBody,
allowlist: []string{"claude-sonnet-4-5"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_blocked",
},
{
// No body model: the pseudo-model stays and fails closed.
name: "count-tokens without a body model fails closed",
url: countTokens,
body: messagesBody,
allowlist: []string{"claude-opus-4-6"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_blocked",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
out := runParserGuardrail(t, tt.url, []byte(tt.body), tt.allowlist)
assert.Equal(t, tt.decision, out.Decision, "unexpected decision for %s", tt.name)
if tt.decision == middleware.DecisionDeny {
require.NotNil(t, out.DenyReason, "deny reason must be set for %s", tt.name)
assert.Equal(t, 403, out.DenyStatus, "deny status must be 403 for %s", tt.name)
assert.Equal(t, tt.denyCode, out.DenyReason.Code, "deny code for %s", tt.name)
}
})
}
}

View File

@@ -253,9 +253,7 @@ func parseVertexPath(reqPath string) (vertexRequest, bool) {
if c := strings.LastIndex(rest, ":"); c >= 0 {
model, action = rest[:c], rest[c+1:]
}
if at := strings.Index(model, "@"); at >= 0 {
model = model[:at]
}
model = llm.NormalizeVertexModel(model)
if model == "" {
return vertexRequest{}, false
}
@@ -276,24 +274,40 @@ func vertexPublisherVendor(publisher string) string {
}
}
// vertexCountTokensModel is the pseudo-model of the Vertex token-count endpoint,
// the one Vertex shape whose real model lives in the body, not the URL path.
const vertexCountTokensModel = "count-tokens"
// invokeVertex emits the model/vendor/session/prompt for a Vertex publisher
// request, using the publisher's parser to read the (vendor-native) body.
func (m middlewareImpl) invokeVertex(in *middleware.Input, vx vertexRequest) *middleware.Output {
out := &middleware.Output{Decision: middleware.DecisionAllow}
vendor := vertexPublisherVendor(vx.publisher)
md := []middleware.KV{}
if vendor != "" {
md = append(md, middleware.KV{Key: middleware.KeyLLMProvider, Value: vendor})
}
md = append(md, middleware.KV{Key: middleware.KeyLLMModel, Value: vx.model})
md = append(md, middleware.KV{Key: middleware.KeyLLMStream, Value: strconv.FormatBool(vx.stream)})
var parser llm.Parser
if vendor != "" {
parser, _ = llm.ParserByName(vendor)
}
model := vx.model
// count-tokens carries its real model in the body; resolve it so the
// guardrail and router evaluate the actual model. An unreadable body keeps
// the pseudo-model and fails closed downstream.
if model == vertexCountTokensModel && parser != nil {
if facts, err := parser.ParseRequest(in.Body); err == nil && facts.Model != "" {
if bodyModel := llm.NormalizeVertexModel(facts.Model); bodyModel != "" {
model = bodyModel
}
}
}
md := []middleware.KV{}
if vendor != "" {
md = append(md, middleware.KV{Key: middleware.KeyLLMProvider, Value: vendor})
}
md = append(md, middleware.KV{Key: middleware.KeyLLMModel, Value: model})
md = append(md, middleware.KV{Key: middleware.KeyLLMStream, Value: strconv.FormatBool(vx.stream)})
sessionID := sessionIDFromHeaders(in.Headers)
if sessionID == "" && parser != nil {
sessionID = parser.ExtractSessionID(in.Body)

View File

@@ -556,14 +556,16 @@ func routeClaimsModel(route ProviderRoute, model string) bool {
if candidate == model {
return true
}
// Bedrock request models reach the router already normalized (the parser
// strips the region / inference-profile prefix and version suffix), but
// the operator may register the raw inference-profile id (e.g.
// "us.anthropic.claude-haiku-4-5"). Normalize the candidate so both sides
// compare equal; otherwise a native Bedrock request denies as not-routable.
// The request model is already normalized by the parser, but the operator
// may register the raw id (Bedrock inference-profile, Vertex "@version").
// Normalize the candidate so both spellings match; else it denies as
// not-routable.
if route.Bedrock && llm.NormalizeBedrockModel(candidate) == model {
return true
}
if route.Vertex && llm.NormalizeVertexModel(candidate) == model {
return true
}
}
return false
}

View File

@@ -0,0 +1,56 @@
package llm_router
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
// A Vertex route registered with the raw "@version" id must match the
// version-stripped request model; non-Vertex routes stay exact.
func TestRouteClaimsModel_VertexNormalizesCandidate(t *testing.T) {
route := ProviderRoute{Vertex: true, Models: []string{"claude-opus-4-6@20250514"}}
assert.True(t, routeClaimsModel(route, "claude-opus-4-6"))
assert.False(t, routeClaimsModel(route, "claude-haiku-4-5"))
bare := ProviderRoute{Vertex: true, Models: []string{"claude-opus-4-6"}}
assert.True(t, routeClaimsModel(bare, "claude-opus-4-6"))
direct := ProviderRoute{Models: []string{"claude-opus-4-6@20250514"}}
assert.False(t, routeClaimsModel(direct, "claude-opus-4-6"),
"non-Vertex routes must not normalize @version candidates")
}
// TestRouter_VertexUnversionedModelRoutes replays the customer-reported request:
// an unversioned model id must route on a provider registered with "@version" ids.
func TestRouter_VertexUnversionedModelRoutes(t *testing.T) {
route := vertexRoute()
route.Models = []string{"claude-opus-4-6@20250514", "claude-sonnet-4-5@20250929"}
mw := New(Config{Providers: []ProviderRoute{route}})
in := pathRoutedInput(
"/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/claude-opus-4-6:rawPredict",
"anthropic",
"claude-opus-4-6",
)
out, err := mw.Invoke(context.Background(), in)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision,
"unversioned request model must route on a provider registered with @version ids")
denied := pathRoutedInput(
"/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/claude-haiku-4-5:rawPredict",
"anthropic",
"claude-haiku-4-5",
)
out, err = mw.Invoke(context.Background(), denied)
require.NoError(t, err)
assert.Equal(t, middleware.DecisionDeny, out.Decision,
"a model outside the registered list must still deny")
require.NotNil(t, out.DenyReason)
assert.Equal(t, denyCodeNotRoutable, out.DenyReason.Code)
}

View File

@@ -0,0 +1,272 @@
package proxy_test
import (
"context"
"net"
"net/http"
"net/http/httptest"
"net/url"
"runtime"
"strings"
"sync/atomic"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/test/bufconn"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork"
agentNetworkTypes "github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
rpservice "github.com/netbirdio/netbird/management/internals/modules/reverseproxy/service"
mgmtgrpc "github.com/netbirdio/netbird/management/internals/shared/grpc"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/proxy/internal/middleware"
"github.com/netbirdio/netbird/proxy/internal/middleware/bodytap"
mwbuiltin "github.com/netbirdio/netbird/proxy/internal/middleware/builtin"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/cost_meter"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_guardrail"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_identity_inject"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_limit_check"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_limit_record"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_request_parser"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_response_parser"
_ "github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_router"
"github.com/netbirdio/netbird/proxy/internal/proxy"
nbproxytypes "github.com/netbirdio/netbird/proxy/internal/types"
"github.com/netbirdio/netbird/shared/management/proto"
log "github.com/sirupsen/logrus"
)
// TestReverseProxy_VertexGuardrail_ModelAllowlist drives Anthropic-on-Vertex
// requests (model in the URL path, not the body) through the full synthesized
// middleware chain against an in-process management stack and a fake upstream.
// With a Sonnet-only guardrail, Opus must be denied (model_blocked) before the
// upstream and Sonnet must reach it. Two provider shapes exercise different
// code: "catch_all" (no models → only the guardrail can block Opus) and
// "versioned_models" (raw "@version" ids → the router must normalize to route
// Sonnet while the guardrail still blocks Opus).
func TestReverseProxy_VertexGuardrail_ModelAllowlist(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("sqlite store not supported on Windows")
}
cases := []struct {
name string
providerModels []agentNetworkTypes.ProviderModel
}{
{name: "catch_all", providerModels: nil},
{name: "versioned_models", providerModels: []agentNetworkTypes.ProviderModel{
{ID: "claude-sonnet-4-5@20250929"},
{ID: "claude-opus-4-6@20250514"},
}},
}
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
runVertexGuardrailCase(t, tc.providerModels)
})
}
}
func runVertexGuardrailCase(t *testing.T, providerModels []agentNetworkTypes.ProviderModel) {
t.Helper()
const (
testAccountID = "acct-vertex-guard-1"
testAdminUser = "user-admin-1"
adminGroupID = "grp-admins"
providerID = "prov-vertex-test"
guardrailID = "ainguard-sonnet-only"
cluster = "test.proxy.local"
subdomain = "vertexguard"
)
testLogger := log.New()
testLogger.SetLevel(log.PanicLevel)
ctx := context.Background()
// Fake Vertex upstream: a hit on the disallowed model means a guardrail miss.
var upstreamHits atomic.Int64
upstreamBody := []byte(`{"id":"msg_x","type":"message","role":"assistant","model":"claude","content":[{"type":"text","text":"pong"}],"usage":{"input_tokens":5,"output_tokens":2}}`)
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamHits.Add(1)
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write(upstreamBody)
}))
t.Cleanup(upstream.Close)
// In-process management gRPC (bufconn) over a real sqlite store + manager.
st, cleanup, err := store.NewTestStoreFromSQL(ctx, "", t.TempDir())
require.NoError(t, err, "real sqlite test store must come up")
t.Cleanup(cleanup)
anMgr := agentnetwork.NewManager(st, nil, nil, nil)
server := &mgmtgrpc.ProxyServiceServer{}
server.SetAgentNetworkLimitsService(anMgr)
lis := bufconn.Listen(1024 * 1024)
srv := grpc.NewServer()
proto.RegisterProxyServiceServer(srv, server)
go func() { _ = srv.Serve(lis) }()
t.Cleanup(srv.Stop)
conn, err := grpc.NewClient("passthrough:///bufnet",
grpc.WithContextDialer(func(_ context.Context, _ string) (net.Conn, error) { return lis.Dial() }),
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
require.NoError(t, err)
t.Cleanup(func() { _ = conn.Close() })
mgmtClient := proto.NewProxyServiceClient(conn)
require.NoError(t, st.SaveAgentNetworkSettings(ctx, &agentNetworkTypes.Settings{
AccountID: testAccountID,
Cluster: cluster,
Subdomain: subdomain,
}))
require.NoError(t, st.SaveAgentNetworkProvider(ctx, &agentNetworkTypes.Provider{
ID: providerID,
AccountID: testAccountID,
ProviderID: "vertex_ai_api",
Name: "vertex-guard-test",
UpstreamURL: upstream.URL,
// A static bearer (not "keyfile::…") so the router injects a static auth
// header instead of minting a GCP token, which needs network egress and
// would deny before the guardrail runs, masking the decision under test.
APIKey: "static-vertex-token",
Enabled: true,
Models: providerModels,
SessionPrivateKey: "priv",
SessionPublicKey: "pub",
}))
// Guardrail allowlisting ONLY Sonnet.
require.NoError(t, st.SaveAgentNetworkGuardrail(ctx, &agentNetworkTypes.Guardrail{
ID: guardrailID,
AccountID: testAccountID,
Name: "sonnet-only",
Checks: agentNetworkTypes.GuardrailChecks{
ModelAllowlist: agentNetworkTypes.GuardrailModelAllowlist{
Enabled: true,
Models: []string{"claude-sonnet-4-5"},
},
},
}))
require.NoError(t, st.SaveAgentNetworkPolicy(ctx, &agentNetworkTypes.Policy{
ID: "ainpol-vertex-guard",
AccountID: testAccountID,
Name: "admins-vertex",
Enabled: true,
SourceGroups: []string{adminGroupID},
DestinationProviderIDs: []string{providerID},
GuardrailIDs: []string{guardrailID},
}))
services, err := agentnetwork.SynthesizeServices(ctx, st, testAccountID)
require.NoError(t, err)
require.Len(t, services, 1, "exactly one synth service expected")
synthSvc := services[0]
require.NotEmpty(t, synthSvc.Targets, "synth target must exist")
mwbuiltin.Configure(ctx, t.TempDir(), nil, testLogger, mgmtClient)
registry := mwbuiltin.DefaultRegistry()
mwMetrics, err := middleware.NewMetrics(nil)
require.NoError(t, err)
mwMgr := middleware.NewManager(0, mwMetrics, testLogger)
mwMgr.SetResolver(middleware.NewResolver(registry))
specs := make([]middleware.Spec, 0, len(synthSvc.Targets[0].Options.Middlewares))
for _, mw := range synthSvc.Targets[0].Options.Middlewares {
var slot middleware.Slot
switch mw.Slot {
case rpservice.MiddlewareSlotOnRequest:
slot = middleware.SlotOnRequest
case rpservice.MiddlewareSlotOnResponse:
slot = middleware.SlotOnResponse
case rpservice.MiddlewareSlotTerminal:
slot = middleware.SlotTerminal
default:
t.Fatalf("unknown middleware slot %q on %s", mw.Slot, mw.ID)
}
specs = append(specs, middleware.Spec{
ID: mw.ID,
Slot: slot,
Enabled: mw.Enabled,
FailMode: middleware.FailOpen,
Timeout: middleware.DefaultTimeout,
RawConfig: append([]byte(nil), mw.ConfigJSON...),
CanMutate: mw.CanMutate,
})
}
serviceIDStr := synthSvc.ID
require.NoError(t, mwMgr.Rebuild(serviceIDStr, []middleware.PathTargetBinding{{
ServiceID: serviceIDStr,
PathID: "/",
Specs: specs,
}}))
upstreamURL, err := url.Parse(upstream.URL)
require.NoError(t, err)
rp := proxy.NewReverseProxy(http.DefaultTransport, "auto", nil, testLogger, proxy.WithMiddlewareManager(mwMgr))
rp.AddMapping(proxy.Mapping{
ID: nbproxytypes.ServiceID(serviceIDStr),
AccountID: nbproxytypes.AccountID(testAccountID),
Host: synthSvc.Domain,
Paths: map[string]*proxy.PathTarget{
"/": {
URL: upstreamURL,
DirectUpstream: true,
AgentNetwork: true,
Middlewares: specs,
CaptureConfig: &bodytap.Config{
MaxRequestBytes: 1 << 20,
MaxResponseBytes: 1 << 20,
ContentTypes: []string{"application/json", "text/event-stream"},
},
},
},
})
// No "model" field — the model lives in the URL path, as Vertex clients send.
const vertexBody = `{"anthropic_version":"vertex-2023-10-16","max_tokens":64,"messages":[{"role":"user","content":"Reply with exactly: pong"}]}`
send := func(t *testing.T, model string) (int, int64, string) {
t.Helper()
before := upstreamHits.Load()
path := "/v1/projects/corp-gcp-it-all-claude/locations/global/publishers/anthropic/models/" + model + ":rawPredict"
req := httptest.NewRequest("POST", "https://"+synthSvc.Domain+path, strings.NewReader(vertexBody))
req.Host = synthSvc.Domain
req.Header.Set("Content-Type", "application/json")
cd := proxy.NewCapturedData("req-" + model)
cd.SetServiceID(nbproxytypes.ServiceID(serviceIDStr))
cd.SetAccountID(nbproxytypes.AccountID(testAccountID))
cd.SetUserID(testAdminUser)
cd.SetUserGroups([]string{adminGroupID})
cd.SetAuthMethod("tunnel_peer")
req = req.WithContext(proxy.WithCapturedData(req.Context(), cd))
w := httptest.NewRecorder()
rp.ServeHTTP(w, req)
return w.Code, upstreamHits.Load() - before, w.Body.String()
}
// Opus must be denied by the guardrail (model_blocked, not the router's
// model_not_routable) before reaching the upstream — the customer-reported bug.
t.Run("opus_denied_by_guardrail", func(t *testing.T) {
code, hits, body := send(t, "claude-opus-4-6")
assert.Equal(t, http.StatusForbidden, code, "Opus must be denied under a Sonnet-only allowlist; body=%s", body)
assert.Contains(t, body, "llm_policy.model_blocked", "denial must come from the guardrail allowlist, not routing; body=%s", body)
assert.Equal(t, int64(0), hits, "a denied request must never reach the Vertex upstream")
})
// The allowed model (Sonnet) passes the guardrail and reaches the upstream.
t.Run("sonnet_allowed", func(t *testing.T) {
code, hits, body := send(t, "claude-sonnet-4-5")
assert.Equal(t, http.StatusOK, code, "Sonnet is allowlisted and must be served; body=%s", body)
assert.Equal(t, int64(1), hits, "the allowed request must reach the Vertex upstream exactly once")
})
}