Compare commits

..

13 Commits

Author SHA1 Message Date
Zoltan Papp
753925032a [client] Move metrics saver into its own metricsstages package
Relocate MetricsStages (per-connection metric stage timestamps) out of
the peer package into client/internal/peer/metricsstages.
2026-07-11 20:15:30 +02:00
Zoltan Papp
9be5f238da [client] Trim WGIface to the methods peer uses and inline it into conn.go
Drop the unused Address() method from WGIface (only ice.Candidate.Address
was ever called, never the wg interface), and move the interface next to
its sole user WgConfig in conn.go, removing iface.go.
2026-07-11 20:09:59 +02:00
Zoltan Papp
791e8b33ae [client] Move ICE and relay workers into the worker package
Relocate WorkerICE (renamed worker.ICE) and WorkerRelay plus ConnPriority
into client/internal/peer/worker. To break the peer<->worker cycle the
workers no longer take *Conn or ConnConfig: callbacks are passed as plain
functions (Conn's unexported methods as method values), and each worker
receives only the fields it needs (key, ICE config, isController) plus a
small services struct. Context is passed to OnNewOffer instead of stored.

Move the worker connection-status helper the other way, out of the worker
package into peer as worker_status.go (WorkerStatus / AtomicWorkerStatus),
since only Conn uses it.
2026-07-11 20:06:46 +02:00
Zoltan Papp
917e85f648 [client] Extract Handshaker and Signaler into a signaling package
Move the signaling protocol out of the peer package into
client/internal/peer/signaling: OfferAnswer, IceCredentials, Handshaker
and Signaler. Break the peer<->signaling cycle by giving the Handshaker
a plain Config and an ICEWorker interface instead of *Conn/*WorkerICE,
and by passing the relay manager directly rather than the relay worker.
Combine the ICE worker's local-credentials and session-id accessors into
a single Credentials() returning a Credentials struct.

Move the ICE session id to the ice package as ice.SessionID, since it
identifies an ICE agent session and is minted there alongside
GenerateICECredentials; signaling only carries it.
2026-07-11 19:09:07 +02:00
Zoltan Papp
9e75d5c732 [client] Decouple ICE and relay workers from Conn via interfaces
WorkerICE and WorkerRelay took a concrete *Conn back-pointer. Replace it
with small callback interfaces (iceCallbacks, relayCallbacks) covering
only the methods each worker invokes on the connection. WorkerICE also
receives its portForwardManager as an explicit parameter instead of
reaching into conn.portForwardManager.
2026-07-11 18:29:26 +02:00
Zoltan Papp
1afc9bcac7 [client] Merge conntype package into peer
The conntype package held only ConnPriority and its constants and was
imported solely by peer. Move it into the peer package as priority.go
and drop the conntype. qualifier from conn.go, event.go and worker_ice.go.
2026-07-11 15:24:54 +02:00
Zoltan Papp
78f3165e85 [client] Extract WGWatcher into its own wg_watcher package
Move the WireGuard handshake watcher out of the peer package into
client/internal/peer/wg_watcher. The test stays an internal test since
it drives the unexported checkPeriod. Update conn.go to reference the
watcher through the package.
2026-07-11 15:17:16 +02:00
Zoltan Papp
4d76fd3c80 [client] Extract stateDump into its own state_dump package
Move the debug state dumper out of the peer package into
client/internal/peer/state_dump with an exported StateDump type and
NewStateDump constructor. Update conn.go, wg_watcher.go and the watcher
test to reference it through the package.
2026-07-11 15:14:22 +02:00
Zoltan Papp
07ffbc9424 [client] Extract peer status recorder into its own package
Move the Status recorder and its state types out of the peer package
into client/internal/peer/status, split by struct across recorder.go,
peer_state.go, full_status.go, events.go, notifier.go and route.go
instead of one 1600-line file. Rename the type Status -> Recorder
(NewRecorder already implied it; avoids status.Status stutter). Split
conn_status.go: the ConnStatus type and its constants move to the status
package, connStatusInputs stays with the peer event loop.

The peer package references the status package directly; a transitional
status_alias.go re-exports the moved symbols for the ~50 external callers
still using peer.Status/State/ConnStatus, to be removed once they are
migrated.
2026-07-11 15:09:45 +02:00
Zoltan Papp
467b2a1712 [client] Deduplicate peer state update methods in Status
The five UpdatePeer* methods repeated the same lock/copy/snapshot/notify
boilerplate. Extract a shared updatePeer helper taking a router-notify
predicate and a mutate closure; each method now only declares which
fields it copies. Replace the repeated inline peer-not-found error with
an errPeerNotExists sentinel.
2026-07-11 14:30:37 +02:00
Zoltan Papp
cb4088484e [client] Suppress ICE events from replaced agents
Track connection state per agent generation instead of the shared
lastKnownState field, which was written from concurrent agent callbacks
without a lock. The connect goroutine now drops the connection if the
agent was replaced during dialing, and a replaced agent's late
disconnected callback no longer reaches the conn after its successor
already reported ready. Only the agent whose connection was last
reported ready may report it down.
2026-07-11 14:22:58 +02:00
Zoltan Papp
7620599961 [client] Fix ICE dialer cancel race in WorkerICE.connect
The connect goroutine read the agentDialerCancel field without holding
muxAgent, racing with OnNewOffer replacing it for a new session. On
failure paths the stale read could cancel the new session's dialer
instead of its own. Pass the cancel func of the owning session as a
parameter, like the dialer context.
2026-07-11 13:37:59 +02:00
Zoltan Papp
5f98524e02 [client] Refactor peer Conn to a single-owner event loop
Replace the mutex-guarded callback model of peer.Conn with a per-peer
event loop that exclusively owns all mutable connection state. External
callers and transport workers post typed events into a non-blocking,
coalescing mailbox instead of contending on conn.mu:

- offers/answers coalesce to the newest message, a new offer flushes
  queued candidates of the superseded session
- candidates are applied in arrival order from a bounded FIFO
- transport state changes are never dropped
- the blocking relay dial runs on a helper goroutine with a single dial
  in flight; signaling I/O (offer/answer sends) runs off the loop

conn.mu now only guards the open/close lifecycle. Close posts a close
event and waits for the loop teardown; the loop also tears down on
engine context cancellation and releases resources of unprocessed
events.

Delete the Handshaker listener machinery (Listen loop, unbuffered
drop-on-busy channels, AsyncOfferListener with its double-processing of
the first offer), the never-wired dispatcher package and the unused
ICEMonitor.ReconnectCh. Fix a goroutine leak in the WG watcher test
that raced with tests mutating the package-level check timing vars.
2026-07-11 13:30:06 +02:00
83 changed files with 2113 additions and 3163 deletions

View File

@@ -17,9 +17,7 @@ import (
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/profilemanager"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/util"
)
@@ -333,14 +331,6 @@ func doForegroundLogin(ctx context.Context, cmd *cobra.Command, setupKey string,
return fmt.Errorf("read config file %s: %v", configFilePath, err)
}
// Mirror runInForegroundMode: recover residual state (DNS, firewall,
// ssh config, legacy routing) from a previous unclean shutdown and
// enable advanced routing before dialing management.
if err := server.RestoreResidualState(ctx, profilemanager.NewServiceManager(configFilePath).GetStatePath()); err != nil {
log.Warnf("failed to restore residual state: %v", err)
}
nbnet.Init()
err = foregroundLogin(ctx, cmd, config, setupKey, activeProf.ID)
if err != nil {
return fmt.Errorf("foreground login failed: %v", err)

View File

@@ -22,8 +22,6 @@ import (
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
"github.com/netbirdio/netbird/util"
@@ -231,24 +229,6 @@ func runInForegroundMode(ctx context.Context, cmd *cobra.Command, activeProf *pr
_, _ = profilemanager.UpdateOldManagementURL(ctx, config, configFilePath)
// Restore residual state left by a previous run that did not shut down
// cleanly, mirroring what the daemon does before connecting: it recovers
// DNS config (a stale resolv.conf takeover can make the management
// hostname unresolvable), firewall rules, ssh config and legacy routing.
// Route cleanup itself happens at engine start; nbnet.Init() below lets
// the management dial bypass a leftover fwmark rule until then.
// Foreground mode is particularly exposed in containers: a crashed
// container restarts inside the same (pod) network namespace, so stale
// state survives while the process does not.
if err := server.RestoreResidualState(ctx, profilemanager.NewServiceManager(configPath).GetStatePath()); err != nil {
log.Warnf("failed to restore residual state: %v", err)
}
// Enable advanced routing (as the daemon does on startup) so the
// management dial bypasses a leftover fwmark rule instead of being
// shunted into a stale routing table.
nbnet.Init()
err = foregroundLogin(ctx, cmd, config, providedSetupKey, activeProf.ID)
if err != nil {
return fmt.Errorf("foreground login failed: %v", err)

View File

@@ -480,6 +480,7 @@ func (g *BundleGenerator) addStatus() error {
fullStatus := g.statusRecorder.GetFullStatus()
protoFullStatus := nbstatus.ToProtoFullStatus(fullStatus)
protoFullStatus.Events = g.statusRecorder.GetEventHistory()
overview := nbstatus.ConvertToStatusOutputOverview(protoFullStatus, nbstatus.ConvertOptions{
Anonymize: g.anonymize,
ProfileName: profName,

View File

@@ -48,6 +48,7 @@ import (
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/peer/guard"
icemaker "github.com/netbirdio/netbird/client/internal/peer/ice"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
"github.com/netbirdio/netbird/client/internal/peerstore"
"github.com/netbirdio/netbird/client/internal/portforward"
"github.com/netbirdio/netbird/client/internal/profilemanager"
@@ -182,7 +183,7 @@ type EngineServices struct {
type Engine struct {
// signal is a Signal Service client
signal signal.Client
signaler *peer.Signaler
signaler *signaling.Signaler
// mgmClient is a Management Service client
mgmClient mgm.Client
// peerConns is a map that holds all the peers that are known to this peer
@@ -318,7 +319,7 @@ func NewEngine(
ctx: ctx,
cancel: cancel,
signal: services.SignalClient,
signaler: peer.NewSignaler(services.SignalClient, config.WgPrivateKey),
signaler: signaling.NewSignaler(services.SignalClient, config.WgPrivateKey),
mgmClient: services.MgmClient,
relayManager: services.RelayManager,
peerStore: peerstore.NewConnStore(),
@@ -2605,14 +2606,13 @@ func (e *Engine) updateForwardRules(rules []*mgmProto.ForwardingRule) ([]firewal
func (e *Engine) toExcludedLazyPeers(rules []firewallManager.ForwardRule, peers []*mgmProto.RemotePeerConfig) map[string]bool {
excludedPeers := make(map[string]bool)
// Ingress forward targets: inbound forwarded traffic is initiated remotely and
// cannot wake a lazy connection, so the peer routing the target must stay
// permanently connected. AllowedIPs are already parsed on the peer conn, so
// reuse those typed prefixes instead of re-parsing the network map strings.
for _, r := range rules {
ip := r.TranslatedAddress
for _, p := range peers {
if e.peerRoutesAddr(p, r.TranslatedAddress) {
for _, allowedIP := range p.GetAllowedIps() {
if allowedIP != ip.String() {
continue
}
log.Infof("exclude forwarder peer from lazy connection: %s", p.GetWgPubKey())
excludedPeers[p.GetWgPubKey()] = true
}
@@ -2622,27 +2622,6 @@ func (e *Engine) toExcludedLazyPeers(rules []firewallManager.ForwardRule, peers
return excludedPeers
}
// peerRoutesAddr reports whether the peer is a router for addr, matched against
// the peer's already-parsed AllowedIPs from the store (the same typed value the
// lazy manager consumes) rather than re-parsing the network map strings.
func (e *Engine) peerRoutesAddr(p *mgmProto.RemotePeerConfig, addr netip.Addr) bool {
prefixes, ok := e.peerStore.AllowedIPs(p.GetWgPubKey())
if !ok {
return false
}
return prefixesContain(prefixes, addr)
}
// prefixesContain reports whether addr falls within any of the prefixes.
func prefixesContain(prefixes []netip.Prefix, addr netip.Addr) bool {
for _, prefix := range prefixes {
if prefix.Contains(addr) {
return true
}
}
return false
}
// isChecksEqual checks if two slices of checks are equal.
func isChecksEqual(checks1, checks2 []*mgmProto.Checks) bool {
normalize := func(checks []*mgmProto.Checks) []string {
@@ -2769,7 +2748,7 @@ func createFile(path string) error {
return file.Close()
}
func convertToOfferAnswer(msg *sProto.Message) (*peer.OfferAnswer, error) {
func convertToOfferAnswer(msg *sProto.Message) (*signaling.OfferAnswer, error) {
remoteCred, err := signal.UnMarshalCredential(msg)
if err != nil {
return nil, err
@@ -2785,9 +2764,9 @@ func convertToOfferAnswer(msg *sProto.Message) (*peer.OfferAnswer, error) {
}
// Handle optional SessionID
var sessionID *peer.ICESessionID
var sessionID *icemaker.SessionID
if sessionBytes := msg.GetBody().GetSessionId(); sessionBytes != nil {
if id, err := peer.ICESessionIDFromBytes(sessionBytes); err != nil {
if id, err := icemaker.SessionIDFromBytes(sessionBytes); err != nil {
log.Warnf("Invalid session ID in message: %v", err)
sessionID = nil // Set to nil if conversion fails
} else {
@@ -2797,8 +2776,8 @@ func convertToOfferAnswer(msg *sProto.Message) (*peer.OfferAnswer, error) {
relayIP := decodeRelayIP(msg.GetBody().GetRelayServerIP())
offerAnswer := peer.OfferAnswer{
IceCredentials: peer.IceCredentials{
offerAnswer := signaling.OfferAnswer{
IceCredentials: signaling.IceCredentials{
UFrag: remoteCred.UFrag,
Pwd: remoteCred.Pwd,
},

View File

@@ -1,87 +0,0 @@
package internal
import (
"net/netip"
"testing"
"github.com/stretchr/testify/require"
firewallManager "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/peerstore"
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
)
func TestPrefixesContain(t *testing.T) {
tests := []struct {
name string
prefixes []string
addr string
want bool
}{
{name: "own overlay /32 matches", prefixes: []string{"100.110.8.145/32"}, addr: "100.110.8.145", want: true},
{name: "addr inside routed subnet", prefixes: []string{"10.121.0.0/16"}, addr: "10.121.208.4", want: true},
{name: "addr outside subnet", prefixes: []string{"10.121.0.0/16"}, addr: "10.122.0.1", want: false},
{name: "different /32", prefixes: []string{"100.110.8.145/32"}, addr: "100.110.8.146", want: false},
{name: "ipv6 /128 matches", prefixes: []string{"fd00::1/128"}, addr: "fd00::1", want: true},
{name: "no prefixes", prefixes: nil, addr: "10.121.208.4", want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
prefixes := make([]netip.Prefix, 0, len(tt.prefixes))
for _, p := range tt.prefixes {
prefixes = append(prefixes, netip.MustParsePrefix(p))
}
require.Equal(t, tt.want, prefixesContain(prefixes, netip.MustParseAddr(tt.addr)))
})
}
}
// TestToExcludedLazyPeers_ForwardTarget guards a regression: the forward-target
// peer (the peer routing a ForwardRule.TranslatedAddress) must be excluded from
// lazy connections, matched via the peer's already-parsed AllowedIPs.
func TestToExcludedLazyPeers_ForwardTarget(t *testing.T) {
const targetPeerKey = "cccccccccccccccccccccccccccccccccccccccccc0="
const otherPeerKey = "dddddddddddddddddddddddddddddddddddddddddd0="
store := peerstore.NewConnStore()
store.AddPeerConn(targetPeerKey, newTestConn(t, targetPeerKey, "100.110.8.145/32"))
store.AddPeerConn(otherPeerKey, newTestConn(t, otherPeerKey, "100.110.9.10/32"))
e := &Engine{peerStore: store}
peers := []*mgmProto.RemotePeerConfig{
{WgPubKey: targetPeerKey, AllowedIps: []string{"100.110.8.145/32"}},
{WgPubKey: otherPeerKey, AllowedIps: []string{"100.110.9.10/32"}},
}
rules := []firewallManager.ForwardRule{
{TranslatedAddress: netip.MustParseAddr("100.110.8.145")},
}
excluded := e.toExcludedLazyPeers(rules, peers)
require.True(t, excluded[targetPeerKey], "forward-target peer must be excluded from lazy connections")
require.False(t, excluded[otherPeerKey], "non-target peer must not be excluded")
require.Len(t, excluded, 1)
}
func TestToExcludedLazyPeers_NoRules(t *testing.T) {
e := &Engine{peerStore: peerstore.NewConnStore()}
peers := []*mgmProto.RemotePeerConfig{
{WgPubKey: "peer-a", AllowedIps: []string{"100.110.8.145/32"}},
}
require.Empty(t, e.toExcludedLazyPeers(nil, peers))
}
func newTestConn(t *testing.T, key, allowedIP string) *peer.Conn {
t.Helper()
conn, err := peer.NewConn(peer.ConnConfig{
Key: key,
WgConfig: peer.WgConfig{AllowedIps: []netip.Prefix{netip.MustParsePrefix(allowedIP)}},
}, peer.ServiceDependencies{})
require.NoError(t, err)
return conn
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,18 +1,5 @@
package peer
import (
log "github.com/sirupsen/logrus"
)
const (
// StatusIdle indicate the peer is in disconnected state
StatusIdle ConnStatus = iota
// StatusConnecting indicate the peer is in connecting state
StatusConnecting
// StatusConnected indicate the peer is in connected state
StatusConnected
)
// connStatusInputs is the primitive-valued snapshot of the state that drives the
// tri-state connection classification. Extracted so the decision logic can be unit-tested
// without constructing full Worker/Handshaker objects.
@@ -21,24 +8,7 @@ type connStatusInputs struct {
peerUsesRelay bool // remote peer advertises relay support AND local has relay
relayConnected bool // statusRelay reports Connected (independent of whether peer uses relay)
remoteSupportsICE bool // remote peer sent ICE credentials
iceWorkerCreated bool // local WorkerICE exists (false in force-relay mode)
iceWorkerCreated bool // local ICE worker exists (false in force-relay mode)
iceStatusConnecting bool // statusICE is anything other than Disconnected
iceInProgress bool // a negotiation is currently in flight
}
// ConnStatus describe the status of a peer's connection
type ConnStatus int32
func (s ConnStatus) String() string {
switch s {
case StatusConnecting:
return "Connecting"
case StatusConnected:
return "Connected"
case StatusIdle:
return "Idle"
default:
log.Errorf("unknown status: %d", s)
return "INVALID_PEER_CONNECTION_STATUS"
}
}

View File

@@ -3,28 +3,33 @@ package peer
import (
"context"
"fmt"
"net/netip"
"os"
"testing"
"time"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/internal/peer/dispatcher"
"github.com/netbirdio/netbird/client/internal/peer/guard"
"github.com/netbirdio/netbird/client/internal/peer/ice"
"github.com/netbirdio/netbird/client/internal/peer/metricsstages"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
"github.com/netbirdio/netbird/client/internal/peer/status"
"github.com/netbirdio/netbird/client/internal/stdnet"
"github.com/netbirdio/netbird/util"
)
var testDispatcher = dispatcher.NewConnectionDispatcher()
var connConf = ConnConfig{
Key: "LLHf3Ma6z6mdLbriAJbqhX7+nM/B71lgw2+91q3LfhU=",
LocalKey: "RRHf3Ma6z6mdLbriAJbqhX7+nM/B71lgw2+91q3LfhU=",
Timeout: time.Second,
LocalWgPort: 51820,
WgConfig: WgConfig{
AllowedIps: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
},
ICEConfig: ice.Config{
InterfaceBlackList: nil,
},
@@ -52,92 +57,37 @@ func TestConn_GetKey(t *testing.T) {
swWatcher := guard.NewSRWatcher(nil, nil, nil, connConf.ICEConfig)
sd := ServiceDependencies{
SrWatcher: swWatcher,
PeerConnDispatcher: testDispatcher,
SrWatcher: swWatcher,
}
conn, err := NewConn(connConf, sd)
if err != nil {
return
}
require.NoError(t, err)
got := conn.GetKey()
assert.Equal(t, got, connConf.Key, "they should be equal")
}
func TestConn_OnRemoteOffer(t *testing.T) {
// TestConn_DiscardMessagesWhenNotOpened: signal messages posted to a not yet
// opened connection must be discarded without blocking or panicking.
func TestConn_DiscardMessagesWhenNotOpened(t *testing.T) {
swWatcher := guard.NewSRWatcher(nil, nil, nil, connConf.ICEConfig)
sd := ServiceDependencies{
StatusRecorder: NewRecorder("https://mgm"),
SrWatcher: swWatcher,
PeerConnDispatcher: testDispatcher,
StatusRecorder: status.NewRecorder("https://mgm"),
SrWatcher: swWatcher,
}
conn, err := NewConn(connConf, sd)
if err != nil {
return
}
require.NoError(t, err)
onNewOfferChan := make(chan struct{})
conn.handshaker.AddRelayListener(func(remoteOfferAnswer *OfferAnswer) {
onNewOfferChan <- struct{}{}
})
conn.OnRemoteOffer(OfferAnswer{
IceCredentials: IceCredentials{
offerAnswer := signaling.OfferAnswer{
IceCredentials: signaling.IceCredentials{
UFrag: "test",
Pwd: "test",
},
WgListenPort: 0,
Version: "",
})
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
select {
case <-onNewOfferChan:
// success
case <-ctx.Done():
t.Error("expected to receive a new offer notification, but timed out")
}
}
func TestConn_OnRemoteAnswer(t *testing.T) {
swWatcher := guard.NewSRWatcher(nil, nil, nil, connConf.ICEConfig)
sd := ServiceDependencies{
StatusRecorder: NewRecorder("https://mgm"),
SrWatcher: swWatcher,
PeerConnDispatcher: testDispatcher,
}
conn, err := NewConn(connConf, sd)
if err != nil {
return
}
onNewOfferChan := make(chan struct{})
conn.handshaker.AddRelayListener(func(remoteOfferAnswer *OfferAnswer) {
onNewOfferChan <- struct{}{}
})
conn.OnRemoteAnswer(OfferAnswer{
IceCredentials: IceCredentials{
UFrag: "test",
Pwd: "test",
},
WgListenPort: 0,
Version: "",
})
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
select {
case <-onNewOfferChan:
// success
case <-ctx.Done():
t.Error("expected to receive a new offer notification, but timed out")
}
conn.OnRemoteOffer(offerAnswer)
conn.OnRemoteAnswer(offerAnswer)
conn.OnRemoteCandidate(nil, nil)
conn.Close(false)
}
func TestConn_presharedKey(t *testing.T) {
@@ -320,7 +270,7 @@ func newWGTimeoutTestConn(rosenpassEnabled bool, disconnected *[]string) *Conn {
ctx: context.Background(),
config: cfg,
Log: log.WithField("peer", cfg.Key),
metricsStages: &MetricsStages{},
metricsStages: &metricsstages.MetricsStages{},
}
conn.SetOnDisconnected(func(remotePeer string) {
*disconnected = append(*disconnected, remotePeer)
@@ -339,20 +289,20 @@ func TestConn_onWGDisconnected_EscalatesToRosenpassReset(t *testing.T) {
conn := newWGTimeoutTestConn(true, &disconnected)
for i := 0; i < wgTimeoutEscalationThreshold-1; i++ {
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
}
assert.Empty(t, disconnected, "escalation must not fire below the threshold")
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
assert.Equal(t, []string{conn.config.WgConfig.RemoteKey}, disconnected,
"reaching the threshold must report the peer disconnected once")
for i := 0; i < wgTimeoutEscalationThreshold-1; i++ {
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
}
assert.Len(t, disconnected, 1, "escalation must restart counting after firing")
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
assert.Len(t, disconnected, 2, "continued timeouts must escalate again")
}
@@ -364,12 +314,12 @@ func TestConn_onWGDisconnected_CheckSuccessResetsEscalation(t *testing.T) {
conn := newWGTimeoutTestConn(true, &disconnected)
for i := 0; i < wgTimeoutEscalationThreshold-1; i++ {
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
}
conn.onWGCheckSuccess()
conn.handleWGCheckSuccess()
for i := 0; i < wgTimeoutEscalationThreshold-1; i++ {
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
}
assert.Empty(t, disconnected, "handshake success must reset the timeout count")
}
@@ -382,7 +332,7 @@ func TestConn_onWGDisconnected_NoEscalationWithoutRosenpass(t *testing.T) {
conn := newWGTimeoutTestConn(false, &disconnected)
for i := 0; i < wgTimeoutEscalationThreshold*3; i++ {
conn.onWGDisconnected(conn.ctx)
conn.handleWGTimeout()
}
assert.Empty(t, disconnected, "escalation must be limited to rosenpass connections")
}

View File

@@ -1,52 +0,0 @@
package dispatcher
import (
"sync"
"github.com/netbirdio/netbird/client/internal/peer/id"
)
type ConnectionListener struct {
OnConnected func(peerID id.ConnID)
OnDisconnected func(peerID id.ConnID)
}
type ConnectionDispatcher struct {
listeners map[*ConnectionListener]struct{}
mu sync.Mutex
}
func NewConnectionDispatcher() *ConnectionDispatcher {
return &ConnectionDispatcher{
listeners: make(map[*ConnectionListener]struct{}),
}
}
func (e *ConnectionDispatcher) AddListener(listener *ConnectionListener) {
e.mu.Lock()
defer e.mu.Unlock()
e.listeners[listener] = struct{}{}
}
func (e *ConnectionDispatcher) RemoveListener(listener *ConnectionListener) {
e.mu.Lock()
defer e.mu.Unlock()
delete(e.listeners, listener)
}
func (e *ConnectionDispatcher) NotifyConnected(peerConnID id.ConnID) {
e.mu.Lock()
defer e.mu.Unlock()
for listener := range e.listeners {
listener.OnConnected(peerConnID)
}
}
func (e *ConnectionDispatcher) NotifyDisconnected(peerConnID id.ConnID) {
e.mu.Lock()
defer e.mu.Unlock()
for listener := range e.listeners {
listener.OnDisconnected(peerConnID)
}
}

View File

@@ -0,0 +1,69 @@
package peer
import (
"time"
"github.com/pion/ice/v4"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
"github.com/netbirdio/netbird/client/internal/peer/worker"
"github.com/netbirdio/netbird/route"
)
// event is a message processed by the Conn event loop. All mutable Conn state
// is owned by that loop; producers deliver events through the mailbox and
// never mutate Conn state directly.
type event any
// evClose asks the event loop to tear down the connection. done is closed
// once the teardown finished.
type evClose struct {
signalToRemote bool
done chan struct{}
}
type evRemoteOffer struct {
offer signaling.OfferAnswer
}
type evRemoteAnswer struct {
answer signaling.OfferAnswer
}
type evRemoteCandidate struct {
candidate ice.Candidate
haRoutes route.HAMap
}
type evICEReady struct {
priority worker.ConnPriority
info worker.ICEConnInfo
}
type evICEDown struct {
sessionChanged bool
}
type evRelayReady struct {
info worker.RelayConnInfo
}
type evRelayDown struct{}
// evRelayDialDone reports that the relay dial helper goroutine finished,
// successfully or not, so the loop may dispatch a pending offer.
type evRelayDialDone struct{}
type evWGTimeout struct{}
// evWGHandshake reports the first WireGuard handshake of the current watcher run.
type evWGHandshake struct {
when time.Time
}
// evWGCheckOK reports a watcher check that observed a fresh handshake,
// including handshakes of connections that were already up.
type evWGCheckOK struct{}
// evGuardTick asks the loop to send a new offer to restore connectivity.
type evGuardTick struct{}

View File

@@ -21,8 +21,6 @@ const (
)
type ICEMonitor struct {
ReconnectCh chan struct{}
iFaceDiscover stdnet.ExternalIFaceDiscover
iceConfig icemaker.Config
tickerPeriod time.Duration
@@ -34,7 +32,6 @@ type ICEMonitor struct {
func NewICEMonitor(iFaceDiscover stdnet.ExternalIFaceDiscover, config icemaker.Config, period time.Duration) *ICEMonitor {
log.Debugf("prepare ICE monitor with period: %s", period)
cm := &ICEMonitor{
ReconnectCh: make(chan struct{}, 1),
iFaceDiscover: iFaceDiscover,
iceConfig: config,
tickerPeriod: period,

View File

@@ -1,246 +0,0 @@
package peer
import (
"context"
"errors"
"net/netip"
"sync"
"sync/atomic"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/version"
)
var (
ErrSignalIsNotReady = errors.New("signal is not ready")
)
// IceCredentials ICE protocol credentials struct
type IceCredentials struct {
UFrag string
Pwd string
}
// OfferAnswer represents a session establishment offer or answer
type OfferAnswer struct {
IceCredentials IceCredentials
// WgListenPort is a remote WireGuard listen port.
// This field is used when establishing a direct WireGuard connection without any proxy.
// We can set the remote peer's endpoint with this port.
WgListenPort int
// Version of NetBird Agent
Version string
// RosenpassPubKey is the Rosenpass public key of the remote peer when receiving this message
// This value is the local Rosenpass server public key when sending the message
RosenpassPubKey []byte
// RosenpassAddr is the Rosenpass server address (IP:port) of the remote peer when receiving this message
// This value is the local Rosenpass server address when sending the message
RosenpassAddr string
// relay server address
RelaySrvAddress string
// RelaySrvIP is the IP the remote peer is connected to on its
// relay server. Used as a dial target if DNS for RelaySrvAddress
// fails. Zero value if the peer did not advertise an IP.
RelaySrvIP netip.Addr
// SessionID is the unique identifier of the session, used to discard old messages
SessionID *ICESessionID
}
func (o *OfferAnswer) hasICECredentials() bool {
return o.IceCredentials.UFrag != "" && o.IceCredentials.Pwd != ""
}
type Handshaker struct {
mu sync.Mutex
log *log.Entry
config ConnConfig
signaler *Signaler
ice *WorkerICE
relay *WorkerRelay
metricsStages *MetricsStages
// relayListener is not blocking because the listener is using a goroutine to process the messages
// and it will only keep the latest message if multiple offers are received in a short time
// this is to avoid blocking the handshaker if the listener is doing some heavy processing
// and also to avoid processing old offers if multiple offers are received in a short time
// the listener will always process the latest offer
relayListener *AsyncOfferListener
iceListener func(remoteOfferAnswer *OfferAnswer)
// remoteICESupported tracks whether the remote peer includes ICE credentials in its offers/answers.
// When false, the local side skips ICE listener dispatch and suppresses ICE credentials in responses.
remoteICESupported atomic.Bool
// remoteOffersCh is a channel used to wait for remote credentials to proceed with the connection
remoteOffersCh chan OfferAnswer
// remoteAnswerCh is a channel used to wait for remote credentials answer (confirmation of our offer) to proceed with the connection
remoteAnswerCh chan OfferAnswer
}
func NewHandshaker(log *log.Entry, config ConnConfig, signaler *Signaler, ice *WorkerICE, relay *WorkerRelay, metricsStages *MetricsStages) *Handshaker {
h := &Handshaker{
log: log,
config: config,
signaler: signaler,
ice: ice,
relay: relay,
metricsStages: metricsStages,
remoteOffersCh: make(chan OfferAnswer),
remoteAnswerCh: make(chan OfferAnswer),
}
// assume remote supports ICE until we learn otherwise from received offers
h.remoteICESupported.Store(ice != nil)
return h
}
func (h *Handshaker) RemoteICESupported() bool {
return h.remoteICESupported.Load()
}
func (h *Handshaker) AddRelayListener(offer func(remoteOfferAnswer *OfferAnswer)) {
h.relayListener = NewAsyncOfferListener(offer)
}
func (h *Handshaker) AddICEListener(offer func(remoteOfferAnswer *OfferAnswer)) {
h.iceListener = offer
}
func (h *Handshaker) Listen(ctx context.Context) {
for {
select {
case remoteOfferAnswer := <-h.remoteOffersCh:
h.log.Infof("received offer, running version %s, remote WireGuard listen port %d, session id: %s, remote ICE supported: %t", remoteOfferAnswer.Version, remoteOfferAnswer.WgListenPort, remoteOfferAnswer.SessionIDString(), remoteOfferAnswer.hasICECredentials())
// Record signaling received for reconnection attempts
if h.metricsStages != nil {
h.metricsStages.RecordSignalingReceived()
}
h.updateRemoteICEState(&remoteOfferAnswer)
if h.relayListener != nil {
h.relayListener.Notify(&remoteOfferAnswer)
}
if h.iceListener != nil && h.RemoteICESupported() {
h.iceListener(&remoteOfferAnswer)
}
if err := h.sendAnswer(); err != nil {
h.log.Errorf("failed to send remote offer confirmation: %s", err)
continue
}
case remoteOfferAnswer := <-h.remoteAnswerCh:
h.log.Infof("received answer, running version %s, remote WireGuard listen port %d, session id: %s, remote ICE supported: %t", remoteOfferAnswer.Version, remoteOfferAnswer.WgListenPort, remoteOfferAnswer.SessionIDString(), remoteOfferAnswer.hasICECredentials())
// Record signaling received for reconnection attempts
if h.metricsStages != nil {
h.metricsStages.RecordSignalingReceived()
}
h.updateRemoteICEState(&remoteOfferAnswer)
if h.relayListener != nil {
h.relayListener.Notify(&remoteOfferAnswer)
}
if h.iceListener != nil && h.RemoteICESupported() {
h.iceListener(&remoteOfferAnswer)
}
case <-ctx.Done():
h.log.Infof("stop listening for remote offers and answers")
return
}
}
}
func (h *Handshaker) SendOffer() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.sendOffer()
}
// OnRemoteOffer handles an offer from the remote peer and returns true if the message was accepted, false otherwise
// doesn't block, discards the message if connection wasn't ready
func (h *Handshaker) OnRemoteOffer(offer OfferAnswer) {
select {
case h.remoteOffersCh <- offer:
return
default:
h.log.Warnf("skipping remote offer message because receiver not ready")
// connection might not be ready yet to receive so we ignore the message
return
}
}
// OnRemoteAnswer handles an offer from the remote peer and returns true if the message was accepted, false otherwise
// doesn't block, discards the message if connection wasn't ready
func (h *Handshaker) OnRemoteAnswer(answer OfferAnswer) {
select {
case h.remoteAnswerCh <- answer:
return
default:
// connection might not be ready yet to receive so we ignore the message
h.log.Warnf("skipping remote answer message because receiver not ready")
return
}
}
// sendOffer prepares local user credentials and signals them to the remote peer
func (h *Handshaker) sendOffer() error {
if !h.signaler.Ready() {
return ErrSignalIsNotReady
}
offer := h.buildOfferAnswer()
h.log.Debugf("sending offer with serial: %s", offer.SessionIDString())
return h.signaler.SignalOffer(offer, h.config.Key)
}
func (h *Handshaker) sendAnswer() error {
answer := h.buildOfferAnswer()
h.log.Debugf("sending answer with serial: %s", answer.SessionIDString())
return h.signaler.SignalAnswer(answer, h.config.Key)
}
func (h *Handshaker) buildOfferAnswer() OfferAnswer {
answer := OfferAnswer{
WgListenPort: h.config.LocalWgPort,
Version: version.NetbirdVersion(),
RosenpassPubKey: h.config.RosenpassConfig.PubKey,
RosenpassAddr: h.config.RosenpassConfig.Addr,
}
if h.ice != nil && h.RemoteICESupported() {
uFrag, pwd := h.ice.GetLocalUserCredentials()
sid := h.ice.SessionID()
answer.IceCredentials = IceCredentials{uFrag, pwd}
answer.SessionID = &sid
}
if addr, ip, err := h.relay.RelayInstanceAddress(); err == nil {
answer.RelaySrvAddress = addr
answer.RelaySrvIP = ip
}
return answer
}
func (h *Handshaker) updateRemoteICEState(offer *OfferAnswer) {
hasICE := offer.hasICECredentials()
prev := h.remoteICESupported.Swap(hasICE)
if prev != hasICE {
if hasICE {
h.log.Infof("remote peer started sending ICE credentials")
} else {
h.log.Infof("remote peer stopped sending ICE credentials")
if h.ice != nil {
h.ice.Close()
}
}
}
}

View File

@@ -1,62 +0,0 @@
package peer
import (
"sync"
)
type callbackFunc func(remoteOfferAnswer *OfferAnswer)
func (oa *OfferAnswer) SessionIDString() string {
if oa.SessionID == nil {
return "unknown"
}
return oa.SessionID.String()
}
type AsyncOfferListener struct {
fn callbackFunc
running bool
latest *OfferAnswer
mu sync.Mutex
}
func NewAsyncOfferListener(fn callbackFunc) *AsyncOfferListener {
return &AsyncOfferListener{
fn: fn,
}
}
func (o *AsyncOfferListener) Notify(remoteOfferAnswer *OfferAnswer) {
o.mu.Lock()
defer o.mu.Unlock()
// Store the latest offer
o.latest = remoteOfferAnswer
// If already running, the running goroutine will pick up this latest value
if o.running {
return
}
// Start processing
o.running = true
// Process in a goroutine to avoid blocking the caller
go func(remoteOfferAnswer *OfferAnswer) {
for {
o.fn(remoteOfferAnswer)
o.mu.Lock()
if o.latest == nil {
// No more work to do
o.running = false
o.mu.Unlock()
return
}
remoteOfferAnswer = o.latest
// Clear the latest to mark it as being processed
o.latest = nil
o.mu.Unlock()
}
}(remoteOfferAnswer)
}

View File

@@ -1,39 +0,0 @@
package peer
import (
"testing"
"time"
)
func Test_newOfferListener(t *testing.T) {
dummyOfferAnswer := &OfferAnswer{}
runChan := make(chan struct{}, 10)
longRunningFn := func(remoteOfferAnswer *OfferAnswer) {
time.Sleep(1 * time.Second)
runChan <- struct{}{}
}
hl := NewAsyncOfferListener(longRunningFn)
hl.Notify(dummyOfferAnswer)
hl.Notify(dummyOfferAnswer)
hl.Notify(dummyOfferAnswer)
// Wait for exactly 2 callbacks
for i := 0; i < 2; i++ {
select {
case <-runChan:
case <-time.After(3 * time.Second):
t.Fatal("Timeout waiting for callback")
}
}
// Verify no additional callbacks happen
select {
case <-runChan:
t.Fatal("Unexpected additional callback")
case <-time.After(100 * time.Millisecond):
t.Log("Correctly received exactly 2 callbacks")
}
}

View File

@@ -1,4 +1,4 @@
package peer
package ice
import (
"crypto/rand"
@@ -9,26 +9,26 @@ import (
const sessionIDSize = 5
type ICESessionID string
type SessionID string
// NewICESessionID generates a new session ID for distinguishing sessions
func NewICESessionID() (ICESessionID, error) {
// NewSessionID generates a new session ID for distinguishing sessions
func NewSessionID() (SessionID, error) {
b := make([]byte, sessionIDSize)
if _, err := io.ReadFull(rand.Reader, b); err != nil {
return "", fmt.Errorf("failed to generate session ID: %w", err)
}
return ICESessionID(hex.EncodeToString(b)), nil
return SessionID(hex.EncodeToString(b)), nil
}
func ICESessionIDFromBytes(b []byte) (ICESessionID, error) {
func SessionIDFromBytes(b []byte) (SessionID, error) {
if len(b) != sessionIDSize {
return "", fmt.Errorf("invalid session ID length: %d", len(b))
}
return ICESessionID(hex.EncodeToString(b)), nil
return SessionID(hex.EncodeToString(b)), nil
}
// Bytes returns the raw bytes of the session ID for protobuf serialization
func (id ICESessionID) Bytes() ([]byte, error) {
func (id SessionID) Bytes() ([]byte, error) {
if len(id) == 0 {
return nil, fmt.Errorf("ICE session ID is empty")
}
@@ -42,6 +42,6 @@ func (id ICESessionID) Bytes() ([]byte, error) {
return b, nil
}
func (id ICESessionID) String() string {
func (id SessionID) String() string {
return string(id)
}

View File

@@ -1,22 +0,0 @@
package peer
import (
"net"
"net/netip"
"time"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"github.com/netbirdio/netbird/client/iface/configurer"
"github.com/netbirdio/netbird/client/iface/wgaddr"
"github.com/netbirdio/netbird/client/iface/wgproxy"
)
type WGIface interface {
UpdatePeer(peerKey string, allowedIps []netip.Prefix, keepAlive time.Duration, endpoint *net.UDPAddr, preSharedKey *wgtypes.Key) error
RemovePeer(peerKey string) error
GetStats() (map[string]configurer.WGStats, error)
GetProxy() wgproxy.Proxy
Address() wgaddr.Address
RemoveEndpointAddress(key string) error
}

View File

@@ -1,11 +0,0 @@
package peer
// Listener is a callback type about the NetBird network connection state
type Listener interface {
OnConnected()
OnDisconnected()
OnConnecting()
OnDisconnecting()
OnAddressChanged(string, string)
OnPeersListChanged(int)
}

View File

@@ -0,0 +1,116 @@
package peer
import (
"sync"
)
// maxQueuedCandidates bounds the remote candidate queue; on overflow the
// oldest candidate is dropped. Lost candidates are recovered by the next
// offer exchange triggered by the guard.
const maxQueuedCandidates = 128
// mailbox is the coalescing inbox of the Conn event loop. Posting never
// blocks. Per message kind either the latest value wins (offer, answer,
// guard tick), the values queue in bounded FIFO order (candidates) or in
// unbounded FIFO order (lifecycle and transport state changes, which are
// low-volume and must not be lost). A new offer flushes the queued
// candidates because they belong to the superseded session.
type mailbox struct {
mu sync.Mutex
closed bool
lifecycle []event
transport []event
offer *evRemoteOffer
answer *evRemoteAnswer
candidates []evRemoteCandidate
guardTick bool
wake chan struct{}
}
func newMailbox() *mailbox {
return &mailbox{
wake: make(chan struct{}, 1),
}
}
// post stores the event and wakes the loop. It reports false if the mailbox
// is already closed and the event was not accepted.
func (m *mailbox) post(ev event) bool {
m.mu.Lock()
if m.closed {
m.mu.Unlock()
return false
}
switch e := ev.(type) {
case evClose:
m.lifecycle = append(m.lifecycle, e)
case evRemoteOffer:
m.offer = &e
m.candidates = nil
case evRemoteAnswer:
m.answer = &e
case evRemoteCandidate:
if len(m.candidates) >= maxQueuedCandidates {
m.candidates = m.candidates[1:]
}
m.candidates = append(m.candidates, e)
case evGuardTick:
m.guardTick = true
default:
m.transport = append(m.transport, ev)
}
m.mu.Unlock()
select {
case m.wake <- struct{}{}:
default:
}
return true
}
// drain returns the pending events in processing order: lifecycle first,
// then transport state changes, the coalesced offer and answer, the queued
// candidates and finally the guard tick.
func (m *mailbox) drain() []event {
m.mu.Lock()
defer m.mu.Unlock()
return m.drainLocked()
}
// closeAndDrain marks the mailbox closed so further posts are rejected and
// returns the events that were still pending.
func (m *mailbox) closeAndDrain() []event {
m.mu.Lock()
defer m.mu.Unlock()
m.closed = true
return m.drainLocked()
}
func (m *mailbox) drainLocked() []event {
evs := make([]event, 0, len(m.lifecycle)+len(m.transport)+len(m.candidates)+3)
evs = append(evs, m.lifecycle...)
evs = append(evs, m.transport...)
if m.offer != nil {
evs = append(evs, *m.offer)
}
if m.answer != nil {
evs = append(evs, *m.answer)
}
for _, c := range m.candidates {
evs = append(evs, c)
}
if m.guardTick {
evs = append(evs, evGuardTick{})
}
m.lifecycle = nil
m.transport = nil
m.offer = nil
m.answer = nil
m.candidates = nil
m.guardTick = false
return evs
}

View File

@@ -0,0 +1,128 @@
package peer
import (
"testing"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestMailbox_OfferCoalescing(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{WgListenPort: 1}}))
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{WgListenPort: 2}}))
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{WgListenPort: 3}}))
evs := mb.drain()
require.Len(t, evs, 1, "consecutive offers must coalesce to a single event")
offer, ok := evs[0].(evRemoteOffer)
require.True(t, ok, "coalesced event must be an offer")
assert.Equal(t, 3, offer.offer.WgListenPort, "the newest offer must win")
}
func TestMailbox_OfferFlushesCandidates(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evRemoteCandidate{}))
require.True(t, mb.post(evRemoteCandidate{}))
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{}}))
evs := mb.drain()
require.Len(t, evs, 1, "candidates of the superseded session must be flushed")
_, ok := evs[0].(evRemoteOffer)
assert.True(t, ok, "only the offer must remain after the flush")
}
func TestMailbox_CandidatesKeepOrderAfterOffer(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{}}))
require.True(t, mb.post(evRemoteCandidate{haRoutes: nil}))
require.True(t, mb.post(evRemoteCandidate{haRoutes: nil}))
evs := mb.drain()
require.Len(t, evs, 3)
_, ok := evs[0].(evRemoteOffer)
assert.True(t, ok, "offer must be processed before the candidates")
for _, ev := range evs[1:] {
_, ok := ev.(evRemoteCandidate)
assert.True(t, ok, "candidates posted after the offer must survive")
}
}
func TestMailbox_CandidateQueueBounded(t *testing.T) {
mb := newMailbox()
for i := 0; i < maxQueuedCandidates+10; i++ {
require.True(t, mb.post(evRemoteCandidate{}))
}
evs := mb.drain()
assert.Len(t, evs, maxQueuedCandidates, "candidate queue must stay bounded")
}
func TestMailbox_DrainOrder(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evGuardTick{}))
require.True(t, mb.post(evRemoteAnswer{answer: signaling.OfferAnswer{}}))
require.True(t, mb.post(evRemoteOffer{offer: signaling.OfferAnswer{}}))
require.True(t, mb.post(evRelayDown{}))
require.True(t, mb.post(evICEDown{sessionChanged: true}))
require.True(t, mb.post(evClose{}))
evs := mb.drain()
require.Len(t, evs, 6)
_, ok := evs[0].(evClose)
assert.True(t, ok, "lifecycle events must come first")
_, ok = evs[1].(evRelayDown)
assert.True(t, ok, "transport events must keep FIFO order")
_, ok = evs[2].(evICEDown)
assert.True(t, ok, "transport events must keep FIFO order")
_, ok = evs[3].(evRemoteOffer)
assert.True(t, ok, "offer must come after transport events")
_, ok = evs[4].(evRemoteAnswer)
assert.True(t, ok, "answer must come after the offer")
_, ok = evs[5].(evGuardTick)
assert.True(t, ok, "guard tick must come last")
}
func TestMailbox_GuardTickCoalesced(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evGuardTick{}))
require.True(t, mb.post(evGuardTick{}))
require.True(t, mb.post(evGuardTick{}))
evs := mb.drain()
assert.Len(t, evs, 1, "guard ticks must coalesce to a single event")
}
func TestMailbox_PostAfterCloseRejected(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evRelayDown{}))
leftovers := mb.closeAndDrain()
assert.Len(t, leftovers, 1, "pending events must be returned on close")
assert.False(t, mb.post(evRelayDown{}), "posts must be rejected after close")
assert.Empty(t, mb.drain(), "no events must remain after close")
}
func TestMailbox_WakeSignal(t *testing.T) {
mb := newMailbox()
require.True(t, mb.post(evRelayDown{}))
require.True(t, mb.post(evGuardTick{}))
select {
case <-mb.wake:
default:
t.Fatal("wake signal must be pending after posts")
}
assert.Len(t, mb.drain(), 2, "a single wake must deliver all pending events")
}

View File

@@ -1,4 +1,4 @@
package peer
package metricsstages
import (
"sync"

View File

@@ -1,4 +1,4 @@
package peer
package metricsstages
import (
"testing"

View File

@@ -0,0 +1,189 @@
package signaling
import (
"errors"
"net/netip"
"sync"
"sync/atomic"
log "github.com/sirupsen/logrus"
icemaker "github.com/netbirdio/netbird/client/internal/peer/ice"
relayClient "github.com/netbirdio/netbird/shared/relay/client"
"github.com/netbirdio/netbird/version"
)
var (
ErrSignalIsNotReady = errors.New("signal is not ready")
)
// IceCredentials ICE protocol credentials struct
type IceCredentials struct {
UFrag string
Pwd string
}
// OfferAnswer represents a session establishment offer or answer
type OfferAnswer struct {
IceCredentials IceCredentials
// WgListenPort is a remote WireGuard listen port.
// This field is used when establishing a direct WireGuard connection without any proxy.
// We can set the remote peer's endpoint with this port.
WgListenPort int
// Version of NetBird Agent
Version string
// RosenpassPubKey is the Rosenpass public key of the remote peer when receiving this message
// This value is the local Rosenpass server public key when sending the message
RosenpassPubKey []byte
// RosenpassAddr is the Rosenpass server address (IP:port) of the remote peer when receiving this message
// This value is the local Rosenpass server address when sending the message
RosenpassAddr string
// relay server address
RelaySrvAddress string
// RelaySrvIP is the IP the remote peer is connected to on its
// relay server. Used as a dial target if DNS for RelaySrvAddress
// fails. Zero value if the peer did not advertise an IP.
RelaySrvIP netip.Addr
// SessionID is the unique identifier of the session, used to discard old messages
SessionID *icemaker.SessionID
}
func (o *OfferAnswer) HasICECredentials() bool {
return o.IceCredentials.UFrag != "" && o.IceCredentials.Pwd != ""
}
func (o *OfferAnswer) SessionIDString() string {
if o.SessionID == nil {
return "unknown"
}
return o.SessionID.String()
}
// Config carries the peer-specific values the Handshaker embeds into offers
// and answers.
type Config struct {
Key string
LocalWgPort int
RosenpassPubKey []byte
RosenpassAddr string
}
// Credentials are the local ICE credentials and session id the Handshaker embeds in offers.
type Credentials struct {
UFrag string
Pwd string
SessionID icemaker.SessionID
}
// ICEWorker is the subset of the ICE worker the Handshaker needs to build offers.
type ICEWorker interface {
Credentials() Credentials
Close()
}
// Handshaker keeps the signaling protocol logic: building and sending offers
// and answers and tracking whether the remote peer supports ICE. Incoming
// message processing is driven by the Conn event loop.
type Handshaker struct {
mu sync.Mutex
log *log.Entry
config Config
signaler *Signaler
ice ICEWorker
relayManager *relayClient.Manager
// remoteICESupported tracks whether the remote peer includes ICE credentials in its offers/answers.
// When false, the local side skips ICE dispatch and suppresses ICE credentials in responses.
remoteICESupported atomic.Bool
}
func NewHandshaker(log *log.Entry, config Config, signaler *Signaler, ice ICEWorker, relayManager *relayClient.Manager) *Handshaker {
h := &Handshaker{
log: log,
config: config,
signaler: signaler,
ice: ice,
relayManager: relayManager,
}
// assume remote supports ICE until we learn otherwise from received offers
h.remoteICESupported.Store(ice != nil)
return h
}
func (h *Handshaker) RemoteICESupported() bool {
return h.remoteICESupported.Load()
}
func (h *Handshaker) SendOffer() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.sendOffer()
}
func (h *Handshaker) SendAnswer() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.sendAnswer()
}
// sendOffer prepares local user credentials and signals them to the remote peer
func (h *Handshaker) sendOffer() error {
if !h.signaler.Ready() {
return ErrSignalIsNotReady
}
offer := h.buildOfferAnswer()
h.log.Debugf("sending offer with serial: %s", offer.SessionIDString())
return h.signaler.SignalOffer(offer, h.config.Key)
}
func (h *Handshaker) sendAnswer() error {
answer := h.buildOfferAnswer()
h.log.Debugf("sending answer with serial: %s", answer.SessionIDString())
return h.signaler.SignalAnswer(answer, h.config.Key)
}
func (h *Handshaker) buildOfferAnswer() OfferAnswer {
answer := OfferAnswer{
WgListenPort: h.config.LocalWgPort,
Version: version.NetbirdVersion(),
RosenpassPubKey: h.config.RosenpassPubKey,
RosenpassAddr: h.config.RosenpassAddr,
}
if h.ice != nil && h.RemoteICESupported() {
creds := h.ice.Credentials()
answer.IceCredentials = IceCredentials{creds.UFrag, creds.Pwd}
sid := creds.SessionID
answer.SessionID = &sid
}
if addr, ip, err := h.relayManager.RelayInstanceAddress(); err == nil {
answer.RelaySrvAddress = addr
answer.RelaySrvIP = ip
}
return answer
}
// UpdateRemoteICEState refreshes the remote ICE support flag from a received
// offer or answer and closes the ICE worker when the remote peer stopped
// sending ICE credentials. Runs on the Conn event loop.
func (h *Handshaker) UpdateRemoteICEState(offer *OfferAnswer) {
hasICE := offer.HasICECredentials()
prev := h.remoteICESupported.Swap(hasICE)
if prev != hasICE {
if hasICE {
h.log.Infof("remote peer started sending ICE credentials")
} else {
h.log.Infof("remote peer stopped sending ICE credentials")
if h.ice != nil {
h.ice.Close()
}
}
}
}

View File

@@ -1,4 +1,4 @@
package peer
package signaling
import (
"github.com/pion/ice/v4"

View File

@@ -1,4 +1,4 @@
package peer
package state_dump
import (
"context"
@@ -6,11 +6,13 @@ import (
"time"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/peer/status"
)
type stateDump struct {
type StateDump struct {
log *log.Entry
status *Status
status *status.Recorder
key string
sentOffer int
@@ -26,15 +28,15 @@ type stateDump struct {
mu sync.Mutex
}
func newStateDump(key string, log *log.Entry, statusRecorder *Status) *stateDump {
return &stateDump{
func NewStateDump(key string, log *log.Entry, statusRecorder *status.Recorder) *StateDump {
return &StateDump{
log: log,
status: statusRecorder,
key: key,
}
}
func (s *stateDump) Start(ctx context.Context) {
func (s *StateDump) Start(ctx context.Context) {
ticker := time.NewTicker(10 * time.Minute)
defer ticker.Stop()
@@ -48,25 +50,25 @@ func (s *stateDump) Start(ctx context.Context) {
}
}
func (s *stateDump) RemoteOffer() {
func (s *StateDump) RemoteOffer() {
s.mu.Lock()
defer s.mu.Unlock()
s.remoteOffer++
}
func (s *stateDump) RemoteCandidate() {
func (s *StateDump) RemoteCandidate() {
s.mu.Lock()
defer s.mu.Unlock()
s.remoteCandidate++
}
func (s *stateDump) SendOffer() {
func (s *StateDump) SendOffer() {
s.mu.Lock()
defer s.mu.Unlock()
s.sentOffer++
}
func (s *stateDump) dumpState() {
func (s *StateDump) dumpState() {
s.mu.Lock()
defer s.mu.Unlock()
@@ -80,41 +82,41 @@ func (s *stateDump) dumpState() {
status, s.sentOffer, s.remoteOffer, s.remoteAnswer, s.remoteCandidate, s.p2pConnected, s.switchToRelay, s.wgCheckSuccess, s.relayConnected, s.localProxies)
}
func (s *stateDump) RemoteAnswer() {
func (s *StateDump) RemoteAnswer() {
s.mu.Lock()
defer s.mu.Unlock()
s.remoteAnswer++
}
func (s *stateDump) P2PConnected() {
func (s *StateDump) P2PConnected() {
s.mu.Lock()
defer s.mu.Unlock()
s.p2pConnected++
}
func (s *stateDump) SwitchToRelay() {
func (s *StateDump) SwitchToRelay() {
s.mu.Lock()
defer s.mu.Unlock()
s.switchToRelay++
}
func (s *stateDump) WGcheckSuccess() {
func (s *StateDump) WGcheckSuccess() {
s.mu.Lock()
defer s.mu.Unlock()
s.wgCheckSuccess++
}
func (s *stateDump) RelayConnected() {
func (s *StateDump) RelayConnected() {
s.mu.Lock()
defer s.mu.Unlock()
s.relayConnected++
}
func (s *stateDump) NewLocalProxy() {
func (s *StateDump) NewLocalProxy() {
s.mu.Lock()
defer s.mu.Unlock()

View File

@@ -0,0 +1,31 @@
package status
import (
log "github.com/sirupsen/logrus"
)
const (
// StatusIdle indicate the peer is in disconnected state
StatusIdle ConnStatus = iota
// StatusConnecting indicate the peer is in connecting state
StatusConnecting
// StatusConnected indicate the peer is in connected state
StatusConnected
)
// ConnStatus describe the status of a peer's connection
type ConnStatus int32
func (s ConnStatus) String() string {
switch s {
case StatusConnecting:
return "Connecting"
case StatusConnected:
return "Connected"
case StatusIdle:
return "Idle"
default:
log.Errorf("unknown status: %d", s)
return "INVALID_PEER_CONNECTION_STATUS"
}
}

View File

@@ -1,4 +1,4 @@
package peer
package status
import (
"testing"

View File

@@ -0,0 +1,48 @@
package status
import (
"slices"
"sync"
"github.com/netbirdio/netbird/client/proto"
)
type EventQueue struct {
maxSize int
events []*proto.SystemEvent
mutex sync.RWMutex
}
func NewEventQueue(size int) *EventQueue {
return &EventQueue{
maxSize: size,
events: make([]*proto.SystemEvent, 0, size),
}
}
func (q *EventQueue) Add(event *proto.SystemEvent) {
q.mutex.Lock()
defer q.mutex.Unlock()
q.events = append(q.events, event)
if len(q.events) > q.maxSize {
q.events = q.events[len(q.events)-q.maxSize:]
}
}
func (q *EventQueue) GetAll() []*proto.SystemEvent {
q.mutex.RLock()
defer q.mutex.RUnlock()
return slices.Clone(q.events)
}
type EventSubscription struct {
id string
events chan *proto.SystemEvent
}
func (s *EventSubscription) Events() <-chan *proto.SystemEvent {
return s.events
}

View File

@@ -0,0 +1,122 @@
package status
import (
"golang.org/x/exp/maps"
"google.golang.org/protobuf/types/known/durationpb"
"google.golang.org/protobuf/types/known/timestamppb"
"github.com/netbirdio/netbird/client/internal/relay"
"github.com/netbirdio/netbird/client/proto"
)
// FullStatus contains the full state held by the Recorder instance
type FullStatus struct {
Peers []State
ManagementState ManagementState
SignalState SignalState
LocalPeerState LocalPeerState
RosenpassState RosenpassState
Relays []relay.ProbeResult
NSGroupStates []NSGroupState
NumOfForwardingRules int
LazyConnectionEnabled bool
Events []*proto.SystemEvent
}
// ToProto converts FullStatus to proto.FullStatus.
func (fs FullStatus) ToProto() *proto.FullStatus {
pbFullStatus := proto.FullStatus{
ManagementState: &proto.ManagementState{},
SignalState: &proto.SignalState{},
LocalPeerState: &proto.LocalPeerState{},
Peers: []*proto.PeerState{},
}
pbFullStatus.ManagementState.URL = fs.ManagementState.URL
pbFullStatus.ManagementState.Connected = fs.ManagementState.Connected
if err := fs.ManagementState.Error; err != nil {
pbFullStatus.ManagementState.Error = err.Error()
}
pbFullStatus.SignalState.URL = fs.SignalState.URL
pbFullStatus.SignalState.Connected = fs.SignalState.Connected
if err := fs.SignalState.Error; err != nil {
pbFullStatus.SignalState.Error = err.Error()
}
pbFullStatus.LocalPeerState.IP = fs.LocalPeerState.IP
pbFullStatus.LocalPeerState.Ipv6 = fs.LocalPeerState.IPv6
pbFullStatus.LocalPeerState.PubKey = fs.LocalPeerState.PubKey
pbFullStatus.LocalPeerState.KernelInterface = fs.LocalPeerState.KernelInterface
pbFullStatus.LocalPeerState.Fqdn = fs.LocalPeerState.FQDN
pbFullStatus.LocalPeerState.WgPort = int32(fs.LocalPeerState.WgPort)
pbFullStatus.LocalPeerState.RosenpassPermissive = fs.RosenpassState.Permissive
pbFullStatus.LocalPeerState.RosenpassEnabled = fs.RosenpassState.Enabled
pbFullStatus.NumberOfForwardingRules = int32(fs.NumOfForwardingRules)
pbFullStatus.LazyConnectionEnabled = fs.LazyConnectionEnabled
pbFullStatus.LocalPeerState.Networks = maps.Keys(fs.LocalPeerState.Routes)
for _, peerState := range fs.Peers {
networks := maps.Keys(peerState.GetRoutes())
pbPeerState := &proto.PeerState{
IP: peerState.IP,
Ipv6: peerState.IPv6,
PubKey: peerState.PubKey,
ConnStatus: peerState.ConnStatus.String(),
ConnStatusUpdate: timestamppb.New(peerState.ConnStatusUpdate),
Relayed: peerState.Relayed,
LocalIceCandidateType: peerState.LocalIceCandidateType,
RemoteIceCandidateType: peerState.RemoteIceCandidateType,
LocalIceCandidateEndpoint: peerState.LocalIceCandidateEndpoint,
RemoteIceCandidateEndpoint: peerState.RemoteIceCandidateEndpoint,
RelayAddress: peerState.RelayServerAddress,
Fqdn: peerState.FQDN,
LastWireguardHandshake: timestamppb.New(peerState.LastWireguardHandshake),
BytesRx: peerState.BytesRx,
BytesTx: peerState.BytesTx,
RosenpassEnabled: peerState.RosenpassEnabled,
Networks: networks,
Latency: durationpb.New(peerState.Latency),
SshHostKey: peerState.SSHHostKey,
}
pbFullStatus.Peers = append(pbFullStatus.Peers, pbPeerState)
}
for _, relayState := range fs.Relays {
pbRelayState := &proto.RelayState{
URI: relayState.URI,
Available: relayState.Err == nil,
Transport: relayState.Transport,
}
if err := relayState.Err; err != nil {
pbRelayState.Error = err.Error()
}
pbFullStatus.Relays = append(pbFullStatus.Relays, pbRelayState)
}
for _, dnsState := range fs.NSGroupStates {
var err string
if dnsState.Error != nil {
err = dnsState.Error.Error()
}
var servers []string
for _, server := range dnsState.Servers {
servers = append(servers, server.String())
}
pbDnsState := &proto.NSGroupState{
Servers: servers,
Domains: dnsState.Domains,
Enabled: dnsState.Enabled,
Error: err,
}
pbFullStatus.DnsServers = append(pbFullStatus.DnsServers, pbDnsState)
}
pbFullStatus.Events = fs.Events
return &pbFullStatus
}

View File

@@ -1,4 +1,4 @@
package peer
package status
import (
"sync"
@@ -11,6 +11,16 @@ const (
stateDisconnecting
)
// Listener is a callback type about the NetBird network connection state
type Listener interface {
OnConnected()
OnDisconnected()
OnConnecting()
OnDisconnecting()
OnAddressChanged(string, string)
OnPeersListChanged(int)
}
type notifier struct {
serverStateLock sync.Mutex
listenersLock sync.Mutex

View File

@@ -1,4 +1,4 @@
package peer
package status
import (
"sync"

View File

@@ -0,0 +1,63 @@
package status
import (
"sync"
"time"
"golang.org/x/exp/maps"
)
// State contains the latest state of a peer
type State struct {
Mux *sync.RWMutex
IP string
IPv6 string
PubKey string
FQDN string
ConnStatus ConnStatus
ConnStatusUpdate time.Time
Relayed bool
LocalIceCandidateType string
RemoteIceCandidateType string
LocalIceCandidateEndpoint string
RemoteIceCandidateEndpoint string
RelayServerAddress string
LastWireguardHandshake time.Time
BytesTx int64
BytesRx int64
Latency time.Duration
RosenpassEnabled bool
SSHHostKey []byte
routes map[string]struct{}
}
// AddRoute add a single route to routes map
func (s *State) AddRoute(network string) {
s.Mux.Lock()
defer s.Mux.Unlock()
if s.routes == nil {
s.routes = make(map[string]struct{})
}
s.routes[network] = struct{}{}
}
// SetRoutes set state routes
func (s *State) SetRoutes(routes map[string]struct{}) {
s.Mux.Lock()
defer s.Mux.Unlock()
s.routes = routes
}
// DeleteRoute removes a route from the network amp
func (s *State) DeleteRoute(network string) {
s.Mux.Lock()
defer s.Mux.Unlock()
delete(s.routes, network)
}
// GetRoutes return routes map
func (s *State) GetRoutes() map[string]struct{} {
s.Mux.RLock()
defer s.Mux.RUnlock()
return maps.Clone(s.routes)
}

View File

@@ -1,4 +1,4 @@
package peer
package status
import (
"context"

View File

@@ -1,4 +1,4 @@
package peer
package status
import (
"net/netip"

View File

@@ -0,0 +1,36 @@
package peer
import "github.com/netbirdio/netbird/client/internal/peer/status"
// Transitional aliases re-exporting the peer status recorder from its own
// package. Callers are being migrated to reference the status package
// directly; these aliases will be removed once the migration completes.
type (
Status = status.Recorder
State = status.State
ConnStatus = status.ConnStatus
FullStatus = status.FullStatus
RouterState = status.RouterState
LocalPeerState = status.LocalPeerState
SignalState = status.SignalState
ManagementState = status.ManagementState
RosenpassState = status.RosenpassState
NSGroupState = status.NSGroupState
ResolvedDomainInfo = status.ResolvedDomainInfo
StatusChangeSubscription = status.StatusChangeSubscription
EventQueue = status.EventQueue
EventSubscription = status.EventSubscription
WGIfaceStatus = status.WGIfaceStatus
Listener = status.Listener
EventListener = status.EventListener
)
const (
StatusIdle = status.StatusIdle
StatusConnecting = status.StatusConnecting
StatusConnected = status.StatusConnected
)
var (
NewRecorder = status.NewRecorder
)

View File

@@ -1,13 +1,15 @@
package peer
package wg_watcher
import (
"context"
"fmt"
"sync"
"time"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/iface/configurer"
"github.com/netbirdio/netbird/client/internal/peer/state_dump"
)
const (
@@ -23,21 +25,21 @@ type WGInterfaceStater interface {
GetStats() (map[string]configurer.WGStats, error)
}
// WGWatcher is single-shot: one instance per connection attempt, run once, then discarded.
// Lifecycle is owned by Conn under conn.mu, so it keeps no "enabled" state to go stale.
type WGWatcher struct {
log *log.Entry
wgIfaceStater WGInterfaceStater
peerKey string
stateDump *stateDump
stateDump *state_dump.StateDump
enabled bool
muEnabled sync.Mutex
// initialHandshake is not thread-safe; never call PrepareInitialHandshake and EnableWgWatcher concurrently.
initialHandshake time.Time
resetCh chan struct{}
}
func NewWGWatcher(log *log.Entry, wgIfaceStater WGInterfaceStater, peerKey string, stateDump *stateDump) *WGWatcher {
func NewWGWatcher(log *log.Entry, wgIfaceStater WGInterfaceStater, peerKey string, stateDump *state_dump.StateDump) *WGWatcher {
return &WGWatcher{
log: log,
wgIfaceStater: wgIfaceStater,
@@ -47,14 +49,25 @@ func NewWGWatcher(log *log.Entry, wgIfaceStater WGInterfaceStater, peerKey strin
}
}
// PrepareInitialHandshake reads the peer's current WireGuard handshake time. It must be
// called before the peer is (re)configured on the WireGuard interface, so the captured
// baseline reflects the state prior to this connection attempt instead of racing with
// that configuration.
func (w *WGWatcher) PrepareInitialHandshake() {
// PrepareInitialHandshake reserves the watcher and reads the peer's current WireGuard
// handshake time. It must be called before the peer is (re)configured on the WireGuard
// interface, so the captured baseline reflects the state prior to this connection attempt
// instead of racing with that configuration. Returns ok=false if the watcher is already
// running, in which case EnableWgWatcher must not be called.
func (w *WGWatcher) PrepareInitialHandshake() (ok bool) {
w.muEnabled.Lock()
if w.enabled {
w.muEnabled.Unlock()
return false
}
w.log.Debugf("enable WireGuard watcher")
w.enabled = true
w.muEnabled.Unlock()
handshake, _ := w.wgState()
w.initialHandshake = handshake
return true
}
// EnableWgWatcher runs the WireGuard watcher loop using the handshake baseline captured by
@@ -64,6 +77,10 @@ func (w *WGWatcher) PrepareInitialHandshake() {
// handshake, including the first.
func (w *WGWatcher) EnableWgWatcher(ctx context.Context, enabledTime time.Time, onDisconnectedFn func(), onHandshakeSuccessFn func(when time.Time), onCheckSuccessFn func()) {
w.periodicHandshakeCheck(ctx, onDisconnectedFn, onHandshakeSuccessFn, onCheckSuccessFn, enabledTime, w.initialHandshake)
w.muEnabled.Lock()
w.enabled = false
w.muEnabled.Unlock()
}
// Reset signals the watcher that the WireGuard peer has been reset and a new
@@ -89,7 +106,6 @@ func (w *WGWatcher) periodicHandshakeCheck(ctx context.Context, onDisconnectedFn
case <-timer.C:
handshake, ok := w.handshakeCheck(lastHandshake)
if !ok {
// early ctx cancel check return
if ctx.Err() != nil {
return
}
@@ -138,9 +154,9 @@ func (w *WGWatcher) handshakeCheck(lastHandshake time.Time) (*time.Time, bool) {
w.log.Tracef("previous handshake, handshake: %v, %v", lastHandshake, handshake)
// the current known handshake did not change
// the current know handshake did not change
if handshake.Equal(lastHandshake) {
w.log.Warnf("WireGuard handshake not updated: %v", handshake)
w.log.Warnf("WireGuard handshake timed out: %v", handshake)
return nil, false
}

View File

@@ -1,4 +1,4 @@
package peer
package wg_watcher
import (
"context"
@@ -7,8 +7,11 @@ import (
"time"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/iface/configurer"
"github.com/netbirdio/netbird/client/internal/peer/state_dump"
"github.com/netbirdio/netbird/client/internal/peer/status"
)
type MocWgIface struct {
@@ -56,23 +59,27 @@ func TestWGWatcher_CheckSuccessCallback(t *testing.T) {
// platforms with coarse clock resolution (Windows), where two time.Now() calls
// microseconds apart can return the same instant and read as a timed-out handshake.
stats := &mockHandshakeStats{handshake: time.Now().Add(-time.Hour)}
watcher := NewWGWatcher(mlog, stats, "", newStateDump("peer", mlog, &Status{}))
watcher := NewWGWatcher(mlog, stats, "", state_dump.NewStateDump("peer", mlog, &status.Recorder{}))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
watcher.PrepareInitialHandshake()
require.True(t, watcher.PrepareInitialHandshake())
firstHandshake := make(chan struct{}, 1)
checkSuccess := make(chan struct{}, 1)
go watcher.EnableWgWatcher(ctx, time.Now(), func() {}, func(when time.Time) {
firstHandshake <- struct{}{}
}, func() {
select {
case checkSuccess <- struct{}{}:
default:
}
})
watcherDone := make(chan struct{})
go func() {
defer close(watcherDone)
watcher.EnableWgWatcher(ctx, time.Now(), func() {}, func(when time.Time) {
firstHandshake <- struct{}{}
}, func() {
select {
case checkSuccess <- struct{}{}:
default:
}
})
}()
stats.advance()
@@ -87,6 +94,11 @@ func TestWGWatcher_CheckSuccessCallback(t *testing.T) {
t.Errorf("first-handshake callback must not fire for a non-zero baseline")
default:
}
// Wait for the watcher goroutine to exit so it cannot race with other
// tests mutating the package-level check timing variables.
cancel()
<-watcherDone
}
func TestWGWatcher_EnableWgWatcher(t *testing.T) {
@@ -95,12 +107,13 @@ func TestWGWatcher_EnableWgWatcher(t *testing.T) {
mlog := log.WithField("peer", "tet")
mocWgIface := &MocWgIface{}
watcher := NewWGWatcher(mlog, mocWgIface, "", newStateDump("peer", mlog, &Status{}))
watcher := NewWGWatcher(mlog, mocWgIface, "", state_dump.NewStateDump("peer", mlog, &status.Recorder{}))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
watcher.PrepareInitialHandshake()
ok := watcher.PrepareInitialHandshake()
require.True(t, ok, "watcher should not be enabled yet")
onDisconnected := make(chan struct{}, 1)
go watcher.EnableWgWatcher(ctx, time.Now(), func() {
@@ -127,10 +140,11 @@ func TestWGWatcher_ReEnable(t *testing.T) {
mlog := log.WithField("peer", "tet")
mocWgIface := &MocWgIface{}
watcher := NewWGWatcher(mlog, mocWgIface, "", newStateDump("peer", mlog, &Status{}))
watcher := NewWGWatcher(mlog, mocWgIface, "", state_dump.NewStateDump("peer", mlog, &status.Recorder{}))
ctx, cancel := context.WithCancel(context.Background())
watcher.PrepareInitialHandshake()
ok := watcher.PrepareInitialHandshake()
require.True(t, ok, "watcher should not be enabled yet")
wg := &sync.WaitGroup{}
wg.Add(1)
@@ -146,7 +160,8 @@ func TestWGWatcher_ReEnable(t *testing.T) {
ctx, cancel = context.WithCancel(context.Background())
defer cancel()
watcher.PrepareInitialHandshake()
ok = watcher.PrepareInitialHandshake()
require.True(t, ok, "watcher should be re-enabled after the previous run stopped")
onDisconnected := make(chan struct{}, 1)
go watcher.EnableWgWatcher(ctx, time.Now(), func() {

View File

@@ -1,4 +1,4 @@
package conntype
package worker
import (
"fmt"

View File

@@ -1,4 +1,4 @@
package peer
package worker
import (
"context"
@@ -13,8 +13,9 @@ import (
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/iface/udpmux"
"github.com/netbirdio/netbird/client/internal/peer/conntype"
icemaker "github.com/netbirdio/netbird/client/internal/peer/ice"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
"github.com/netbirdio/netbird/client/internal/peer/status"
"github.com/netbirdio/netbird/client/internal/portforward"
"github.com/netbirdio/netbird/client/internal/stdnet"
"github.com/netbirdio/netbird/route"
@@ -32,57 +33,68 @@ type ICEConnInfo struct {
RelayedOnLocal bool
}
type WorkerICE struct {
ctx context.Context
log *log.Entry
config ConnConfig
conn *Conn
signaler *Signaler
iFaceDiscover stdnet.ExternalIFaceDiscover
statusRecorder *Status
hasRelayOnLocally bool
type ICEDependencies struct {
Signaler *signaling.Signaler
IFaceDiscover stdnet.ExternalIFaceDiscover
StatusRecorder *status.Recorder
PortForwardManager *portforward.Manager
}
type ICE struct {
log *log.Entry
key string
iceConfig icemaker.Config
isController bool
onConnReady func(priority ConnPriority, iceConnInfo ICEConnInfo)
onStatusDisconnect func(sessionChanged bool)
signaler *signaling.Signaler
iFaceDiscover stdnet.ExternalIFaceDiscover
statusRecorder *status.Recorder
portForwardManager *portforward.Manager
hasRelayOnLocally bool
agent *icemaker.ThreadSafeAgent
agentDialerCancel context.CancelFunc
agentConnecting bool // while it is true, drop all incoming offers
lastSuccess time.Time // with this avoid the too frequent ICE agent recreation
// connectedAgent is the agent whose connection was last reported ready; guarded by muxAgent
connectedAgent *icemaker.ThreadSafeAgent
// remoteSessionID represents the peer's session identifier from the latest remote offer.
remoteSessionID ICESessionID
remoteSessionID icemaker.SessionID
// sessionID is used to track the current session ID of the ICE agent
// increase by one when disconnecting the agent
// with it the remote peer can discard the already deprecated offer/answer
// Without it the remote peer may recreate a workable ICE connection
sessionID ICESessionID
sessionID icemaker.SessionID
remoteSessionChanged bool
muxAgent sync.Mutex
localUfrag string
localPwd string
// we record the last known state of the ICE agent to avoid duplicate on disconnected events
lastKnownState ice.ConnectionState
// portForwardAttempted tracks if we've already tried port forwarding this session
portForwardAttempted bool
}
func NewWorkerICE(ctx context.Context, log *log.Entry, config ConnConfig, conn *Conn, signaler *Signaler, ifaceDiscover stdnet.ExternalIFaceDiscover, statusRecorder *Status, hasRelayOnLocally bool) (*WorkerICE, error) {
sessionID, err := NewICESessionID()
func NewICE(log *log.Entry, key string, iceConfig icemaker.Config, isController bool, onConnReady func(ConnPriority, ICEConnInfo), onStatusDisconnect func(bool), services ICEDependencies, hasRelayOnLocally bool) (*ICE, error) {
sessionID, err := icemaker.NewSessionID()
if err != nil {
return nil, err
}
w := &WorkerICE{
ctx: ctx,
log: log,
config: config,
conn: conn,
signaler: signaler,
iFaceDiscover: ifaceDiscover,
statusRecorder: statusRecorder,
hasRelayOnLocally: hasRelayOnLocally,
lastKnownState: ice.ConnectionStateDisconnected,
sessionID: sessionID,
w := &ICE{
log: log,
key: key,
iceConfig: iceConfig,
isController: isController,
onConnReady: onConnReady,
onStatusDisconnect: onStatusDisconnect,
signaler: services.Signaler,
iFaceDiscover: services.IFaceDiscover,
statusRecorder: services.StatusRecorder,
portForwardManager: services.PortForwardManager,
hasRelayOnLocally: hasRelayOnLocally,
sessionID: sessionID,
}
localUfrag, localPwd, err := icemaker.GenerateICECredentials()
@@ -94,7 +106,7 @@ func NewWorkerICE(ctx context.Context, log *log.Entry, config ConnConfig, conn *
return w, nil
}
func (w *WorkerICE) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
func (w *ICE) OnNewOffer(ctx context.Context, remoteOfferAnswer *signaling.OfferAnswer) {
w.log.Debugf("OnNewOffer for ICE, serial: %s", remoteOfferAnswer.SessionIDString())
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
@@ -118,7 +130,7 @@ func (w *WorkerICE) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
}
}
sessionID, err := NewICESessionID()
sessionID, err := icemaker.NewSessionID()
if err != nil {
w.log.Errorf("failed to create new session ID: %s", err)
}
@@ -136,8 +148,8 @@ func (w *WorkerICE) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
if remoteOfferAnswer.SessionID != nil {
w.log.Debugf("recreate ICE agent: %s / %s", w.sessionID, *remoteOfferAnswer.SessionID)
}
dialerCtx, dialerCancel := context.WithCancel(w.ctx)
agent, err := w.reCreateAgent(dialerCancel, preferredCandidateTypes)
dialerCtx, dialerCancel := context.WithCancel(ctx)
agent, err := w.reCreateAgent(ctx, dialerCancel, preferredCandidateTypes)
if err != nil {
w.log.Errorf("failed to recreate ICE Agent: %s", err)
return
@@ -151,14 +163,14 @@ func (w *WorkerICE) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
w.remoteSessionID = ""
}
go w.connect(dialerCtx, agent, remoteOfferAnswer)
go w.connect(dialerCtx, dialerCancel, agent, remoteOfferAnswer)
}
// OnRemoteCandidate Handles ICE connection Candidate provided by the remote peer.
func (w *WorkerICE) OnRemoteCandidate(candidate ice.Candidate, haRoutes route.HAMap) {
func (w *ICE) OnRemoteCandidate(candidate ice.Candidate, haRoutes route.HAMap) {
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
w.log.Debugf("OnRemoteCandidate from peer %s -> %s", w.config.Key, candidate.String())
w.log.Debugf("OnRemoteCandidate from peer %s -> %s", w.key, candidate.String())
if w.agent == nil {
w.log.Warnf("ICE Agent is not initialized yet")
return
@@ -185,18 +197,24 @@ func (w *WorkerICE) OnRemoteCandidate(candidate ice.Candidate, haRoutes route.HA
}
}
func (w *WorkerICE) GetLocalUserCredentials() (frag string, pwd string) {
return w.localUfrag, w.localPwd
func (w *ICE) Credentials() signaling.Credentials {
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
return signaling.Credentials{
UFrag: w.localUfrag,
Pwd: w.localPwd,
SessionID: w.sessionID,
}
}
func (w *WorkerICE) InProgress() bool {
func (w *ICE) InProgress() bool {
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
return w.agentConnecting
}
func (w *WorkerICE) Close() {
func (w *ICE) Close() {
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
@@ -212,10 +230,10 @@ func (w *WorkerICE) Close() {
w.agent = nil
}
func (w *WorkerICE) reCreateAgent(dialerCancel context.CancelFunc, candidates []ice.CandidateType) (*icemaker.ThreadSafeAgent, error) {
func (w *ICE) reCreateAgent(ctx context.Context, dialerCancel context.CancelFunc, candidates []ice.CandidateType) (*icemaker.ThreadSafeAgent, error) {
w.portForwardAttempted = false
agent, err := icemaker.NewAgent(w.ctx, w.iFaceDiscover, w.config.ICEConfig, candidates, w.localUfrag, w.localPwd)
agent, err := icemaker.NewAgent(ctx, w.iFaceDiscover, w.iceConfig, candidates, w.localUfrag, w.localPwd)
if err != nil {
return nil, fmt.Errorf("create agent: %w", err)
}
@@ -237,7 +255,7 @@ func (w *WorkerICE) reCreateAgent(dialerCancel context.CancelFunc, candidates []
return agent, nil
}
func (w *WorkerICE) SessionID() ICESessionID {
func (w *ICE) getSessionID() icemaker.SessionID {
w.muxAgent.Lock()
defer w.muxAgent.Unlock()
@@ -247,11 +265,11 @@ func (w *WorkerICE) SessionID() ICESessionID {
// will block until connection succeeded
// but it won't release if ICE Agent went into Disconnected or Failed state,
// so we have to cancel it with the provided context once agent detected a broken connection
func (w *WorkerICE) connect(ctx context.Context, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *OfferAnswer) {
func (w *ICE) connect(ctx context.Context, dialerCancel context.CancelFunc, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *signaling.OfferAnswer) {
w.log.Debugf("gather candidates")
if err := agent.GatherCandidates(); err != nil {
w.log.Warnf("failed to gather candidates: %s", err)
w.closeAgent(agent, w.agentDialerCancel)
w.closeAgent(agent, dialerCancel)
return
}
@@ -259,19 +277,19 @@ func (w *WorkerICE) connect(ctx context.Context, agent *icemaker.ThreadSafeAgent
remoteConn, err := w.turnAgentDial(ctx, agent, remoteOfferAnswer)
if err != nil {
w.log.Debugf("failed to dial the remote peer: %s", err)
w.closeAgent(agent, w.agentDialerCancel)
w.closeAgent(agent, dialerCancel)
return
}
w.log.Debugf("agent dial succeeded")
pair, err := agent.GetSelectedCandidatePair()
if err != nil {
w.closeAgent(agent, w.agentDialerCancel)
w.closeAgent(agent, dialerCancel)
return
}
if pair == nil {
w.log.Warnf("selected candidate pair is nil, cannot proceed")
w.closeAgent(agent, w.agentDialerCancel)
w.closeAgent(agent, dialerCancel)
return
}
@@ -299,17 +317,22 @@ func (w *WorkerICE) connect(ctx context.Context, agent *icemaker.ThreadSafeAgent
}
w.log.Debugf("on ICE conn is ready to use")
w.log.Infof("connection succeeded with offer session: %s", remoteOfferAnswer.SessionIDString())
w.muxAgent.Lock()
if w.agent != agent {
w.muxAgent.Unlock()
w.log.Debugf("agent has been replaced during connect, dropping obsolete connection")
return
}
w.agentConnecting = false
w.lastSuccess = time.Now()
w.connectedAgent = agent
w.muxAgent.Unlock()
// todo: the potential problem is a race between the onConnectionStateChange
w.conn.onICEConnectionIsReady(selectedPriority(pair), ci)
w.log.Infof("connection succeeded with offer session: %s", remoteOfferAnswer.SessionIDString())
w.onConnReady(selectedPriority(pair), ci)
}
func (w *WorkerICE) closeAgent(agent *icemaker.ThreadSafeAgent, cancel context.CancelFunc) bool {
func (w *ICE) closeAgent(agent *icemaker.ThreadSafeAgent, cancel context.CancelFunc) bool {
cancel()
if err := agent.Close(); err != nil {
w.log.Warnf("failed to close ICE agent: %s", err)
@@ -323,7 +346,7 @@ func (w *WorkerICE) closeAgent(agent *icemaker.ThreadSafeAgent, cancel context.C
if w.agent == agent {
// consider to remove from here and move to the OnNewOffer
sessionID, err := NewICESessionID()
sessionID, err := icemaker.NewSessionID()
if err != nil {
w.log.Errorf("failed to create new session ID: %s", err)
}
@@ -335,7 +358,7 @@ func (w *WorkerICE) closeAgent(agent *icemaker.ThreadSafeAgent, cancel context.C
return sessionChanged
}
func (w *WorkerICE) punchRemoteWGPort(pair *ice.CandidatePair, remoteWgPort int) {
func (w *ICE) punchRemoteWGPort(pair *ice.CandidatePair, remoteWgPort int) {
// wait local endpoint configuration
time.Sleep(time.Second)
addr, err := net.ResolveUDPAddr("udp", net.JoinHostPort(pair.Remote.Address(), strconv.Itoa(remoteWgPort)))
@@ -344,7 +367,7 @@ func (w *WorkerICE) punchRemoteWGPort(pair *ice.CandidatePair, remoteWgPort int)
return
}
mux, ok := w.config.ICEConfig.UDPMuxSrflx.(*udpmux.UniversalUDPMuxDefault)
mux, ok := w.iceConfig.UDPMuxSrflx.(*udpmux.UniversalUDPMuxDefault)
if !ok {
w.log.Warn("invalid udp mux conversion")
return
@@ -357,7 +380,7 @@ func (w *WorkerICE) punchRemoteWGPort(pair *ice.CandidatePair, remoteWgPort int)
// onICECandidate is a callback attached to an ICE Agent to receive new local connection candidates
// and then signals them to the remote peer
func (w *WorkerICE) onICECandidate(candidate ice.Candidate) {
func (w *ICE) onICECandidate(candidate ice.Candidate) {
// nil means candidate gathering has been ended
if candidate == nil {
return
@@ -366,9 +389,9 @@ func (w *WorkerICE) onICECandidate(candidate ice.Candidate) {
// TODO: reported port is incorrect for CandidateTypeHost, makes understanding ICE use via logs confusing as port is ignored
w.log.Debugf("discovered local candidate %s", candidate.String())
go func() {
err := w.signaler.SignalICECandidate(candidate, w.config.Key)
err := w.signaler.SignalICECandidate(candidate, w.key)
if err != nil {
w.log.Errorf("failed signaling candidate to the remote peer %s %s", w.config.Key, err)
w.log.Errorf("failed signaling candidate to the remote peer %s %s", w.key, err)
}
}()
@@ -378,8 +401,8 @@ func (w *WorkerICE) onICECandidate(candidate ice.Candidate) {
}
// injectPortForwardedCandidate signals an additional candidate using the pre-created port mapping.
func (w *WorkerICE) injectPortForwardedCandidate(srflxCandidate ice.Candidate) {
pfManager := w.conn.portForwardManager
func (w *ICE) injectPortForwardedCandidate(srflxCandidate ice.Candidate) {
pfManager := w.portForwardManager
if pfManager == nil {
return
}
@@ -407,7 +430,7 @@ func (w *WorkerICE) injectPortForwardedCandidate(srflxCandidate ice.Candidate) {
forwardedCandidate.String(), mapping.InternalPort, mapping.ExternalPort, mapping.NATType, forwardedCandidate.Priority())
go func() {
if err := w.signaler.SignalICECandidate(forwardedCandidate, w.config.Key); err != nil {
if err := w.signaler.SignalICECandidate(forwardedCandidate, w.key); err != nil {
w.log.Errorf("signal port-forwarded candidate: %v", err)
}
}()
@@ -415,7 +438,7 @@ func (w *WorkerICE) injectPortForwardedCandidate(srflxCandidate ice.Candidate) {
// createForwardedCandidate creates a new server reflexive candidate with the forwarded port.
// It uses the NAT gateway's external IP with the forwarded port.
func (w *WorkerICE) createForwardedCandidate(srflxCandidate ice.Candidate, mapping *portforward.Mapping) (ice.Candidate, error) {
func (w *ICE) createForwardedCandidate(srflxCandidate ice.Candidate, mapping *portforward.Mapping) (ice.Candidate, error) {
var externalIP string
if mapping.ExternalIP != nil && !mapping.ExternalIP.IsUnspecified() {
externalIP = mapping.ExternalIP.String()
@@ -460,9 +483,9 @@ func (w *WorkerICE) createForwardedCandidate(srflxCandidate ice.Candidate, mappi
return candidate, nil
}
func (w *WorkerICE) onICESelectedCandidatePair(agent *icemaker.ThreadSafeAgent, c1, c2 ice.Candidate) {
func (w *ICE) onICESelectedCandidatePair(agent *icemaker.ThreadSafeAgent, c1, c2 ice.Candidate) {
w.log.Debugf("selected candidate pair [local <-> remote] -> [%s <-> %s], peer %s", c1.String(), c2.String(),
w.config.Key)
w.key)
pairStat, ok := agent.GetSelectedCandidatePairStats()
if !ok {
@@ -471,14 +494,14 @@ func (w *WorkerICE) onICESelectedCandidatePair(agent *icemaker.ThreadSafeAgent,
}
duration := time.Duration(pairStat.CurrentRoundTripTime * float64(time.Second))
if err := w.statusRecorder.UpdateLatency(w.config.Key, duration); err != nil {
if err := w.statusRecorder.UpdateLatency(w.key, duration); err != nil {
w.log.Debugf("failed to update latency for peer: %s", err)
return
}
}
func (w *WorkerICE) logSuccessfulPaths(agent *icemaker.ThreadSafeAgent) {
sessionID := w.SessionID()
func (w *ICE) logSuccessfulPaths(agent *icemaker.ThreadSafeAgent) {
sessionID := w.getSessionID()
stats := agent.GetCandidatePairsStats()
localCandidates, _ := agent.GetLocalCandidates()
remoteCandidates, _ := agent.GetRemoteCandidates()
@@ -508,32 +531,44 @@ func (w *WorkerICE) logSuccessfulPaths(agent *icemaker.ThreadSafeAgent) {
}
}
func (w *WorkerICE) onConnectionStateChange(agent *icemaker.ThreadSafeAgent, dialerCancel context.CancelFunc) func(ice.ConnectionState) {
func (w *ICE) onConnectionStateChange(agent *icemaker.ThreadSafeAgent, dialerCancel context.CancelFunc) func(ice.ConnectionState) {
// per-agent state; pion delivers callbacks of one agent sequentially
var connected bool
return func(state ice.ConnectionState) {
w.log.Debugf("ICE ConnectionState has changed to %s", state.String())
switch state {
case ice.ConnectionStateConnected:
w.lastKnownState = ice.ConnectionStateConnected
connected = true
w.logSuccessfulPaths(agent)
return
case ice.ConnectionStateFailed, ice.ConnectionStateDisconnected, ice.ConnectionStateClosed:
// ice.ConnectionStateClosed happens when we recreate the agent. For the P2P to TURN switch important to
// notify the conn.onICEStateDisconnected changes to update the current used priority
sessionChanged := w.closeAgent(agent, dialerCancel)
if w.lastKnownState == ice.ConnectionStateConnected {
w.lastKnownState = ice.ConnectionStateDisconnected
w.conn.onICEStateDisconnected(sessionChanged)
if !connected {
return
}
default:
return
connected = false
w.muxAgent.Lock()
stale := w.connectedAgent != agent
if !stale {
w.connectedAgent = nil
}
w.muxAgent.Unlock()
if stale {
w.log.Debugf("suppress disconnected event of replaced ICE agent")
return
}
w.onStatusDisconnect(sessionChanged)
}
}
}
func (w *WorkerICE) turnAgentDial(ctx context.Context, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *OfferAnswer) (*ice.Conn, error) {
if isController(w.config) {
func (w *ICE) turnAgentDial(ctx context.Context, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *signaling.OfferAnswer) (*ice.Conn, error) {
if w.isController {
return agent.Dial(ctx, remoteOfferAnswer.IceCredentials.UFrag, remoteOfferAnswer.IceCredentials.Pwd)
} else {
return agent.Accept(ctx, remoteOfferAnswer.IceCredentials.UFrag, remoteOfferAnswer.IceCredentials.Pwd)
@@ -595,10 +630,10 @@ func isRelayed(pair *ice.CandidatePair) bool {
return false
}
func selectedPriority(pair *ice.CandidatePair) conntype.ConnPriority {
func selectedPriority(pair *ice.CandidatePair) ConnPriority {
if isRelayed(pair) {
return conntype.ICETurn
return ICETurn
} else {
return conntype.ICEP2P
return ICEP2P
}
}

View File

@@ -1,4 +1,4 @@
package peer
package worker
import (
"context"
@@ -10,22 +10,23 @@ import (
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/peer/signaling"
relayClient "github.com/netbirdio/netbird/shared/relay/client"
)
type RelayConnInfo struct {
relayedConn net.Conn
rosenpassPubKey []byte
rosenpassAddr string
RelayedConn net.Conn
RosenpassPubKey []byte
RosenpassAddr string
}
type WorkerRelay struct {
peerCtx context.Context
log *log.Entry
isController bool
config ConnConfig
conn *Conn
relayManager *relayClient.Manager
log *log.Entry
key string
isController bool
onConnReady func(RelayConnInfo)
onDisconnected func()
relayManager *relayClient.Manager
relayedConn net.Conn
relayLock sync.Mutex
@@ -33,19 +34,19 @@ type WorkerRelay struct {
relaySupportedOnRemotePeer atomic.Bool
}
func NewWorkerRelay(ctx context.Context, log *log.Entry, ctrl bool, config ConnConfig, conn *Conn, relayManager *relayClient.Manager) *WorkerRelay {
func NewWorkerRelay(log *log.Entry, key string, isController bool, onConnReady func(RelayConnInfo), onDisconnected func(), relayManager *relayClient.Manager) *WorkerRelay {
r := &WorkerRelay{
peerCtx: ctx,
log: log,
isController: ctrl,
config: config,
conn: conn,
relayManager: relayManager,
log: log,
key: key,
isController: isController,
onConnReady: onConnReady,
onDisconnected: onDisconnected,
relayManager: relayManager,
}
return r
}
func (w *WorkerRelay) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
func (w *WorkerRelay) OnNewOffer(ctx context.Context, remoteOfferAnswer *signaling.OfferAnswer) {
if !w.isRelaySupported(remoteOfferAnswer) {
w.log.Infof("Relay is not supported by remote peer")
w.relaySupportedOnRemotePeer.Store(false)
@@ -66,7 +67,7 @@ func (w *WorkerRelay) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
serverIP = remoteOfferAnswer.RelaySrvIP
}
relayedConn, err := w.relayManager.OpenConn(w.peerCtx, srv, w.config.Key, serverIP)
relayedConn, err := w.relayManager.OpenConn(ctx, srv, w.key, serverIP)
if err != nil {
if errors.Is(err, relayClient.ErrConnAlreadyExists) {
w.log.Debugf("handled offer by reusing existing relay connection")
@@ -88,10 +89,10 @@ func (w *WorkerRelay) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
}
w.log.Debugf("peer conn opened via Relay: %s", srv)
go w.conn.onRelayConnectionIsReady(RelayConnInfo{
relayedConn: relayedConn,
rosenpassPubKey: remoteOfferAnswer.RosenpassPubKey,
rosenpassAddr: remoteOfferAnswer.RosenpassAddr,
w.onConnReady(RelayConnInfo{
RelayedConn: relayedConn,
RosenpassPubKey: remoteOfferAnswer.RosenpassPubKey,
RosenpassAddr: remoteOfferAnswer.RosenpassAddr,
})
}
@@ -119,7 +120,7 @@ func (w *WorkerRelay) CloseConn() {
}
}
func (w *WorkerRelay) isRelaySupported(answer *OfferAnswer) bool {
func (w *WorkerRelay) isRelaySupported(answer *signaling.OfferAnswer) bool {
if !w.relayManager.HasRelayAddress() {
return false
}
@@ -134,5 +135,5 @@ func (w *WorkerRelay) preferredRelayServer(myRelayAddress, remoteRelayAddress st
}
func (w *WorkerRelay) onRelayClientDisconnected() {
go w.conn.onRelayDisconnected()
w.onDisconnected()
}

View File

@@ -1,4 +1,4 @@
package worker
package peer
import (
"sync/atomic"
@@ -7,17 +7,17 @@ import (
)
const (
StatusDisconnected Status = iota
StatusConnected
WorkerStatusDisconnected WorkerStatus = iota
WorkerStatusConnected
)
type Status int32
type WorkerStatus int32
func (s Status) String() string {
func (s WorkerStatus) String() string {
switch s {
case StatusDisconnected:
case WorkerStatusDisconnected:
return "Disconnected"
case StatusConnected:
case WorkerStatusConnected:
return "Connected"
default:
log.Errorf("unknown status: %d", s)
@@ -37,16 +37,16 @@ func NewAtomicStatus() *AtomicWorkerStatus {
}
// Get returns the current connection status
func (acs *AtomicWorkerStatus) Get() Status {
return Status(acs.status.Load())
func (acs *AtomicWorkerStatus) Get() WorkerStatus {
return WorkerStatus(acs.status.Load())
}
func (acs *AtomicWorkerStatus) SetConnected() {
acs.status.Store(int32(StatusConnected))
acs.status.Store(int32(WorkerStatusConnected))
}
func (acs *AtomicWorkerStatus) SetDisconnected() {
acs.status.Store(int32(StatusDisconnected))
acs.status.Store(int32(WorkerStatusDisconnected))
}
// String returns the string representation of the current status

View File

@@ -95,7 +95,7 @@ func (d *DnsInterceptor) RemoveRoute() error {
// AllowedIPs should use real IPs
if d.currentPeerKey != "" {
if _, err := d.allowedIPsRefcounter.Decrement(prefix, d.currentPeerKey); err != nil {
if _, err := d.allowedIPsRefcounter.Decrement(prefix); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove allowed IP %s: %v", prefix, err))
}
}
@@ -172,7 +172,7 @@ func (d *DnsInterceptor) removeAllowedIP(realPrefix netip.Prefix) error {
}
// AllowedIPs use real IPs
if _, err := d.allowedIPsRefcounter.Decrement(realPrefix, d.currentPeerKey); err != nil {
if _, err := d.allowedIPsRefcounter.Decrement(realPrefix); err != nil {
return fmt.Errorf("remove allowed IP %s: %v", realPrefix, err)
}
@@ -205,7 +205,7 @@ func (d *DnsInterceptor) RemoveAllowedIPs() error {
for _, prefixes := range d.interceptedDomains {
for _, prefix := range prefixes {
// AllowedIPs use real IPs
if _, err := d.allowedIPsRefcounter.Decrement(prefix, d.currentPeerKey); err != nil {
if _, err := d.allowedIPsRefcounter.Decrement(prefix); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove allowed IP %s: %v", prefix, err))
}
}

View File

@@ -135,7 +135,7 @@ func (r *Route) RemoveAllowedIPs() error {
var merr *multierror.Error
for _, domainPrefixes := range r.dynamicDomains {
for _, prefix := range domainPrefixes {
if _, err := r.allowedIPsRefcounter.Decrement(prefix, r.currentPeerKey); err != nil {
if _, err := r.allowedIPsRefcounter.Decrement(prefix); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove allowed IP %s: %w", prefix, err))
}
}
@@ -320,7 +320,7 @@ func (r *Route) removeRoutes(prefixes []netip.Prefix) ([]netip.Prefix, error) {
merr = multierror.Append(merr, fmt.Errorf("remove dynamic route for IP %s: %w", prefix, err))
}
if r.currentPeerKey != "" {
if _, err := r.allowedIPsRefcounter.Decrement(prefix, r.currentPeerKey); err != nil {
if _, err := r.allowedIPsRefcounter.Decrement(prefix); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove allowed IP %s: %w", prefix, err))
}
}

View File

@@ -215,7 +215,7 @@ func (m *DefaultManager) setupRefCounters(useNoop bool) {
)
}
m.allowedIPsRefCounter = refcounter.NewAllowedIPs(
m.allowedIPsRefCounter = refcounter.New(
func(prefix netip.Prefix, peerKey string) (string, error) {
// save peerKey to use it in the remove function
return peerKey, m.wgInterface.AddAllowedIP(peerKey, prefix)

View File

@@ -1,185 +0,0 @@
package refcounter
import (
"errors"
"fmt"
"net/netip"
"sort"
"sync"
"github.com/hashicorp/go-multierror"
nberrors "github.com/netbirdio/netbird/client/errors"
)
// allowedIPsEntry holds the per-peer reference counts for a single prefix and which peer is
// currently installed in WireGuard. WireGuard allows a prefix on exactly one peer, so at most
// one peer is active at a time even when several peers reference the prefix.
type allowedIPsEntry struct {
// peers maps a peerKey to the number of references holding the prefix for that peer.
peers map[string]int
// active is the peerKey currently installed in WireGuard for this prefix ("" if none).
active string
// total is the sum of all per-peer reference counts (kept in sync with peers).
total int
}
// AllowedIPsRefCounter is a peer-aware reference counter for WireGuard AllowedIPs.
//
// The generic Counter keys only by prefix and remembers a single Out value set by the first
// caller, which it never changes. That is wrong for AllowedIPs: two independent watchers (or
// multiple resolved domains) can reference the same prefix through different peers, and when the
// peer currently installed in WireGuard releases its last reference the prefix must be handed over
// to a surviving peer instead of being left pointing at the released one.
//
// It calls add/remove (which program WireGuard) only on the transitions that matter:
// - add on the first reference for a prefix, or when swapping the active peer;
// - remove on the last reference for a prefix, or on the old peer during a swap.
type AllowedIPsRefCounter struct {
mu sync.Mutex
entries map[netip.Prefix]*allowedIPsEntry
add AddFunc[netip.Prefix, string, string]
remove RemoveFunc[netip.Prefix, string]
}
// NewAllowedIPs creates a new peer-aware AllowedIPs reference counter.
// add programs a prefix on a peer in WireGuard and returns the peerKey to store as the active peer.
// remove unprograms the prefix from the given peer.
func NewAllowedIPs(add AddFunc[netip.Prefix, string, string], remove RemoveFunc[netip.Prefix, string]) *AllowedIPsRefCounter {
return &AllowedIPsRefCounter{
entries: map[netip.Prefix]*allowedIPsEntry{},
add: add,
remove: remove,
}
}
// Increment adds a reference to prefix for peerKey. WireGuard is programmed only for the first
// reference to a prefix; while a different peer is already installed the prefix is left with it
// (first peer wins, HA at the WireGuard layer is not possible) and only the reference count is kept.
func (rm *AllowedIPsRefCounter) Increment(prefix netip.Prefix, peerKey string) (Ref[string], error) {
rm.mu.Lock()
defer rm.mu.Unlock()
e, ok := rm.entries[prefix]
if !ok {
e = &allowedIPsEntry{peers: map[string]int{}}
rm.entries[prefix] = e
}
logCallerF("Increasing allowed IP ref count for prefix %v peer %s [peer %d -> %d, total %d -> %d, active %q]",
prefix, peerKey, e.peers[peerKey], e.peers[peerKey]+1, e.total, e.total+1, e.active)
// Program WireGuard only when nothing is installed yet for this prefix.
if e.active == "" {
out, err := rm.add(prefix, peerKey)
if errors.Is(err, ErrIgnore) {
if e.total == 0 {
delete(rm.entries, prefix)
}
return Ref[string]{Count: e.total, Out: e.active}, nil
}
if err != nil {
if e.total == 0 {
delete(rm.entries, prefix)
}
return Ref[string]{}, fmt.Errorf("failed to add allowed IP %v for peer %s: %w", prefix, peerKey, err)
}
e.active = out
}
e.peers[peerKey]++
e.total++
return Ref[string]{Count: e.total, Out: e.active}, nil
}
// Decrement removes a reference to prefix for peerKey. When the peer currently installed in
// WireGuard releases its last reference, the prefix is swapped to a surviving peer if one exists,
// otherwise it is removed from WireGuard.
func (rm *AllowedIPsRefCounter) Decrement(prefix netip.Prefix, peerKey string) (Ref[string], error) {
rm.mu.Lock()
defer rm.mu.Unlock()
e, ok := rm.entries[prefix]
if !ok {
logCallerF("No allowed IP reference found for prefix %v", prefix)
return Ref[string]{}, nil
}
if e.peers[peerKey] > 0 {
logCallerF("Decreasing allowed IP ref count for prefix %v peer %s [peer %d -> %d, total %d -> %d, active %q]",
prefix, peerKey, e.peers[peerKey], e.peers[peerKey]-1, e.total, e.total-1, e.active)
e.peers[peerKey]--
e.total--
if e.peers[peerKey] == 0 {
delete(e.peers, peerKey)
}
} else {
logCallerF("No allowed IP reference found for prefix %v peer %s", prefix, peerKey)
}
// If the peer currently installed in WireGuard still holds references, nothing to reprogram.
// Keying the check on the active peer (not the one just released) makes this self-healing:
// a prior swap whose remove/add failed leaves e.active pointing at a peer with no references,
// and this retries the hand-off on the next Decrement instead of getting stuck.
if e.active != "" && e.peers[e.active] > 0 {
return Ref[string]{Count: e.total, Out: e.active}, nil
}
// Detach the stale/gone active peer from WireGuard before reprogramming.
if e.active != "" {
if err := rm.remove(prefix, e.active); err != nil {
return Ref[string]{Count: e.total, Out: e.active}, fmt.Errorf("remove allowed IP %v for peer %s: %w", prefix, e.active, err)
}
e.active = ""
}
// Hand the prefix over to a surviving peer, or drop the entry when none remain.
if survivor, ok := pickSurvivor(e.peers); ok {
out, err := rm.add(prefix, survivor)
if err != nil {
return Ref[string]{Count: e.total, Out: ""}, fmt.Errorf("swap allowed IP %v to peer %s: %w", prefix, survivor, err)
}
e.active = out
return Ref[string]{Count: e.total, Out: e.active}, nil
}
delete(rm.entries, prefix)
return Ref[string]{Count: 0, Out: ""}, nil
}
// Flush removes all prefixes from WireGuard and clears the counter.
func (rm *AllowedIPsRefCounter) Flush() error {
rm.mu.Lock()
defer rm.mu.Unlock()
var merr *multierror.Error
for prefix, e := range rm.entries {
if e.active == "" {
continue
}
logCallerF("Flushing allowed IP for prefix %v peer %s", prefix, e.active)
if err := rm.remove(prefix, e.active); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove allowed IP %v for peer %s: %w", prefix, e.active, err))
}
}
clear(rm.entries)
return nberrors.FormatErrorOrNil(merr)
}
// pickSurvivor deterministically selects a peer still referencing the prefix. WireGuard cannot do
// multipath for a single prefix, so any surviving peer is a valid winner; the choice is made stable
// (lowest peerKey) for predictable behavior and testability.
func pickSurvivor(peers map[string]int) (string, bool) {
if len(peers) == 0 {
return "", false
}
keys := make([]string, 0, len(peers))
for k := range peers {
keys = append(keys, k)
}
sort.Strings(keys)
return keys[0], true
}

View File

@@ -1,241 +0,0 @@
package refcounter
import (
"errors"
"net/netip"
"testing"
)
// fakeWG models WireGuard's cryptokey routing: a prefix can be installed on exactly one peer.
// failAdd/failRemove make the next add/remove fail once, to exercise the self-healing error paths.
type fakeWG struct {
installed map[netip.Prefix]string
adds int
removes int
failAdd bool
failRemove bool
}
func newFakeWG() *fakeWG {
return &fakeWG{installed: map[netip.Prefix]string{}}
}
func (f *fakeWG) counter() *AllowedIPsRefCounter {
return NewAllowedIPs(
func(prefix netip.Prefix, peerKey string) (string, error) {
if f.failAdd {
f.failAdd = false
return "", errors.New("add failed")
}
f.adds++
f.installed[prefix] = peerKey
return peerKey, nil
},
func(prefix netip.Prefix, peerKey string) error {
if f.failRemove {
f.failRemove = false
return errors.New("remove failed")
}
f.removes++
// only clear if this peer is the one installed, mirroring wg semantics
if f.installed[prefix] == peerKey {
delete(f.installed, prefix)
}
return nil
},
)
}
func mustPrefix(t *testing.T, s string) netip.Prefix {
t.Helper()
p, err := netip.ParsePrefix(s)
if err != nil {
t.Fatalf("parse prefix %q: %v", s, err)
}
return p
}
func mustIncrement(t *testing.T, c *AllowedIPsRefCounter, p netip.Prefix, peer string) Ref[string] {
t.Helper()
ref, err := c.Increment(p, peer)
if err != nil {
t.Fatalf("Increment(%v, %s): %v", p, peer, err)
}
return ref
}
func mustDecrement(t *testing.T, c *AllowedIPsRefCounter, p netip.Prefix, peer string) Ref[string] {
t.Helper()
ref, err := c.Decrement(p, peer)
if err != nil {
t.Fatalf("Decrement(%v, %s): %v", p, peer, err)
}
return ref
}
// TestAllowedIPs_SwapOnActivePeerRemoval reproduces the reported bug: two networks with the same
// prefix routed by different peers. Removing the network whose peer is installed must hand the
// prefix over to the surviving peer instead of leaving it on the removed one.
func TestAllowedIPs_SwapOnActivePeerRemoval(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
mustIncrement(t, c, p, "peerA")
mustIncrement(t, c, p, "peerB")
// First peer wins while both are present.
if got := f.installed[p]; got != "peerA" {
t.Fatalf("expected peerA installed, got %q", got)
}
// Remove the active peer's network -> must swap to peerB.
mustDecrement(t, c, p, "peerA")
if got := f.installed[p]; got != "peerB" {
t.Fatalf("BUG: prefix stuck on removed peer, want peerB got %q", got)
}
// Remove the last one -> prefix gone.
mustDecrement(t, c, p, "peerB")
if _, ok := f.installed[p]; ok {
t.Fatalf("expected prefix removed, still installed on %q", f.installed[p])
}
}
// TestAllowedIPs_RemoveNonActivePeer removing a non-installed peer must not touch WireGuard.
func TestAllowedIPs_RemoveNonActivePeer(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
mustIncrement(t, c, p, "peerA")
mustIncrement(t, c, p, "peerB")
removesBefore := f.removes
mustDecrement(t, c, p, "peerB")
if f.installed[p] != "peerA" {
t.Fatalf("active peer must stay peerA, got %q", f.installed[p])
}
if f.removes != removesBefore {
t.Fatalf("removing a non-active peer must not call wg remove")
}
}
// TestAllowedIPs_SamePeerMultipleRefs two references via the same peer must keep the prefix until
// the last reference is released (the reason the per-peer count must be an int, not a set).
func TestAllowedIPs_SamePeerMultipleRefs(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
mustIncrement(t, c, p, "peerA")
mustIncrement(t, c, p, "peerA")
if f.adds != 1 {
t.Fatalf("expected a single wg add for the same peer, got %d", f.adds)
}
mustDecrement(t, c, p, "peerA")
if f.installed[p] != "peerA" {
t.Fatalf("prefix must stay while a reference remains, got %q", f.installed[p])
}
if f.removes != 0 {
t.Fatalf("no wg remove expected while a reference remains, got %d", f.removes)
}
mustDecrement(t, c, p, "peerA")
if _, ok := f.installed[p]; ok {
t.Fatalf("prefix must be removed after last reference")
}
}
// TestAllowedIPs_RefCountAndActive checks the Ref returned to callers (used for the HA-disabled log).
func TestAllowedIPs_RefCountAndActive(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
ref := mustIncrement(t, c, p, "peerA")
if ref.Count != 1 || ref.Out != "peerA" {
t.Fatalf("want {1, peerA}, got {%d, %q}", ref.Count, ref.Out)
}
ref = mustIncrement(t, c, p, "peerB")
if ref.Count != 2 || ref.Out != "peerA" {
t.Fatalf("want {2, peerA}, got {%d, %q}", ref.Count, ref.Out)
}
}
// TestAllowedIPs_Flush removes everything installed and clears the counter.
func TestAllowedIPs_Flush(t *testing.T) {
f := newFakeWG()
c := f.counter()
p1 := mustPrefix(t, "10.44.8.0/24")
p2 := mustPrefix(t, "10.44.9.0/24")
mustIncrement(t, c, p1, "peerA")
mustIncrement(t, c, p2, "peerB")
if err := c.Flush(); err != nil {
t.Fatal(err)
}
if len(f.installed) != 0 {
t.Fatalf("expected all prefixes removed, got %v", f.installed)
}
// After flush, a fresh increment must add again.
mustIncrement(t, c, p1, "peerC")
if f.installed[p1] != "peerC" {
t.Fatalf("counter not reset after flush")
}
}
// TestAllowedIPs_SelfHealAfterSwapAddError ensures a failed add during a swap does not permanently
// strand the prefix: the next Decrement (or Increment) must retry and install a surviving peer.
func TestAllowedIPs_SelfHealAfterSwapAddError(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
mustIncrement(t, c, p, "peerA")
mustIncrement(t, c, p, "peerB")
mustIncrement(t, c, p, "peerC")
// Removing the active peerA triggers a swap to a survivor; make the add fail once.
f.failAdd = true
if _, err := c.Decrement(p, "peerA"); err == nil {
t.Fatalf("expected error from failed swap add")
}
if _, ok := f.installed[p]; ok {
t.Fatalf("nothing should be installed after a failed swap add, got %q", f.installed[p])
}
// A later Decrement of a non-active survivor must retry the hand-off (self-heal), not stay stuck.
ref := mustDecrement(t, c, p, "peerC")
if got := f.installed[p]; got == "" {
t.Fatalf("self-heal failed: prefix left unrouted after add recovered")
}
if ref.Out == "" {
t.Fatalf("expected an active peer after self-heal, got empty")
}
}
// TestAllowedIPs_SelfHealAfterRemoveError ensures a failed remove during a swap is retried instead
// of leaving e.active stuck on a peer that no longer holds references.
func TestAllowedIPs_SelfHealAfterRemoveError(t *testing.T) {
f := newFakeWG()
c := f.counter()
p := mustPrefix(t, "10.44.8.0/24")
mustIncrement(t, c, p, "peerA")
mustIncrement(t, c, p, "peerB")
// Releasing active peerA must detach it (remove) then add peerB; fail the remove once.
f.failRemove = true
if _, err := c.Decrement(p, "peerA"); err == nil {
t.Fatalf("expected error from failed remove")
}
// Next Decrement of the non-active survivor retries: removes stale peerA, installs peerB.
mustDecrement(t, c, p, "peerB")
// peerB had only one ref, so after retry the prefix is fully released.
if _, ok := f.installed[p]; ok {
t.Fatalf("expected prefix released after self-heal, still on %q", f.installed[p])
}
}

View File

@@ -5,7 +5,5 @@ import "net/netip"
// RouteRefCounter is a Counter for Route, it doesn't take any input on Increment and doesn't use any output on Decrement
type RouteRefCounter = Counter[netip.Prefix, struct{}, struct{}]
// AllowedIPsRefCounter tracks WireGuard AllowedIPs per prefix. Unlike the generic Counter it is peer-aware:
// a prefix can be claimed by several peers at once and WireGuard allows a given prefix on exactly one peer,
// so the counter records the per-peer reference count and swaps the installed peer when the active one is released.
// See allowedips.go.
// AllowedIPsRefCounter is a Counter for AllowedIPs, it takes a peer key on Increment and passes it back to Decrement
type AllowedIPsRefCounter = Counter[netip.Prefix, string, string]

View File

@@ -15,11 +15,6 @@ type Route struct {
route *route.Route
routeRefCounter *refcounter.RouteRefCounter
allowedIPsRefcounter *refcounter.AllowedIPsRefCounter
// currentPeerKey is the routing peer this watcher currently has the prefix installed on
// (the HA winner elected by the watcher). It can differ from route.Peer and change on
// failover, so it is recorded on AddAllowedIPs and used on RemoveAllowedIPs to decrement
// the exact peer that was incremented.
currentPeerKey string
}
func NewRoute(params common.HandlerParams) *Route {
@@ -57,15 +52,12 @@ func (r *Route) AddAllowedIPs(peerKey string) error {
ref.Out,
)
}
r.currentPeerKey = peerKey
return nil
}
func (r *Route) RemoveAllowedIPs() error {
var err error
if _, decErr := r.allowedIPsRefcounter.Decrement(r.route.Network, r.currentPeerKey); decErr != nil {
err = fmt.Errorf("remove allowed IP %s: %w", r.route.Network, decErr)
if _, err := r.allowedIPsRefcounter.Decrement(r.route.Network); err != nil {
return err
}
r.currentPeerKey = ""
return err
return nil
}

View File

@@ -22,7 +22,6 @@ var allKeys = []string{
KeyDisableMetricsCollection,
KeyAllowServerSSH,
KeyDisableAutoConnect,
KeyDisableAutostart,
KeyPreSharedKey,
KeyRosenpassEnabled,
KeyRosenpassPermissive,

View File

@@ -20,10 +20,10 @@ import (
// names (lowerCamelCase) so the daemon can map a Policy key directly to a
// configuration field.
const (
KeyManagementURL = "managementURL"
KeyDisableUpdateSettings = "disableUpdateSettings"
KeyDisableProfiles = "disableProfiles"
KeyDisableNetworks = "disableNetworks"
KeyManagementURL = "managementURL"
KeyDisableUpdateSettings = "disableUpdateSettings"
KeyDisableProfiles = "disableProfiles"
KeyDisableNetworks = "disableNetworks"
// KeyDisableAdvancedView gates the advanced-view section in the
// upcoming UI revision. UI-only: NOT stored on Config, not
// applied by applyMDMPolicy, not rejectable via SetConfig. The
@@ -37,16 +37,10 @@ const (
KeyDisableMetricsCollection = "disableMetricsCollection"
KeyAllowServerSSH = "allowServerSSH"
KeyDisableAutoConnect = "disableAutoConnect"
// KeyDisableAutostart suppresses the GUI's fresh-install
// launch-on-login default and marks the Settings toggle as
// MDM-managed. UI-only: NOT stored on Config and not applied by
// applyMDMPolicy; the GUI reads it directly and it appears in
// GetConfigResponse.mDMManagedFields when set.
KeyDisableAutostart = "disableAutostart"
KeyPreSharedKey = "preSharedKey"
KeyRosenpassEnabled = "rosenpassEnabled"
KeyRosenpassPermissive = "rosenpassPermissive"
KeyWireguardPort = "wireguardPort"
KeyPreSharedKey = "preSharedKey"
KeyRosenpassEnabled = "rosenpassEnabled"
KeyRosenpassPermissive = "rosenpassPermissive"
KeyWireguardPort = "wireguardPort"
// Split tunnel is modeled as a single conceptual policy with two
// registry/plist values. KeySplitTunnelMode is the discriminator

View File

@@ -181,7 +181,7 @@ func (s *Server) Start() error {
log.Warnf("failed to redirect stderr: %v", err)
}
if err := RestoreResidualState(s.rootCtx, s.profileManager.GetStatePath()); err != nil {
if err := restoreResidualState(s.rootCtx, s.profileManager.GetStatePath()); err != nil {
log.Warnf(errRestoreResidualState, err)
}
@@ -551,7 +551,7 @@ func (s *Server) Login(callerCtx context.Context, msg *proto.LoginRequest) (*pro
s.actCancel = cancel
s.mutex.Unlock()
if err := RestoreResidualState(s.rootCtx, s.profileManager.GetStatePath()); err != nil {
if err := restoreResidualState(s.rootCtx, s.profileManager.GetStatePath()); err != nil {
log.Warnf(errRestoreResidualState, err)
}
@@ -858,7 +858,7 @@ func (s *Server) Up(callerCtx context.Context, msg *proto.UpRequest) (*proto.UpR
return s.waitForUp(callerCtx)
}
if err := RestoreResidualState(callerCtx, s.profileManager.GetStatePath()); err != nil {
if err := restoreResidualState(callerCtx, s.profileManager.GetStatePath()); err != nil {
log.Warnf(errRestoreResidualState, err)
}

View File

@@ -46,7 +46,7 @@ func (s *Server) CleanState(ctx context.Context, req *proto.CleanStateRequest) (
if req.All {
// Reuse existing cleanup logic for all states
if err := RestoreResidualState(ctx, statePath); err != nil {
if err := restoreResidualState(ctx, statePath); err != nil {
return nil, status.Errorf(codes.Internal, "failed to clean all states: %v", err)
}
@@ -113,9 +113,9 @@ func (s *Server) DeleteState(ctx context.Context, req *proto.DeleteStateRequest)
}, nil
}
// RestoreResidualState checks if the client was not shut down in a clean way and restores residual if required.
// restoreResidualState checks if the client was not shut down in a clean way and restores residual if required.
// Otherwise, we might not be able to connect to the management server to retrieve new config.
func RestoreResidualState(ctx context.Context, statePath string) error {
func restoreResidualState(ctx context.Context, statePath string) error {
if statePath == "" {
return nil
}

View File

@@ -746,8 +746,6 @@ func ToProtoFullStatus(fullStatus peer.FullStatus) *proto.FullStatus {
pbFullStatus.DnsServers = append(pbFullStatus.DnsServers, pbDnsState)
}
pbFullStatus.Events = fullStatus.Events
return &pbFullStatus
}

View File

@@ -1,107 +0,0 @@
//go:build !android && !ios && !freebsd && !js
package main
import (
"context"
"os"
"path/filepath"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/mdm"
"github.com/netbirdio/netbird/client/ui/preferences"
"github.com/netbirdio/netbird/client/ui/services"
)
// autostartDefaultState carries the guard inputs of the one-time autostart
// default decision so the decision itself stays a pure, testable function.
type autostartDefaultState struct {
supported bool
mdmDisabled bool
priorInstall bool
}
// shouldEnableAutostartDefault applies the first-run guards in order and
// returns whether autostart may be enabled, plus the reason when it may not.
func shouldEnableAutostartDefault(s autostartDefaultState) (bool, string) {
switch {
case !s.supported:
return false, "autostart not supported on this platform"
case s.mdmDisabled:
return false, "autostart disabled by MDM policy"
case s.priorInstall:
return false, "existing NetBird installation"
}
return true, ""
}
// autostartDisabledByMDM reports whether the MDM policy manages the
// disableAutostart key in a way that must suppress the default. An
// unparseable managed value is treated as disabled to stay on the safe side.
func autostartDisabledByMDM(policy *mdm.Policy) bool {
if !policy.HasKey(mdm.KeyDisableAutostart) {
return false
}
disabled, ok := policy.GetBool(mdm.KeyDisableAutostart)
return !ok || disabled
}
// netbirdFootprintExists reports whether the machine already carries NetBird
// daemon config or state, meaning this is not a genuinely fresh install. It is
// the update-safety gate for the autostart default: upgrading users always
// have a footprint, so an update can never trigger a login-item write.
func netbirdFootprintExists() bool {
candidates := []string{
profilemanager.DefaultConfigPath,
filepath.Join(profilemanager.DefaultConfigPathDir, "config.json"),
filepath.Join(profilemanager.DefaultConfigPathDir, "state.json"),
}
for _, path := range candidates {
if path != "" && fileExists(path) {
return true
}
}
return false
}
// applyAutostartDefault runs the one-time launch-on-login default for genuinely
// fresh installs. The autostartInitialized marker is persisted before any
// enable attempt so a crash mid-flow degrades to "never enabled" instead of
// retrying login-item writes on every launch. A user's later disable in
// Settings is never overridden: the marker guarantees at-most-once, ever.
func applyAutostartDefault(ctx context.Context, autostart *services.Autostart, prefs *preferences.Store, prefsFileExisted bool) {
priorFootprint := netbirdFootprintExists() || prefsFileExisted
if prefs.Get().AutostartInitialized {
return
}
if err := prefs.SetAutostartInitialized(true); err != nil {
log.Warnf("persist autostart marker, skipping autostart default: %v", err)
return
}
state := autostartDefaultState{
supported: autostart.Supported(ctx),
mdmDisabled: autostartDisabledByMDM(mdm.LoadPolicy()),
priorInstall: priorFootprint,
}
enable, reason := shouldEnableAutostartDefault(state)
if !enable {
log.Debugf("skipping autostart default: %s", reason)
return
}
if err := autostart.SetEnabled(ctx, true); err != nil {
log.Warnf("enable autostart on fresh install: %v", err)
return
}
log.Info("autostart enabled by default on fresh install")
}
// fileExists reports whether path exists.
func fileExists(path string) bool {
_, err := os.Stat(path)
return err == nil
}

View File

@@ -1,125 +0,0 @@
//go:build !android && !ios && !freebsd && !js
package main
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/netbirdio/netbird/client/mdm"
)
func TestShouldEnableAutostartDefault(t *testing.T) {
allPass := autostartDefaultState{
supported: true,
mdmDisabled: false,
priorInstall: false,
}
tests := []struct {
name string
mutate func(*autostartDefaultState)
wantEnable bool
wantReason string
}{
{
name: "fresh install with all guards passing enables",
mutate: func(*autostartDefaultState) {},
wantEnable: true,
},
{
name: "unsupported platform skips",
mutate: func(s *autostartDefaultState) { s.supported = false },
wantReason: "autostart not supported on this platform",
},
{
name: "MDM disable skips",
mutate: func(s *autostartDefaultState) { s.mdmDisabled = true },
wantReason: "autostart disabled by MDM policy",
},
{
name: "existing installation (upgrade) skips",
mutate: func(s *autostartDefaultState) { s.priorInstall = true },
wantReason: "existing NetBird installation",
},
{
name: "unsupported wins over every other guard",
mutate: func(s *autostartDefaultState) {
s.supported = false
s.mdmDisabled = true
s.priorInstall = true
},
wantReason: "autostart not supported on this platform",
},
{
name: "MDM disable wins over prior install",
mutate: func(s *autostartDefaultState) {
s.mdmDisabled = true
s.priorInstall = true
},
wantReason: "autostart disabled by MDM policy",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
state := allPass
tc.mutate(&state)
enable, reason := shouldEnableAutostartDefault(state)
assert.Equal(t, tc.wantEnable, enable, "enable decision should match for state %+v", state)
assert.Equal(t, tc.wantReason, reason, "skip reason should identify the failing guard")
})
}
}
func TestAutostartDisabledByMDM(t *testing.T) {
tests := []struct {
name string
values map[string]any
want bool
}{
{
name: "empty policy does not disable",
values: nil,
want: false,
},
{
name: "unrelated managed keys do not disable",
values: map[string]any{mdm.KeyDisableAutoConnect: true},
want: false,
},
{
name: "disableAutostart true disables",
values: map[string]any{mdm.KeyDisableAutostart: true},
want: true,
},
{
name: "disableAutostart registry DWORD 1 disables",
values: map[string]any{mdm.KeyDisableAutostart: int64(1)},
want: true,
},
{
name: "disableAutostart string true disables",
values: map[string]any{mdm.KeyDisableAutostart: "true"},
want: true,
},
{
name: "disableAutostart explicit false allows",
values: map[string]any{mdm.KeyDisableAutostart: false},
want: false,
},
{
name: "unparseable managed value is treated as disabled",
values: map[string]any{mdm.KeyDisableAutostart: "not-a-bool"},
want: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := autostartDisabledByMDM(mdm.NewPolicy(tc.values))
assert.Equal(t, tc.want, got, "MDM disable decision should match for values %v", tc.values)
})
}
}

View File

@@ -51,14 +51,7 @@ async function runSsoLogin(
if (uri) await openBrowserLoginUri(uri);
const cancelPromise = buildSsoCancelPromise(state, signal);
// Combine wait + up in Go so the connection comes up the moment SSO
// completes. During SSO the tray window is hidden and the webview is
// suspended, so a frontend-driven Up (a promise continuation) would not
// fire until the user woke the window (e.g. hovering the tray icon).
const waitPromise = Connection.WaitSSOLoginAndUp(
{ userCode: result.userCode, hostname: "" },
{ profileName: "", username: "" },
);
const waitPromise = Connection.WaitSSOLogin({ userCode: result.userCode, hostname: "" });
try {
await Promise.race([waitPromise, cancelPromise]);
@@ -96,13 +89,13 @@ export async function startConnection(onSettled?: () => void, signal?: AbortSign
if (signal?.aborted) state.cancelled = true;
if (!state.cancelled && result.needsSsoLogin) {
// runSsoLogin brings the connection up in Go once SSO completes.
await runSsoLogin(result, state, signal);
} else {
if (!state.cancelled && signal?.aborted) state.cancelled = true;
if (!state.cancelled) {
await Connection.Up({ profileName: "", username: "" });
}
}
if (!state.cancelled && signal?.aborted) state.cancelled = true;
if (!state.cancelled) {
await Connection.Up({ profileName: "", username: "" });
}
} catch (e) {
WindowManager.CloseBrowserLogin().catch(console.error);

View File

@@ -197,9 +197,6 @@ func main() {
// daemon may keep the main window from showing, so the OS toast is the
// only reliable signal the user gets.
go notifyIfDaemonOutdated(compat, notifier, localizer)
// One-time launch-on-login default for fresh installs; gated by the
// NetBird footprint check, MDM policy, and the persisted marker.
go applyAutostartDefault(context.Background(), services.NewAutostart(app.Autostart), prefStore, prefStore.ExistedAtLoad())
})
if err := app.Run(); err != nil {

View File

@@ -54,10 +54,6 @@ type UIPreferences struct {
Language i18n.LanguageCode `json:"language"`
ViewMode ViewMode `json:"viewMode"`
OnboardingCompleted bool `json:"onboardingCompleted"`
// AutostartInitialized records that the one-time autostart default
// decision has run for this OS user. It only ever transitions to true
// and is never reset, so the default-on flow runs at most once, ever.
AutostartInitialized bool `json:"autostartInitialized"`
}
// LanguageValidator rejects SetLanguage inputs with no shipped bundle.
@@ -76,9 +72,8 @@ type Emitter interface {
type Store struct {
path string
mu sync.RWMutex
current UIPreferences
existedAtLoad bool
mu sync.RWMutex
current UIPreferences
subsMu sync.Mutex
subs []chan UIPreferences
@@ -162,27 +157,6 @@ func (s *Store) SetOnboardingCompleted(done bool) error {
return nil
}
// SetAutostartInitialized persists the one-time autostart decision marker.
// No-op if unchanged.
func (s *Store) SetAutostartInitialized(done bool) error {
s.mu.Lock()
if s.current.AutostartInitialized == done {
s.mu.Unlock()
return nil
}
next := s.current
next.AutostartInitialized = done
if err := s.persistLocked(next); err != nil {
s.mu.Unlock()
return fmt.Errorf("persist preferences: %w", err)
}
s.current = next
s.mu.Unlock()
s.broadcast(next)
return nil
}
// SetLanguage validates, persists, and broadcasts. No-op if unchanged.
func (s *Store) SetLanguage(lang i18n.LanguageCode) error {
if lang == "" {
@@ -232,29 +206,13 @@ func (s *Store) Subscribe() (<-chan UIPreferences, func()) {
return ch, unsubscribe
}
// ExistedAtLoad reports whether the backing preferences file was present on
// disk when the store loaded. It distinguishes a user who ran a prior GUI
// version from a brand-new OS user with no preferences yet.
func (s *Store) ExistedAtLoad() bool {
s.mu.RLock()
defer s.mu.RUnlock()
return s.existedAtLoad
}
// load reads the file into current. A missing file is not an error (the
// in-memory default stands); malformed contents return an error.
func (s *Store) load() error {
if _, err := os.Stat(s.path); err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return fmt.Errorf("stat preferences: %w", err)
if _, err := os.Stat(s.path); errors.Is(err, os.ErrNotExist) {
return nil
}
s.mu.Lock()
s.existedAtLoad = true
s.mu.Unlock()
var loaded UIPreferences
if _, err := util.ReadJson(s.path, &loaded); err != nil {
return err

View File

@@ -215,46 +215,6 @@ func TestStore_FileShapeIsJSON(t *testing.T) {
assert.Equal(t, i18n.LanguageCode("hu"), parsed.Language)
}
func TestStore_SetAutostartInitializedPersistsAcrossReload(t *testing.T) {
withTempConfigDir(t)
emitter := &recordingEmitter{}
s, err := NewStore(nil, emitter)
require.NoError(t, err)
assert.False(t, s.Get().AutostartInitialized, "marker must default to false when no file is on disk")
require.NoError(t, s.SetAutostartInitialized(true))
assert.True(t, s.Get().AutostartInitialized, "Get should reflect the persisted marker")
require.Len(t, emitter.calledWith(EventPreferencesChanged), 1, "first marker write should broadcast")
// Re-setting the same value must be a no-op: no disk write, no broadcast.
require.NoError(t, s.SetAutostartInitialized(true))
assert.Len(t, emitter.calledWith(EventPreferencesChanged), 1, "idempotent marker write should not broadcast again")
// A fresh Store (new GUI launch) must see the marker so the autostart
// default decision never runs twice.
reloaded, err := NewStore(nil, nil)
require.NoError(t, err)
assert.True(t, reloaded.Get().AutostartInitialized, "marker must survive a reload from disk")
}
func TestStore_ExistedAtLoad(t *testing.T) {
withTempConfigDir(t)
// Brand-new OS user: no preferences file on disk yet.
fresh, err := NewStore(nil, nil)
require.NoError(t, err)
assert.False(t, fresh.ExistedAtLoad(), "ExistedAtLoad must be false when no file is on disk")
// Persisting a value writes the file to disk.
require.NoError(t, fresh.SetLanguage("en"))
// A subsequent GUI launch reopens the now-present file.
reopened, err := NewStore(nil, nil)
require.NoError(t, err)
assert.True(t, reopened.ExistedAtLoad(), "ExistedAtLoad must be true after the store has persisted and is reopened")
}
func TestStore_ErrUnsupportedSentinel(t *testing.T) {
// Verifies callers can match on the sentinel error rather than parsing
// strings — protects against accidental %v -> %w changes that would

View File

@@ -35,7 +35,7 @@ type LoginResult struct {
VerificationURIComplete string `json:"verificationUriComplete"`
}
// WaitSSOParams are the inputs to waitSSOLogin.
// WaitSSOParams are the inputs to WaitSSOLogin.
type WaitSSOParams struct {
UserCode string `json:"userCode"`
Hostname string `json:"hostname"`
@@ -125,6 +125,23 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
}, nil
}
func (s *Connection) WaitSSOLogin(ctx context.Context, p WaitSSOParams) (string, error) {
cli, err := s.conn.Client()
if err != nil {
return "", err
}
log.Infof("waiting for SSO login to complete")
resp, err := cli.WaitSSOLogin(ctx, &proto.WaitSSOLoginRequest{
UserCode: p.UserCode,
Hostname: p.Hostname,
})
if err != nil {
return "", s.classifyDaemonError(err)
}
log.Infof("SSO login completed, daemon reported success")
return resp.GetEmail(), nil
}
func (s *Connection) Up(ctx context.Context, p UpParams) error {
cli, err := s.conn.Client()
if err != nil {
@@ -145,27 +162,6 @@ func (s *Connection) Up(ctx context.Context, p UpParams) error {
return nil
}
// WaitSSOLoginAndUp blocks until the SSO login completes and then brings the
// connection up, both from the Go side. Keeping the post-login Up here rather
// than as a frontend continuation is deliberate: during SSO the tray window is
// hidden and the webview is suspended (macOS App Nap / hidden-window timer
// throttling), so a frontend-driven Up would not run until the user woke the
// window (e.g. by hovering the tray icon). Doing it in Go connects the moment
// the daemon reports SSO success. Returns the authenticated user's email.
func (s *Connection) WaitSSOLoginAndUp(ctx context.Context, wait WaitSSOParams, up UpParams) (string, error) {
email, err := s.waitSSOLogin(ctx, wait)
if err != nil {
return "", err
}
if err := ctx.Err(); err != nil {
return "", err
}
if err := s.Up(ctx, up); err != nil {
return "", err
}
return email, nil
}
func (s *Connection) Down(ctx context.Context) error {
cli, err := s.conn.Client()
if err != nil {
@@ -225,26 +221,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return nil
}
// waitSSOLogin blocks until the daemon reports the SSO login result and returns
// the authenticated user's email. It is unexported because the frontend drives
// SSO through the exported WaitSSOLoginAndUp.
func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string, error) {
cli, err := s.conn.Client()
if err != nil {
return "", err
}
log.Infof("waiting for SSO login to complete")
resp, err := cli.WaitSSOLogin(ctx, &proto.WaitSSOLoginRequest{
UserCode: p.UserCode,
Hostname: p.Hostname,
})
if err != nil {
return "", s.classifyDaemonError(err)
}
log.Infof("SSO login completed, daemon reported success")
return resp.GetEmail(), nil
}
// classifyDaemonError maps a gRPC error to a localised ClientError.
func (s *Connection) classifyDaemonError(err error) *ClientError {
return s.classifier.classify(err)

View File

@@ -20,12 +20,11 @@ type MDMFields struct {
DisableServerRoutes bool `json:"disableServerRoutes"`
AllowServerSSH *bool `json:"allowServerSSH"`
DisableAutoConnect bool `json:"disableAutoConnect"`
DisableAutostart bool `json:"disableAutostart"`
BlockInbound bool `json:"blockInbound"`
DisableMetricsCollection bool `json:"disableMetricsCollection"`
SplitTunnelMode bool `json:"splitTunnelMode"`
SplitTunnelApps bool `json:"splitTunnelApps"`
DisableAdvancedView bool `json:"disableAdvancedView"`
DisableAdvancedView bool `json:"disableAdvancedView"`
}
type Features struct {

View File

@@ -91,7 +91,7 @@ func availableProviders() []providerCase {
if region == "" {
region = "us-east-1"
}
ps = append(ps, providerCase{name: "bedrock", catalogID: "bedrock_api", upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k, model: "us.anthropic.claude-haiku-4-5", kind: harness.WireBedrock})
ps = append(ps, providerCase{name: "bedrock", catalogID: "bedrock_api", upstream: "https://bedrock-runtime." + region + ".amazonaws.com", apiKey: k, model: "us.anthropic.claude-haiku-4-5", kind: harness.WireMessages})
}
return ps
}
@@ -224,12 +224,9 @@ func TestProvidersMatrix(t *testing.T) {
var c int
var b string
var cerr error
switch pc.kind {
case harness.WireVertex:
if pc.kind == harness.WireVertex {
c, b, cerr = cl.Vertex(ctx, settings.Endpoint, proxyIP, pc.project, pc.region, pc.model, "Reply with exactly: pong", sessionID)
case harness.WireBedrock:
c, b, cerr = cl.Bedrock(ctx, settings.Endpoint, proxyIP, pc.model, "Reply with exactly: pong", sessionID)
default:
} else {
c, b, cerr = cl.Chat(ctx, settings.Endpoint, proxyIP, pc.kind, pc.model, "Reply with exactly: pong", sessionID)
}
if cerr == nil {

View File

@@ -1,168 +0,0 @@
//go:build e2e
package agentnetwork
import (
"context"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// catalogModel returns the normalized catalog id the proxy stamps for a
// path-routed provider's configured model — the form the guardrail allowlist is
// compared against (region prefix / @version stripped).
func catalogModel(pc providerCase) string {
switch pc.kind {
case harness.WireBedrock:
return strings.TrimPrefix(pc.model, "us.")
case harness.WireVertex:
return strings.SplitN(pc.model, "@", 2)[0]
default:
return pc.model
}
}
// disallowedModel returns a valid-shaped model id for the provider that is NOT
// the configured/allowed one, so the guardrail must reject it before the
// request ever reaches the upstream.
func disallowedModel(pc providerCase) string {
switch pc.kind {
case harness.WireBedrock:
return "us.anthropic.claude-opus-4-8"
case harness.WireVertex:
return "claude-opus-4-8@20250101"
default:
return "unlisted-model"
}
}
// sendModel drives one request for the given model through the provider's native
// wire shape and returns the HTTP status.
func sendModel(ctx context.Context, t *testing.T, cl *harness.Client, endpoint, proxyIP string, pc providerCase, model string) int {
t.Helper()
var code int
var err error
switch pc.kind {
case harness.WireBedrock:
code, _, err = cl.Bedrock(ctx, endpoint, proxyIP, model, "Reply with exactly: pong", "")
case harness.WireVertex:
code, _, err = cl.Vertex(ctx, endpoint, proxyIP, pc.project, pc.region, model, "Reply with exactly: pong", "")
default:
code, _, err = cl.Chat(ctx, endpoint, proxyIP, pc.kind, model, "Reply with exactly: pong", "")
}
require.NoError(t, err, "request must reach the proxy for %s", pc.name)
return code
}
// TestModelAllowlistEnforced provisions a Model Allowlist guardrail limiting each
// path-routed provider (Bedrock, Vertex) to its configured model, then drives
// requests over the tunnel: the allowed model returns 200 while a model outside
// the allowlist is denied 403 by the guardrail before it reaches the upstream.
// This is the coverage missing for #6751 — the model for these providers travels
// in the URL path, and the allowlist must be enforced there.
func TestModelAllowlistEnforced(t *testing.T) {
var providers []providerCase
for _, pc := range availableProviders() {
if pc.kind == harness.WireBedrock || pc.kind == harness.WireVertex {
providers = append(providers, pc)
}
}
if len(providers) == 0 {
t.Skip("no path-routed provider keys set (AWS_BEARER_TOKEN_BEDROCK / GOOGLE_VERTEX_*); source ~/.llm-keys")
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
grp, err := srv.API().Groups.Create(ctx, api.PostApiGroupsJSONRequestBody{Name: "e2e-allowlist"})
require.NoError(t, err, "create group")
t.Cleanup(func() { _ = srv.API().Groups.Delete(context.Background(), grp.Id) })
ephemeral := false
sk, err := srv.API().SetupKeys.Create(ctx, api.PostApiSetupKeysJSONRequestBody{
Name: "e2e-allowlist-client",
Type: "reusable",
ExpiresIn: 86400,
UsageLimit: 0,
AutoGroups: []string{grp.Id},
Ephemeral: &ephemeral,
})
require.NoError(t, err, "mint setup key")
// Providers with their configured (allowed) models; the first bootstraps the cluster.
ids := make([]string, 0, len(providers))
allowed := make([]string, 0, len(providers))
for i, pc := range providers {
req := providerRequest(pc)
if i == 0 {
req.BootstrapCluster = ptr(harness.AgentNetworkCluster)
}
prov, perr := srv.CreateProvider(ctx, req)
require.NoError(t, perr, "create provider %s", pc.name)
id := prov.Id
ids = append(ids, id)
allowed = append(allowed, catalogModel(pc))
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), id) })
}
// Guardrail allowlisting exactly the configured models.
var gr api.AgentNetworkGuardrailRequest
gr.Name = "e2e-allowlist"
gr.Checks.ModelAllowlist.Enabled = true
gr.Checks.ModelAllowlist.Models = allowed
guard, err := srv.CreateGuardrail(ctx, gr)
require.NoError(t, err, "create guardrail")
t.Cleanup(func() { _ = srv.DeleteGuardrail(context.Background(), guard.Id) })
enabled := true
pol, err := srv.CreatePolicy(ctx, api.AgentNetworkPolicyRequest{
Name: "e2e-allowlist",
Enabled: &enabled,
SourceGroups: []string{grp.Id},
DestinationProviderIds: ids,
GuardrailIds: &[]string{guard.Id},
})
require.NoError(t, err, "create policy")
t.Cleanup(func() { _ = srv.DeletePolicy(context.Background(), pol.Id) })
settings, err := srv.GetSettings(ctx)
require.NoError(t, err, "read settings for endpoint")
require.NotEmpty(t, settings.Endpoint, "agent-network endpoint must be assigned")
proxyToken, err := srv.CreateProxyTokenCLI(ctx, "e2e-proxy-allowlist")
require.NoError(t, err, "mint proxy token via CLI")
px, err := harness.StartProxy(ctx, srv, proxyToken)
require.NoError(t, err, "start proxy")
t.Cleanup(func() { _ = px.Terminate(context.Background()) })
cl, err := harness.StartClient(ctx, srv, sk.Key)
require.NoError(t, err, "start client")
t.Cleanup(func() { _ = cl.Terminate(context.Background()) })
require.NoError(t, cl.WaitConnected(ctx, 90*time.Second), "client must connect to management")
if err := cl.WaitProxyPeer(ctx, 180*time.Second); err != nil {
t.Fatalf("client did not see the proxy peer: %v\n=== proxy logs ===\n%s", err, px.Logs(context.Background()))
}
proxyIP, err := cl.ResolveProxyIP(ctx, settings.Endpoint)
require.NoError(t, err, "resolve agent-network endpoint to proxy IP")
for _, pc := range providers {
pc := pc
t.Run(pc.name, func(t *testing.T) {
// The admin's allowlisted model is served end to end.
assert.Equal(t, 200, sendModel(ctx, t, cl, settings.Endpoint, proxyIP, pc, pc.model),
"allowlisted model must be permitted for %s", pc.name)
// A model outside the allowlist is rejected by the guardrail (before
// the upstream), regardless of whether it is a real catalog model.
assert.Equal(t, 403, sendModel(ctx, t, cl, settings.Endpoint, proxyIP, pc, disallowedModel(pc)),
"model outside the allowlist must be denied for %s", pc.name)
})
}
}

View File

@@ -107,17 +107,6 @@ func (c *Combined) DeletePolicy(ctx context.Context, id string) error {
return anDelete(ctx, c, "/api/agent-network/policies/"+id)
}
// CreateGuardrail creates an agent-network guardrail (e.g. a model allowlist)
// that can then be attached to a policy via its GuardrailIds.
func (c *Combined) CreateGuardrail(ctx context.Context, req api.AgentNetworkGuardrailRequest) (api.AgentNetworkGuardrail, error) {
return anRequest[api.AgentNetworkGuardrail](ctx, c, http.MethodPost, "/api/agent-network/guardrails", req)
}
// DeleteGuardrail removes a guardrail by id.
func (c *Combined) DeleteGuardrail(ctx context.Context, id string) error {
return anDelete(ctx, c, "/api/agent-network/guardrails/"+id)
}
// GetSettings returns the account's agent-network settings row. It exists only
// after the first provider create bootstraps it.
func (c *Combined) GetSettings(ctx context.Context) (api.AgentNetworkSettings, error) {

View File

@@ -194,11 +194,6 @@ const (
// WireVertex is the Anthropic-on-Vertex rawPredict shape: the client posts
// the full Vertex model path and the proxy mints the SA OAuth token.
WireVertex = "vertex"
// WireBedrock is the native AWS Bedrock InvokeModel shape: the model id
// travels in the URL path (/model/{id}/invoke), not the body, so the proxy
// routes by path. This is what a Bedrock SDK client sends and the shape the
// model-allowlist guardrail must enforce.
WireBedrock = "bedrock"
)
// Chat issues a chat-completion POST to the agent-network endpoint over the
@@ -231,17 +226,6 @@ func (cl *Client) Vertex(ctx context.Context, endpoint, proxyIP, project, region
return cl.post(ctx, endpoint, proxyIP, path, body, withSessionID(nil, sessionID))
}
// Bedrock issues a native AWS Bedrock InvokeModel POST over the tunnel. The
// model id is carried in the request path (/model/{id}/invoke), so the proxy
// routes by path; the body uses the bedrock anthropic_version rather than a
// model field. A non-empty sessionID is sent as the universal x-session-id
// header the proxy records.
func (cl *Client) Bedrock(ctx context.Context, endpoint, proxyIP, model, prompt, sessionID string) (int, string, error) {
path := "/model/" + model + "/invoke"
body := fmt.Sprintf(`{"anthropic_version":"bedrock-2023-05-31","max_tokens":64,"messages":[{"role":"user","content":%q}]}`, prompt)
return cl.post(ctx, endpoint, proxyIP, path, body, withSessionID(nil, sessionID))
}
// withSessionID appends the x-session-id header when sessionID is non-empty.
func withSessionID(headers []string, sessionID string) []string {
if sessionID == "" {

View File

@@ -215,7 +215,7 @@ func (e *EphemeralManager) cleanup(ctx context.Context) {
}
for accountID, peerIDs := range peerIDsPerAccount {
log.WithContext(ctx).Debugf("cleanup: deleting %d ephemeral peers for account %s: %s", len(peerIDs), accountID, peerIDs)
log.WithContext(ctx).Tracef("cleanup: deleting %d ephemeral peers for account %s", len(peerIDs), accountID)
err := e.peersManager.DeletePeers(ctx, accountID, peerIDs, activity.SystemInitiator, true)
if err != nil {
log.WithContext(ctx).Errorf("failed to delete ephemeral peers: %s", err)

View File

@@ -184,8 +184,6 @@ func (m *managerImpl) DeletePeers(ctx context.Context, accountID string, peerIDs
return err
}
log.WithContext(ctx).Debugf("DeletePeers: deleted peer %s", peerID)
if !(peer.ProxyMeta.Embedded || peer.Meta.KernelVersion == "wasm") {
eventsToStore = append(eventsToStore, func() {
m.accountManager.StoreEvent(ctx, userID, peer.ID, accountID, activity.PeerRemovedByUser, peer.EventMeta(dnsDomain))
@@ -226,6 +224,30 @@ func (m *managerImpl) CreateProxyPeer(ctx context.Context, accountID string, pee
return nil
}
// Dedupe stale embedded peer records for the same (account, cluster).
// The proxy generates a fresh WireGuard keypair on every startup
// (proxy/internal/roundtrip/netbird.go), so without this sweep the
// prior embedded peer would linger forever — holding its CGNAT IP
// allocation, polluting other peers' rosters, and (most visibly)
// leaving the synth DNS pointing at the dead address. The
// (account, cluster) tuple identifies "the embedded peer for this
// proxy instance at this cluster"; any record matching that tuple
// with a different pubkey is by definition stale and must go.
staleIDs, err := m.findStaleEmbeddedProxyPeers(ctx, accountID, cluster, peerKey)
if err != nil {
return fmt.Errorf("scan for stale embedded proxy peers: %w", err)
}
if len(staleIDs) > 0 {
// userID="" + checkConnected=false: the deletion is initiated
// by management itself on behalf of the freshly-registering
// proxy, not by an end user; the stale peer may still be
// marked Connected from its prior session, but its session is
// dead by definition (its key no longer exists).
if err := m.DeletePeers(ctx, accountID, staleIDs, "", false); err != nil {
return fmt.Errorf("delete stale embedded proxy peers %v: %w", staleIDs, err)
}
}
name := fmt.Sprintf("proxy-%s", xid.New().String())
newPeer := &peer.Peer{
Ephemeral: true,
@@ -251,3 +273,29 @@ func (m *managerImpl) CreateProxyPeer(ctx context.Context, accountID string, pee
return nil
}
// findStaleEmbeddedProxyPeers returns the peer IDs of embedded proxy peer
// records in accountID that target the same cluster but carry a different
// WireGuard pubkey than the freshly-registering one. Used by CreateProxyPeer
// to garbage-collect stale records left behind when the proxy restarts with a
// regenerated keypair.
func (m *managerImpl) findStaleEmbeddedProxyPeers(ctx context.Context, accountID, cluster, newKey string) ([]string, error) {
account, err := m.store.GetAccount(ctx, accountID)
if err != nil {
return nil, err
}
var stale []string
for _, p := range account.Peers {
if p == nil || !p.ProxyMeta.Embedded {
continue
}
if p.ProxyMeta.Cluster != cluster {
continue
}
if p.Key == newKey {
continue
}
stale = append(stale, p.ID)
}
return stale, nil
}

View File

@@ -161,8 +161,6 @@ func (m *TimeBasedAuthSecretsManager) SetupRefresh(ctx context.Context, accountI
m.turnCancelMap[peerID] = turnCancel
go m.refreshTURNTokens(ctx, accountID, peerID, turnCancel)
log.WithContext(ctx).Debugf("starting TURN refresh for %s", peerID)
} else {
log.WithContext(ctx).Debugf("no TURN configuration, skipping TURN refresh for %s", peerID)
}
if m.relayCfg != nil {
@@ -170,8 +168,6 @@ func (m *TimeBasedAuthSecretsManager) SetupRefresh(ctx context.Context, accountI
m.relayCancelMap[peerID] = relayCancel
go m.refreshRelayTokens(ctx, accountID, peerID, relayCancel)
log.WithContext(ctx).Tracef("starting relay refresh for %s", peerID)
} else {
log.WithContext(ctx).Tracef("no relay configuration, skipping relay refresh for %s", peerID)
}
}

View File

@@ -1,24 +1,19 @@
package main
import (
"log"
"net/http"
// nolint:gosec
_ "net/http/pprof"
"os"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/management/cmd"
)
func main() {
if pprofAddr := os.Getenv("NB_PPROF_ADDR"); pprofAddr != "" {
log.Infof("pprof enabled, listening on: %s", pprofAddr)
go func() {
log.Println(http.ListenAndServe(pprofAddr, nil))
}()
}
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
if err := cmd.Execute(); err != nil {
os.Exit(1)
}

View File

@@ -0,0 +1,199 @@
package server
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/peers"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork"
agenttypes "github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
nbpeer "github.com/netbirdio/netbird/management/server/peer"
"github.com/netbirdio/netbird/management/server/permissions"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
)
// TestAgentNetwork_ProxyRestart_PropagatesNewPeerAndDropsStale is the no-mock
// regression guard for the bug the user reported: restarting the proxy creates
// a fresh embedded peer with a NEW WireGuard public key (the proxy generates
// the keypair on every startup at proxy/internal/roundtrip/netbird.go:312).
// The PRIOR embedded peer record is never deleted on management, so the
// account accumulates a stale peer holding a stale CGNAT IP. Other peers
// in the account either keep routing to the dead IP, or — if synth DNS
// picks the wrong record — never see the new IP at all.
//
// What this test exercises (no mocks):
// - real SQLite test store
// - real DefaultAccountManager, network-map controller, peer-update channels
// - real peers.Manager.CreateProxyPeer path (the very method the proxy
// invokes over gRPC on every startup)
// - real agentnetwork.Manager + synth chain so the client receives a
// concrete DNS record that must point at the LATEST proxy peer.
//
// Pre-fix expected behavior (red): two embedded peers exist after the
// "restart"; the synth DNS record points at the stale one; the client
// receives an update reflecting the new peer but the old one lingers.
// Post-fix expected behavior (green): exactly one embedded peer exists
// after restart (with the new key) AND the client's network map carries
// the synth DNS pointing at that new peer's CGNAT IP.
func TestAgentNetwork_ProxyRestart_PropagatesNewPeerAndDropsStale(t *testing.T) {
am, updateManager, err := createManager(t)
require.NoError(t, err, "createManager must succeed")
ctx := context.Background()
const (
accountID = "an-restart-acct"
adminUserID = "an-restart-admin"
groupAID = "an-restart-grp-A"
clusterAddr = "eu.proxy.netbird.io"
clientKey = "BhRPtynAAYRDy08+q4HTMsos8fs4plTP4NOSh7C1ry8="
// Two different proxy pubkeys — the "before" and "after" of a
// proxy-process restart with fresh-keypair generation.
proxyKey1 = "Aaaaa1aaaaYRDy08+q4HTMsos8fs4plTP4NOSh7C1ry8="
proxyKey2 = "Bbbbb2bbbbYRDy08+q4HTMsos8fs4plTP4NOSh7C1ry8="
)
// --- Account scaffold ---
account := newAccountWithId(ctx, accountID, adminUserID, "an-restart.test", "", "", false)
require.NoError(t, am.Store.SaveAccount(ctx, account))
clientPeer := &nbpeer.Peer{
Key: clientKey,
Name: "an-restart-client",
DNSLabel: "an-restart-client",
Meta: nbpeer.PeerSystemMeta{Hostname: "an-restart-client", GoOS: "linux", WtVersion: "development"},
}
addedClient, _, _, _, err := am.AddPeer(ctx, "", "", adminUserID, clientPeer, false)
require.NoError(t, err, "AddPeer for client must succeed")
require.NoError(t, am.MarkPeerConnected(ctx, clientKey, accountID, time.Now().UnixNano(), &types.NetworkMap{}),
"MarkPeerConnected for the client peer must succeed (affected-peer fan-out skips disconnected peers)")
// Place the client in group A so the synth policy reaches it.
account, err = am.Store.GetAccount(ctx, accountID)
require.NoError(t, err)
account.Groups[groupAID] = &types.Group{ID: groupAID, Name: "groupA", Peers: []string{addedClient.ID}}
require.NoError(t, am.Store.SaveAccount(ctx, account), "SaveAccount must persist group A")
// --- Real peers + agent-network managers ---
permMgr := permissions.NewManager(am.Store)
peersMgr := peers.NewManager(am.Store, permMgr)
peersMgr.SetAccountManager(am)
peersMgr.SetNetworkMapController(am.networkMapController)
agentMgr := agentnetwork.NewManager(am.Store, permMgr, am, nil)
// Subscribe BEFORE any state-mutating call so we don't lose the update
// that contains the synth DNS record.
clientCh := updateManager.CreateChannel(ctx, addedClient.ID)
t.Cleanup(func() { updateManager.CloseChannel(ctx, addedClient.ID) })
drain(clientCh)
// --- First proxy startup: register peer key K1, then mark it
// connected. In production the proxy follows CreateProxyPeer with the
// regular sync stream which lands on MarkPeerConnected; the synth DNS
// path filters out peers that aren't Connected (types/account.go:323),
// so without this step no DNS record would be emitted.
require.NoError(t, peersMgr.CreateProxyPeer(ctx, accountID, proxyKey1, clusterAddr),
"first CreateProxyPeer (proxy startup) must succeed")
peer1ID, err := am.Store.GetPeerIDByKey(ctx, store.LockingStrengthNone, proxyKey1)
require.NoError(t, err, "proxy peer for K1 must be persisted after CreateProxyPeer")
require.NotEmpty(t, peer1ID)
require.NoError(t, am.MarkPeerConnected(ctx, proxyKey1, accountID, time.Now().UnixNano(), &types.NetworkMap{}),
"MarkPeerConnected for K1 must succeed")
account, err = am.Store.GetAccount(ctx, accountID)
require.NoError(t, err)
proxyIP1 := account.Peers[peer1ID].IP.String()
require.NotEmpty(t, proxyIP1, "K1 must have an assigned overlay IP")
// --- Provider + policy. CreateProvider / CreatePolicy trigger the
// agentnetwork reconcile which runs UpdateAccountPeers; the resulting
// NetworkMap delivered to the client carries the synth DNS record
// pointing at K1's IP. ---
provider, err := agentMgr.CreateProvider(ctx, adminUserID, &agenttypes.Provider{
AccountID: accountID,
ProviderID: "openai_api",
Name: "openai-test",
UpstreamURL: "https://api.openai.com",
APIKey: "sk-test-key",
Enabled: true,
Models: []agenttypes.ProviderModel{{ID: "gpt-5.4"}},
}, clusterAddr)
require.NoError(t, err, "CreateProvider must succeed")
_, err = agentMgr.CreatePolicy(ctx, adminUserID, &agenttypes.Policy{
AccountID: accountID,
Name: "p1",
Enabled: true,
SourceGroups: []string{groupAID},
DestinationProviderIDs: []string{provider.ID},
})
require.NoError(t, err, "CreatePolicy must succeed")
settings, err := am.Store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, accountID)
require.NoError(t, err)
fqdn := settings.Endpoint()
rdata1 := awaitZoneRData(clientCh, clusterAddr, fqdn, true)
require.Equal(t, proxyIP1, rdata1,
"client must receive a synth DNS record pointing at K1's overlay IP after the synth path runs")
drain(clientCh)
// --- Proxy restart: NEW keypair K2, same account, same cluster ---
require.NoError(t, peersMgr.CreateProxyPeer(ctx, accountID, proxyKey2, clusterAddr),
"second CreateProxyPeer (proxy restart with fresh keypair) must succeed")
peer2ID, err := am.Store.GetPeerIDByKey(ctx, store.LockingStrengthNone, proxyKey2)
require.NoError(t, err, "proxy peer for K2 must be persisted after restart")
require.NotEmpty(t, peer2ID)
require.NoError(t, am.MarkPeerConnected(ctx, proxyKey2, accountID, time.Now().UnixNano(), &types.NetworkMap{}),
"MarkPeerConnected for K2 must succeed")
// In production the agent's sync stream pulls a fresh NetworkMap as
// part of its normal reconcile cadence; in this isolated test
// MarkPeerConnected's affected-peer fan-out can race the channel-side
// buffer in a way that swallows the synth-DNS-bearing update before
// our await reads it. Trigger an explicit account-wide fan-out so the
// assertion below tests what production actually delivers, not the
// in-test buffer race.
am.UpdateAccountPeers(ctx, accountID, types.UpdateReason{Resource: types.UpdateResourcePeer, Operation: types.UpdateOperationUpdate})
account, err = am.Store.GetAccount(ctx, accountID)
require.NoError(t, err)
proxyIP2 := account.Peers[peer2ID].IP.String()
require.NotEmpty(t, proxyIP2, "K2 must have an assigned overlay IP")
require.NotEqual(t, proxyIP1, proxyIP2, "K2 must get a different overlay IP than K1 (sanity)")
// CRITICAL ASSERTION 1: K1 must no longer be in the store. The SqlStore
// returns ("", nil) for a missing key rather than NotFound, so assert
// on the returned ID being empty.
staleID, err := am.Store.GetPeerIDByKey(ctx, store.LockingStrengthNone, proxyKey1)
require.NoError(t, err, "GetPeerIDByKey for a missing peer must not error")
assert.Empty(t, staleID,
"stale embedded proxy peer K1 must be removed when a new embedded peer registers for the same (account, cluster); pre-fix this assertion fails because management never cleans up the prior peer record")
// CRITICAL ASSERTION 2: exactly one embedded proxy peer remains, and it
// is K2.
account, err = am.Store.GetAccount(ctx, accountID)
require.NoError(t, err)
embeddedKeys := []string{}
for _, p := range account.Peers {
if p.ProxyMeta.Embedded {
embeddedKeys = append(embeddedKeys, p.Key)
}
}
assert.Equal(t, []string{proxyKey2}, embeddedKeys,
"after a proxy restart exactly one embedded proxy peer should remain — the one with the new key K2")
// CRITICAL ASSERTION 3: the synth DNS record the client receives now
// points at K2's IP, not K1's.
rdata2 := awaitZoneRData(clientCh, clusterAddr, fqdn, true)
assert.Equal(t, proxyIP2, rdata2,
"after proxy restart, the client's synth DNS record must point at the NEW embedded peer's IP, not the stale K1 IP")
}

View File

@@ -286,21 +286,6 @@ func (h *handler) updateAccountRequestSettings(req api.PutApiAccountsAccountIdJS
if req.Settings.MetricsPushEnabled != nil {
returnSettings.MetricsPushEnabled = *req.Settings.MetricsPushEnabled
}
if req.Settings.AgentNetworkOnly != nil {
returnSettings.AgentNetworkOnly = *req.Settings.AgentNetworkOnly
}
if req.Settings.DashboardFeatures != nil {
returnSettings.DashboardFeatures = &types.DashboardFeatures{
AgentNetwork: req.Settings.DashboardFeatures.AgentNetwork,
}
}
if returnSettings.AgentNetworkOnly &&
(returnSettings.DashboardFeatures == nil ||
returnSettings.DashboardFeatures.AgentNetwork == nil ||
!*returnSettings.DashboardFeatures.AgentNetwork) {
return nil, status.Errorf(status.InvalidArgument, "agent network only mode requires dashboard_features.agent_network to be enabled")
}
return returnSettings, nil
}
@@ -432,7 +417,6 @@ func toAccountResponse(accountID string, settings *types.Settings, meta *types.A
AutoUpdateAlways: &settings.AutoUpdateAlways,
Ipv6EnabledGroups: &settings.IPv6EnabledGroups,
MetricsPushEnabled: &settings.MetricsPushEnabled,
AgentNetworkOnly: &settings.AgentNetworkOnly,
EmbeddedIdpEnabled: &settings.EmbeddedIdpEnabled,
LocalAuthDisabled: &settings.LocalAuthDisabled,
LocalMfaEnabled: &settings.LocalMfaEnabled,
@@ -446,11 +430,6 @@ func toAccountResponse(accountID string, settings *types.Settings, meta *types.A
networkRangeV6Str := settings.NetworkRangeV6.String()
apiSettings.NetworkRangeV6 = &networkRangeV6Str
}
if settings.DashboardFeatures != nil {
apiSettings.DashboardFeatures = &api.AccountDashboardFeatures{
AgentNetwork: settings.DashboardFeatures.AgentNetwork,
}
}
apiOnboarding := api.AccountOnboarding{
OnboardingFlowPending: onboarding.OnboardingFlowPending,

View File

@@ -130,7 +130,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
@@ -159,7 +158,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
@@ -188,7 +186,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr("latest"),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
@@ -217,7 +214,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
@@ -246,7 +242,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
@@ -275,109 +270,6 @@ func TestAccounts_AccountsHandler(t *testing.T) {
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
},
expectedArray: false,
expectedID: accountID,
},
{
name: "PutAccount OK enabling agent_network_only",
expectedBody: true,
requestType: http.MethodPut,
requestPath: "/api/accounts/" + accountID,
requestBody: bytes.NewBufferString("{\"settings\": {\"peer_login_expiration\": 15552000,\"peer_login_expiration_enabled\": true,\"agent_network_only\": true,\"dashboard_features\": {\"agent_network\": true}},\"onboarding\": {\"onboarding_flow_pending\": true,\"signup_form_pending\": true}}"),
expectedStatus: http.StatusOK,
expectedSettings: api.AccountSettings{
PeerLoginExpiration: 15552000,
PeerLoginExpirationEnabled: true,
GroupsPropagationEnabled: br(false),
JwtGroupsClaimName: sr(""),
JwtGroupsEnabled: br(false),
JwtAllowGroups: &[]string{},
RegularUsersViewBlocked: false,
RoutingPeerDnsResolutionEnabled: br(false),
LazyConnectionEnabled: br(false),
DnsDomain: sr(""),
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(true),
DashboardFeatures: &api.AccountDashboardFeatures{
AgentNetwork: br(true),
},
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
},
expectedArray: false,
expectedID: accountID,
},
{
name: "PutAccount fails enabling agent_network_only without dashboard_features",
expectedBody: true,
requestType: http.MethodPut,
requestPath: "/api/accounts/" + accountID,
requestBody: bytes.NewBufferString("{\"settings\": {\"peer_login_expiration\": 15552000,\"peer_login_expiration_enabled\": true,\"agent_network_only\": true},\"onboarding\": {\"onboarding_flow_pending\": true,\"signup_form_pending\": true}}"),
expectedStatus: http.StatusUnprocessableEntity,
expectedArray: false,
},
{
name: "PutAccount OK setting dashboard_features agent_network",
expectedBody: true,
requestType: http.MethodPut,
requestPath: "/api/accounts/" + accountID,
requestBody: bytes.NewBufferString("{\"settings\": {\"peer_login_expiration\": 15552000,\"peer_login_expiration_enabled\": true,\"dashboard_features\": {\"agent_network\": true}},\"onboarding\": {\"onboarding_flow_pending\": true,\"signup_form_pending\": true}}"),
expectedStatus: http.StatusOK,
expectedSettings: api.AccountSettings{
PeerLoginExpiration: 15552000,
PeerLoginExpirationEnabled: true,
GroupsPropagationEnabled: br(false),
JwtGroupsClaimName: sr(""),
JwtGroupsEnabled: br(false),
JwtAllowGroups: &[]string{},
RegularUsersViewBlocked: false,
RoutingPeerDnsResolutionEnabled: br(false),
LazyConnectionEnabled: br(false),
DnsDomain: sr(""),
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
DashboardFeatures: &api.AccountDashboardFeatures{
AgentNetwork: br(true),
},
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),
},
expectedArray: false,
expectedID: accountID,
},
{
name: "PutAccount OK disabling agent_network_only again",
expectedBody: true,
requestType: http.MethodPut,
requestPath: "/api/accounts/" + accountID,
requestBody: bytes.NewBufferString("{\"settings\": {\"peer_login_expiration\": 15552000,\"peer_login_expiration_enabled\": true,\"agent_network_only\": false},\"onboarding\": {\"onboarding_flow_pending\": true,\"signup_form_pending\": true}}"),
expectedStatus: http.StatusOK,
expectedSettings: api.AccountSettings{
PeerLoginExpiration: 15552000,
PeerLoginExpirationEnabled: true,
GroupsPropagationEnabled: br(false),
JwtGroupsClaimName: sr(""),
JwtGroupsEnabled: br(false),
JwtAllowGroups: &[]string{},
RegularUsersViewBlocked: false,
RoutingPeerDnsResolutionEnabled: br(false),
LazyConnectionEnabled: br(false),
DnsDomain: sr(""),
AutoUpdateAlways: br(false),
AutoUpdateVersion: sr(""),
MetricsPushEnabled: br(false),
AgentNetworkOnly: br(false),
EmbeddedIdpEnabled: br(false),
LocalAuthDisabled: br(false),
LocalMfaEnabled: br(false),

View File

@@ -106,13 +106,11 @@ func (am *DefaultAccountManager) MarkPeerConnected(ctx context.Context, peerPubK
}
if !updated {
am.metrics.AccountManagerMetrics().CountPeerStatusUpdate(telemetry.PeerStatusConnect, telemetry.PeerStatusStale)
log.WithContext(ctx).Debugf("peer %s already has a newer session in store, skipping connect", peer.ID)
log.WithContext(ctx).Tracef("peer %s already has a newer session in store, skipping connect", peer.ID)
return nil
}
am.metrics.AccountManagerMetrics().CountPeerStatusUpdate(telemetry.PeerStatusConnect, telemetry.PeerStatusApplied)
log.WithContext(ctx).Debugf("mark peer %s connected", peer.ID)
if err = am.schedulePeerExpirations(ctx, accountID, peer); err != nil {
return err
}
@@ -182,14 +180,12 @@ func (am *DefaultAccountManager) MarkPeerDisconnected(ctx context.Context, peerP
}
if !updated {
am.metrics.AccountManagerMetrics().CountPeerStatusUpdate(telemetry.PeerStatusDisconnect, telemetry.PeerStatusStale)
log.WithContext(ctx).Debugf("peer %s session token mismatch on disconnect (token=%d), skipping",
log.WithContext(ctx).Tracef("peer %s session token mismatch on disconnect (token=%d), skipping",
peer.ID, sessionStartedAt)
return nil
}
am.metrics.AccountManagerMetrics().CountPeerStatusUpdate(telemetry.PeerStatusDisconnect, telemetry.PeerStatusApplied)
log.WithContext(ctx).Debugf("mark peer %s disconnected", peer.ID)
// Symmetric with MarkPeerConnected: when an embedded proxy peer goes
// offline, refresh the peers that had synthesized records pointing at
// it so they pull the stale entries instead of waiting out TTL.

View File

@@ -1605,8 +1605,7 @@ func (s *SqlStore) getAccount(ctx context.Context, accountID string) (*types.Acc
settings_jwt_groups_enabled, settings_jwt_groups_claim_name, settings_jwt_allow_groups,
settings_routing_peer_dns_resolution_enabled, settings_dns_domain, settings_network_range,
settings_network_range_v6, settings_ipv6_enabled_groups, settings_lazy_connection_enabled,
settings_local_mfa_enabled, settings_metrics_push_enabled, settings_agent_network_only,
settings_dashboard_features,
settings_local_mfa_enabled, settings_metrics_push_enabled,
-- Embedded ExtraSettings
settings_extra_peer_approval_enabled, settings_extra_user_approval_required,
settings_extra_integrated_validator, settings_extra_integrated_validator_groups
@@ -1630,8 +1629,6 @@ func (s *SqlStore) getAccount(ctx context.Context, accountID string) (*types.Acc
sLazyConnectionEnabled sql.NullBool
sLocalMFAEnabled sql.NullBool
sMetricsPushEnabled sql.NullBool
sAgentNetworkOnly sql.NullBool
sDashboardFeatures sql.NullString
sExtraPeerApprovalEnabled sql.NullBool
sExtraUserApprovalRequired sql.NullBool
sExtraIntegratedValidator sql.NullString
@@ -1654,8 +1651,7 @@ func (s *SqlStore) getAccount(ctx context.Context, accountID string) (*types.Acc
&sJWTGroupsEnabled, &sJWTGroupsClaimName, &sJWTAllowGroups,
&sRoutingPeerDNSResolutionEnabled, &sDNSDomain, &sNetworkRange,
&sNetworkRangeV6, &sIPv6EnabledGroups, &sLazyConnectionEnabled,
&sLocalMFAEnabled, &sMetricsPushEnabled, &sAgentNetworkOnly,
&sDashboardFeatures,
&sLocalMFAEnabled, &sMetricsPushEnabled,
&sExtraPeerApprovalEnabled, &sExtraUserApprovalRequired,
&sExtraIntegratedValidator, &sExtraIntegratedValidatorGroups,
)
@@ -1724,14 +1720,6 @@ func (s *SqlStore) getAccount(ctx context.Context, accountID string) (*types.Acc
if sMetricsPushEnabled.Valid {
account.Settings.MetricsPushEnabled = sMetricsPushEnabled.Bool
}
if sAgentNetworkOnly.Valid {
account.Settings.AgentNetworkOnly = sAgentNetworkOnly.Bool
}
if sDashboardFeatures.Valid && sDashboardFeatures.String != "" {
if err := json.Unmarshal([]byte(sDashboardFeatures.String), &account.Settings.DashboardFeatures); err != nil {
log.WithContext(ctx).Warnf("failed to unmarshal dashboard features for account %s: %v", accountID, err)
}
}
if sJWTAllowGroups.Valid {
_ = json.Unmarshal([]byte(sJWTAllowGroups.String), &account.Settings.JWTAllowGroups)
}

View File

@@ -1245,61 +1245,6 @@ func TestSqlite_CreateAndGetObjectInTransaction(t *testing.T) {
assert.NoError(t, err)
}
func TestSqlStore_SaveAccountPersistsAgentNetworkOnly(t *testing.T) {
store, cleanup, err := NewTestStoreFromSQL(context.Background(), "../testdata/store.sql", t.TempDir())
t.Cleanup(cleanup)
require.NoError(t, err)
accountID := "bf1c8084-ba50-4ce7-9439-34653001fc3b"
account, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.False(t, account.Settings.AgentNetworkOnly, "setting should default to false")
account.Settings.AgentNetworkOnly = true
require.NoError(t, store.SaveAccount(context.Background(), account))
reloaded, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.True(t, reloaded.Settings.AgentNetworkOnly, "setting should survive a save/load round-trip")
reloaded.Settings.AgentNetworkOnly = false
require.NoError(t, store.SaveAccount(context.Background(), reloaded))
disabled, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.False(t, disabled.Settings.AgentNetworkOnly, "disabling should persist")
}
func TestSqlStore_SaveAccountPersistsDashboardFeatures(t *testing.T) {
store, cleanup, err := NewTestStoreFromSQL(context.Background(), "../testdata/store.sql", t.TempDir())
t.Cleanup(cleanup)
require.NoError(t, err)
accountID := "bf1c8084-ba50-4ce7-9439-34653001fc3b"
account, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.Nil(t, account.Settings.DashboardFeatures, "dashboard features should default to unset")
agentNetwork := true
account.Settings.DashboardFeatures = &types.DashboardFeatures{AgentNetwork: &agentNetwork}
require.NoError(t, store.SaveAccount(context.Background(), account))
reloaded, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.NotNil(t, reloaded.Settings.DashboardFeatures, "dashboard features should survive a save/load round-trip")
require.NotNil(t, reloaded.Settings.DashboardFeatures.AgentNetwork, "agent network flag should be set")
require.True(t, *reloaded.Settings.DashboardFeatures.AgentNetwork, "agent network flag should persist as true")
disabled := false
reloaded.Settings.DashboardFeatures = &types.DashboardFeatures{AgentNetwork: &disabled}
require.NoError(t, store.SaveAccount(context.Background(), reloaded))
reloadedDisabled, err := store.GetAccount(context.Background(), accountID)
require.NoError(t, err)
require.NotNil(t, reloadedDisabled.Settings.DashboardFeatures.AgentNetwork, "agent network flag should remain set")
require.False(t, *reloadedDisabled.Settings.DashboardFeatures.AgentNetwork, "explicit false should persist")
}
func TestSqlStore_GetAccountUsers(t *testing.T) {
store, cleanup, err := NewTestStoreFromSQL(context.Background(), "../testdata/extended-store.sql", t.TempDir())
t.Cleanup(cleanup)

View File

@@ -7,7 +7,6 @@ import (
"slices"
"strconv"
"strings"
"sync"
"time"
"github.com/netbirdio/netbird/client/ssh/auth"
@@ -43,14 +42,6 @@ type NetworkMapComponents struct {
PostureFailedPeers map[string]map[string]struct{}
RouterPeers map[string]*nbpeer.Peer
routesByPeerOnce sync.Once
routesByPeerIdx map[string][]routeIndexEntry
}
type routeIndexEntry struct {
route *route.Route
viaGroup bool
}
type AccountSettingsInfo struct {
@@ -539,43 +530,33 @@ func (c *NetworkMapComponents) getRoutingPeerRoutes(peerID string) (enabledRoute
disabledRoutes = append(disabledRoutes, r)
}
for _, entry := range c.routesByPeer()[peerID] {
if entry.viaGroup {
newPeerRoute := entry.route.Copy()
newPeerRoute.PeerGroups = nil
newPeerRoute.ID = route.ID(string(entry.route.ID) + ":" + peerID)
takeRoute(newPeerRoute)
continue
for _, r := range c.Routes {
for _, groupID := range r.PeerGroups {
group := c.GetGroupInfo(groupID)
if group == nil {
continue
}
for _, id := range group.Peers {
if id != peerID {
continue
}
newPeerRoute := r.Copy()
newPeerRoute.Peer = id
newPeerRoute.PeerGroups = nil
newPeerRoute.ID = route.ID(string(r.ID) + ":" + id)
takeRoute(newPeerRoute)
break
}
}
if r.Peer == peerID {
takeRoute(r.Copy())
}
takeRoute(entry.route.Copy())
}
return enabledRoutes, disabledRoutes
}
func (c *NetworkMapComponents) routesByPeer() map[string][]routeIndexEntry {
c.routesByPeerOnce.Do(func() {
idx := make(map[string][]routeIndexEntry)
for _, r := range c.Routes {
for _, groupID := range r.PeerGroups {
group := c.GetGroupInfo(groupID)
if group == nil {
continue
}
for _, id := range group.Peers {
idx[id] = append(idx[id], routeIndexEntry{route: r, viaGroup: true})
}
}
if r.Peer != "" {
idx[r.Peer] = append(idx[r.Peer], routeIndexEntry{route: r})
}
}
c.routesByPeerIdx = idx
})
return c.routesByPeerIdx
}
func (c *NetworkMapComponents) filterRoutesByGroups(routes []*route.Route, groupListMap LookupMap) []*route.Route {
var filteredRoutes []*route.Route
for _, r := range routes {

View File

@@ -76,15 +76,6 @@ type Settings struct {
// MetricsPushEnabled globally enables or disables client metrics push for the account
MetricsPushEnabled bool `gorm:"default:false"`
// AgentNetworkOnly limits the dashboard to the Agent Network surface for this account.
// Set for accounts created via netbird.ai signups; users can disable it later.
AgentNetworkOnly bool `gorm:"default:false"`
// DashboardFeatures holds per-account dashboard section visibility overrides.
// It serializes to a single JSON column so new sections can be added without
// a schema change.
DashboardFeatures *DashboardFeatures `gorm:"serializer:json"`
// EmbeddedIdpEnabled indicates if the embedded identity provider is enabled.
// This is a runtime-only field, not stored in the database.
EmbeddedIdpEnabled bool `gorm:"-"`
@@ -123,7 +114,6 @@ func (s *Settings) Copy() *Settings {
AutoUpdateAlways: s.AutoUpdateAlways,
IPv6EnabledGroups: slices.Clone(s.IPv6EnabledGroups),
MetricsPushEnabled: s.MetricsPushEnabled,
AgentNetworkOnly: s.AgentNetworkOnly,
EmbeddedIdpEnabled: s.EmbeddedIdpEnabled,
LocalAuthDisabled: s.LocalAuthDisabled,
LocalMfaEnabled: s.LocalMfaEnabled,
@@ -131,31 +121,9 @@ func (s *Settings) Copy() *Settings {
if s.Extra != nil {
settings.Extra = s.Extra.Copy()
}
if s.DashboardFeatures != nil {
settings.DashboardFeatures = s.DashboardFeatures.Copy()
}
return settings
}
// DashboardFeatures holds per-account dashboard section visibility overrides.
// Nil fields are unset and follow the default dashboard behavior; an explicit
// value forces that section shown or hidden for the account.
type DashboardFeatures struct {
// AgentNetwork, when set, forces the Agent Network menu shown (true) or
// hidden (false) regardless of the deployment feature flag.
AgentNetwork *bool `json:"agent_network,omitempty"`
}
// Copy returns a deep copy of the DashboardFeatures struct.
func (d *DashboardFeatures) Copy() *DashboardFeatures {
c := &DashboardFeatures{}
if d.AgentNetwork != nil {
v := *d.AgentNetwork
c.AgentNetwork = &v
}
return c
}
type ExtraSettings struct {
// PeerApprovalEnabled enables or disables the need for peers bo be approved by an administrator
PeerApprovalEnabled bool

View File

@@ -25,14 +25,6 @@ const (
denyCodeModel = "llm_policy.model_blocked"
denyReasonModel = "model_blocked"
denyMessageModel = "model is not in the policy allowlist"
// Deny reason used when an allowlist is configured but the request model
// could not be determined. URL/path-routed providers (AWS Bedrock, Google
// Vertex, ...) carry the model outside the JSON body, so a request shape the
// parser does not recognise reaches the guardrail with no model. Such a
// request must be denied (fail closed), never waved through.
denyCodeModelUnknown = "llm_policy.model_unknown"
denyReasonModelUnknown = "model_unknown"
denyMessageModelUnknown = "request model could not be determined for the policy allowlist"
)
// Middleware enforces the model allowlist and optionally captures the
@@ -116,37 +108,23 @@ func (m *Middleware) evaluateAllowlist(model string, modelPresent bool) *middlew
if len(m.cfg.ModelAllowlist) == 0 {
return nil
}
// Fail closed: with an allowlist configured, a request whose model the
// upstream parser could not extract (absent or empty) must be denied rather
// than allowed. This is what enforces the allowlist for URL/path-routed
// providers (Bedrock, Vertex, ...) whose model lives outside the JSON body.
if !modelPresent || normaliseModel(model) == "" {
return denyModel("", denyCodeModelUnknown, denyMessageModelUnknown, denyReasonModelUnknown)
if !modelPresent {
return nil
}
if m.modelInAllowlist(model) {
return nil
}
return denyModel(model, denyCodeModel, denyMessageModel, denyReasonModel)
}
// denyModel builds a 403 deny Output for a model-allowlist rejection. model is
// included in the details only when non-empty.
func denyModel(model, code, message, reason string) *middleware.Output {
details := map[string]string{}
if model != "" {
details["model"] = model
}
return &middleware.Output{
Decision: middleware.DecisionDeny,
DenyStatus: 403,
DenyReason: &middleware.DenyReason{
Code: code,
Message: message,
Details: details,
Code: denyCodeModel,
Message: denyMessageModel,
Details: map[string]string{"model": model},
},
Metadata: []middleware.KV{
{Key: middleware.KeyLLMPolicyDecision, Value: "deny"},
{Key: middleware.KeyLLMPolicyReason, Value: reason},
{Key: middleware.KeyLLMPolicyReason, Value: denyReasonModel},
},
}
}

View File

@@ -102,44 +102,13 @@ func TestAllowlistCaseInsensitive(t *testing.T) {
}
}
func TestAllowlistMissingModelKeyDenies(t *testing.T) {
// Fail closed: with an allowlist configured, a request whose model the
// parser could not extract (URL/path-routed providers such as Bedrock or
// Vertex whose shape wasn't recognised) must be denied, not allowed.
func TestAllowlistMissingModelKeyAllows(t *testing.T) {
mw := New(Config{ModelAllowlist: []string{"gpt-4o"}})
out, err := mw.Invoke(context.Background(), newInput())
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionDeny, out.Decision, "absent model must be denied when an allowlist is set")
assert.Equal(t, 403, out.DenyStatus, "deny status must be 403")
require.NotNil(t, out.DenyReason, "deny reason must be populated")
assert.Equal(t, "llm_policy.model_unknown", out.DenyReason.Code, "deny code must be model_unknown")
assert.Equal(t, middleware.DecisionAllow, out.Decision, "missing model key must allow even with non-empty allowlist")
dec, _ := metaValue(t, out.Metadata, middleware.KeyLLMPolicyDecision)
assert.Equal(t, "deny", dec, "decision must be deny when model key is absent")
reason, _ := metaValue(t, out.Metadata, middleware.KeyLLMPolicyReason)
assert.Equal(t, "model_unknown", reason, "reason metadata must be model_unknown")
}
func TestAllowlistEmptyModelValueDenies(t *testing.T) {
// A present-but-empty model is as undeterminable as an absent one.
mw := New(Config{ModelAllowlist: []string{"gpt-4o"}})
out, err := mw.Invoke(context.Background(), newInput(
middleware.KV{Key: middleware.KeyLLMModel, Value: " "},
))
require.NoError(t, err)
require.NotNil(t, out)
assert.Equal(t, middleware.DecisionDeny, out.Decision, "empty model must be denied when an allowlist is set")
require.NotNil(t, out.DenyReason, "deny reason must be populated")
assert.Equal(t, "llm_policy.model_unknown", out.DenyReason.Code, "deny code must be model_unknown")
}
func TestAllowlistEmptyListAllowsMissingModel(t *testing.T) {
// Without an allowlist there is nothing to enforce, so a missing model is
// still allowed — the fail-closed rule only applies when a list is set.
mw := New(Config{})
out, err := mw.Invoke(context.Background(), newInput())
require.NoError(t, err)
assert.Equal(t, middleware.DecisionAllow, out.Decision, "no allowlist must allow even without a model")
assert.Equal(t, "allow", dec, "decision must be allow when model key is absent")
}
func TestPromptCaptureDisabledEmitsNoPrompt(t *testing.T) {

View File

@@ -1,106 +0,0 @@
package llm_request_parser
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/proxy/internal/middleware"
"github.com/netbirdio/netbird/proxy/internal/middleware/builtin/llm_guardrail"
)
// runParserGuardrail runs the request parser then the model-allowlist guardrail
// in SlotOnRequest order, threading the parser's metadata into the guardrail the
// same way the real chain does. It returns the guardrail decision so tests can
// assert allowlist enforcement for URL/path-routed providers end to end.
func runParserGuardrail(t *testing.T, url string, body []byte, allowlist []string) *middleware.Output {
t.Helper()
parser := newMiddleware(t)
parsed, err := parser.Invoke(context.Background(), &middleware.Input{
Slot: middleware.SlotOnRequest,
URL: url,
Body: body,
})
require.NoError(t, err, "parser must not error")
guard := llm_guardrail.New(llm_guardrail.Config{ModelAllowlist: allowlist})
out, err := guard.Invoke(context.Background(), &middleware.Input{
Slot: middleware.SlotOnRequest,
Metadata: parsed.Metadata,
})
require.NoError(t, err, "guardrail must not error")
require.NotNil(t, out, "guardrail must return an output")
return out
}
// TestModelAllowlist_URLRoutedProviders validates that the model allowlist is
// enforced for providers whose model travels in the URL path (AWS Bedrock,
// Google Vertex) rather than the JSON body. The "unknown action" case is the
// regression guard for #6751: a Bedrock request shape the parser cannot map to a
// model must fail closed under an allowlist instead of bypassing it.
func TestModelAllowlist_URLRoutedProviders(t *testing.T) {
const bedrockBody = `{"anthropic_version":"bedrock-2023-05-31","messages":[{"role":"user","content":"hi"}]}`
const vertexBody = `{"anthropic_version":"vertex-2023-10-16","messages":[{"role":"user","content":"hi"}]}`
tests := []struct {
name string
url string
body string
allowlist []string
decision middleware.Decision
denyCode string
}{
{
name: "bedrock allowed model passes",
url: "https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-haiku-4-5-v1:0/invoke",
body: bedrockBody,
allowlist: []string{"anthropic.claude-haiku-4-5"},
decision: middleware.DecisionAllow,
},
{
name: "bedrock disallowed model denied",
url: "https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-opus-4-8-v1:0/invoke",
body: bedrockBody,
allowlist: []string{"anthropic.claude-haiku-4-5"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_blocked",
},
{
name: "bedrock unknown action fails closed",
url: "https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-opus-4-8-v1:0/some-future-action",
body: bedrockBody,
allowlist: []string{"anthropic.claude-haiku-4-5"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_unknown",
},
{
name: "vertex disallowed model denied",
url: "/v1/projects/p/locations/global/publishers/anthropic/models/claude-opus-4-8@20250101:rawPredict",
body: vertexBody,
allowlist: []string{"claude-haiku-4-5"},
decision: middleware.DecisionDeny,
denyCode: "llm_policy.model_blocked",
},
{
name: "vertex allowed model passes",
url: "/v1/projects/p/locations/global/publishers/anthropic/models/claude-haiku-4-5@20250101:rawPredict",
body: vertexBody,
allowlist: []string{"claude-haiku-4-5"},
decision: middleware.DecisionAllow,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
out := runParserGuardrail(t, tt.url, []byte(tt.body), tt.allowlist)
assert.Equal(t, tt.decision, out.Decision, "unexpected decision for %s", tt.name)
if tt.decision == middleware.DecisionDeny {
require.NotNil(t, out.DenyReason, "deny reason must be set for %s", tt.name)
assert.Equal(t, 403, out.DenyStatus, "deny status must be 403 for %s", tt.name)
assert.Equal(t, tt.denyCode, out.DenyReason.Code, "deny code for %s", tt.name)
}
})
}
}

View File

@@ -375,12 +375,6 @@ components:
description: Enables or disables client metrics push for all peers in the account
type: boolean
example: false
agent_network_only:
description: Limits the dashboard to the Agent Network surface for this account. Set for accounts created via netbird.ai signups and can be disabled later. Enabling this requires dashboard_features.agent_network to be true in the same request.
type: boolean
example: false
dashboard_features:
$ref: '#/components/schemas/AccountDashboardFeatures'
embedded_idp_enabled:
description: Indicates whether the embedded identity provider (Dex) is enabled for this account. This is a read-only field.
type: boolean
@@ -409,14 +403,6 @@ components:
- regular_users_view_blocked
- peer_expose_enabled
- peer_expose_groups
AccountDashboardFeatures:
description: Per-account dashboard section visibility overrides. Omitted keys follow the default dashboard behavior.
type: object
properties:
agent_network:
description: Controls the Agent Network menu for the account regardless of the deployment feature flag. When true the menu is shown, when false it is hidden, and when omitted the default behavior applies. Must be true when agent_network_only is enabled.
type: boolean
example: true
AccountExtraSettings:
type: object
properties:
@@ -10504,7 +10490,7 @@ paths:
- EDR Intune Integrations
summary: Delete EDR Intune Integration
description: Deletes an EDR Intune Integration by its ID.
operationId: deleteEDRIntuneIntegration
operationId: deleteIntegration
responses:
'200':
description: Integration deleted successfully. Returns an empty object.
@@ -12588,7 +12574,7 @@ paths:
- Event Streaming Integrations
summary: Delete Event Streaming Integration
description: Deletes an event streaming integration by its ID.
operationId: deleteEventStreamingIntegration
operationId: deleteIntegration
responses:
'200':
description: Integration deleted successfully. Returns an empty object.

View File

@@ -1612,12 +1612,6 @@ type Account struct {
Settings AccountSettings `json:"settings"`
}
// AccountDashboardFeatures Per-account dashboard section visibility overrides. Omitted keys follow the default dashboard behavior.
type AccountDashboardFeatures struct {
// AgentNetwork Controls the Agent Network menu for the account regardless of the deployment feature flag. When true the menu is shown, when false it is hidden, and when omitted the default behavior applies. Must be true when agent_network_only is enabled.
AgentNetwork *bool `json:"agent_network,omitempty"`
}
// AccountExtraSettings defines model for AccountExtraSettings.
type AccountExtraSettings struct {
// NetworkTrafficLogsEnabled Enables or disables network traffic logging. If enabled, all network traffic events from peers will be stored.
@@ -1653,18 +1647,12 @@ type AccountRequest struct {
// AccountSettings defines model for AccountSettings.
type AccountSettings struct {
// AgentNetworkOnly Limits the dashboard to the Agent Network surface for this account. Set for accounts created via netbird.ai signups and can be disabled later. Enabling this requires dashboard_features.agent_network to be true in the same request.
AgentNetworkOnly *bool `json:"agent_network_only,omitempty"`
// AutoUpdateAlways When true, updates are installed automatically in the background. When false, updates require user interaction from the UI.
AutoUpdateAlways *bool `json:"auto_update_always,omitempty"`
// AutoUpdateVersion Set Clients auto-update version. "latest", "disabled", or a specific version (e.g "0.50.1")
AutoUpdateVersion *string `json:"auto_update_version,omitempty"`
// DashboardFeatures Per-account dashboard section visibility overrides. Omitted keys follow the default dashboard behavior.
DashboardFeatures *AccountDashboardFeatures `json:"dashboard_features,omitempty"`
// DnsDomain Allows to define a custom dns domain for the account
DnsDomain *string `json:"dns_domain,omitempty"`

View File

@@ -10,7 +10,7 @@ import (
const (
earlyMsgTTL = 5 * time.Second
earlyMsgCapacity = 10000
earlyMsgCapacity = 1000
)
// earlyMsgBuffer buffers transport messages that arrive before the corresponding