Compare commits

..

20 Commits

Author SHA1 Message Date
Zoltán Papp
310f9cca3f [client] Reset gRPC channel backoff when the network returns
Dials attempted by the gRPC channel while offline grow its internal
exponential backoff, so after the network came back the reconnect loop sat
in WaitForStateChange until that timer expired. Reset the channel backoff
alongside the retry backoff so the management and signal reconnects dial
immediately.
2026-08-19 17:40:42 +02:00
Zoltán Papp
bd99d89bac [client] Sweep connections on network loss via a shared netevents manager
Losing the last network only flipped the availability state: the dead
management, signal and relay sockets stayed silently connected until their
own timeouts, so the client kept reporting Connected with no network at all.

Introduce client/netevents with a Manager that ties the availability state,
the connection sweeper and the status recorder together, and move the
netstate and netsweep packages under it (netsweep renamed to sweep).
SetNetworkAvailable(false) now also sweeps the registered connections so
their owners redial and the listener reaches the NoNetwork state.

The Android and iOS bindings own a Manager instance and inject it through
the constructors; consumers hold the concrete *Manager whose nil zero value
reports always-online and never sweeps, with interfaces kept only as
parameter contracts. The relay guard settle wait moved into the Manager as
WaitSettled, removing the netevents import from the relay package.
2026-08-19 17:15:57 +02:00
Zoltan Papp
77791b5858 [client] Report network addresses on Android for posture checks (#7235)
Android never reported its local network interfaces, so PeerNetworkRange posture checks could not be evaluated: NetworkAddresses always arrived empty.

net.Interfaces() is unusable on Android 11+ (SELinux blocks netlink), so the addresses are parsed from the interface description the host app already provides via stdnet.ExternalIFaceDiscover. The MAC filter is skipped, mirroring #5906
for iOS, since Android does not expose MACs either and nothing reads Mac server side.
2026-08-19 11:47:57 +02:00
Zoltan Papp
ad98b99fc5 [client] Stop the UI before a silent Windows update and suppress the installer reboot (#7209)
Stop the UI before a silent Windows update and suppress the installer reboot

On silent MSI updates msiexec could reboot the machine on its own. The running UI holds a lock on its own exe, and since msiexec runs as LocalSystem it cannot close the interactive user's UI via Restart Manager, so the MSI scheduled the
file replacement for the next reboot and marked the install restart-required.

Terminate netbird-ui.exe before launching the installer and wait until its image file is released; the existing deferred restart brings it back after the install on every exit path
Run msiexec with /norestart REBOOT=ReallySuppress so it never reboots on its own
Treat exit codes 3010/1641 as success with a warning instead of a failure

---------

Co-authored-by: Viktor Liu <viktor@netbird.io>
2026-08-19 11:41:24 +02:00
Zoltan Papp
6d223042eb [client] Clear stale installer result before starting update (#7204)
* [client] Clear stale installer result before starting update

The installer result file could survive a previous update attempt (e.g.
when the updater wrote it after the restarted daemon already ran its
startup check). A new install attempt left the old file in place, so the
GUI progress window's first GetInstallerResult poll read the outdated
result: a stale success made the GUI quit mid-install, which cancelled
the TriggerUpdate context and aborted the artifact verification; a stale
error surfaced a bogus failure dialog for a succeeding update.

Remove any leftover result file at the start of RunInstallation, before
the download begins, so result watchers only see the current attempt's
outcome.

* [client] Align stale-result warning with log message style
2026-08-19 10:27:06 +02:00
Zoltan Papp
070a0a7bf1 [client, android] Handle network changes without restarting the engine (#7144)
On network changes the client restarted the whole engine. That is heavy-handed and slow: it tears down working state to recover from a transition the engine could handle itself. This replaces the restart with proper network event handling.

Suspend the retry loops while no network is available. Instead of burning through backoff intervals against an unreachable network, the reconnection loops park until the OS reports a usable network again.

Reconnect immediately on a network switch. When the OS hands us a new network, connections bound to the old one are swept and re-dialed right away, rather than waiting for a timeout to notice they are dead.
2026-08-19 10:05:12 +02:00
Zoltan Papp
ecfbd686b8 [client, android] Expose ssh functionality for Android (#7156)
Adds an SSHClient gomobile binding so the Android app can run an SSH session over the tunnel with a PTY, exposed through a listener interface for the in-app terminal.

Server type is auto-detected from the SSH banner, which selects the auth path: JWT device-code flow, NetBird key, or a regular server (NetBird key first, then password). Host keys are verified against the peer registry for NetBird servers and trust-on-first-use for regular ones.
2026-08-18 18:49:13 +02:00
Maycon Santos
d5b283dca8 [management] Refuse a usage limit a one-off setup key cannot honour (#7220)
refuse creating one-off keys without limits set to 1
2026-08-18 18:36:42 +02:00
Viktor Liu
6210399e65 [client] Declare multi-buffer support for the loopback XDP program (#7230) 2026-08-17 13:10:12 +02:00
Viktor Liu
939b686d05 [client] Delete NRPT rules by enumerating the registry instead of a rule count (#7195) 2026-08-17 12:52:17 +02:00
Zoltan Papp
70f192344b [client] Update golang.org/x/mobile to v0.0.0-20260816165457-f98cc9b3c733 (#7229) 2026-08-17 12:10:19 +02:00
Misha Bragin
4e5b632490 [infrastructure] Don't override the dashboard image on enterprise migration (#7206) 2026-08-16 16:40:20 +02:00
Misha Bragin
93e97f4bf1 [doc] Agent network docs update (#7020)
* [docs] Update agent-network docs for management-owned pricing

  The docs still described the retired proxy-side pricing: pricing.Loader,
  pricing_path, MiddlewareDataDir, embedded defaults_pricing.yaml, and the
  symlink-safe Unix loader. Rewrite them for the current design — management
  synthesizes the whole table and ships it in cost_meter's ConfigJSON, so the
  proxy carries no price list and has nothing to reload.
2026-08-15 19:31:49 +02:00
Zoltan Papp
16544dbc58 [client] Pass stored email as login hint from UI and keep it on logout (#7199)
* [client] Pass stored email as login hint from UI and keep it on logout

Follow the CLI pattern: the Wails UI now reads the account email from the
user-owned profile state file and passes it as the OIDC login_hint on login
and session extend, since the daemon-side fallback runs as root and cannot
see the user's state file. Logout no longer deletes the stored email, so a
later login preselects the account at the IdP; profile removal remains the
operation that deletes it.

* [client] Log ignored profile lookup errors in extend-session hint fallback
2026-08-15 11:21:57 +02:00
Zoltan Papp
f458c1f265 [client] Skip IPv6 route tests when the default nexthop is unusable (#7212)
* [client] Skip IPv6 route tests when the default nexthop is unusable

ensureIPv6DefaultRoute treated a successful netlink RouteAdd as proof that
a usable IPv6 nexthop exists. Installing ::/0 via loopback can succeed while
the kernel still rejects that nexthop for a concrete prefix, which surfaced
on ubuntu22/20260810.260 runners as:

    add route to table: netlink add route: invalid argument

Probe the resolved nexthop by installing and removing a discard-prefix route
through the same code path the tests use, and skip when it fails. EEXIST
means the nexthop already carries a route, so it counts as usable.

* [client] Probe the IPv6 nexthop through raw netlink

addRoute swallows EAFNOSUPPORT and EOPNOTSUPP via isOpErr, so a nil return
did not prove the probe route was installed. Call netlink directly so an
unsupported operation skips the test instead of passing as usable.
2026-08-15 10:13:06 +02:00
Viktor Liu
ec6f1b8c27 [client] Rank Windows route candidates by combined route and interface metric (#7210) 2026-08-15 09:06:22 +02:00
Zoltan Papp
2cfe14d7ec [client] Keep account email on Android logout, drop it on profile removal (#7200)
Align Android logout semantics with the desktop UI and CLI: logging out no
longer deletes the stored account email, so the next login passes it as the
OIDC login_hint and the IdP preselects the account. Removing the profile is
now the operation that deletes the email; previously RemoveProfile left the
account file behind, which the fixed-name default profile would have
inherited on recreation.
2026-08-14 18:13:52 +02:00
Eduard Gert
85dd335836 [client] Add CI check for translation key parity (#6852)
English (en) is the source of truth for UI translation keys; the other
nine locales rely on runtime English fallback for any missing key, so a
gap never surfaces in CI. Add a dependency-free Node check that fails
when any locale declared in _index.json does not carry the exact same
key set as en (missing or orphaned keys), wired into a dedicated
UI Translations workflow that runs on locale changes.

Also close the one existing gap the check found: ja was missing
daemon.outdated.download ("Download Latest").

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-14 10:57:11 +02:00
Viktor Liu
5544761b47 [client] Add Windows DNS configuration to the debug bundle (#7196) 2026-08-13 20:07:37 +02:00
Kim Harre
1d372bb634 [infrastructure] Support non-interactive installation in getting-started.sh (#7168) 2026-08-13 18:58:03 +03:00
113 changed files with 6299 additions and 1325 deletions

42
.github/workflows/ui-translations.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: UI Translations
on:
pull_request:
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
- ".github/workflows/ui-translations.yml"
push:
branches:
- main
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
jobs:
check-translations:
name: Check translation key parity
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "22"
# English (en) is the source of truth for translation keys; every other
# locale declared in _index.json must carry the exact same key set.
- name: Check translation key parity
run: node client/ui/i18n/check-translations.mjs

View File

@@ -1,6 +1,6 @@
# NetBird Agent Guidelines
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
network with a control plane. The **agent** (`client/`) runs on user machines as
a privileged daemon and manages the WireGuard interface, routing, firewall, and
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,

View File

@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
## Checklist before submitting a PR
As a critical network service and open-source project, we must enforce a few
As a critical network service and open source project, we must enforce a few
things before submitting a pull request. The
[pull request template](/.github/pull_request_template.md) mirrors this list —
fill it in rather than deleting it.

View File

@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
![CISPA_Logo_BLACK_EN_RZ_RGB (1)](https://user-images.githubusercontent.com/700848/203091324-c6d311a0-22b5-4b05-a288-91cbc6cdcc46.png)
### Acknowledgements
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
### Legal
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.

View File

@@ -14,7 +14,7 @@ Report security issues one of these two ways:
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
- **Email** — `security@netbird.io`.
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
filing a repository report.
### What to include

View File

@@ -26,6 +26,7 @@ import (
"github.com/netbirdio/netbird/client/internal/routemanager"
"github.com/netbirdio/netbird/client/internal/stdnet"
"github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/netevents"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/formatter"
"github.com/netbirdio/netbird/route"
@@ -40,11 +41,6 @@ const (
AnonymizeLevelStrict = nbAnonymize.LevelStrictString
)
// ConnectionListener export internal Listener for mobile
type ConnectionListener interface {
peer.Listener
}
// TunAdapter export internal TunAdapter for mobile
type TunAdapter interface {
device.TunAdapter
@@ -85,6 +81,10 @@ type Client struct {
deviceName string
uiVersion string
networkChangeListener listener.NetworkChangeListener
// netMgr outlives engine restarts: it mirrors the OS connectivity, not
// the engine lifecycle. Run and RunWithoutLogin inject its state and
// sweeper into each new ConnectClient.
netMgr *netevents.Manager
stateMu sync.RWMutex
connectClient *internal.ConnectClient
@@ -148,14 +148,17 @@ func NewClient(androidSDKVersion int, deviceName string, uiVersion string, tunAd
execWorkaround(androidSDKVersion)
net.SetAndroidProtectSocketFn(tunAdapter.ProtectSocket)
system.SetIFaceDiscover(iFaceDiscover)
recorder := peer.NewRecorder("")
return &Client{
deviceName: deviceName,
uiVersion: uiVersion,
tunAdapter: tunAdapter,
iFaceDiscover: iFaceDiscover,
recorder: peer.NewRecorder(""),
recorder: recorder,
ctxCancelLock: &sync.Mutex{},
networkChangeListener: networkChangeListener,
netMgr: netevents.NewManager(recorder),
}
}
@@ -196,7 +199,9 @@ func (c *Client) Run(platformFiles PlatformFiles, urlOpener URLOpener, isAndroid
}
// todo do not throw error in case of cancelled context
ctx = internal.CtxInitState(ctx)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder,
internal.WithNetEvents(c.netMgr))
c.setState(cfg, cacheDir, cfgFile, connectClient)
// This path runs the interactive SSO flow, so reaching here means the peer
// is authenticated again — release the latch Status() reports from. Clear
@@ -237,7 +242,8 @@ func (c *Client) RunWithoutLogin(platformFiles PlatformFiles, dns *DNSList, dnsR
// todo do not throw error in case of cancelled context
ctx = internal.CtxInitState(ctx)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder,
internal.WithNetEvents(c.netMgr))
c.setState(cfg, cacheDir, cfgFile, connectClient)
return connectClient.RunOnAndroid(c.tunAdapter, c.iFaceDiscover, c.networkChangeListener, slices.Clone(dns.items), dnsReadyListener, stateFile, cacheDir)
}
@@ -285,6 +291,26 @@ func (c *Client) GetTunSettings() (*TunSettings, error) {
}, nil
}
// SetNetworkAvailable feeds OS-reported network availability into the client.
// While unavailable, the internal reconnect loops suspend their attempts and
// the connection listener reports NoNetwork instead of Connecting; when
// availability returns, the loops resume immediately with a fresh backoff.
// Losing the last network also sweeps the registered connections: nothing can
// redial while offline, so the stale sockets would otherwise stay silently
// "connected" until their own timeouts and the client would keep reporting
// Connected with no network at all.
func (c *Client) SetNetworkAvailable(available bool) {
c.netMgr.SetNetworkAvailable(available)
}
// NotifyNetworkChange marks the management, signal and relay connections
// stale after the OS switched networks and schedules a sweep that cuts
// whatever has not redialed on the new network by then. The engine and the
// TUN device stay untouched.
func (c *Client) NotifyNetworkChange() {
c.netMgr.NotifyNetworkChange()
}
// DebugBundle generates a debug bundle, uploads it, and returns the upload key.
// It works both with and without a running engine. anonymizeLevel is "default"
// or "strict"; strict also anonymizes internal IP ranges, peer names, and
@@ -525,7 +551,11 @@ func (c *Client) OnUpdatedHostDNS(list *DNSList) error {
// SetConnectionListener set the network connection listener
func (c *Client) SetConnectionListener(listener ConnectionListener) {
c.recorder.SetConnectionListener(listener)
if listener == nil {
c.recorder.RemoveConnectionListener()
return
}
c.recorder.SetConnectionListener(connectionListenerAdapter{listener})
}
// RemoveConnectionListener remove connection listener

View File

@@ -0,0 +1,41 @@
//go:build android
package android
import (
"github.com/netbirdio/netbird/client/internal/peer"
)
// Client state values delivered via ConnectionListener.OnStateChanged,
// re-exported as basic constants so gomobile emits them into the generated
// Java bindings. They mirror peer.ClientState*: append-only, never reorder.
const (
ClientStateDisconnected = int(peer.ClientStateDisconnected)
ClientStateConnected = int(peer.ClientStateConnected)
ClientStateConnecting = int(peer.ClientStateConnecting)
ClientStateDisconnecting = int(peer.ClientStateDisconnecting)
ClientStateNoNetwork = int(peer.ClientStateNoNetwork)
)
// ConnectionListener export internal Listener for mobile. It mirrors
// peer.Listener with OnStateChanged taking a plain int (one of the
// ClientState* constants), because gomobile cannot bind named types.
type ConnectionListener interface {
OnStateChanged(state int)
OnConnected()
OnDisconnected()
OnConnecting()
OnDisconnecting()
OnAddressChanged(string, string)
OnPeersListChanged(int)
}
// connectionListenerAdapter adapts the gomobile-facing ConnectionListener to
// peer.Listener, converting the typed state to the int the binding carries.
type connectionListenerAdapter struct {
ConnectionListener
}
func (a connectionListenerAdapter) OnStateChanged(state peer.ClientState) {
a.ConnectionListener.OnStateChanged(int(state))
}

View File

@@ -191,39 +191,49 @@ func (a *Auth) login(urlOpener URLOpener, isAndroidTV bool) error {
return nil
}
// loginHintSetter is implemented by both concrete flows (PKCE and device code)
// but absent from the OAuthFlow interface, hence the assertion below — the same
// way internal/auth wires it in authenticateWithPKCEFlow.
type loginHintSetter interface {
SetLoginHint(hint string)
}
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, isAndroidTV bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV, profileLoginHint(a.cfgPath))
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}
// An empty hint is deliberate, not a fallback: a fresh or logged-out profile
// leaves the choice to the IdP, which is how accounts get switched.
if a.cfgPath != "" {
if hint := readProfileEmail(a.cfgPath); hint != "" {
if setter, ok := oAuthFlow.(loginHintSetter); ok {
setter.SetLoginHint(hint)
}
}
return runOAuthFlow(a.ctx, oAuthFlow, urlOpener, nil)
}
// profileLoginHint returns the stored account email for the profile at cfgPath.
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
func profileLoginHint(cfgPath string) string {
if cfgPath == "" {
return ""
}
return readProfileEmail(cfgPath)
}
// runOAuthFlow drives an already acquired OAuth flow to a token: requests the
// flow info, presents the verification URL through the opener and waits for
// the browser round-trip. Open is called synchronously — it is what marks the
// surface as opened on the client side, and a fast token's OnLoginSuccess is
// a no-op until it has, so the dismissal would be dropped rather than
// delayed. Openers must therefore not block: they post their UI work and
// return. onWaiting, when set, runs after the URL is shown, right before the
// blocking wait.
func runOAuthFlow(ctx context.Context, flow auth.OAuthFlow, urlOpener URLOpener, onWaiting func()) (*auth.TokenInfo, error) {
flowInfo, err := flow.RequestAuthInfo(ctx)
if err != nil {
return nil, fmt.Errorf("request auth info: %w", err)
}
flowInfo, err := oAuthFlow.RequestAuthInfo(context.TODO())
if err != nil {
return nil, fmt.Errorf("getting a request OAuth flow info failed: %v", err)
urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
if onWaiting != nil {
onWaiting()
}
go urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
tokenInfo, err := oAuthFlow.WaitToken(a.ctx, flowInfo)
tokenInfo, err := flow.WaitToken(ctx, flowInfo)
if err != nil {
return nil, fmt.Errorf("waiting for browser login failed: %v", err)
return nil, fmt.Errorf("wait for token: %w", err)
}
return &tokenInfo, nil

View File

@@ -22,7 +22,8 @@ type Profile struct {
ID string
Name string
// Email is the account this profile last logged in with, "" if it never
// completed an SSO login or was logged out. See profile_state.go.
// completed an SSO login. Kept across logouts; cleared when the profile is
// removed. See profile_state.go.
Email string
IsActive bool
}
@@ -200,11 +201,9 @@ func (pm *ProfileManager) LogoutProfile(id string) error {
return fmt.Errorf("failed to save config: %w", err)
}
// Not fatal: a stale hint costs an account switch, not the logout itself.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to clear stored account email for profile %s: %v", id, err)
}
// The stored account email is kept on purpose, matching the desktop and CLI
// logout semantics: the next login passes it as the login_hint so the IdP
// preselects the account. Removing the profile is what deletes it.
log.Infof("logged out from profile: %s", id)
return nil
}
@@ -224,11 +223,24 @@ func (pm *ProfileManager) RenameProfile(id string, newName string) error {
// RemoveProfile deletes a profile
func (pm *ProfileManager) RemoveProfile(id string) error {
configPath, err := pm.getProfileConfigPath(id)
if err != nil {
return err
}
// Use ServiceManager (removes profile from profiles/ directory)
if err := pm.serviceMgr.RemoveProfile(profilemanager.ID(id), androidUsername); err != nil {
return fmt.Errorf("failed to remove profile: %w", err)
}
// The account file is this package's, not the ServiceManager's, so it must
// go here. The default profile has a fixed filename, so a recreated one
// would otherwise inherit the deleted profile's email as its login_hint.
// Not fatal: the profile itself is gone.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to remove stored account email for profile %s: %v", id, err)
}
log.Infof("removed profile: %s", id)
return nil
}

View File

@@ -0,0 +1,38 @@
//go:build android
package android
import (
"fmt"
"github.com/netbirdio/netbird/client/internal/profilemanager"
)
type prefsStore interface {
Get(namespace string, v any) (bool, error)
Put(namespace string, v any) error
}
type profilePrefs struct {
prefs *profilemanager.Prefs
}
func newProfilePrefs(configDir, profileID string) (*profilePrefs, error) {
if configDir == "" || profileID == "" {
return nil, fmt.Errorf("profile prefs require a config dir and profile ID")
}
pm := NewProfileManager(configDir)
prefs, err := pm.serviceMgr.ProfilePrefs(profilemanager.ID(profileID), androidUsername)
if err != nil {
return nil, fmt.Errorf("resolve profile prefs: %w", err)
}
return &profilePrefs{prefs: prefs}, nil
}
func (p *profilePrefs) Get(namespace string, v any) (bool, error) {
return p.prefs.Get(namespace, v)
}
func (p *profilePrefs) Put(namespace string, v any) error {
return p.prefs.Put(namespace, v)
}

View File

@@ -90,10 +90,10 @@ func writeProfileEmail(configPath string, email string) error {
return nil
}
// removeProfileEmail drops the stored account email. Called on logout: while the
// email is on disk it goes out as a login_hint, which would steer the next login
// straight back into the account just logged out of. Mirrors the desktop UI's
// RemoveProfileState call.
// removeProfileEmail drops the stored account email. Called on profile removal,
// not on logout: a logged-out profile keeps its email so the next login passes
// it as the login_hint, matching the desktop and CLI semantics. Mirrors the
// desktop UI's RemoveProfileState call.
func removeProfileEmail(configPath string) error {
accountPath, err := profileAccountPathFor(configPath)
if err != nil {

View File

@@ -127,10 +127,10 @@ func TestWriteThenReadProfileEmail(t *testing.T) {
t.Fatalf("remove: %v", err)
}
if got := readProfileEmail(configPath); got != "" {
t.Errorf("expected no email after logout, got %q", got)
t.Errorf("expected no email after removal, got %q", got)
}
// Logout may run on a never-logged-in profile, so a second remove must pass.
// Removal may run on a never-logged-in profile, so a second remove must pass.
if err := removeProfileEmail(configPath); err != nil {
t.Fatalf("second remove should be a no-op: %v", err)
}

View File

@@ -0,0 +1,649 @@
//go:build android
package android
import (
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
gossh "golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/profilemanager"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/ssh/detection"
)
const (
sshDialTimeout = 30 * time.Second
sshDetectionTimeout = 5 * time.Second
)
// PasswordRequiredMarker tells Java to prompt for a password and retry. It is
// a string because gomobile flattens errors to their message, so a sentinel
// value would not survive the binding.
const PasswordRequiredMarker = "netbird-ssh-password-required"
// HostKeyUnknownMarker tells Java to show the fingerprint and, on confirmation,
// retry with TrustHostKey set. The presented fingerprint is appended after the
// marker so the prompt can display it and the retry can guard against a key
// that changed between the two connects. Only regular (non-NetBird) servers
// reach this: NetBird peers verify against the registry.
const HostKeyUnknownMarker = "netbird-ssh-hostkey-unknown"
var (
errPasswordRequired = errors.New(PasswordRequiredMarker)
errClientClosed = errors.New("ssh client closed")
)
// errHostKeyUnknown carries the presented fingerprint so Connect can build the
// marker message the Java side parses.
type errHostKeyUnknown struct {
fingerprint string
}
func (e *errHostKeyUnknown) Error() string {
return HostKeyUnknownMarker + ":" + e.fingerprint
}
// SSHTerminalListener receives SSH session events. It is implemented in Java.
//
// All callbacks are invoked from goroutines and may run concurrently with each
// other; the implementation must be safe to call from any thread.
type SSHTerminalListener interface {
OnConnected()
OnData(data []byte)
OnClose(reason string)
OnError(message string)
}
// SSHClient is a NetBird-aware SSH client exposed to Java via gomobile.
//
// It dials through the running NetBird tunnel and runs a standard SSH session
// on top with PTY enabled. Host-key verification uses the NetBird-provided
// peer SSH host keys, identical to the desktop client.
type SSHClient struct {
nb *Client
mu sync.Mutex
listener SSHTerminalListener
urlOpener URLOpener
sshClient *gossh.Client
session *gossh.Session
stdin io.WriteCloser
closed bool
// gen identifies the current connection attempt. Connect and Close bump it,
// so an in-flight dial or a reader left over from a previous connection
// finds itself stale and stays silent instead of publishing OnConnected or
// OnClose for a connection the caller already abandoned.
gen uint64
dialCancel context.CancelFunc
// knownHostsConfigDir and knownHostsProfile locate the TOFU store for
// regular SSH servers in the profile's preferences. Java supplies them,
// since an overlay IP is a different host under a different profile. Empty
// until set: without them a regular server cannot be verified and Connect
// refuses one.
knownHostsConfigDir string
knownHostsProfile string
// trustHostKey carries the fingerprint the user confirmed on a previous
// attempt, so the retry accepts exactly that key and persists it.
trustHostKey string
}
// NewSSHClient creates a new SSH client bound to the running NetBird Client.
func NewSSHClient(c *Client) *SSHClient {
return &SSHClient{nb: c}
}
// SetListener registers the Java listener. Must be called before Connect to
// receive any events.
func (s *SSHClient) SetListener(l SSHTerminalListener) {
s.mu.Lock()
s.listener = l
s.mu.Unlock()
}
// SetURLOpener registers the Java URL opener used to display the device-code
// authorization page in a Custom Tabs window when the target peer requires
// JWT authentication. Must be set before Connect to be effective.
func (s *SSHClient) SetURLOpener(opener URLOpener) {
s.mu.Lock()
s.urlOpener = opener
s.mu.Unlock()
}
// SetKnownHostsStore points the TOFU host-key store at a profile's preferences.
// Must be set before connecting to a regular SSH server; without it such a
// server cannot be verified and Connect refuses one.
func (s *SSHClient) SetKnownHostsStore(configDir, profileID string) {
s.mu.Lock()
s.knownHostsConfigDir = configDir
s.knownHostsProfile = profileID
s.mu.Unlock()
}
// TrustHostKey records the fingerprint the user confirmed for a regular server,
// so the next Connect accepts that exact key and adds it to the known-hosts
// store. Passing a fingerprint that no longer matches makes the connect fail
// rather than trust a key that changed since the prompt.
func (s *SSHClient) TrustHostKey(fingerprint string) {
s.mu.Lock()
s.trustHostKey = fingerprint
s.mu.Unlock()
}
// Connect dials the SSH server through the NetBird tunnel and performs the
// SSH handshake. It auto-detects the server type via SSH banner inspection
// and selects the appropriate authentication path:
//
// - NetBird-SSH server requiring JWT: launches the OAuth 2.0 device-code
// flow, opens the verification URL through the registered URLOpener, and
// uses the resulting token as the SSH password. Host-key verification
// uses the NetBird peer registry.
// - NetBird-SSH server without JWT: authenticates with the NetBird SSH
// private key. Host-key verification uses the NetBird peer registry.
// - Regular SSH server (e.g. OpenSSH): authenticates with the NetBird key
// first (so a user-installed NetBird public key works), then falls back
// to the supplied password if non-empty. Host-key verification is
// trust-on-first-use against the per-profile known-hosts store.
//
// The password parameter is only consulted for regular SSH servers.
func (s *SSHClient) Connect(host string, port int, user, password string) error {
if port < 1 || port > 65535 {
return fmt.Errorf("invalid port: %d", port)
}
cfg, cfgPath, cc := s.nb.authSnapshot()
if cc == nil {
return errors.New("netbird client not running")
}
if cfg == nil {
return errors.New("netbird config not loaded")
}
engine := cc.Engine()
if engine == nil {
return errors.New("netbird engine not available")
}
s.mu.Lock()
s.gen++
gen := s.gen
s.mu.Unlock()
serverType := detectServerType(host, port)
log.Debugf("SSH server type: %s", serverType)
authMethods, hostKeyCallback, err := s.buildAuth(cfg, cfgPath, engine, serverType, password)
if err != nil {
return err
}
clientConfig := &gossh.ClientConfig{
User: user,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: sshDialTimeout,
}
err = s.dialAndHandshake(gen, host, port, clientConfig)
// An unknown host key is a prompt, not a failure: return the marker intact
// (rootCause would unwrap it) so Java can show the fingerprint and retry.
var unknownHost *errHostKeyUnknown
if errors.As(err, &unknownHost) {
return errors.New(unknownHost.Error())
}
// A regular server may still accept a password, so let the caller ask for
// one instead of failing. NetBird servers never use a password, so a
// failure there is genuine.
if err != nil && serverType != detection.ServerTypeNetBirdJWT &&
serverType != detection.ServerTypeNetBirdNoJWT && isAuthFailure(err) &&
passwordCouldHelp(err, password != "") {
return errPasswordRequired
}
if err != nil {
return rootCause(err)
}
return nil
}
// StartSession requests a PTY and starts an interactive shell. Output from
// the session is forwarded to the listener via OnData.
func (s *SSHClient) StartSession(cols, rows int) error {
err := s.startSession(cols, rows)
if err != nil {
log.Infof("SSH: start session failed: %v", err)
return rootCause(err)
}
return nil
}
// Write sends data to the SSH session stdin.
func (s *SSHClient) Write(data []byte) error {
s.mu.Lock()
stdin := s.stdin
s.mu.Unlock()
if stdin == nil {
return errors.New("ssh session not started")
}
if _, err := stdin.Write(data); err != nil {
return fmt.Errorf("write stdin: %w", err)
}
return nil
}
// Resize updates the PTY window size.
func (s *SSHClient) Resize(cols, rows int) error {
s.mu.Lock()
session := s.session
s.mu.Unlock()
if session == nil {
return errors.New("ssh session not started")
}
return session.WindowChange(rows, cols)
}
// Reset makes a closed client usable for another Connect: Close leaves the
// one-shot guard set, and clearing it lets the same client back a reconnect.
func (s *SSHClient) Reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.closed = false
}
// Close terminates the SSH session and underlying connection. Safe to call
// multiple times.
func (s *SSHClient) Close() error {
s.mu.Lock()
s.gen++
if s.dialCancel != nil {
s.dialCancel()
s.dialCancel = nil
}
sshClient := s.sshClient
session := s.session
stdin := s.stdin
s.sshClient = nil
s.session = nil
s.stdin = nil
notify := !s.closed
s.closed = true
listener := s.listener
s.mu.Unlock()
if stdin != nil {
if err := stdin.Close(); err != nil {
log.Debugf("ssh: stdin close: %v", err)
}
}
if session != nil {
if err := session.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: session close: %v", err)
}
}
var firstErr error
if sshClient != nil {
if err := sshClient.Close(); err != nil {
firstErr = err
}
}
if notify && listener != nil {
listener.OnClose("closed by client")
}
return firstErr
}
func (s *SSHClient) startSession(cols, rows int) error {
log.Debugf("SSH: starting session %dx%d", cols, rows)
s.mu.Lock()
sshClient := s.sshClient
gen := s.gen
s.mu.Unlock()
if sshClient == nil {
return errors.New("ssh client not connected")
}
pty, err := nbssh.StartPTYSession(sshClient, cols, rows)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(pty.Session, "stale session")
return errClientClosed
}
s.session = pty.Session
s.stdin = pty.Stdin
s.mu.Unlock()
readerDone := make(chan string, 2)
go func() { readerDone <- s.readLoop(pty.Stdout, "stdout") }()
go func() { readerDone <- s.readLoop(pty.Stderr, "stderr") }()
go func() {
reason := <-readerDone
if second := <-readerDone; reason == "" {
reason = second
}
s.notifyClose(gen, reason)
}()
log.Debug("SSH: session started, shell running")
return nil
}
func (s *SSHClient) buildAuth(cfg *profilemanager.Config, cfgPath string, engine *internal.Engine,
serverType detection.ServerType, password string) ([]gossh.AuthMethod, gossh.HostKeyCallback, error) {
switch serverType {
case detection.ServerTypeNetBirdJWT:
token, err := s.requestJWTToken(cfg, cfgPath)
if err != nil {
return nil, nil, fmt.Errorf("jwt: %w", err)
}
auths := []gossh.AuthMethod{gossh.Password(token)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeNetBirdNoJWT:
if cfg.SSHKey == "" {
return nil, nil, errors.New("no NetBird SSH key available")
}
signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey))
if err != nil {
return nil, nil, fmt.Errorf("parse netbird ssh key: %w", err)
}
auths := []gossh.AuthMethod{gossh.PublicKeys(signer)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeRegular:
var auths []gossh.AuthMethod
if cfg.SSHKey != "" {
if signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey)); err == nil {
auths = append(auths, gossh.PublicKeys(signer))
} else {
log.Debugf("ssh: parse netbird key for regular auth: %v", err)
}
}
if password != "" {
pw := password
auths = append(auths, gossh.Password(pw))
auths = append(auths, gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range questions {
answers[i] = pw
}
return answers, nil
}))
}
if len(auths) == 0 {
// Nothing to offer at all: ask for a password rather than failing,
// so the caller can retry once the user supplies one.
return nil, nil, errPasswordRequired
}
callback, err := s.tofuHostKeyCallback()
if err != nil {
return nil, nil, err
}
return auths, callback, nil
default:
return nil, nil, fmt.Errorf("unsupported SSH server type: %v", serverType)
}
}
// tofuHostKeyCallback verifies a regular server's host key against the
// per-profile known-hosts store. An unknown host returns errHostKeyUnknown so
// Java can show the fingerprint and, once confirmed, retry with the key
// trusted; a changed key is rejected outright, as OpenSSH does. When the user
// has confirmed a fingerprint, the callback accepts exactly that key and
// appends it to the store.
func (s *SSHClient) tofuHostKeyCallback() (gossh.HostKeyCallback, error) {
s.mu.Lock()
configDir := s.knownHostsConfigDir
profileID := s.knownHostsProfile
trusted := s.trustHostKey
s.mu.Unlock()
if configDir == "" || profileID == "" {
return nil, errors.New("no known-hosts store configured for regular SSH")
}
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return nil, fmt.Errorf("load known-hosts store: %w", err)
}
return func(hostname string, remote net.Addr, key gossh.PublicKey) error {
verdict, err := store.verify(hostname, remote, key)
if err != nil {
return err
}
if verdict == hostKeyMatched {
return nil
}
if verdict == hostKeyChanged {
return fmt.Errorf("SSH host key changed for %s (possible attack)", hostname)
}
fingerprint := gossh.FingerprintSHA256(key)
if trusted == "" {
return &errHostKeyUnknown{fingerprint: fingerprint}
}
if trusted != fingerprint {
return fmt.Errorf("SSH host key changed since it was confirmed for %s", hostname)
}
if err := store.append(hostname, remote, key); err != nil {
return fmt.Errorf("persist trusted host key: %w", err)
}
// The confirmation is spent: now that the key is stored, a later
// reconnect must verify against the file, not re-accept this fingerprint.
s.mu.Lock()
s.trustHostKey = ""
s.mu.Unlock()
return nil
}, nil
}
func (s *SSHClient) requestJWTToken(cfg *profilemanager.Config, cfgPath string) (string, error) {
s.mu.Lock()
urlOpener := s.urlOpener
s.mu.Unlock()
if urlOpener == nil {
return "", errors.New("URL opener not configured for JWT auth")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
flow, err := auth.NewOAuthFlow(ctx, cfg, false, true, profileLoginHint(cfgPath))
if err != nil {
return "", fmt.Errorf("create oauth flow: %w", err)
}
// The status callback covers the browser round-trip, which would
// otherwise leave the terminal blank.
tokenInfo, err := runOAuthFlow(ctx, flow, urlOpener, func() {
s.notifyStatus("Waiting for browser authentication...")
})
if err != nil {
return "", err
}
token := tokenInfo.GetTokenToUse()
if token == "" {
return "", errors.New("empty token returned by IdP")
}
// Tells the client the browser round-trip is over so it can dismiss the
// surface it opened, the same way the login and session-extend flows do.
// Without it the Custom Tab stays in front of the terminal even though the
// token has already been collected.
urlOpener.OnLoginSuccess()
return token, nil
}
func (s *SSHClient) dialAndHandshake(gen uint64, host string, port int, clientConfig *gossh.ClientConfig) error {
addr := net.JoinHostPort(host, strconv.Itoa(port))
ctx, cancel := context.WithTimeout(context.Background(), sshDialTimeout)
defer cancel()
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
return errClientClosed
}
s.dialCancel = cancel
s.mu.Unlock()
var dialer net.Dialer
conn, err := dialer.DialContext(ctx, "tcp", addr)
if err != nil {
return fmt.Errorf("dial %s: %w", addr, err)
}
client, err := nbssh.Handshake(ctx, conn, addr, clientConfig)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(client, "stale ssh client")
return errClientClosed
}
s.sshClient = client
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnConnected()
}
return nil
}
func (s *SSHClient) readLoop(r io.Reader, name string) string {
buf := make([]byte, 4096)
for {
n, err := r.Read(buf)
if n > 0 {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
chunk := make([]byte, n)
copy(chunk, buf[:n])
listener.OnData(chunk)
}
}
if err != nil {
// EOF is a normal shell exit, so report it without a reason.
if errors.Is(err, io.EOF) {
return ""
}
log.Debugf("ssh %s read: %v", name, err)
return rootCause(err).Error()
}
}
}
// notifyStatus writes a progress line to the terminal through the normal
// output path, so long steps are visible while nothing else is arriving.
func (s *SSHClient) notifyStatus(text string) {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnData([]byte("\r\n\x1b[33m" + text + "\x1b[0m\r\n"))
}
}
func (s *SSHClient) notifyClose(gen uint64, reason string) {
s.mu.Lock()
if gen != s.gen || s.closed {
s.mu.Unlock()
return
}
s.closed = true
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnClose(reason)
}
}
func closeQuiet(c io.Closer, label string) {
if c == nil {
return
}
if err := c.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: close %s: %v", label, err)
}
}
func detectServerType(host string, port int) detection.ServerType {
ctx, cancel := context.WithTimeout(context.Background(), sshDetectionTimeout)
defer cancel()
dialer := &net.Dialer{}
serverType, err := detection.DetectSSHServerType(ctx, dialer, host, port)
if err != nil {
log.Debugf("ssh: server detection failed: %v (assuming regular SSH)", err)
return detection.ServerTypeRegular
}
return serverType
}
// rootCause returns the innermost error of a %w chain, so the terminal shows
// "i/o timeout" rather than every layer that added context on the way up.
func rootCause(err error) error {
for {
// A joined error has no single root, so keep it as-is.
if _, ok := err.(interface{ Unwrap() []error }); ok {
return err
}
next := errors.Unwrap(err)
if next == nil {
return err
}
err = next
}
}
// isAuthFailure distinguishes credential rejection from dial, timeout and
// host-key errors, which retrying with a password would not fix.
func isAuthFailure(err error) bool {
if errors.Is(err, errPasswordRequired) {
return true
}
var partial *gossh.PartialSuccessError
if errors.As(err, &partial) {
return true
}
return strings.Contains(err.Error(), "unable to authenticate")
}
// passwordCouldHelp reports whether prompting for a password again can change
// the outcome. gossh lists a method under "attempted methods" only when the
// server offered it, so a supplied password that was never attempted means the
// server does not accept passwords and the real error should surface instead.
func passwordCouldHelp(err error, passwordOffered bool) bool {
if !passwordOffered {
return true
}
msg := err.Error()
return strings.Contains(msg, "password") || strings.Contains(msg, "keyboard-interactive")
}

View File

@@ -0,0 +1,168 @@
//go:build android
package android
import (
"bytes"
"net"
"strconv"
"strings"
"sync"
gossh "golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
const knownHostsNamespace = "ssh"
const (
hostKeyUnknown hostKeyVerdict = iota
hostKeyMatched
hostKeyChanged
)
var knownHostsMu sync.Mutex
type hostKeyVerdict uint8
type knownHostsSection struct {
KnownHosts []string `json:"knownHosts"`
}
type knownHostsStore struct {
prefs prefsStore
}
// RemoveKnownHost deletes every known-hosts entry for host:port from the
// profile's store, so a host trusted for a session that is being deleted does
// not linger. Java calls this only once no session targets that host, so a
// shared host stays trusted. A missing entry is not an error: the goal state
// is "absent".
func RemoveKnownHost(configDir, profileID, host string, port int) error {
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return err
}
return store.removeHost(host, port)
}
func openKnownHostsStore(configDir, profileID string) (*knownHostsStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &knownHostsStore{prefs: prefs}, nil
}
func (st *knownHostsStore) verify(hostname string, remote net.Addr, key gossh.PublicKey) (hostKeyVerdict, error) {
lines, err := st.lines()
if err != nil {
return hostKeyUnknown, err
}
targets := knownHostsTargets(hostname, remote)
verdict := hostKeyUnknown
for _, line := range lines {
pubKey, ok := knownHostsLineKey(line, targets)
if !ok {
continue
}
if pubKey.Type() == key.Type() && bytes.Equal(pubKey.Marshal(), key.Marshal()) {
return hostKeyMatched, nil
}
verdict = hostKeyChanged
}
return verdict, nil
}
func (st *knownHostsStore) append(hostname string, remote net.Addr, key gossh.PublicKey) error {
line := knownhosts.Line(knownHostsTargets(hostname, remote), key)
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: append(lines, line)})
}
func (st *knownHostsStore) removeHost(host string, port int) error {
target := knownhosts.Normalize(net.JoinHostPort(host, strconv.Itoa(port)))
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
kept := make([]string, 0, len(lines))
for _, line := range lines {
if knownHostsLineMatches(line, target) {
continue
}
kept = append(kept, line)
}
if len(kept) == len(lines) {
return nil
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: kept})
}
func (st *knownHostsStore) lines() ([]string, error) {
var section knownHostsSection
if _, err := st.prefs.Get(knownHostsNamespace, &section); err != nil {
return nil, err
}
return section.KnownHosts, nil
}
func knownHostsTargets(hostname string, remote net.Addr) []string {
targets := []string{knownhosts.Normalize(hostname)}
if remote != nil {
if normalized := knownhosts.Normalize(remote.String()); normalized != targets[0] {
targets = append(targets, normalized)
}
}
return targets
}
func knownHostsLineKey(line string, targets []string) (gossh.PublicKey, bool) {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return nil, false
}
_, hosts, pubKey, _, _, err := gossh.ParseKnownHosts([]byte(trimmed))
if err != nil {
return nil, false
}
for _, host := range hosts {
for _, target := range targets {
if host == target {
return pubKey, true
}
}
}
return nil, false
}
// knownHostsLineMatches reports whether a known-hosts line's address list
// contains the normalized target. Comment and blank lines never match.
func knownHostsLineMatches(line, target string) bool {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return false
}
fields := strings.Fields(trimmed)
if len(fields) == 0 {
return false
}
for _, addr := range strings.Split(fields[0], ",") {
if addr == target {
return true
}
}
return false
}

View File

@@ -0,0 +1,104 @@
//go:build android
package android
const (
sshSessionsNamespace = "ssh-sessions"
maxStoredSSHSessions = 50
)
type sshSessionRecord struct {
ID string `json:"id"`
Host string `json:"host"`
Port int `json:"port"`
User string `json:"user"`
}
type sshSessionsSection struct {
Sessions []sshSessionRecord `json:"sessions"`
}
// SSHSessionEntry is one stored SSH session, without any credential.
type SSHSessionEntry struct {
ID string
Host string
Port int
User string
}
// SSHSessionArray wraps stored SSH sessions for gomobile compatibility.
type SSHSessionArray struct {
items []*SSHSessionEntry
}
// NewSSHSessionArray creates an empty session array to fill via Add.
func NewSSHSessionArray() *SSHSessionArray {
return &SSHSessionArray{}
}
// Add appends a session entry, oldest first.
func (a *SSHSessionArray) Add(id, host string, port int, user string) {
a.items = append(a.items, &SSHSessionEntry{ID: id, Host: host, Port: port, User: user})
}
// Length returns the number of entries.
func (a *SSHSessionArray) Length() int {
return len(a.items)
}
// Get returns the entry at index i, or nil when out of range.
func (a *SSHSessionArray) Get(i int) *SSHSessionEntry {
if i < 0 || i >= len(a.items) {
return nil
}
return a.items[i]
}
// SSHSessionStore reads and writes a profile's stored SSH sessions.
type SSHSessionStore struct {
prefs prefsStore
}
// NewSSHSessionStore opens the session store of the given profile.
func NewSSHSessionStore(configDir, profileID string) (*SSHSessionStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &SSHSessionStore{prefs: prefs}, nil
}
// Load returns the stored sessions, oldest first.
func (s *SSHSessionStore) Load() (*SSHSessionArray, error) {
var section sshSessionsSection
if _, err := s.prefs.Get(sshSessionsNamespace, &section); err != nil {
return nil, err
}
out := NewSSHSessionArray()
for _, record := range section.Sessions {
if record.ID == "" || record.Host == "" {
continue
}
out.Add(record.ID, record.Host, record.Port, record.User)
}
return out, nil
}
// Save replaces the stored sessions, keeping only the newest entries when the
// list exceeds the storage cap.
func (s *SSHSessionStore) Save(sessions *SSHSessionArray) error {
var items []*SSHSessionEntry
if sessions != nil {
items = sessions.items
}
if len(items) > maxStoredSSHSessions {
items = items[len(items)-maxStoredSSHSessions:]
}
records := make([]sshSessionRecord, 0, len(items))
for _, item := range items {
records = append(records, sshSessionRecord{ID: item.ID, Host: item.Host, Port: item.Port, User: item.User})
}
return s.prefs.Put(sshSessionsNamespace, sshSessionsSection{Sessions: records})
}

View File

@@ -305,6 +305,12 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
return domain
}
// A reverse zone names an address prefix, so it follows the address rules,
// which also keeps its digit labels intact.
if zone, ok := a.anonymizeReverseZone(baseDomain); ok {
return withTrailingDot(zone, hasDot)
}
if suffix := protectedSuffix(baseDomain); suffix != "" {
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
return domain
@@ -405,6 +411,10 @@ func (a *Anonymizer) AnonymizeString(str string) string {
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
// Reverse zones go first and are then held out of the passes below: their
// labels are digits, which the address patterns would otherwise consume.
str, restoreZones := a.replaceReverseZones(str)
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
@@ -425,7 +435,7 @@ func (a *Anonymizer) AnonymizeString(str string) string {
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
}
return str
return restoreZones(str)
}
// sortedDomains returns the domain mappings longest-first, so a full-FQDN

View File

@@ -0,0 +1,174 @@
package anonymize
import (
"encoding/hex"
"net/netip"
"regexp"
"strconv"
"strings"
)
const (
reverseZoneSuffixV4 = ".in-addr.arpa"
reverseZoneSuffixV6 = ".ip6.arpa"
v6Nibbles = 32
v4Octets = 4
)
// reverseZoneRegexes match a reverse zone or a full reverse name in free text.
// They are applied before the address passes of AnonymizeString, whose IPv4
// pattern would otherwise consume the digit labels of a zone and replace parts
// of it with unrelated addresses.
var reverseZoneRegexes = []*regexp.Regexp{
regexp.MustCompile(`(?:[0-9]{1,3}\.){1,4}in-addr\.arpa\b`),
regexp.MustCompile(`(?:[0-9a-fA-F]\.){1,32}ip6\.arpa\b`),
}
// anonymizeReverseZone maps a reverse zone to the zone of the anonymized form
// of the prefix it encodes, so it follows the address rules rather than the
// domain ones: the zone of an address that is preserved is preserved too, and
// the zone of one that is replaced names the replacement. This keeps a reverse
// zone recognizable as such, and consistent with the addresses it belongs to
// elsewhere in the same output. It reports false for anything that is not a
// reverse zone.
func (a *Anonymizer) anonymizeReverseZone(domain string) (string, bool) {
prefix, labelCount, suffix, ok := parseReverseZone(domain)
if !ok {
return "", false
}
anonymized := a.AnonymizeIP(prefix)
if anonymized == prefix {
return domain, true
}
return reverseZoneName(anonymized, labelCount) + suffix, true
}
// replaceReverseZones anonymizes every reverse zone in str and swaps each one
// for a placeholder, returning a function that puts the anonymized zones back.
// The placeholders carry no dots, digits or colons, so no later pass matches
// them.
func (a *Anonymizer) replaceReverseZones(str string) (string, func(string) string) {
var zones []string
for _, re := range reverseZoneRegexes {
str = re.ReplaceAllStringFunc(str, func(match string) string {
zone, ok := a.anonymizeReverseZone(match)
if !ok {
return match
}
zones = append(zones, zone)
return reverseZonePlaceholder(len(zones) - 1)
})
}
if len(zones) == 0 {
return str, func(s string) string { return s }
}
return str, func(s string) string {
for i, zone := range zones {
s = strings.ReplaceAll(s, reverseZonePlaceholder(i), zone)
}
return s
}
}
func reverseZonePlaceholder(index int) string {
return "\x00reversezone" + strconv.Itoa(index) + "\x00"
}
// parseReverseZone turns a reverse zone into the address of the prefix its
// labels spell backwards, padding the absent low-order part with zeroes, and
// returns the label count and zone suffix so the name can be rebuilt.
func parseReverseZone(domain string) (netip.Addr, int, string, bool) {
lower := strings.ToLower(domain)
switch {
case strings.HasSuffix(lower, reverseZoneSuffixV4):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV4), ".")
addr, ok := reverseZoneAddrV4(labels)
return addr, len(labels), reverseZoneSuffixV4, ok
case strings.HasSuffix(lower, reverseZoneSuffixV6):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV6), ".")
addr, ok := reverseZoneAddrV6(labels)
return addr, len(labels), reverseZoneSuffixV6, ok
default:
return netip.Addr{}, 0, "", false
}
}
func reverseZoneAddrV4(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v4Octets {
return netip.Addr{}, false
}
var octets [v4Octets]byte
for i, label := range labels {
octet, err := strconv.ParseUint(label, 10, 8)
if err != nil {
return netip.Addr{}, false
}
octets[len(labels)-1-i] = byte(octet)
}
return netip.AddrFrom4(octets), true
}
func reverseZoneAddrV6(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v6Nibbles {
return netip.Addr{}, false
}
nibbles := make([]byte, 0, v6Nibbles)
for i := len(labels) - 1; i >= 0; i-- {
if len(labels[i]) != 1 || !isHexDigit(labels[i][0]) {
return netip.Addr{}, false
}
nibbles = append(nibbles, labels[i][0])
}
for len(nibbles) < v6Nibbles {
nibbles = append(nibbles, '0')
}
var groups []string
for i := 0; i < len(nibbles); i += 4 {
groups = append(groups, string(nibbles[i:i+4]))
}
addr, err := netip.ParseAddr(strings.Join(groups, ":"))
if err != nil {
return netip.Addr{}, false
}
return addr, true
}
// reverseZoneName spells the first labelCount labels of addr backwards, the
// inverse of parseReverseZone, without the zone suffix.
func reverseZoneName(addr netip.Addr, labelCount int) string {
labels := make([]string, 0, labelCount)
if addr.Is4() {
octets := addr.As4()
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, strconv.Itoa(int(octets[i])))
}
return strings.Join(labels, ".")
}
address := addr.As16()
nibbles := hex.EncodeToString(address[:])
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, string(nibbles[i]))
}
return strings.Join(labels, ".")
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}

View File

@@ -0,0 +1,171 @@
package anonymize
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newLeveledAnonymizer(level Level) *Anonymizer {
a := NewAnonymizer(DefaultAddresses())
a.SetLevel(level)
return a
}
// TestAnonymizeDomainReverseZone covers reverse zones going through the address
// rules instead of the domain ones, so a zone stays a zone and an address that
// is preserved keeps the zone that names it.
func TestAnonymizeDomainReverseZone(t *testing.T) {
// 100.64.0.0/10 is the overlay range, which is CGNAT: preserved at the
// default level and replaced from the internal pool at the strict one
const overlayZone = "64.100.in-addr.arpa"
t.Run("overlay zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, overlayZone, a.AnonymizeDomain(overlayZone), "should keep the zone of a preserved address")
})
t.Run("private zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "168.192.in-addr.arpa", a.AnonymizeDomain("168.192.in-addr.arpa"), "should keep the zone of a private address")
})
t.Run("overlay zone replaced at the strict level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelStrict)
got := a.AnonymizeDomain(overlayZone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, overlayZone, got, "should replace the encoded prefix")
assert.Len(t, strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV4), "."), 2,
"should keep the label count, got %q", got)
})
t.Run("public zone replaced at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("113.0.203.in-addr.arpa")
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, "113.0.203.in-addr.arpa", got, "should replace a public prefix")
})
t.Run("zone of an address keeps that address mapping", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
anonymizedAddr := a.AnonymizeIPString("203.0.113.7")
got := a.AnonymizeDomain("7.113.0.203.in-addr.arpa")
octets := strings.Split(anonymizedAddr, ".")
want := octets[3] + "." + octets[2] + "." + octets[1] + "." + octets[0] + reverseZoneSuffixV4
assert.Equal(t, want, got, "should name the same replacement as the address itself")
})
t.Run("ipv6 nibble labels stay single digits", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeDomain(zone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV6), "should stay a reverse zone, got %q", got)
labels := strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV6), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
for _, label := range labels {
assert.Len(t, label, 1, "nibble label %q should stay a single digit", label)
}
})
t.Run("trailing dot is kept", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "64.100.in-addr.arpa.", a.AnonymizeDomain("64.100.in-addr.arpa."), "should keep the trailing dot")
})
t.Run("a domain that only looks like a zone is anonymized as a domain", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("not-a-zone.in-addr.arpa")
assert.NotContains(t, got, "in-addr.arpa", "should fall back to domain anonymization")
})
}
// TestAnonymizeStringReverseZone verifies that a zone inside free text, such as
// a DNS log line, is not chewed up by the address passes. The IPv4 pattern
// matches any run of dotted digits, which a reverse zone is made of.
func TestAnonymizeStringReverseZone(t *testing.T) {
t.Run("ipv6 zone survives the address passes", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeString("question: domain=" + zone + " type=PTR")
assert.Contains(t, got, "type=PTR", "should keep the rest of the line")
assert.NotContains(t, got, "198.51.100", "should not rewrite nibble labels as an address")
labels := strings.Split(strings.TrimSuffix(strings.TrimPrefix(got, "question: domain="), reverseZoneSuffixV6+" type=PTR"), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
})
t.Run("preserved ipv4 zone is untouched", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
line := "reverse zone 64.100.in-addr.arpa registered"
assert.Equal(t, line, a.AnonymizeString(line), "should keep the zone of a preserved address")
})
t.Run("public ipv4 zone is replaced consistently", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeString("zone 113.0.203.in-addr.arpa and address 203.0.113.7")
assert.NotContains(t, got, "113.0.203.in-addr.arpa", "should replace the zone")
assert.NotContains(t, got, "203.0.113.7", "should replace the address")
assert.Contains(t, got, reverseZoneSuffixV4, "should keep the zone suffix")
})
}
func TestParseReverseZone(t *testing.T) {
tests := []struct {
name string
zone string
addr string
labels int
}{
{name: "v4 two labels", zone: "0.100" + reverseZoneSuffixV4, addr: "100.0.0.0", labels: 2},
{name: "v4 three labels", zone: "1.168.192" + reverseZoneSuffixV4, addr: "192.168.1.0", labels: 3},
{name: "v4 full address", zone: "7.113.0.203" + reverseZoneSuffixV4, addr: "203.0.113.7", labels: 4},
{
name: "v6 prefix",
zone: "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6,
addr: "2::",
labels: 28,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
addr, labels, suffix, ok := parseReverseZone(tc.zone)
require.True(t, ok, "should decode the reverse zone")
assert.Equal(t, tc.addr, addr.String(), "should decode to the encoded prefix")
assert.Equal(t, tc.labels, labels, "should count the labels")
assert.Equal(t, tc.zone, reverseZoneName(addr, labels)+suffix, "should re-encode to the original zone")
})
}
}
func TestParseReverseZoneRejectsNonZones(t *testing.T) {
tests := []string{
"example.com",
"in-addr.arpa",
"x.100" + reverseZoneSuffixV4,
"256" + reverseZoneSuffixV4,
"1.2.3.4.5" + reverseZoneSuffixV4,
"ab" + reverseZoneSuffixV6,
"g" + reverseZoneSuffixV6,
}
for _, zone := range tests {
t.Run(zone, func(t *testing.T) {
_, _, _, ok := parseReverseZone(zone)
assert.False(t, ok, "should reject %q", zone)
})
}
}

View File

@@ -410,7 +410,7 @@ func foregroundGetTokenInfo(ctx context.Context, cmd *cobra.Command, config *pro
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, util.HasGraphicalSession(), false, hint)
if err != nil {
return nil, auth.WithSetupKeyAdvice(err)
return nil, err
}
flowInfo, err := oAuthFlow.RequestAuthInfo(context.TODO())

View File

@@ -21,7 +21,7 @@ import (
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
sshcommon "github.com/netbirdio/netbird/client/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
@@ -521,12 +521,7 @@ func (c *Client) VerifySSHHostKey(peerAddress string, key []byte) error {
return err
}
storedKey, found := engine.GetPeerSSHKey(peerAddress)
if !found {
return sshcommon.ErrPeerNotFound
}
return sshcommon.VerifyHostKey(storedKey, key, peerAddress)
return nbssh.PeerKeyLookup(engine.GetPeerSSHKey).VerifySSHHostKey(peerAddress, key)
}
// SetPerformance retunes a running Client. Only PreallocatedBuffersPerPool

View File

@@ -16,28 +16,52 @@ import (
"google.golang.org/grpc"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/netevents/sweep"
)
// Sweeper registers in-flight dials for the network change sweep.
type Sweeper interface {
StartDial(ctx context.Context) *sweep.Dial
}
func WithCustomDialer(_ bool, _ string) grpc.DialOption {
return grpc.WithContextDialer(dialContext)
}
// WithSweeper dials like WithCustomDialer but registers connections and
// dials with the sweeper. Append it after WithCustomDialer: gRPC applies
// dial options in order, so the later context dialer wins.
func WithSweeper(sweeper Sweeper) grpc.DialOption {
return grpc.WithContextDialer(func(ctx context.Context, addr string) (net.Conn, error) {
if runtime.GOOS == "linux" {
currentUser, err := user.Current()
if err != nil {
return nil, status.Errorf(codes.FailedPrecondition, "failed to get current user: %v", err)
}
dial := sweeper.StartDial(ctx)
defer dial.Release()
// the custom dialer requires root permissions which are not required for use cases run as non-root
if currentUser.Uid != "0" {
log.Debug("Not running as root, using standard dialer")
dialer := &net.Dialer{}
return dialer.DialContext(ctx, "tcp", addr)
}
}
conn, err := nbnet.NewDialer().DialContext(ctx, "tcp", addr)
conn, err := dialContext(dial.Ctx(), addr)
if err != nil {
return nil, fmt.Errorf("nbnet.NewDialer().DialContext: %w", err)
return nil, err
}
return conn, nil
return dial.WrapConn(conn)
})
}
func dialContext(ctx context.Context, addr string) (net.Conn, error) {
if runtime.GOOS == "linux" {
currentUser, err := user.Current()
if err != nil {
return nil, status.Errorf(codes.FailedPrecondition, "failed to get current user: %v", err)
}
// the custom dialer requires root permissions which are not required for use cases run as non-root
if currentUser.Uid != "0" {
log.Debug("Not running as root, using standard dialer")
dialer := &net.Dialer{}
return dialer.DialContext(ctx, "tcp", addr)
}
}
conn, err := nbnet.NewDialer().DialContext(ctx, "tcp", addr)
if err != nil {
return nil, fmt.Errorf("nbnet.NewDialer().DialContext: %w", err)
}
return conn, nil
}

View File

@@ -1,13 +1,26 @@
package grpc
import (
"context"
"google.golang.org/grpc"
"github.com/netbirdio/netbird/client/netevents/sweep"
"github.com/netbirdio/netbird/util/wsproxy/client"
)
// Sweeper registers in-flight dials for the network change sweep.
type Sweeper interface {
StartDial(ctx context.Context) *sweep.Dial
}
// WithCustomDialer returns a gRPC dial option that uses WebSocket transport for WASM/JS environments.
// The component parameter specifies the WebSocket proxy component path (e.g., "/management", "/signal").
func WithCustomDialer(tlsEnabled bool, component string) grpc.DialOption {
return client.WithWebSocketDialer(tlsEnabled, component)
}
// WithSweeper is a no-op on WASM/JS: there is no network change signal.
func WithSweeper(_ Sweeper) grpc.DialOption {
return grpc.EmptyDialOption{}
}

56
client/grpc/retry.go Normal file
View File

@@ -0,0 +1,56 @@
package grpc
import (
"context"
"errors"
"time"
"github.com/cenkalti/backoff/v4"
)
// ChangeWatcher exposes OS network availability transitions.
type ChangeWatcher interface {
Changed() <-chan struct{}
}
// Retry mirrors backoff.Retry, but the sleep between attempts also wakes on
// OS network availability transitions: an operation cut down by a network
// change retries the moment the network settles instead of sleeping through
// the recovery. A nil watcher never fires, leaving plain backoff.Retry
// behavior.
func Retry(ctx context.Context, operation backoff.Operation, bo backoff.BackOff, watcher ChangeWatcher) error {
bo.Reset()
for {
err := operation()
if err == nil {
return nil
}
var permanent *backoff.PermanentError
if errors.As(err, &permanent) {
return permanent.Err
}
next := bo.NextBackOff()
if next == backoff.Stop {
if cerr := ctx.Err(); cerr != nil {
return cerr
}
return err
}
var changed <-chan struct{}
if watcher != nil {
changed = watcher.Changed()
}
timer := time.NewTimer(next)
select {
case <-timer.C:
case <-changed:
timer.Stop()
case <-ctx.Done():
timer.Stop()
return ctx.Err()
}
}
}

91
client/grpc/retry_test.go Normal file
View File

@@ -0,0 +1,91 @@
package grpc
import (
"context"
"errors"
"testing"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/netevents/netstate"
)
func TestRetryWakesOnNetworkChange(t *testing.T) {
ns := netstate.New()
attempts := 0
operation := func() error {
attempts++
if attempts == 1 {
return errors.New("cut by network change")
}
return nil
}
go func() {
time.Sleep(20 * time.Millisecond)
ns.Set(false)
}()
start := time.Now()
err := Retry(context.Background(), operation, backoff.NewConstantBackOff(time.Minute), ns)
require.NoError(t, err)
assert.Equal(t, 2, attempts, "network change must cause one immediate retry")
assert.Less(t, time.Since(start), time.Second, "the transition must cut the minute-long sleep short")
}
func TestRetryPermanentError(t *testing.T) {
sentinel := errors.New("permission denied")
operation := func() error {
return backoff.Permanent(sentinel)
}
err := Retry(context.Background(), operation, backoff.NewConstantBackOff(time.Millisecond), nil)
assert.ErrorIs(t, err, sentinel, "permanent errors must stop retries")
}
func TestRetryNilNetState(t *testing.T) {
attempts := 0
operation := func() error {
attempts++
if attempts < 3 {
return errors.New("transient")
}
return nil
}
err := Retry(context.Background(), operation, backoff.NewConstantBackOff(time.Millisecond), nil)
require.NoError(t, err)
assert.Equal(t, 3, attempts, "nil network state must preserve timed retries")
}
func TestRetryStops(t *testing.T) {
failure := errors.New("still failing")
operation := func() error {
return failure
}
err := Retry(context.Background(), operation, &backoff.StopBackOff{}, nil)
assert.ErrorIs(t, err, failure, "stop backoff must return the operation error")
}
func TestRetryCtxCancelDuringSleep(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
operation := func() error {
return errors.New("failing")
}
go func() {
time.Sleep(20 * time.Millisecond)
cancel()
}()
start := time.Now()
err := Retry(ctx, operation, backoff.NewConstantBackOff(time.Minute), netstate.New())
assert.ErrorIs(t, err, context.Canceled, "context cancellation must stop the retry loop")
assert.Less(t, time.Since(start), time.Second, "context cancellation must interrupt backoff sleep")
}

View File

@@ -83,15 +83,6 @@ func NewAuth(ctx context.Context, privateKey string, mgmURL *url.URL, config *pr
}, nil
}
// grpcClient returns the current management connection. Callers must go through it rather than
// reading a.client: reconnect replaces that field while other goroutines are using it.
func (a *Auth) grpcClient() *mgm.GrpcClient {
a.mutex.RLock()
defer a.mutex.RUnlock()
return a.client
}
// Close closes the management client connection
func (a *Auth) Close() error {
a.mutex.Lock()
@@ -147,22 +138,38 @@ func (a *Auth) IsSSOSupported(ctx context.Context) (bool, error) {
// GetOAuthFlow returns an OAuth flow (PKCE or Device) using the existing management connection
// This avoids creating a new connection to the management server
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool) (OAuthFlow, error) {
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool, hint string) (OAuthFlow, error) {
var flow OAuthFlow
// the connection is owned by a and outlives this call, so a later fallback reuses it
newAuth := func(context.Context) (*Auth, func(), error) {
return a, func() {}, nil
}
err := a.withRetry(ctx, func(client *mgm.GrpcClient) error {
var err error
flow, err = oauthFlowWithFallback(a, client, flowOrder(forceDeviceAuth, true), "", newAuth)
if IsSSOUnavailable(err) {
return backoff.Permanent(err)
if forceDeviceAuth {
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
return err
// Try PKCE flow first
pkceFlow, err := a.getPKCEFlow(client)
if err != nil {
// If PKCE not supported, try Device flow
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
return err
}
pkceFlow.SetLoginHint(hint)
flow = pkceFlow
return nil
})
return flow, err

View File

@@ -48,17 +48,8 @@ type DeviceAuthProviderConfig struct {
LoginHint string
}
// validateDeviceAuthConfig validates device authorization provider configuration. A missing
// value means management does not have this flow configured, so the error wraps
// errFlowNotConfigured and the caller can fall back to the other flow.
// validateDeviceAuthConfig validates device authorization provider configuration
func validateDeviceAuthConfig(config *DeviceAuthProviderConfig) error {
if err := checkDeviceAuthConfig(config); err != nil {
return fmt.Errorf("%w: %w", errFlowNotConfigured, err)
}
return nil
}
func checkDeviceAuthConfig(config *DeviceAuthProviderConfig) error {
errorMsgFormat := "invalid provider configuration received from management: %s value is empty. Contact your NetBird administrator"
if config.Audience == "" {
@@ -170,12 +161,8 @@ func (d *DeviceAuthorizationFlow) RequestAuthInfo(ctx context.Context) (AuthFlow
return AuthFlowInfo{}, fmt.Errorf("reading body failed with error: %v", err)
}
if res.StatusCode != http.StatusOK {
reqErr := fmt.Errorf("request device code returned status %d error: %s", res.StatusCode, string(body))
if deviceGrantUnsupported(res.StatusCode, body) {
return AuthFlowInfo{}, fmt.Errorf("%w: %w", errFlowNotConfigured, reqErr)
}
return AuthFlowInfo{}, reqErr
if res.StatusCode != 200 {
return AuthFlowInfo{}, fmt.Errorf("request device code returned status %d error: %s", res.StatusCode, string(body))
}
deviceCode := AuthFlowInfo{}
@@ -199,34 +186,6 @@ func (d *DeviceAuthorizationFlow) RequestAuthInfo(ctx context.Context) (AuthFlow
return deviceCode, err
}
// deviceGrantUnsupported reports whether the IdP's answer to a device code request means it does
// not serve the device authorization grant at all, rather than a transient or request-specific
// failure. An IdP that does not route the endpoint answers 404/405/501; one that knows the
// endpoint but has the grant disabled for this client answers with an OAuth 2.0 error code.
func deviceGrantUnsupported(statusCode int, body []byte) bool {
switch statusCode {
case http.StatusNotFound, http.StatusMethodNotAllowed, http.StatusNotImplemented:
return true
case http.StatusBadRequest, http.StatusUnauthorized, http.StatusForbidden:
default:
return false
}
var oauthErr struct {
Error string `json:"error"`
}
if err := json.Unmarshal(body, &oauthErr); err != nil {
return false
}
switch oauthErr.Error {
case "unsupported_grant_type", "unauthorized_client":
return true
default:
return false
}
}
func appendLoginHint(uri, loginHint string) string {
if uri == "" || loginHint == "" {
return uri

View File

@@ -2,19 +2,15 @@ package auth
import (
"context"
"errors"
"fmt"
"net/http"
"net/url"
"runtime"
"sync"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
mgm "github.com/netbirdio/netbird/shared/management/client"
)
// OAuthFlow represents an interface for authorization using different OAuth 2.0 flows
@@ -63,327 +59,73 @@ func (t TokenInfo) GetTokenToUse() string {
return t.AccessToken
}
// errFlowNotConfigured marks a flow this deployment does not offer: management returned no
// configuration for it, the configuration it returned is incomplete, or the IdP refuses to serve
// the grant. It is the only condition that makes the client try the other flow, so that a
// transient failure keeps failing on the flow the user actually wants.
var errFlowNotConfigured = errors.New("authorization flow is not configured")
// ssoUnavailableError reports that the management server offers no usable SSO flow at all.
// Retrying cannot help, so callers should surface it to the user instead of backing off.
type ssoUnavailableError struct {
msg string
func shouldUseDeviceFlow(force bool, isUnixDesktopClient bool) bool {
return force || (runtime.GOOS == "linux" || runtime.GOOS == "freebsd") && !isUnixDesktopClient
}
func (e *ssoUnavailableError) Error() string {
return e.msg
}
// oauthFlowInit names one of the OAuth flows and builds it from the management configuration.
type oauthFlowInit struct {
name string
init func(a *Auth, client *mgm.GrpcClient, hint string) (OAuthFlow, error)
}
// authFactory hands out a management connection to build a flow with, plus the cleanup that
// releases it. Callers that own a long-lived connection return it with a no-op cleanup.
type authFactory func(ctx context.Context) (*Auth, func(), error)
// loginHintSetter is implemented by both concrete flows but is deliberately not part of
// OAuthFlow, so callers reach it through a type assertion.
type loginHintSetter interface {
SetLoginHint(hint string)
}
// fallbackFlow wraps the flow that was picked at initialization time with the flows that were
// not tried. Whether the IdP actually serves a flow only shows up when the flow is run: an IdP
// with the device grant disabled answers the device code request with 404 even though
// management handed out a device flow configuration. When that happens the wrapper swaps in the
// next flow instead of failing the login.
type fallbackFlow struct {
mu sync.Mutex
active OAuthFlow
remaining []oauthFlowInit
hint string
newAuth authFactory
}
func (f *fallbackFlow) RequestAuthInfo(ctx context.Context) (AuthFlowInfo, error) {
info, err := f.current().RequestAuthInfo(ctx)
if err == nil || !isFlowUnavailable(err) {
return info, err
}
next, nextErr := f.initNext(ctx)
if nextErr != nil {
log.Debugf("failed to fall back to another authorization flow: %v", nextErr)
return AuthFlowInfo{}, err
}
return next.RequestAuthInfo(ctx)
}
func (f *fallbackFlow) WaitToken(ctx context.Context, info AuthFlowInfo) (TokenInfo, error) {
return f.current().WaitToken(ctx, info)
}
func (f *fallbackFlow) GetClientID(ctx context.Context) string {
return f.current().GetClientID(ctx)
}
// SetLoginHint forwards the hint to the active flow and keeps it for a flow a later fallback
// initializes. Callers that set the hint after building the flow reach the concrete flow through
// a type assertion, which the OAuthFlow interface does not carry, so the wrapper has to offer it
// too or the hint is silently dropped.
func (f *fallbackFlow) SetLoginHint(hint string) {
f.mu.Lock()
defer f.mu.Unlock()
f.hint = hint
if setter, ok := f.active.(loginHintSetter); ok {
setter.SetLoginHint(hint)
}
}
func (f *fallbackFlow) current() OAuthFlow {
f.mu.Lock()
defer f.mu.Unlock()
return f.active
}
// initNext initializes the next flow this deployment offers and makes it the active one.
func (f *fallbackFlow) initNext(ctx context.Context) (OAuthFlow, error) {
f.mu.Lock()
defer f.mu.Unlock()
if len(f.remaining) == 0 {
return nil, errors.New("no authorization flow left to try")
}
a, cleanup, err := f.newAuth(ctx)
if err != nil {
return nil, err
}
defer cleanup()
flow, remaining, err := initFirstAvailableFlow(a, a.grpcClient(), f.remaining, f.hint)
if err != nil {
return nil, err
}
log.Infof("the identity provider does not serve the selected authorization flow, continuing with the next one")
f.active = flow
f.remaining = remaining
return flow, nil
}
// preferDeviceFlow reports whether the device code flow should be tried before PKCE. PKCE needs
// a browser on this host and a loopback listener to receive the redirect, neither of which
// exists on a Unix host without a graphical session. The GOOS guard keeps a caller that reports
// no graphical session on a platform that always has one from changing the preference.
func preferDeviceFlow(hasGraphicalSession bool) bool {
return (runtime.GOOS == "linux" || runtime.GOOS == "freebsd") && !hasGraphicalSession
}
// flowOrder returns the flows to attempt, in order.
// NewOAuthFlow initializes and returns the appropriate OAuth flow based on the management configuration
//
// force leaves the device code flow on its own rather than first: it marks a device with no
// browser at all, such as Android TV or tvOS. PKCE cannot work there even from another device,
// because the redirect has to arrive on the loopback listener of the device being enrolled, so
// offering it as a fallback would only replace a clear error with a login that cannot complete.
func flowOrder(force bool, hasGraphicalSession bool) []oauthFlowInit {
pkce := oauthFlowInit{name: "pkce authorization flow", init: initPKCEFlow}
device := oauthFlowInit{name: "device code flow", init: initDeviceFlow}
switch {
case force:
return []oauthFlowInit{device}
case preferDeviceFlow(hasGraphicalSession):
return []oauthFlowInit{device, pkce}
default:
return []oauthFlowInit{pkce, device}
}
}
func initPKCEFlow(a *Auth, client *mgm.GrpcClient, hint string) (OAuthFlow, error) {
flow, err := a.getPKCEFlow(client)
if err != nil {
return nil, err
}
if hint != "" {
flow.SetLoginHint(hint)
}
return flow, nil
}
func initDeviceFlow(a *Auth, client *mgm.GrpcClient, hint string) (OAuthFlow, error) {
flow, err := a.getDeviceFlow(client)
if err != nil {
return nil, err
}
if hint != "" {
flow.SetLoginHint(hint)
}
return flow, nil
}
// NewOAuthFlow initializes and returns an OAuth flow based on the management configuration.
// It starts by initializing the PKCE.If this process fails, it resorts to the Device Code Flow,
// and if that also fails, the authentication process is deemed unsuccessful
//
// Both flows are optional server side: management answers NotFound for a flow it has no
// configuration for. The preferred flow is tried first and the other one is used as a fallback,
// so a server that only offers one of them still works. forceDeviceCodeFlow restricts the client
// to the device code flow with no fallback, for a device that has no browser at all.
func NewOAuthFlow(ctx context.Context, config *profilemanager.Config, hasGraphicalSession bool, forceDeviceCodeFlow bool, hint string) (OAuthFlow, error) {
// On Linux distros without desktop environment support, it only tries to initialize the Device Code Flow
// forceDeviceCodeFlow can be used to skip PKCE and go directly to Device Code Flow (e.g., for Android TV)
func NewOAuthFlow(ctx context.Context, config *profilemanager.Config, isUnixDesktopClient bool, forceDeviceCodeFlow bool, hint string) (OAuthFlow, error) {
if shouldUseDeviceFlow(forceDeviceCodeFlow, isUnixDesktopClient) {
return authenticateWithDeviceCodeFlow(ctx, config, hint)
}
pkceFlow, err := authenticateWithPKCEFlow(ctx, config, hint)
if err != nil {
log.Debugf("failed to initialize pkce authentication with error: %v\n", err)
log.Debug("falling back to device code flow")
return authenticateWithDeviceCodeFlow(ctx, config, hint)
}
return pkceFlow, nil
}
// authenticateWithPKCEFlow initializes the Proof Key for Code Exchange flow auth flow
func authenticateWithPKCEFlow(ctx context.Context, config *profilemanager.Config, hint string) (OAuthFlow, error) {
authClient, err := NewAuth(ctx, config.PrivateKey, config.ManagementURL, config)
if err != nil {
return nil, fmt.Errorf("create auth client: %w", err)
return nil, fmt.Errorf("failed to create auth client: %v", err)
}
defer func() {
if err := authClient.Close(); err != nil {
log.Debugf("failed to close auth client: %v", err)
}
}()
defer authClient.Close()
// the connection above is closed on return, so a later fallback opens its own
newAuth := func(ctx context.Context) (*Auth, func(), error) {
a, err := NewAuth(ctx, config.PrivateKey, config.ManagementURL, config)
if err != nil {
return nil, nil, fmt.Errorf("create auth client: %w", err)
}
return a, func() {
if err := a.Close(); err != nil {
log.Debugf("failed to close auth client: %v", err)
}
}, nil
}
flows := flowOrder(forceDeviceCodeFlow, hasGraphicalSession)
return oauthFlowWithFallback(authClient, authClient.grpcClient(), flows, hint, newAuth)
}
// oauthFlowWithFallback initializes the first flow this deployment offers, moving on to the next
// one when a flow is not configured here. It only fails once every flow has been tried, and any
// flow left untried is handed to the returned flow so it can still fall back if the IdP rejects
// the flow that was picked.
func oauthFlowWithFallback(a *Auth, client *mgm.GrpcClient, flows []oauthFlowInit, hint string, newAuth authFactory) (OAuthFlow, error) {
flow, remaining, err := initFirstAvailableFlow(a, client, flows, hint)
pkceFlowInfo, err := authClient.getPKCEFlow(authClient.client)
if err != nil {
return nil, err
return nil, fmt.Errorf("getting pkce authorization flow info failed with error: %v", err)
}
if len(remaining) == 0 {
return flow, nil
}
pkceFlowInfo.SetLoginHint(hint)
return &fallbackFlow{
active: flow,
remaining: remaining,
hint: hint,
newAuth: newAuth,
}, nil
return pkceFlowInfo, nil
}
// initFirstAvailableFlow returns the first flow that could be initialized along with the flows
// after it, which are still untried.
func initFirstAvailableFlow(a *Auth, client *mgm.GrpcClient, flows []oauthFlowInit, hint string) (OAuthFlow, []oauthFlowInit, error) {
var errs []error
for i, f := range flows {
flow, err := f.init(a, client, hint)
if err == nil {
return flow, flows[i+1:], nil
}
// authenticateWithDeviceCodeFlow initializes the Device Code auth Flow
func authenticateWithDeviceCodeFlow(ctx context.Context, config *profilemanager.Config, hint string) (OAuthFlow, error) {
authClient, err := NewAuth(ctx, config.PrivateKey, config.ManagementURL, config)
if err != nil {
return nil, fmt.Errorf("failed to create auth client: %v", err)
}
defer authClient.Close()
errs = append(errs, fmt.Errorf("%s: %w", f.name, err))
// only a flow this deployment does not offer is worth replacing with another one
if !isFlowUnavailable(err) {
break
}
if i < len(flows)-1 {
log.Infof("%s is not configured (%v), falling back to %s", f.name, err, flows[i+1].name)
deviceFlowInfo, err := authClient.getDeviceFlow(authClient.client)
if err != nil {
switch s, ok := gstatus.FromError(err); {
case ok && s.Code() == codes.NotFound:
return nil, fmt.Errorf("no SSO provider returned from management. " +
"Please proceed with setting up this device using setup keys " +
"https://docs.netbird.io/how-to/register-machines-using-setup-keys")
case ok && s.Code() == codes.Unimplemented:
return nil, fmt.Errorf("the management server, %s, does not support SSO providers, "+
"please update your server or use Setup Keys to login", config.ManagementURL)
default:
return nil, fmt.Errorf("getting device authorization flow info failed with error: %v", err)
}
}
return nil, nil, flowInitError(a.mgmURL, errs)
}
// flowInitError turns the per-flow initialization errors into a single actionable error. The
// message stays neutral about what to do instead: SSO is also how a peer extends its session and
// authenticates SSH, where a setup key is no alternative. Callers that are enrolling a device add
// that advice themselves, see IsSSOUnavailable.
func flowInitError(mgmURL *url.URL, errs []error) error {
if allMatch(errs, isFlowUnimplemented) {
return &ssoUnavailableError{msg: fmt.Sprintf("the management server, %s, does not support SSO providers, "+
"please update your server", mgmURL)}
}
if allMatch(errs, isFlowUnavailable) {
return &ssoUnavailableError{msg: "the management server has no SSO provider configured: " +
"neither the pkce authorization flow nor the device code flow is available"}
}
return fmt.Errorf("initialize authorization flow: %w", errors.Join(errs...))
}
// IsSSOUnavailable reports whether err means the management server offers no usable SSO flow, so
// no retry and no other flow can help. Enrollment paths use it to point the user at setup keys.
func IsSSOUnavailable(err error) bool {
var ssoUnavailable *ssoUnavailableError
return errors.As(err, &ssoUnavailable)
}
// WithSetupKeyAdvice appends enrollment guidance to an SSO-unavailable error and returns any
// other error unchanged. Only enrollment can fall back to a setup key: extending a session and
// authenticating SSH cannot, so those paths must not call this.
//
// The login paths that do call it cannot tell an unregistered peer from an SSO-enrolled one
// whose session expired, since both answer PermissionDenied, so the advice names the case it
// applies to rather than telling an enrolled peer to do something that cannot work.
func WithSetupKeyAdvice(err error) error {
if !IsSSOUnavailable(err) {
return err
}
return fmt.Errorf("%w. If this device is not enrolled yet, enroll it with a setup key instead: "+
"https://docs.netbird.io/how-to/register-machines-using-setup-keys", err)
}
func allMatch(errs []error, match func(error) bool) bool {
if len(errs) == 0 {
return false
}
for _, err := range errs {
if !match(err) {
return false
}
}
return true
}
// isFlowUnavailable reports whether the flow is not on offer here: management has no
// configuration for it (NotFound), predates the RPC entirely (Unimplemented), returned an
// incomplete configuration, or the IdP does not serve the grant.
func isFlowUnavailable(err error) bool {
return errors.Is(err, errFlowNotConfigured) ||
hasStatusCode(err, codes.NotFound) ||
hasStatusCode(err, codes.Unimplemented)
}
func isFlowUnimplemented(err error) bool {
return hasStatusCode(err, codes.Unimplemented)
}
func hasStatusCode(err error, code codes.Code) bool {
s, ok := gstatus.FromError(err)
if !ok {
return false
}
return s.Code() == code
deviceFlowInfo.SetLoginHint(hint)
return deviceFlowInfo, nil
}

View File

@@ -1,330 +0,0 @@
package auth
import (
"context"
"errors"
"fmt"
"net/url"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
mgm "github.com/netbirdio/netbird/shared/management/client"
)
// stubFlow is a minimal OAuthFlow returned by the fake initializers below. requestErr, when set,
// is what its RequestAuthInfo returns, standing in for an IdP that rejects the flow.
type stubFlow struct {
name string
hint string
requestErr error
}
func (s *stubFlow) RequestAuthInfo(context.Context) (AuthFlowInfo, error) {
if s.requestErr != nil {
return AuthFlowInfo{}, s.requestErr
}
return AuthFlowInfo{UserCode: s.name}, nil
}
func (s *stubFlow) WaitToken(context.Context, AuthFlowInfo) (TokenInfo, error) {
return TokenInfo{}, nil
}
func (s *stubFlow) GetClientID(context.Context) string {
return ""
}
func (s *stubFlow) SetLoginHint(hint string) {
s.hint = hint
}
// stubInit returns a flow initializer that yields a named stub flow, or err when err is non-nil.
func stubInit(name string, err error) oauthFlowInit {
return stubInitFlow(name, err, nil)
}
// stubInitFlow is stubInit with control over what the resulting flow's RequestAuthInfo returns.
func stubInitFlow(name string, initErr, requestErr error) oauthFlowInit {
return oauthFlowInit{
name: name,
init: func(_ *Auth, _ *mgm.GrpcClient, hint string) (OAuthFlow, error) {
if initErr != nil {
return nil, initErr
}
return &stubFlow{name: name, hint: hint, requestErr: requestErr}, nil
},
}
}
// stubAuthFactory hands out an Auth without a management connection, which the stub
// initializers above never touch.
func stubAuthFactory(a *Auth) authFactory {
return func(context.Context) (*Auth, func(), error) {
return a, func() {}, nil
}
}
func TestOAuthFlowWithFallback(t *testing.T) {
notFound := status.Error(codes.NotFound, "no device authorization flow information available")
unimplemented := status.Error(codes.Unimplemented, "unknown method")
incompleteConfig := fmt.Errorf("%w: Client ID value is empty", errFlowNotConfigured)
unreachable := status.Error(codes.Unavailable, "connection refused")
tests := []struct {
name string
flows []oauthFlowInit
expectedFlow string
expectedErr string
expectedNoSSO bool
}{
{
name: "preferred flow is used",
flows: []oauthFlowInit{stubInit("device", nil), stubInit("pkce", nil)},
expectedFlow: "device",
},
{
// the RedHat case: device code flow disabled on management, PKCE configured
name: "falls back when preferred flow is not configured",
flows: []oauthFlowInit{stubInit("device", notFound), stubInit("pkce", nil)},
expectedFlow: "pkce",
},
{
name: "falls back on an incomplete flow configuration",
flows: []oauthFlowInit{stubInit("pkce", incompleteConfig), stubInit("device", nil)},
expectedFlow: "device",
},
{
name: "does not fall back when the preferred flow fails for another reason",
flows: []oauthFlowInit{stubInit("pkce", unreachable), stubInit("device", nil)},
expectedErr: "connection refused",
},
{
// stays neutral about the remedy: --extend and SSH auth cannot use a setup key
name: "neither flow configured reports no SSO provider",
flows: []oauthFlowInit{stubInit("device", notFound), stubInit("pkce", notFound)},
expectedErr: "no SSO provider configured",
expectedNoSSO: true,
},
{
name: "old server without the flow RPCs asks for an update",
flows: []oauthFlowInit{stubInit("device", unimplemented), stubInit("pkce", unimplemented)},
expectedErr: "does not support SSO providers",
expectedNoSSO: true,
},
}
mgmURL, err := url.Parse("https://api.netbird.io:443")
require.NoError(t, err)
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
a := &Auth{mgmURL: mgmURL}
flow, err := oauthFlowWithFallback(a, nil, tt.flows, "user@example.com", stubAuthFactory(a))
if tt.expectedErr != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.expectedErr)
var ssoUnavailable *ssoUnavailableError
assert.Equal(t, tt.expectedNoSSO, errors.As(err, &ssoUnavailable),
"terminal SSO-unavailable classification mismatch for %v", err)
return
}
require.NoError(t, err)
stub := activeStub(t, flow)
assert.Equal(t, tt.expectedFlow, stub.name)
assert.Equal(t, "user@example.com", stub.hint, "login hint must be passed to the flow")
})
}
}
// activeStub unwraps the flow currently in use, which is behind a fallbackFlow whenever an
// untried flow is left.
func activeStub(t *testing.T, flow OAuthFlow) *stubFlow {
t.Helper()
if fallback, ok := flow.(*fallbackFlow); ok {
flow = fallback.current()
}
stub, ok := flow.(*stubFlow)
require.True(t, ok, "unexpected flow type %T", flow)
return stub
}
// TestFallbackFlowRequestAuthInfo covers the failure the RedHat report hit: management hands out
// a device flow configuration, but the IdP does not serve the grant and only says so when the
// device code is requested.
func TestFallbackFlowRequestAuthInfo(t *testing.T) {
mgmURL, err := url.Parse("https://api.netbird.io:443")
require.NoError(t, err)
a := &Auth{mgmURL: mgmURL}
idpRejects := fmt.Errorf("%w: request device code returned status 404", errFlowNotConfigured)
t.Run("swaps in the untried flow", func(t *testing.T) {
flows := []oauthFlowInit{stubInitFlow("device", nil, idpRejects), stubInit("pkce", nil)}
flow, err := oauthFlowWithFallback(a, nil, flows, "", stubAuthFactory(a))
require.NoError(t, err)
require.Equal(t, "device", activeStub(t, flow).name)
info, err := flow.RequestAuthInfo(context.Background())
require.NoError(t, err)
assert.Equal(t, "pkce", info.UserCode, "the request must be served by the fallback flow")
assert.Equal(t, "pkce", activeStub(t, flow).name, "the fallback flow must stay active for WaitToken")
})
t.Run("keeps the original error when nothing else is configured", func(t *testing.T) {
flows := []oauthFlowInit{
stubInitFlow("device", nil, idpRejects),
stubInit("pkce", status.Error(codes.NotFound, "no pkce authorization flow information available")),
}
flow, err := oauthFlowWithFallback(a, nil, flows, "", stubAuthFactory(a))
require.NoError(t, err)
_, err = flow.RequestAuthInfo(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "status 404")
})
t.Run("keeps the original error when the fallback cannot reach management", func(t *testing.T) {
flows := []oauthFlowInit{stubInitFlow("device", nil, idpRejects), stubInit("pkce", nil)}
unreachable := func(context.Context) (*Auth, func(), error) {
return nil, nil, errors.New("connect to management: connection refused")
}
flow, err := oauthFlowWithFallback(a, nil, flows, "", unreachable)
require.NoError(t, err)
_, err = flow.RequestAuthInfo(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "status 404", "the IdP error must survive a failed fallback")
assert.Equal(t, "device", activeStub(t, flow).name, "a failed fallback must not swap the flow")
})
t.Run("does not swap flows on an unrelated failure", func(t *testing.T) {
flows := []oauthFlowInit{
stubInitFlow("device", nil, errors.New("timeout talking to the IdP")),
stubInit("pkce", nil),
}
flow, err := oauthFlowWithFallback(a, nil, flows, "", stubAuthFactory(a))
require.NoError(t, err)
_, err = flow.RequestAuthInfo(context.Background())
require.Error(t, err)
assert.Equal(t, "device", activeStub(t, flow).name, "the preferred flow must stay active")
})
}
// TestForcedDeviceFlowHasNoFallback covers Android TV and tvOS: a browserless device must get the
// device code error rather than a PKCE flow it can never complete.
func TestForcedDeviceFlowHasNoFallback(t *testing.T) {
mgmURL, err := url.Parse("https://api.netbird.io:443")
require.NoError(t, err)
a := &Auth{mgmURL: mgmURL}
notFound := status.Error(codes.NotFound, "no device authorization flow information available")
t.Run("no wrapper when the device flow works", func(t *testing.T) {
// flowOrder(force) yields this single-entry list, see TestFlowOrder
forced := []oauthFlowInit{stubInit("device", nil)}
flow, err := oauthFlowWithFallback(a, nil, forced, "", stubAuthFactory(a))
require.NoError(t, err)
_, wrapped := flow.(*fallbackFlow)
assert.False(t, wrapped, "nothing may swap the flow later on a browserless device")
})
t.Run("reports the device flow error instead of falling back", func(t *testing.T) {
forced := []oauthFlowInit{stubInit("device", notFound)}
_, err := oauthFlowWithFallback(a, nil, forced, "", stubAuthFactory(a))
require.Error(t, err)
assert.True(t, IsSSOUnavailable(err), "the caller must see that SSO is unavailable here")
})
}
// TestFallbackFlowSetLoginHint covers the Android SDK's pattern: it sets the login hint after the
// flow is built, through a type assertion that the wrapper must satisfy.
func TestFallbackFlowSetLoginHint(t *testing.T) {
mgmURL, err := url.Parse("https://api.netbird.io:443")
require.NoError(t, err)
a := &Auth{mgmURL: mgmURL}
idpRejects := fmt.Errorf("%w: request device code returned status 404", errFlowNotConfigured)
flows := []oauthFlowInit{stubInitFlow("device", nil, idpRejects), stubInit("pkce", nil)}
flow, err := oauthFlowWithFallback(a, nil, flows, "", stubAuthFactory(a))
require.NoError(t, err)
setter, ok := flow.(loginHintSetter)
require.True(t, ok, "the wrapper must accept a login hint like the concrete flows do")
setter.SetLoginHint("user@example.com")
assert.Equal(t, "user@example.com", activeStub(t, flow).hint, "the active flow must get the hint")
// the device flow is rejected by the IdP here, so the hint has to survive into the fallback
_, err = flow.RequestAuthInfo(context.Background())
require.NoError(t, err)
assert.Equal(t, "pkce", activeStub(t, flow).name)
assert.Equal(t, "user@example.com", activeStub(t, flow).hint, "the fallback flow must get the hint too")
}
func TestWithSetupKeyAdvice(t *testing.T) {
other := errors.New("connection refused")
assert.Equal(t, other, WithSetupKeyAdvice(other), "only an SSO-unavailable error gets advice")
advised := WithSetupKeyAdvice(&ssoUnavailableError{msg: "no SSO provider configured"})
assert.Contains(t, advised.Error(), "no SSO provider configured", "the original message must survive")
assert.Contains(t, advised.Error(), "setup key")
// a setup key cannot re-enrol a peer whose SSO session expired, and the login paths cannot
// tell that peer apart from an unregistered one, so the advice must state its condition
assert.Contains(t, advised.Error(), "not enrolled yet")
assert.True(t, IsSSOUnavailable(advised), "advice must keep the error classifiable")
}
func flowNames(flows []oauthFlowInit) []string {
names := make([]string, 0, len(flows))
for _, f := range flows {
names = append(names, f.name)
}
return names
}
func TestFlowOrder(t *testing.T) {
const pkce, device = "pkce authorization flow", "device code flow"
assert.Equal(t, []string{pkce, device}, flowNames(flowOrder(false, true)),
"a device with a browser tries PKCE first and keeps the device code flow as a fallback")
// only a unix host without a graphical session lacks a browser; the other platforms have one
headless := []string{pkce, device}
if runtime.GOOS == "linux" || runtime.GOOS == "freebsd" {
headless = []string{device, pkce}
}
assert.Equal(t, headless, flowNames(flowOrder(false, false)), "on %s", runtime.GOOS)
// Android TV and tvOS have no browser, so PKCE cannot complete there even from another
// device: the redirect must reach the loopback listener of the device being enrolled.
assert.Equal(t, []string{device}, flowNames(flowOrder(true, false)),
"a forced device code flow must not fall back to PKCE")
assert.Equal(t, []string{device}, flowNames(flowOrder(true, true)),
"force wins over a reported graphical session")
}
func TestPreferDeviceFlow(t *testing.T) {
isUnix := runtime.GOOS == "linux" || runtime.GOOS == "freebsd"
assert.Equal(t, isUnix, preferDeviceFlow(false), "headless unix hosts prefer the device flow")
assert.False(t, preferDeviceFlow(true), "clients with a graphical session prefer PKCE")
}

View File

@@ -62,17 +62,8 @@ type PKCEAuthProviderConfig struct {
LoginHint string
}
// validatePKCEConfig validates PKCE provider configuration. A missing value means management
// does not have this flow configured, so the error wraps errFlowNotConfigured and the caller can
// fall back to the other flow.
// validatePKCEConfig validates PKCE provider configuration
func validatePKCEConfig(config *PKCEAuthProviderConfig) error {
if err := checkPKCEConfig(config); err != nil {
return fmt.Errorf("%w: %w", errFlowNotConfigured, err)
}
return nil
}
func checkPKCEConfig(config *PKCEAuthProviderConfig) error {
errorMsgFormat := "invalid provider configuration received from management: %s value is empty. Contact your NetBird administrator"
if config.ClientID == "" {

View File

@@ -38,6 +38,7 @@ import (
"github.com/netbirdio/netbird/client/internal/updater"
"github.com/netbirdio/netbird/client/internal/updater/installer"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/netevents"
cProto "github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/ssh"
sshconfig "github.com/netbirdio/netbird/client/ssh/config"
@@ -70,18 +71,31 @@ type ConnectClient struct {
updateManager *updater.Manager
persistSyncResponse bool
// netEvents gates every reconnection loop on OS-reported network
// availability and sweeps connections on network change.
netEvents *netevents.Manager
}
// ConnectClientOption configures optional ConnectClient behavior.
type ConnectClientOption func(*ConnectClient)
// WithNetEvents injects the OS network event handling.
func WithNetEvents(events *netevents.Manager) ConnectClientOption {
return func(c *ConnectClient) { c.netEvents = events }
}
func NewConnectClient(
ctx context.Context,
config *profilemanager.Config,
statusRecorder *peer.Status,
opts ...ConnectClientOption,
) *ConnectClient {
// Derive the run context here so Stop owns the cancel that unblocks the run
// loop. runCancel is set once at construction, so Stop can call it without
// racing the run loop's startup. Callers therefore need not cancel before Stop.
runCtx, runCancel := context.WithCancel(ctx)
return &ConnectClient{
c := &ConnectClient{
ctx: runCtx,
runCancel: runCancel,
runExited: make(chan struct{}),
@@ -89,6 +103,10 @@ func NewConnectClient(
statusRecorder: statusRecorder,
engineMutex: sync.Mutex{},
}
for _, opt := range opts {
opt(c)
}
return c
}
func (c *ConnectClient) SetUpdateManager(um *updater.Manager) {
@@ -274,6 +292,13 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
return nil
}
// suspend connection attempts while the OS reports no usable network
if waited, err := c.netEvents.Wait(c.ctx); err != nil {
return nil
} else if waited {
backOff.Reset()
}
state.Set(StatusConnecting)
engineCtx, cancel := context.WithCancel(c.ctx)
@@ -285,7 +310,8 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
}()
log.Debugf("connecting to the Management service %s", c.config.ManagementURL.Host)
mgmClient, err := mgm.NewClient(engineCtx, c.config.ManagementURL.Host, myPrivateKey, mgmTlsEnabled)
mgmClient, err := mgm.NewClient(engineCtx, c.config.ManagementURL.Host, myPrivateKey, mgmTlsEnabled,
mgm.WithNetEvents(c.netEvents))
if err != nil {
// On daemon shutdown / Down() the parent context is cancelled
// and the dial fails with "context canceled". Wrapping that
@@ -360,7 +386,7 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
}()
// with the global Netbird config in hand connect (just a connection, no stream yet) Signal
signalClient, err := connectToSignal(engineCtx, loginResp.GetNetbirdConfig(), myPrivateKey)
signalClient, err := connectToSignal(engineCtx, loginResp.GetNetbirdConfig(), myPrivateKey, c.netEvents)
if err != nil {
log.Error(err)
return wrapErr(err)
@@ -396,7 +422,8 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
engineConfig.StateDir = filepath.Dir(path)
}
relayManager := relayClient.NewManager(engineCtx, relayURLs, myPrivateKey.PublicKey().String(), engineConfig.MTU)
relayManager := relayClient.NewManager(engineCtx, relayURLs, myPrivateKey.PublicKey().String(), engineConfig.MTU,
relayClient.WithNetEvents(c.netEvents))
c.statusRecorder.SetRelayMgr(relayManager)
if len(relayURLs) > 0 {
if token != nil {
@@ -424,6 +451,7 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
UpdateManager: c.updateManager,
ClientMetrics: c.clientMetrics,
MetricsCtx: c.ctx,
NetState: c.netEvents,
}, mobileDependency)
engine.SetSyncResponsePersistence(c.persistSyncResponse)
c.engine = engine
@@ -480,6 +508,16 @@ func (c *ConnectClient) run(mobileDependency MobileDependency, runningChan chan
// status stream stuck at Connecting.
err = backoff.Retry(operation, backoff.WithContext(backOff, c.ctx))
if err != nil {
// Once the client context is cancelled backoff.WithContext surfaces the
// bare context error, and any attempt torn down mid-flight reports the
// same. That cancellation is the caller asking us to stop (Stop, Down or
// an engine restart), so exit cleanly instead of handing back a failure
// the caller would have to distinguish from a real one.
if c.ctx.Err() != nil && errors.Is(err, context.Canceled) {
log.Info("exiting client retry loop, context cancelled")
return nil
}
log.Debugf("exiting client retry loop due to unrecoverable error: %s", err)
if s, ok := gstatus.FromError(err); ok && (s.Code() == codes.PermissionDenied) {
state.Set(StatusNeedsLogin)
@@ -673,7 +711,7 @@ func selectMTU(localMTU uint16, peerMTU int32) uint16 {
}
// connectToSignal creates Signal Service client and established a connection
func connectToSignal(ctx context.Context, wtConfig *mgmProto.NetbirdConfig, ourPrivateKey wgtypes.Key) (*signal.GrpcClient, error) {
func connectToSignal(ctx context.Context, wtConfig *mgmProto.NetbirdConfig, ourPrivateKey wgtypes.Key, netEvents *netevents.Manager) (*signal.GrpcClient, error) {
var sigTLSEnabled bool
if wtConfig.Signal.Protocol == mgmProto.HostConfig_HTTPS {
sigTLSEnabled = true
@@ -681,7 +719,8 @@ func connectToSignal(ctx context.Context, wtConfig *mgmProto.NetbirdConfig, ourP
sigTLSEnabled = false
}
signalClient, err := signal.NewClient(ctx, wtConfig.Signal.Uri, ourPrivateKey, sigTLSEnabled)
signalClient, err := signal.NewClient(ctx, wtConfig.Signal.Uri, ourPrivateKey, sigTLSEnabled,
signal.WithNetEvents(netEvents))
if err != nil {
log.Errorf("error while connecting to the Signal Exchange Service %s: %s", wtConfig.Signal.Uri, err)
return nil, gstatus.Errorf(codes.FailedPrecondition, "failed connecting to Signal Service : %s", err)

View File

@@ -51,6 +51,7 @@ nftables.txt: Anonymized nftables rules with packet counters across all families
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
config.txt: Anonymized configuration information of the NetBird client.
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
@@ -237,6 +238,13 @@ scutil_dns.txt (macOS only):
- Shows DNS configuration for all network interfaces
- Includes search domains, nameservers, and DNS resolver settings
- All IP addresses and domain names are anonymized
dns_windows.txt (Windows only):
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
- All IP addresses and domain names are anonymized
`
const (

View File

@@ -1,4 +1,4 @@
//go:build !unix
//go:build !unix && !windows
package debug

View File

@@ -0,0 +1,443 @@
//go:build windows
package debug
import (
"encoding/hex"
"errors"
"fmt"
"net/netip"
"strings"
"unsafe"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
nbdns "github.com/netbirdio/netbird/client/internal/dns"
)
const dnsInfoFileName = "dns_windows.txt"
const (
gpoDNSClientRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient`
tcpipParamsPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters`
dnscacheParams = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters`
)
// interfaceDNSValues are the per-interface values that decide how a name is
// resolved and registered. Everything the DNS host manager writes is in here,
// so a bundle shows both what we set and what it replaced.
var interfaceDNSValues = []string{
"NameServer",
"DhcpNameServer",
"Domain",
"DhcpDomain",
"SearchList",
"RegistrationEnabled",
"DisableDynamicUpdate",
"MaxNumberOfAddressesToRegister",
"EnableDHCP",
}
// addDNSInfo collects and adds DNS configuration information to the archive
func (g *BundleGenerator) addDNSInfo() error {
if err := g.addFileToZip(strings.NewReader(g.collectDNSInfo()), dnsInfoFileName); err != nil {
return fmt.Errorf("add DNS info to zip: %w", err)
}
return nil
}
// collectDNSInfo renders the report. Everything below it reaches the platform
// through COM and through lazily resolved procedures, which panic when a
// procedure is missing rather than returning an error, and a debug bundle is not
// allowed to take the daemon down. The panic is contained here, and whatever was
// collected before it is kept and reported with it.
func (g *BundleGenerator) collectDNSInfo() (content string) {
var sb strings.Builder
defer func() {
if r := recover(); r != nil {
log.Errorf("collecting Windows DNS configuration panicked: %v", r)
fmt.Fprintf(&sb, "\nerror: collection stopped: %v\n", r)
}
content = sb.String()
}()
sb.WriteString("Windows DNS configuration\n")
sb.WriteString("=========================\n")
adapters, adaptersErr := adapterAddresses()
g.writeNRPTRules(&sb, "NRPT rules, local policy store", nbdns.DNSPolicyConfigRoot)
g.writeNRPTRules(&sb, "NRPT rules, group policy store", nbdns.GPODNSPolicyConfigRoot)
g.writeEffectiveNRPTPolicies(&sb)
g.writeRegistryKey(&sb, "DNS client group policy", gpoDNSClientRoot)
g.writeRegistryKey(&sb, "Global TCP/IP parameters", tcpipParamsPath)
g.writeRegistryKey(&sb, "Dnscache parameters", dnscacheParams)
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv4", nbdns.InterfaceConfigPath, adapterNames(adapters))
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv6", nbdns.InterfaceConfigPathV6, adapterNames(adapters))
g.writeAdapterDNS(&sb, adapters, adaptersErr)
return sb.String()
}
// writeNRPTRules lists every rule in a policy store, ours and any other
// product's, since a foreign rule for the same namespace decides resolution
// just as ours does. Rules the client wrote are marked.
func (g *BundleGenerator) writeNRPTRules(sb *strings.Builder, title, root string) {
writeSection(sb, title, root)
names, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(names) == 0 {
sb.WriteString("no rules\n")
return
}
for _, name := range names {
owner := ""
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(nbdns.NRPTKeyPrefix)) {
owner = " (netbird)"
}
fmt.Fprintf(sb, "%s%s\n", name, owner)
g.writeValues(sb, root+`\`+name, nil, " ")
}
}
// writeEffectiveNRPTPolicies reports the table the resolver answers from, which
// the registry cannot show: a rule is written before it is loaded, and it keeps
// being enforced after its key is gone until the resolver reloads its policy.
func (g *BundleGenerator) writeEffectiveNRPTPolicies(sb *strings.Builder) {
writeSection(sb, "NRPT policy table in effect", nrptPolicyClass+"."+nrptPolicyMethod+" in "+nrptPolicyNamespace)
entries, err := effectiveNRPTPolicies()
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(entries) == 0 {
sb.WriteString("no policies\n")
return
}
for _, entry := range entries {
fmt.Fprintf(sb, "%s\n", g.anonymizeValue("Namespace", entry.namespace))
for _, value := range entry.values {
fmt.Fprintf(sb, " %s: %s\n", value.name, g.anonymizeValue(value.name, value.value))
}
}
}
// writeInterfaceDNS reports the DNS values of every interface that has any, so
// the netbird interface can be compared against the physical ones. The registry
// keys the values by GUID, so each is named from the adapter list; a GUID with
// no adapter is a leftover key of an interface that no longer exists.
func (g *BundleGenerator) writeInterfaceDNS(sb *strings.Builder, title, root string, names map[string]string) {
writeSection(sb, title, root)
guids, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
var reported int
for _, guid := range guids {
var iface strings.Builder
g.writeValues(&iface, root+`\`+guid, interfaceDNSValues, " ")
if iface.Len() == 0 {
continue
}
name, ok := names[strings.ToLower(guid)]
if !ok {
name = "no adapter with this GUID"
}
reported++
fmt.Fprintf(sb, "%s (%s)\n%s", guid, name, iface.String())
}
if reported == 0 {
sb.WriteString("no interface holds DNS values\n")
}
}
// writeRegistryKey reports the values of a single key, without its subkeys.
func (g *BundleGenerator) writeRegistryKey(sb *strings.Builder, title, path string) {
writeSection(sb, title, path)
var values strings.Builder
g.writeValues(&values, path, nil, "")
if values.Len() == 0 {
sb.WriteString("no values\n")
return
}
sb.WriteString(values.String())
}
// writeValues renders the values of a key. A nil names list reports every
// value, otherwise only those named and present.
func (g *BundleGenerator) writeValues(sb *strings.Builder, path string, names []string, indent string) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, path, registry.QUERY_VALUE)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, windows.ERROR_PATH_NOT_FOUND):
// an absent key is the normal state for the GPO store and for
// interfaces without DNS settings
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", path)
return
case err != nil:
fmt.Fprintf(sb, "%serror: open HKEY_LOCAL_MACHINE\\%s: %v\n", indent, path, err)
return
}
defer closeKey(k)
if names == nil {
names, err = k.ReadValueNames(-1)
if err != nil {
fmt.Fprintf(sb, "%serror: read value names: %v\n", indent, err)
return
}
}
for _, name := range names {
value, err := readRegistryValue(k, name)
switch {
case errors.Is(err, registry.ErrNotExist):
// the caller asks for a fixed set of values, most of which a
// given interface does not carry
continue
case err != nil:
// report rather than omit: a value that is there but cannot be
// read reads as unset otherwise
fmt.Fprintf(sb, "%s%s: error: %v\n", indent, name, err)
continue
}
fmt.Fprintf(sb, "%s%s: %s\n", indent, name, g.anonymizeValue(name, value))
}
}
// anonymizeValue redacts a registry value according to what its name says it
// holds. Domains and addresses are handled per entry rather than by the string
// pass: the pass only replaces domains something else in the bundle already
// seeded, and its address regex would eat the digit labels of a reverse zone.
func (g *BundleGenerator) anonymizeValue(name, value string) string {
if !g.anonymize || value == "" {
return value
}
switch {
case holdsDomains(name):
return joinValueEntries(splitValueEntries(value), g.anonymizeDomain)
case holdsAddresses(name):
return joinValueEntries(splitValueEntries(value), g.anonymizer.AnonymizeIPString)
default:
return g.anonymizer.AnonymizeString(value)
}
}
// holdsDomains reports whether a value name holds domains: the domain list of
// an NRPT rule (Name) or of the policy table (Namespace), a search list, the
// DNS suffix values of the TCP/IP and policy keys, which all end in "Domain"
// (Domain, DhcpDomain, NV Domain, ICSDomain), and a proxy host name.
func holdsDomains(name string) bool {
lower := strings.ToLower(name)
return lower == "name" || lower == "namespace" || lower == "searchlist" ||
strings.HasSuffix(lower, "domain") || strings.HasSuffix(lower, "proxyname")
}
// holdsAddresses reports whether a value name holds DNS server addresses
// (NameServer, DhcpNameServer, GenericDNSServers, NameServers).
func holdsAddresses(name string) bool {
lower := strings.ToLower(name)
return strings.Contains(lower, "nameserver") || strings.Contains(lower, "dnsserver")
}
// adapterNames maps adapter GUIDs, as the registry keys the interfaces, to the
// names an operator sees.
func adapterNames(adapters []*windows.IpAdapterAddresses) map[string]string {
names := make(map[string]string, len(adapters))
for _, adapter := range adapters {
guid := windows.BytePtrToString(adapter.AdapterName)
names[strings.ToLower(guid)] = windows.UTF16PtrToString(adapter.FriendlyName)
}
return names
}
// writeAdapterDNS reports the resolver configuration in effect per adapter,
// which is what the resolver uses for a name no NRPT rule matches.
func (g *BundleGenerator) writeAdapterDNS(sb *strings.Builder, adapters []*windows.IpAdapterAddresses, err error) {
writeSection(sb, "Adapter DNS configuration", "GetAdaptersAddresses")
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
for _, adapter := range adapters {
name := windows.UTF16PtrToString(adapter.FriendlyName)
suffix := g.anonymizeDomain(windows.UTF16PtrToString(adapter.DnsSuffix))
fmt.Fprintf(sb, "%s (index %d, oper status %d)\n", name, adapter.IfIndex, adapter.OperStatus)
fmt.Fprintf(sb, " DNS suffix: %s\n", suffix)
var servers []string
for server := adapter.FirstDnsServerAddress; server != nil; server = server.Next {
addr, ok := netip.AddrFromSlice(server.Address.IP())
if !ok {
continue
}
addr = addr.Unmap()
if g.anonymize {
addr = g.anonymizer.AnonymizeIP(addr)
}
servers = append(servers, addr.String())
}
fmt.Fprintf(sb, " DNS servers: %s\n", strings.Join(servers, ", "))
}
}
// anonymizeDomain anonymizes a single domain, keeping the leading dot an NRPT
// match domain carries.
func (g *BundleGenerator) anonymizeDomain(entry string) string {
if !g.anonymize {
return entry
}
domain, dot := strings.CutPrefix(entry, ".")
if domain == "" {
return entry
}
anonymized := g.anonymizer.AnonymizeDomain(domain)
if dot {
anonymized = "." + anonymized
}
return anonymized
}
// splitValueEntries splits a registry value that holds a list. The separator
// differs per value: a REG_MULTI_SZ arrives joined with ", ", a SearchList is
// comma separated and a NameServer may use commas or spaces.
func splitValueEntries(value string) []string {
return strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == ';' || r == ' ' || r == '\t'
})
}
func joinValueEntries(entries []string, anonymize func(string) string) string {
for i, entry := range entries {
entries[i] = anonymize(entry)
}
return strings.Join(entries, ", ")
}
func writeSection(sb *strings.Builder, title, source string) {
fmt.Fprintf(sb, "\n%s\n%s\n%s\n", title, strings.Repeat("-", len(title)), source)
}
func subKeyNames(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
if err != nil {
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closeKey(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
return names, nil
}
// readRegistryValue renders a value as text regardless of its type, so an
// unexpected type in a policy key still shows up instead of being dropped.
func readRegistryValue(k registry.Key, name string) (string, error) {
_, valueType, err := k.GetValue(name, nil)
if err != nil {
return "", fmt.Errorf("get value %s: %w", name, err)
}
switch valueType {
case registry.SZ, registry.EXPAND_SZ:
value, _, err := k.GetStringValue(name)
if err != nil {
return "", fmt.Errorf("get string value %s: %w", name, err)
}
return value, nil
case registry.MULTI_SZ:
values, _, err := k.GetStringsValue(name)
if err != nil {
return "", fmt.Errorf("get strings value %s: %w", name, err)
}
return strings.Join(values, ", "), nil
case registry.DWORD, registry.QWORD:
value, _, err := k.GetIntegerValue(name)
if err != nil {
return "", fmt.Errorf("get integer value %s: %w", name, err)
}
return fmt.Sprintf("%d (0x%x)", value, value), nil
case registry.BINARY:
value, _, err := k.GetBinaryValue(name)
if err != nil {
return "", fmt.Errorf("get binary value %s: %w", name, err)
}
return hex.EncodeToString(value), nil
default:
return fmt.Sprintf("<unhandled registry type %d>", valueType), nil
}
}
// adapterAddresses returns the adapter list including DNS servers. The call
// reports the size it needs, so grow the buffer and retry until it fits.
func adapterAddresses() (adapters []*windows.IpAdapterAddresses, err error) {
// GetAdaptersAddresses is resolved on first use and panics when it is
// missing, so this reports it as an error and leaves the rest of the
// report intact.
defer func() {
if r := recover(); r != nil {
adapters, err = nil, fmt.Errorf("GetAdaptersAddresses: %v", r)
}
}()
const flags = windows.GAA_FLAG_SKIP_ANYCAST | windows.GAA_FLAG_SKIP_MULTICAST
size := uint32(15000)
for range 3 {
buf := make([]byte, size)
first := (*windows.IpAdapterAddresses)(unsafe.Pointer(&buf[0]))
err := windows.GetAdaptersAddresses(windows.AF_UNSPEC, flags, 0, first, &size)
if errors.Is(err, windows.ERROR_BUFFER_OVERFLOW) {
continue
}
if err != nil {
return nil, fmt.Errorf("GetAdaptersAddresses: %w", err)
}
for adapter := first; adapter != nil; adapter = adapter.Next {
adapters = append(adapters, adapter)
}
return adapters, nil
}
return nil, fmt.Errorf("GetAdaptersAddresses: buffer kept growing")
}
func closeKey(k registry.Key) {
if err := k.Close(); err != nil {
log.Debugf("close registry key: %v", err)
}
}

View File

@@ -0,0 +1,146 @@
//go:build windows
package debug
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/anonymize"
)
func newDNSValueGenerator(level anonymize.Level) *BundleGenerator {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(level)
return &BundleGenerator{
anonymize: true,
anonymizeLevel: level,
anonymizer: anonymizer,
}
}
// TestAnonymizeValueByName covers the value kinds of the DNS registry keys. The
// names decide the treatment, because the string pass alone replaces only
// domains another part of the bundle already seeded.
func TestAnonymizeValueByName(t *testing.T) {
tests := []struct {
name string
valueName string
value string
assert func(t *testing.T, got string)
}{
{
name: "NRPT match domains keep the leading dot",
valueName: "Name",
value: ".internal.example.com, .corp.example.org",
assert: func(t *testing.T, got string) {
t.Helper()
for _, entry := range strings.Split(got, ", ") {
assert.True(t, strings.HasPrefix(entry, "."), "entry %q should keep its leading dot", entry)
assert.NotContains(t, entry, "example", "entry %q should not keep the original domain", entry)
}
},
},
{
name: "any value name ending in Domain is treated as a domain",
valueName: "ICSDomain",
value: "mshome.net",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "mshome", "should anonymize a domain suffix value")
},
},
{
name: "search list is a comma separated domain list",
valueName: "SearchList",
value: "corp.example.com,branch.example.com",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "example", "should anonymize every search domain")
assert.Len(t, strings.Split(got, ", "), 2, "should keep both search domains")
},
},
{
name: "name servers are anonymized as addresses",
valueName: "DhcpNameServer",
value: "203.0.113.10 8.8.8.8",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "203.0.113.10", "should anonymize a public resolver address")
// well-known resolvers stay readable at every level
assert.Contains(t, got, "8.8.8.8", "should keep a well-known resolver address")
},
},
{
name: "opaque values are left to the string pass",
valueName: "DataBasePath",
value: `%SystemRoot%\System32\drivers\etc`,
assert: func(t *testing.T, got string) {
t.Helper()
assert.Equal(t, `%SystemRoot%\System32\drivers\etc`, got, "should not alter a path")
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
g := newDNSValueGenerator(anonymize.LevelDefault)
tc.assert(t, g.anonymizeValue(tc.valueName, tc.value))
})
}
}
// TestParseNRPTPolicyTable parses the MOF text of the policy table out
// parameters, as the provider on a client with one NRPT rule renders it.
func TestParseNRPTPolicyTable(t *testing.T) {
const text = `[abstract]
class __PARAMETERS
{
[Out, EmbeddedInstance("DnsClientPolicyConfiguration"): ToSubClass, ID(2): DisableOverride ToInstance] DnsClientPolicyConfiguration cmdletOutput[] = {
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
DirectAccessQueryIPsecRequired = FALSE;
NameEncoding = "Utf8WithoutMapping";
Namespace = ".0.100.in-addr.arpa";
},
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
NameEncoding = "Utf8WithoutMapping";
NameServers = {"100.0.255.254", "100.0.255.253"};
Namespace = ".nb.internal";
}};
[in] boolean Effective;
[out] uint32 ReturnValue = 0;
};
`
entries := parseNRPTPolicyTable(text)
require.Len(t, entries, 2, "should parse both embedded instances")
assert.Equal(t, ".0.100.in-addr.arpa", entries[0].namespace, "should read the namespace of the first instance")
assert.Equal(t, ".nb.internal", entries[1].namespace, "should read the namespace of the second instance")
assert.Equal(t, []registryValue{
{name: "DirectAccessProxyType", value: "NoProxy"},
{name: "DirectAccessQueryIPsecRequired", value: "FALSE"},
{name: "NameEncoding", value: "Utf8WithoutMapping"},
}, entries[0].values, "should keep the remaining values in order")
assert.Contains(t, entries[1].values, registryValue{name: "NameServers", value: "100.0.255.254, 100.0.255.253"},
"should flatten a MOF array")
for _, value := range entries[1].values {
assert.NotContains(t, value.name, "ReturnValue", "should not read the class level parameters as values")
}
}
func TestParseNRPTPolicyTableEmpty(t *testing.T) {
assert.Empty(t, parseNRPTPolicyTable(""), "should parse no entries from empty text")
assert.Empty(t, parseNRPTPolicyTable("class __PARAMETERS\n{\n};\n"), "should parse no entries from a table with no instances")
}

View File

@@ -0,0 +1,317 @@
//go:build windows
package debug
import (
"errors"
"fmt"
"runtime"
"strings"
"time"
"github.com/go-ole/go-ole"
"github.com/go-ole/go-ole/oleutil"
log "github.com/sirupsen/logrus"
)
const (
// The NRPT policy table is reachable through the CIM class that backs
// Get-DnsClientNrptPolicy. Unlike the rules in the registry, the table is
// what the resolver currently has loaded, which is the only way to tell an
// applied rule from one that is merely written, in either direction.
nrptPolicyNamespace = `root\Microsoft\Windows\DNS`
nrptPolicyClass = "PS_DnsClientNrptPolicy"
nrptPolicyMethod = "Get"
// The class has no instances, so the table comes from the out parameters
// of a static method call, rendered as MOF text: the embedded instances
// arrive as a safe array of objects, which cannot be read back through the
// COM bindings, and the text form carries all of them.
nrptPolicyInstanceKeyword = "instance of DnsClientPolicyConfiguration"
nrptPolicyTimeout = 15 * time.Second
)
// COM initialization results that leave the calling thread usable: S_FALSE for
// a thread this process already initialized, RPC_E_CHANGED_MODE for one that
// belongs to another apartment.
const (
sFalse = 0x00000001
rpcEChangedMode = 0x80010106
)
// nrptQueryInFlight admits one read of the policy table at a time. A provider
// that stops answering keeps its goroutine and the OS thread that goroutine
// pinned, so a later bundle reports that instead of pinning another one.
var nrptQueryInFlight = make(chan struct{}, 1)
// nrptPolicyEntry is one namespace of the effective policy table, holding the
// values of an embedded DnsClientPolicyConfiguration instance in the order the
// provider reported them.
type nrptPolicyEntry struct {
namespace string
values []registryValue
}
// registryValue is a name and its rendered value, shared by the registry and
// policy table readers so both anonymize by value name the same way.
type registryValue struct {
name string
value string
}
// effectiveNRPTPolicies reads the effective NRPT table. The call is bounded
// because a WMI provider can block indefinitely and a debug bundle must not.
func effectiveNRPTPolicies() ([]nrptPolicyEntry, error) {
type result struct {
text string
err error
}
select {
case nrptQueryInFlight <- struct{}{}:
default:
return nil, errors.New("an earlier read of the policy table has not returned")
}
done := make(chan result, 1)
go func() {
// the slot is released here rather than by the caller, so a read that
// outlives the timeout holds it until the provider answers
defer func() { <-nrptQueryInFlight }()
text, err := nrptPolicyTableText()
done <- result{text: text, err: err}
}()
select {
case res := <-done:
if res.err != nil {
return nil, res.err
}
return parseNRPTPolicyTable(res.text), nil
case <-time.After(nrptPolicyTimeout):
return nil, errors.New("read of the policy table timed out")
}
}
// nrptPolicyTableText calls the policy table method and returns the MOF text of
// its out parameters.
func nrptPolicyTableText() (text string, err error) {
// COM is per thread, and the collection is short lived, so the thread is
// pinned for the duration rather than initialized for the process.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
defer func() {
// The COM call chain is dynamically typed, so a provider that answers
// with an unexpected shape must not take the daemon down with it.
if r := recover(); r != nil {
err = fmt.Errorf("read NRPT policy table: %v", r)
}
}()
owns, err := coInitialize()
if err != nil {
return "", err
}
if owns {
defer ole.CoUninitialize()
}
locator, err := oleutil.CreateObject("WbemScripting.SWbemLocator")
if err != nil {
return "", fmt.Errorf("create WMI locator: %w", err)
}
defer locator.Release()
dispatch, err := locator.QueryInterface(ole.IID_IDispatch)
if err != nil {
return "", fmt.Errorf("query WMI locator interface: %w", err)
}
defer dispatch.Release()
service, err := dispatchCall(dispatch, "ConnectServer", nil, nrptPolicyNamespace)
if err != nil {
return "", fmt.Errorf("connect to %s: %w", nrptPolicyNamespace, err)
}
defer service.Release()
inParams, err := spawnMethodInParams(service)
if err != nil {
return "", err
}
defer inParams.Release()
// The effective table is the merge of the local and the group policy
// store, which is what the resolver answers from.
if _, err := oleutil.PutProperty(inParams, "Effective", true); err != nil {
return "", fmt.Errorf("set Effective parameter: %w", err)
}
outParams, err := dispatchCall(service, "ExecMethod", nrptPolicyClass, nrptPolicyMethod, inParams)
if err != nil {
return "", fmt.Errorf("call %s.%s: %w", nrptPolicyClass, nrptPolicyMethod, err)
}
defer outParams.Release()
textVariant, err := oleutil.CallMethod(outParams, "GetObjectText_")
if err != nil {
return "", fmt.Errorf("render policy table: %w", err)
}
defer func() {
if err := textVariant.Clear(); err != nil {
log.Debugf("clear policy table variant: %v", err)
}
}()
return textVariant.ToString(), nil
}
// spawnMethodInParams builds the in parameters instance the method needs. The
// provider rejects the call without one, even when every parameter is optional.
func spawnMethodInParams(service *ole.IDispatch) (*ole.IDispatch, error) {
class, err := dispatchCall(service, "Get", nrptPolicyClass)
if err != nil {
return nil, fmt.Errorf("get class %s: %w", nrptPolicyClass, err)
}
defer class.Release()
methods, err := dispatchProperty(class, "Methods_")
if err != nil {
return nil, fmt.Errorf("get class methods: %w", err)
}
defer methods.Release()
method, err := dispatchCall(methods, "Item", nrptPolicyMethod)
if err != nil {
return nil, fmt.Errorf("get method %s: %w", nrptPolicyMethod, err)
}
defer method.Release()
params, err := dispatchProperty(method, "InParameters")
if err != nil {
return nil, fmt.Errorf("get method parameters: %w", err)
}
defer params.Release()
inParams, err := dispatchCall(params, "SpawnInstance_")
if err != nil {
return nil, fmt.Errorf("spawn parameter instance: %w", err)
}
return inParams, nil
}
// parseNRPTPolicyTable pulls the embedded instances out of the MOF text. Each
// instance is a namespace of the table, with one name and value per line.
func parseNRPTPolicyTable(text string) []nrptPolicyEntry {
var entries []nrptPolicyEntry
var current *nrptPolicyEntry
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(line), ";"))
switch {
case strings.HasPrefix(line, nrptPolicyInstanceKeyword):
entries = append(entries, nrptPolicyEntry{})
current = &entries[len(entries)-1]
continue
case strings.HasPrefix(line, "}"):
// closes an instance, and the array with the last one, so the
// class level parameters that follow are not read as values
current = nil
continue
case current == nil, line == "{":
continue
}
name, value, ok := strings.Cut(line, " = ")
if !ok {
continue
}
value = unquoteMOFValue(value)
if name == "Namespace" {
current.namespace = value
continue
}
current.values = append(current.values, registryValue{name: name, value: value})
}
return entries
}
// unquoteMOFValue renders a MOF scalar or array as plain text: "a" becomes a,
// and {"a", "b"} becomes a, b.
func unquoteMOFValue(value string) string {
value = strings.TrimSpace(value)
if inner, ok := strings.CutPrefix(value, "{"); ok {
value = strings.TrimSuffix(inner, "}")
entries := strings.Split(value, ",")
for i, entry := range entries {
entries[i] = strings.Trim(strings.TrimSpace(entry), `"`)
}
return strings.Join(entries, ", ")
}
return strings.Trim(value, `"`)
}
// coInitialize prepares the calling thread for COM and reports whether this
// call owns the initialization, which decides whether it may be balanced with
// CoUninitialize. S_FALSE took a reference on a thread this process had already
// initialized and so has to be released, while RPC_E_CHANGED_MODE took none:
// the thread belongs to another apartment, which is usable but is not ours to
// uninitialize.
func coInitialize() (bool, error) {
err := ole.CoInitializeEx(0, ole.COINIT_MULTITHREADED)
if err == nil {
return true, nil
}
var oleErr *ole.OleError
if errors.As(err, &oleErr) {
switch oleErr.Code() {
case sFalse:
return true, nil
case rpcEChangedMode:
return false, nil
}
}
return false, fmt.Errorf("initialize COM: %w", err)
}
// dispatchCall calls a COM method that returns an object.
func dispatchCall(dispatch *ole.IDispatch, method string, params ...any) (*ole.IDispatch, error) {
variant, err := oleutil.CallMethod(dispatch, method, params...)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("%s returned no object", method)
}
return object, nil
}
// dispatchProperty reads a COM property that holds an object.
func dispatchProperty(dispatch *ole.IDispatch, property string) (*ole.IDispatch, error) {
variant, err := oleutil.GetProperty(dispatch, property)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("property %s holds no object", property)
}
return object, nil
}

View File

@@ -31,10 +31,30 @@ var (
dnsFlushResolverCacheFn = dnsapi.NewProc("DnsFlushResolverCache")
)
// Registry locations of the host DNS configuration this package programs,
// exported so a diagnostic reader reports the same locations that are written.
const (
dnsPolicyConfigMatchPath = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig\NetBird-Match`
gpoDnsPolicyRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
gpoDnsPolicyConfigMatchPath = gpoDnsPolicyRoot + `\NetBird-Match`
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
// Older versions used different layouts under the same prefix: a single
// unsuffixed key, then one key per domain, now one key per batch of domains.
NRPTKeyPrefix = "NetBird-Match"
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
DNSPolicyConfigRoot = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig`
// GPODNSPolicyConfigRoot holds the NRPT rules of the group policy store,
// which takes precedence over the local one when it is present.
GPODNSPolicyConfigRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
// InterfaceConfigPath and InterfaceConfigPathV6 hold the per-interface DNS
// settings, keyed by interface GUID, in separate hives per address family.
InterfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
InterfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
)
const (
dnsPolicyConfigMatchPath = DNSPolicyConfigRoot + `\` + NRPTKeyPrefix
gpoDnsPolicyConfigMatchPath = GPODNSPolicyConfigRoot + `\` + NRPTKeyPrefix
dnsPolicyConfigVersionKey = "Version"
dnsPolicyConfigVersionValue = 2
@@ -45,8 +65,6 @@ const (
nrptMaxDomainsPerRule = 50
interfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
interfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
interfaceConfigNameServerKey = "NameServer"
interfaceConfigDhcpNameSrvKey = "DhcpNameServer"
interfaceConfigSearchListKey = "SearchList"
@@ -73,7 +91,6 @@ type registryConfigurator struct {
guid string
routingAll bool
gpo bool
nrptEntryCount int
origNameservers []netip.Addr
}
@@ -84,7 +101,7 @@ func newHostManager(wgInterface WGIface) (*registryConfigurator, error) {
}
var useGPO bool
k, err := registry.OpenKey(registry.LOCAL_MACHINE, gpoDnsPolicyRoot, registry.QUERY_VALUE)
k, err := registry.OpenKey(registry.LOCAL_MACHINE, GPODNSPolicyConfigRoot, registry.QUERY_VALUE)
if err != nil {
log.Debugf("failed to open GPO DNS policy root: %v", err)
} else {
@@ -123,7 +140,7 @@ func (r *registryConfigurator) captureOriginalNameservers() ([]netip.Addr, error
seen := make(map[netip.Addr]struct{})
var out []netip.Addr
var merr *multierror.Error
for _, root := range []string{interfaceConfigPath, interfaceConfigPathV6} {
for _, root := range []string{InterfaceConfigPath, InterfaceConfigPathV6} {
addrs, err := r.captureFromTcpipRoot(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("%s: %w", root, err))
@@ -306,14 +323,9 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
}
if len(matchDomains) != 0 {
count, err := r.addDNSMatchPolicy(matchDomains, config.ServerIP)
// Update count even on error to ensure cleanup covers partially created rules
r.nrptEntryCount = count
if err != nil {
if err := r.addDNSMatchPolicy(matchDomains, config.ServerIP); err != nil {
return fmt.Errorf("add dns match policy: %w", err)
}
} else {
r.nrptEntryCount = 0
}
r.updateState(stateManager)
@@ -329,9 +341,8 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
func (r *registryConfigurator) updateState(stateManager *statemanager.Manager) {
if err := stateManager.UpdateState(&ShutdownState{
Guid: r.guid,
GPO: r.gpo,
NRPTEntryCount: r.nrptEntryCount,
Guid: r.guid,
GPO: r.gpo,
}); err != nil {
log.Errorf("failed to update shutdown state: %s", err)
}
@@ -346,7 +357,7 @@ func (r *registryConfigurator) addDNSSetupForAll(ip netip.Addr) error {
return nil
}
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) (int, error) {
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) error {
// if the gpo key is present, we need to put our DNS settings there, otherwise our config might be ignored
// see https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-gpnrpt/8cc31cb9-20cb-4140-9e85-3e08703b4745
@@ -363,19 +374,17 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, ruleIndex)
if err := r.configureDNSPolicy(localPath, batchDomains, ip); err != nil {
return ruleIndex, fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
return fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
}
// Increment immediately so the caller's cleanup path knows about this rule
ruleIndex++
if r.gpo {
if err := r.configureDNSPolicy(gpoPath, batchDomains, ip); err != nil {
return ruleIndex, fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex-1, err)
return fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex, err)
}
}
log.Debugf("added NRPT rule %d with %d domains", ruleIndex-1, len(batchDomains))
log.Debugf("added NRPT rule %d with %d domains", ruleIndex, len(batchDomains))
ruleIndex++
}
if r.gpo {
@@ -385,7 +394,7 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
}
log.Infof("added %d NRPT rules for %d domains", ruleIndex, len(domains))
return ruleIndex, nil
return nil
}
func (r *registryConfigurator) configureDNSPolicy(policyPath string, domains []string, ip netip.Addr) error {
@@ -496,7 +505,7 @@ func (r *registryConfigurator) deleteInterfaceRegistryKeyProperty(propertyKey st
}
func (r *registryConfigurator) getInterfaceRegistryKey() (registry.Key, error) {
regKeyPath := interfaceConfigPath + "\\" + r.guid
regKeyPath := InterfaceConfigPath + "\\" + r.guid
regKey, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.SET_VALUE)
if err != nil {
return regKey, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", regKeyPath, err)
@@ -518,28 +527,28 @@ func (r *registryConfigurator) restoreHostDNS() error {
return nil
}
// removeDNSMatchPolicies deletes every NRPT rule this client may have created,
// from the local and the GPO policy store. The rules are found by enumerating
// the registry, the only authoritative record of what was written. Cleanup must
// not depend on a rule count: the in-memory one is scoped to a single
// registryConfigurator and the persisted one is deleted on every clean
// disconnect, and a rule left behind keeps resolving names over an interface
// that is gone, until reboot discards the volatile key.
func (r *registryConfigurator) removeDNSMatchPolicies() error {
var merr *multierror.Error
// Try to remove the base entries (for backward compatibility)
if err := removeRegistryKeyFromDNSPolicyConfig(dnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local base entry: %w", err))
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoDnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO base entry: %w", err))
}
for i := 0; i < r.nrptEntryCount; i++ {
localPath := fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i)
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, i)
if err := removeRegistryKeyFromDNSPolicyConfig(localPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local entry %d: %w", i, err))
for _, root := range []string{DNSPolicyConfigRoot, GPODNSPolicyConfigRoot} {
names, err := listNRPTRuleKeys(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("list rule keys under %s: %w", root, err))
continue
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO entry %d: %w", i, err))
for _, name := range names {
path := root + `\` + name
if err := removeRegistryKeyFromDNSPolicyConfig(path); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove entry %s: %w", path, err))
}
}
}
@@ -554,6 +563,39 @@ func (r *registryConfigurator) restoreUncleanShutdownDNS() error {
return r.restoreHostDNS()
}
// listNRPTRuleKeys returns the names of our NRPT rule keys under a policy store
// root. An absent root holds nothing to clean up, which is the normal state of
// the GPO store on a machine without DNS Client policy.
func listNRPTRuleKeys(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, syscall.ERROR_PATH_NOT_FOUND):
// the GPO store is absent on a machine without DNS client policy
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", root)
return nil, nil
case err != nil:
// any other failure has to reach the caller: reporting no rules would
// report a successful cleanup while leaving the rules in place
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closer(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
var ruleKeys []string
for _, name := range names {
// registry key names are case insensitive
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(NRPTKeyPrefix)) {
ruleKeys = append(ruleKeys, name)
}
}
return ruleKeys, nil
}
func removeRegistryKeyFromDNSPolicyConfig(regKeyPath string) error {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.QUERY_VALUE)
if err != nil {

View File

@@ -25,7 +25,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
interfacePath := InterfaceConfigPath + `\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -56,7 +56,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify 3 NRPT rules exist
assert.Equal(t, 3, cfg.nrptEntryCount, "Should create 3 NRPT rules for 125 domains")
assert.Equal(t, 3, countNRPTRuleKeys(t), "Should create 3 NRPT rules for 125 domains")
for i := 0; i < 3; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -81,7 +81,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify first 2 NRPT rules exist
assert.Equal(t, 2, cfg.nrptEntryCount, "Should create 2 NRPT rules for 75 domains")
assert.Equal(t, 2, countNRPTRuleKeys(t), "Should create 2 NRPT rules for 75 domains")
for i := 0; i < 2; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -106,9 +106,65 @@ func registryKeyExists(path string) (bool, error) {
return true, nil
}
// TestNRPTCleanupWithoutRuleCount verifies that rules written by a previous run
// are removed by a configurator that has no record of how many there are: an
// unclean exit loses the in-memory count and a clean disconnect deletes the
// persisted one, so cleanup cannot depend on either.
func TestNRPTCleanupWithoutRuleCount(t *testing.T) {
if testing.Short() {
t.Skip("skipping registry integration test in short mode")
}
defer cleanupRegistryKeys(t)
cleanupRegistryKeys(t)
testIP := netip.MustParseAddr("100.64.0.1")
// 75 domains produce two indexed rules, as the current layout does
domains := make([]string, 75)
for i := range domains {
domains[i] = fmt.Sprintf(".domain%d.com", i+1)
}
previousRun := &registryConfigurator{}
require.NoError(t, previousRun.addDNSMatchPolicy(domains, testIP))
// the unsuffixed key an older version would have written
require.NoError(t, previousRun.configureDNSPolicy(dnsPolicyConfigMatchPath, []string{".legacy.example.com"}, testIP))
// a policy owned by someone else, which cleanup must not touch
foreignPath := DNSPolicyConfigRoot + `\DnsPolicyConfigTestForeign`
foreignKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, foreignPath, registry.SET_VALUE)
require.NoError(t, err, "Should create foreign policy key")
foreignKey.Close()
defer func() {
_ = registry.DeleteKey(registry.LOCAL_MACHINE, foreignPath)
}()
require.Equal(t, 3, countNRPTRuleKeys(t), "Should have two indexed rules and the legacy one")
// a configurator that never applied a DNS config, as one built after a
// restart or from a shutdown state without a count is
freshRun := &registryConfigurator{}
require.NoError(t, freshRun.removeDNSMatchPolicies())
assert.Equal(t, 0, countNRPTRuleKeys(t), "Should remove every rule left by the previous run")
exists, err := registryKeyExists(foreignPath)
require.NoError(t, err)
assert.True(t, exists, "Should not remove a policy that is not ours")
}
func countNRPTRuleKeys(t *testing.T) int {
t.Helper()
names, err := listNRPTRuleKeys(DNSPolicyConfigRoot)
require.NoError(t, err, "Should list NRPT rule keys")
return len(names)
}
func cleanupRegistryKeys(*testing.T) {
// Clean up more entries to account for batching tests with many domains
cfg := &registryConfigurator{nrptEntryCount: 20}
cfg := &registryConfigurator{}
_ = cfg.removeDNSMatchPolicies()
}
@@ -125,7 +181,7 @@ func TestNRPTDomainBatching(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
interfacePath := InterfaceConfigPath + `\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -193,7 +249,7 @@ func TestNRPTDomainBatching(t *testing.T) {
require.NoError(t, err)
// Verify that exactly expectedRuleCount rules were created
assert.Equal(t, tc.expectedRuleCount, cfg.nrptEntryCount,
assert.Equal(t, tc.expectedRuleCount, countNRPTRuleKeys(t),
"Should create %d NRPT rules for %d domains", tc.expectedRuleCount, tc.domainCount)
// Verify all expected rules exist

View File

@@ -5,9 +5,8 @@ import (
)
type ShutdownState struct {
Guid string
GPO bool
NRPTEntryCount int
Guid string
GPO bool
}
func (s *ShutdownState) Name() string {
@@ -16,9 +15,8 @@ func (s *ShutdownState) Name() string {
func (s *ShutdownState) Cleanup() error {
manager := &registryConfigurator{
guid: s.Guid,
gpo: s.GPO,
nrptEntryCount: s.NRPTEntryCount,
guid: s.Guid,
gpo: s.GPO,
}
if err := manager.restoreUncleanShutdownDNS(); err != nil {

View File

@@ -2,17 +2,21 @@ package ebpf
import (
_ "embed"
"fmt"
"net"
"sync"
"github.com/cilium/ebpf/link"
"github.com/cilium/ebpf/rlimit"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
"github.com/netbirdio/netbird/client/internal/ebpf/manager"
)
const (
xdpProgName = "nb_xdp_prog"
mapKeyFeatures uint32 = 0
featureFlagWGProxy = 0b00000001
@@ -68,21 +72,50 @@ func (tf *GeneralManager) loadXdp() error {
return err
}
// load pre-compiled programs into the kernel.
err = loadBpfObjects(&tf.bpfObjs, nil)
// lo has no native XDP, so the program runs in generic mode. Unless it
// declares multi-buffer support the kernel must linearize every non-linear
// skb before running it. Loopback packets are up to 64 KB, so that is a
// contiguous GFP_ATOMIC allocation per packet, and when it fails the packet
// is dropped before the program runs, stalling local TCP connections.
// Multi-buffer XDP in generic mode requires kernel 6.3, so fall back to a
// plain attach when the kernel rejects it.
err = tf.attachXdp(iFace.Index, true)
if err == nil {
return nil
}
log.Debugf("failed to attach multi-buffer xdp program, retrying without it: %s", err)
return tf.attachXdp(iFace.Index, false)
}
func (tf *GeneralManager) attachXdp(iFaceIndex int, multiBuffer bool) error {
spec, err := loadBpf()
if err != nil {
return err
return fmt.Errorf("load bpf spec: %w", err)
}
if multiBuffer {
prog, ok := spec.Programs[xdpProgName]
if !ok {
return fmt.Errorf("program %s not found in bpf spec", xdpProgName)
}
prog.Flags |= unix.BPF_F_XDP_HAS_FRAGS
}
if err := spec.LoadAndAssign(&tf.bpfObjs, nil); err != nil {
return fmt.Errorf("load bpf objects: %w", err)
}
tf.link, err = link.AttachXDP(link.XDPOptions{
Program: tf.bpfObjs.NbXdpProg,
Interface: iFace.Index,
Interface: iFaceIndex,
})
if err != nil {
_ = tf.bpfObjs.Close()
if closeErr := tf.bpfObjs.Close(); closeErr != nil {
log.Debugf("failed to close bpf objects after xdp attach error: %s", closeErr)
}
tf.link = nil
return err
return fmt.Errorf("attach xdp: %w", err)
}
return nil
}

View File

@@ -59,6 +59,7 @@ import (
"github.com/netbirdio/netbird/client/internal/syncstore"
"github.com/netbirdio/netbird/client/internal/updater"
"github.com/netbirdio/netbird/client/jobexec"
"github.com/netbirdio/netbird/client/netevents"
cProto "github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/system"
nbdns "github.com/netbirdio/netbird/dns"
@@ -181,6 +182,9 @@ type EngineServices struct {
UpdateManager *updater.Manager
ClientMetrics *metrics.ClientMetrics
MetricsCtx context.Context
// NetState gates the reconnection loops on OS-reported network
// availability; nil disables gating.
NetState *netevents.Manager
}
// Engine is a mechanism responsible for reacting on Signal and Management stream events and managing connections to the remote peers.
@@ -204,6 +208,10 @@ type Engine struct {
config *EngineConfig
mobileDep MobileDependency
// netState gates the peer reconnection guards on OS-reported network
// availability; nil disables gating.
netState *netevents.Manager
// STUNs is a list of STUN servers used by ICE
STUNs []*stun.URI
// TURNs is a list of STUN servers used by ICE
@@ -337,6 +345,7 @@ func NewEngine(
syncMsgMux: &sync.Mutex{},
config: config,
mobileDep: mobileDep,
netState: services.NetState,
STUNs: []*stun.URI{},
TURNs: []*stun.URI{},
networkSerial: 0,
@@ -1893,7 +1902,8 @@ func (e *Engine) createPeerConn(pubKey string, allowedIPs []netip.Prefix, agentV
Addr: e.getRosenpassAddr(),
PermissiveMode: e.config.RosenpassPermissive,
},
ICEConfig: e.createICEConfig(),
ICEConfig: e.createICEConfig(),
NetworkState: e.netState,
}
serviceDependencies := peer.ServiceDependencies{

View File

@@ -26,6 +26,7 @@ import (
"github.com/netbirdio/netbird/client/internal/portforward"
"github.com/netbirdio/netbird/client/internal/rosenpass"
"github.com/netbirdio/netbird/client/internal/stdnet"
"github.com/netbirdio/netbird/client/netevents"
"github.com/netbirdio/netbird/route"
relayClient "github.com/netbirdio/netbird/shared/relay/client"
)
@@ -93,6 +94,10 @@ type ConnConfig struct {
// ICEConfig ICE protocol configuration
ICEConfig icemaker.Config
// NetworkState gates the reconnection guard on OS-reported network
// availability; nil disables gating.
NetworkState *netevents.Manager
}
type Conn struct {
@@ -254,7 +259,7 @@ func (conn *Conn) open(engineCtx context.Context, firstPacket []byte) error {
conn.handshaker.AddICEListener(conn.workerICE.OnNewOffer)
}
conn.guard = guard.NewGuard(conn.Log, conn.isConnectedOnAllWay, conn.config.Timeout, conn.srWatcher)
conn.guard = guard.NewGuard(conn.Log, conn.isConnectedOnAllWay, conn.config.Timeout, conn.srWatcher, conn.config.NetworkState)
conn.wg.Add(1)
go func() {

View File

@@ -22,6 +22,12 @@ const (
type connStatusFunc func() ConnStatus
// NetworkWatcher is the availability view the guard gates reconnects on.
type NetworkWatcher interface {
IsOnline() bool
Changed() <-chan struct{}
}
// Guard is responsible for the reconnection logic.
// It will trigger to send an offer to the peer then has connection issues.
// Watch these events:
@@ -31,20 +37,26 @@ type connStatusFunc func() ConnStatus
// - Relayed connection disconnected
// - ICE candidate changes
type Guard struct {
log *log.Entry
isConnectedOnAllWay connStatusFunc
timeout time.Duration
srWatcher *SRWatcher
log *log.Entry
isConnectedOnAllWay connStatusFunc
timeout time.Duration
srWatcher *SRWatcher
// netState gates reconnect attempts on OS-reported network availability;
// nil disables gating.
netState NetworkWatcher
relayedConnDisconnected chan struct{}
iCEConnDisconnected chan struct{}
}
func NewGuard(log *log.Entry, isConnectedFn connStatusFunc, timeout time.Duration, srWatcher *SRWatcher) *Guard {
// NewGuard creates a reconnection guard for a peer connection. A nil netState
// disables network availability gating.
func NewGuard(log *log.Entry, isConnectedFn connStatusFunc, timeout time.Duration, srWatcher *SRWatcher, netState NetworkWatcher) *Guard {
return &Guard{
log: log,
isConnectedOnAllWay: isConnectedFn,
timeout: timeout,
srWatcher: srWatcher,
netState: netState,
relayedConnDisconnected: make(chan struct{}, 1),
iCEConnDisconnected: make(chan struct{}, 1),
}
@@ -96,9 +108,19 @@ func (g *Guard) reconnectLoopWithRetry(ctx context.Context, callback func()) {
iceState := &iceRetryState{log: g.log}
defer iceState.reset()
var netChanged <-chan struct{}
if g.netState != nil {
netChanged = g.netState.Changed()
}
for {
select {
case <-tickerChannel:
// skip attempts while the OS reports no usable network; the
// netChanged case below resumes the loop once it returns
if g.netState != nil && !g.netState.IsOnline() {
continue
}
switch g.isConnectedOnAllWay() {
case ConnStatusConnected:
// all good, nothing to do
@@ -135,6 +157,23 @@ func (g *Guard) reconnectLoopWithRetry(ctx context.Context, callback func()) {
tickerChannel = ticker.C
iceState.reset()
case <-netChanged:
// Re-arm for the next transition before acting on this one.
netChanged = g.netState.Changed()
if !g.netState.IsOnline() {
continue
}
// Ticks skipped while offline drove the backoff towards its
// maximum without ever attempting, and left the ICE budget
// frozen — possibly in hourly mode. Recover on our own so the
// peer does not depend on a signal or relay event that never
// comes when both stayed up across the outage.
g.log.Debugf("network is back, reset reconnection ticker")
ticker.Stop()
ticker = g.newReconnectTicker(ctx)
tickerChannel = ticker.C
iceState.reset()
case <-ctx.Done():
g.log.Debugf("context is done, stop reconnect loop")
return

View File

@@ -15,7 +15,7 @@ import (
func newTestGuard(status connStatusFunc) *Guard {
srw := NewSRWatcher(nil, nil, nil, ice.Config{})
return NewGuard(log.WithField("test", "guard"), status, 50*time.Millisecond, srw)
return NewGuard(log.WithField("test", "guard"), status, 50*time.Millisecond, srw, nil)
}
// countBackoffTickerGoroutines returns how many goroutines are currently sitting

View File

@@ -0,0 +1,107 @@
package guard
import (
"context"
"sync/atomic"
"testing"
"time"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/peer/ice"
"github.com/netbirdio/netbird/client/netevents/netstate"
)
// newTestGuardWithNetState builds a guard with a realistic MaxInterval: the
// backoff must be able to grow well past the outage, as it does in production
// where the timeout is seconds to minutes.
func newTestGuardWithNetState(status connStatusFunc, netState *netstate.State) *Guard {
srw := NewSRWatcher(nil, nil, nil, ice.Config{})
return NewGuard(log.WithField("test", "guard"), status, 30*time.Second, srw, netState)
}
// TestGuard_RecoversAfterOfflineToOnline covers a peer that stays disconnected
// across a network outage while neither signal nor relay reports an event —
// both stayed up, as on a short airplane mode toggle over Wi-Fi.
//
// Every tick taken while offline is skipped, but it still advances the
// exponential backoff, so by the time the network returns the next tick can be
// tens of seconds away. Without an explicit reaction to the transition the
// peer waits out that interval for a recovery that could start immediately.
func TestGuard_RecoversAfterOfflineToOnline(t *testing.T) {
netState := netstate.New()
var attempts atomic.Int32
g := newTestGuardWithNetState(func() ConnStatus { return ConnStatusDisconnected }, netState)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Start from the reconnect ticker (800ms initial interval), the state a
// peer is in after it loses its connection.
go g.Start(ctx, func() { attempts.Add(1) })
g.SetRelayedConnDisconnected()
// Let the backoff climb: 0.8s, 1.6s, 3.2s, 6.4s ... every tick is skipped
// while offline, but each one doubles the wait for the next.
netState.Set(false)
time.Sleep(8 * time.Second)
offlineAttempts := attempts.Load()
if offlineAttempts != 0 {
t.Fatalf("callback ran %d times while offline, want 0", offlineAttempts)
}
netState.Set(true)
// The next organic tick is now several seconds out, so anything within
// this window can only come from reacting to the transition itself.
pollCtx, stopPolling := context.WithTimeout(ctx, 2*time.Second)
defer stopPolling()
select {
case <-pollCtx.Done():
t.Fatal("peer was not retried within 2s of the network coming back, " +
"with neither a signal nor a relay event to fall back on")
case <-pollUntil(pollCtx, func() bool { return attempts.Load() > 0 }):
}
}
// TestGuard_OfflineTransitionDoesNotRetry checks the other direction: going
// offline must not itself trigger an attempt.
func TestGuard_OfflineTransitionDoesNotRetry(t *testing.T) {
netState := netstate.New()
var attempts atomic.Int32
g := newTestGuardWithNetState(func() ConnStatus { return ConnStatusDisconnected }, netState)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go g.Start(ctx, func() { attempts.Add(1) })
netState.Set(false)
time.Sleep(5 * time.Second)
if got := attempts.Load(); got != 0 {
t.Fatalf("callback ran %d times after going offline, want 0", got)
}
}
// pollUntil closes the returned channel once cond holds. It gives up when ctx
// is done, so the polling goroutine never outlives the test that started it.
func pollUntil(ctx context.Context, cond func() bool) <-chan struct{} {
done := make(chan struct{})
go func() {
for {
if cond() {
close(done)
return
}
select {
case <-ctx.Done():
return
case <-time.After(10 * time.Millisecond):
}
}
}()
return done
}

View File

@@ -1,11 +1,40 @@
package peer
// ClientState identifies the client connection state delivered via
// Listener.OnStateChanged.
type ClientState int
// Client states. The numeric values cross the gomobile boundary (the mobile
// bindings re-export them as integer constants), so they are a wire format:
// append new states at the end, never reorder or insert.
const (
ClientStateDisconnected ClientState = iota
ClientStateConnected
ClientStateConnecting
ClientStateDisconnecting
// ClientStateNoNetwork is an overlay state: it is never stored as the
// last notification, only derived from ClientStateConnecting while the
// OS reports no usable network (see notifier.effectiveState).
ClientStateNoNetwork
)
// Listener is a callback type about the NetBird network connection state
type Listener interface {
// OnStateChanged reports every client state transition. New states are
// delivered only through this callback; the per-state callbacks below
// are kept for compatibility and will be removed once all consumers
// have migrated.
OnStateChanged(state ClientState)
// Deprecated: consume OnStateChanged instead.
OnConnected()
// Deprecated: consume OnStateChanged instead.
OnDisconnected()
// Deprecated: consume OnStateChanged instead.
OnConnecting()
// Deprecated: consume OnStateChanged instead.
OnDisconnecting()
OnAddressChanged(string, string)
OnPeersListChanged(int)
}

View File

@@ -4,31 +4,64 @@ import (
"sync"
)
const (
stateDisconnected = iota
stateConnected
stateConnecting
stateDisconnecting
)
type notifier struct {
// publishLock orders state publication: it is held across computing the
// effective state and handing it to the listener, so a transition cannot
// overtake a newer one and leave the listener on a stale state.
publishLock sync.Mutex
serverStateLock sync.Mutex
listenersLock sync.Mutex
listener Listener
currentClientState bool
lastNotification int
lastNotification ClientState
lastNumberOfPeers int
lastFqdnAddress string
lastIPAddress string
networkAvailable bool
}
func newNotifier() *notifier {
return &notifier{}
return &notifier{
networkAvailable: true,
}
}
// effectiveState maps the computed state to what listeners should see:
// while the OS reports no usable network, "Connecting" would be a lie —
// connection attempts are suspended — so it is reported as NoNetwork.
// Caller must hold serverStateLock.
func (n *notifier) effectiveState(state ClientState) ClientState {
if !n.networkAvailable && state == ClientStateConnecting {
return ClientStateNoNetwork
}
return state
}
// setNetworkAvailable records the OS network availability and re-notifies
// the listener when the flag flips the effective state (Connecting <->
// NoNetwork).
func (n *notifier) setNetworkAvailable(available bool) {
n.publishLock.Lock()
defer n.publishLock.Unlock()
n.serverStateLock.Lock()
if n.networkAvailable == available {
n.serverStateLock.Unlock()
return
}
previous := n.effectiveState(n.lastNotification)
n.networkAvailable = available
current := n.effectiveState(n.lastNotification)
n.serverStateLock.Unlock()
if previous != current {
n.notify(current)
}
}
func (n *notifier) setListener(listener Listener) {
n.serverStateLock.Lock()
lastNotification := n.lastNotification
lastNotification := n.effectiveState(n.lastNotification)
numOfPeers := n.lastNumberOfPeers
fqdnAddress := n.lastFqdnAddress
address := n.lastIPAddress
@@ -52,6 +85,9 @@ func (n *notifier) removeListener() {
}
func (n *notifier) updateServerStates(mgmState bool, signalState bool) {
n.publishLock.Lock()
defer n.publishLock.Unlock()
n.serverStateLock.Lock()
calculatedState := n.calculateState(mgmState, signalState)
@@ -61,43 +97,54 @@ func (n *notifier) updateServerStates(mgmState bool, signalState bool) {
}
n.lastNotification = calculatedState
effective := n.effectiveState(calculatedState)
n.serverStateLock.Unlock()
n.notify(calculatedState)
n.notify(effective)
}
func (n *notifier) clientStart() {
n.publishLock.Lock()
defer n.publishLock.Unlock()
n.serverStateLock.Lock()
n.currentClientState = true
n.lastNotification = stateConnecting
n.lastNotification = ClientStateConnecting
effective := n.effectiveState(ClientStateConnecting)
n.serverStateLock.Unlock()
n.notify(stateConnecting)
n.notify(effective)
}
func (n *notifier) clientStop() {
n.publishLock.Lock()
defer n.publishLock.Unlock()
n.serverStateLock.Lock()
n.currentClientState = false
n.lastNotification = stateDisconnected
n.lastNotification = ClientStateDisconnected
n.serverStateLock.Unlock()
n.notify(stateDisconnected)
n.notify(ClientStateDisconnected)
}
func (n *notifier) clientTearDown() {
n.publishLock.Lock()
defer n.publishLock.Unlock()
n.serverStateLock.Lock()
n.currentClientState = false
n.lastNotification = stateDisconnecting
n.lastNotification = ClientStateDisconnecting
n.serverStateLock.Unlock()
n.notify(stateDisconnecting)
n.notify(ClientStateDisconnecting)
}
func (n *notifier) isServerStateChanged(newState int) bool {
func (n *notifier) isServerStateChanged(newState ClientState) bool {
return n.lastNotification != newState
}
func (n *notifier) notify(state int) {
func (n *notifier) notify(state ClientState) {
n.listenersLock.Lock()
listener := n.listener
n.listenersLock.Unlock()
@@ -109,20 +156,20 @@ func (n *notifier) notify(state int) {
notifyListener(listener, state)
}
func (n *notifier) calculateState(managementConn, signalConn bool) int {
func (n *notifier) calculateState(managementConn, signalConn bool) ClientState {
if managementConn && signalConn {
return stateConnected
return ClientStateConnected
}
if !managementConn && !signalConn && !n.currentClientState {
return stateDisconnected
return ClientStateDisconnected
}
if n.lastNotification == stateDisconnecting {
return stateDisconnecting
if n.lastNotification == ClientStateDisconnecting {
return ClientStateDisconnecting
}
return stateConnecting
return ClientStateConnecting
}
func (n *notifier) peerListChanged(numOfPeers int) {
@@ -159,15 +206,19 @@ func (n *notifier) localAddressChanged(fqdn, address string) {
listener.OnAddressChanged(fqdn, address)
}
func notifyListener(l Listener, state int) {
func notifyListener(l Listener, state ClientState) {
// legacy per-state callbacks; NoNetwork is delivered only via
// OnStateChanged below
switch state {
case stateDisconnected:
case ClientStateDisconnected:
l.OnDisconnected()
case stateConnected:
case ClientStateConnected:
l.OnConnected()
case stateConnecting:
case ClientStateConnecting:
l.OnConnecting()
case stateDisconnecting:
case ClientStateDisconnecting:
l.OnDisconnecting()
}
l.OnStateChanged(state)
}

View File

@@ -0,0 +1,108 @@
package peer
import (
"sync"
"testing"
"time"
)
type recordingListener struct {
mu sync.Mutex
states []ClientState
onState func(ClientState)
}
func (l *recordingListener) OnStateChanged(state ClientState) {
l.mu.Lock()
l.states = append(l.states, state)
hook := l.onState
l.mu.Unlock()
if hook != nil {
hook(state)
}
}
func (l *recordingListener) last() (ClientState, bool) {
l.mu.Lock()
defer l.mu.Unlock()
if len(l.states) == 0 {
return 0, false
}
return l.states[len(l.states)-1], true
}
func (l *recordingListener) snapshot() []ClientState {
l.mu.Lock()
defer l.mu.Unlock()
return append([]ClientState(nil), l.states...)
}
func (l *recordingListener) OnConnected() {}
func (l *recordingListener) OnDisconnected() {}
func (l *recordingListener) OnConnecting() {}
func (l *recordingListener) OnDisconnecting() {}
func (l *recordingListener) OnAddressChanged(string, string) {}
func (l *recordingListener) OnPeersListChanged(int) {}
// TestNotifier_ConcurrentAvailabilityFlipOrdersPublication holds the first
// transition inside the listener callback and flips availability again from
// another goroutine while it is parked. The second flip must not publish
// ahead of the one in flight, otherwise the listener ends up on a state the
// notifier already superseded.
func TestNotifier_ConcurrentAvailabilityFlipOrdersPublication(t *testing.T) {
n := newNotifier()
n.currentClientState = true
n.lastNotification = ClientStateConnecting
entered := make(chan struct{})
release := make(chan struct{})
l := &recordingListener{}
l.onState = func(state ClientState) {
if state != ClientStateNoNetwork {
return
}
l.mu.Lock()
l.onState = nil
l.mu.Unlock()
close(entered)
<-release
}
n.listener = l
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
n.setNetworkAvailable(false)
}()
<-entered
flipped := make(chan struct{})
go func() {
defer close(flipped)
n.setNetworkAvailable(true)
}()
select {
case <-flipped:
t.Fatal("the online transition published while the offline one was " +
"still in flight; publication is not serialized")
case <-time.After(200 * time.Millisecond):
}
close(release)
<-flipped
wg.Wait()
got, ok := l.last()
if !ok {
t.Fatal("listener never observed a state")
}
if got != ClientStateConnecting {
t.Fatalf("listener holds %v after the network came back, want Connecting; sequence: %v",
got, l.snapshot())
}
}

View File

@@ -6,29 +6,32 @@ import (
)
type mocListener struct {
lastState int
lastState ClientState
wg sync.WaitGroup
peersWg sync.WaitGroup
peers int
}
func (l *mocListener) OnConnected() {
l.lastState = stateConnected
l.lastState = ClientStateConnected
l.wg.Done()
}
func (l *mocListener) OnDisconnected() {
l.lastState = stateDisconnected
l.lastState = ClientStateDisconnected
l.wg.Done()
}
func (l *mocListener) OnConnecting() {
l.lastState = stateConnecting
l.lastState = ClientStateConnecting
l.wg.Done()
}
func (l *mocListener) OnDisconnecting() {
l.lastState = stateDisconnecting
l.lastState = ClientStateDisconnecting
l.wg.Done()
}
func (l *mocListener) OnStateChanged(state ClientState) {
}
func (l *mocListener) OnAddressChanged(host, addr string) {
}
@@ -57,15 +60,15 @@ func Test_notifier_serverState(t *testing.T) {
type scenario struct {
name string
expected int
expected ClientState
mgmState bool
signalState bool
}
scenarios := []scenario{
{"connected", stateConnected, true, true},
{"mgm down", stateConnecting, false, true},
{"signal down", stateConnecting, true, false},
{"disconnected", stateDisconnected, false, false},
{"connected", ClientStateConnected, true, true},
{"mgm down", ClientStateConnecting, false, true},
{"signal down", ClientStateConnecting, true, false},
{"disconnected", ClientStateDisconnected, false, false},
}
for _, tt := range scenarios {
@@ -85,7 +88,7 @@ func Test_notifier_SetListener(t *testing.T) {
listener.setPeersWaiter()
n := newNotifier()
n.lastNotification = stateConnecting
n.lastNotification = ClientStateConnecting
n.setListener(listener)
listener.wait()
listener.waitPeers()
@@ -99,7 +102,7 @@ func Test_notifier_RemoveListener(t *testing.T) {
listener.setWaiter()
listener.setPeersWaiter()
n := newNotifier()
n.lastNotification = stateConnecting
n.lastNotification = ClientStateConnecting
n.setListener(listener)
// setListener replays cached state on a goroutine; wait for both the state
// and peers callbacks to finish so we don't race on listener.peers.

View File

@@ -1211,6 +1211,12 @@ func (d *Status) ClientTeardown() {
d.notifyStateChange()
}
// SetNetworkAvailable records the OS-reported network availability; while
// unavailable, listeners see NoNetwork instead of Connecting.
func (d *Status) SetNetworkAvailable(available bool) {
d.notifier.setNetworkAvailable(available)
}
// SetConnectionListener set a listener to the notifier
func (d *Status) SetConnectionListener(listener Listener) {
d.notifier.setListener(listener)

View File

@@ -0,0 +1,130 @@
package profilemanager
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"github.com/netbirdio/netbird/util"
)
const prefsFileSuffix = ".prefs.json"
var prefsMu sync.Mutex
// Prefs is a namespaced per-profile preference store backed by a single JSON
// file next to the profile config; it is deleted together with the profile.
type Prefs struct {
path string
}
// ProfilePrefs returns the preference store of the profile identified by id.
func (s *ServiceManager) ProfilePrefs(id ID, username string) (*Prefs, error) {
if !IsValidProfileFilenameStem(id) {
return nil, fmt.Errorf("invalid profile ID: %q", id)
}
if id == defaultProfileName {
return &Prefs{path: filepath.Join(filepath.Dir(DefaultConfigPath), id.String()+prefsFileSuffix)}, nil
}
configDir, err := s.getConfigDir(username)
if err != nil {
return nil, fmt.Errorf("get config directory for user %s: %w", username, err)
}
return &Prefs{path: filepath.Join(configDir, id.String()+prefsFileSuffix)}, nil
}
// Get unmarshals the namespace section into v and reports whether it exists.
func (p *Prefs) Get(namespace string, v any) (bool, error) {
if namespace == "" {
return false, fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return false, err
}
raw, ok := sections[namespace]
if !ok {
return false, nil
}
if err := json.Unmarshal(raw, v); err != nil {
return false, fmt.Errorf("decode prefs namespace %q: %w", namespace, err)
}
return true, nil
}
// Put stores v as the namespace section, replacing any previous value.
func (p *Prefs) Put(namespace string, v any) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
raw, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("encode prefs namespace %q: %w", namespace, err)
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
sections[namespace] = raw
return writePrefsFile(p.path, sections)
}
// Remove deletes the namespace section; a missing one is not an error.
func (p *Prefs) Remove(namespace string) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
if _, ok := sections[namespace]; !ok {
return nil
}
delete(sections, namespace)
return writePrefsFile(p.path, sections)
}
func removePrefsFile(path string) error {
prefsMu.Lock()
defer prefsMu.Unlock()
return os.Remove(path)
}
func readPrefsFile(path string) (map[string]json.RawMessage, error) {
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return map[string]json.RawMessage{}, nil
}
if err != nil {
return nil, fmt.Errorf("read prefs: %w", err)
}
sections := map[string]json.RawMessage{}
if err := json.Unmarshal(data, &sections); err != nil {
return nil, fmt.Errorf("decode prefs: %w", err)
}
return sections, nil
}
func writePrefsFile(path string, sections map[string]json.RawMessage) error {
if err := util.WriteJsonWithRestrictedPermission(context.Background(), path, sections); err != nil {
return fmt.Errorf("write prefs: %w", err)
}
return nil
}

View File

@@ -0,0 +1,138 @@
package profilemanager
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type testPrefsSection struct {
Mode uint8 `json:"mode"`
Dest string `json:"dest"`
}
func TestProfilePrefs_RoundTrip(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2, Dest: "/tmp/x"}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, testPrefsSection{Mode: 2, Dest: "/tmp/x"}, got)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_GetMissingNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
})
}
func TestProfilePrefs_RemoveNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
require.NoError(t, prefs.Remove("filedrop"))
require.NoError(t, prefs.Remove("missing"))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_RejectsInvalidID(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
_, err := sm.ProfilePrefs("../escape", username)
assert.Error(t, err)
})
}
func TestProfilePrefs_RejectsEmptyNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
_, err = prefs.Get("", &testPrefsSection{})
assert.Error(t, err)
assert.Error(t, prefs.Put("", testPrefsSection{}))
assert.Error(t, prefs.Remove(""))
})
}
func TestProfilePrefs_DefaultProfile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
prefs, err := sm.ProfilePrefs(defaultProfileName, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
expected := filepath.Join(filepath.Dir(DefaultConfigPath), "default"+prefsFileSuffix)
_, err = os.Stat(expected)
require.NoError(t, err)
})
}
func TestRemoveProfile_DeletesPrefsFile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2}))
configDir, err := sm.getConfigDir(username)
require.NoError(t, err)
prefsPath := filepath.Join(configDir, created.ID.String()+prefsFileSuffix)
_, err = os.Stat(prefsPath)
require.NoError(t, err)
require.NoError(t, sm.RemoveProfile(created.ID, username))
_, err = os.Stat(prefsPath)
assert.True(t, errors.Is(err, os.ErrNotExist), "prefs file should be removed")
})
}

View File

@@ -420,6 +420,11 @@ func (s *ServiceManager) RemoveProfile(id ID, username string) error {
log.Warnf("failed to remove profile state file %s: %v", stateFile, err)
}
prefsFile := filepath.Join(filepath.Dir(target.Path), id.String()+prefsFileSuffix)
if err := removePrefsFile(prefsFile); err != nil && !os.IsNotExist(err) {
log.Warnf("failed to remove profile prefs file %s: %v", prefsFile, err)
}
return nil
}

View File

@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -0,0 +1,82 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
}
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
return candidates[i].routeMetric < candidates[j].routeMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest route metric
// 3. Lowest interface metric
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -5,6 +5,7 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -37,23 +37,32 @@
// Updater Process (Setup):
//
// 1. Receives parameters from service via command-line arguments
// 2. Runs installer with appropriate silent/quiet flags:
// 2. Terminates the UI so the installer does not have to replace a locked image
// file, which would otherwise leave the install needing a reboot
// 3. Runs installer with appropriate silent/quiet flags:
// - Windows EXE: installer.exe /S
// - Windows MSI: msiexec.exe /i installer.msi /quiet /qn /l*v msi.log
// - Windows MSI: msiexec.exe /i installer.msi /qn /norestart REBOOT=ReallySuppress /l*v msi.log
// - macOS PKG: installer -pkg installer.pkg -target /
// - macOS Homebrew: brew upgrade netbirdio/tap/netbird
// 3. Installer terminates daemon and UI processes
// 4. Installer replaces binaries with new version
// 5. Updater waits for installer to complete
// 6. Updater restarts daemon:
// 4. Installer terminates the daemon
// 5. Installer replaces binaries with new version
// 6. Updater waits for installer to complete. On Windows, MSI exit codes 3010
// (ERROR_SUCCESS_REBOOT_REQUIRED) and 1641 (ERROR_SUCCESS_REBOOT_INITIATED)
// are a pending-reboot outcome, not a failure: the install succeeded, but
// some files are only replaced on the next restart (the reboot itself is
// suppressed via /norestart and REBOOT=ReallySuppress), and the flow
// continues as on success
// 7. Updater restarts daemon:
// - Windows: netbird.exe service start
// - macOS/Linux: netbird service start
// 7. Updater restarts UI:
// - Windows: Launches netbird-ui.exe as active console user using CreateProcessAsUser
// 8. Updater restarts UI:
// - Windows: Launches netbird-ui.exe using CreateProcessAsUser in every
// session it was terminated in, falling back to the active console session
// - macOS: Uses launchctl asuser to launch NetBird.app for console user
// - Linux: Not implemented (UI typically auto-starts)
// 8. Updater writes result.json with success/error status
// 9. Updater process exits
// 9. Updater writes result.json with success/error status (a pending reboot is
// recorded as success)
// 10. Updater process exits
//
// # Result Communication
//

View File

@@ -42,6 +42,9 @@ func NewWithDir(tempDir string) *Installer {
// This will run by the original service process
func (u *Installer) RunInstallation(ctx context.Context, targetVersion string) (err error) {
resultHandler := NewResultHandler(u.tempDir)
if err := resultHandler.ClearStaleResult(); err != nil {
log.Warnf("clear stale installer result: %v", err)
}
defer func() {
if err != nil {

View File

@@ -2,6 +2,7 @@ package installer
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
@@ -22,6 +23,12 @@ const (
msiLogFile = "msi.log"
// ERROR_SUCCESS_REBOOT_REQUIRED and ERROR_SUCCESS_REBOOT_INITIATED
msiRebootRequired = 3010
msiRebootInitiated = 1641
processExitWait = 10 * time.Second
msiDownloadURL = "https://github.com/netbirdio/netbird/releases/download/v%version/netbird_installer_%version_windows_%arch.msi"
exeDownloadURL = "https://github.com/netbirdio/netbird/releases/download/v%version/netbird_installer_%version_windows_%arch.exe"
)
@@ -38,6 +45,8 @@ var (
func (u *Installer) Setup(ctx context.Context, dryRun bool, installerFile string, daemonFolder string) (resultErr error) {
resultHandler := NewResultHandler(u.tempDir)
var uiSessions []uint32
// Always ensure daemon and UI are restarted after setup
defer func() {
log.Infof("starting daemon back")
@@ -46,7 +55,7 @@ func (u *Installer) Setup(ctx context.Context, dryRun bool, installerFile string
}
log.Infof("starting UI back")
if err := u.startUIAsUser(daemonFolder); err != nil {
if err := u.startUI(daemonFolder, uiSessions); err != nil {
log.Errorf("failed to start UI: %v", err)
}
@@ -75,6 +84,14 @@ func (u *Installer) Setup(ctx context.Context, dryRun bool, installerFile string
return
}
// The UI holds an open handle on its own image. Left running, Restart Manager
// cannot shut it down (msiexec runs as LocalSystem here, the UI as the
// interactive user), so the MSI falls back to replacing the file on reboot and
// marks the install as restart-required. The deferred close-application action
// in the package runs too late to prevent that, it happens after
// InstallValidate has already registered the file as in use.
uiSessions = killUI()
var cmd *exec.Cmd
switch installerType {
case TypeExe:
@@ -84,7 +101,9 @@ func (u *Installer) Setup(ctx context.Context, dryRun bool, installerFile string
installerDir := filepath.Dir(installerFile)
logPath := filepath.Join(installerDir, msiLogFile)
log.Infof("run msi installer: %s", installerFile)
cmd = exec.CommandContext(ctx, "msiexec.exe", "/i", filepath.Base(installerFile), "/quiet", "/qn", "/l*v", logPath)
// REBOOT=ReallySuppress: a silent install has no way to ask, so without it
// msiexec reboots the machine on its own if it decides one is needed.
cmd = exec.CommandContext(ctx, "msiexec.exe", "/i", filepath.Base(installerFile), "/qn", "/norestart", "REBOOT=ReallySuppress", "/l*v", logPath)
}
cmd.Dir = filepath.Dir(installerFile)
@@ -95,9 +114,13 @@ func (u *Installer) Setup(ctx context.Context, dryRun bool, installerFile string
}
log.Infof("installer started with PID %d", cmd.Process.Pid)
if resultErr = cmd.Wait(); resultErr != nil {
log.Errorf("installer process finished with error: %v", resultErr)
return
if err := cmd.Wait(); err != nil {
if !isRebootPending(err) {
resultErr = err
log.Errorf("installer process finished with error: %v", err)
return
}
log.Warnf("installer completed but reported a pending reboot, some files will be replaced on the next restart")
}
return nil
@@ -117,16 +140,142 @@ func (u *Installer) startDaemon(daemonFolder string) error {
return nil
}
func (u *Installer) startUIAsUser(daemonFolder string) error {
func (u *Installer) startUI(daemonFolder string, sessionIDs []uint32) error {
uiPath := filepath.Join(daemonFolder, uiName)
log.Infof("starting netbird-ui: %s", uiPath)
// Get the active console session ID
sessionID := windows.WTSGetActiveConsoleSessionId()
if sessionID == 0xFFFFFFFF {
return fmt.Errorf("no active user session found")
if len(sessionIDs) == 0 {
sessionID := windows.WTSGetActiveConsoleSessionId()
if sessionID == 0xFFFFFFFF {
return fmt.Errorf("no active user session found")
}
sessionIDs = []uint32{sessionID}
}
var errs []error
for _, sessionID := range sessionIDs {
if err := startUIInSession(uiPath, sessionID); err != nil {
errs = append(errs, fmt.Errorf("session %d: %w", sessionID, err))
continue
}
log.Infof("netbird-ui started successfully in session %d", sessionID)
}
return errors.Join(errs...)
}
// isRebootPending reports whether the installer exit code means it succeeded but
// left work for the next restart. The reboot itself is suppressed, so this is not
// a failure.
func isRebootPending(err error) bool {
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
return false
}
switch exitErr.ExitCode() {
case msiRebootRequired, msiRebootInitiated:
return true
default:
return false
}
}
// killUI terminates any running netbird-ui process and returns the IDs of the
// interactive sessions the terminated processes belonged to. Setup starts the
// UI again in those sessions once the installer is done.
func killUI() []uint32 {
pids, err := processIDsByName(uiName)
if err != nil {
log.Warnf("failed to look up %s processes: %v", uiName, err)
return nil
}
sessions := make(map[uint32]struct{})
for _, pid := range pids {
var sessionID uint32
if err := windows.ProcessIdToSessionId(pid, &sessionID); err != nil {
log.Warnf("failed to look up session of %s (PID %d): %v", uiName, pid, err)
}
if err := terminateProcess(pid); err != nil {
log.Warnf("failed to terminate %s (PID %d): %v", uiName, pid, err)
continue
}
log.Infof("terminated %s (PID %d) in session %d", uiName, pid, sessionID)
if sessionID != 0 {
sessions[sessionID] = struct{}{}
}
}
sessionIDs := make([]uint32, 0, len(sessions))
for sessionID := range sessions {
sessionIDs = append(sessionIDs, sessionID)
}
return sessionIDs
}
func processIDsByName(name string) ([]uint32, error) {
snapshot, err := windows.CreateToolhelp32Snapshot(windows.TH32CS_SNAPPROCESS, 0)
if err != nil {
return nil, fmt.Errorf("create process snapshot: %w", err)
}
defer func() {
if err := windows.CloseHandle(snapshot); err != nil {
log.Warnf("failed to close process snapshot: %v", err)
}
}()
var entry windows.ProcessEntry32
entry.Size = uint32(unsafe.Sizeof(entry))
var pids []uint32
for err = windows.Process32First(snapshot, &entry); err == nil; err = windows.Process32Next(snapshot, &entry) {
if strings.EqualFold(windows.UTF16ToString(entry.ExeFile[:]), name) {
pids = append(pids, entry.ProcessID)
}
}
if !errors.Is(err, windows.ERROR_NO_MORE_FILES) {
return nil, fmt.Errorf("enumerate processes: %w", err)
}
return pids, nil
}
func terminateProcess(pid uint32) error {
handle, err := windows.OpenProcess(windows.PROCESS_TERMINATE|windows.SYNCHRONIZE, false, pid)
if err != nil {
// The process may have exited between enumeration and now.
if errors.Is(err, windows.ERROR_INVALID_PARAMETER) {
return nil
}
return fmt.Errorf("open process: %w", err)
}
defer func() {
if err := windows.CloseHandle(handle); err != nil {
log.Warnf("failed to close process handle: %v", err)
}
}()
if err := windows.TerminateProcess(handle, 0); err != nil {
return fmt.Errorf("terminate process: %w", err)
}
// Wait for the handle to signal so the image file is released before the
// installer tries to overwrite it. A timeout is reported through the returned
// event, not through err, which stays nil unless the wait itself failed.
event, err := windows.WaitForSingleObject(handle, uint32(processExitWait.Milliseconds()))
if err != nil {
return fmt.Errorf("wait for process exit: %w", err)
}
if event != windows.WAIT_OBJECT_0 {
return fmt.Errorf("wait for process exit: unexpected wait result %#x", event)
}
return nil
}
func startUIInSession(uiPath string, sessionID uint32) error {
// Get the user token for that session
var userToken windows.Token
err := windows.WTSQueryUserToken(sessionID, &userToken)
@@ -197,7 +346,6 @@ func (u *Installer) startUIAsUser(daemonFolder string) error {
log.Warnf("failed to close thread handle: %v", err)
}
log.Infof("netbird-ui started successfully in session %d", sessionID)
return nil
}

View File

@@ -0,0 +1,108 @@
package installer
import (
"errors"
"os/exec"
"slices"
"strconv"
"testing"
)
// exitErrorWithCode returns a real *exec.ExitError carrying the given exit code.
func exitErrorWithCode(t *testing.T, code int) error {
t.Helper()
err := exec.Command("cmd.exe", "/c", "exit "+strconv.Itoa(code)).Run()
if err == nil {
t.Fatalf("expected a non-zero exit for code %d", code)
}
return err
}
func TestIsRebootPending(t *testing.T) {
tests := []struct {
name string
code int
want bool
}{
{name: "reboot required", code: msiRebootRequired, want: true},
{name: "reboot initiated", code: msiRebootInitiated, want: true},
{name: "generic failure", code: 1603, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isRebootPending(exitErrorWithCode(t, tt.code)); got != tt.want {
t.Errorf("isRebootPending(exit %d) = %v, want %v", tt.code, got, tt.want)
}
})
}
}
// TestProcessIDsByNameAndTerminate spawns a long-running system process, finds it
// by name and terminates it, covering the path the updater uses to release the UI
// image file before the installer replaces it.
func TestProcessIDsByNameAndTerminate(t *testing.T) {
cmd := exec.Command("ping.exe", "-n", "60", "127.0.0.1")
if err := cmd.Start(); err != nil {
t.Fatalf("start ping: %v", err)
}
pid := uint32(cmd.Process.Pid)
killed := false
t.Cleanup(func() {
if !killed {
_ = cmd.Process.Kill()
}
_ = cmd.Wait()
})
// Name matching must be case-insensitive: the snapshot reports PING.EXE.
pids, err := processIDsByName("ping.exe")
if err != nil {
t.Fatalf("processIDsByName: %v", err)
}
if !slices.Contains(pids, pid) {
t.Fatalf("PID %d not among the ping.exe processes found: %v", pid, pids)
}
if err := terminateProcess(pid); err != nil {
t.Fatalf("terminateProcess: %v", err)
}
killed = true
// terminateProcess only returns once the handle has signalled, so the process
// is already gone and Wait must not block. It exits with the code passed to
// TerminateProcess, which is 0, so Wait reports no error.
if err := cmd.Wait(); err != nil {
t.Fatalf("wait for terminated ping: %v", err)
}
if !cmd.ProcessState.Exited() {
t.Error("process did not exit after terminateProcess")
}
remaining, err := processIDsByName("ping.exe")
if err != nil {
t.Fatalf("processIDsByName after terminate: %v", err)
}
if slices.Contains(remaining, pid) {
t.Errorf("PID %d still listed after terminateProcess", pid)
}
}
func TestProcessIDsByNameNoMatch(t *testing.T) {
pids, err := processIDsByName("netbird-nonexistent-process.exe")
if err != nil {
t.Fatalf("processIDsByName: %v", err)
}
if len(pids) != 0 {
t.Errorf("expected no matches, got %v", pids)
}
}
func TestIsRebootPendingNonExitError(t *testing.T) {
if isRebootPending(errors.New("start installer: file not found")) {
t.Error("a non-exit error must not be treated as a pending reboot")
}
}

View File

@@ -54,6 +54,12 @@ func (rh *ResultHandler) GetErrorResultReason() string {
return ""
}
// ClearStaleResult removes a result file left over from a previous installation
// attempt so result watchers cannot read an outdated outcome for the current attempt.
func (rh *ResultHandler) ClearStaleResult() error {
return rh.cleanup()
}
func (rh *ResultHandler) WriteSuccess() error {
result := Result{
Success: true,

View File

@@ -22,6 +22,7 @@ import (
"github.com/netbirdio/netbird/client/internal/listener"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/netevents"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/formatter"
"github.com/netbirdio/netbird/route"
@@ -36,11 +37,6 @@ const (
AnonymizeLevelStrict = nbAnonymize.LevelStrictString
)
// ConnectionListener export internal Listener for mobile
type ConnectionListener interface {
peer.Listener
}
// RouteListener export internal RouteListener for mobile
type NetworkChangeListener interface {
listener.NetworkChangeListener
@@ -87,6 +83,10 @@ type Client struct {
onHostDnsFn func([]string)
dnsManager dns.IosDnsManager
loginComplete bool
// netMgr outlives engine restarts: it mirrors the OS connectivity, not
// the engine lifecycle. Run injects its state and sweeper into each new
// ConnectClient.
netMgr *netevents.Manager
// preloadedConfig holds config loaded from JSON (used on tvOS where file writes are blocked)
preloadedConfig *profilemanager.Config
@@ -97,6 +97,7 @@ type Client struct {
// NewClient instantiate a new Client
func NewClient(cfgFile, stateFile, cacheDir, logFilePath, deviceName string, osVersion string, osName string, networkChangeListener NetworkChangeListener, dnsManager DnsManager) *Client {
recorder := peer.NewRecorder("")
return &Client{
cfgFile: cfgFile,
stateFile: stateFile,
@@ -105,10 +106,11 @@ func NewClient(cfgFile, stateFile, cacheDir, logFilePath, deviceName string, osV
deviceName: deviceName,
osName: osName,
osVersion: osVersion,
recorder: peer.NewRecorder(""),
recorder: recorder,
ctxCancelLock: &sync.Mutex{},
networkChangeListener: networkChangeListener,
dnsManager: dnsManager,
netMgr: netevents.NewManager(recorder),
}
}
@@ -184,7 +186,8 @@ func (c *Client) Run(fd int32, interfaceName string, envList *EnvList) error {
c.onHostDnsFn = func([]string) {}
cfg.WgIface = interfaceName
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder,
internal.WithNetEvents(c.netMgr))
c.setState(cfg, connectClient)
// Persist the latest sync response so DebugBundle can include the network
// map. On iOS this is backed by disk to keep it out of the constrained
@@ -193,6 +196,25 @@ func (c *Client) Run(fd int32, interfaceName string, envList *EnvList) error {
return connectClient.RunOniOS(fd, c.networkChangeListener, c.dnsManager, c.stateFile, c.cacheDir, c.logFilePath)
}
// SetNetworkAvailable feeds OS-reported network availability into the client
// (e.g. from NWPathMonitor). While unavailable, the internal reconnect loops
// suspend their attempts and the connection listener reports NoNetwork
// instead of Connecting; when availability returns, the loops resume
// immediately with a fresh backoff. Losing the last network also sweeps the
// registered connections, so the client does not keep reporting Connected
// over stale sockets with no network at all.
func (c *Client) SetNetworkAvailable(available bool) {
c.netMgr.SetNetworkAvailable(available)
}
// NotifyNetworkChange marks the management, signal and relay connections
// stale after the OS switched networks and schedules a sweep that cuts
// whatever has not redialed on the new network by then. The engine and the
// TUN device stay untouched.
func (c *Client) NotifyNetworkChange() {
c.netMgr.NotifyNetworkChange()
}
// Stop the internal client and free the resources
func (c *Client) Stop() {
c.ctxCancelLock.Lock()
@@ -331,7 +353,11 @@ func (c *Client) GetStatusDetails() *StatusDetails {
// SetConnectionListener set the network connection listener
func (c *Client) SetConnectionListener(listener ConnectionListener) {
c.recorder.SetConnectionListener(listener)
if listener == nil {
c.recorder.RemoveConnectionListener()
return
}
c.recorder.SetConnectionListener(connectionListenerAdapter{listener})
}
// RemoveConnectionListener remove connection listener

View File

@@ -0,0 +1,43 @@
//go:build ios
package NetBirdSDK
import (
"github.com/netbirdio/netbird/client/internal/peer"
)
// Client state values, re-exported as basic constants so gomobile emits them
// into the generated bindings. They mirror peer.ClientState*: append-only,
// never reorder.
const (
ClientStateDisconnected = int(peer.ClientStateDisconnected)
ClientStateConnected = int(peer.ClientStateConnected)
ClientStateConnecting = int(peer.ClientStateConnecting)
ClientStateDisconnecting = int(peer.ClientStateDisconnecting)
ClientStateNoNetwork = int(peer.ClientStateNoNetwork)
)
// ConnectionListener export internal Listener for mobile.
//
// It intentionally lacks OnStateChanged for now: adding a method to a gomobile
// interface breaks every Swift implementation, so the iOS app keeps building
// against the legacy per-state callbacks. A follow-up will extend it together
// with the app.
type ConnectionListener interface {
OnConnected()
OnDisconnected()
OnConnecting()
OnDisconnecting()
OnAddressChanged(string, string)
OnPeersListChanged(int)
}
// connectionListenerAdapter adapts the gomobile-facing ConnectionListener to
// peer.Listener.
type connectionListenerAdapter struct {
ConnectionListener
}
// OnStateChanged is dropped on iOS until the app adopts the state callback;
// the legacy per-state callbacks continue to fire.
func (a connectionListenerAdapter) OnStateChanged(peer.ClientState) {}

View File

@@ -323,7 +323,7 @@ func (a *Auth) login(urlOpener URLOpener, forceDeviceAuth bool, deviceName strin
const authInfoRequestTimeout = 30 * time.Second
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, forceDeviceAuth bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth, "")
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}

View File

@@ -0,0 +1,158 @@
// Package netevents owns the OS network event handling shared by the mobile
// bindings: availability changes park or wake the reconnection loops and drive
// the NoNetwork listener state, and both losing the last network and switching
// networks sweep the stale connections so their owners redial immediately.
package netevents
import (
"context"
"time"
"github.com/cenkalti/backoff/v4"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/netevents/netstate"
"github.com/netbirdio/netbird/client/netevents/sweep"
)
// Recorder receives the availability changes for listener state reporting.
type Recorder interface {
SetNetworkAvailable(available bool)
}
// Manager ties the network availability state, the connection sweeper and the
// status recorder together; it outlives engine restarts. A nil *Manager is
// the valid no-events value: the read methods report always-online and never
// sweep.
type Manager struct {
netState *netstate.State
sweeper *sweep.Sweeper
recorder Recorder
}
// NewManager creates a Manager reporting into recorder, starting online.
func NewManager(recorder Recorder) *Manager {
return &Manager{
netState: netstate.New(),
sweeper: sweep.New(),
recorder: recorder,
}
}
// SetNetworkAvailable records OS-reported network availability. While
// unavailable, the reconnection loops suspend their attempts and the
// connection listener reports NoNetwork instead of Connecting; when
// availability returns, the loops resume immediately with a fresh backoff.
// Losing the last network also sweeps the registered connections: nothing can
// redial while offline, so the stale sockets would otherwise stay silently
// "connected" until their own timeouts and the client would keep reporting
// Connected with no network at all.
func (m *Manager) SetNetworkAvailable(available bool) {
if !available && m.netState.IsOnline() {
m.sweeper.MarkNetworkChange()
}
m.netState.Set(available)
m.recorder.SetNetworkAvailable(available)
}
// NotifyNetworkChange marks the management, signal and relay connections
// stale after the OS switched networks and schedules a sweep that cuts
// whatever has not redialed on the new network by then. The engine and the
// TUN device stay untouched.
func (m *Manager) NotifyNetworkChange() {
m.sweeper.MarkNetworkChange()
log.Infof("network change: connections marked stale")
}
// IsOnline reports whether the OS reports at least one usable network.
func (m *Manager) IsOnline() bool {
if m == nil {
return true
}
return m.netState.IsOnline()
}
// Changed returns a channel closed on the next availability transition.
func (m *Manager) Changed() <-chan struct{} {
if m == nil {
return nil
}
return m.netState.Changed()
}
// Wait blocks while the network is offline; see netstate.State.Wait.
func (m *Manager) Wait(ctx context.Context) (bool, error) {
if m == nil {
return false, nil
}
return m.netState.Wait(ctx)
}
// WaitSettled waits until an online verdict holds for a full settleWindow, or
// while offline until the budget runs out. Returns false when ctx is
// cancelled. The settle window exists because a disconnect often precedes the
// OS offline flag by a few milliseconds, so a fresh online verdict cannot be
// trusted immediately. A nil Manager has no events to watch: it degrades to a
// fixed budget-long sleep.
func (m *Manager) WaitSettled(ctx context.Context, budget, settleWindow time.Duration) bool {
if m == nil {
select {
case <-time.After(budget):
return true
case <-ctx.Done():
return false
}
}
budgetTimer := time.NewTimer(budget)
defer budgetTimer.Stop()
settle := time.NewTimer(settleWindow)
defer settle.Stop()
for {
// Channel first, flag second: a flip in between still fires the channel.
changedCh := m.netState.Changed()
if m.netState.IsOnline() {
select {
case <-settle.C:
return true
case <-changedCh:
case <-ctx.Done():
return false
}
} else {
select {
case <-budgetTimer.C:
return true
case <-changedCh:
case <-ctx.Done():
return false
}
}
if !settle.Stop() {
select {
case <-settle.C:
default:
}
}
settle.Reset(settleWindow)
}
}
// StartDial registers an in-flight dial with the sweeper; see sweep.Sweeper.StartDial.
func (m *Manager) StartDial(ctx context.Context) *sweep.Dial {
if m == nil {
return (*sweep.Sweeper)(nil).StartDial(ctx)
}
return m.sweeper.StartDial(ctx)
}
// QuickRetryBackoff wraps bo for a quick retry after a network change; see
// sweep.Sweeper.QuickRetryBackoff.
func (m *Manager) QuickRetryBackoff(ctx context.Context, bo backoff.BackOff) backoff.BackOff {
if m == nil {
return bo
}
return m.sweeper.QuickRetryBackoff(ctx, bo, m.netState)
}

View File

@@ -0,0 +1,34 @@
package netevents
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
type recorderStub struct{}
func (recorderStub) SetNetworkAvailable(bool) {}
func TestWaitSettledAfterOutage(t *testing.T) {
const budget = 1500 * time.Millisecond
const settleWindow = 200 * time.Millisecond
const outage = 2 * settleWindow
m := NewManager(recorderStub{})
m.SetNetworkAvailable(false)
start := time.Now()
go func() {
time.Sleep(outage)
m.SetNetworkAvailable(true)
}()
ok := m.WaitSettled(context.Background(), budget, settleWindow)
elapsed := time.Since(start)
assert.True(t, ok, "recovered network must let the caller proceed")
assert.GreaterOrEqual(t, elapsed, outage+settleWindow, "an online verdict must hold a full settle window before it is trusted")
}

View File

@@ -0,0 +1,110 @@
// Package netstate tracks OS-reported network availability for the client.
//
// A State instance is owned by the platform integration (e.g. the Android or
// iOS bindings, fed from ConnectivityManager callbacks or NWPathMonitor) and
// is injected into the connection retry loops (management, signal, relay,
// peer guards and the top-level connect loop), which consult it to avoid
// burning CPU and battery on reconnect attempts while the device has no
// network at all (e.g. airplane mode), and to reset their backoff as soon as
// the network returns.
//
// Consumers hold a *State that may be nil — every non-mobile platform leaves
// it unset. The read methods are safe on a nil receiver: they report online
// and never block, so consumers behave as if this package did not exist.
package netstate
import (
"context"
"sync"
log "github.com/sirupsen/logrus"
)
// State holds the OS-reported network availability. The zero value is not
// usable; create instances with New.
type State struct {
mu sync.Mutex
online bool
changed chan struct{}
}
// New creates a State that starts online. Platforms without network tracking
// pass a nil *State instead: the read methods treat nil as always online and
// never block, so consumers need no nil guards.
func New() *State {
return &State{
online: true,
changed: make(chan struct{}),
}
}
// Set records whether the OS reports any usable network. Transitions wake up
// all Wait callers immediately. Unlike the read methods, Set is not nil-safe:
// it is only for the platform owner that created the State with New.
func (s *State) Set(online bool) {
s.mu.Lock()
defer s.mu.Unlock()
if s.online == online {
return
}
s.online = online
close(s.changed)
s.changed = make(chan struct{})
log.Infof("OS network availability changed: online=%t", online)
}
// IsOnline reports whether the OS reports at least one usable network. On a
// nil receiver — no State injected — it reports online.
func (s *State) IsOnline() bool {
if s == nil {
return true
}
s.mu.Lock()
defer s.mu.Unlock()
return s.online
}
// Changed returns a channel closed on the next availability transition, for
// callers that already own a select loop and cannot block in Wait. Re-read it
// after every fire: each transition installs a fresh channel. On a nil
// receiver — no State injected — it returns nil, which blocks forever in a
// select, so the caller simply never observes a transition.
func (s *State) Changed() <-chan struct{} {
if s == nil {
return nil
}
s.mu.Lock()
defer s.mu.Unlock()
return s.changed
}
// Wait blocks while the network is offline. It reports whether it had to
// wait, so callers can reset their backoff after an outage. It returns early
// with the context error when ctx is done. On a nil receiver — no State
// injected — it returns immediately.
func (s *State) Wait(ctx context.Context) (bool, error) {
if s == nil {
return false, nil
}
waited := false
for {
s.mu.Lock()
if s.online {
s.mu.Unlock()
return waited, nil
}
ch := s.changed
s.mu.Unlock()
if !waited {
waited = true
log.Debugf("network is offline, pausing connection attempts")
}
select {
case <-ctx.Done():
return waited, ctx.Err()
case <-ch:
}
}
}

View File

@@ -0,0 +1,170 @@
package netstate
import (
"context"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewStateIsOnline(t *testing.T) {
assert.True(t, New().IsOnline(), "a fresh State should start online")
}
func TestSetTogglesOnlineState(t *testing.T) {
s := New()
s.Set(false)
assert.False(t, s.IsOnline(), "state should be offline after Set(false)")
s.Set(true)
assert.True(t, s.IsOnline(), "state should be online after Set(true)")
}
func TestWaitReturnsImmediatelyWhenOnline(t *testing.T) {
s := New()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
waited, err := s.Wait(ctx)
require.NoError(t, err)
assert.False(t, waited, "Wait should not block when the network is online")
}
func TestWaitBlocksUntilOnline(t *testing.T) {
s := New()
s.Set(false)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
result := make(chan bool, 1)
go func() {
waited, err := s.Wait(ctx)
if err != nil {
result <- false
return
}
result <- waited
}()
// Verify Wait is actually blocking while offline
select {
case <-result:
t.Fatal("Wait should block while the network is offline")
case <-time.After(100 * time.Millisecond):
}
s.Set(true)
select {
case waited := <-result:
assert.True(t, waited, "Wait should report that it had to wait for the network")
case <-time.After(2 * time.Second):
t.Fatal("Wait should return promptly after the network becomes available")
}
}
func TestWaitReturnsOnContextCancel(t *testing.T) {
s := New()
s.Set(false)
ctx, cancel := context.WithCancel(context.Background())
result := make(chan error, 1)
go func() {
_, err := s.Wait(ctx)
result <- err
}()
cancel()
select {
case err := <-result:
assert.ErrorIs(t, err, context.Canceled)
case <-time.After(2 * time.Second):
t.Fatal("Wait should return promptly after context cancellation")
}
}
func TestWaitWakesAllWaiters(t *testing.T) {
s := New()
s.Set(false)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
const waiters = 10
var wg sync.WaitGroup
results := make(chan bool, waiters)
for i := 0; i < waiters; i++ {
wg.Add(1)
go func() {
defer wg.Done()
waited, err := s.Wait(ctx)
if err != nil {
results <- false
return
}
results <- waited
}()
}
time.Sleep(100 * time.Millisecond)
s.Set(true)
wg.Wait()
close(results)
count := 0
for waited := range results {
assert.True(t, waited, "every waiter should report that it waited")
count++
}
assert.Equal(t, waiters, count, "all waiters should have returned")
}
func TestNilStateReadsAreNoops(t *testing.T) {
var s *State
assert.True(t, s.IsOnline(), "nil State should report online")
waited, err := s.Wait(context.Background())
require.NoError(t, err)
assert.False(t, waited, "nil State's Wait should not block")
}
func TestConcurrentSetAndWait(t *testing.T) {
s := New()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var wg sync.WaitGroup
for i := 0; i < 4; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < 100; j++ {
s.Set(j%2 == 0)
s.IsOnline()
}
}()
}
for i := 0; i < 4; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < 100; j++ {
if _, err := s.Wait(ctx); err != nil {
return
}
}
}()
}
wg.Wait()
}

View File

@@ -0,0 +1,39 @@
package sweep
import (
"time"
"github.com/cenkalti/backoff/v4"
"github.com/netbirdio/netbird/client/netevents/netstate"
)
const quickRetryDelay = 200 * time.Millisecond
type quickRetryBackoff struct {
backoff.BackOff
sweeper *Sweeper
netState *netstate.State
used bool
}
func newQuickRetryBackoff(bo backoff.BackOff, sweeper *Sweeper, netState *netstate.State) *quickRetryBackoff {
return &quickRetryBackoff{
BackOff: bo,
sweeper: sweeper,
netState: netState,
}
}
func (b *quickRetryBackoff) NextBackOff() time.Duration {
if !b.used && b.sweeper.markedRecently() && b.netState.IsOnline() {
b.used = true
return quickRetryDelay
}
return b.BackOff.NextBackOff()
}
func (b *quickRetryBackoff) Reset() {
b.used = false
b.BackOff.Reset()
}

View File

@@ -0,0 +1,58 @@
package sweep
import (
"context"
"testing"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/stretchr/testify/assert"
)
func TestQuickRetryAfterRecentMark(t *testing.T) {
sweeper := New()
sweeper.MarkNetworkChange()
slow := backoff.NewConstantBackOff(5 * time.Second)
bo := sweeper.QuickRetryBackoff(context.Background(), slow, nil)
assert.Equal(t, quickRetryDelay, bo.NextBackOff(), "first retry after a mark must be quick")
assert.Equal(t, 5*time.Second, bo.NextBackOff(), "second retry must fall back to the wrapped backoff")
bo.Reset()
assert.Equal(t, quickRetryDelay, bo.NextBackOff(), "reset must re-arm the quick retry")
}
func TestQuickRetryWithoutMarkKeepsSpread(t *testing.T) {
sweeper := New()
slow := backoff.NewConstantBackOff(5 * time.Second)
bo := sweeper.QuickRetryBackoff(context.Background(), slow, nil)
assert.Equal(t, 5*time.Second, bo.NextBackOff(), "without a mark the wrapped backoff decides")
sweeper.mu.Lock()
sweeper.lastMark = time.Now().Add(-recentMarkWindow)
sweeper.mu.Unlock()
assert.Equal(t, 5*time.Second, bo.NextBackOff(), "a stale mark must not trigger the quick retry")
}
func TestQuickRetryNilSweeperPassthrough(t *testing.T) {
var sweeper *Sweeper
slow := backoff.NewConstantBackOff(5 * time.Second)
bo := sweeper.QuickRetryBackoff(context.Background(), slow, nil)
assert.Equal(t, backoff.BackOff(slow), bo, "nil sweeper must return the backoff unchanged")
}
func TestQuickRetryHonorsContext(t *testing.T) {
sweeper := New()
sweeper.MarkNetworkChange()
ctx, cancel := context.WithCancel(context.Background())
cancel()
bo := sweeper.QuickRetryBackoff(ctx, backoff.NewConstantBackOff(time.Millisecond), nil)
assert.Equal(t, backoff.Stop, bo.NextBackOff(), "cancelled context must stop the retry loop")
}

View File

@@ -0,0 +1,267 @@
// Package sweep cuts network-bound activity when the OS switches networks:
// a sweep closes the registered connections and aborts the in-flight dials, so
// their owners redial immediately instead of waiting for the old sockets to
// time out.
//
// A nil *Sweeper disables everything: all methods are nil-safe no-ops.
package sweep
import (
"context"
"errors"
"net"
"sync"
"time"
"github.com/cenkalti/backoff/v4"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/netevents/netstate"
)
// DefaultSweepDelay absorbs network flapping while the OS settles on a
// default network before the stale registrations are cut.
const DefaultSweepDelay = 500 * time.Millisecond
const recentMarkWindow = 3 * time.Second
// Config customizes a Sweeper. The zero value applies the defaults.
type Config struct {
// SweepDelay overrides DefaultSweepDelay when positive.
SweepDelay time.Duration
}
// ErrSwept reports that a dial finished after a network change swept its
// registration. The connection is already closed; the caller must treat it
// as a failed dial and redial on the new network.
var ErrSwept = errors.New("sweep: connection swept by network change")
// sweepID identifies one registration in a sweeper. Connections and dials
// draw from the same counter, so an id is unique across both registries.
type sweepID uint64
type connEntry struct {
conn net.Conn
gen uint64
}
// Dial tracks one dial from start to connection registration. It hands the
// dialed connection to the sweeper atomically, so a sweep can never fall
// between the dial finishing and the connection being registered.
type Dial struct {
sweeper *Sweeper
ctx context.Context
cancel context.CancelFunc
id sweepID
done bool // set by a sweep, WrapConn or Release; guarded by sweeper.mu
gen uint64
}
// Ctx returns the dial's context. A sweep cancels it, so a dial started on the
// old network aborts instead of waiting out its handshake timeout.
func (d *Dial) Ctx() context.Context {
return d.ctx
}
// Release ends the dial's registration and cancels its context. It is
// idempotent and safe after WrapConn, so callers can defer it.
func (d *Dial) Release() {
s := d.sweeper
if s == nil {
return
}
s.mu.Lock()
d.done = true
delete(s.dials, d.id)
s.mu.Unlock()
d.cancel()
}
// sweptConn deregisters itself from the sweeper when closed.
type sweptConn struct {
net.Conn
sweeper *Sweeper
id sweepID
}
func (c *sweptConn) Close() error {
c.sweeper.deregister(c.id)
return c.Conn.Close()
}
// Sweeper registers live connections and in-flight dials so the
// network-change sweep can cut everything registered before the change.
type Sweeper struct {
mu sync.Mutex
conns map[sweepID]connEntry
dials map[sweepID]*Dial
nextID sweepID
gen uint64
timer *time.Timer
sweepDelay time.Duration
lastMark time.Time
}
// New creates an empty sweeper with the default configuration.
func New() *Sweeper {
return NewWithConfig(Config{})
}
// NewWithConfig creates an empty sweeper customized by cfg.
func NewWithConfig(cfg Config) *Sweeper {
delay := cfg.SweepDelay
if delay <= 0 {
delay = DefaultSweepDelay
}
return &Sweeper{
conns: make(map[sweepID]connEntry),
dials: make(map[sweepID]*Dial),
sweepDelay: delay,
}
}
// StartDial registers an in-flight dial. Dial with Ctx, hand the result to
// WrapConn, and Release the dial when the attempt is over, typically deferred.
func (s *Sweeper) StartDial(ctx context.Context) *Dial {
if s == nil {
return &Dial{ctx: ctx}
}
ctx, cancel := context.WithCancel(ctx)
d := &Dial{sweeper: s, ctx: ctx, cancel: cancel}
s.mu.Lock()
d.id = s.nextID
s.nextID++
d.gen = s.gen
s.dials[d.id] = d
s.mu.Unlock()
return d
}
// WrapConn hands conn over to the sweeper. If a sweep ran since StartDial,
// the connection belongs to the old network: it is closed and ErrSwept is
// returned. Otherwise conn is registered against the next sweep and returned
// wrapped, deregistering itself on Close. Call it once, before Release.
func (d *Dial) WrapConn(conn net.Conn) (net.Conn, error) {
s := d.sweeper
if s == nil {
return conn, nil
}
s.mu.Lock()
if d.done {
s.mu.Unlock()
if err := conn.Close(); err != nil {
log.Debugf("swept dial close error: %v", err)
}
return nil, ErrSwept
}
d.done = true
delete(s.dials, d.id)
id := s.nextID
s.nextID++
// The conn inherits the dial's generation: the socket was bound to the
// network that was default when the dial started, not when it finished.
s.conns[id] = connEntry{conn: conn, gen: d.gen}
s.mu.Unlock()
return &sweptConn{Conn: conn, sweeper: s, id: id}, nil
}
// MarkNetworkChange records that the OS switched networks: everything
// registered so far becomes stale, and a sweep is (re)scheduled after the
// configured delay to cut whatever is still stale by then. Owners that
// redialed in the meantime hold fresh-generation registrations and survive,
// so no cancellation is needed around the sweep.
func (s *Sweeper) MarkNetworkChange() {
if s == nil {
return
}
s.mu.Lock()
s.gen++
cutoff := s.gen
s.lastMark = time.Now()
if s.timer != nil {
s.timer.Stop()
}
s.timer = time.AfterFunc(s.sweepDelay, func() {
n := s.sweep(cutoff)
log.Infof("network change sweep: closed %d stale connections", n)
})
s.mu.Unlock()
}
// QuickRetryBackoff wraps bo so that after each Reset the first retry comes
// quickly when the disconnect followed a recent network change and the
// network is online. Any other failure keeps bo's spread, so the clients of
// a restarted server still scatter their reconnects. A nil sweeper returns
// bo unchanged.
func (s *Sweeper) QuickRetryBackoff(ctx context.Context, bo backoff.BackOff, netState *netstate.State) backoff.BackOff {
if s == nil {
return bo
}
return backoff.WithContext(newQuickRetryBackoff(bo, s, netState), ctx)
}
func (s *Sweeper) markedRecently() bool {
if s == nil {
return false
}
s.mu.Lock()
defer s.mu.Unlock()
return !s.lastMark.IsZero() && time.Since(s.lastMark) < recentMarkWindow
}
// sweep closes the registered connections and aborts the in-flight dials
// older than cutoff, and returns how many connections it closed. A dial
// whose connection was not yet handed to WrapConn is marked, so the late
// WrapConn closes it instead of registering it.
func (s *Sweeper) sweep(cutoff uint64) int {
if s == nil {
return 0
}
s.mu.Lock()
var conns []net.Conn
for id, e := range s.conns {
if e.gen < cutoff {
delete(s.conns, id)
conns = append(conns, e.conn)
}
}
var dials []*Dial
for id, d := range s.dials {
if d.gen < cutoff {
d.done = true
delete(s.dials, id)
dials = append(dials, d)
}
}
s.mu.Unlock()
if len(dials) > 0 {
log.Debugf("aborting %d in-flight dials", len(dials))
for _, d := range dials {
d.cancel()
}
}
for _, conn := range conns {
log.Debugf("sweeping connection %s -> %s", conn.LocalAddr(), conn.RemoteAddr())
if err := conn.Close(); err != nil {
log.Debugf("swept connection close error: %v", err)
}
}
return len(conns)
}
func (s *Sweeper) deregister(id sweepID) {
s.mu.Lock()
delete(s.conns, id)
s.mu.Unlock()
}

View File

@@ -0,0 +1,241 @@
package sweep
import (
"context"
"math"
"net"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSweepClosesRegisteredConns(t *testing.T) {
sweeper := New()
c1 := wrap(t, sweeper, connPair(t))
c2 := wrap(t, sweeper, connPair(t))
assert.Equal(t, 2, sweeper.sweepAll(), "both live connections should be closed")
// The wrappers must report closed now.
buf := make([]byte, 1)
_, err := c1.Read(buf)
assert.Error(t, err, "first connection should be unusable after the sweep")
_, err = c2.Read(buf)
assert.Error(t, err, "second connection should be unusable after the sweep")
assert.Equal(t, 0, sweeper.sweepAll(), "second sweep should find nothing")
}
func TestCloseDeregisters(t *testing.T) {
sweeper := New()
conn := wrap(t, sweeper, connPair(t))
require.NoError(t, conn.Close())
assert.Equal(t, 0, sweeper.sweepAll(), "closed connection must leave the registry")
}
func TestCloseIsIdempotent(t *testing.T) {
sweeper := New()
conn := wrap(t, sweeper, connPair(t))
require.NoError(t, conn.Close())
assert.Error(t, conn.Close(), "double close surfaces the underlying error but must not panic")
}
func TestSweepOnlyAffectsOlderConns(t *testing.T) {
sweeper := New()
_ = wrap(t, sweeper, connPair(t))
assert.Equal(t, 1, sweeper.sweepAll())
// A connection dialed after the sweep must survive until the next one.
_ = wrap(t, sweeper, connPair(t))
assert.Equal(t, 1, sweeper.sweepAll(), "post-sweep connection belongs to the next sweep")
}
func TestSweepAbortsInFlightDials(t *testing.T) {
sweeper := New()
dial := sweeper.StartDial(context.Background())
defer dial.Release()
sweeper.sweepAll()
assert.ErrorIs(t, dial.Ctx().Err(), context.Canceled, "sweep must cancel the in-flight dial context")
}
func TestReleasedDialIsNotAborted(t *testing.T) {
sweeper := New()
// Simulate a dial that finished before the sweep.
released := sweeper.StartDial(context.Background())
released.Release()
// A dial still in flight during the sweep.
pending := sweeper.StartDial(context.Background())
defer pending.Release()
sweeper.sweepAll()
assert.ErrorIs(t, pending.Ctx().Err(), context.Canceled, "pending dial must be aborted")
}
func TestSweepBetweenDialAndHandoffClosesConn(t *testing.T) {
sweeper := New()
dial := sweeper.StartDial(context.Background())
defer dial.Release()
// The dial succeeds on the old network, then the sweep lands before the
// connection is handed over.
conn := connPair(t)
assert.Equal(t, 0, sweeper.sweepAll(), "the connection is not registered yet")
wrapped, err := dial.WrapConn(conn)
require.ErrorIs(t, err, ErrSwept)
require.Nil(t, wrapped)
buf := make([]byte, 1)
_, err = conn.Read(buf)
assert.Error(t, err, "the old-network connection must be closed, not leaked")
assert.Equal(t, 0, sweeper.sweepAll(), "nothing may leak into the next sweep")
}
func TestMarkNetworkChangeSparesFreshConns(t *testing.T) {
sweeper := NewWithConfig(Config{SweepDelay: 10 * time.Millisecond})
stale := wrap(t, sweeper, connPair(t))
sweeper.MarkNetworkChange()
_ = wrap(t, sweeper, connPair(t))
_ = stale.SetReadDeadline(time.Now().Add(time.Second))
buf := make([]byte, 1)
_, err := stale.Read(buf)
require.ErrorIs(t, err, net.ErrClosed, "stale connection must be closed by the delayed sweep")
assert.Equal(t, 1, sweeper.sweepAll(), "the fresh connection must survive the stale sweep")
}
func TestMarkNetworkChangeAbortsStaleDials(t *testing.T) {
sweeper := NewWithConfig(Config{SweepDelay: 10 * time.Millisecond})
stale := sweeper.StartDial(context.Background())
defer stale.Release()
sweeper.MarkNetworkChange()
fresh := sweeper.StartDial(context.Background())
defer fresh.Release()
assert.Eventually(t, func() bool {
return stale.Ctx().Err() != nil
}, time.Second, 5*time.Millisecond, "stale dial must be aborted by the delayed sweep")
assert.NoError(t, fresh.Ctx().Err(), "post-mark dial must not be aborted")
}
func TestConnInheritsDialGeneration(t *testing.T) {
sweeper := NewWithConfig(Config{SweepDelay: 20 * time.Millisecond})
// The dial starts before the network change but completes after it: the
// socket is bound to the old network, so the sweep must still cut it.
dial := sweeper.StartDial(context.Background())
defer dial.Release()
sweeper.MarkNetworkChange()
wrapped, err := dial.WrapConn(connPair(t))
require.NoError(t, err)
_ = wrapped.SetReadDeadline(time.Now().Add(time.Second))
buf := make([]byte, 1)
_, err = wrapped.Read(buf)
require.ErrorIs(t, err, net.ErrClosed, "old-generation connection must be swept")
}
func TestRepeatedMarksCoalesce(t *testing.T) {
sweeper := NewWithConfig(Config{SweepDelay: 20 * time.Millisecond})
first := wrap(t, sweeper, connPair(t))
sweeper.MarkNetworkChange()
second := wrap(t, sweeper, connPair(t))
sweeper.MarkNetworkChange()
_ = wrap(t, sweeper, connPair(t))
buf := make([]byte, 1)
for _, conn := range []net.Conn{first, second} {
_ = conn.SetReadDeadline(time.Now().Add(time.Second))
_, err := conn.Read(buf)
require.ErrorIs(t, err, net.ErrClosed, "every pre-mark connection must be swept by the rescheduled sweep")
}
assert.Equal(t, 1, sweeper.sweepAll(), "only the newest-generation connection may remain")
}
func TestNilSweeperIsNoop(t *testing.T) {
var sweeper *Sweeper
conn := connPair(t)
dial := sweeper.StartDial(context.Background())
defer dial.Release()
wrapped, err := dial.WrapConn(conn)
require.NoError(t, err)
assert.Equal(t, conn, wrapped, "nil sweeper must return the conn unchanged")
assert.NoError(t, dial.Ctx().Err(), "nil sweeper must not cancel the dial context")
assert.Equal(t, 0, sweeper.sweepAll(), "nil sweeper closes nothing")
}
// wrap registers conn with the sweeper through a completed dial.
func wrap(t *testing.T, sweeper *Sweeper, conn net.Conn) net.Conn {
t.Helper()
dial := sweeper.StartDial(context.Background())
defer dial.Release()
wrapped, err := dial.WrapConn(conn)
require.NoError(t, err)
return wrapped
}
// connPair dials a loopback TCP connection and keeps the accepted peer open
// until the test ends: a peer that closed early would make the connection
// unreadable on its own, so a read error after the sweep would prove nothing.
func connPair(t *testing.T) net.Conn {
t.Helper()
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() {
if err := l.Close(); err != nil {
t.Logf("listener close error: %v", err)
}
})
accepted := make(chan net.Conn, 1)
go func() {
conn, err := l.Accept()
if err != nil {
close(accepted)
return
}
accepted <- conn
}()
conn, err := net.Dial("tcp", l.Addr().String())
require.NoError(t, err)
peer, ok := <-accepted
require.True(t, ok, "listener must accept the dialed connection")
t.Cleanup(func() {
if err := peer.Close(); err != nil {
t.Logf("peer close error: %v", err)
}
})
return conn
}
// sweepAll cuts every registration regardless of generation.
func (s *Sweeper) sweepAll() int {
return s.sweep(math.MaxUint64)
}

View File

@@ -682,11 +682,6 @@ func (s *Server) Login(callerCtx context.Context, msg *proto.LoginRequest) (*pro
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.IsUnixDesktopClient, false, hint)
if err != nil {
state.Set(internal.StatusLoginFailed)
// enrolling a device is the one flow a setup key can replace. NotFound so the CLI
// stops its backoff loop and shows this instead of retrying a permanent condition.
if auth.IsSSOUnavailable(err) {
return nil, gstatus.Error(codes.NotFound, auth.WithSetupKeyAdvice(err).Error())
}
return nil, err
}

View File

@@ -313,21 +313,23 @@ func Dial(ctx context.Context, addr, user string, opts DialOptions) (*Client, er
// dialSSH establishes an SSH connection without JWT authentication
func dialSSH(ctx context.Context, network, addr string, config *ssh.ClientConfig) (*Client, error) {
if config.Timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, config.Timeout)
defer cancel()
}
dialer := &net.Dialer{}
conn, err := dialer.DialContext(ctx, network, addr)
if err != nil {
return nil, fmt.Errorf("dial %s: %w", addr, err)
}
clientConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
client, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
if closeErr := conn.Close(); closeErr != nil {
log.Debugf("connection close after handshake failure: %v", closeErr)
}
return nil, fmt.Errorf("ssh handshake: %w", err)
return nil, err
}
client := ssh.NewClient(clientConn, chans, reqs)
return &Client{
client: client,
}, nil

View File

@@ -12,6 +12,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
"golang.org/x/term"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
func (c *Client) setupTerminalMode(ctx context.Context, session *ssh.Session) error {
@@ -82,37 +84,7 @@ func (c *Client) setupTerminal(session *ssh.Session, fd int) error {
return fmt.Errorf("get terminal size: %w", err)
}
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
// Ctrl+C
ssh.VINTR: 3,
// Ctrl+\
ssh.VQUIT: 28,
// Backspace
ssh.VERASE: 127,
// Ctrl+U
ssh.VKILL: 21,
// Ctrl+D
ssh.VEOF: 4,
ssh.VEOL: 0,
ssh.VEOL2: 0,
// Ctrl+Q
ssh.VSTART: 17,
// Ctrl+S
ssh.VSTOP: 19,
// Ctrl+Z
ssh.VSUSP: 26,
// Ctrl+O
ssh.VDISCARD: 15,
// Ctrl+R
ssh.VREPRINT: 18,
// Ctrl+W
ssh.VWERASE: 23,
// Ctrl+V
ssh.VLNEXT: 22,
}
modes := nbssh.DefaultTerminalModes
terminal := os.Getenv("TERM")
if terminal == "" {

View File

@@ -10,6 +10,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
const (
@@ -80,28 +82,14 @@ func (c *Client) setupTerminalMode(_ context.Context, session *ssh.Session) erro
w, h := c.getWindowsConsoleSize()
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
ssh.VREPRINT: 18, // Ctrl+R
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
}
for mode, value := range nbssh.DefaultTerminalModes {
modes[mode] = value
}
if err := session.RequestPty("xterm-256color", h, w, modes); err != nil {

View File

@@ -35,6 +35,19 @@ type HostKeyVerifier interface {
VerifySSHHostKey(peerAddress string, key []byte) error
}
// PeerKeyLookup returns the stored SSH host key for a peer address.
type PeerKeyLookup func(peerAddress string) ([]byte, bool)
// VerifySSHHostKey implements HostKeyVerifier by looking up the stored key
// and comparing it against the presented key.
func (l PeerKeyLookup) VerifySSHHostKey(peerAddress string, presentedKey []byte) error {
storedKey, found := l(peerAddress)
if !found {
return ErrPeerNotFound
}
return VerifyHostKey(storedKey, presentedKey, peerAddress)
}
// DaemonHostKeyVerifier implements HostKeyVerifier using the NetBird daemon
type DaemonHostKeyVerifier struct {
client proto.DaemonServiceClient

45
client/ssh/handshake.go Normal file
View File

@@ -0,0 +1,45 @@
package ssh
import (
"context"
"fmt"
"io"
"net"
"time"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// Handshake runs the SSH client handshake on an already dialed conn and
// returns the resulting client. Dialing bounds only the TCP establishment;
// without a deadline on the socket a peer that accepts and then goes silent
// blocks the handshake forever, so the context deadline is applied to conn
// for the duration of the handshake. conn is closed on any error.
func Handshake(ctx context.Context, conn net.Conn, addr string, config *ssh.ClientConfig) (*ssh.Client, error) {
if deadline, ok := ctx.Deadline(); ok {
if err := conn.SetDeadline(deadline); err != nil {
closeHandshake(conn, "conn after deadline error")
return nil, fmt.Errorf("set handshake deadline: %w", err)
}
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
if err != nil {
closeHandshake(conn, "conn after handshake error")
return nil, fmt.Errorf("ssh handshake: %w", err)
}
if err := conn.SetDeadline(time.Time{}); err != nil {
closeHandshake(sshConn, "ssh conn after deadline clear error")
return nil, fmt.Errorf("clear handshake deadline: %w", err)
}
return ssh.NewClient(sshConn, chans, reqs), nil
}
func closeHandshake(c io.Closer, label string) {
if err := c.Close(); err != nil {
log.Debugf("ssh: close %s: %v", label, err)
}
}

View File

@@ -610,13 +610,10 @@ func (p *SSHProxy) dialBackend(ctx context.Context, addr, user, jwtToken string)
return nil, fmt.Errorf("connect to server: %w", err)
}
clientConn, chans, reqs, err := cryptossh.NewClientConn(conn, addr, config)
if err != nil {
_ = conn.Close()
return nil, fmt.Errorf("SSH handshake: %w", err)
}
handshakeCtx, cancel := context.WithTimeout(ctx, sshHandshakeTimeout)
defer cancel()
return cryptossh.NewClient(clientConn, chans, reqs), nil
return nbssh.Handshake(handshakeCtx, conn, addr, config)
}
func (p *SSHProxy) verifyHostKey(hostname string, remote net.Addr, key cryptossh.PublicKey) error {

84
client/ssh/session.go Normal file
View File

@@ -0,0 +1,84 @@
package ssh
import (
"fmt"
"io"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// DefaultTerminalModes are the PTY modes used by the interactive terminal clients.
var DefaultTerminalModes = ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VREPRINT: 18, // Ctrl+R
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
}
// PTYSession is an interactive shell session with a PTY and its I/O pipes.
type PTYSession struct {
Session *ssh.Session
Stdin io.WriteCloser
Stdout io.Reader
Stderr io.Reader
}
// StartPTYSession opens a session on the client, requests an xterm-256color PTY
// with the default terminal modes, wires up the I/O pipes and starts a shell.
// The session is closed on any error.
func StartPTYSession(client *ssh.Client, cols, rows int) (*PTYSession, error) {
session, err := client.NewSession()
if err != nil {
return nil, fmt.Errorf("new session: %w", err)
}
pty, err := setupPTYSession(session, cols, rows)
if err != nil {
if closeErr := session.Close(); closeErr != nil {
log.Debugf("ssh: session close after setup error: %v", closeErr)
}
return nil, err
}
return pty, nil
}
// setupPTYSession requests the PTY, opens the pipes and starts the shell on an
// already created session.
func setupPTYSession(session *ssh.Session, cols, rows int) (*PTYSession, error) {
if err := session.RequestPty("xterm-256color", rows, cols, DefaultTerminalModes); err != nil {
return nil, fmt.Errorf("request pty: %w", err)
}
stdin, err := session.StdinPipe()
if err != nil {
return nil, fmt.Errorf("stdin pipe: %w", err)
}
stdout, err := session.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("stdout pipe: %w", err)
}
stderr, err := session.StderrPipe()
if err != nil {
return nil, fmt.Errorf("stderr pipe: %w", err)
}
if err := session.Shell(); err != nil {
return nil, fmt.Errorf("start shell: %w", err)
}
return &PTYSession{Session: session, Stdin: stdin, Stdout: stdout, Stderr: stderr}, nil
}

View File

@@ -30,6 +30,11 @@ func GetInfo(ctx context.Context) *Info {
kernelVersion = osInfo[2]
}
addrs, err := networkAddresses()
if err != nil {
log.Warnf("discover network addresses: %s", err)
}
gio := &Info{
GoOS: runtime.GOOS,
Kernel: kernel,
@@ -41,6 +46,7 @@ func GetInfo(ctx context.Context) *Info {
NetbirdVersion: version.NetbirdVersion(),
UIVersion: extractUIVersion(ctx),
KernelVersion: kernelVersion,
NetworkAddresses: addrs,
SystemSerialNumber: serial(),
SystemProductName: productModel(),
SystemManufacturer: productManufacturer(),

View File

@@ -1,4 +1,4 @@
//go:build !ios
//go:build !ios && !android
package system

View File

@@ -0,0 +1,89 @@
package system
import (
"net/netip"
"strings"
)
var iFaceDiscover IFaceDiscover
type IFaceDiscover interface {
IFaces() (string, error)
}
// SetIFaceDiscover configures the Android interface discovery provider.
func SetIFaceDiscover(discover IFaceDiscover) {
iFaceDiscover = discover
}
func networkAddresses() ([]NetworkAddress, error) {
if iFaceDiscover == nil {
return nil, nil
}
ifaces, err := iFaceDiscover.IFaces()
if err != nil {
return nil, err
}
var netAddresses []NetworkAddress
for _, line := range strings.Split(ifaces, "\n") {
addresses, ok := interfaceAddresses(line)
if !ok {
continue
}
for _, address := range addresses {
netAddr, ok := toNetworkAddress(address)
if !ok {
continue
}
if isDuplicated(netAddresses, netAddr) {
continue
}
netAddresses = append(netAddresses, netAddr)
}
}
return netAddresses, nil
}
func interfaceAddresses(line string) ([]string, bool) {
parts := strings.Split(line, "|")
if len(parts) != 2 {
return nil, false
}
flags := strings.Fields(parts[0])
if len(flags) != 8 {
return nil, false
}
up, loopback := flags[3], flags[5]
if up != "true" || loopback == "true" {
return nil, false
}
return strings.Fields(parts[1]), true
}
func toNetworkAddress(address string) (NetworkAddress, bool) {
prefix, err := netip.ParsePrefix(address)
if err != nil {
return NetworkAddress{}, false
}
if prefix.Addr().Is4In6() {
if prefix.Bits() < 96 {
return NetworkAddress{}, false
}
prefix = netip.PrefixFrom(prefix.Addr().Unmap(), prefix.Bits()-96)
}
ip := prefix.Addr()
if ip.IsLoopback() || ip.IsLinkLocalUnicast() || ip.IsMulticast() {
return NetworkAddress{}, false
}
return NetworkAddress{NetIP: prefix}, true
}
func isDuplicated(addresses []NetworkAddress, addr NetworkAddress) bool {
for _, duplicated := range addresses {
if duplicated.NetIP == addr.NetIP {
return true
}
}
return false
}

View File

@@ -1,4 +1,4 @@
//go:build !ios
//go:build !ios && !android
package system

View File

@@ -6,9 +6,11 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -60,9 +62,19 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
if p.Hint != "" {
h := p.Hint
req.Hint = &h
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -15,7 +15,8 @@
"lint": "eslint \"src/**/*.{ts,tsx}\"",
"lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix",
"check": "pnpm lint && pnpm typecheck && pnpm format:check",
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck"
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck",
"i18n:check": "node ../i18n/check-translations.mjs"
},
"dependencies": {
"@radix-ui/react-dialog": "^1.1.15",

View File

@@ -0,0 +1,104 @@
#!/usr/bin/env node
// Validates that every shipped translation bundle carries exactly the same set
// of keys as the English source of truth. English (en) defines the keys; every
// other locale declared in _index.json must match it 1:1:
//
// - no missing keys — a missing key silently falls back to English at runtime
// (see i18n bundle fallback), so the gap never surfaces to users or CI
// without this check;
// - no orphaned keys — keys left behind after an English key is renamed or
// removed are dead weight and a sign the locale is drifting.
//
// Pure Node, no dependencies, so it runs without installing the frontend
// toolchain.
//
// Local: node client/ui/i18n/check-translations.mjs (or: pnpm i18n:check)
// CI: .github/workflows/ui-translations.yml
import { readdirSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const SOURCE = "en";
const localesDir = join(dirname(fileURLToPath(import.meta.url)), "locales");
const isCI = Boolean(process.env.GITHUB_ACTIONS);
function readJSON(path) {
return JSON.parse(readFileSync(path, "utf8"));
}
function keysOf(langCode) {
return Object.keys(readJSON(join(localesDir, langCode, "common.json")));
}
// Emit a GitHub Actions annotation so failures render inline on the PR diff.
function annotate(file, message) {
if (isCI) console.log(`::error file=${file}::${message}`);
}
const index = readJSON(join(localesDir, "_index.json"));
const declared = index.languages.map((l) => l.code);
if (!declared.includes(SOURCE)) {
console.error(`FATAL: source language "${SOURCE}" is not declared in _index.json`);
process.exit(1);
}
const sourceKeys = keysOf(SOURCE);
const sourceSet = new Set(sourceKeys);
console.log(`Source of truth: ${SOURCE}/common.json — ${sourceKeys.length} keys\n`);
let failed = false;
for (const code of declared) {
if (code === SOURCE) continue;
const file = `client/ui/i18n/locales/${code}/common.json`;
let keys;
try {
keys = keysOf(code);
} catch (e) {
failed = true;
const msg = `bundle is declared in _index.json but common.json is missing or invalid (${e.message})`;
console.error(`${code}: ${msg}`);
annotate("client/ui/i18n/locales/_index.json", `${code}: ${msg}`);
continue;
}
const set = new Set(keys);
const missing = sourceKeys.filter((k) => !set.has(k));
const extra = keys.filter((k) => !sourceSet.has(k));
if (missing.length === 0 && extra.length === 0) {
console.log(`${code}: ${keys.length} keys`);
continue;
}
failed = true;
console.error(`${code}: ${keys.length} keys (expected ${sourceKeys.length})`);
if (missing.length) {
console.error(` missing ${missing.length}: ${missing.join(", ")}`);
annotate(file, `Missing ${missing.length} key(s) present in ${SOURCE}: ${missing.join(", ")}`);
}
if (extra.length) {
console.error(` extra ${extra.length}: ${extra.join(", ")}`);
annotate(file, `Has ${extra.length} key(s) not present in ${SOURCE}: ${extra.join(", ")}`);
}
}
// Locale directories present on disk but not declared in _index.json are never
// loaded by the app — surface them so dead translation files don't rot silently.
const onDisk = readdirSync(localesDir, { withFileTypes: true })
.filter((e) => e.isDirectory())
.map((e) => e.name);
const undeclared = onDisk.filter((d) => !declared.includes(d));
if (undeclared.length) {
console.warn(`\n⚠ locale directories not declared in _index.json (not shipped): ${undeclared.join(", ")}`);
}
console.log();
if (failed) {
console.error("Translation check FAILED — every locale must match the English key set.");
process.exit(1);
}
console.log("Translation check passed — all locales match the English key set.");

View File

@@ -1312,6 +1312,9 @@
"daemon.outdated.description": {
"message": "このアプリを使用するには NetBird サービスを更新してください。"
},
"daemon.outdated.download": {
"message": "最新版をダウンロード"
},
"error.jwt_clock_skew": {
"message": "サインインに失敗しました: このデバイスの時計がサーバーと同期していません。システムの時計を同期してからもう一度お試しください。"
},

View File

@@ -123,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
}
resp, err := cli.Login(ctx, req)
@@ -228,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -261,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Logout below).
// root and the per-profile state file is user-owned (see Profiles.List).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -80,13 +80,12 @@ func (c *Client) Connect(host string, port int, username, jwtToken string, ipVer
return fmt.Errorf("dial %s: %w", addr, err)
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
sshClient, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
closeWithLog(conn, "connection after handshake error")
return fmt.Errorf("SSH handshake: %w", err)
return err
}
c.sshClient = ssh.NewClient(sshConn, chans, reqs)
c.sshClient = sshClient
logrus.Infof("SSH: Connected to %s", addr)
return nil
@@ -119,57 +118,26 @@ func (c *Client) getAuthMethods(jwtToken string) ([]ssh.AuthMethod, error) {
return []ssh.AuthMethod{ssh.PublicKeys(signer)}, nil
}
// StartSession starts an SSH session with PTY
// StartSession starts an SSH session with PTY. It holds the client lock for
// the whole startup so Close cannot tear the client down mid-setup and the
// new session cannot be installed into an already closed client.
func (c *Client) StartSession(cols, rows int) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.sshClient == nil {
return fmt.Errorf("SSH client not connected")
}
session, err := c.sshClient.NewSession()
pty, err := nbssh.StartPTYSession(c.sshClient, cols, rows)
if err != nil {
return fmt.Errorf("create session: %w", err)
return err
}
c.mu.Lock()
defer c.mu.Unlock()
c.session = session
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3,
ssh.VQUIT: 28,
ssh.VERASE: 127,
}
if err := session.RequestPty("xterm-256color", rows, cols, modes); err != nil {
closeWithLog(session, "session after PTY error")
return fmt.Errorf("PTY request: %w", err)
}
c.stdin, err = session.StdinPipe()
if err != nil {
closeWithLog(session, "session after stdin error")
return fmt.Errorf("get stdin: %w", err)
}
c.stdout, err = session.StdoutPipe()
if err != nil {
closeWithLog(session, "session after stdout error")
return fmt.Errorf("get stdout: %w", err)
}
c.stderr, err = session.StderrPipe()
if err != nil {
closeWithLog(session, "session after stderr error")
return fmt.Errorf("get stderr: %w", err)
}
if err := session.Shell(); err != nil {
closeWithLog(session, "session after shell error")
return fmt.Errorf("start shell: %w", err)
}
c.session = pty.Session
c.stdin = pty.Stdin
c.stdout = pty.Stdout
c.stderr = pty.Stderr
logrus.Info("SSH: Session started with PTY")
return nil

View File

@@ -115,7 +115,7 @@ sequenceDiagram
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)

View File

@@ -15,6 +15,10 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
| ---- | ---- |
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
@@ -48,6 +52,8 @@ flowchart TD
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
K2 --> P
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
@@ -60,6 +66,84 @@ flowchart TD
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### LLM pricing (management is the sole authority)
**The proxy carries no price list.** Management synthesizes the entire pricing
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
table and there is nothing to reload on the proxy side.
```mermaid
flowchart TD
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
B --> C{AgentNetwork.PricingDefaultsFile}
C -- absent --> D[compiled-in table serves]
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
E --> F[mergedTable atomic.Pointer]
D --> G[DefaultTable]
F --> G
G --> H[buildCostMeterConfigJSON — pricing.defaults]
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
K --> L[pricing.providers keyed by provider record ID]
H --> M[cost_meter ConfigJSON]
L --> M
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
```
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
id → rates. The **full** default table ships to every account: it is small
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
models, so they claim every model) priced.
- `pricing.providers` — provider **record** id → normalized model id → rates,
matched against the `llm.resolved_provider_id` the router stamps. Entries are
**fully materialized here**, at synth time: `materializeEntry` starts from the
default entry for that model so cache rates the operator didn't state are
inherited, overlays operator `input`/`output` verbatim (**including an explicit
0**, which prices a self-hosted or internal endpoint as free rather than
silently reverting to list price), and overlays cache-rate **pointers only when
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
discount, bill this bucket at the input rate". The proxy therefore does two map
lookups and no merging.
Same orphan rule as the router: a provider no enabled policy authorises is
unreachable, so its prices aren't shipped. Model ids are normalized with the
**same** functions the request parser uses (`NormalizeBedrockModel` /
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
first-occurrence-wins, matching the routing dedup order.
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
replace default rates without a rebuild. Schema is `surface → model → rates`
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
- An **explicitly configured** path is *required to load*: a typo or malformed
file fails startup, because the operator believes those rates are live. The
conventional probe is optional — an absent file just serves compiled-in
defaults, and the path stays watched in case it appears later.
- File entries **replace** the compiled-in entry for the same (surface, model)
**whole** — they are not field-merged, so an entry must repeat the cache rates
it wants to keep. Everything the file doesn't mention keeps built-in rates.
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
finite and non-negative — the same constraints the HTTP API enforces on
operator per-provider prices.
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
a parse error keeps the previous table, a deleted file reverts to compiled-in
defaults. A mid-edit save can never take pricing down.
The live table feeds **both** consumers, which is what keeps them consistent: the
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
golden-tested, so operators start from a file matching the built-in rates exactly.
### Budget rule resolution (min-wins, group+user bound)
```mermaid
@@ -124,7 +208,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | |
| on_response | 6 | `cost_meter` | `{}` | |
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
@@ -139,6 +223,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
## Things to scrutinize
@@ -176,10 +266,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
@@ -188,6 +280,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
## Test coverage
@@ -198,6 +291,9 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |

View File

@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing catalog this chain delegates to.
adapters + pricing table and cost formula this chain delegates to.
---
@@ -34,7 +34,7 @@ rewrites.
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:2640](../../../proxy/internal/middleware/builtin/all_test.go)
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
@@ -168,12 +168,46 @@ token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`).
**Management owns pricing.** The proxy carries no embedded price list: the whole
table arrives in this middleware's `ConfigJSON` as
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
plus the operator's stored per-provider prices
([factory.go:1334](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
construction, so a non-finite or negative rate fails the chain build. A price
change is an ordinary mapping push — the chain rebuild yields a fresh instance
over a fresh immutable table, so there is no data dir, no pricing file, no
reload goroutine, and nothing to invalidate.
**Two-tier lookup**
([middleware.go:165183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
1. **Per-provider-record** — the operator's stored price for the route that
actually served the request, keyed by the `llm.resolved_provider_id` that
`llm_router` stamped on the allow path, then by normalized model id. Entries
arrive fully materialized (management folds default cache rates in at synth
time), so there is no merging here. Absent metadata — no router in the chain
— skips this tier.
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
providers, which enumerate no models and therefore get no per-record entry.
**Backward compatibility:** a config with no `pricing` block means management
predates config-delivered pricing. The factory logs one warning at build time
and the instance records `cost.skipped=unknown_model` ($0) for every request
rather than falling back to a stale built-in price list
([factory.go:5560](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
(sibling doc) and is selected by the **surface**, not by which tier the entry
came from — `cost_meter` stays provider-agnostic, and a per-record override on
an Anthropic route still bills its cache buckets additively.
### llm_limit_record
@@ -246,12 +280,14 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time.
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
block carrying a negative or non-finite rate fails the build too, rather than
mispricing live traffic.
## Invariants
@@ -320,10 +356,11 @@ non-object `metadata` field
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
middleware config and never mutated, so the lookup path needs no lock or atomic
swap — a price change replaces the whole instance. Every middleware is
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
@@ -349,13 +386,13 @@ counter accuracy.
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing loader.
+ SSE framer + pricing table and cost formula.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing loader.
— the 8 middlewares that consume this package's parsers + pricing table.
---
@@ -24,8 +24,9 @@ proxy-framework dependencies:
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/`embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `pricing/`immutable pricing table + the per-surface cost formula. The
rates themselves come from management inside `cost_meter`'s middleware
config; this package holds no price list and reads no files.
- `fixtures/` — captured request/response/stream bodies the tests replay.
The package carries zero proxy-framework dependencies so the same parsers can
@@ -47,12 +48,9 @@ be reused later by a WASM adapter
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream |
## Request body → parser dispatch
@@ -188,9 +186,11 @@ response leg, covering both Bedrock body shapes:
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
management ships, keyed by the **normalised** model id (region prefix + version
suffix stripped by the request parser; management normalises its keys the same
way at synth time so the two compare equal). `ParseResponse` returns
`ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
@@ -205,11 +205,34 @@ response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing catalog
### Pricing table
`Table.Cost`
([pricing.go:129174](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
**Management is the sole pricing authority.** The proxy carries no embedded
price list and reads no pricing file: the whole table arrives inside
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
is just another push — the chain rebuild constructs a fresh `Table`, so there
is nothing to reload
([pricing.go:17](../../../proxy/internal/llm/pricing/pricing.go)). The
management side of the contract (catalog defaults, the operator's stored
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
management-side module guide; `cost_meter`'s wire shape is in
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
`EntryJSON`
([pricing.go:3645](../../../proxy/internal/llm/pricing/pricing.go)) is the
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
direct struct conversion rather than field-by-field copying (a new rate can't
be silently dropped in transit).
`EntryCosts`
([pricing.go:183234](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module. The
**surface** (the `llm.provider` value the request parser stamped) selects the
formula, never the tier the entry came from: a per-provider-record override on
an Anthropic route still bills its cache buckets additively.
| Provider | Formula |
|---|---|
@@ -218,7 +241,7 @@ is the cost formula — most security-relevant math in this module:
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
([pricing.go:214229](../../../proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
@@ -226,15 +249,12 @@ to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Loader`
([pricing.go:212268](../../../proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
`Costs`
([pricing.go:143163](../../../proxy/internal/llm/pricing/pricing.go)) is the
per-request split. The four per-bucket fields are the base; `TotalUSD` and
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
the breakdown. `InputUSD` is always the non-cached input bucket on both
provider shapes, so input and cached-input never double-count.
## Public contracts
@@ -264,29 +284,38 @@ Order matters: `DetectFromURL` ties resolve by registration order.
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` lookup**
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
**`Pricing` construction + lookup**
([pricing.go:60130](../../../proxy/internal/llm/pricing/pricing.go)):
```go
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
func (t *Table) Lookup(provider, model string) (Entry, bool)
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
```
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130132](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
reject any non-finite or negative rate, so a corrupt config fails the chain
build rather than mispricing silently. Nil input yields an empty,
never-matching table.
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
([pricing.go:9699](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means the surface or model is absent from the table management sent;
the caller emits `cost.skipped=unknown_model`.
## Invariants
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:1416](../../../proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
1. **The pricing package is pure and platform-independent.** No file I/O, no
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
there is nothing platform-specific left to port. Anything reintroducing a
read-from-disk path here re-splits pricing authority between management and
the proxy, which is exactly what this design removed.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
@@ -298,38 +327,45 @@ emits `cost.skipped=unknown_model`.
usage rather than aborting
([streaming.go:6873, 144150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:2930](../../../proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:4249](../../../proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
3. **Management is the only source of rates.** `Table` has no constructor that
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
management sent. A missing or empty `pricing` block therefore means *no
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
never a stale built-in fallback that would silently bill list price.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370394](../../../proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
4. **Tables are immutable once built.** `Table.entries` is written only in
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
map is likewise build-time-only
([pricing.go:4752](../../../proxy/internal/llm/pricing/pricing.go)). This
is what makes the no-reload design safe: a price change arrives as a mapping
push that builds a new middleware instance over a new table, so concurrent
readers can't observe a half-updated price list and no atomic swap or lock
is needed on the hot path.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:2557](../../../proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
5. **Rate validation happens at chain-build time, not per request.**
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
([pricing.go:6083](../../../proxy/internal/llm/pricing/pricing.go)), naming
the offending surface/model/field in the error. Management enforces the same
constraints at its API boundary and in its YAML parser, so this is
defense-in-depth — but it means a corrupt push fails loudly at build instead
of producing negative costs on live traffic. Test:
`TestNewTable_ValidatesRates`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397398](../../../proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
`pricing.Entry` together.** `NewEntries` converts by direct struct
conversion `Entry(e)`
([pricing.go:7678](../../../proxy/internal/llm/pricing/pricing.go)), which
only compiles while the two structs stay field-identical — so the proxy half
is compiler-enforced. The management half is not: a rate added there but not
here unmarshals into nothing and prices that bucket at `InputPer1K`.
## Things to scrutinise
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147149](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
**Correctness.** Verify the OpenAI cached-prompt clamp at
[pricing.go:203206](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. Negative token counts are clamped to zero up
front ([pricing.go:186197](../../../proxy/internal/llm/pricing/pricing.go)) so
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
@@ -338,22 +374,27 @@ noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
oversize surface is gone entirely — the config channel (an authenticated
mapping push from management) is now the only way rates enter the proxy, and
`NewEntries` is the validation boundary on it. A new rate added to management's
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
path (see invariant 6).
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Concurrency.** Nothing in this package is shared mutable state: tables are
built once and never written again, so `cost_meter`'s hot path is lock-free by
construction rather than by atomic swap. Per-call `Scanner` instances mean no
shared state across concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
`ReadString('\n')` per line. No background goroutines and no per-request
allocation of pricing state.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
**Observability.** A config carrying no `pricing` block logs one warning at
chain-build time (`cost_meter` factory) and then records
`cost.skipped=unknown_model` per request, so an old-management deployment is
visible in both logs and the access log rather than quietly reporting $0.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
@@ -365,7 +406,7 @@ right `cost.skipped` reason.
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
@@ -373,14 +414,15 @@ right `cost.skipped` reason.
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
No pricing fixture: the table is config-delivered, so pricing tests construct
it in-process from a wire-shape map.
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewLoader`.
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -1,7 +1,7 @@
# proxy/runtime — translate + serve + log
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
## Module boundary
@@ -114,8 +114,7 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
## Public contracts touched
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).

View File

@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic``llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
parser, priced under the **`anthropic`** surface of the pricing table
management ships (the parser emits the standard Anthropic provider label, so
Vertex Claude reuses first-party Anthropic prices).
- `openai``llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
@@ -104,8 +104,9 @@ is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
> those regions read low. Operators who need exact regional billing set the
> affected models' prices on the provider record, or replace the default entries
> via management's `AgentNetwork.PricingDefaultsFile`.
## AWS Bedrock (`bedrock_api`)
@@ -211,15 +212,19 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
`model_prices_and_context_window.json`. The **catalog is the source of default
prices**: management's `pricing.DefaultTable` folds every catalog provider's
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
is structural rather than maintained in a parallel file
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
providers contribute the same (surface, model) at different rates. Bedrock
entries are keyed by the **normalised** id the request parser emits (region
prefix + version suffix stripped) — management applies the same normalisation to
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
surface.
## Things to scrutinise
@@ -232,16 +237,17 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
pricing key — a model that normalises to something absent from the shipped
pricing table meters at `cost.skipped=unknown_model` rather than failing the
request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
~10% premium not modelled by base pricing — flagged in the catalog comment.
Operators needing exact regional billing set per-provider prices on the model
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
## Cross-references

View File

@@ -6,7 +6,7 @@
"name": "NetBird GmbH",
"email": "hello@netbird.io",
"phone": "",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open-source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open-source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"webpageUrl": {
"url": "https://github.com/netbirdio"
}
@@ -15,7 +15,7 @@
{
"guid": "netbird",
"name": "NetBird",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open-source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"webpageUrl": {
"url": "https://github.com/netbirdio/netbird"
},
@@ -59,7 +59,7 @@
"guid": "support-yearly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Yearly",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "yearly",
@@ -72,7 +72,7 @@
"guid": "support-one-time-year",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Year",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "one-time",
@@ -85,7 +85,7 @@
"guid": "support-one-time-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Monthly",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",
@@ -98,7 +98,7 @@
"guid": "support-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Month",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",

14
go.mod
View File

@@ -19,7 +19,7 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/vishvananda/netlink v1.3.1
golang.org/x/crypto v0.54.0
golang.org/x/crypto v0.55.0
golang.org/x/sys v0.47.0
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
@@ -57,6 +57,7 @@ require (
github.com/fsnotify/fsnotify v1.9.0
github.com/gliderlabs/ssh v0.3.8
github.com/go-jose/go-jose/v4 v4.1.4
github.com/go-ole/go-ole v1.3.0
github.com/gobwas/ws v1.4.0
github.com/goccy/go-yaml v1.18.0
github.com/godbus/dbus/v5 v5.2.2
@@ -126,9 +127,9 @@ require (
go.uber.org/zap v1.27.0
goauthentik.io/api/v3 v3.2023051.3
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab
golang.org/x/mod v0.37.0
golang.org/x/net v0.56.0
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733
golang.org/x/mod v0.39.0
golang.org/x/net v0.58.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.22.0
golang.org/x/term v0.45.0
@@ -199,7 +200,6 @@ require (
github.com/go-ldap/ldap/v3 v3.4.13 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.2 // indirect
github.com/go-openapi/jsonpointer v0.21.1 // indirect
@@ -313,8 +313,8 @@ require (
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/text v0.40.0 // indirect
golang.org/x/tools v0.47.0 // indirect
golang.org/x/text v0.41.0 // indirect
golang.org/x/tools v0.49.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect

24
go.sum
View File

@@ -728,13 +728,13 @@ golang.org/x/crypto v0.18.0/go.mod h1:R0j02AL6hcrfOiy9T4ZYp/rcWeMxM3L6QYxlOuEG1m
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab h1:Iqyc+2zr7aGyLuEadIm0KRJP0Wwt+fhlXLa51Fxf1+Q=
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab/go.mod h1:Eq3Nh/5pFSWug2ohiudJ1iyU59SO78QFuh4qTTN++I0=
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733 h1:XKMObIaAElmkdO+4SQh1iCfzwciZHJi1OblnX9BED9k=
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733/go.mod h1:jMwjxoDSx9jqhNaZqPnr6nnKzb7cs+Dy1Czk7wdX+R8=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
@@ -744,8 +744,8 @@ golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
@@ -764,8 +764,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.20.0/go.mod h1:z8BVo6PvndSri0LbOE3hAn0apkU+1YvI6E70E9jsnvY=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/oauth2 v0.8.0/go.mod h1:yr7u4HXZRm1R1kBWqr/xKNqewf0plRYoB7sla+BCIXE=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
@@ -843,8 +843,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -858,8 +858,8 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

Some files were not shown because too many files have changed in this diff Show More