Compare commits

..

22 Commits

Author SHA1 Message Date
Zoltán Papp
14aab0fc6e [client] Store per-profile settings in the profile manager
Introduce a namespaced preference store owned by the profile manager,
persisted next to the profile config as <id>.prefs.json and deleted with
the profile. Sections are opaque JSON, so the profile manager stays free
of any feature schema, and writes reuse the state file's atomic path.

Migrate the Android SSH known-hosts store and session list onto it. Both
previously lived outside the profile lifecycle: known hosts in a
per-profile file under filesDir, the session list in Java
SharedPreferences, each needing its own sweep against the live profile
list to avoid outliving the profile they belonged to. A profile ID that
got reused would have inherited the trusted keys of a deleted profile.
Both now share the "ssh" and "ssh-sessions" namespaces of the profile's
preferences, so deleting a profile takes them along and the Java-side
pruning is gone.

The known-hosts entries keep the OpenSSH line format, only the container
changed, and host key verification keeps rejecting a changed key
outright. SetKnownHostsPath becomes SetKnownHostsStore, taking the
config dir and profile ID instead of a file path.

Existing known-hosts files are not migrated: hosts trusted before this
change prompt for confirmation once more, which errs towards safety.
2026-08-16 19:13:50 +02:00
Zoltan Papp
f06b8c7624 Merge remote-tracking branch 'origin/main' into feature/android-client-ssh
# Conflicts:
#	client/android/login.go
2026-08-14 23:25:50 +02:00
Zoltan Papp
ee3aeadf8f [client] Pass the login hint to GetOAuthFlow at construction
GetOAuthFlow was the only flow factory without a hint parameter, which
forced its callers to apply the hint afterwards through a local setter
interface and a type assertion. Give it the same constructor-style hint
as NewOAuthFlow and set the hint on the concrete flows before they are
handed out as the interface, so a flow is always complete when built
and the caller-side ordering constraint disappears.

An empty hint is a valid value meaning the IdP chooses the account, so
the flows set it unconditionally.
2026-08-14 23:09:43 +02:00
Zoltan Papp
c28cf2fa61 [android] Deduplicate the OAuth token flow and fix the SSH login hint
Extract the shared RequestAuthInfo -> Open -> WaitToken sequence from the
login flow and the SSH JWT flow into runOAuthFlow. Open is now called
synchronously by both flows, matching iOS; openers must post their UI
work instead of blocking, which the app-side openers already do.

The SSH flow read its login hint via profilemanager.GetLoginHint, which
resolves desktop-layout files that the Android app never writes, so the
hint was always empty and the device-code flow could prompt for account
selection. Both flows now read the hint from the profile account file
via the config path, taken from authSnapshot so a concurrent profile
switch cannot pair one profile's config with another's hint.
2026-08-14 22:31:54 +02:00
Zoltan Papp
78c95bb8ec [client] Unify SSH PTY terminal modes on the full CLI table
The shared table used by the Android and wasm terminals was a strict
subset of the CLI one, leaving Ctrl+U, Ctrl+D, Ctrl+Z and friends
without explicit mappings. Export the full table from the ssh package
and derive both CLI variants from it; Windows adds its console-specific
modes to a copy so the shared map is never mutated.
2026-08-14 22:01:36 +02:00
Zoltan Papp
1aa1f915a2 [client] Deduplicate SSH client handshake and bound it with a deadline
Extract the dial-then-handshake sequence into nbssh.Handshake, which
applies the context deadline to the socket for the duration of the
handshake. Previously only the Android client did this; the CLI, wasm
and SSH proxy paths could block forever on a peer that accepts the TCP
connection and then goes silent, since ClientConfig.Timeout is not used
by NewClientConn.
2026-08-14 21:57:21 +02:00
Zoltan Papp
0bb49fa144 [client] Replace Engine SSH host key verifier with PeerKeyLookup
Remove Engine.VerifySSHHostKey and keep GetPeerSSHKey as the only SSH
key API on the Engine. Verification now lives in the ssh package as a
PeerKeyLookup func type implementing HostKeyVerifier, shared by the
android and embed clients.
2026-08-14 21:50:04 +02:00
Zoltan Papp
ceb1719f9a [client] Serialize wasm SSH session startup with Close
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 17:55:20 +02:00
Zoltan Papp
2da4512272 [client] Deduplicate SSH PTY session setup and host key verification
Extract the identical PTY session setup shared by the wasm and Android
terminal clients into ssh.StartPTYSession, and move the stored-key host
verification onto the engine so the embed client delegates and the
Android client passes the engine directly as HostKeyVerifier.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 17:49:31 +02:00
Zoltán Papp
16f7e1e148 Code formate 2026-08-12 12:54:39 +02:00
Zoltán Papp
9531c9cf79 [android] invalidate stale SSH operations across reconnects 2026-08-12 12:52:24 +02:00
Zoltán Papp
ba16475ad6 [android] deliver both output streams before OnClose 2026-08-12 12:45:32 +02:00
Zoltán Papp
a98ced399e Merge remote-tracking branch 'origin/main' into feature/android-client-ssh 2026-08-12 11:51:58 +02:00
Zoltán Papp
a8d2e5b0b2 [android] verify regular SSH host keys with trust-on-first-use
Regular (non-NetBird) servers used InsecureIgnoreHostKey while also
offering the user's password, so an impersonating endpoint could collect
it. Replace that with a per-profile known-hosts store: an unknown host
returns a marker carrying the fingerprint so the client can show it and,
once confirmed, retry with the key trusted and persisted; a changed key
is rejected outright, as OpenSSH does. The confirmation is single-use and
cleared once the key is stored.

The server-type switch now handles the regular case explicitly and
rejects unknown types instead of routing them through the unverified
path. Java sets the store path (per profile, since an overlay IP is a
different host under a different profile) and can drop a host's key once
no session targets it.
2026-08-11 17:38:47 +02:00
Zoltán Papp
cc0702396c [android] bound the SSH handshake with a deadline
DialContext limited only the TCP establishment, so a peer that accepted
the connection and then stayed silent left gossh.NewClientConn blocking
forever and the terminal stuck on "Connecting".

Set the socket deadline from the dial context before the handshake and
clear it on success, so the handshake shares the dial timeout instead of
being able to hang. Verified against a silent listener: the connect now
returns i/o timeout instead of blocking.
2026-08-11 16:50:22 +02:00
Zoltán Papp
6a83476831 [android] stop prompting for a password the server will not take
Any authentication failure on a regular server returned the
password-required marker, so against a server with password
authentication disabled the client asked again after every attempt and
reported each one as a wrong password.

gossh only lists a method under "attempted methods" when the server
offered it. When a supplied password never got attempted, surface the
real error instead of the marker, the same way the desktop client
reports it. A first connect without a password still prompts.
2026-08-11 16:40:12 +02:00
Zoltán Papp
c4c8e2fe1e [android] keep SSH endpoints out of the logs
The connect path logged the target host, port and username at info level,
which the guidelines reserve for debug and below.

Drop the two connect messages entirely rather than lowering them: both sat
directly in front of a return, so the same error already reaches the caller
and the terminal, and OnConnected reports the success. Keep the detected
server type, since it decides the auth path, but log it without the
endpoint.
2026-08-11 15:09:31 +02:00
Zoltán Papp
a33e981c26 [android] reject an out-of-range SSH port
The port arrives as an int because gomobile cannot carry uint16 across the
Java boundary, so nothing rejected a value outside the valid range. It
reached strconv.Itoa and only surfaced as a dial failure, after the server
detection had already spent its timeout.
2026-08-11 15:05:58 +02:00
Zoltán Papp
c1c8ee832e [android] call the SSH auth URL opener synchronously
Open and OnLoginSuccess were each started in their own goroutine, so they
raced. Open is what marks the surface as opened on the client side, and
OnLoginSuccess does nothing until it has, so a token that arrived quickly
left the browser sitting in front of the terminal — the dismissal was
dropped rather than delayed.

The login and session-extend flows do not hit this because their two calls
live in separate functions with a blocking wait between them. Here both
are in one function, so ordering has to come from calling them in turn.

Also groups the file's helpers with the code they serve.
2026-08-11 13:43:30 +02:00
Zoltán Papp
9ee5c04687 [android] dismiss the SSH auth browser once the token arrives
The JWT device-code flow opened the verification URL through the URL
opener but never told it the round-trip had finished, so the Custom Tab
stayed in front of the terminal after the token had already been
collected and the user had to dismiss it by hand.

Call OnLoginSuccess once a non-empty token is in hand, which is what the
login and session-extend flows already do; the Android side reacts by
bringing its own activity forward.
2026-08-11 12:46:02 +02:00
Zoltán Papp
26f7ed858d [android] ask for an SSH password only when the server needs one
Connect() reports a password-required marker instead of a raw handshake
error when a regular SSH server turns down the NetBird key, so the caller
can prompt and retry as often as the user needs. NetBird servers are
excluded: they authenticate with a JWT or the NetBird key, so a failure
there is genuine. The marker is a string because gomobile flattens errors
to their message across the binding.

Errors that reach the terminal are unwrapped to their root cause, so a
dial failure reads "i/o timeout" rather than repeating every layer that
added context; the full chain still goes to the log. A normal shell exit
no longer surfaces as "EOF".

Reset() lets a closed client back a reconnect, which keeps the Java-side
session and its scrollback alive across a drop, and the JWT flow now
reports that it is waiting on the browser instead of blocking silently.
2026-08-09 11:28:56 +02:00
Zoltan Papp
82e799f095 [android] add SSHClient gomobile binding for in-app terminal
Exposes SSHClient + SSHTerminalListener to the Android app. Connect()
auto-detects the server type via banner inspection and selects the auth
path: NetBird-SSH with JWT triggers the device-code OAuth flow via the
existing URLOpener; NetBird-SSH without JWT uses the NetBird private
key; regular SSH falls back to NetBird key then optional password. The
client dials through the running tunnel using a plain net.Dialer and
relies on the gomobile-bound listener for streaming PTY output back to
Java for rendering in an xterm.js WebView.
2026-08-09 08:55:28 +02:00
57 changed files with 1779 additions and 897 deletions

View File

@@ -1,6 +1,6 @@
# NetBird Agent Guidelines
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
network with a control plane. The **agent** (`client/`) runs on user machines as
a privileged daemon and manages the WireGuard interface, routing, firewall, and
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,

View File

@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
## Checklist before submitting a PR
As a critical network service and open source project, we must enforce a few
As a critical network service and open-source project, we must enforce a few
things before submitting a pull request. The
[pull request template](/.github/pull_request_template.md) mirrors this list —
fill it in rather than deleting it.

View File

@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
![CISPA_Logo_BLACK_EN_RZ_RGB (1)](https://user-images.githubusercontent.com/700848/203091324-c6d311a0-22b5-4b05-a288-91cbc6cdcc46.png)
### Acknowledgements
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
### Legal
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.

View File

@@ -14,7 +14,7 @@ Report security issues one of these two ways:
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
- **Email** — `security@netbird.io`.
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
filing a repository report.
### What to include

View File

@@ -191,40 +191,49 @@ func (a *Auth) login(urlOpener URLOpener, isAndroidTV bool) error {
return nil
}
// loginHintSetter is implemented by both concrete flows (PKCE and device code)
// but absent from the OAuthFlow interface, hence the assertion below — the same
// way internal/auth wires it in authenticateWithPKCEFlow.
type loginHintSetter interface {
SetLoginHint(hint string)
}
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, isAndroidTV bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV, profileLoginHint(a.cfgPath))
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
if a.cfgPath != "" {
if hint := readProfileEmail(a.cfgPath); hint != "" {
if setter, ok := oAuthFlow.(loginHintSetter); ok {
setter.SetLoginHint(hint)
}
}
return runOAuthFlow(a.ctx, oAuthFlow, urlOpener, nil)
}
// profileLoginHint returns the stored account email for the profile at cfgPath.
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
func profileLoginHint(cfgPath string) string {
if cfgPath == "" {
return ""
}
return readProfileEmail(cfgPath)
}
// runOAuthFlow drives an already acquired OAuth flow to a token: requests the
// flow info, presents the verification URL through the opener and waits for
// the browser round-trip. Open is called synchronously — it is what marks the
// surface as opened on the client side, and a fast token's OnLoginSuccess is
// a no-op until it has, so the dismissal would be dropped rather than
// delayed. Openers must therefore not block: they post their UI work and
// return. onWaiting, when set, runs after the URL is shown, right before the
// blocking wait.
func runOAuthFlow(ctx context.Context, flow auth.OAuthFlow, urlOpener URLOpener, onWaiting func()) (*auth.TokenInfo, error) {
flowInfo, err := flow.RequestAuthInfo(ctx)
if err != nil {
return nil, fmt.Errorf("request auth info: %w", err)
}
flowInfo, err := oAuthFlow.RequestAuthInfo(context.TODO())
if err != nil {
return nil, fmt.Errorf("getting a request OAuth flow info failed: %v", err)
urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
if onWaiting != nil {
onWaiting()
}
go urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
tokenInfo, err := oAuthFlow.WaitToken(a.ctx, flowInfo)
tokenInfo, err := flow.WaitToken(ctx, flowInfo)
if err != nil {
return nil, fmt.Errorf("waiting for browser login failed: %v", err)
return nil, fmt.Errorf("wait for token: %w", err)
}
return &tokenInfo, nil

View File

@@ -0,0 +1,38 @@
//go:build android
package android
import (
"fmt"
"github.com/netbirdio/netbird/client/internal/profilemanager"
)
type prefsStore interface {
Get(namespace string, v any) (bool, error)
Put(namespace string, v any) error
}
type profilePrefs struct {
prefs *profilemanager.Prefs
}
func newProfilePrefs(configDir, profileID string) (*profilePrefs, error) {
if configDir == "" || profileID == "" {
return nil, fmt.Errorf("profile prefs require a config dir and profile ID")
}
pm := NewProfileManager(configDir)
prefs, err := pm.serviceMgr.ProfilePrefs(profilemanager.ID(profileID), androidUsername)
if err != nil {
return nil, fmt.Errorf("resolve profile prefs: %w", err)
}
return &profilePrefs{prefs: prefs}, nil
}
func (p *profilePrefs) Get(namespace string, v any) (bool, error) {
return p.prefs.Get(namespace, v)
}
func (p *profilePrefs) Put(namespace string, v any) error {
return p.prefs.Put(namespace, v)
}

View File

@@ -0,0 +1,649 @@
//go:build android
package android
import (
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
gossh "golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/profilemanager"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/ssh/detection"
)
const (
sshDialTimeout = 30 * time.Second
sshDetectionTimeout = 5 * time.Second
)
// PasswordRequiredMarker tells Java to prompt for a password and retry. It is
// a string because gomobile flattens errors to their message, so a sentinel
// value would not survive the binding.
const PasswordRequiredMarker = "netbird-ssh-password-required"
// HostKeyUnknownMarker tells Java to show the fingerprint and, on confirmation,
// retry with TrustHostKey set. The presented fingerprint is appended after the
// marker so the prompt can display it and the retry can guard against a key
// that changed between the two connects. Only regular (non-NetBird) servers
// reach this: NetBird peers verify against the registry.
const HostKeyUnknownMarker = "netbird-ssh-hostkey-unknown"
var (
errPasswordRequired = errors.New(PasswordRequiredMarker)
errClientClosed = errors.New("ssh client closed")
)
// errHostKeyUnknown carries the presented fingerprint so Connect can build the
// marker message the Java side parses.
type errHostKeyUnknown struct {
fingerprint string
}
func (e *errHostKeyUnknown) Error() string {
return HostKeyUnknownMarker + ":" + e.fingerprint
}
// SSHTerminalListener receives SSH session events. It is implemented in Java.
//
// All callbacks are invoked from goroutines and may run concurrently with each
// other; the implementation must be safe to call from any thread.
type SSHTerminalListener interface {
OnConnected()
OnData(data []byte)
OnClose(reason string)
OnError(message string)
}
// SSHClient is a NetBird-aware SSH client exposed to Java via gomobile.
//
// It dials through the running NetBird tunnel and runs a standard SSH session
// on top with PTY enabled. Host-key verification uses the NetBird-provided
// peer SSH host keys, identical to the desktop client.
type SSHClient struct {
nb *Client
mu sync.Mutex
listener SSHTerminalListener
urlOpener URLOpener
sshClient *gossh.Client
session *gossh.Session
stdin io.WriteCloser
closed bool
// gen identifies the current connection attempt. Connect and Close bump it,
// so an in-flight dial or a reader left over from a previous connection
// finds itself stale and stays silent instead of publishing OnConnected or
// OnClose for a connection the caller already abandoned.
gen uint64
dialCancel context.CancelFunc
// knownHostsConfigDir and knownHostsProfile locate the TOFU store for
// regular SSH servers in the profile's preferences. Java supplies them,
// since an overlay IP is a different host under a different profile. Empty
// until set: without them a regular server cannot be verified and Connect
// refuses one.
knownHostsConfigDir string
knownHostsProfile string
// trustHostKey carries the fingerprint the user confirmed on a previous
// attempt, so the retry accepts exactly that key and persists it.
trustHostKey string
}
// NewSSHClient creates a new SSH client bound to the running NetBird Client.
func NewSSHClient(c *Client) *SSHClient {
return &SSHClient{nb: c}
}
// SetListener registers the Java listener. Must be called before Connect to
// receive any events.
func (s *SSHClient) SetListener(l SSHTerminalListener) {
s.mu.Lock()
s.listener = l
s.mu.Unlock()
}
// SetURLOpener registers the Java URL opener used to display the device-code
// authorization page in a Custom Tabs window when the target peer requires
// JWT authentication. Must be set before Connect to be effective.
func (s *SSHClient) SetURLOpener(opener URLOpener) {
s.mu.Lock()
s.urlOpener = opener
s.mu.Unlock()
}
// SetKnownHostsStore points the TOFU host-key store at a profile's preferences.
// Must be set before connecting to a regular SSH server; without it such a
// server cannot be verified and Connect refuses one.
func (s *SSHClient) SetKnownHostsStore(configDir, profileID string) {
s.mu.Lock()
s.knownHostsConfigDir = configDir
s.knownHostsProfile = profileID
s.mu.Unlock()
}
// TrustHostKey records the fingerprint the user confirmed for a regular server,
// so the next Connect accepts that exact key and adds it to the known-hosts
// store. Passing a fingerprint that no longer matches makes the connect fail
// rather than trust a key that changed since the prompt.
func (s *SSHClient) TrustHostKey(fingerprint string) {
s.mu.Lock()
s.trustHostKey = fingerprint
s.mu.Unlock()
}
// Connect dials the SSH server through the NetBird tunnel and performs the
// SSH handshake. It auto-detects the server type via SSH banner inspection
// and selects the appropriate authentication path:
//
// - NetBird-SSH server requiring JWT: launches the OAuth 2.0 device-code
// flow, opens the verification URL through the registered URLOpener, and
// uses the resulting token as the SSH password. Host-key verification
// uses the NetBird peer registry.
// - NetBird-SSH server without JWT: authenticates with the NetBird SSH
// private key. Host-key verification uses the NetBird peer registry.
// - Regular SSH server (e.g. OpenSSH): authenticates with the NetBird key
// first (so a user-installed NetBird public key works), then falls back
// to the supplied password if non-empty. Host-key verification is
// trust-on-first-use against the per-profile known-hosts store.
//
// The password parameter is only consulted for regular SSH servers.
func (s *SSHClient) Connect(host string, port int, user, password string) error {
if port < 1 || port > 65535 {
return fmt.Errorf("invalid port: %d", port)
}
cfg, cfgPath, cc := s.nb.authSnapshot()
if cc == nil {
return errors.New("netbird client not running")
}
if cfg == nil {
return errors.New("netbird config not loaded")
}
engine := cc.Engine()
if engine == nil {
return errors.New("netbird engine not available")
}
s.mu.Lock()
s.gen++
gen := s.gen
s.mu.Unlock()
serverType := detectServerType(host, port)
log.Debugf("SSH server type: %s", serverType)
authMethods, hostKeyCallback, err := s.buildAuth(cfg, cfgPath, engine, serverType, password)
if err != nil {
return err
}
clientConfig := &gossh.ClientConfig{
User: user,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: sshDialTimeout,
}
err = s.dialAndHandshake(gen, host, port, clientConfig)
// An unknown host key is a prompt, not a failure: return the marker intact
// (rootCause would unwrap it) so Java can show the fingerprint and retry.
var unknownHost *errHostKeyUnknown
if errors.As(err, &unknownHost) {
return errors.New(unknownHost.Error())
}
// A regular server may still accept a password, so let the caller ask for
// one instead of failing. NetBird servers never use a password, so a
// failure there is genuine.
if err != nil && serverType != detection.ServerTypeNetBirdJWT &&
serverType != detection.ServerTypeNetBirdNoJWT && isAuthFailure(err) &&
passwordCouldHelp(err, password != "") {
return errPasswordRequired
}
if err != nil {
return rootCause(err)
}
return nil
}
// StartSession requests a PTY and starts an interactive shell. Output from
// the session is forwarded to the listener via OnData.
func (s *SSHClient) StartSession(cols, rows int) error {
err := s.startSession(cols, rows)
if err != nil {
log.Infof("SSH: start session failed: %v", err)
return rootCause(err)
}
return nil
}
// Write sends data to the SSH session stdin.
func (s *SSHClient) Write(data []byte) error {
s.mu.Lock()
stdin := s.stdin
s.mu.Unlock()
if stdin == nil {
return errors.New("ssh session not started")
}
if _, err := stdin.Write(data); err != nil {
return fmt.Errorf("write stdin: %w", err)
}
return nil
}
// Resize updates the PTY window size.
func (s *SSHClient) Resize(cols, rows int) error {
s.mu.Lock()
session := s.session
s.mu.Unlock()
if session == nil {
return errors.New("ssh session not started")
}
return session.WindowChange(rows, cols)
}
// Reset makes a closed client usable for another Connect: Close leaves the
// one-shot guard set, and clearing it lets the same client back a reconnect.
func (s *SSHClient) Reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.closed = false
}
// Close terminates the SSH session and underlying connection. Safe to call
// multiple times.
func (s *SSHClient) Close() error {
s.mu.Lock()
s.gen++
if s.dialCancel != nil {
s.dialCancel()
s.dialCancel = nil
}
sshClient := s.sshClient
session := s.session
stdin := s.stdin
s.sshClient = nil
s.session = nil
s.stdin = nil
notify := !s.closed
s.closed = true
listener := s.listener
s.mu.Unlock()
if stdin != nil {
if err := stdin.Close(); err != nil {
log.Debugf("ssh: stdin close: %v", err)
}
}
if session != nil {
if err := session.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: session close: %v", err)
}
}
var firstErr error
if sshClient != nil {
if err := sshClient.Close(); err != nil {
firstErr = err
}
}
if notify && listener != nil {
listener.OnClose("closed by client")
}
return firstErr
}
func (s *SSHClient) startSession(cols, rows int) error {
log.Debugf("SSH: starting session %dx%d", cols, rows)
s.mu.Lock()
sshClient := s.sshClient
gen := s.gen
s.mu.Unlock()
if sshClient == nil {
return errors.New("ssh client not connected")
}
pty, err := nbssh.StartPTYSession(sshClient, cols, rows)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(pty.Session, "stale session")
return errClientClosed
}
s.session = pty.Session
s.stdin = pty.Stdin
s.mu.Unlock()
readerDone := make(chan string, 2)
go func() { readerDone <- s.readLoop(pty.Stdout, "stdout") }()
go func() { readerDone <- s.readLoop(pty.Stderr, "stderr") }()
go func() {
reason := <-readerDone
if second := <-readerDone; reason == "" {
reason = second
}
s.notifyClose(gen, reason)
}()
log.Debug("SSH: session started, shell running")
return nil
}
func (s *SSHClient) buildAuth(cfg *profilemanager.Config, cfgPath string, engine *internal.Engine,
serverType detection.ServerType, password string) ([]gossh.AuthMethod, gossh.HostKeyCallback, error) {
switch serverType {
case detection.ServerTypeNetBirdJWT:
token, err := s.requestJWTToken(cfg, cfgPath)
if err != nil {
return nil, nil, fmt.Errorf("jwt: %w", err)
}
auths := []gossh.AuthMethod{gossh.Password(token)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeNetBirdNoJWT:
if cfg.SSHKey == "" {
return nil, nil, errors.New("no NetBird SSH key available")
}
signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey))
if err != nil {
return nil, nil, fmt.Errorf("parse netbird ssh key: %w", err)
}
auths := []gossh.AuthMethod{gossh.PublicKeys(signer)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeRegular:
var auths []gossh.AuthMethod
if cfg.SSHKey != "" {
if signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey)); err == nil {
auths = append(auths, gossh.PublicKeys(signer))
} else {
log.Debugf("ssh: parse netbird key for regular auth: %v", err)
}
}
if password != "" {
pw := password
auths = append(auths, gossh.Password(pw))
auths = append(auths, gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range questions {
answers[i] = pw
}
return answers, nil
}))
}
if len(auths) == 0 {
// Nothing to offer at all: ask for a password rather than failing,
// so the caller can retry once the user supplies one.
return nil, nil, errPasswordRequired
}
callback, err := s.tofuHostKeyCallback()
if err != nil {
return nil, nil, err
}
return auths, callback, nil
default:
return nil, nil, fmt.Errorf("unsupported SSH server type: %v", serverType)
}
}
// tofuHostKeyCallback verifies a regular server's host key against the
// per-profile known-hosts store. An unknown host returns errHostKeyUnknown so
// Java can show the fingerprint and, once confirmed, retry with the key
// trusted; a changed key is rejected outright, as OpenSSH does. When the user
// has confirmed a fingerprint, the callback accepts exactly that key and
// appends it to the store.
func (s *SSHClient) tofuHostKeyCallback() (gossh.HostKeyCallback, error) {
s.mu.Lock()
configDir := s.knownHostsConfigDir
profileID := s.knownHostsProfile
trusted := s.trustHostKey
s.mu.Unlock()
if configDir == "" || profileID == "" {
return nil, errors.New("no known-hosts store configured for regular SSH")
}
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return nil, fmt.Errorf("load known-hosts store: %w", err)
}
return func(hostname string, remote net.Addr, key gossh.PublicKey) error {
verdict, err := store.verify(hostname, remote, key)
if err != nil {
return err
}
if verdict == hostKeyMatched {
return nil
}
if verdict == hostKeyChanged {
return fmt.Errorf("SSH host key changed for %s (possible attack)", hostname)
}
fingerprint := gossh.FingerprintSHA256(key)
if trusted == "" {
return &errHostKeyUnknown{fingerprint: fingerprint}
}
if trusted != fingerprint {
return fmt.Errorf("SSH host key changed since it was confirmed for %s", hostname)
}
if err := store.append(hostname, remote, key); err != nil {
return fmt.Errorf("persist trusted host key: %w", err)
}
// The confirmation is spent: now that the key is stored, a later
// reconnect must verify against the file, not re-accept this fingerprint.
s.mu.Lock()
s.trustHostKey = ""
s.mu.Unlock()
return nil
}, nil
}
func (s *SSHClient) requestJWTToken(cfg *profilemanager.Config, cfgPath string) (string, error) {
s.mu.Lock()
urlOpener := s.urlOpener
s.mu.Unlock()
if urlOpener == nil {
return "", errors.New("URL opener not configured for JWT auth")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
flow, err := auth.NewOAuthFlow(ctx, cfg, false, true, profileLoginHint(cfgPath))
if err != nil {
return "", fmt.Errorf("create oauth flow: %w", err)
}
// The status callback covers the browser round-trip, which would
// otherwise leave the terminal blank.
tokenInfo, err := runOAuthFlow(ctx, flow, urlOpener, func() {
s.notifyStatus("Waiting for browser authentication...")
})
if err != nil {
return "", err
}
token := tokenInfo.GetTokenToUse()
if token == "" {
return "", errors.New("empty token returned by IdP")
}
// Tells the client the browser round-trip is over so it can dismiss the
// surface it opened, the same way the login and session-extend flows do.
// Without it the Custom Tab stays in front of the terminal even though the
// token has already been collected.
urlOpener.OnLoginSuccess()
return token, nil
}
func (s *SSHClient) dialAndHandshake(gen uint64, host string, port int, clientConfig *gossh.ClientConfig) error {
addr := net.JoinHostPort(host, strconv.Itoa(port))
ctx, cancel := context.WithTimeout(context.Background(), sshDialTimeout)
defer cancel()
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
return errClientClosed
}
s.dialCancel = cancel
s.mu.Unlock()
var dialer net.Dialer
conn, err := dialer.DialContext(ctx, "tcp", addr)
if err != nil {
return fmt.Errorf("dial %s: %w", addr, err)
}
client, err := nbssh.Handshake(ctx, conn, addr, clientConfig)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(client, "stale ssh client")
return errClientClosed
}
s.sshClient = client
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnConnected()
}
return nil
}
func (s *SSHClient) readLoop(r io.Reader, name string) string {
buf := make([]byte, 4096)
for {
n, err := r.Read(buf)
if n > 0 {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
chunk := make([]byte, n)
copy(chunk, buf[:n])
listener.OnData(chunk)
}
}
if err != nil {
// EOF is a normal shell exit, so report it without a reason.
if errors.Is(err, io.EOF) {
return ""
}
log.Debugf("ssh %s read: %v", name, err)
return rootCause(err).Error()
}
}
}
// notifyStatus writes a progress line to the terminal through the normal
// output path, so long steps are visible while nothing else is arriving.
func (s *SSHClient) notifyStatus(text string) {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnData([]byte("\r\n\x1b[33m" + text + "\x1b[0m\r\n"))
}
}
func (s *SSHClient) notifyClose(gen uint64, reason string) {
s.mu.Lock()
if gen != s.gen || s.closed {
s.mu.Unlock()
return
}
s.closed = true
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnClose(reason)
}
}
func closeQuiet(c io.Closer, label string) {
if c == nil {
return
}
if err := c.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: close %s: %v", label, err)
}
}
func detectServerType(host string, port int) detection.ServerType {
ctx, cancel := context.WithTimeout(context.Background(), sshDetectionTimeout)
defer cancel()
dialer := &net.Dialer{}
serverType, err := detection.DetectSSHServerType(ctx, dialer, host, port)
if err != nil {
log.Debugf("ssh: server detection failed: %v (assuming regular SSH)", err)
return detection.ServerTypeRegular
}
return serverType
}
// rootCause returns the innermost error of a %w chain, so the terminal shows
// "i/o timeout" rather than every layer that added context on the way up.
func rootCause(err error) error {
for {
// A joined error has no single root, so keep it as-is.
if _, ok := err.(interface{ Unwrap() []error }); ok {
return err
}
next := errors.Unwrap(err)
if next == nil {
return err
}
err = next
}
}
// isAuthFailure distinguishes credential rejection from dial, timeout and
// host-key errors, which retrying with a password would not fix.
func isAuthFailure(err error) bool {
if errors.Is(err, errPasswordRequired) {
return true
}
var partial *gossh.PartialSuccessError
if errors.As(err, &partial) {
return true
}
return strings.Contains(err.Error(), "unable to authenticate")
}
// passwordCouldHelp reports whether prompting for a password again can change
// the outcome. gossh lists a method under "attempted methods" only when the
// server offered it, so a supplied password that was never attempted means the
// server does not accept passwords and the real error should surface instead.
func passwordCouldHelp(err error, passwordOffered bool) bool {
if !passwordOffered {
return true
}
msg := err.Error()
return strings.Contains(msg, "password") || strings.Contains(msg, "keyboard-interactive")
}

View File

@@ -0,0 +1,168 @@
//go:build android
package android
import (
"bytes"
"net"
"strconv"
"strings"
"sync"
gossh "golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
const knownHostsNamespace = "ssh"
const (
hostKeyUnknown hostKeyVerdict = iota
hostKeyMatched
hostKeyChanged
)
var knownHostsMu sync.Mutex
type hostKeyVerdict uint8
type knownHostsSection struct {
KnownHosts []string `json:"knownHosts"`
}
type knownHostsStore struct {
prefs prefsStore
}
// RemoveKnownHost deletes every known-hosts entry for host:port from the
// profile's store, so a host trusted for a session that is being deleted does
// not linger. Java calls this only once no session targets that host, so a
// shared host stays trusted. A missing entry is not an error: the goal state
// is "absent".
func RemoveKnownHost(configDir, profileID, host string, port int) error {
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return err
}
return store.removeHost(host, port)
}
func openKnownHostsStore(configDir, profileID string) (*knownHostsStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &knownHostsStore{prefs: prefs}, nil
}
func (st *knownHostsStore) verify(hostname string, remote net.Addr, key gossh.PublicKey) (hostKeyVerdict, error) {
lines, err := st.lines()
if err != nil {
return hostKeyUnknown, err
}
targets := knownHostsTargets(hostname, remote)
verdict := hostKeyUnknown
for _, line := range lines {
pubKey, ok := knownHostsLineKey(line, targets)
if !ok {
continue
}
if pubKey.Type() == key.Type() && bytes.Equal(pubKey.Marshal(), key.Marshal()) {
return hostKeyMatched, nil
}
verdict = hostKeyChanged
}
return verdict, nil
}
func (st *knownHostsStore) append(hostname string, remote net.Addr, key gossh.PublicKey) error {
line := knownhosts.Line(knownHostsTargets(hostname, remote), key)
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: append(lines, line)})
}
func (st *knownHostsStore) removeHost(host string, port int) error {
target := knownhosts.Normalize(net.JoinHostPort(host, strconv.Itoa(port)))
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
kept := make([]string, 0, len(lines))
for _, line := range lines {
if knownHostsLineMatches(line, target) {
continue
}
kept = append(kept, line)
}
if len(kept) == len(lines) {
return nil
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: kept})
}
func (st *knownHostsStore) lines() ([]string, error) {
var section knownHostsSection
if _, err := st.prefs.Get(knownHostsNamespace, &section); err != nil {
return nil, err
}
return section.KnownHosts, nil
}
func knownHostsTargets(hostname string, remote net.Addr) []string {
targets := []string{knownhosts.Normalize(hostname)}
if remote != nil {
if normalized := knownhosts.Normalize(remote.String()); normalized != targets[0] {
targets = append(targets, normalized)
}
}
return targets
}
func knownHostsLineKey(line string, targets []string) (gossh.PublicKey, bool) {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return nil, false
}
_, hosts, pubKey, _, _, err := gossh.ParseKnownHosts([]byte(trimmed))
if err != nil {
return nil, false
}
for _, host := range hosts {
for _, target := range targets {
if host == target {
return pubKey, true
}
}
}
return nil, false
}
// knownHostsLineMatches reports whether a known-hosts line's address list
// contains the normalized target. Comment and blank lines never match.
func knownHostsLineMatches(line, target string) bool {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return false
}
fields := strings.Fields(trimmed)
if len(fields) == 0 {
return false
}
for _, addr := range strings.Split(fields[0], ",") {
if addr == target {
return true
}
}
return false
}

View File

@@ -0,0 +1,104 @@
//go:build android
package android
const (
sshSessionsNamespace = "ssh-sessions"
maxStoredSSHSessions = 50
)
type sshSessionRecord struct {
ID string `json:"id"`
Host string `json:"host"`
Port int `json:"port"`
User string `json:"user"`
}
type sshSessionsSection struct {
Sessions []sshSessionRecord `json:"sessions"`
}
// SSHSessionEntry is one stored SSH session, without any credential.
type SSHSessionEntry struct {
ID string
Host string
Port int
User string
}
// SSHSessionArray wraps stored SSH sessions for gomobile compatibility.
type SSHSessionArray struct {
items []*SSHSessionEntry
}
// NewSSHSessionArray creates an empty session array to fill via Add.
func NewSSHSessionArray() *SSHSessionArray {
return &SSHSessionArray{}
}
// Add appends a session entry, oldest first.
func (a *SSHSessionArray) Add(id, host string, port int, user string) {
a.items = append(a.items, &SSHSessionEntry{ID: id, Host: host, Port: port, User: user})
}
// Length returns the number of entries.
func (a *SSHSessionArray) Length() int {
return len(a.items)
}
// Get returns the entry at index i, or nil when out of range.
func (a *SSHSessionArray) Get(i int) *SSHSessionEntry {
if i < 0 || i >= len(a.items) {
return nil
}
return a.items[i]
}
// SSHSessionStore reads and writes a profile's stored SSH sessions.
type SSHSessionStore struct {
prefs prefsStore
}
// NewSSHSessionStore opens the session store of the given profile.
func NewSSHSessionStore(configDir, profileID string) (*SSHSessionStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &SSHSessionStore{prefs: prefs}, nil
}
// Load returns the stored sessions, oldest first.
func (s *SSHSessionStore) Load() (*SSHSessionArray, error) {
var section sshSessionsSection
if _, err := s.prefs.Get(sshSessionsNamespace, &section); err != nil {
return nil, err
}
out := NewSSHSessionArray()
for _, record := range section.Sessions {
if record.ID == "" || record.Host == "" {
continue
}
out.Add(record.ID, record.Host, record.Port, record.User)
}
return out, nil
}
// Save replaces the stored sessions, keeping only the newest entries when the
// list exceeds the storage cap.
func (s *SSHSessionStore) Save(sessions *SSHSessionArray) error {
var items []*SSHSessionEntry
if sessions != nil {
items = sessions.items
}
if len(items) > maxStoredSSHSessions {
items = items[len(items)-maxStoredSSHSessions:]
}
records := make([]sshSessionRecord, 0, len(items))
for _, item := range items {
records = append(records, sshSessionRecord{ID: item.ID, Host: item.Host, Port: item.Port, User: item.User})
}
return s.prefs.Put(sshSessionsNamespace, sshSessionsSection{Sessions: records})
}

View File

@@ -21,7 +21,7 @@ import (
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
sshcommon "github.com/netbirdio/netbird/client/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
@@ -521,12 +521,7 @@ func (c *Client) VerifySSHHostKey(peerAddress string, key []byte) error {
return err
}
storedKey, found := engine.GetPeerSSHKey(peerAddress)
if !found {
return sshcommon.ErrPeerNotFound
}
return sshcommon.VerifyHostKey(storedKey, key, peerAddress)
return nbssh.PeerKeyLookup(engine.GetPeerSSHKey).VerifySSHHostKey(peerAddress, key)
}
// SetPerformance retunes a running Client. Only PreallocatedBuffersPerPool

View File

@@ -53,15 +53,15 @@ func NewProxyBind(bind Bind, mtu uint16) *ProxyBind {
return p
}
// AddRelayedConn adds a new connection to the bind.
// AddTurnConn adds a new connection to the bind.
// endpoint is the NetBird address of the remote peer. The SetEndpoint return with the address what will be used in the
// WireGuard configuration.
//
// Parameters:
// - ctx: Context is used for proxyToLocal to avoid unnecessary error messages
// - nbAddr: The NetBird UDP address of the remote peer, it required to generate fake address
// - remoteConn: The established relayed connection to the remote peer
func (p *ProxyBind) AddRelayedConn(ctx context.Context, nbAddr *net.UDPAddr, remoteConn net.Conn) error {
// - remoteConn: The established TURN connection to the remote peer
func (p *ProxyBind) AddTurnConn(ctx context.Context, nbAddr *net.UDPAddr, remoteConn net.Conn) error {
fakeNetIP, err := fakeAddress(nbAddr)
if err != nil {
return err

View File

@@ -30,9 +30,9 @@ type WGEBPFProxy struct {
proxyPort int
mtu uint16
ebpfManager ebpfMgr.Manager
relayedConnStore map[uint16]net.Conn
relayedConnMutex sync.Mutex
ebpfManager ebpfMgr.Manager
turnConnStore map[uint16]net.Conn
turnConnMutex sync.Mutex
lastUsedPort uint16
rawConnIPv4 net.PacketConn
@@ -50,7 +50,7 @@ func NewWGEBPFProxy(wgPort int, mtu uint16) *WGEBPFProxy {
localWGListenPort: wgPort,
mtu: mtu,
ebpfManager: ebpf.GetEbpfManagerInstance(),
relayedConnStore: make(map[uint16]net.Conn),
turnConnStore: make(map[uint16]net.Conn),
}
return wgProxy
}
@@ -110,14 +110,14 @@ func (p *WGEBPFProxy) Listen() error {
return nil
}
// AddRelayedConn add new relayed connection for the proxy
func (p *WGEBPFProxy) AddRelayedConn(relayedConn net.Conn) (*net.UDPAddr, error) {
wgEndpointPort, err := p.storeRelayedConn(relayedConn)
// AddTurnConn add new turn connection for the proxy
func (p *WGEBPFProxy) AddTurnConn(turnConn net.Conn) (*net.UDPAddr, error) {
wgEndpointPort, err := p.storeTurnConn(turnConn)
if err != nil {
return nil, err
}
log.Infof("relayed conn added to wg proxy store: %s, endpoint port: :%d", relayedConn.RemoteAddr(), wgEndpointPort)
log.Infof("turn conn added to wg proxy store: %s, endpoint port: :%d", turnConn.RemoteAddr(), wgEndpointPort)
wgEndpoint := &net.UDPAddr{
IP: net.ParseIP(loopbackAddr),
@@ -186,48 +186,48 @@ func (p *WGEBPFProxy) readAndForwardPacket(buf []byte) error {
return fmt.Errorf("failed to read UDP packet from WG: %w", err)
}
p.relayedConnMutex.Lock()
conn, ok := p.relayedConnStore[uint16(addr.Port)]
p.relayedConnMutex.Unlock()
p.turnConnMutex.Lock()
conn, ok := p.turnConnStore[uint16(addr.Port)]
p.turnConnMutex.Unlock()
if !ok {
if p.ctx.Err() == nil {
log.Debugf("relayed conn not found by port because conn already has been closed: %d", addr.Port)
log.Debugf("turn conn not found by port because conn already has been closed: %d", addr.Port)
}
return nil
}
if _, err := conn.Write(buf[:n]); err != nil {
return fmt.Errorf("forward local WG packet (%d) to remote relayed conn: %w", addr.Port, err)
return fmt.Errorf("failed to forward local WG packet (%d) to remote turn conn: %w", addr.Port, err)
}
return nil
}
func (p *WGEBPFProxy) storeRelayedConn(relayedConn net.Conn) (uint16, error) {
p.relayedConnMutex.Lock()
defer p.relayedConnMutex.Unlock()
func (p *WGEBPFProxy) storeTurnConn(turnConn net.Conn) (uint16, error) {
p.turnConnMutex.Lock()
defer p.turnConnMutex.Unlock()
np, err := p.nextFreePort()
if err != nil {
return np, err
}
p.relayedConnStore[np] = relayedConn
p.turnConnStore[np] = turnConn
return np, nil
}
func (p *WGEBPFProxy) removeRelayedConn(relayedConnID uint16) {
p.relayedConnMutex.Lock()
defer p.relayedConnMutex.Unlock()
func (p *WGEBPFProxy) removeTurnConn(turnConnID uint16) {
p.turnConnMutex.Lock()
defer p.turnConnMutex.Unlock()
_, ok := p.relayedConnStore[relayedConnID]
_, ok := p.turnConnStore[turnConnID]
if ok {
log.Debugf("remove relayed conn from store by port: %d", relayedConnID)
log.Debugf("remove turn conn from store by port: %d", turnConnID)
}
delete(p.relayedConnStore, relayedConnID)
delete(p.turnConnStore, turnConnID)
}
func (p *WGEBPFProxy) nextFreePort() (uint16, error) {
if len(p.relayedConnStore) == 65535 {
return 0, fmt.Errorf("reached maximum relayed connection numbers")
if len(p.turnConnStore) == 65535 {
return 0, fmt.Errorf("reached maximum turn connection numbers")
}
generatePort:
if p.lastUsedPort == 65535 {
@@ -236,7 +236,7 @@ generatePort:
p.lastUsedPort++
}
if _, ok := p.relayedConnStore[p.lastUsedPort]; ok {
if _, ok := p.turnConnStore[p.lastUsedPort]; ok {
goto generatePort
}
return p.lastUsedPort, nil

View File

@@ -9,32 +9,32 @@ import (
func TestWGEBPFProxy_connStore(t *testing.T) {
wgProxy := NewWGEBPFProxy(1, 1280)
p, _ := wgProxy.storeRelayedConn(nil)
p, _ := wgProxy.storeTurnConn(nil)
if p != 1 {
t.Errorf("invalid initial port: %d", wgProxy.lastUsedPort)
}
numOfConns := 10
for i := 0; i < numOfConns; i++ {
p, _ = wgProxy.storeRelayedConn(nil)
p, _ = wgProxy.storeTurnConn(nil)
}
if p != uint16(numOfConns)+1 {
t.Errorf("invalid last used port: %d, expected: %d", p, numOfConns+1)
}
if len(wgProxy.relayedConnStore) != numOfConns+1 {
t.Errorf("invalid store size: %d, expected: %d", len(wgProxy.relayedConnStore), numOfConns+1)
if len(wgProxy.turnConnStore) != numOfConns+1 {
t.Errorf("invalid store size: %d, expected: %d", len(wgProxy.turnConnStore), numOfConns+1)
}
}
func TestWGEBPFProxy_portCalculation_overflow(t *testing.T) {
wgProxy := NewWGEBPFProxy(1, 1280)
_, _ = wgProxy.storeRelayedConn(nil)
_, _ = wgProxy.storeTurnConn(nil)
wgProxy.lastUsedPort = 65535
p, _ := wgProxy.storeRelayedConn(nil)
p, _ := wgProxy.storeTurnConn(nil)
if len(wgProxy.relayedConnStore) != 2 {
t.Errorf("invalid store size: %d, expected: %d", len(wgProxy.relayedConnStore), 2)
if len(wgProxy.turnConnStore) != 2 {
t.Errorf("invalid store size: %d, expected: %d", len(wgProxy.turnConnStore), 2)
}
if p != 2 {
@@ -46,11 +46,11 @@ func TestWGEBPFProxy_portCalculation_maxConn(t *testing.T) {
wgProxy := NewWGEBPFProxy(1, 1280)
for i := 0; i < 65535; i++ {
_, _ = wgProxy.storeRelayedConn(nil)
_, _ = wgProxy.storeTurnConn(nil)
}
_, err := wgProxy.storeRelayedConn(nil)
_, err := wgProxy.storeTurnConn(nil)
if err == nil {
t.Errorf("invalid relayed conn store calculation")
t.Errorf("invalid turn conn store calculation")
}
}

View File

@@ -121,10 +121,10 @@ func NewProxyWrapper(proxy *WGEBPFProxy) *ProxyWrapper {
}
}
func (p *ProxyWrapper) AddRelayedConn(ctx context.Context, _ *net.UDPAddr, remoteConn net.Conn) error {
addr, err := p.wgeBPFProxy.AddRelayedConn(remoteConn)
func (p *ProxyWrapper) AddTurnConn(ctx context.Context, _ *net.UDPAddr, remoteConn net.Conn) error {
addr, err := p.wgeBPFProxy.AddTurnConn(remoteConn)
if err != nil {
return fmt.Errorf("add relayed conn: %w", err)
return fmt.Errorf("add turn conn: %w", err)
}
headers, err := NewPacketHeaders(p.wgeBPFProxy.localWGListenPort, addr)
@@ -252,7 +252,7 @@ func (p *ProxyWrapper) CloseConn() error {
}
func (p *ProxyWrapper) proxyToLocal(ctx context.Context) {
defer p.wgeBPFProxy.removeRelayedConn(uint16(p.wgRelayedEndpointAddr.Port))
defer p.wgeBPFProxy.removeTurnConn(uint16(p.wgRelayedEndpointAddr.Port))
buf := make([]byte, p.wgeBPFProxy.mtu+bufsize.WGBufferOverhead)
for {
@@ -273,7 +273,7 @@ func (p *ProxyWrapper) proxyToLocal(ctx context.Context) {
if ctx.Err() != nil {
return
}
log.Errorf("failed to write out relayed pkg to local conn: %v", err)
log.Errorf("failed to write out turn pkg to local conn: %v", err)
}
}
}
@@ -286,7 +286,7 @@ func (p *ProxyWrapper) readFromRemote(ctx context.Context, buf []byte) (int, err
}
p.closeListener.Notify()
if !errors.Is(err, io.EOF) {
log.Errorf("failed to read from relayed conn (endpoint: :%d): %s", p.wgRelayedEndpointAddr.Port, err)
log.Errorf("failed to read from turn conn (endpoint: :%d): %s", p.wgRelayedEndpointAddr.Port, err)
}
return 0, err
}

View File

@@ -7,7 +7,7 @@ import (
// Proxy is a transfer layer between the relayed connection and the WireGuard
type Proxy interface {
AddRelayedConn(ctx context.Context, endpoint *net.UDPAddr, remoteConn net.Conn) error
AddTurnConn(ctx context.Context, endpoint *net.UDPAddr, remoteConn net.Conn) error
EndpointAddr() *net.UDPAddr // EndpointAddr returns the address of the WireGuard peer endpoint
Work() // Work start or resume the proxy
Pause() // Pause to forward the packages from remote connection to WireGuard. The opposite way still works.

View File

@@ -95,7 +95,7 @@ func TestProxyCloseByRemoteConn(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
addr, _ := net.ResolveUDPAddr("udp", "100.108.135.221:51892")
relayedConn := newMockConn()
err := tt.proxy.AddRelayedConn(ctx, addr, relayedConn)
err := tt.proxy.AddTurnConn(ctx, addr, relayedConn)
if err != nil {
t.Errorf("error: %v", err)
}
@@ -157,7 +157,7 @@ func redirectTraffic(t *testing.T, proxy Proxy, wgPort int, endPointAddr *net.UD
_ = relayedServer.Close()
}()
if err := proxy.AddRelayedConn(context.Background(), endPointAddr, relayedConn); err != nil {
if err := proxy.AddTurnConn(context.Background(), endPointAddr, relayedConn); err != nil {
t.Errorf("error: %v", err)
}
defer func() {

View File

@@ -119,9 +119,9 @@ func testRedirectAs(t *testing.T, proxy Proxy, wgPort int, nbAddr, p2pEndpoint *
}
defer relayConn.Close()
// Add relayed connection to proxy
if err := proxy.AddRelayedConn(ctx, nbAddr, relayConn); err != nil {
t.Fatalf("failed to add relayed connection: %v", err)
// Add TURN connection to proxy
if err := proxy.AddTurnConn(ctx, nbAddr, relayConn); err != nil {
t.Fatalf("failed to add TURN connection: %v", err)
}
defer func() {
if err := proxy.CloseConn(); err != nil {
@@ -304,8 +304,8 @@ func TestRedirectAs_Multiple_Switches(t *testing.T) {
Port: 38746,
}
if err := proxy.AddRelayedConn(ctx, nbAddr, relayConn); err != nil {
t.Fatalf("failed to add relayed connection: %v", err)
if err := proxy.AddTurnConn(ctx, nbAddr, relayConn); err != nil {
t.Fatalf("failed to add TURN connection: %v", err)
}
defer func() {
if err := proxy.CloseConn(); err != nil {

View File

@@ -51,12 +51,12 @@ func NewWGUDPProxy(wgPort int, mtu uint16) *WGUDPProxy {
return p
}
// AddRelayedConn dials the local WireGuard port and stores the relayed connection.
// AddTurnConn
// The provided Context must be non-nil. If the context expires before
// the connection is complete, an error is returned. Once successfully
// connected, any expiration of the context will not affect the
// connection.
func (p *WGUDPProxy) AddRelayedConn(ctx context.Context, _ *net.UDPAddr, remoteConn net.Conn) error {
func (p *WGUDPProxy) AddTurnConn(ctx context.Context, _ *net.UDPAddr, remoteConn net.Conn) error {
dialer := net.Dialer{}
localConn, err := dialer.DialContext(ctx, "udp", fmt.Sprintf(":%d", p.localWGListenPort))
if err != nil {

View File

@@ -138,26 +138,37 @@ func (a *Auth) IsSSOSupported(ctx context.Context) (bool, error) {
// GetOAuthFlow returns an OAuth flow (PKCE or Device) using the existing management connection
// This avoids creating a new connection to the management server
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool) (OAuthFlow, error) {
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool, hint string) (OAuthFlow, error) {
var flow OAuthFlow
var err error
err = a.withRetry(ctx, func(client *mgm.GrpcClient) error {
err := a.withRetry(ctx, func(client *mgm.GrpcClient) error {
if forceDeviceAuth {
flow, err = a.getDeviceFlow(client)
return err
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
// Try PKCE flow first
flow, err = a.getPKCEFlow(client)
pkceFlow, err := a.getPKCEFlow(client)
if err != nil {
// If PKCE not supported, try Device flow
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
flow, err = a.getDeviceFlow(client)
return err
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
return err
}
pkceFlow.SetLoginHint(hint)
flow = pkceFlow
return nil
})

View File

@@ -97,9 +97,7 @@ func authenticateWithPKCEFlow(ctx context.Context, config *profilemanager.Config
return nil, fmt.Errorf("getting pkce authorization flow info failed with error: %v", err)
}
if hint != "" {
pkceFlowInfo.SetLoginHint(hint)
}
pkceFlowInfo.SetLoginHint(hint)
return pkceFlowInfo, nil
}
@@ -127,9 +125,7 @@ func authenticateWithDeviceCodeFlow(ctx context.Context, config *profilemanager.
}
}
if hint != "" {
deviceFlowInfo.SetLoginHint(hint)
}
deviceFlowInfo.SetLoginHint(hint)
return deviceFlowInfo, nil
}

View File

@@ -35,8 +35,6 @@ var (
// exported so a diagnostic reader reports the same locations that are written.
const (
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
// Older versions used different layouts under the same prefix: a single
// unsuffixed key, then one key per domain, now one key per batch of domains.
NRPTKeyPrefix = "NetBird-Match"
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
@@ -91,6 +89,7 @@ type registryConfigurator struct {
guid string
routingAll bool
gpo bool
nrptEntryCount int
origNameservers []netip.Addr
}
@@ -323,9 +322,14 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
}
if len(matchDomains) != 0 {
if err := r.addDNSMatchPolicy(matchDomains, config.ServerIP); err != nil {
count, err := r.addDNSMatchPolicy(matchDomains, config.ServerIP)
// Update count even on error to ensure cleanup covers partially created rules
r.nrptEntryCount = count
if err != nil {
return fmt.Errorf("add dns match policy: %w", err)
}
} else {
r.nrptEntryCount = 0
}
r.updateState(stateManager)
@@ -341,8 +345,9 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
func (r *registryConfigurator) updateState(stateManager *statemanager.Manager) {
if err := stateManager.UpdateState(&ShutdownState{
Guid: r.guid,
GPO: r.gpo,
Guid: r.guid,
GPO: r.gpo,
NRPTEntryCount: r.nrptEntryCount,
}); err != nil {
log.Errorf("failed to update shutdown state: %s", err)
}
@@ -357,7 +362,7 @@ func (r *registryConfigurator) addDNSSetupForAll(ip netip.Addr) error {
return nil
}
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) error {
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) (int, error) {
// if the gpo key is present, we need to put our DNS settings there, otherwise our config might be ignored
// see https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-gpnrpt/8cc31cb9-20cb-4140-9e85-3e08703b4745
@@ -374,17 +379,19 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, ruleIndex)
if err := r.configureDNSPolicy(localPath, batchDomains, ip); err != nil {
return fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
return ruleIndex, fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
}
// Increment immediately so the caller's cleanup path knows about this rule
ruleIndex++
if r.gpo {
if err := r.configureDNSPolicy(gpoPath, batchDomains, ip); err != nil {
return fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex, err)
return ruleIndex, fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex-1, err)
}
}
log.Debugf("added NRPT rule %d with %d domains", ruleIndex, len(batchDomains))
ruleIndex++
log.Debugf("added NRPT rule %d with %d domains", ruleIndex-1, len(batchDomains))
}
if r.gpo {
@@ -394,7 +401,7 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
}
log.Infof("added %d NRPT rules for %d domains", ruleIndex, len(domains))
return nil
return ruleIndex, nil
}
func (r *registryConfigurator) configureDNSPolicy(policyPath string, domains []string, ip netip.Addr) error {
@@ -527,28 +534,28 @@ func (r *registryConfigurator) restoreHostDNS() error {
return nil
}
// removeDNSMatchPolicies deletes every NRPT rule this client may have created,
// from the local and the GPO policy store. The rules are found by enumerating
// the registry, the only authoritative record of what was written. Cleanup must
// not depend on a rule count: the in-memory one is scoped to a single
// registryConfigurator and the persisted one is deleted on every clean
// disconnect, and a rule left behind keeps resolving names over an interface
// that is gone, until reboot discards the volatile key.
func (r *registryConfigurator) removeDNSMatchPolicies() error {
var merr *multierror.Error
for _, root := range []string{DNSPolicyConfigRoot, GPODNSPolicyConfigRoot} {
names, err := listNRPTRuleKeys(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("list rule keys under %s: %w", root, err))
continue
// Try to remove the base entries (for backward compatibility)
if err := removeRegistryKeyFromDNSPolicyConfig(dnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local base entry: %w", err))
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoDnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO base entry: %w", err))
}
for i := 0; i < r.nrptEntryCount; i++ {
localPath := fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i)
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, i)
if err := removeRegistryKeyFromDNSPolicyConfig(localPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local entry %d: %w", i, err))
}
for _, name := range names {
path := root + `\` + name
if err := removeRegistryKeyFromDNSPolicyConfig(path); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove entry %s: %w", path, err))
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO entry %d: %w", i, err))
}
}
@@ -563,39 +570,6 @@ func (r *registryConfigurator) restoreUncleanShutdownDNS() error {
return r.restoreHostDNS()
}
// listNRPTRuleKeys returns the names of our NRPT rule keys under a policy store
// root. An absent root holds nothing to clean up, which is the normal state of
// the GPO store on a machine without DNS Client policy.
func listNRPTRuleKeys(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, syscall.ERROR_PATH_NOT_FOUND):
// the GPO store is absent on a machine without DNS client policy
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", root)
return nil, nil
case err != nil:
// any other failure has to reach the caller: reporting no rules would
// report a successful cleanup while leaving the rules in place
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closer(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
var ruleKeys []string
for _, name := range names {
// registry key names are case insensitive
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(NRPTKeyPrefix)) {
ruleKeys = append(ruleKeys, name)
}
}
return ruleKeys, nil
}
func removeRegistryKeyFromDNSPolicyConfig(regKeyPath string) error {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.QUERY_VALUE)
if err != nil {

View File

@@ -25,7 +25,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := InterfaceConfigPath + `\` + testGUID
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -56,7 +56,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify 3 NRPT rules exist
assert.Equal(t, 3, countNRPTRuleKeys(t), "Should create 3 NRPT rules for 125 domains")
assert.Equal(t, 3, cfg.nrptEntryCount, "Should create 3 NRPT rules for 125 domains")
for i := 0; i < 3; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -81,7 +81,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify first 2 NRPT rules exist
assert.Equal(t, 2, countNRPTRuleKeys(t), "Should create 2 NRPT rules for 75 domains")
assert.Equal(t, 2, cfg.nrptEntryCount, "Should create 2 NRPT rules for 75 domains")
for i := 0; i < 2; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -106,65 +106,9 @@ func registryKeyExists(path string) (bool, error) {
return true, nil
}
// TestNRPTCleanupWithoutRuleCount verifies that rules written by a previous run
// are removed by a configurator that has no record of how many there are: an
// unclean exit loses the in-memory count and a clean disconnect deletes the
// persisted one, so cleanup cannot depend on either.
func TestNRPTCleanupWithoutRuleCount(t *testing.T) {
if testing.Short() {
t.Skip("skipping registry integration test in short mode")
}
defer cleanupRegistryKeys(t)
cleanupRegistryKeys(t)
testIP := netip.MustParseAddr("100.64.0.1")
// 75 domains produce two indexed rules, as the current layout does
domains := make([]string, 75)
for i := range domains {
domains[i] = fmt.Sprintf(".domain%d.com", i+1)
}
previousRun := &registryConfigurator{}
require.NoError(t, previousRun.addDNSMatchPolicy(domains, testIP))
// the unsuffixed key an older version would have written
require.NoError(t, previousRun.configureDNSPolicy(dnsPolicyConfigMatchPath, []string{".legacy.example.com"}, testIP))
// a policy owned by someone else, which cleanup must not touch
foreignPath := DNSPolicyConfigRoot + `\DnsPolicyConfigTestForeign`
foreignKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, foreignPath, registry.SET_VALUE)
require.NoError(t, err, "Should create foreign policy key")
foreignKey.Close()
defer func() {
_ = registry.DeleteKey(registry.LOCAL_MACHINE, foreignPath)
}()
require.Equal(t, 3, countNRPTRuleKeys(t), "Should have two indexed rules and the legacy one")
// a configurator that never applied a DNS config, as one built after a
// restart or from a shutdown state without a count is
freshRun := &registryConfigurator{}
require.NoError(t, freshRun.removeDNSMatchPolicies())
assert.Equal(t, 0, countNRPTRuleKeys(t), "Should remove every rule left by the previous run")
exists, err := registryKeyExists(foreignPath)
require.NoError(t, err)
assert.True(t, exists, "Should not remove a policy that is not ours")
}
func countNRPTRuleKeys(t *testing.T) int {
t.Helper()
names, err := listNRPTRuleKeys(DNSPolicyConfigRoot)
require.NoError(t, err, "Should list NRPT rule keys")
return len(names)
}
func cleanupRegistryKeys(*testing.T) {
cfg := &registryConfigurator{}
// Clean up more entries to account for batching tests with many domains
cfg := &registryConfigurator{nrptEntryCount: 20}
_ = cfg.removeDNSMatchPolicies()
}
@@ -181,7 +125,7 @@ func TestNRPTDomainBatching(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := InterfaceConfigPath + `\` + testGUID
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -249,7 +193,7 @@ func TestNRPTDomainBatching(t *testing.T) {
require.NoError(t, err)
// Verify that exactly expectedRuleCount rules were created
assert.Equal(t, tc.expectedRuleCount, countNRPTRuleKeys(t),
assert.Equal(t, tc.expectedRuleCount, cfg.nrptEntryCount,
"Should create %d NRPT rules for %d domains", tc.expectedRuleCount, tc.domainCount)
// Verify all expected rules exist

View File

@@ -5,8 +5,9 @@ import (
)
type ShutdownState struct {
Guid string
GPO bool
Guid string
GPO bool
NRPTEntryCount int
}
func (s *ShutdownState) Name() string {
@@ -15,8 +16,9 @@ func (s *ShutdownState) Name() string {
func (s *ShutdownState) Cleanup() error {
manager := &registryConfigurator{
guid: s.Guid,
gpo: s.GPO,
guid: s.Guid,
gpo: s.GPO,
nrptEntryCount: s.NRPTEntryCount,
}
if err := manager.restoreUncleanShutdownDNS(); err != nil {

View File

@@ -2,21 +2,17 @@ package ebpf
import (
_ "embed"
"fmt"
"net"
"sync"
"github.com/cilium/ebpf/link"
"github.com/cilium/ebpf/rlimit"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
"github.com/netbirdio/netbird/client/internal/ebpf/manager"
)
const (
xdpProgName = "nb_xdp_prog"
mapKeyFeatures uint32 = 0
featureFlagWGProxy = 0b00000001
@@ -72,50 +68,21 @@ func (tf *GeneralManager) loadXdp() error {
return err
}
// lo has no native XDP, so the program runs in generic mode. Unless it
// declares multi-buffer support the kernel must linearize every non-linear
// skb before running it. Loopback packets are up to 64 KB, so that is a
// contiguous GFP_ATOMIC allocation per packet, and when it fails the packet
// is dropped before the program runs, stalling local TCP connections.
// Multi-buffer XDP in generic mode requires kernel 6.3, so fall back to a
// plain attach when the kernel rejects it.
err = tf.attachXdp(iFace.Index, true)
if err == nil {
return nil
}
log.Debugf("failed to attach multi-buffer xdp program, retrying without it: %s", err)
return tf.attachXdp(iFace.Index, false)
}
func (tf *GeneralManager) attachXdp(iFaceIndex int, multiBuffer bool) error {
spec, err := loadBpf()
// load pre-compiled programs into the kernel.
err = loadBpfObjects(&tf.bpfObjs, nil)
if err != nil {
return fmt.Errorf("load bpf spec: %w", err)
}
if multiBuffer {
prog, ok := spec.Programs[xdpProgName]
if !ok {
return fmt.Errorf("program %s not found in bpf spec", xdpProgName)
}
prog.Flags |= unix.BPF_F_XDP_HAS_FRAGS
}
if err := spec.LoadAndAssign(&tf.bpfObjs, nil); err != nil {
return fmt.Errorf("load bpf objects: %w", err)
return err
}
tf.link, err = link.AttachXDP(link.XDPOptions{
Program: tf.bpfObjs.NbXdpProg,
Interface: iFaceIndex,
Interface: iFace.Index,
})
if err != nil {
if closeErr := tf.bpfObjs.Close(); closeErr != nil {
log.Debugf("failed to close bpf objects after xdp attach error: %s", closeErr)
}
_ = tf.bpfObjs.Close()
tf.link = nil
return fmt.Errorf("attach xdp: %w", err)
return err
}
return nil
}

View File

@@ -440,7 +440,7 @@ func (conn *Conn) onICEConnectionIsReady(priority conntype.ConnPriority, iceConn
conn.dumpState.NewLocalProxy()
wgProxy, err = conn.newProxy(iceConnInfo.RemoteConn)
if err != nil {
conn.Log.Errorf("failed to add relayed net.Conn to local proxy: %v", err)
conn.Log.Errorf("failed to add turn net.Conn to local proxy: %v", err)
return
}
ep = wgProxy.EndpointAddr()
@@ -878,8 +878,9 @@ func (conn *Conn) newProxy(remoteConn net.Conn) (wgproxy.Proxy, error) {
}
wgProxy := conn.config.WgConfig.WgInterface.GetProxy()
if err := wgProxy.AddRelayedConn(conn.ctx, udpAddr, remoteConn); err != nil {
return nil, fmt.Errorf("add relayed conn to proxy: %w", err)
if err := wgProxy.AddTurnConn(conn.ctx, udpAddr, remoteConn); err != nil {
conn.Log.Errorf("failed to add turn net.Conn to local proxy: %v", err)
return nil, err
}
return wgProxy, nil
}

View File

@@ -255,8 +255,8 @@ func (w *WorkerICE) connect(ctx context.Context, agent *icemaker.ThreadSafeAgent
return
}
w.log.Debugf("agent dial")
remoteConn, err := w.agentDial(ctx, agent, remoteOfferAnswer)
w.log.Debugf("turn agent dial")
remoteConn, err := w.turnAgentDial(ctx, agent, remoteOfferAnswer)
if err != nil {
w.log.Debugf("failed to dial the remote peer: %s", err)
w.closeAgent(agent, w.agentDialerCancel)
@@ -517,8 +517,8 @@ func (w *WorkerICE) onConnectionStateChange(agent *icemaker.ThreadSafeAgent, dia
w.logSuccessfulPaths(agent)
return
case ice.ConnectionStateFailed, ice.ConnectionStateDisconnected, ice.ConnectionStateClosed:
// ice.ConnectionStateClosed happens when we recreate the agent. The P2P to relay switch requires
// notifying conn.onICEStateDisconnected so it can update the currently used priority.
// ice.ConnectionStateClosed happens when we recreate the agent. For the P2P to TURN switch important to
// notify the conn.onICEStateDisconnected changes to update the current used priority
sessionChanged := w.closeAgent(agent, dialerCancel)
@@ -532,7 +532,7 @@ func (w *WorkerICE) onConnectionStateChange(agent *icemaker.ThreadSafeAgent, dia
}
}
func (w *WorkerICE) agentDial(ctx context.Context, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *OfferAnswer) (*ice.Conn, error) {
func (w *WorkerICE) turnAgentDial(ctx context.Context, agent *icemaker.ThreadSafeAgent, remoteOfferAnswer *OfferAnswer) (*ice.Conn, error) {
if isController(w.config) {
return agent.Dial(ctx, remoteOfferAnswer.IceCredentials.UFrag, remoteOfferAnswer.IceCredentials.Pwd)
} else {

View File

@@ -0,0 +1,130 @@
package profilemanager
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"github.com/netbirdio/netbird/util"
)
const prefsFileSuffix = ".prefs.json"
var prefsMu sync.Mutex
// Prefs is a namespaced per-profile preference store backed by a single JSON
// file next to the profile config; it is deleted together with the profile.
type Prefs struct {
path string
}
// ProfilePrefs returns the preference store of the profile identified by id.
func (s *ServiceManager) ProfilePrefs(id ID, username string) (*Prefs, error) {
if !IsValidProfileFilenameStem(id) {
return nil, fmt.Errorf("invalid profile ID: %q", id)
}
if id == defaultProfileName {
return &Prefs{path: filepath.Join(filepath.Dir(DefaultConfigPath), id.String()+prefsFileSuffix)}, nil
}
configDir, err := s.getConfigDir(username)
if err != nil {
return nil, fmt.Errorf("get config directory for user %s: %w", username, err)
}
return &Prefs{path: filepath.Join(configDir, id.String()+prefsFileSuffix)}, nil
}
// Get unmarshals the namespace section into v and reports whether it exists.
func (p *Prefs) Get(namespace string, v any) (bool, error) {
if namespace == "" {
return false, fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return false, err
}
raw, ok := sections[namespace]
if !ok {
return false, nil
}
if err := json.Unmarshal(raw, v); err != nil {
return false, fmt.Errorf("decode prefs namespace %q: %w", namespace, err)
}
return true, nil
}
// Put stores v as the namespace section, replacing any previous value.
func (p *Prefs) Put(namespace string, v any) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
raw, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("encode prefs namespace %q: %w", namespace, err)
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
sections[namespace] = raw
return writePrefsFile(p.path, sections)
}
// Remove deletes the namespace section; a missing one is not an error.
func (p *Prefs) Remove(namespace string) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
if _, ok := sections[namespace]; !ok {
return nil
}
delete(sections, namespace)
return writePrefsFile(p.path, sections)
}
func removePrefsFile(path string) error {
prefsMu.Lock()
defer prefsMu.Unlock()
return os.Remove(path)
}
func readPrefsFile(path string) (map[string]json.RawMessage, error) {
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return map[string]json.RawMessage{}, nil
}
if err != nil {
return nil, fmt.Errorf("read prefs: %w", err)
}
sections := map[string]json.RawMessage{}
if err := json.Unmarshal(data, &sections); err != nil {
return nil, fmt.Errorf("decode prefs: %w", err)
}
return sections, nil
}
func writePrefsFile(path string, sections map[string]json.RawMessage) error {
if err := util.WriteJsonWithRestrictedPermission(context.Background(), path, sections); err != nil {
return fmt.Errorf("write prefs: %w", err)
}
return nil
}

View File

@@ -0,0 +1,138 @@
package profilemanager
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type testPrefsSection struct {
Mode uint8 `json:"mode"`
Dest string `json:"dest"`
}
func TestProfilePrefs_RoundTrip(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2, Dest: "/tmp/x"}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, testPrefsSection{Mode: 2, Dest: "/tmp/x"}, got)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_GetMissingNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
})
}
func TestProfilePrefs_RemoveNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
require.NoError(t, prefs.Remove("filedrop"))
require.NoError(t, prefs.Remove("missing"))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_RejectsInvalidID(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
_, err := sm.ProfilePrefs("../escape", username)
assert.Error(t, err)
})
}
func TestProfilePrefs_RejectsEmptyNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
_, err = prefs.Get("", &testPrefsSection{})
assert.Error(t, err)
assert.Error(t, prefs.Put("", testPrefsSection{}))
assert.Error(t, prefs.Remove(""))
})
}
func TestProfilePrefs_DefaultProfile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
prefs, err := sm.ProfilePrefs(defaultProfileName, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
expected := filepath.Join(filepath.Dir(DefaultConfigPath), "default"+prefsFileSuffix)
_, err = os.Stat(expected)
require.NoError(t, err)
})
}
func TestRemoveProfile_DeletesPrefsFile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2}))
configDir, err := sm.getConfigDir(username)
require.NoError(t, err)
prefsPath := filepath.Join(configDir, created.ID.String()+prefsFileSuffix)
_, err = os.Stat(prefsPath)
require.NoError(t, err)
require.NoError(t, sm.RemoveProfile(created.ID, username))
_, err = os.Stat(prefsPath)
assert.True(t, errors.Is(err, os.ErrNotExist), "prefs file should be removed")
})
}

View File

@@ -420,6 +420,11 @@ func (s *ServiceManager) RemoveProfile(id ID, username string) error {
log.Warnf("failed to remove profile state file %s: %v", stateFile, err)
}
prefsFile := filepath.Join(filepath.Dir(target.Path), id.String()+prefsFileSuffix)
if err := removePrefsFile(prefsFile); err != nil && !os.IsNotExist(err) {
log.Warnf("failed to remove profile prefs file %s: %v", prefsFile, err)
}
return nil
}

View File

@@ -87,10 +87,9 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -1,82 +0,0 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -882,40 +882,26 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
}
return candidates[i].routeMetric < candidates[j].routeMetric
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
// 2. Lowest route metric
// 3. Lowest interface metric
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -939,6 +925,7 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -5,7 +5,6 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -30,7 +29,6 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -40,36 +38,4 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -323,7 +323,7 @@ func (a *Auth) login(urlOpener URLOpener, forceDeviceAuth bool, deviceName strin
const authInfoRequestTimeout = 30 * time.Second
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, forceDeviceAuth bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth, "")
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}

View File

@@ -313,21 +313,23 @@ func Dial(ctx context.Context, addr, user string, opts DialOptions) (*Client, er
// dialSSH establishes an SSH connection without JWT authentication
func dialSSH(ctx context.Context, network, addr string, config *ssh.ClientConfig) (*Client, error) {
if config.Timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, config.Timeout)
defer cancel()
}
dialer := &net.Dialer{}
conn, err := dialer.DialContext(ctx, network, addr)
if err != nil {
return nil, fmt.Errorf("dial %s: %w", addr, err)
}
clientConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
client, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
if closeErr := conn.Close(); closeErr != nil {
log.Debugf("connection close after handshake failure: %v", closeErr)
}
return nil, fmt.Errorf("ssh handshake: %w", err)
return nil, err
}
client := ssh.NewClient(clientConn, chans, reqs)
return &Client{
client: client,
}, nil

View File

@@ -12,6 +12,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
"golang.org/x/term"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
func (c *Client) setupTerminalMode(ctx context.Context, session *ssh.Session) error {
@@ -82,37 +84,7 @@ func (c *Client) setupTerminal(session *ssh.Session, fd int) error {
return fmt.Errorf("get terminal size: %w", err)
}
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
// Ctrl+C
ssh.VINTR: 3,
// Ctrl+\
ssh.VQUIT: 28,
// Backspace
ssh.VERASE: 127,
// Ctrl+U
ssh.VKILL: 21,
// Ctrl+D
ssh.VEOF: 4,
ssh.VEOL: 0,
ssh.VEOL2: 0,
// Ctrl+Q
ssh.VSTART: 17,
// Ctrl+S
ssh.VSTOP: 19,
// Ctrl+Z
ssh.VSUSP: 26,
// Ctrl+O
ssh.VDISCARD: 15,
// Ctrl+R
ssh.VREPRINT: 18,
// Ctrl+W
ssh.VWERASE: 23,
// Ctrl+V
ssh.VLNEXT: 22,
}
modes := nbssh.DefaultTerminalModes
terminal := os.Getenv("TERM")
if terminal == "" {

View File

@@ -10,6 +10,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
const (
@@ -80,28 +82,14 @@ func (c *Client) setupTerminalMode(_ context.Context, session *ssh.Session) erro
w, h := c.getWindowsConsoleSize()
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
ssh.VREPRINT: 18, // Ctrl+R
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
}
for mode, value := range nbssh.DefaultTerminalModes {
modes[mode] = value
}
if err := session.RequestPty("xterm-256color", h, w, modes); err != nil {

View File

@@ -35,6 +35,19 @@ type HostKeyVerifier interface {
VerifySSHHostKey(peerAddress string, key []byte) error
}
// PeerKeyLookup returns the stored SSH host key for a peer address.
type PeerKeyLookup func(peerAddress string) ([]byte, bool)
// VerifySSHHostKey implements HostKeyVerifier by looking up the stored key
// and comparing it against the presented key.
func (l PeerKeyLookup) VerifySSHHostKey(peerAddress string, presentedKey []byte) error {
storedKey, found := l(peerAddress)
if !found {
return ErrPeerNotFound
}
return VerifyHostKey(storedKey, presentedKey, peerAddress)
}
// DaemonHostKeyVerifier implements HostKeyVerifier using the NetBird daemon
type DaemonHostKeyVerifier struct {
client proto.DaemonServiceClient

45
client/ssh/handshake.go Normal file
View File

@@ -0,0 +1,45 @@
package ssh
import (
"context"
"fmt"
"io"
"net"
"time"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// Handshake runs the SSH client handshake on an already dialed conn and
// returns the resulting client. Dialing bounds only the TCP establishment;
// without a deadline on the socket a peer that accepts and then goes silent
// blocks the handshake forever, so the context deadline is applied to conn
// for the duration of the handshake. conn is closed on any error.
func Handshake(ctx context.Context, conn net.Conn, addr string, config *ssh.ClientConfig) (*ssh.Client, error) {
if deadline, ok := ctx.Deadline(); ok {
if err := conn.SetDeadline(deadline); err != nil {
closeHandshake(conn, "conn after deadline error")
return nil, fmt.Errorf("set handshake deadline: %w", err)
}
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
if err != nil {
closeHandshake(conn, "conn after handshake error")
return nil, fmt.Errorf("ssh handshake: %w", err)
}
if err := conn.SetDeadline(time.Time{}); err != nil {
closeHandshake(sshConn, "ssh conn after deadline clear error")
return nil, fmt.Errorf("clear handshake deadline: %w", err)
}
return ssh.NewClient(sshConn, chans, reqs), nil
}
func closeHandshake(c io.Closer, label string) {
if err := c.Close(); err != nil {
log.Debugf("ssh: close %s: %v", label, err)
}
}

View File

@@ -610,13 +610,10 @@ func (p *SSHProxy) dialBackend(ctx context.Context, addr, user, jwtToken string)
return nil, fmt.Errorf("connect to server: %w", err)
}
clientConn, chans, reqs, err := cryptossh.NewClientConn(conn, addr, config)
if err != nil {
_ = conn.Close()
return nil, fmt.Errorf("SSH handshake: %w", err)
}
handshakeCtx, cancel := context.WithTimeout(ctx, sshHandshakeTimeout)
defer cancel()
return cryptossh.NewClient(clientConn, chans, reqs), nil
return nbssh.Handshake(handshakeCtx, conn, addr, config)
}
func (p *SSHProxy) verifyHostKey(hostname string, remote net.Addr, key cryptossh.PublicKey) error {

84
client/ssh/session.go Normal file
View File

@@ -0,0 +1,84 @@
package ssh
import (
"fmt"
"io"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// DefaultTerminalModes are the PTY modes used by the interactive terminal clients.
var DefaultTerminalModes = ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VREPRINT: 18, // Ctrl+R
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
}
// PTYSession is an interactive shell session with a PTY and its I/O pipes.
type PTYSession struct {
Session *ssh.Session
Stdin io.WriteCloser
Stdout io.Reader
Stderr io.Reader
}
// StartPTYSession opens a session on the client, requests an xterm-256color PTY
// with the default terminal modes, wires up the I/O pipes and starts a shell.
// The session is closed on any error.
func StartPTYSession(client *ssh.Client, cols, rows int) (*PTYSession, error) {
session, err := client.NewSession()
if err != nil {
return nil, fmt.Errorf("new session: %w", err)
}
pty, err := setupPTYSession(session, cols, rows)
if err != nil {
if closeErr := session.Close(); closeErr != nil {
log.Debugf("ssh: session close after setup error: %v", closeErr)
}
return nil, err
}
return pty, nil
}
// setupPTYSession requests the PTY, opens the pipes and starts the shell on an
// already created session.
func setupPTYSession(session *ssh.Session, cols, rows int) (*PTYSession, error) {
if err := session.RequestPty("xterm-256color", rows, cols, DefaultTerminalModes); err != nil {
return nil, fmt.Errorf("request pty: %w", err)
}
stdin, err := session.StdinPipe()
if err != nil {
return nil, fmt.Errorf("stdin pipe: %w", err)
}
stdout, err := session.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("stdout pipe: %w", err)
}
stderr, err := session.StderrPipe()
if err != nil {
return nil, fmt.Errorf("stderr pipe: %w", err)
}
if err := session.Shell(); err != nil {
return nil, fmt.Errorf("start shell: %w", err)
}
return &PTYSession{Session: session, Stdin: stdin, Stdout: stdout, Stderr: stderr}, nil
}

View File

@@ -6,11 +6,9 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -62,19 +60,9 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
if p.Hint != "" {
h := p.Hint
req.Hint = &h
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -123,16 +123,8 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
}
resp, err := cli.Login(ctx, req)
@@ -236,6 +228,16 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -259,7 +261,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Profiles.List).
// root and the per-profile state file is user-owned (see Logout below).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,9 +162,8 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -80,13 +80,12 @@ func (c *Client) Connect(host string, port int, username, jwtToken string, ipVer
return fmt.Errorf("dial %s: %w", addr, err)
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
sshClient, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
closeWithLog(conn, "connection after handshake error")
return fmt.Errorf("SSH handshake: %w", err)
return err
}
c.sshClient = ssh.NewClient(sshConn, chans, reqs)
c.sshClient = sshClient
logrus.Infof("SSH: Connected to %s", addr)
return nil
@@ -119,57 +118,26 @@ func (c *Client) getAuthMethods(jwtToken string) ([]ssh.AuthMethod, error) {
return []ssh.AuthMethod{ssh.PublicKeys(signer)}, nil
}
// StartSession starts an SSH session with PTY
// StartSession starts an SSH session with PTY. It holds the client lock for
// the whole startup so Close cannot tear the client down mid-setup and the
// new session cannot be installed into an already closed client.
func (c *Client) StartSession(cols, rows int) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.sshClient == nil {
return fmt.Errorf("SSH client not connected")
}
session, err := c.sshClient.NewSession()
pty, err := nbssh.StartPTYSession(c.sshClient, cols, rows)
if err != nil {
return fmt.Errorf("create session: %w", err)
return err
}
c.mu.Lock()
defer c.mu.Unlock()
c.session = session
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3,
ssh.VQUIT: 28,
ssh.VERASE: 127,
}
if err := session.RequestPty("xterm-256color", rows, cols, modes); err != nil {
closeWithLog(session, "session after PTY error")
return fmt.Errorf("PTY request: %w", err)
}
c.stdin, err = session.StdinPipe()
if err != nil {
closeWithLog(session, "session after stdin error")
return fmt.Errorf("get stdin: %w", err)
}
c.stdout, err = session.StdoutPipe()
if err != nil {
closeWithLog(session, "session after stdout error")
return fmt.Errorf("get stdout: %w", err)
}
c.stderr, err = session.StderrPipe()
if err != nil {
closeWithLog(session, "session after stderr error")
return fmt.Errorf("get stderr: %w", err)
}
if err := session.Shell(); err != nil {
closeWithLog(session, "session after shell error")
return fmt.Errorf("start shell: %w", err)
}
c.session = pty.Session
c.stdin = pty.Stdin
c.stdout = pty.Stdout
c.stderr = pty.Stderr
logrus.Info("SSH: Session started with PTY")
return nil

View File

@@ -115,7 +115,7 @@ sequenceDiagram
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)

View File

@@ -15,10 +15,6 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
| ---- | ---- |
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
@@ -52,8 +48,6 @@ flowchart TD
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
K2 --> P
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
@@ -66,84 +60,6 @@ flowchart TD
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### LLM pricing (management is the sole authority)
**The proxy carries no price list.** Management synthesizes the entire pricing
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
table and there is nothing to reload on the proxy side.
```mermaid
flowchart TD
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
B --> C{AgentNetwork.PricingDefaultsFile}
C -- absent --> D[compiled-in table serves]
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
E --> F[mergedTable atomic.Pointer]
D --> G[DefaultTable]
F --> G
G --> H[buildCostMeterConfigJSON — pricing.defaults]
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
K --> L[pricing.providers keyed by provider record ID]
H --> M[cost_meter ConfigJSON]
L --> M
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
```
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
id → rates. The **full** default table ships to every account: it is small
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
models, so they claim every model) priced.
- `pricing.providers` — provider **record** id → normalized model id → rates,
matched against the `llm.resolved_provider_id` the router stamps. Entries are
**fully materialized here**, at synth time: `materializeEntry` starts from the
default entry for that model so cache rates the operator didn't state are
inherited, overlays operator `input`/`output` verbatim (**including an explicit
0**, which prices a self-hosted or internal endpoint as free rather than
silently reverting to list price), and overlays cache-rate **pointers only when
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
discount, bill this bucket at the input rate". The proxy therefore does two map
lookups and no merging.
Same orphan rule as the router: a provider no enabled policy authorises is
unreachable, so its prices aren't shipped. Model ids are normalized with the
**same** functions the request parser uses (`NormalizeBedrockModel` /
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
first-occurrence-wins, matching the routing dedup order.
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
replace default rates without a rebuild. Schema is `surface → model → rates`
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
- An **explicitly configured** path is *required to load*: a typo or malformed
file fails startup, because the operator believes those rates are live. The
conventional probe is optional — an absent file just serves compiled-in
defaults, and the path stays watched in case it appears later.
- File entries **replace** the compiled-in entry for the same (surface, model)
**whole** — they are not field-merged, so an entry must repeat the cache rates
it wants to keep. Everything the file doesn't mention keeps built-in rates.
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
finite and non-negative — the same constraints the HTTP API enforces on
operator per-provider prices.
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
a parse error keeps the previous table, a deleted file reverts to compiled-in
defaults. A mid-edit save can never take pricing down.
The live table feeds **both** consumers, which is what keeps them consistent: the
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
golden-tested, so operators start from a file matching the built-in rates exactly.
### Budget rule resolution (min-wins, group+user bound)
```mermaid
@@ -208,7 +124,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | |
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | |
| on_response | 6 | `cost_meter` | `{}` | |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
@@ -223,12 +139,6 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
## Things to scrutinize
@@ -266,12 +176,10 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
@@ -280,7 +188,6 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
## Test coverage
@@ -291,9 +198,6 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |

View File

@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing table and cost formula this chain delegates to.
adapters + pricing catalog this chain delegates to.
---
@@ -34,7 +34,7 @@ rewrites.
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:2640](../../../proxy/internal/middleware/builtin/all_test.go)
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
@@ -168,46 +168,12 @@ token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`).
**Management owns pricing.** The proxy carries no embedded price list: the whole
table arrives in this middleware's `ConfigJSON` as
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
plus the operator's stored per-provider prices
([factory.go:1334](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
construction, so a non-finite or negative rate fails the chain build. A price
change is an ordinary mapping push — the chain rebuild yields a fresh instance
over a fresh immutable table, so there is no data dir, no pricing file, no
reload goroutine, and nothing to invalidate.
**Two-tier lookup**
([middleware.go:165183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
1. **Per-provider-record** — the operator's stored price for the route that
actually served the request, keyed by the `llm.resolved_provider_id` that
`llm_router` stamped on the allow path, then by normalized model id. Entries
arrive fully materialized (management folds default cache rates in at synth
time), so there is no merging here. Absent metadata — no router in the chain
— skips this tier.
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
providers, which enumerate no models and therefore get no per-record entry.
**Backward compatibility:** a config with no `pricing` block means management
predates config-delivered pricing. The factory logs one warning at build time
and the instance records `cost.skipped=unknown_model` ($0) for every request
rather than falling back to a stale built-in price list
([factory.go:5560](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
(sibling doc) and is selected by the **surface**, not by which tier the entry
came from — `cost_meter` stays provider-agnostic, and a per-record override on
an Anthropic route still bills its cache buckets additively.
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
### llm_limit_record
@@ -280,14 +246,12 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
block carrying a negative or non-finite rate fails the build too, rather than
mispricing live traffic.
build time.
## Invariants
@@ -356,11 +320,10 @@ non-object `metadata` field
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
middleware config and never mutated, so the lookup path needs no lock or atomic
swap — a price change replaces the whole instance. Every middleware is
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
race detector is the meaningful bar.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
@@ -386,13 +349,13 @@ counter accuracy.
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing table and cost formula.
+ SSE framer + pricing loader.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing table.
— the 8 middlewares that consume this package's parsers + pricing loader.
---
@@ -24,9 +24,8 @@ proxy-framework dependencies:
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/`immutable pricing table + the per-surface cost formula. The
rates themselves come from management inside `cost_meter`'s middleware
config; this package holds no price list and reads no files.
- `pricing/`embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `fixtures/` — captured request/response/stream bodies the tests replay.
The package carries zero proxy-framework dependencies so the same parsers can
@@ -48,9 +47,12 @@ be reused later by a WASM adapter
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
## Request body → parser dispatch
@@ -186,11 +188,9 @@ response leg, covering both Bedrock body shapes:
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
management ships, keyed by the **normalised** model id (region prefix + version
suffix stripped by the request parser; management normalises its keys the same
way at synth time so the two compare equal). `ParseResponse` returns
`ErrStreamingUnsupported` for an
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
@@ -205,34 +205,11 @@ response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing table
### Pricing catalog
**Management is the sole pricing authority.** The proxy carries no embedded
price list and reads no pricing file: the whole table arrives inside
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
is just another push — the chain rebuild constructs a fresh `Table`, so there
is nothing to reload
([pricing.go:17](../../../proxy/internal/llm/pricing/pricing.go)). The
management side of the contract (catalog defaults, the operator's stored
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
management-side module guide; `cost_meter`'s wire shape is in
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
`EntryJSON`
([pricing.go:3645](../../../proxy/internal/llm/pricing/pricing.go)) is the
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
direct struct conversion rather than field-by-field copying (a new rate can't
be silently dropped in transit).
`EntryCosts`
([pricing.go:183234](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module. The
**surface** (the `llm.provider` value the request parser stamped) selects the
formula, never the tier the entry came from: a per-provider-record override on
an Anthropic route still bills its cache buckets additively.
`Table.Cost`
([pricing.go:129174](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
| Provider | Formula |
|---|---|
@@ -241,7 +218,7 @@ an Anthropic route still bills its cache buckets additively.
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:214229](../../../proxy/internal/llm/pricing/pricing.go)):
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
@@ -249,12 +226,15 @@ to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Costs`
([pricing.go:143163](../../../proxy/internal/llm/pricing/pricing.go)) is the
per-request split. The four per-bucket fields are the base; `TotalUSD` and
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
the breakdown. `InputUSD` is always the non-cached input bucket on both
provider shapes, so input and cached-input never double-count.
`Loader`
([pricing.go:212268](../../../proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
## Public contracts
@@ -284,38 +264,29 @@ Order matters: `DetectFromURL` ties resolve by registration order.
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` construction + lookup**
([pricing.go:60130](../../../proxy/internal/llm/pricing/pricing.go)):
**`Pricing` lookup**
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
```go
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
func (t *Table) Lookup(provider, model string) (Entry, bool)
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
```
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
reject any non-finite or negative rate, so a corrupt config fails the chain
build rather than mispricing silently. Nil input yields an empty,
never-matching table.
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
([pricing.go:9699](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means the surface or model is absent from the table management sent;
the caller emits `cost.skipped=unknown_model`.
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130132](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
## Invariants
1. **The pricing package is pure and platform-independent.** No file I/O, no
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
there is nothing platform-specific left to port. Anything reintroducing a
read-from-disk path here re-splits pricing authority between management and
the proxy, which is exactly what this design removed.
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:1416](../../../proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
@@ -327,45 +298,38 @@ the caller emits `cost.skipped=unknown_model`.
usage rather than aborting
([streaming.go:6873, 144150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **Management is the only source of rates.** `Table` has no constructor that
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
management sent. A missing or empty `pricing` block therefore means *no
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
never a stale built-in fallback that would silently bill list price.
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:2930](../../../proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:4249](../../../proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
4. **Tables are immutable once built.** `Table.entries` is written only in
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
map is likewise build-time-only
([pricing.go:4752](../../../proxy/internal/llm/pricing/pricing.go)). This
is what makes the no-reload design safe: a price change arrives as a mapping
push that builds a new middleware instance over a new table, so concurrent
readers can't observe a half-updated price list and no atomic swap or lock
is needed on the hot path.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370394](../../../proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
5. **Rate validation happens at chain-build time, not per request.**
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
([pricing.go:6083](../../../proxy/internal/llm/pricing/pricing.go)), naming
the offending surface/model/field in the error. Management enforces the same
constraints at its API boundary and in its YAML parser, so this is
defense-in-depth — but it means a corrupt push fails loudly at build instead
of producing negative costs on live traffic. Test:
`TestNewTable_ValidatesRates`.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:2557](../../../proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
`pricing.Entry` together.** `NewEntries` converts by direct struct
conversion `Entry(e)`
([pricing.go:7678](../../../proxy/internal/llm/pricing/pricing.go)), which
only compiles while the two structs stay field-identical — so the proxy half
is compiler-enforced. The management half is not: a rate added there but not
here unmarshals into nothing and prices that bucket at `InputPer1K`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397398](../../../proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
## Things to scrutinise
**Correctness.** Verify the OpenAI cached-prompt clamp at
[pricing.go:203206](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. Negative token counts are clamped to zero up
front ([pricing.go:186197](../../../proxy/internal/llm/pricing/pricing.go)) so
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147149](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
@@ -374,27 +338,22 @@ noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
oversize surface is gone entirely — the config channel (an authenticated
mapping push from management) is now the only way rates enter the proxy, and
`NewEntries` is the validation boundary on it. A new rate added to management's
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
path (see invariant 6).
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
**Concurrency.** Nothing in this package is shared mutable state: tables are
built once and never written again, so `cost_meter`'s hot path is lock-free by
construction rather than by atomic swap. Per-call `Scanner` instances mean no
shared state across concurrent response-parser calls.
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
`ReadString('\n')` per line. No background goroutines and no per-request
allocation of pricing state.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Observability.** A config carrying no `pricing` block logs one warning at
chain-build time (`cost_meter` factory) and then records
`cost.skipped=unknown_model` per request, so an old-management deployment is
visible in both logs and the access log rather than quietly reporting $0.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
@@ -406,7 +365,7 @@ right `cost.skipped` reason.
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
@@ -414,15 +373,14 @@ right `cost.skipped` reason.
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
No pricing fixture: the table is config-delivered, so pricing tests construct
it in-process from a wire-shape map.
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
`llm.NewScanner`, `pricing.NewLoader`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -1,7 +1,7 @@
# proxy/runtime — translate + serve + log
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
## Module boundary
@@ -114,7 +114,8 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
## Public contracts touched
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).

View File

@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic``llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** surface of the pricing table
management ships (the parser emits the standard Anthropic provider label, so
Vertex Claude reuses first-party Anthropic prices).
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
- `openai``llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
@@ -104,9 +104,8 @@ is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing set the
> affected models' prices on the provider record, or replace the default entries
> via management's `AgentNetwork.PricingDefaultsFile`.
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
## AWS Bedrock (`bedrock_api`)
@@ -212,19 +211,15 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The **catalog is the source of default
prices**: management's `pricing.DefaultTable` folds every catalog provider's
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
is structural rather than maintained in a parallel file
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
providers contribute the same (surface, model) at different rates. Bedrock
entries are keyed by the **normalised** id the request parser emits (region
prefix + version suffix stripped) — management applies the same normalisation to
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
surface.
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
## Things to scrutinise
@@ -237,17 +232,16 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something absent from the shipped
pricing table meters at `cost.skipped=unknown_model` rather than failing the
request. The
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in the catalog comment.
Operators needing exact regional billing set per-provider prices on the model
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
## Cross-references

View File

@@ -6,7 +6,7 @@
"name": "NetBird GmbH",
"email": "hello@netbird.io",
"phone": "",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open-source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open-source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"webpageUrl": {
"url": "https://github.com/netbirdio"
}
@@ -15,7 +15,7 @@
{
"guid": "netbird",
"name": "NetBird",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open-source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"webpageUrl": {
"url": "https://github.com/netbirdio/netbird"
},
@@ -59,7 +59,7 @@
"guid": "support-yearly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Yearly",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "yearly",
@@ -72,7 +72,7 @@
"guid": "support-one-time-year",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Year",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "one-time",
@@ -85,7 +85,7 @@
"guid": "support-one-time-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Monthly",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",
@@ -98,7 +98,7 @@
"guid": "support-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Month",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",

12
go.mod
View File

@@ -19,7 +19,7 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/vishvananda/netlink v1.3.1
golang.org/x/crypto v0.55.0
golang.org/x/crypto v0.54.0
golang.org/x/sys v0.47.0
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
@@ -127,9 +127,9 @@ require (
go.uber.org/zap v1.27.0
goauthentik.io/api/v3 v3.2023051.3
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733
golang.org/x/mod v0.39.0
golang.org/x/net v0.58.0
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab
golang.org/x/mod v0.37.0
golang.org/x/net v0.56.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.22.0
golang.org/x/term v0.45.0
@@ -313,8 +313,8 @@ require (
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/text v0.41.0 // indirect
golang.org/x/tools v0.49.0 // indirect
golang.org/x/text v0.40.0 // indirect
golang.org/x/tools v0.47.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect

24
go.sum
View File

@@ -728,13 +728,13 @@ golang.org/x/crypto v0.18.0/go.mod h1:R0j02AL6hcrfOiy9T4ZYp/rcWeMxM3L6QYxlOuEG1m
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733 h1:XKMObIaAElmkdO+4SQh1iCfzwciZHJi1OblnX9BED9k=
golang.org/x/mobile v0.0.0-20260816165457-f98cc9b3c733/go.mod h1:jMwjxoDSx9jqhNaZqPnr6nnKzb7cs+Dy1Czk7wdX+R8=
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab h1:Iqyc+2zr7aGyLuEadIm0KRJP0Wwt+fhlXLa51Fxf1+Q=
golang.org/x/mobile v0.0.0-20251113184115-a159579294ab/go.mod h1:Eq3Nh/5pFSWug2ohiudJ1iyU59SO78QFuh4qTTN++I0=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
@@ -744,8 +744,8 @@ golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
@@ -764,8 +764,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.20.0/go.mod h1:z8BVo6PvndSri0LbOE3hAn0apkU+1YvI6E70E9jsnvY=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.8.0/go.mod h1:yr7u4HXZRm1R1kBWqr/xKNqewf0plRYoB7sla+BCIXE=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
@@ -843,8 +843,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -858,8 +858,8 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

View File

@@ -221,6 +221,9 @@ render_override() {
# Remove this file (and config.yaml.enterprise if present) to revert.
services:
${DASHBOARD_SERVICE}:
image: \${NETBIRD_DASHBOARD_IMAGE:-ghcr.io/netbirdio/dashboard-cloud:latest}
${COMBINED_SERVICE}:
image: \${NETBIRD_SERVER_IMAGE:-ghcr.io/netbirdio/netbird-server-cloud:latest}
environment:

View File

@@ -4608,7 +4608,7 @@ components:
FleetDMMatchAttributes:
type: object
description: Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
description: Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
additionalProperties: false
properties:
disk_encryption_enabled:

View File

@@ -2876,7 +2876,7 @@ type EDRFleetDMRequest struct {
// LastSyncedInterval The devices last sync requirement interval in hours. Minimum value is 24 hours
LastSyncedInterval int `json:"last_synced_interval"`
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
MatchAttributes FleetDMMatchAttributes `json:"match_attributes"`
}
@@ -2909,7 +2909,7 @@ type EDRFleetDMResponse struct {
// LastSyncedInterval The devices last sync requirement interval in hours.
LastSyncedInterval int `json:"last_synced_interval"`
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
MatchAttributes FleetDMMatchAttributes `json:"match_attributes"`
// UpdatedAt Timestamp of when the integration was last updated.
@@ -3129,7 +3129,7 @@ type Event struct {
// EventActivityCode The string code of the activity that occurred during the event
type EventActivityCode string
// FleetDMMatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
// FleetDMMatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
type FleetDMMatchAttributes struct {
// DiskEncryptionEnabled Whether disk encryption (FileVault/BitLocker) must be enabled on the host
DiskEncryptionEnabled *bool `json:"disk_encryption_enabled,omitempty"`