Compare commits

..

13 Commits

Author SHA1 Message Date
Theodor S. Midtlien
8a46353ae2 Add tests 2026-07-21 15:40:39 +02:00
Zoltán Papp
d663da8f82 Merge branch 'main' into refactor/relay-foreign-cache
# Conflicts:
#	shared/relay/client/manager.go
2026-07-21 11:27:09 +02:00
Maycon Santos
82fdfa84b8 [proxy] match Bedrock provider models against the normalized request model (#6773)
## Describe your changes

Native AWS Bedrock requests carry the model in the URL path as a
cross-region inference-profile id (e.g.
`us.anthropic.claude-haiku-4-5`). The request parser normalizes that to
the catalog key (`anthropic.claude-haiku-4-5`) before the router runs,
but the router matched it against the operator's registered provider
models with exact string equality. So a Bedrock provider registered with
the id Bedrock actually uses (`us.anthropic…`) never matched a
normalized request → the request denied with
`llm_policy.model_not_routable` ("no provider configured for model …").
Only a provider registered with the already-stripped catalog id worked,
which is not how Bedrock ids appear.

Fix: introduce a single shared `llm.NormalizeBedrockModel` (the same
ARN/region-prefix/version-suffix stripping the parser already does) and,
in the router's `routeClaimsModel`, normalize a **Bedrock** route's
candidate models before comparing. Now a Bedrock provider registered
with either the raw inference-profile id or the normalized catalog id
matches the request. Non-Bedrock routes keep exact matching.

Surfaced by the new native-Bedrock e2e (`WireBedrock`,
`/model/{id}/invoke`); the old e2e used the Anthropic body shape, which
never normalized either side and so hid this.

The request parser keeps its own identical normalizer for now;
de-duplicating it onto `llm.NormalizeBedrockModel` is a trivial
follow-up.

## Issue ticket number and link

N/A — follow-up to the Agent Network Bedrock support / model-allowlist
work.

## Stack

<!-- branch-stack -->

### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [x] Created tests that fail without the change (if possible)
- [x] This change does **not** modify the public API, gRPC protocols,
functionality behavior, CLI / service flags, or introduce a new feature
— **OR** I have discussed it with the NetBird team beforehand (link the
issue / Slack thread in the description). See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

Internal routing correctness fix; no user-facing surface change.

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

## Tests

- `proxy/internal/llm`: `NormalizeBedrockModel` unit cases (region
prefixes, version suffixes, ARN).
- `proxy/internal/middleware/builtin/llm_router`: `routeClaimsModel`
matches a Bedrock route registered with the raw `us.anthropic…` id
against a normalized request model; non-Bedrock routes still match
exactly.

Note: full through-tunnel e2e verification of this (the native-Bedrock
`TestProvidersMatrix/bedrock`) also needs the DNS lazy-connection
warm-up (separate PR) to get the client past the proxy-peer gate; they
converge once both land.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved Amazon Bedrock model matching across ARN formats, regional
prefixes, and version or throughput suffixes.
* Bedrock routes now correctly match equivalent model identifiers even
when requests and route configurations use different formats.
  * Non-Bedrock model matching remains exact.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-21 10:10:12 +02:00
Zoltan Papp
ca80e49aa0 [client] Refresh WireGuard stats in mobile debug bundles (#6814)
The iOS and Android DebugBundle paths built GeneratorDependencies
without setting RefreshStatus, so the bundle's status.txt read the
cached peer state instead of live WireGuard interface stats. When the
periodic health probe had not run yet, connected relayed peers showed
"handshake: -" and "0 B/0 B" even though the interface was passing
traffic.

Wire RefreshStatus to RunHealthProbes on both platforms, matching the
desktop daemon path in client/server/debug.go. The engine reference is
already available in the cc.Engine() block used for client metrics.

## Describe your changes

## Issue ticket number and link

## Stack

<!-- branch-stack -->

### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [ ] Created tests that fail without the change (if possible)
- [ ] This change does **not** modify the public API, gRPC protocols,
functionality behavior, CLI / service flags, or introduce a new feature
— **OR** I have discussed it with the NetBird team beforehand (link the
issue / Slack thread in the description). See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/netbirdio/codesmith/netbird/pr/6814"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img
alt="View with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a>
<a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1786881440&installation_id=146802194&pr_number=6814&repository=netbirdio%2Fnetbird&return_to=https%3A%2F%2Fgithub.com%2Fnetbirdio%2Fnetbird%2Fpull%2F6814&signature=c07cbcef1028a366312f7743e6479f2fddafd568aac498ab95828a372ac6843c"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a>
<sup>Need help on this PR? Tag <code>/codesmith</code> with what you
need. Autofix is disabled.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved debug bundle generation on Android and iOS by refreshing
connection health status before collecting diagnostic information.
* Ensured debug bundles include more current health-related data for
troubleshooting.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-21 09:25:33 +02:00
Zoltan Papp
51f17bf919 [client] Update wails to v3.0.0-alpha2.117 (#6837)
## Describe your changes

Update wails to v3.0.0-alpha2.117

## Issue ticket number and link

## Stack

<!-- branch-stack -->

### Checklist
- [ ] Is it a bug fix
- [ ] Is a typo/documentation fix
- [x] Is a feature enhancement
- [ ] It is a refactor
- [ ] Created tests that fail without the change (if possible)
- [ ] This change does **not** modify the public API, gRPC protocols,
functionality behavior, CLI / service flags, or introduce a new feature
— **OR** I have discussed it with the NetBird team beforehand (link the
issue / Slack thread in the description). See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/netbirdio/codesmith/netbird/pr/6837"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img
alt="View with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a>
<a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1787156447&installation_model_id=427504&pr_number=6837&repository=netbirdio%2Fnetbird&return_to=https%3A%2F%2Fgithub.com%2Fnetbirdio%2Fnetbird%2Fpull%2F6837&signature=600f7cb9ecda9ee75b7fe841ed81149507d00dbf6323c61aa92ac2acdaea2487"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a>
<sup>Need help on this PR? Tag <code>/codesmith</code> with what you
need. Autofix is disabled.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated the application framework dependency to a newer release.
  * Removed an obsolete supporting dependency requirement.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-20 21:12:22 +02:00
Zoltan Papp
cf101c44b4 [relay] Count winning attempt before draining losers
The success path in handleResult left settled at 0, so drainLoser
waited on a result that was already consumed and blocked forever,
leaking a goroutine and the results channel on every successful race.
Increment settled before stop() so drainLoser only waits for actual
started losers.
2026-07-02 12:22:01 +02:00
Zoltan Papp
79b51a79e4 [relay] Release home relay lock before the connection race
OpenConn held relayClientMu for the whole call, including the network-bound
FallbackOpener.Run, which blocked onServerDisconnected/storeClient from taking
the write lock and delayed reconnect by up to the race timeout. Snapshot the
home relay client under the read lock, release it, and run the foreign check
and the race on the snapshot. isForeignServer now takes the client so it uses
the snapshot instead of re-reading the field after the lock is dropped.
2026-07-01 19:22:41 +02:00
Zoltan Papp
a737504ec9 Fix timeout 2026-07-01 17:11:23 +02:00
Zoltan Papp
3b1beb3497 [relay] Return raceOutcome struct from handleResult to avoid nilnil
handleResult returned (net.Conn, error, bool) and produced (nil, nil, false)
on the non-terminal path, which the nilnil linter rejects. Return a
raceOutcome{conn, err, done} struct instead so the not-done case carries no
ambiguous nil value / nil error pair.
2026-07-01 17:05:46 +02:00
Zoltan Papp
49c0aeb6ce [relay] Update relay client tests for the new OpenConn signature
Manager.OpenConn now takes a RelayServer struct and a preferForeign
flag instead of a server address string and a serverIP. Adjust the
manager tests accordingly: bundle addr/IP into RelayServer and set
preferForeign=true for foreign-relay dials, false for home-relay dials.
2026-07-01 17:00:20 +02:00
Zoltan Papp
6774a43eae [relay] Rename ConnRacer to FallbackOpener and align file names with types
Rename the ConnRacer type to FallbackOpener and its constructor to
NewFallbackOpener. Rename fallback.go to fallback_opener.go and
foreign_relays.go to foreign_relays_store.go so file names mirror their
primary struct.
2026-07-01 16:53:42 +02:00
Zoltan Papp
50a29c07ce [relay] Race home and foreign relay for peer connection setup
When a peer advertises a relay different from the home relay, open the
peer connection by racing the home relay and the remote peer's relay in
parallel: start the preferred one (controller prefers home, otherwise the
remote relay), fall back to the other after a delay, use whichever connects
first and drain the loser.

Expose the foreign relay cache as ForeignRelaysStore and the race as
ConnRacer. The manager passes the remote relay as a RelayServer{Addr, IP}.
OpenConn no longer returns the winning server address; the worker derives it
from the returned conn's RemoteAddr to register the close listener, matching
the previous main-branch behavior.
2026-06-30 18:26:13 +02:00
Zoltan Papp
7d8e20030b [relay] Extract foreign relay client cache into a dedicated type
Move the foreign-relay client cache out of Manager into a foreignRelays
type. Concurrent first-time connects to the same server are deduplicated
with singleflight, so the cache mutex is never held during a network
connect (removing the previous stall where a slow connect blocked all map
operations). A per-entry in-use refcount prevents the cleanup loop from
closing a client while a connection is being opened on it.

This drops RelayTrack, its per-track lock and the hand-over-hand locking
between the map lock and the track lock. The exported API is unchanged.
2026-06-29 00:41:47 +02:00
51 changed files with 1130 additions and 1449 deletions

View File

@@ -247,6 +247,9 @@ func (c *Client) DebugBundle(platformFiles PlatformFiles, anonymize bool) (strin
deps.SyncResponse = resp
if e := cc.Engine(); e != nil {
deps.RefreshStatus = func() {
e.RunHealthProbes(context.Background(), true)
}
if cm := e.GetClientMetrics(); cm != nil {
deps.ClientMetrics = cm
}

View File

@@ -6,7 +6,6 @@ import (
"fmt"
"io"
"io/fs"
"net"
"os"
"os/signal"
"path"
@@ -80,8 +79,6 @@ var (
updateSettingsDisabled bool
captureEnabled bool
networksDisabled bool
socketOwner string
strictSocketDisabled bool
rootCmd = &cobra.Command{
Use: "netbird",
@@ -146,12 +143,10 @@ func init() {
defaultDaemonAddr := "unix:///var/run/netbird.sock"
if runtime.GOOS == "windows" {
// Named pipe (not loopback TCP): the pipe SDDL gates who may connect and
// the pipe client token carries the caller's SID for per-RPC authorization.
defaultDaemonAddr = "npipe://netbird"
defaultDaemonAddr = "tcp://127.0.0.1:41731"
}
rootCmd.PersistentFlags().StringVar(&daemonAddr, "daemon-addr", defaultDaemonAddr, "Daemon service address to serve CLI requests [unix|tcp|npipe]://[path|host:port|name]")
rootCmd.PersistentFlags().StringVar(&daemonAddr, "daemon-addr", defaultDaemonAddr, "Daemon service address to serve CLI requests [unix|tcp]://[path|host:port]")
rootCmd.PersistentFlags().StringVarP(&managementURL, "management-url", "m", "", fmt.Sprintf("Management Service URL [http|https]://[host]:[port] (default \"%s\")", profilemanager.DefaultManagementURL))
rootCmd.PersistentFlags().StringVar(&adminURL, "admin-url", "", fmt.Sprintf("Admin Panel URL [http|https]://[host]:[port] (default \"%s\")", profilemanager.DefaultAdminURL))
rootCmd.PersistentFlags().StringVarP(&logLevel, "log-level", "l", "info", "sets NetBird log level")
@@ -270,31 +265,16 @@ func FlagNameToEnvVar(cmdFlag string, prefix string) string {
}
// DialClientGRPCServer returns client connection to the daemon server.
//
// The daemon reads the caller's kernel identity from the transport (SO_PEERCRED
// on a Unix socket, the client token on a Windows named pipe), so the client
// side uses insecure (plaintext) credentials — it needs no cooperation to be
// identified. For npipe addresses we install a context dialer since gRPC's
// resolver does not understand Windows named pipes.
func DialClientGRPCServer(ctx context.Context, addr string) (*grpc.ClientConn, error) {
ctx, cancel := context.WithTimeout(ctx, time.Second*10)
defer cancel()
opts := []grpc.DialOption{
return grpc.DialContext(
ctx,
strings.TrimPrefix(addr, "tcp://"),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithBlock(),
}
target := strings.TrimPrefix(addr, "tcp://")
if strings.HasPrefix(addr, "npipe://") {
path := pipePath(strings.TrimPrefix(addr, "npipe://"))
opts = append(opts, grpc.WithContextDialer(func(dialCtx context.Context, _ string) (net.Conn, error) {
return dialNamedPipe(dialCtx, path)
}))
target = "passthrough:///netbird-daemon-pipe"
}
return grpc.DialContext(ctx, target, opts...)
)
}
// WithBackOff execute function in backoff cycle.

View File

@@ -56,9 +56,6 @@ func init() {
serviceCmd.PersistentFlags().BoolVar(&enableJSONSocket, "enable-json-socket", false, "Enables the HTTP/JSON API socket served by grpc-gateway. To persist, use: netbird service install --enable-json-socket")
serviceCmd.PersistentFlags().StringVar(&jsonSocket, "json-socket", defaultJSONSocket, "HTTP/JSON API socket address [unix|tcp]://[path|host:port]. Requires --enable-json-socket to serve. To persist, use: netbird service install --enable-json-socket --json-socket")
serviceCmd.PersistentFlags().StringVar(&socketOwner, "socket-owner", "", "user to own the daemon control socket; restricts it to that user plus the netbird group (0660). If unset, the first client to connect claims ownership (trust-on-first-use). Persisted via: netbird service install --socket-owner")
serviceCmd.PersistentFlags().BoolVar(&strictSocketDisabled, "disable-strict-socket", false, "leave the daemon control socket world-writable (0666) instead of restricting it (root-only, discouraged). Persisted via: netbird service install --disable-strict-socket")
rootCmd.PersistentFlags().StringVarP(&serviceName, "service", "s", defaultServiceName, "Netbird system service name")
serviceEnvDesc := `Sets extra environment variables for the service. ` +
`You can specify a comma-separated list of KEY=VALUE pairs. ` +

View File

@@ -5,7 +5,6 @@ package cmd
import (
"context"
"fmt"
"runtime"
"time"
"github.com/kardianos/service"
@@ -14,31 +13,12 @@ import (
"github.com/spf13/cobra"
"google.golang.org/grpc"
"github.com/netbirdio/netbird/client/internal/ipcauth"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/util"
)
// daemonServerOptions returns the gRPC server options that install peer-identity
// transport credentials on the daemon control channel. Identity extraction is
// only possible over a Unix socket (SO_PEERCRED) or Windows named pipe (client
// token); over TCP, or on platforms without a peer-credential primitive, the
// daemon runs without per-caller authorization and logs a warning.
func daemonServerOptions(network string) []grpc.ServerOption {
creds := ipcauth.NewTransportCredentials()
if creds == nil {
log.Warnf("daemon control channel has no peer-identity primitive on %s; per-caller authorization is disabled", runtime.GOOS)
return nil
}
if network == "tcp" {
log.Warnf("daemon is listening on TCP (%s); peer identity cannot be authenticated over TCP, per-caller authorization is disabled", daemonAddr)
return nil
}
return []grpc.ServerOption{grpc.Creds(creds)}
}
func validateJSONSocketFlags() error {
if serviceCmd.PersistentFlags().Changed("json-socket") && !enableJSONSocket {
return fmt.Errorf("--json-socket requires --enable-json-socket to configure the daemon JSON gateway")
@@ -57,13 +37,8 @@ func (p *program) Start(svc service.Service) error {
// Collect static system and platform information
system.UpdateStaticInfoAsync()
network, _, err := parseListenAddress(daemonAddr)
if err != nil {
return fmt.Errorf("parse daemon address: %w", err)
}
// in any case, even if configuration does not exist we run daemon to serve the CLI gRPC API.
p.serv = grpc.NewServer(daemonServerOptions(network)...)
// in any case, even if configuration does not exists we run daemon to serve CLI gRPC API.
p.serv = grpc.NewServer()
daemonListener, err := listenOnAddress(daemonAddr)
if err != nil {
@@ -87,8 +62,7 @@ func (p *program) Start(svc service.Service) error {
defer jsonListener.Close()
}
serveListener, err := secureDaemonListener(daemonListener)
if err != nil {
if err := daemonListener.chmodUnixSocket("daemon"); err != nil {
log.Error(err)
return
}
@@ -110,7 +84,6 @@ func (p *program) Start(svc service.Service) error {
p.serverInstanceMu.Unlock()
if jsonListener != nil {
log.Warnf("JSON gateway (--enable-json-socket) re-dials the daemon locally as the daemon's own identity and BYPASSES per-caller authorization; restrict access to %s separately", jsonSocket)
if err := p.startJSONGateway(jsonListener, daemonAddr); err != nil {
log.Fatalf("failed to start daemon JSON server: %v", err)
}
@@ -119,7 +92,7 @@ func (p *program) Start(svc service.Service) error {
}
log.Printf("started daemon server: %v", daemonListener.address)
if err := p.serv.Serve(serveListener); err != nil {
if err := p.serv.Serve(daemonListener.Listener); err != nil {
log.Errorf("failed to serve daemon requests: %v", err)
}
}()

View File

@@ -71,14 +71,6 @@ func buildServiceArguments() []string {
args = append(args, "--enable-json-socket", "--json-socket", jsonSocket)
}
if socketOwner != "" {
args = append(args, "--socket-owner", socketOwner)
}
if strictSocketDisabled {
args = append(args, "--disable-strict-socket")
}
return args
}

View File

@@ -32,8 +32,6 @@ type serviceParams struct {
EnableCapture bool `json:"enable_capture,omitempty"`
DisableNetworks bool `json:"disable_networks,omitempty"`
EnableJSONSocket bool `json:"enable_json_socket,omitempty"`
SocketOwner string `json:"socket_owner,omitempty"`
DisableStrictSocket bool `json:"disable_strict_socket,omitempty"`
ServiceEnvVars map[string]string `json:"service_env_vars,omitempty"`
}
@@ -88,8 +86,6 @@ func currentServiceParams() *serviceParams {
EnableCapture: captureEnabled,
DisableNetworks: networksDisabled,
EnableJSONSocket: enableJSONSocket,
SocketOwner: socketOwner,
DisableStrictSocket: strictSocketDisabled,
}
if len(serviceEnvVars) > 0 {
@@ -169,14 +165,6 @@ func applyServiceParams(cmd *cobra.Command, params *serviceParams) {
networksDisabled = params.DisableNetworks
}
if !serviceCmd.PersistentFlags().Changed("socket-owner") {
socketOwner = params.SocketOwner
}
if !serviceCmd.PersistentFlags().Changed("disable-strict-socket") {
strictSocketDisabled = params.DisableStrictSocket
}
applyServiceEnvParams(cmd, params)
}

View File

@@ -1,20 +0,0 @@
//go:build !windows
package cmd
import (
"context"
"fmt"
"net"
"runtime"
)
// listenNamedPipe is unsupported off Windows; named pipes are a Windows-only transport.
func listenNamedPipe(string) (net.Listener, error) {
return nil, fmt.Errorf("named pipe daemon socket is only supported on Windows, not %s", runtime.GOOS)
}
// dialNamedPipe is unsupported off Windows.
func dialNamedPipe(context.Context, string) (net.Conn, error) {
return nil, fmt.Errorf("named pipe daemon socket is only supported on Windows, not %s", runtime.GOOS)
}

View File

@@ -1,32 +0,0 @@
//go:build windows
package cmd
import (
"context"
"net"
"time"
"github.com/Microsoft/go-winio"
"github.com/netbirdio/netbird/client/internal/ipcauth"
)
// listenNamedPipe creates the daemon control named pipe with a tight SDDL
// (SYSTEM + Administrators + interactive users). ListenPipe fails if the pipe
// already exists (first-instance semantics), which prevents a squatting process
// from pre-creating it — we surface that error loudly rather than falling back.
func listenNamedPipe(path string) (net.Listener, error) {
return winio.ListenPipe(path, &winio.PipeConfig{
SecurityDescriptor: ipcauth.DefaultPipeSDDL(),
})
}
// dialNamedPipe connects to the daemon control named pipe.
func dialNamedPipe(ctx context.Context, path string) (net.Conn, error) {
if deadline, ok := ctx.Deadline(); ok {
timeout := time.Until(deadline)
return winio.DialPipe(path, &timeout)
}
return winio.DialPipeContext(ctx, path)
}

View File

@@ -26,15 +26,6 @@ func listenOnAddress(addr string) (*socketListener, error) {
return nil, err
}
if network == "npipe" {
path := pipePath(address)
listener, err := listenNamedPipe(path)
if err != nil {
return nil, err
}
return &socketListener{Listener: listener, network: network, address: path}, nil
}
if network == "unix" {
removeStaleUnixSocket(address)
}
@@ -54,23 +45,13 @@ func parseListenAddress(addr string) (string, string, error) {
}
switch network {
case "unix", "tcp", "npipe":
case "unix", "tcp":
return network, address, nil
default:
return "", "", fmt.Errorf("unsupported daemon address protocol: %v", network)
}
}
// pipePath maps a daemon-addr npipe name (e.g. "netbird" from "npipe://netbird")
// to a Windows named-pipe path (\\.\pipe\netbird). A caller may also pass a full
// \\.\pipe\ path, which is returned unchanged.
func pipePath(name string) string {
if strings.HasPrefix(name, `\\`) {
return name
}
return `\\.\pipe\` + name
}
func removeStaleUnixSocket(path string) {
stat, err := os.Lstat(path)
if err != nil {

View File

@@ -1,11 +0,0 @@
//go:build windows
package cmd
import "net"
// secureDaemonListener is a no-op on Windows: the named-pipe SDDL gates who may
// connect (Layer 1), and the pipe client token supplies per-RPC identity.
func secureDaemonListener(l *socketListener) (net.Listener, error) {
return l.Listener, nil
}

View File

@@ -1,225 +0,0 @@
//go:build !windows && !ios && !android
package cmd
import (
"errors"
"fmt"
"net"
"os"
"os/exec"
"os/user"
"strconv"
"sync"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/ipcauth"
"github.com/netbirdio/netbird/client/internal/shell"
)
// secureDaemonListener applies the Layer-1 access control to the daemon control
// socket and returns the listener to serve on. For a Unix socket this restricts
// the socket to an owner (plus the netbird group); for anything else it is a
// no-op (TCP is legacy/unauthenticated; named pipes are gated by their SDDL).
func secureDaemonListener(l *socketListener) (net.Listener, error) {
if l.network != "unix" {
return l.Listener, nil
}
owner := effectiveSocketOwner()
switch {
case strictSocketDisabled:
// Root-only opt-out (via service.json): leave it world-writable.
if err := os.Chmod(l.address, 0666); err != nil {
return nil, fmt.Errorf("set daemon socket permissions: %w", err)
}
log.Warnf("daemon control socket left world-writable (0666) by --disable-strict-socket")
return l.Listener, nil
case owner != "":
// Seeded owner (flag, MDM, or persisted TOFU result): restrict before
// serving so there is no open window.
uid, err := lookupUser(owner)
if err != nil {
return nil, fmt.Errorf("lookup socket owner %q: %w", owner, err)
}
if err := restrictSocket(l.address, uid); err != nil {
return nil, fmt.Errorf("restrict socket to %q: %w", owner, err)
}
return l.Listener, nil
default:
// Trust-on-first-use: open the socket now; tofuListener locks it to the
// first caller's uid on the first connection.
if err := os.Chmod(l.address, 0666); err != nil {
return nil, fmt.Errorf("set daemon socket permissions: %w", err)
}
return &tofuListener{Listener: l.Listener, path: l.address, owner: -1}, nil
}
}
func lookupUser(username string) (int, error) {
u, err := shell.LookupWithGetent(username)
if err != nil {
return -1, fmt.Errorf("lookup user %s: %w", username, err)
}
uid, err := strconv.Atoi(u.Uid)
if err != nil {
return -1, fmt.Errorf("parse uid %s: %w", u.Uid, err)
}
return uid, nil
}
// addGroup creates a system group if it doesn't already exist and returns the gid.
// Must run as root.
func addGroup(name string) (int, error) {
group, err := shell.LookupGroupWithGetent(name)
if err == nil {
gid, err := strconv.ParseInt(group.Gid, 10, 64)
return int(gid), err
}
groupadd, err := exec.LookPath("groupadd")
if err != nil {
// Fallback for Alpine/BusyBox systems.
if groupadd, err = exec.LookPath("addgroup"); err != nil {
return -1, errors.New("neither groupadd nor addgroup found")
}
}
// Use --system for a service/daemon group (no login, low GID).
out, err := exec.Command(groupadd, "--system", name).CombinedOutput()
if err != nil {
return -1, fmt.Errorf("create group %q: %w: %s", name, err, out)
}
if group, err := shell.LookupGroupWithGetent(name); err == nil {
gid, err := strconv.ParseInt(group.Gid, 10, 64)
return int(gid), err
}
return -1, fmt.Errorf("lookup group %q: %w", name, err)
}
// restrictSocket locks the unix socket down to the owner uid plus the netbird
// group (0660). If the group cannot be created or applied, it fails closed to
// owner-only 0600 — it never leaves the socket world-writable.
func restrictSocket(path string, uid int) error {
gid, err := addGroup("netbird")
if err != nil {
log.Errorf("create netbird group, failing closed to owner-only 0600: %v", err)
return chownChmod(path, uid, -1, 0600)
}
if err := chownChmod(path, uid, gid, 0660); err != nil {
log.Errorf("apply netbird group to socket, failing closed to owner-only 0600: %v", err)
return chownChmod(path, uid, -1, 0600)
}
return nil
}
// chownChmod sets ownership and mode on the socket. A gid of -1 leaves the
// group unchanged.
func chownChmod(path string, uid, gid int, mode os.FileMode) error {
if err := os.Chown(path, uid, gid); err != nil {
return fmt.Errorf("chown socket %s: %w", path, err)
}
if err := os.Chmod(path, mode); err != nil {
return fmt.Errorf("chmod socket %s: %w", path, err)
}
return nil
}
// tofuListener implements trust-on-first-use for the daemon control socket.
// The socket starts world-writable; the first caller's uid (read via SO_PEERCRED)
// becomes the owner. On that first connection the socket is restricted and the
// owner persisted so the open window never reopens on later starts. Connections
// that raced in during the open window and are neither the owner nor root are
// dropped. Changing the socket mode does not disturb the already-open
// connection, so the first caller's request is served normally.
type tofuListener struct {
net.Listener
path string
mu sync.Mutex
owner int // -1 until claimed
}
func (l *tofuListener) Accept() (net.Conn, error) {
for {
c, err := l.Listener.Accept()
if err != nil {
return nil, err
}
id, err := ipcauth.PeerIdentity(c)
if err != nil {
log.Errorf("read peer credentials, dropping connection: %v", err)
_ = c.Close()
continue
}
uid := int(id.UID)
l.mu.Lock()
if l.owner == -1 {
if err := restrictSocket(l.path, uid); err != nil {
l.mu.Unlock()
_ = c.Close()
// Refuse to serve on a socket we could not lock down.
return nil, fmt.Errorf("restrict socket on first connection: %w", err)
}
l.owner = uid
persistSocketOwner(uid)
log.Infof("control socket restricted to first caller (uid %d)", uid)
l.mu.Unlock()
return c, nil
}
owner := l.owner
l.mu.Unlock()
// New connects are already gated by the 0660 perms set above; this only
// drops anything that slipped in during the brief open window.
if uid != owner && uid != 0 {
log.Warnf("dropping non-owner connection (uid %d) during socket bootstrap", uid)
_ = c.Close()
continue
}
return c, nil
}
}
// effectiveSocketOwner returns the configured socket owner: the --socket-owner
// flag when set, otherwise the owner persisted by a previous TOFU migration.
func effectiveSocketOwner() string {
if socketOwner != "" {
return socketOwner
}
params, err := loadServiceParams()
if err != nil {
log.Errorf("load service params for socket owner: %v", err)
return ""
}
if params != nil {
return params.SocketOwner
}
return ""
}
// persistSocketOwner records the TOFU-selected owner (by username) so the next
// daemon start restricts the socket immediately, with no open window.
func persistSocketOwner(uid int) {
u, err := user.LookupId(strconv.Itoa(uid))
if err != nil {
log.Errorf("resolve uid %d to username for persistence: %v", uid, err)
return
}
params, err := loadServiceParams()
if err != nil {
log.Errorf("load service params to persist socket owner: %v", err)
return
}
if params == nil {
params = currentServiceParams()
}
params.SocketOwner = u.Username
if err := saveServiceParams(params); err != nil {
log.Errorf("persist socket owner: %v", err)
}
}

View File

@@ -1,13 +0,0 @@
//go:build !linux && !darwin && !freebsd && !windows
package ipcauth
import "google.golang.org/grpc/credentials"
// NewTransportCredentials returns nil on platforms without a peer-identity
// primitive. The daemon falls back to insecure credentials and skips per-RPC
// authorization (logging a warning), preserving pre-hardening behavior until
// the transport gains an identity primitive.
func NewTransportCredentials() credentials.TransportCredentials {
return nil
}

View File

@@ -1,48 +0,0 @@
//go:build linux || darwin || freebsd
package ipcauth
import (
"context"
"net"
"google.golang.org/grpc/credentials"
)
// NewTransportCredentials returns gRPC transport credentials that extract the
// caller's kernel-authenticated identity from a Unix-socket connection and
// expose it via IdentityFromContext. It is non-nil on platforms with a
// peer-credential primitive.
func NewTransportCredentials() credentials.TransportCredentials {
return unixCreds{}
}
// unixCreds implements credentials.TransportCredentials over a Unix socket.
// The server side reads SO_PEERCRED/LOCAL_PEERCRED during the handshake; the
// client side is a no-op (the kernel supplies the peer identity to the server
// without any client cooperation).
type unixCreds struct{}
func (unixCreds) ClientHandshake(_ context.Context, _ string, conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
return conn, AuthInfo{}, nil
}
// ServerHandshake extracts the peer identity and fails closed if it cannot be read.
func (unixCreds) ServerHandshake(conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
id, err := PeerIdentity(conn)
if err != nil {
return nil, nil, err
}
return conn, AuthInfo{
CommonAuthInfo: credentials.CommonAuthInfo{SecurityLevel: credentials.NoSecurity},
Identity: id,
}, nil
}
func (unixCreds) Info() credentials.ProtocolInfo {
return credentials.ProtocolInfo{SecurityProtocol: "netbird-ipc-peercred"}
}
func (unixCreds) Clone() credentials.TransportCredentials { return unixCreds{} }
func (unixCreds) OverrideServerName(string) error { return nil }

View File

@@ -1,142 +0,0 @@
//go:build windows
package ipcauth
import (
"context"
"fmt"
"net"
"runtime"
"golang.org/x/sys/windows"
"google.golang.org/grpc/credentials"
)
var (
modadvapi32 = windows.NewLazySystemDLL("advapi32.dll")
procImpersonateNamedPipeClient = modadvapi32.NewProc("ImpersonateNamedPipeClient")
procRevertToSelf = modadvapi32.NewProc("RevertToSelf")
)
// Windows group-SID attribute flags (winnt.h): a group only counts toward
// membership when it is enabled and not marked use-for-deny-only.
const (
seGroupEnabled = 0x00000004
seGroupUseForDenyOnly = 0x00000010
)
// DefaultPipeSDDL restricts the daemon control pipe to LocalSystem (SY), the
// Administrators group (BA), and interactive logon users (IU). It deliberately
// excludes Authenticated Users / Everyone so remote or arbitrary service
// principals cannot connect. This is the Layer-1 channel gate; the interceptor
// (Layer 2) further restricts by per-profile ownership.
func DefaultPipeSDDL() string {
return "D:P(A;;GA;;;SY)(A;;GA;;;BA)(A;;GA;;;IU)"
}
// NewTransportCredentials returns gRPC transport credentials that derive the
// caller's identity from the named-pipe client token, following Microsoft's
// "Verifying Client Access with ACLs" pattern: ImpersonateNamedPipeClient ->
// OpenThreadToken -> RevertToSelf. Per threat-model M-NOIMP, impersonation is
// used only to read the client token for identity, never to perform privileged work.
func NewTransportCredentials() credentials.TransportCredentials {
return winpipeCreds{}
}
type winpipeCreds struct{}
func (winpipeCreds) ClientHandshake(_ context.Context, _ string, conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
return conn, AuthInfo{}, nil
}
// ServerHandshake extracts the connecting client's identity from the pipe token.
// Fails closed if the handle or token cannot be read.
func (winpipeCreds) ServerHandshake(conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
// go-winio's pipe connection embeds *win32File, which exposes Fd().
fdConn, ok := conn.(interface{ Fd() uintptr })
if !ok {
return nil, nil, fmt.Errorf("connection %T does not expose a pipe handle", conn)
}
handle := windows.Handle(fdConn.Fd())
id, err := pipeClientIdentity(handle)
if err != nil {
return nil, nil, err
}
return conn, AuthInfo{
CommonAuthInfo: credentials.CommonAuthInfo{SecurityLevel: credentials.NoSecurity},
Identity: id,
}, nil
}
func (winpipeCreds) Info() credentials.ProtocolInfo {
return credentials.ProtocolInfo{SecurityProtocol: "netbird-ipc-peercred"}
}
func (winpipeCreds) Clone() credentials.TransportCredentials { return winpipeCreds{} }
func (winpipeCreds) OverrideServerName(string) error { return nil }
// pipeClientIdentity reads the connecting client's user SID and enabled group
// SIDs from the named-pipe handle. The impersonation window is kept as small as
// possible and pinned to the OS thread (impersonation is thread-local).
func pipeClientIdentity(handle windows.Handle) (Identity, error) {
var pid uint32
hasPID := windows.GetNamedPipeClientProcessId(handle, &pid) == nil
runtime.LockOSThread()
defer runtime.UnlockOSThread()
if err := impersonateNamedPipeClient(handle); err != nil {
return Identity{}, fmt.Errorf("impersonate named pipe client: %w", err)
}
defer func() { _ = revertToSelf() }()
// openAsSelf=true: the token is opened using the daemon's process context
// (LocalSystem), not the impersonated client's, so the open always succeeds.
var token windows.Token
if err := windows.OpenThreadToken(windows.CurrentThread(), windows.TOKEN_QUERY, true, &token); err != nil {
return Identity{}, fmt.Errorf("open thread token: %w", err)
}
defer token.Close()
tu, err := token.GetTokenUser()
if err != nil {
return Identity{}, fmt.Errorf("get token user: %w", err)
}
tg, err := token.GetTokenGroups()
if err != nil {
return Identity{}, fmt.Errorf("get token groups: %w", err)
}
var groups []string
for _, g := range tg.AllGroups() {
if g.Attributes&seGroupEnabled == 0 || g.Attributes&seGroupUseForDenyOnly != 0 {
continue
}
groups = append(groups, g.Sid.String())
}
return Identity{
SID: tu.User.Sid.String(),
Groups: groups,
PID: int32(pid),
HasPID: hasPID,
}, nil
}
func impersonateNamedPipeClient(h windows.Handle) error {
r, _, e := procImpersonateNamedPipeClient.Call(uintptr(h))
if r == 0 {
return e
}
return nil
}
func revertToSelf() error {
r, _, e := procRevertToSelf.Call()
if r == 0 {
return e
}
return nil
}

View File

@@ -1,93 +0,0 @@
// Package ipcauth provides kernel-authenticated caller identity for the daemon's
// local IPC (gRPC) channel and the transport credentials that populate it.
//
// It is the identity foundation shared by two layers of the local-IPC hardening:
// - the socket-permission layer (Layer 1, client/cmd), which reads the peer
// identity to gate who may connect and to run trust-on-first-use; and
// - the per-RPC authorization interceptor (Layer 2), which reads the same
// identity from the gRPC context to enforce per-profile ownership.
//
// On Unix the identity is read from the kernel via SO_PEERCRED (Linux) or
// LOCAL_PEERCRED (Darwin/FreeBSD). On Windows it is derived from the named-pipe
// client token. Platforms without a peer-identity primitive get no credentials
// and therefore no enforcement (the daemon logs a warning and stays open,
// preserving today's behavior until the transport is hardened).
package ipcauth
import (
"context"
"fmt"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/peer"
)
// Identity is the kernel-authenticated identity of a local IPC caller.
//
// The zero value is not a valid identity; callers obtain one via
// IdentityFromContext (which reports presence) or PeerIdentity.
type Identity struct {
// UID and GID are the caller's Unix user ID and primary group ID.
// Zero on Windows, where SID is authoritative instead.
UID uint32
GID uint32
// PID is the caller's process ID, for audit only. HasPID is false when the
// platform cannot supply it (e.g. Darwin/FreeBSD xucred carries no PID).
PID int32
HasPID bool
// SID is the caller's Windows security identifier (empty on Unix).
SID string
// Groups holds the caller's Windows group SIDs, captured from the client
// token at handshake time (empty on Unix, where supplementary group
// membership is resolved on demand via NSS/getent by the authorizer).
Groups []string
}
// IsWindows reports whether this identity is a Windows principal (SID-based)
// rather than a Unix uid/gid principal.
func (i Identity) IsWindows() bool {
return i.SID != ""
}
// String renders the identity for audit logs.
func (i Identity) String() string {
if i.IsWindows() {
if i.HasPID {
return fmt.Sprintf("sid=%s pid=%d", i.SID, i.PID)
}
return fmt.Sprintf("sid=%s", i.SID)
}
if i.HasPID {
return fmt.Sprintf("uid=%d gid=%d pid=%d", i.UID, i.GID, i.PID)
}
return fmt.Sprintf("uid=%d gid=%d", i.UID, i.GID)
}
// AuthInfo carries the peer Identity as a gRPC credentials.AuthInfo so the
// interceptor can retrieve it from the request context via IdentityFromContext.
type AuthInfo struct {
credentials.CommonAuthInfo
Identity Identity
}
// AuthType identifies the authentication scheme.
func (AuthInfo) AuthType() string { return "netbird-ipc-peercred" }
// IdentityFromContext extracts the caller's kernel-authenticated identity from
// the gRPC peer context. The second return value is false when no IPC transport
// credentials were negotiated (e.g. an unsupported platform, or a caller that
// did not come through the daemon socket) — callers MUST fail closed in that case.
func IdentityFromContext(ctx context.Context) (Identity, bool) {
p, ok := peer.FromContext(ctx)
if !ok {
return Identity{}, false
}
info, ok := p.AuthInfo.(AuthInfo)
if !ok {
return Identity{}, false
}
return info.Identity, true
}

View File

@@ -1,47 +0,0 @@
package ipcauth
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/peer"
)
func TestIdentityFromContext_NoPeer(t *testing.T) {
_, ok := IdentityFromContext(context.Background())
assert.False(t, ok, "bare context must report no identity (fail closed)")
}
func TestIdentityFromContext_WrongAuthInfo(t *testing.T) {
ctx := peer.NewContext(context.Background(), &peer.Peer{})
_, ok := IdentityFromContext(ctx)
assert.False(t, ok, "peer without our AuthInfo must report no identity")
}
func TestIdentityFromContext_Present(t *testing.T) {
want := Identity{UID: 1000, GID: 1000, PID: 4242, HasPID: true}
ctx := peer.NewContext(context.Background(), &peer.Peer{
AuthInfo: AuthInfo{
CommonAuthInfo: credentials.CommonAuthInfo{SecurityLevel: credentials.NoSecurity},
Identity: want,
},
})
got, ok := IdentityFromContext(ctx)
assert.True(t, ok)
assert.Equal(t, want, got)
}
func TestIdentity_String(t *testing.T) {
assert.Equal(t, "uid=1000 gid=1000 pid=42", Identity{UID: 1000, GID: 1000, PID: 42, HasPID: true}.String())
assert.Equal(t, "uid=1000 gid=1000", Identity{UID: 1000, GID: 1000}.String())
assert.Equal(t, "sid=S-1-5-21-1 pid=42", Identity{SID: "S-1-5-21-1", PID: 42, HasPID: true}.String())
assert.Equal(t, "sid=S-1-5-21-1", Identity{SID: "S-1-5-21-1"}.String())
}
func TestIdentity_IsWindows(t *testing.T) {
assert.True(t, Identity{SID: "S-1-5-18"}.IsWindows())
assert.False(t, Identity{UID: 0}.IsWindows())
}

View File

@@ -1,43 +0,0 @@
//go:build darwin || freebsd
package ipcauth
import (
"fmt"
"net"
"golang.org/x/sys/unix"
)
// PeerIdentity reads the kernel-authenticated identity of the process on the
// other end of a Unix socket connection via LOCAL_PEERCRED (xucred). xucred
// carries the uid and group list but no pid, so audit on these platforms is
// uid/gid-based (HasPID is false); PID via LOCAL_PEERPID is a possible follow-up.
func PeerIdentity(c net.Conn) (Identity, error) {
uc, ok := c.(*net.UnixConn)
if !ok {
return Identity{}, fmt.Errorf("connection is not a unix socket: %T", c)
}
raw, err := uc.SyscallConn()
if err != nil {
return Identity{}, fmt.Errorf("raw conn: %w", err)
}
var cred *unix.Xucred
var credErr error
if err := raw.Control(func(fd uintptr) {
cred, credErr = unix.GetsockoptXucred(int(fd), unix.SOL_LOCAL, unix.LOCAL_PEERCRED)
}); err != nil {
return Identity{}, fmt.Errorf("getsockopt control: %w", err)
}
if credErr != nil {
return Identity{}, fmt.Errorf("LOCAL_PEERCRED: %w", credErr)
}
id := Identity{UID: cred.Uid}
// Groups[0] is the effective (primary) GID; guard against an empty list.
if cred.Ngroups > 0 {
id.GID = cred.Groups[0]
}
return id, nil
}

View File

@@ -1,43 +0,0 @@
//go:build linux
package ipcauth
import (
"fmt"
"net"
"golang.org/x/sys/unix"
)
// PeerIdentity reads the kernel-authenticated identity of the process on the
// other end of a Unix socket connection via SO_PEERCRED. The credentials are
// captured by the kernel at connect() time and cannot be spoofed or changed for
// the life of the connection.
func PeerIdentity(c net.Conn) (Identity, error) {
uc, ok := c.(*net.UnixConn)
if !ok {
return Identity{}, fmt.Errorf("connection is not a unix socket: %T", c)
}
raw, err := uc.SyscallConn()
if err != nil {
return Identity{}, fmt.Errorf("raw conn: %w", err)
}
var cred *unix.Ucred
var credErr error
if err := raw.Control(func(fd uintptr) {
cred, credErr = unix.GetsockoptUcred(int(fd), unix.SOL_SOCKET, unix.SO_PEERCRED)
}); err != nil {
return Identity{}, fmt.Errorf("getsockopt control: %w", err)
}
if credErr != nil {
return Identity{}, fmt.Errorf("SO_PEERCRED: %w", credErr)
}
return Identity{
UID: cred.Uid,
GID: cred.Gid,
PID: cred.Pid,
HasPID: true,
}, nil
}

View File

@@ -1,16 +0,0 @@
//go:build !linux && !darwin && !freebsd
package ipcauth
import (
"fmt"
"net"
"runtime"
)
// PeerIdentity is unimplemented on platforms without a Unix-socket peer-credential
// primitive. Windows derives identity from the named-pipe client token instead
// (see the Windows transport credentials), so it never calls this.
func PeerIdentity(net.Conn) (Identity, error) {
return Identity{}, fmt.Errorf("peer credential check not supported on %s", runtime.GOOS)
}

View File

@@ -1,113 +0,0 @@
//go:build linux || darwin || freebsd
package ipcauth
import (
"net"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestPeerIdentity_MatchesCurrentProcess connects to a real Unix socket and
// verifies the extracted UID/GID match the running process (both ends are us).
func TestPeerIdentity_MatchesCurrentProcess(t *testing.T) {
sock := filepath.Join(t.TempDir(), "peer.sock")
ln, err := net.Listen("unix", sock)
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
type result struct {
id Identity
err error
}
done := make(chan result, 1)
go func() {
c, aerr := ln.Accept()
if aerr != nil {
done <- result{err: aerr}
return
}
defer func() { _ = c.Close() }()
id, ierr := PeerIdentity(c)
done <- result{id: id, err: ierr}
}()
client, err := net.Dial("unix", sock)
require.NoError(t, err)
t.Cleanup(func() { _ = client.Close() })
res := <-done
require.NoError(t, res.err)
assert.Equal(t, uint32(os.Getuid()), res.id.UID, "UID should match current process")
assert.Equal(t, uint32(os.Getgid()), res.id.GID, "primary GID should match current process")
}
// TestPeerIdentity_NonUnixConn rejects non-Unix connections (fail closed).
func TestPeerIdentity_NonUnixConn(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
done := make(chan error, 1)
go func() {
c, aerr := ln.Accept()
if aerr != nil {
done <- aerr
return
}
defer func() { _ = c.Close() }()
_, ierr := PeerIdentity(c)
done <- ierr
}()
client, err := net.Dial("tcp", ln.Addr().String())
require.NoError(t, err)
t.Cleanup(func() { _ = client.Close() })
assert.Error(t, <-done, "PeerIdentity must reject a non-Unix connection")
}
// TestUnixCreds_ServerHandshake exercises the transport-credentials path end to end.
func TestUnixCreds_ServerHandshake(t *testing.T) {
creds := NewTransportCredentials()
require.NotNil(t, creds)
sock := filepath.Join(t.TempDir(), "hs.sock")
ln, err := net.Listen("unix", sock)
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
type result struct {
info interface{ AuthType() string }
err error
}
done := make(chan result, 1)
go func() {
c, aerr := ln.Accept()
if aerr != nil {
done <- result{err: aerr}
return
}
_, ai, herr := creds.ServerHandshake(c)
if herr != nil {
done <- result{err: herr}
return
}
done <- result{info: ai}
}()
client, err := net.Dial("unix", sock)
require.NoError(t, err)
t.Cleanup(func() { _ = client.Close() })
res := <-done
require.NoError(t, res.err)
ai, ok := res.info.(AuthInfo)
require.True(t, ok, "expected ipcauth.AuthInfo, got %T", res.info)
assert.Equal(t, uint32(os.Getuid()), ai.Identity.UID)
assert.Equal(t, "netbird-ipc-peercred", ai.AuthType())
}

View File

@@ -54,19 +54,15 @@ func (w *WorkerRelay) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
w.relaySupportedOnRemotePeer.Store(true)
// the relayManager will return with error in case if the connection has lost with relay server
currentRelayAddress, _, err := w.relayManager.RelayInstanceAddress()
_, _, err := w.relayManager.RelayInstanceAddress()
if err != nil {
w.log.Errorf("failed to handle new offer: %s", err)
return
}
srv := w.preferredRelayServer(currentRelayAddress, remoteOfferAnswer.RelaySrvAddress)
var serverIP netip.Addr
if srv == remoteOfferAnswer.RelaySrvAddress {
serverIP = remoteOfferAnswer.RelaySrvIP
}
relayedConn, err := w.relayManager.OpenConn(w.peerCtx, srv, w.config.Key, serverIP)
preferForeign := !w.isController
remoteRelayServer := relayClient.RelayServer{Addr: remoteOfferAnswer.RelaySrvAddress, IP: remoteOfferAnswer.RelaySrvIP}
relayedConn, err := w.relayManager.OpenConn(w.peerCtx, remoteRelayServer, w.config.Key, preferForeign)
if err != nil {
if errors.Is(err, relayClient.ErrConnAlreadyExists) {
w.log.Debugf("handled offer by reusing existing relay connection")
@@ -80,14 +76,13 @@ func (w *WorkerRelay) OnNewOffer(remoteOfferAnswer *OfferAnswer) {
w.relayedConn = relayedConn
w.relayLock.Unlock()
err = w.relayManager.AddCloseListener(srv, w.onRelayClientDisconnected)
if err != nil {
log.Errorf("failed to add close listener: %s", err)
if err := w.relayManager.AddCloseListener(relayedConn.RemoteAddr().String(), w.onRelayClientDisconnected); err != nil {
w.log.Errorf("failed to add close listener: %s", err)
_ = relayedConn.Close()
return
}
w.log.Debugf("peer conn opened via Relay: %s", srv)
w.log.Debugf("peer conn opened via Relay: %s", relayedConn.RemoteAddr())
go w.conn.onRelayConnectionIsReady(RelayConnInfo{
relayedConn: relayedConn,
rosenpassPubKey: remoteOfferAnswer.RosenpassPubKey,
@@ -126,13 +121,6 @@ func (w *WorkerRelay) isRelaySupported(answer *OfferAnswer) bool {
return answer.RelaySrvAddress != ""
}
func (w *WorkerRelay) preferredRelayServer(myRelayAddress, remoteRelayAddress string) string {
if w.isController {
return myRelayAddress
}
return remoteRelayAddress
}
func (w *WorkerRelay) onRelayClientDisconnected() {
go w.conn.onRelayDisconnected()
}

View File

@@ -1,29 +0,0 @@
//go:build cgo && !osusergo && !windows
package shell
import "os/user"
// LookupWithGetent with CGO delegates directly to os/user.Lookup.
// When CGO is enabled, os/user uses libc (getpwnam_r) which goes through
// the NSS stack natively. If it fails, the user truly doesn't exist and
// getent would also fail.
func LookupWithGetent(username string) (*user.User, error) {
return user.Lookup(username)
}
// CurrentUserWithGetent with CGO delegates directly to os/user.Current.
func CurrentUserWithGetent() (*user.User, error) {
return user.Current()
}
// LookupGroupWithGetent returns the resolved group from either a gid or groupname.
func LookupGroupWithGetent(name string) (*user.Group, error) {
return user.LookupGroup(name)
}
// GroupIdsWithFallback with CGO delegates directly to user.GroupIds.
// libc's getgrouplist handles NSS groups natively.
func GroupIdsWithFallback(u *user.User) ([]string, error) {
return u.GroupIds()
}

View File

@@ -1,31 +0,0 @@
//go:build windows
package shell
import "os/user"
// LookupWithGetent on Windows just delegates to os/user.Lookup.
// Windows does not use NSS/getent; its user lookup works without CGO.
func LookupWithGetent(username string) (*user.User, error) {
return user.Lookup(username)
}
// CurrentUserWithGetent on Windows just delegates to os/user.Current.
func CurrentUserWithGetent() (*user.User, error) {
return user.Current()
}
// LookupGroupWithGetent on Windows just delegates to os/user.LookupGroup.
func LookupGroupWithGetent(name string) (*user.Group, error) {
return user.LookupGroup(name)
}
// GetShellFromGetent is a no-op on Windows; shell resolution uses PowerShell detection.
func GetShellFromGetent(_ string) string {
return ""
}
// GroupIdsWithFallback on Windows just delegates to u.GroupIds().
func GroupIdsWithFallback(u *user.User) ([]string, error) {
return u.GroupIds()
}

View File

@@ -233,6 +233,9 @@ func (c *Client) DebugBundle(anonymize bool) (string, error) {
deps.SyncResponse = resp
if e := cc.Engine(); e != nil {
deps.RefreshStatus = func() {
e.RunHealthProbes(context.Background(), true)
}
if cm := e.GetClientMetrics(); cm != nil {
deps.ClientMetrics = cm
}

View File

@@ -20,8 +20,6 @@ import (
"github.com/creack/pty"
"github.com/gliderlabs/ssh"
log "github.com/sirupsen/logrus"
shellutil "github.com/netbirdio/netbird/client/internal/shell"
)
// ptyManager manages Pty file operations with thread safety
@@ -148,10 +146,10 @@ func (s *Server) createShellCommand(ctx context.Context, shell string, args []st
// prepareCommandEnv prepares environment variables for command execution on Unix
func (s *Server) prepareCommandEnv(_ *log.Entry, localUser *user.User, session ssh.Session) []string {
env := shellutil.PrepareUserEnv(localUser, shellutil.GetUserShell(localUser.Uid))
env := prepareUserEnv(localUser, getUserShell(localUser.Uid))
env = append(env, prepareSSHEnv(session)...)
for _, v := range session.Environ() {
if shellutil.AcceptEnv(v) {
if acceptEnv(v) {
env = append(env, v)
}
}

View File

@@ -15,7 +15,6 @@ import (
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
shellutil "github.com/netbirdio/netbird/client/internal/shell"
"github.com/netbirdio/netbird/client/ssh/server/winpty"
)
@@ -248,10 +247,10 @@ func (s *Server) prepareCommandEnv(logger *log.Entry, localUser *user.User, sess
userEnv, err := s.getUserEnvironment(logger, username, domain)
if err != nil {
log.Debugf("failed to get user environment for %s\\%s, using fallback: %v", domain, username, err)
env := shellutil.PrepareUserEnv(localUser, shellutil.GetUserShell(localUser.Uid))
env := prepareUserEnv(localUser, getUserShell(localUser.Uid))
env = append(env, prepareSSHEnv(session)...)
for _, v := range session.Environ() {
if shellutil.AcceptEnv(v) {
if acceptEnv(v) {
env = append(env, v)
}
}
@@ -261,7 +260,7 @@ func (s *Server) prepareCommandEnv(logger *log.Entry, localUser *user.User, sess
env := userEnv
env = append(env, prepareSSHEnv(session)...)
for _, v := range session.Environ() {
if shellutil.AcceptEnv(v) {
if acceptEnv(v) {
env = append(env, v)
}
}
@@ -274,7 +273,7 @@ func (s *Server) handlePtyLogin(logger *log.Entry, session ssh.Session, privileg
return false
}
shell := shellutil.GetUserShell(privilegeResult.User.Uid)
shell := getUserShell(privilegeResult.User.Uid)
logger.Infof("starting interactive shell: %s", shell)
s.executeCommandWithPty(logger, session, nil, privilegeResult, ptyReq, nil)
@@ -385,7 +384,7 @@ func (s *Server) executeCommandWithPty(logger *log.Entry, session ssh.Session, _
}
username, domain := s.parseUsername(localUser.Username)
shell := shellutil.GetUserShell(localUser.Uid)
shell := getUserShell(localUser.Uid)
req := PtyExecutionRequest{
Shell: shell,

View File

@@ -0,0 +1,24 @@
//go:build cgo && !osusergo && !windows
package server
import "os/user"
// lookupWithGetent with CGO delegates directly to os/user.Lookup.
// When CGO is enabled, os/user uses libc (getpwnam_r) which goes through
// the NSS stack natively. If it fails, the user truly doesn't exist and
// getent would also fail.
func lookupWithGetent(username string) (*user.User, error) {
return user.Lookup(username)
}
// currentUserWithGetent with CGO delegates directly to os/user.Current.
func currentUserWithGetent() (*user.User, error) {
return user.Current()
}
// groupIdsWithFallback with CGO delegates directly to user.GroupIds.
// libc's getgrouplist handles NSS groups natively.
func groupIdsWithFallback(u *user.User) ([]string, error) {
return u.GroupIds()
}

View File

@@ -1,6 +1,6 @@
//go:build (!cgo || osusergo) && !windows
package shell
package server
import (
"os"
@@ -10,10 +10,10 @@ import (
log "github.com/sirupsen/logrus"
)
// LookupWithGetent looks up a user by name, falling back to getent if os/user fails.
// lookupWithGetent looks up a user by name, falling back to getent if os/user fails.
// Without CGO, os/user only reads /etc/passwd and misses NSS-provided users.
// getent goes through the host's NSS stack.
func LookupWithGetent(username string) (*user.User, error) {
func lookupWithGetent(username string) (*user.User, error) {
u, err := user.Lookup(username)
if err == nil {
return u, nil
@@ -22,7 +22,7 @@ func LookupWithGetent(username string) (*user.User, error) {
stdErr := err
log.Debugf("os/user.Lookup(%q) failed, trying getent: %v", username, err)
u, _, getentErr := runGetentPasswd(username)
u, _, getentErr := runGetent(username)
if getentErr != nil {
log.Debugf("getent fallback for %q also failed: %v", username, getentErr)
return nil, stdErr
@@ -31,26 +31,8 @@ func LookupWithGetent(username string) (*user.User, error) {
return u, nil
}
// LookupGroupWithGetent returns the resolved group from either a gid or groupname,
// falling back to getent if os/user fails (NSS groups under nocgo).
func LookupGroupWithGetent(name string) (*user.Group, error) {
g, err := user.LookupGroup(name)
if err == nil {
return g, nil
}
stdErr := err
log.Debugf("os/user.LookupGroup(%q) failed, trying getent: %v", name, err)
g, getentErr := runGetentGroup(name)
if getentErr != nil {
log.Debugf("getent fallback for %q also failed: %v", name, getentErr)
return nil, stdErr
}
return g, nil
}
// CurrentUserWithGetent gets the current user, falling back to getent if os/user fails.
func CurrentUserWithGetent() (*user.User, error) {
// currentUserWithGetent gets the current user, falling back to getent if os/user fails.
func currentUserWithGetent() (*user.User, error) {
u, err := user.Current()
if err == nil {
return u, nil
@@ -60,7 +42,7 @@ func CurrentUserWithGetent() (*user.User, error) {
uid := strconv.Itoa(os.Getuid())
log.Debugf("os/user.Current() failed, trying getent with UID %s: %v", uid, err)
u, _, getentErr := runGetentPasswd(uid)
u, _, getentErr := runGetent(uid)
if getentErr != nil {
return nil, stdErr
}
@@ -68,14 +50,14 @@ func CurrentUserWithGetent() (*user.User, error) {
return u, nil
}
// GroupIdsWithFallback gets group IDs for a user via the id command first,
// groupIdsWithFallback gets group IDs for a user via the id command first,
// falling back to user.GroupIds().
// NOTE: unlike LookupWithGetent/CurrentUserWithGetent which try stdlib first,
// NOTE: unlike lookupWithGetent/currentUserWithGetent which try stdlib first,
// this intentionally tries `id -G` first because without CGO, user.GroupIds()
// only reads /etc/group and silently returns incomplete results for NSS users
// (no error, just missing groups). The id command goes through NSS and returns
// the full set.
func GroupIdsWithFallback(u *user.User) ([]string, error) {
func groupIdsWithFallback(u *user.User) ([]string, error) {
ids, err := runIdGroups(u.Username)
if err == nil {
return ids, nil

View File

@@ -1,4 +1,4 @@
package shell
package server
import (
"os/user"
@@ -15,7 +15,7 @@ func TestLookupWithGetent_CurrentUser(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
u, err := LookupWithGetent(current.Username)
u, err := lookupWithGetent(current.Username)
require.NoError(t, err)
assert.Equal(t, current.Username, u.Username)
assert.Equal(t, current.Uid, u.Uid)
@@ -23,7 +23,7 @@ func TestLookupWithGetent_CurrentUser(t *testing.T) {
}
func TestLookupWithGetent_NonexistentUser(t *testing.T) {
_, err := LookupWithGetent("nonexistent_user_xyzzy_12345")
_, err := lookupWithGetent("nonexistent_user_xyzzy_12345")
require.Error(t, err, "should fail for nonexistent user")
}
@@ -31,7 +31,7 @@ func TestCurrentUserWithGetent(t *testing.T) {
stdUser, err := user.Current()
require.NoError(t, err)
u, err := CurrentUserWithGetent()
u, err := currentUserWithGetent()
require.NoError(t, err)
assert.Equal(t, stdUser.Uid, u.Uid)
assert.Equal(t, stdUser.Username, u.Username)
@@ -41,7 +41,7 @@ func TestGroupIdsWithFallback_CurrentUser(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
groups, err := GroupIdsWithFallback(current)
groups, err := groupIdsWithFallback(current)
require.NoError(t, err)
require.NotEmpty(t, groups, "current user should have at least one group")
@@ -56,7 +56,7 @@ func TestGroupIdsWithFallback_CurrentUser(t *testing.T) {
func TestGetShellFromGetent_CurrentUser(t *testing.T) {
if runtime.GOOS == "windows" {
// Windows stub always returns empty, which is correct
shell := GetShellFromGetent("1000")
shell := getShellFromGetent("1000")
assert.Empty(t, shell, "Windows stub should return empty")
return
}
@@ -65,9 +65,9 @@ func TestGetShellFromGetent_CurrentUser(t *testing.T) {
require.NoError(t, err)
// getent may not be available on all systems (e.g., macOS without Homebrew getent)
shell := GetShellFromGetent(current.Uid)
shell := getShellFromGetent(current.Uid)
if shell == "" {
t.Log("GetShellFromGetent returned empty, getent may not be available")
t.Log("getShellFromGetent returned empty, getent may not be available")
return
}
assert.True(t, shell[0] == '/', "shell should be an absolute path, got %q", shell)
@@ -78,7 +78,7 @@ func TestLookupWithGetent_RootUser(t *testing.T) {
t.Skip("no root user on Windows")
}
u, err := LookupWithGetent("root")
u, err := lookupWithGetent("root")
if err != nil {
t.Skip("root user not available on this system")
}
@@ -86,25 +86,25 @@ func TestLookupWithGetent_RootUser(t *testing.T) {
}
// TestIntegration_FullLookupChain exercises the complete user lookup chain
// against the real system, testing that all wrappers (LookupWithGetent,
// CurrentUserWithGetent, GroupIdsWithFallback, GetShellFromGetent) produce
// against the real system, testing that all wrappers (lookupWithGetent,
// currentUserWithGetent, groupIdsWithFallback, getShellFromGetent) produce
// consistent and correct results when composed together.
func TestIntegration_FullLookupChain(t *testing.T) {
// Step 1: CurrentUserWithGetent must resolve the running user.
current, err := CurrentUserWithGetent()
require.NoError(t, err, "CurrentUserWithGetent must resolve the running user")
// Step 1: currentUserWithGetent must resolve the running user.
current, err := currentUserWithGetent()
require.NoError(t, err, "currentUserWithGetent must resolve the running user")
require.NotEmpty(t, current.Uid)
require.NotEmpty(t, current.Username)
// Step 2: LookupWithGetent by the same username must return matching identity.
byName, err := LookupWithGetent(current.Username)
// Step 2: lookupWithGetent by the same username must return matching identity.
byName, err := lookupWithGetent(current.Username)
require.NoError(t, err)
assert.Equal(t, current.Uid, byName.Uid, "lookup by name should return same UID")
assert.Equal(t, current.Gid, byName.Gid, "lookup by name should return same GID")
assert.Equal(t, current.HomeDir, byName.HomeDir, "lookup by name should return same home")
// Step 3: GroupIdsWithFallback must return at least the primary GID.
groups, err := GroupIdsWithFallback(current)
// Step 3: groupIdsWithFallback must return at least the primary GID.
groups, err := groupIdsWithFallback(current)
require.NoError(t, err)
require.NotEmpty(t, groups, "user must have at least one group")
@@ -120,10 +120,10 @@ func TestIntegration_FullLookupChain(t *testing.T) {
}
assert.True(t, foundPrimary, "primary GID %s should appear in supplementary groups", current.Gid)
// Step 4: GetShellFromGetent should either return a valid shell path or empty
// Step 4: getShellFromGetent should either return a valid shell path or empty
// (empty is OK when getent is not available, e.g. macOS without Homebrew getent).
if runtime.GOOS != "windows" {
shell := GetShellFromGetent(current.Uid)
shell := getShellFromGetent(current.Uid)
if shell != "" {
assert.True(t, shell[0] == '/', "shell should be an absolute path, got %q", shell)
}
@@ -131,17 +131,17 @@ func TestIntegration_FullLookupChain(t *testing.T) {
}
// TestIntegration_LookupAndGroupsConsistency verifies that a user resolved via
// LookupWithGetent can have their groups resolved via GroupIdsWithFallback,
// lookupWithGetent can have their groups resolved via groupIdsWithFallback,
// testing the handoff between the two functions as used by the SSH server.
func TestIntegration_LookupAndGroupsConsistency(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
// Simulate the SSH server flow: lookup user, then get their groups.
resolved, err := LookupWithGetent(current.Username)
resolved, err := lookupWithGetent(current.Username)
require.NoError(t, err)
groups, err := GroupIdsWithFallback(resolved)
groups, err := groupIdsWithFallback(resolved)
require.NoError(t, err)
require.NotEmpty(t, groups, "resolved user must have groups")
@@ -156,7 +156,7 @@ func TestIntegration_LookupAndGroupsConsistency(t *testing.T) {
}
// TestIntegration_ShellLookupChain tests the full shell resolution chain
// (getShellFromPasswd -> GetShellFromGetent -> $SHELL -> default) on Unix.
// (getShellFromPasswd -> getShellFromGetent -> $SHELL -> default) on Unix.
func TestIntegration_ShellLookupChain(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Unix shell lookup not applicable on Windows")
@@ -165,8 +165,8 @@ func TestIntegration_ShellLookupChain(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
// GetUserShell is the top-level function used by the SSH server.
shell := GetUserShell(current.Uid)
require.NotEmpty(t, shell, "GetUserShell must always return a shell")
// getUserShell is the top-level function used by the SSH server.
shell := getUserShell(current.Uid)
require.NotEmpty(t, shell, "getUserShell must always return a shell")
assert.True(t, shell[0] == '/', "shell should be an absolute path, got %q", shell)
}

View File

@@ -1,6 +1,6 @@
//go:build !windows
package shell
package server
import (
"context"
@@ -14,26 +14,19 @@ import (
const getentTimeout = 5 * time.Second
// GetShellFromGetent gets a user's login shell via getent by UID.
// getShellFromGetent gets a user's login shell via getent by UID.
// This is needed even with CGO because getShellFromPasswd reads /etc/passwd
// directly and won't find NSS-provided users there.
func GetShellFromGetent(userID string) string {
_, shell, err := runGetentPasswd(userID)
func getShellFromGetent(userID string) string {
_, shell, err := runGetent(userID)
if err != nil {
return ""
}
return shell
}
// GetUserFromGetent returns the resolved user from either a uid or username,
// going through the host's NSS stack.
func GetUserFromGetent(query string) (*user.User, error) {
u, _, err := runGetentPasswd(query)
return u, err
}
// runGetentPasswd executes `getent passwd <query>` and returns the user and login shell.
func runGetentPasswd(query string) (*user.User, string, error) {
// runGetent executes `getent passwd <query>` and returns the user and login shell.
func runGetent(query string) (*user.User, string, error) {
if !validateGetentInput(query) {
return nil, "", fmt.Errorf("invalid getent input: %q", query)
}
@@ -49,24 +42,7 @@ func runGetentPasswd(query string) (*user.User, string, error) {
return parseGetentPasswd(string(out))
}
// runGetentGroup executes `getent group <query>` and returns the group.
func runGetentGroup(query string) (*user.Group, error) {
if !validateGetentInput(query) {
return nil, fmt.Errorf("invalid getent input: %q", query)
}
ctx, cancel := context.WithTimeout(context.Background(), getentTimeout)
defer cancel()
out, err := exec.CommandContext(ctx, "getent", "group", query).Output()
if err != nil {
return nil, fmt.Errorf("getent group %s: %w", query, err)
}
return parseGetentGroup(string(out))
}
// parseGetentPasswd parses getent passwd output: "name:x:uid:gid:gecos:home:shell".
// parseGetentPasswd parses getent passwd output: "name:x:uid:gid:gecos:home:shell"
func parseGetentPasswd(output string) (*user.User, string, error) {
fields := strings.SplitN(strings.TrimSpace(output), ":", 8)
if len(fields) < 6 {
@@ -91,20 +67,6 @@ func parseGetentPasswd(output string) (*user.User, string, error) {
}, shell, nil
}
// parseGetentGroup parses getent group output: "group:x:gid:members".
func parseGetentGroup(output string) (*user.Group, error) {
fields := strings.SplitN(strings.TrimSpace(output), ":", 8)
if len(fields) < 4 {
return nil, fmt.Errorf("unexpected getent output (need 4+ fields): %q", output)
}
if fields[0] == "" || fields[2] == "" {
return nil, fmt.Errorf("missing required fields in getent output: %q", output)
}
return &user.Group{Gid: fields[2], Name: fields[0]}, nil
}
// validateGetentInput checks that the input is safe to pass to getent or id.
// Allows POSIX usernames, numeric UIDs, and common NSS extensions
// (@ for Kerberos, $ for Samba, + for NIS compat). A leading hyphen is

View File

@@ -1,6 +1,6 @@
//go:build !windows
package shell
package server
import (
"os/exec"
@@ -198,7 +198,7 @@ func TestRunGetent_RootUser(t *testing.T) {
t.Skip("getent not available on this system")
}
u, shell, err := runGetentPasswd("root")
u, shell, err := runGetent("root")
require.NoError(t, err)
assert.Equal(t, "root", u.Username)
assert.Equal(t, "0", u.Uid)
@@ -211,7 +211,7 @@ func TestRunGetent_ByUID(t *testing.T) {
t.Skip("getent not available on this system")
}
u, _, err := runGetentPasswd("0")
u, _, err := runGetent("0")
require.NoError(t, err)
assert.Equal(t, "root", u.Username)
assert.Equal(t, "0", u.Uid)
@@ -222,15 +222,15 @@ func TestRunGetent_NonexistentUser(t *testing.T) {
t.Skip("getent not available on this system")
}
_, _, err := runGetentPasswd("nonexistent_user_xyzzy_12345")
_, _, err := runGetent("nonexistent_user_xyzzy_12345")
assert.Error(t, err)
}
func TestRunGetent_InvalidInput(t *testing.T) {
_, _, err := runGetentPasswd("")
_, _, err := runGetent("")
assert.Error(t, err)
_, _, err = runGetentPasswd("user\x00name")
_, _, err = runGetent("user\x00name")
assert.Error(t, err)
}
@@ -239,7 +239,7 @@ func TestRunGetent_NotAvailable(t *testing.T) {
t.Skip("getent is available, can't test missing case")
}
_, _, err := runGetentPasswd("root")
_, _, err := runGetent("root")
assert.Error(t, err, "should fail when getent is not installed")
}
@@ -286,7 +286,7 @@ func TestGetentResultsMatchStdlib(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
getentUser, _, err := runGetentPasswd(current.Username)
getentUser, _, err := runGetent(current.Username)
require.NoError(t, err)
assert.Equal(t, current.Username, getentUser.Username, "username should match")
@@ -303,7 +303,7 @@ func TestGetentResultsMatchStdlib_ByUID(t *testing.T) {
current, err := user.Current()
require.NoError(t, err)
getentUser, _, err := runGetentPasswd(current.Uid)
getentUser, _, err := runGetent(current.Uid)
require.NoError(t, err)
assert.Equal(t, current.Username, getentUser.Username, "username should match when looked up by UID")
@@ -359,7 +359,7 @@ func TestGetShellFromPasswd_CurrentUser(t *testing.T) {
assert.True(t, shell[0] == '/', "shell should be an absolute path, got %q", shell)
if _, err := exec.LookPath("getent"); err == nil {
_, getentShell, getentErr := runGetentPasswd(current.Uid)
_, getentShell, getentErr := runGetent(current.Uid)
if getentErr == nil && getentShell != "" {
assert.Equal(t, getentShell, shell, "shell from /etc/passwd should match getent")
}
@@ -403,7 +403,7 @@ func TestGetShellFromPasswd_MatchesGetentForKnownUsers(t *testing.T) {
continue
}
_, getentShell, err := runGetentPasswd(uid)
_, getentShell, err := runGetent(uid)
if err != nil {
continue
}

View File

@@ -0,0 +1,26 @@
//go:build windows
package server
import "os/user"
// lookupWithGetent on Windows just delegates to os/user.Lookup.
// Windows does not use NSS/getent; its user lookup works without CGO.
func lookupWithGetent(username string) (*user.User, error) {
return user.Lookup(username)
}
// currentUserWithGetent on Windows just delegates to os/user.Current.
func currentUserWithGetent() (*user.User, error) {
return user.Current()
}
// getShellFromGetent is a no-op on Windows; shell resolution uses PowerShell detection.
func getShellFromGetent(_ string) string {
return ""
}
// groupIdsWithFallback on Windows just delegates to u.GroupIds().
func groupIdsWithFallback(u *user.User) ([]string, error) {
return u.GroupIds()
}

View File

@@ -1,14 +1,17 @@
package shell
package server
import (
"bufio"
"fmt"
"net"
"os"
"os/exec"
"os/user"
"runtime"
"strconv"
"strings"
"github.com/gliderlabs/ssh"
log "github.com/sirupsen/logrus"
)
@@ -19,9 +22,9 @@ const (
powershellExe = "powershell.exe"
)
// GetUserShell returns the appropriate shell for the given user ID.
// Handles all platform-specific logic and fallbacks consistently.
func GetUserShell(userID string) string {
// getUserShell returns the appropriate shell for the given user ID
// Handles all platform-specific logic and fallbacks consistently
func getUserShell(userID string) string {
switch runtime.GOOS {
case "windows":
return getWindowsUserShell()
@@ -53,7 +56,7 @@ func getUnixUserShell(userID string) string {
return shell
}
if shell := GetShellFromGetent(userID); shell != "" {
if shell := getShellFromGetent(userID); shell != "" {
return shell
}
@@ -64,7 +67,7 @@ func getUnixUserShell(userID string) string {
return defaultUnixShell
}
// getShellFromPasswd reads the shell from /etc/passwd for the given user ID.
// getShellFromPasswd reads the shell from /etc/passwd for the given user ID
func getShellFromPasswd(userID string) string {
file, err := os.Open("/etc/passwd")
if err != nil {
@@ -98,8 +101,8 @@ func getShellFromPasswd(userID string) string {
return ""
}
// PrepareUserEnv prepares environment variables for user execution.
func PrepareUserEnv(user *user.User, shell string) []string {
// prepareUserEnv prepares environment variables for user execution
func prepareUserEnv(user *user.User, shell string) []string {
pathValue := "/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"
if runtime.GOOS == "windows" {
pathValue = `C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0`
@@ -114,9 +117,9 @@ func PrepareUserEnv(user *user.User, shell string) []string {
}
}
// AcceptEnv checks if an environment variable from an SSH client should be accepted.
// This is a whitelist of variables that SSH clients can send to the server.
func AcceptEnv(envVar string) bool {
// acceptEnv checks if environment variable from SSH client should be accepted
// This is a whitelist of variables that SSH clients can send to the server
func acceptEnv(envVar string) bool {
varName := envVar
if idx := strings.Index(envVar, "="); idx != -1 {
varName = envVar[:idx]
@@ -153,3 +156,29 @@ func AcceptEnv(envVar string) bool {
return false
}
// prepareSSHEnv prepares SSH protocol-specific environment variables
// These variables provide information about the SSH connection itself
func prepareSSHEnv(session ssh.Session) []string {
remoteAddr := session.RemoteAddr()
localAddr := session.LocalAddr()
remoteHost, remotePort, err := net.SplitHostPort(remoteAddr.String())
if err != nil {
remoteHost = remoteAddr.String()
remotePort = "0"
}
localHost, localPort, err := net.SplitHostPort(localAddr.String())
if err != nil {
localHost = localAddr.String()
localPort = strconv.Itoa(InternalSSHPort)
}
return []string{
// SSH_CLIENT format: "client_ip client_port server_port"
fmt.Sprintf("SSH_CLIENT=%s %s %s", remoteHost, remotePort, localPort),
// SSH_CONNECTION format: "client_ip client_port server_ip server_port"
fmt.Sprintf("SSH_CONNECTION=%s %s %s %s", remoteHost, remotePort, localHost, localPort),
}
}

View File

@@ -1,35 +0,0 @@
package server
import (
"fmt"
"net"
"strconv"
"github.com/gliderlabs/ssh"
)
// prepareSSHEnv prepares SSH protocol-specific environment variables
// These variables provide information about the SSH connection itself
func prepareSSHEnv(session ssh.Session) []string {
remoteAddr := session.RemoteAddr()
localAddr := session.LocalAddr()
remoteHost, remotePort, err := net.SplitHostPort(remoteAddr.String())
if err != nil {
remoteHost = remoteAddr.String()
remotePort = "0"
}
localHost, localPort, err := net.SplitHostPort(localAddr.String())
if err != nil {
localHost = localAddr.String()
localPort = strconv.Itoa(InternalSSHPort)
}
return []string{
// SSH_CLIENT format: "client_ip client_port server_port"
fmt.Sprintf("SSH_CLIENT=%s %s %s", remoteHost, remotePort, localPort),
// SSH_CONNECTION format: "client_ip client_port server_ip server_port"
fmt.Sprintf("SSH_CONNECTION=%s %s %s %s", remoteHost, remotePort, localHost, localPort),
}
}

View File

@@ -9,8 +9,6 @@ import (
"strings"
log "github.com/sirupsen/logrus"
shellutil "github.com/netbirdio/netbird/client/internal/shell"
)
var (
@@ -25,8 +23,8 @@ func isPlatformUnix() bool {
// Dependency injection variables for testing - allows mocking dynamic runtime checks
var (
getCurrentUser = shellutil.CurrentUserWithGetent
lookupUser = shellutil.LookupWithGetent
getCurrentUser = currentUserWithGetent
lookupUser = lookupWithGetent
getCurrentOS = func() string { return runtime.GOOS }
getIsProcessPrivileged = isCurrentProcessPrivileged

View File

@@ -16,8 +16,6 @@ import (
"github.com/gliderlabs/ssh"
log "github.com/sirupsen/logrus"
shellutil "github.com/netbirdio/netbird/client/internal/shell"
)
// POSIX portable filename character set regex: [a-zA-Z0-9._-]
@@ -162,7 +160,7 @@ func (s *Server) parseUserCredentials(localUser *user.User) (uint32, uint32, []u
// getSupplementaryGroups retrieves supplementary group IDs for a user.
// Uses id/getent fallback for NSS users in CGO_ENABLED=0 builds.
func (s *Server) getSupplementaryGroups(u *user.User) ([]uint32, error) {
groupIDStrings, err := shellutil.GroupIdsWithFallback(u)
groupIDStrings, err := groupIdsWithFallback(u)
if err != nil {
return nil, fmt.Errorf("get group IDs for user %s: %w", u.Username, err)
}
@@ -198,7 +196,7 @@ func (s *Server) createExecutorCommand(logger *log.Entry, session ssh.Session, l
GID: gid,
Groups: groups,
WorkingDir: localUser.HomeDir,
Shell: shellutil.GetUserShell(localUser.Uid),
Shell: getUserShell(localUser.Uid),
Command: session.RawCommand(),
PTY: hasPty,
}
@@ -230,7 +228,7 @@ func (s *Server) createPtyCommand(privilegeResult PrivilegeCheckResult, ptyReq s
func (s *Server) createDirectPtyCommand(session ssh.Session, localUser *user.User, ptyReq ssh.Pty) *exec.Cmd {
log.Debugf("creating direct Pty command for user %s (no user switching needed)", localUser.Username)
shell := shellutil.GetUserShell(localUser.Uid)
shell := getUserShell(localUser.Uid)
args := s.getShellCommandArgs(shell, session.RawCommand())
cmd := s.createShellCommand(session.Context(), shell, args)
@@ -247,12 +245,12 @@ func (s *Server) preparePtyEnv(localUser *user.User, ptyReq ssh.Pty, session ssh
termType = "xterm-256color"
}
env := shellutil.PrepareUserEnv(localUser, shellutil.GetUserShell(localUser.Uid))
env := prepareUserEnv(localUser, getUserShell(localUser.Uid))
env = append(env, prepareSSHEnv(session)...)
env = append(env, fmt.Sprintf("TERM=%s", termType))
for _, v := range session.Environ() {
if shellutil.AcceptEnv(v) {
if acceptEnv(v) {
env = append(env, v)
}
}

View File

@@ -13,8 +13,6 @@ import (
"github.com/gliderlabs/ssh"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
shellutil "github.com/netbirdio/netbird/client/internal/shell"
)
// validateUsername validates Windows usernames according to SAM Account Name rules
@@ -106,7 +104,7 @@ func (s *Server) createExecutorCommand(logger *log.Entry, session ssh.Session, l
func (s *Server) createUserSwitchCommand(logger *log.Entry, session ssh.Session, localUser *user.User) (*exec.Cmd, func(), error) {
username, domain := s.parseUsername(localUser.Username)
shell := shellutil.GetUserShell(localUser.Uid)
shell := getUserShell(localUser.Uid)
rawCmd := session.RawCommand()
var command string

5
go.mod
View File

@@ -30,7 +30,6 @@ require (
require (
github.com/DeRuina/timberjack v1.4.2
github.com/Microsoft/go-winio v0.6.2
github.com/awnumar/memguard v0.23.0
github.com/aws/aws-sdk-go-v2 v1.38.3
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.1
@@ -114,7 +113,7 @@ require (
github.com/ti-mo/conntrack v0.5.1
github.com/ti-mo/netfilter v0.5.2
github.com/vmihailenco/msgpack/v5 v5.4.1
github.com/wailsapp/wails/v3 v3.0.0-alpha2.111
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117
github.com/yusufpapurcu/wmi v1.2.4
github.com/zcalusic/sysinfo v1.1.3
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0
@@ -157,6 +156,7 @@ require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/adrg/xdg v0.5.3 // indirect
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be // indirect
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
@@ -303,7 +303,6 @@ require (
github.com/tklauser/numcpus v0.10.0 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
github.com/wailsapp/wails/webview2 v1.0.27 // indirect
github.com/wlynxg/anet v0.0.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/zeebo/blake3 v0.2.3 // indirect

6
go.sum
View File

@@ -660,10 +660,8 @@ github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IU
github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
github.com/wailsapp/wails/v3 v3.0.0-alpha2.111 h1:MKx1nOnhnDuEGrRBmtxLOJq1NERwailu2cI4BvzWhi4=
github.com/wailsapp/wails/v3 v3.0.0-alpha2.111/go.mod h1:wrdvmyeCsB/K3YqJDoH8E3MwcN8NXAMnEFaDTW46w60=
github.com/wailsapp/wails/webview2 v1.0.27 h1:wjgAi/I8BBZ7kUGU8um3XF3ILEfzr96Q2Q1G4GPjMns=
github.com/wailsapp/wails/webview2 v1.0.27/go.mod h1:zdM4jcO1IaC61RiJL5F1BzgoqBHFIdacz8gPr5exr0o=
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117 h1:udyjqPG3AIgkod5QDR/WblCkpV8R86BFPSrsWxSyt5Y=
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117/go.mod h1:74WH2FScMsgucZvHHvv7eOefDXCm/CjuIxqhhZgPhKg=
github.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU=
github.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=

View File

@@ -0,0 +1,38 @@
package llm
import (
"regexp"
"strings"
)
// bedrockRegionPrefixes are the cross-region inference-profile prefixes that
// front a Bedrock model id (e.g. "eu.anthropic.claude-...").
var bedrockRegionPrefixes = []string{"us.", "eu.", "apac.", "global."}
// bedrockVersionSuffix matches the trailing "-vN[:N]" or "-YYYYMMDD-vN[:N]"
// version/throughput suffix of a Bedrock model id.
var bedrockVersionSuffix = regexp.MustCompile(`-(\d{8}-)?v\d+(:\d+)?$`)
// NormalizeBedrockModel strips an ARN wrapper, a cross-region inference-profile
// prefix, and the version/throughput suffix from a Bedrock model id so it
// matches the catalog/pricing key, e.g.
// "eu.anthropic.claude-sonnet-4-5-20250929-v1:0" -> "anthropic.claude-sonnet-4-5"
// and the inference-profile ARN's last segment likewise. It is the single
// source of truth shared by the request parser (which normalizes the request
// model from the URL path) and the router (which normalizes the operator's
// registered Bedrock model ids so both sides compare equal).
func NormalizeBedrockModel(modelID string) string {
m := modelID
if strings.HasPrefix(m, "arn:") {
if i := strings.LastIndex(m, "/"); i >= 0 {
m = m[i+1:]
}
}
for _, p := range bedrockRegionPrefixes {
if strings.HasPrefix(m, p) {
m = m[len(p):]
break
}
}
return bedrockVersionSuffix.ReplaceAllString(m, "")
}

View File

@@ -0,0 +1,23 @@
package llm
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestNormalizeBedrockModel(t *testing.T) {
cases := map[string]string{
"eu.anthropic.claude-sonnet-4-5-20250929-v1:0": "anthropic.claude-sonnet-4-5",
"us.anthropic.claude-haiku-4-5": "anthropic.claude-haiku-4-5",
"us.anthropic.claude-opus-4-8-20250101-v1:0": "anthropic.claude-opus-4-8",
"anthropic.claude-sonnet-4-5-20250929-v1:0": "anthropic.claude-sonnet-4-5",
"meta.llama3-3-70b-instruct-v1:0": "meta.llama3-3-70b-instruct",
"amazon.nova-pro-v1:0": "amazon.nova-pro",
// Inference-profile ARN — model id lives in the last path segment.
"arn:aws:bedrock:eu-central-1:123456789012:inference-profile/eu.anthropic.claude-sonnet-4-5-20250929-v1:0": "anthropic.claude-sonnet-4-5",
}
for in, want := range cases {
require.Equal(t, want, NormalizeBedrockModel(in), "normalize %q", in)
}
}

View File

@@ -0,0 +1,30 @@
package llm_router
import (
"testing"
"github.com/stretchr/testify/assert"
)
// TestRouteClaimsModel_BedrockNormalizesCandidate guards the fix for the native
// Bedrock routing gap: the request model reaches the router already normalized
// (the parser strips the region/inference-profile prefix and version suffix),
// so a provider registered with the raw inference-profile id must still match.
func TestRouteClaimsModel_BedrockNormalizesCandidate(t *testing.T) {
route := ProviderRoute{Bedrock: true, Models: []string{"us.anthropic.claude-haiku-4-5"}}
assert.True(t, routeClaimsModel(route, "anthropic.claude-haiku-4-5"),
"raw region-prefixed Bedrock model must match the normalized request model")
assert.False(t, routeClaimsModel(route, "anthropic.claude-opus-4-8"),
"a model outside the provider's list must not match")
// A provider registered with the already-normalized id also matches.
normalized := ProviderRoute{Bedrock: true, Models: []string{"anthropic.claude-haiku-4-5"}}
assert.True(t, routeClaimsModel(normalized, "anthropic.claude-haiku-4-5"),
"normalized Bedrock model must match")
// Non-Bedrock routes keep exact matching (no prefix stripping).
openai := ProviderRoute{Models: []string{"gpt-4o"}}
assert.True(t, routeClaimsModel(openai, "gpt-4o"), "exact model must match")
assert.False(t, routeClaimsModel(openai, "us.gpt-4o"),
"non-Bedrock routes must not strip a us. prefix")
}

View File

@@ -23,6 +23,7 @@ import (
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
"github.com/netbirdio/netbird/proxy/internal/llm"
"github.com/netbirdio/netbird/proxy/internal/middleware"
)
@@ -555,6 +556,14 @@ func routeClaimsModel(route ProviderRoute, model string) bool {
if candidate == model {
return true
}
// Bedrock request models reach the router already normalized (the parser
// strips the region / inference-profile prefix and version suffix), but
// the operator may register the raw inference-profile id (e.g.
// "us.anthropic.claude-haiku-4-5"). Normalize the candidate so both sides
// compare equal; otherwise a native Bedrock request denies as not-routable.
if route.Bedrock && llm.NormalizeBedrockModel(candidate) == model {
return true
}
}
return false
}

View File

@@ -0,0 +1,176 @@
package client
import (
"context"
"errors"
"net"
"time"
log "github.com/sirupsen/logrus"
)
const (
raceTotalTimeout = 40 * time.Second
raceFallbackDelay = 10 * time.Second
)
type raceAttempt struct {
conn net.Conn
err error
}
type raceOutcome struct {
conn net.Conn
err error
done bool
}
type connRace struct {
opener *FallbackOpener
peerKey string
remoteRelayServer RelayServer
preferForeign bool
raceCtx context.Context
otherCtx context.Context
cancelPreferred context.CancelFunc
cancelOther context.CancelFunc
results chan raceAttempt
fallbackTimer *time.Timer
otherStarted bool
settled int
lastErr error
}
type FallbackOpener struct {
home *Client
foreignStore *ForeignRelaysStore
fallbackDelay time.Duration
totalTimeout time.Duration
// openFn performs a single attempt. It is overridable in tests; when nil the
// real home/foreign dispatch in open is used.
openFn func(ctx context.Context, peerKey string, remoteRelayServer RelayServer, foreign bool) raceAttempt
}
func NewFallbackOpener(home *Client, foreignStore *ForeignRelaysStore) *FallbackOpener {
return &FallbackOpener{
home: home,
foreignStore: foreignStore,
fallbackDelay: raceFallbackDelay,
totalTimeout: raceTotalTimeout,
}
}
func (r *FallbackOpener) Run(ctx context.Context, peerKey string, remoteRelayServer RelayServer, preferForeign bool) (net.Conn, error) {
raceCtx, cancel := context.WithTimeout(ctx, r.totalTimeout)
defer cancel()
preferredCtx, cancelPreferred := context.WithCancel(raceCtx)
otherCtx, cancelOther := context.WithCancel(raceCtx)
race := &connRace{
opener: r,
peerKey: peerKey,
remoteRelayServer: remoteRelayServer,
preferForeign: preferForeign,
raceCtx: raceCtx,
otherCtx: otherCtx,
cancelPreferred: cancelPreferred,
cancelOther: cancelOther,
results: make(chan raceAttempt, 2),
fallbackTimer: time.NewTimer(r.fallbackDelay),
}
defer race.fallbackTimer.Stop()
go func() {
race.results <- r.open(preferredCtx, peerKey, remoteRelayServer, preferForeign)
}()
for {
select {
case <-race.fallbackTimer.C:
race.startOther()
case res := <-race.results:
if o := race.handleResult(res); o.done {
return o.conn, o.err
}
case <-raceCtx.Done():
return race.onTimeout()
}
}
}
func (c *connRace) startOther() {
if c.otherStarted {
return
}
c.otherStarted = true
c.fallbackTimer.Stop()
go func() {
c.results <- c.opener.open(c.otherCtx, c.peerKey, c.remoteRelayServer, !c.preferForeign)
}()
}
func (c *connRace) handleResult(res raceAttempt) raceOutcome {
if (res.err == nil && res.conn != nil) || errors.Is(res.err, ErrConnAlreadyExists) {
c.settled++
c.stop()
return raceOutcome{conn: res.conn, err: res.err, done: true}
}
c.lastErr = res.err
c.settled++
if !c.otherStarted {
c.startOther()
return raceOutcome{}
}
if c.settled == 2 {
c.cancelPreferred()
c.cancelOther()
return raceOutcome{err: c.lastErr, done: true}
}
return raceOutcome{}
}
func (c *connRace) onTimeout() (net.Conn, error) {
c.stop()
if c.lastErr != nil {
return nil, c.lastErr
}
return nil, c.raceCtx.Err()
}
func (c *connRace) stop() {
c.cancelPreferred()
c.cancelOther()
go c.opener.drainLoser(c.results, c.settled, c.otherStarted)
}
func (r *FallbackOpener) open(ctx context.Context, peerKey string, remoteRelayServer RelayServer, foreign bool) raceAttempt {
if r.openFn != nil {
return r.openFn(ctx, peerKey, remoteRelayServer, foreign)
}
if foreign {
conn, err := r.foreignStore.OpenConn(ctx, peerKey, remoteRelayServer)
return raceAttempt{conn: conn, err: err}
}
conn, err := r.home.OpenConn(ctx, peerKey)
return raceAttempt{conn: conn, err: err}
}
func (r *FallbackOpener) drainLoser(results chan raceAttempt, settled int, otherStarted bool) {
started := 1
if otherStarted {
started = 2
}
for i := settled; i < started; i++ {
res := <-results
if res.conn != nil {
if err := res.conn.Close(); err != nil {
log.Debugf("failed to close losing relay connection: %v", err)
}
}
}
}

View File

@@ -0,0 +1,450 @@
package client
import (
"context"
"errors"
"fmt"
"net"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// The FallbackOpener race is driven by a single goroutine (Run's select loop)
// with worker goroutines that communicate only through the buffered results
// channel and the two cancel contexts. These tests exercise that state machine
// in isolation via an injected openFn, so no relay server or network is needed.
// Timing is scaled down through the fallbackDelay/totalTimeout fields.
// raceFakeConn tracks whether Close was called. Only Close is exercised by the
// race logic (drainLoser closes losers; Run returns the winner untouched).
type raceFakeConn struct {
net.Conn
label string
closed atomic.Bool
}
func (c *raceFakeConn) Close() error {
c.closed.Store(true)
return nil
}
// raceAttemptScript describes how a single scripted attempt behaves.
type raceAttemptScript struct {
delay time.Duration
conn *raceFakeConn // non-nil => the attempt succeeds and returns this conn
err error // returned when conn is nil
// ignoreCtx makes the attempt complete after delay even if its context is
// cancelled. It models an OpenConn that produced a real connection right as
// the race cancelled it - exactly the case drainLoser must clean up.
ignoreCtx bool
}
// fakeOpener replaces FallbackOpener.open. Scripts are keyed by the foreign
// flag, so which script is "preferred" depends on the preferForeign argument
// passed to Run.
type fakeOpener struct {
mu sync.Mutex
scripts map[bool]raceAttemptScript
calls []bool // foreign flag of each open() invocation, in order
}
func (f *fakeOpener) open(ctx context.Context, _ string, _ RelayServer, foreign bool) raceAttempt {
f.mu.Lock()
f.calls = append(f.calls, foreign)
s, ok := f.scripts[foreign]
f.mu.Unlock()
if !ok {
return raceAttempt{err: fmt.Errorf("no script for foreign=%v", foreign)}
}
if s.delay > 0 {
timer := time.NewTimer(s.delay)
defer timer.Stop()
if s.ignoreCtx {
<-timer.C
} else {
select {
case <-timer.C:
case <-ctx.Done():
return raceAttempt{err: ctx.Err()}
}
}
}
if s.conn != nil {
return raceAttempt{conn: s.conn}
}
return raceAttempt{err: s.err}
}
func (f *fakeOpener) callCount() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.calls)
}
func (f *fakeOpener) firstCallForeign(t *testing.T) bool {
t.Helper()
f.mu.Lock()
defer f.mu.Unlock()
require.NotEmpty(t, f.calls, "expected at least one open attempt")
return f.calls[0]
}
func newTestOpener(f *fakeOpener, fallbackDelay, totalTimeout time.Duration) *FallbackOpener {
o := NewFallbackOpener(nil, nil)
o.openFn = f.open
o.fallbackDelay = fallbackDelay
o.totalTimeout = totalTimeout
return o
}
const (
// controller prefers the home relay, i.e. preferForeign == false.
preferHome = false
preferForeign = true
)
var errAttempt = errors.New("attempt failed")
// The preferred attempt wins before the fallback timer fires, so the other
// attempt is never started.
func TestFallbackOpener_PreferredWinsImmediately(t *testing.T) {
homeConn := &raceFakeConn{label: "home"}
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {conn: homeConn}, // preferred (home): instant success
true: {delay: 5 * time.Second, conn: foreignConn}, // would never finish in time
}}
o := newTestOpener(f, 40*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.NoError(t, err)
require.Same(t, homeConn, conn)
assert.Equal(t, 1, f.callCount(), "other attempt must not start when preferred wins first")
assert.False(t, f.firstCallForeign(t), "home must be tried first when preferring home")
assert.False(t, foreignConn.closed.Load())
}
// preferForeign flips which relay is tried first.
func TestFallbackOpener_PreferForeignRoutesForeignFirst(t *testing.T) {
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
true: {conn: foreignConn}, // preferred (foreign): instant success
false: {delay: 5 * time.Second, conn: &raceFakeConn{}},
}}
o := newTestOpener(f, 40*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferForeign)
require.NoError(t, err)
require.Same(t, foreignConn, conn)
assert.Equal(t, 1, f.callCount())
assert.True(t, f.firstCallForeign(t), "foreign must be tried first when preferring foreign")
}
// ErrConnAlreadyExists counts as success: Run returns it and does not start the
// other attempt.
func TestFallbackOpener_ErrConnAlreadyExistsIsSuccess(t *testing.T) {
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {err: ErrConnAlreadyExists},
true: {delay: 5 * time.Second, conn: &raceFakeConn{}},
}}
o := newTestOpener(f, 40*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.ErrorIs(t, err, ErrConnAlreadyExists)
assert.Nil(t, conn)
assert.Equal(t, 1, f.callCount(), "other attempt must not start on ErrConnAlreadyExists")
}
// A preferred failure starts the other attempt immediately, without waiting for
// the fallback timer.
func TestFallbackOpener_PreferredFailsStartsOtherBeforeTimer(t *testing.T) {
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {err: errAttempt}, // preferred fails instantly
true: {delay: 5 * time.Millisecond, conn: foreignConn},
}}
fallbackDelay := 500 * time.Millisecond
o := newTestOpener(f, fallbackDelay, 2*time.Second)
start := time.Now()
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
elapsed := time.Since(start)
require.NoError(t, err)
require.Same(t, foreignConn, conn)
assert.Equal(t, 2, f.callCount())
assert.Less(t, elapsed, fallbackDelay/2, "fallback must not wait for the timer after a preferred failure")
}
// When the preferred attempt is slow, the fallback timer starts the other
// attempt and its success wins.
func TestFallbackOpener_TimerStartsOtherWhenPreferredSlow(t *testing.T) {
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {delay: 5 * time.Second}, // preferred hangs until cancelled
true: {delay: 5 * time.Millisecond, conn: foreignConn},
}}
fallbackDelay := 40 * time.Millisecond
o := newTestOpener(f, fallbackDelay, 2*time.Second)
start := time.Now()
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
elapsed := time.Since(start)
require.NoError(t, err)
require.Same(t, foreignConn, conn)
assert.Equal(t, 2, f.callCount())
assert.GreaterOrEqual(t, elapsed, fallbackDelay, "other must not start before the fallback timer fires")
}
// Both attempts fail: Run returns the last error and tries both relays.
func TestFallbackOpener_BothFail(t *testing.T) {
errOther := errors.New("other failed")
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {err: errAttempt},
true: {err: errOther},
}}
o := newTestOpener(f, 40*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.Error(t, err)
assert.Nil(t, conn)
assert.ErrorIs(t, err, errOther, "the most recent error should be surfaced")
assert.Equal(t, 2, f.callCount())
}
// When both attempts succeed, drainLoser must close the losing connection so it
// is not leaked. Here the preferred attempt wins and the foreign loser - which
// produced a real conn despite being cancelled - is closed.
func TestFallbackOpener_DoubleSuccessClosesLoser(t *testing.T) {
homeConn := &raceFakeConn{label: "home"}
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {delay: 30 * time.Millisecond, conn: homeConn}, // preferred wins
true: {delay: 80 * time.Millisecond, conn: foreignConn, ignoreCtx: true}, // loser yields a conn after cancel
}}
o := newTestOpener(f, 15*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.NoError(t, err)
require.Same(t, homeConn, conn)
assert.Equal(t, 2, f.callCount())
assert.False(t, homeConn.closed.Load(), "the winning connection must not be closed")
require.Eventually(t, foreignConn.closed.Load, time.Second, 5*time.Millisecond,
"the losing connection must be closed by drainLoser")
}
// Winner selection is purely by result arrival order, not by preference: when
// the non-preferred attempt returns first it wins even though home was
// preferred. This is the mechanism behind the split-relay concern - two peers
// racing independently have no shared tie-break, so under adversarial timing
// they can settle on different relays. Documented here as current behavior.
func TestFallbackOpener_FasterOtherWinsDespitePreference(t *testing.T) {
homeConn := &raceFakeConn{label: "home"}
foreignConn := &raceFakeConn{label: "foreign"}
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {delay: 60 * time.Millisecond, conn: homeConn, ignoreCtx: true}, // preferred but slower
true: {delay: 5 * time.Millisecond, conn: foreignConn}, // other is faster
}}
o := newTestOpener(f, 15*time.Millisecond, 2*time.Second)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.NoError(t, err)
require.Same(t, foreignConn, conn, "the first successful attempt wins regardless of preference")
require.Eventually(t, homeConn.closed.Load, time.Second, 5*time.Millisecond,
"the slower preferred attempt becomes the loser and is closed")
}
// The whole race is bounded by totalTimeout. With no attempt succeeding or
// failing, Run returns the deadline error.
func TestFallbackOpener_TotalTimeout(t *testing.T) {
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {delay: 5 * time.Second},
true: {delay: 5 * time.Second},
}}
o := newTestOpener(f, 20*time.Millisecond, 80*time.Millisecond)
conn, err := o.Run(context.Background(), "peer", RelayServer{Addr: "srv"}, preferHome)
require.Error(t, err)
assert.Nil(t, conn)
assert.ErrorIs(t, err, context.DeadlineExceeded)
}
// Cancelling the caller's context aborts the race promptly with the cancel
// error, even before the fallback timer would fire.
func TestFallbackOpener_ParentContextCanceled(t *testing.T) {
f := &fakeOpener{scripts: map[bool]raceAttemptScript{
false: {delay: 5 * time.Second},
true: {delay: 5 * time.Second},
}}
// fallbackDelay large so the timer never fires; only the parent cancel ends the race.
o := newTestOpener(f, 5*time.Second, 5*time.Second)
ctx, cancel := context.WithCancel(context.Background())
go func() {
time.Sleep(20 * time.Millisecond)
cancel()
}()
start := time.Now()
conn, err := o.Run(ctx, "peer", RelayServer{Addr: "srv"}, preferHome)
require.Error(t, err)
assert.Nil(t, conn)
assert.ErrorIs(t, err, context.Canceled)
assert.Less(t, time.Since(start), time.Second, "must return shortly after the parent context is cancelled")
assert.Equal(t, 1, f.callCount(), "the other attempt must not start")
}
// rendezvous models the relay-level requirement that a relayed connection is
// established only once BOTH peers subscribe to the same relay server. arrive
// records a peer's presence on a relay and returns a channel that closes when
// the second peer arrives, so an attempt can only complete after a real
// rendezvous - the same coupling the production code depends on.
type rendezvous struct {
mu sync.Mutex
arrivals map[string]int
gates map[string]chan struct{}
}
func newRendezvous() *rendezvous {
return &rendezvous{arrivals: map[string]int{}, gates: map[string]chan struct{}{}}
}
func (r *rendezvous) arrive(relay string) <-chan struct{} {
r.mu.Lock()
defer r.mu.Unlock()
g, ok := r.gates[relay]
if !ok {
g = make(chan struct{})
r.gates[relay] = g
}
r.arrivals[relay]++
if r.arrivals[relay] == 2 {
close(g)
}
return g
}
// splitPeer is one peer's view of the two relays. relayFor maps the foreign
// flag to a relay name; postDelay is how long after the rendezvous that peer's
// OpenConn takes to return (its per-relay subscribe latency). Different values
// per peer model the asymmetric timing that triggers finding #1.
type splitPeer struct {
rv *rendezvous
relayFor map[bool]string
postDelay map[string]time.Duration
}
func (p *splitPeer) open(ctx context.Context, _ string, _ RelayServer, foreign bool) raceAttempt {
relay := p.relayFor[foreign]
select {
case <-p.rv.arrive(relay):
case <-ctx.Done():
return raceAttempt{err: ctx.Err()}
}
timer := time.NewTimer(p.postDelay[relay])
defer timer.Stop()
select {
case <-timer.C:
case <-ctx.Done():
return raceAttempt{err: ctx.Err()}
}
return raceAttempt{conn: &raceFakeConn{label: relay}}
}
// TestFallbackOpener_SplitRelaySelection reproduces finding #1: the two peers
// run FallbackOpener.Run independently with no shared tie-break, so the winner
// is chosen purely by local result-arrival order. Under an adversarial - but
// self-consistent - timing profile they settle on DIFFERENT relays.
//
// Both peers prefer relayA (the controller's home). The split needs each peer's
// preferred relayA to be slow enough that both start their fallback (so both
// relays actually rendezvous), and then each peer's fast path to be a different
// relay:
// - peerA: relayA slow (abandoned), relayB fast -> peerA wins relayB
// - peerB: relayA fast (wins), relayB slow -> peerB wins relayA
//
// Each winner then cancels its attempt on the relay the OTHER peer actually
// kept, leaving two half-open relayed connections that were both reported as
// successful. When a deterministic cross-peer tie-break is added to fix this,
// invert the assertion below to require convergence.
func TestFallbackOpener_SplitRelaySelection(t *testing.T) {
const (
relayA = "relayA" // controller's home relay; both peers prefer it
relayB = "relayB" // non-controller's home relay
)
rv := newRendezvous()
peerA := &splitPeer{
rv: rv,
relayFor: map[bool]string{false: relayA, true: relayB}, // home=relayA
postDelay: map[string]time.Duration{
relayA: 500 * time.Millisecond, // preferred but slow -> abandoned
relayB: 10 * time.Millisecond, // fallback is fast -> peerA wins relayB
},
}
peerB := &splitPeer{
rv: rv,
relayFor: map[bool]string{false: relayB, true: relayA}, // home=relayB
postDelay: map[string]time.Duration{
relayA: 80 * time.Millisecond, // preferred, wins - but only after starting fallback
relayB: 500 * time.Millisecond, // fallback (home) is slow -> abandoned
},
}
fallbackDelay := 30 * time.Millisecond
newPeerOpener := func(p *splitPeer) *FallbackOpener {
o := NewFallbackOpener(nil, nil)
o.openFn = p.open
o.fallbackDelay = fallbackDelay
o.totalTimeout = 5 * time.Second
return o
}
oA := newPeerOpener(peerA)
oB := newPeerOpener(peerB)
type result struct {
conn net.Conn
err error
}
var ra, rb result
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
ra.conn, ra.err = oA.Run(context.Background(), "peerB", RelayServer{Addr: relayB}, preferHome)
}()
go func() {
defer wg.Done()
rb.conn, rb.err = oB.Run(context.Background(), "peerA", RelayServer{Addr: relayA}, preferForeign)
}()
wg.Wait()
require.NoError(t, ra.err)
require.NoError(t, rb.err)
aRelay := ra.conn.(*raceFakeConn).label
bRelay := rb.conn.(*raceFakeConn).label
t.Logf("peerA settled on %s, peerB settled on %s", aRelay, bRelay)
assert.Equal(t, aRelay, bRelay,
"peers selected different relays with no cross-peer tie-break")
assert.Equal(t, relayB, aRelay, "peerA abandoned its slow preferred relay and won the fallback")
assert.Equal(t, relayA, bRelay, "peerB won its preferred relay after starting the fallback")
}

View File

@@ -0,0 +1,155 @@
package client
import (
"context"
"net"
"sync"
"time"
log "github.com/sirupsen/logrus"
"golang.org/x/sync/singleflight"
relayAuth "github.com/netbirdio/netbird/shared/relay/auth/hmac"
)
type foreignRelay struct {
client *Client
created time.Time
inUse int
}
type ForeignRelaysStore struct {
mu sync.RWMutex
clients map[string]*foreignRelay
group singleflight.Group
ctx context.Context
tokenStore *relayAuth.TokenStore
peerID string
mtu uint16
transportFallback *transportFallback
onDisconnect func(string)
keepUnusedServerTime time.Duration
}
func NewForeignRelaysStore(ctx context.Context, tokenStore *relayAuth.TokenStore, peerID string, mtu uint16, transportFallback *transportFallback, onDisconnect func(string), keepUnusedServerTime time.Duration) *ForeignRelaysStore {
return &ForeignRelaysStore{
clients: make(map[string]*foreignRelay),
ctx: ctx,
tokenStore: tokenStore,
peerID: peerID,
mtu: mtu,
transportFallback: transportFallback,
onDisconnect: onDisconnect,
keepUnusedServerTime: keepUnusedServerTime,
}
}
func (f *ForeignRelaysStore) OpenConn(ctx context.Context, peerKey string, remoteRelayServer RelayServer) (net.Conn, error) {
fr, err := f.acquire(remoteRelayServer)
if err != nil {
return nil, err
}
defer f.release(fr)
return fr.client.OpenConn(ctx, peerKey)
}
func (f *ForeignRelaysStore) acquire(remoteRelayServer RelayServer) (*foreignRelay, error) {
f.mu.Lock()
if fr, ok := f.clients[remoteRelayServer.Addr]; ok {
fr.inUse++
f.mu.Unlock()
return fr, nil
}
f.mu.Unlock()
v, err, _ := f.group.Do(remoteRelayServer.Addr, func() (any, error) {
f.mu.RLock()
fr, ok := f.clients[remoteRelayServer.Addr]
f.mu.RUnlock()
if ok {
return fr, nil
}
relayClient := NewClientWithServerIP(remoteRelayServer.Addr, remoteRelayServer.IP, f.tokenStore, f.peerID, f.mtu)
relayClient.SetTransportFallback(f.transportFallback)
if err := relayClient.Connect(f.ctx); err != nil {
return nil, err
}
relayClient.SetOnDisconnectListener(f.onDisconnect)
f.mu.Lock()
fr = &foreignRelay{client: relayClient, created: time.Now()}
f.clients[remoteRelayServer.Addr] = fr
f.mu.Unlock()
return fr, nil
})
if err != nil {
return nil, err
}
fr := v.(*foreignRelay)
f.mu.Lock()
if cur, ok := f.clients[remoteRelayServer.Addr]; !ok || cur != fr {
f.mu.Unlock()
return f.acquire(remoteRelayServer)
}
fr.inUse++
f.mu.Unlock()
return fr, nil
}
func (f *ForeignRelaysStore) release(fr *foreignRelay) {
f.mu.Lock()
fr.inUse--
f.mu.Unlock()
}
func (f *ForeignRelaysStore) evict(serverAddress string) {
f.mu.Lock()
defer f.mu.Unlock()
if _, ok := f.clients[serverAddress]; ok {
delete(f.clients, serverAddress)
log.Debugf("evicted disconnected foreign relay client: %s", serverAddress)
}
}
func (f *ForeignRelaysStore) cleanupUnused() {
f.mu.Lock()
defer f.mu.Unlock()
for addr, fr := range f.clients {
if time.Since(fr.created) <= f.keepUnusedServerTime {
continue
}
if fr.inUse > 0 {
continue
}
if fr.client.HasConns() {
continue
}
fr.client.SetOnDisconnectListener(nil)
go func() {
_ = fr.client.Close()
}()
log.Debugf("clean up unused relay server connection: %s", addr)
delete(f.clients, addr)
}
}
func (f *ForeignRelaysStore) states() []RelayConnState {
f.mu.RLock()
clients := make([]*Client, 0, len(f.clients))
for _, fr := range f.clients {
clients = append(clients, fr.client)
}
f.mu.RUnlock()
states := make([]RelayConnState, 0, len(clients))
for _, c := range clients {
states = append(states, relayConnState(c))
}
return states
}

View File

@@ -22,27 +22,6 @@ var (
ErrRelayClientNotConnected = fmt.Errorf("relay client not connected")
)
// RelayTrack hold the relay clients for the foreign relay servers.
// With the mutex can ensure we can open new connection in case the relay connection has been established with
// the relay server.
type RelayTrack struct {
sync.RWMutex
relayClient *Client
err error
created time.Time
// ready is closed once the dial started by openConnVia finishes (relayClient
// or err is set). Callers reusing a track wait on this instead of the track
// lock, so the dial never runs under rt.Lock.
ready chan struct{}
}
func NewRelayTrack() *RelayTrack {
return &RelayTrack{
created: time.Now(),
ready: make(chan struct{}),
}
}
type OnServerCloseListener func()
// ManagerOption configures a Manager at construction time.
@@ -59,6 +38,11 @@ type RelayConnState struct {
Err error
}
type RelayServer struct {
Addr string
IP netip.Addr
}
// WithMaxBackoffInterval caps the exponential backoff between reconnect
// attempts to the home relay. A non-positive value keeps the default.
func WithMaxBackoffInterval(d time.Duration) ManagerOption {
@@ -83,8 +67,7 @@ type Manager struct {
relayClientMu sync.RWMutex
reconnectGuard *Guard
relayClients map[string]*RelayTrack
relayClientsMutex sync.RWMutex
foreign *ForeignRelaysStore
onDisconnectedListeners map[string]*list.List
onReconnectedListenerFn func()
@@ -120,7 +103,6 @@ func NewManager(ctx context.Context, serverURLs []string, peerID string, mtu uin
ConnectionTimeout: defaultConnectionTimeout,
TransportFallback: tf,
},
relayClients: make(map[string]*RelayTrack),
onDisconnectedListeners: make(map[string]*list.List),
cleanupInterval: relayCleanupInterval,
keepUnusedServerTime: keepUnusedServerTime,
@@ -128,6 +110,7 @@ func NewManager(ctx context.Context, serverURLs []string, peerID string, mtu uin
for _, opt := range opts {
opt(m)
}
m.foreign = NewForeignRelaysStore(ctx, tokenStore, peerID, mtu, tf, m.onServerDisconnected, m.keepUnusedServerTime)
m.serverPicker.ServerURLs.Store(serverURLs)
m.reconnectGuard = NewGuard(m.serverPicker, m.maxBackoffInterval)
return m
@@ -159,40 +142,26 @@ func (m *Manager) Serve() error {
return err
}
// OpenConn opens a connection to the given peer key. If the peer is on the same relay server, the connection will be
// established via the relay server. If the peer is on a different relay server, the manager will establish a new
// connection to the relay server. It returns back with a net.Conn what represent the remote peer connection.
//
// serverIP, when valid and serverAddress is foreign, is used as a dial target if the FQDN-based dial fails.
// Ignored for the local home-server path. TLS verification still uses the FQDN via SNI.
func (m *Manager) OpenConn(ctx context.Context, serverAddress, peerKey string, serverIP netip.Addr) (net.Conn, error) {
func (m *Manager) OpenConn(ctx context.Context, remoteRelayServer RelayServer, peerKey string, preferForeign bool) (net.Conn, error) {
m.relayClientMu.RLock()
defer m.relayClientMu.RUnlock()
relayClient := m.relayClient
m.relayClientMu.RUnlock()
if m.relayClient == nil {
if relayClient == nil {
return nil, ErrRelayClientNotConnected
}
foreign, err := m.isForeignServer(serverAddress)
foreign, err := m.isForeignServer(relayClient, remoteRelayServer.Addr)
if err != nil {
return nil, err
}
var (
netConn net.Conn
)
if !foreign {
log.Debugf("open peer connection via permanent server: %s", peerKey)
netConn, err = m.relayClient.OpenConn(ctx, peerKey)
} else {
log.Debugf("open peer connection via foreign server: %s", serverAddress)
netConn, err = m.openConnVia(ctx, serverAddress, peerKey, serverIP)
}
if err != nil {
return nil, err
return relayClient.OpenConn(ctx, peerKey)
}
return netConn, err
opener := NewFallbackOpener(relayClient, m.foreign)
return opener.Run(ctx, peerKey, remoteRelayServer, preferForeign)
}
// Ready returns true if the home Relay client is connected to the relay server.
@@ -223,7 +192,7 @@ func (m *Manager) AddCloseListener(serverAddress string, onClosedListener OnServ
return ErrRelayClientNotConnected
}
foreign, err := m.isForeignServer(serverAddress)
foreign, err := m.isForeignServer(m.relayClient, serverAddress)
if err != nil {
return err
}
@@ -287,26 +256,7 @@ func (m *Manager) RelayStates() []RelayConnState {
states = append(states, st)
}
// Snapshot the tracks, then query each outside the map lock: a track can be
// held by an in-progress Connect, and blocking on it must not stall other
// relay operations.
m.relayClientsMutex.RLock()
tracks := make([]*RelayTrack, 0, len(m.relayClients))
for _, rt := range m.relayClients {
tracks = append(tracks, rt)
}
m.relayClientsMutex.RUnlock()
// Only connected foreign relays carry state; a failed connect is evicted
// immediately (openConnVia), so there is no error state to surface.
for _, rt := range tracks {
rt.RLock()
rc := rt.relayClient
rt.RUnlock()
if rc != nil {
states = append(states, relayConnState(rc))
}
}
states = append(states, m.foreign.states()...)
return states
}
@@ -327,76 +277,6 @@ func (m *Manager) UpdateToken(token *relayAuth.Token) error {
return m.tokenStore.UpdateToken(token)
}
func (m *Manager) openConnVia(ctx context.Context, serverAddress, peerKey string, serverIP netip.Addr) (net.Conn, error) {
// check if already has a connection to the desired relay server
m.relayClientsMutex.RLock()
rt, ok := m.relayClients[serverAddress]
m.relayClientsMutex.RUnlock()
if ok {
return m.openConnOnTrack(ctx, rt, peerKey)
}
// if not, establish a new connection but check it again (because changed the lock type) before starting the
// connection
m.relayClientsMutex.Lock()
rt, ok = m.relayClients[serverAddress]
if ok {
m.relayClientsMutex.Unlock()
return m.openConnOnTrack(ctx, rt, peerKey)
}
// Publish the track and release the map lock BEFORE dialing, so the dial does
// not run under rt.Lock (which would block RelayStates and the cleanup loop
// for the full dial). Concurrent callers find this track and wait on rt.ready.
rt = NewRelayTrack()
m.relayClients[serverAddress] = rt
m.relayClientsMutex.Unlock()
relayClient := NewClientWithServerIP(serverAddress, serverIP, m.tokenStore, m.peerID, m.mtu)
relayClient.SetTransportFallback(m.transportFallback)
err := relayClient.Connect(m.ctx)
if err != nil {
rt.Lock()
rt.err = err
rt.Unlock()
close(rt.ready)
m.relayClientsMutex.Lock()
delete(m.relayClients, serverAddress)
m.relayClientsMutex.Unlock()
return nil, err
}
// if connection closed then delete the relay client from the list
relayClient.SetOnDisconnectListener(m.onServerDisconnected)
rt.Lock()
rt.relayClient = relayClient
rt.Unlock()
close(rt.ready)
return relayClient.OpenConn(ctx, peerKey)
}
// openConnOnTrack opens a peer connection through an existing relay track,
// waiting for the dial started by another openConnVia call to finish. It waits
// on rt.ready rather than the track lock, so it neither holds nor contends the
// track lock across the dial.
func (m *Manager) openConnOnTrack(ctx context.Context, rt *RelayTrack, peerKey string) (net.Conn, error) {
select {
case <-rt.ready:
case <-ctx.Done():
return nil, ctx.Err()
}
rt.RLock()
defer rt.RUnlock()
if rt.err != nil {
return nil, rt.err
}
if rt.relayClient == nil {
return nil, ErrRelayClientNotConnected
}
return rt.relayClient.OpenConn(ctx, peerKey)
}
func (m *Manager) onServerConnected() {
m.listenerLock.Lock()
defer m.listenerLock.Unlock()
@@ -422,21 +302,12 @@ func (m *Manager) onServerDisconnected(serverAddress string) {
m.relayClientMu.Unlock()
if !isHome {
m.evictForeignRelay(serverAddress)
m.foreign.evict(serverAddress)
}
m.notifyOnDisconnectListeners(serverAddress)
}
func (m *Manager) evictForeignRelay(serverAddress string) {
m.relayClientsMutex.Lock()
defer m.relayClientsMutex.Unlock()
if _, ok := m.relayClients[serverAddress]; ok {
delete(m.relayClients, serverAddress)
log.Debugf("evicted disconnected foreign relay client: %s", serverAddress)
}
}
func (m *Manager) listenGuardEvent(ctx context.Context) {
for {
select {
@@ -459,8 +330,8 @@ func (m *Manager) storeClient(client *Client) {
m.relayClient.SetOnDisconnectListener(m.onServerDisconnected)
}
func (m *Manager) isForeignServer(address string) (bool, error) {
rAddr, err := m.relayClient.ServerInstanceURL()
func (m *Manager) isForeignServer(relayClient *Client, address string) (bool, error) {
rAddr, err := relayClient.ServerInstanceURL()
if err != nil {
return false, fmt.Errorf("relay client not connected")
}
@@ -475,50 +346,11 @@ func (m *Manager) startCleanupLoop() {
case <-m.ctx.Done():
return
case <-ticker.C:
m.cleanUpUnusedRelays()
m.foreign.cleanupUnused()
}
}
}
func (m *Manager) cleanUpUnusedRelays() {
m.relayClientsMutex.Lock()
defer m.relayClientsMutex.Unlock()
for addr, rt := range m.relayClients {
rt.Lock()
// if the connection failed to the server the relay client will be nil
// but the instance will be kept in the relayClients until the next locking
if rt.err != nil {
rt.Unlock()
continue
}
// dial still in progress (openConnVia publishes the track before Connect
// completes and no longer holds rt.Lock during it), nothing to clean up.
if rt.relayClient == nil {
rt.Unlock()
continue
}
if time.Since(rt.created) <= m.keepUnusedServerTime {
rt.Unlock()
continue
}
if rt.relayClient.HasConns() {
rt.Unlock()
continue
}
rt.relayClient.SetOnDisconnectListener(nil)
go func() {
_ = rt.relayClient.Close()
}()
log.Debugf("clean up unused relay server connection: %s", addr)
delete(m.relayClients, addr)
rt.Unlock()
}
}
func (m *Manager) addListener(serverAddress string, onClosedListener OnServerCloseListener) {
m.listenerLock.Lock()
defer m.listenerLock.Unlock()

View File

@@ -2,17 +2,14 @@ package client
import (
"context"
"net/netip"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// TestCleanUpUnusedRelays_DoesNotBlockOnRealHangingDial drives a real, hanging foreign
// relay dial and asserts cleanUpUnusedRelays does not stall behind it.
// relay dial and asserts the foreign store cleanup does not stall behind it.
func TestCleanUpUnusedRelays_DoesNotBlockOnRealHangingDial(t *testing.T) {
serverAddr := stallingRelayListener(t)
serverAddr, accepted := stallingRelayListener(t)
mCtx, mCancel := context.WithCancel(context.Background())
t.Cleanup(mCancel)
@@ -22,39 +19,32 @@ func TestCleanUpUnusedRelays_DoesNotBlockOnRealHangingDial(t *testing.T) {
dialDone := make(chan struct{})
go func() {
defer close(dialDone)
_, _ = m.openConnVia(mCtx, serverAddr, "peerKey", netip.Addr{})
_, _ = m.foreign.OpenConn(mCtx, "peerKey", RelayServer{Addr: serverAddr})
}()
// The track appears in the map once the dial is in flight.
require.Eventually(t, func() bool {
m.relayClientsMutex.RLock()
defer m.relayClientsMutex.RUnlock()
_, ok := m.relayClients[serverAddr]
return ok
}, 5*time.Second, 5*time.Millisecond, "relay dial did not start")
select {
case <-accepted:
case <-time.After(5 * time.Second):
t.Fatal("relay dial did not reach the listener")
}
cleanupDone := make(chan struct{})
go func() {
defer close(cleanupDone)
m.cleanUpUnusedRelays()
m.foreign.cleanupUnused()
}()
select {
case <-cleanupDone:
case <-time.After(2 * time.Second):
t.Fatal("cleanUpUnusedRelays blocked on an in-progress relay dial while holding the relay map lock")
t.Fatal("cleanupUnused blocked on an in-progress relay dial")
}
m.relayClientsMutex.RLock()
_, stillTracked := m.relayClients[serverAddr]
m.relayClientsMutex.RUnlock()
require.True(t, stillTracked, "an in-progress relay dial must not be evicted by cleanup")
// Release the hanging dial so the goroutine can exit cleanly.
mCancel()
select {
case <-dialDone:
case <-time.After(5 * time.Second):
t.Fatal("openConnVia did not return after context cancellation")
t.Fatal("foreign OpenConn did not return after context cancellation")
}
}

View File

@@ -3,7 +3,6 @@ package client
import (
"context"
"net"
"net/netip"
"sync"
"testing"
"time"
@@ -13,13 +12,16 @@ import (
// stallingRelayListener accepts TCP connections and holds them open without ever
// responding, so a relay handshake dialed against it blocks until its context is
// cancelled. It returns the "rel://host:port" URL to dial.
func stallingRelayListener(t *testing.T) string {
// cancelled. accepted is signalled once per incoming connection so a caller can
// wait until a dial has actually reached the listener. It returns the
// "rel://host:port" URL to dial.
func stallingRelayListener(t *testing.T) (string, <-chan struct{}) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
accepted := make(chan struct{}, 1)
var mu sync.Mutex
var conns []net.Conn
go func() {
@@ -31,6 +33,10 @@ func stallingRelayListener(t *testing.T) string {
mu.Lock()
conns = append(conns, c)
mu.Unlock()
select {
case accepted <- struct{}{}:
default:
}
}
}()
t.Cleanup(func() {
@@ -42,14 +48,14 @@ func stallingRelayListener(t *testing.T) string {
mu.Unlock()
})
return "rel://" + ln.Addr().String()
return "rel://" + ln.Addr().String(), accepted
}
// TestRelayStates_DoesNotBlockOnRealHangingDial is a regression test for
// RelayStates() called by a "status -d command" hanging behind an in-progress
// relay dial.
// foreign relay dial.
func TestRelayStates_DoesNotBlockOnRealHangingDial(t *testing.T) {
serverAddr := stallingRelayListener(t)
serverAddr, accepted := stallingRelayListener(t)
mCtx, mCancel := context.WithCancel(context.Background())
t.Cleanup(mCancel)
@@ -59,15 +65,14 @@ func TestRelayStates_DoesNotBlockOnRealHangingDial(t *testing.T) {
dialDone := make(chan struct{})
go func() {
defer close(dialDone)
_, _ = m.openConnVia(mCtx, serverAddr, "peerKey", netip.Addr{})
_, _ = m.foreign.OpenConn(mCtx, "peerKey", RelayServer{Addr: serverAddr})
}()
require.Eventually(t, func() bool {
m.relayClientsMutex.RLock()
defer m.relayClientsMutex.RUnlock()
_, ok := m.relayClients[serverAddr]
return ok
}, 5*time.Second, 5*time.Millisecond, "relay dial did not start")
select {
case <-accepted:
case <-time.After(5 * time.Second):
t.Fatal("relay dial did not reach the listener")
}
done := make(chan []RelayConnState, 1)
go func() {
@@ -86,6 +91,6 @@ func TestRelayStates_DoesNotBlockOnRealHangingDial(t *testing.T) {
select {
case <-dialDone:
case <-time.After(5 * time.Second):
t.Fatal("openConnVia did not return after context cancellation")
t.Fatal("foreign OpenConn did not return after context cancellation")
}
}

View File

@@ -3,7 +3,6 @@ package client
import (
"context"
"io"
"net/netip"
"testing"
"time"
@@ -85,7 +84,7 @@ func TestManager_ForeignRelayServerIP(t *testing.T) {
t.Run("no server IP, dial fails", func(t *testing.T) {
dialCtx, dialCancel := context.WithTimeout(ctx, 5*time.Second)
defer dialCancel()
_, err := mgrAlice.OpenConn(dialCtx, brokenFQDN, "bob", netip.Addr{})
_, err := mgrAlice.OpenConn(dialCtx, RelayServer{Addr: brokenFQDN}, "bob", true)
if err == nil {
t.Fatalf("expected OpenConn to fail without server IP, got success")
}
@@ -95,7 +94,7 @@ func TestManager_ForeignRelayServerIP(t *testing.T) {
// Bob waits for Alice's incoming peer connection on his side.
bobSideCh := make(chan error, 1)
go func() {
conn, err := mgrBob.OpenConn(ctx, bobRealAddr, "alice", netip.Addr{})
conn, err := mgrBob.OpenConn(ctx, RelayServer{Addr: bobRealAddr}, "alice", false)
if err != nil {
bobSideCh <- err
return
@@ -113,7 +112,7 @@ func TestManager_ForeignRelayServerIP(t *testing.T) {
bobSideCh <- nil
}()
aliceConn, err := mgrAlice.OpenConn(ctx, brokenFQDN, "bob", bobAdvertisedIP)
aliceConn, err := mgrAlice.OpenConn(ctx, RelayServer{Addr: brokenFQDN, IP: bobAdvertisedIP}, "bob", true)
if err != nil {
t.Fatalf("alice OpenConn with server IP: %s", err)
}

View File

@@ -3,7 +3,6 @@ package client
import (
"context"
"fmt"
"net/netip"
"testing"
"time"
@@ -106,11 +105,11 @@ func TestForeignConn(t *testing.T) {
if err != nil {
t.Fatalf("failed to get relay address: %s", err)
}
connAliceToBob, err := clientAlice.OpenConn(ctx, bobsSrvAddr, "bob", netip.Addr{})
connAliceToBob, err := clientAlice.OpenConn(ctx, RelayServer{Addr: bobsSrvAddr}, "bob", true)
if err != nil {
t.Fatalf("failed to bind channel: %s", err)
}
connBobToAlice, err := clientBob.OpenConn(ctx, bobsSrvAddr, "alice", netip.Addr{})
connBobToAlice, err := clientBob.OpenConn(ctx, RelayServer{Addr: bobsSrvAddr}, "alice", false)
if err != nil {
t.Fatalf("failed to bind channel: %s", err)
}
@@ -210,7 +209,7 @@ func TestForeginConnClose(t *testing.T) {
if err != nil {
t.Fatalf("failed to serve manager: %s", err)
}
conn, err := mgr.OpenConn(ctx, toURL(srvCfg2)[0], "bob", netip.Addr{})
conn, err := mgr.OpenConn(ctx, RelayServer{Addr: toURL(srvCfg2)[0]}, "bob", true)
if err != nil {
t.Fatalf("failed to bind channel: %s", err)
}
@@ -302,7 +301,7 @@ func TestForeignAutoClose(t *testing.T) {
}
t.Log("open connection to another peer")
if _, err = mgr.OpenConn(ctx, foreignServerURL, "anotherpeer", netip.Addr{}); err == nil {
if _, err = mgr.OpenConn(ctx, RelayServer{Addr: foreignServerURL}, "anotherpeer", true); err == nil {
t.Fatalf("should have failed to open connection to another peer")
}
@@ -372,7 +371,7 @@ func TestAutoReconnect(t *testing.T) {
if err != nil {
t.Errorf("failed to get relay address: %s", err)
}
conn, err := clientAlice.OpenConn(ctx, ra, "bob", netip.Addr{})
conn, err := clientAlice.OpenConn(ctx, RelayServer{Addr: ra}, "bob", false)
if err != nil {
t.Errorf("failed to bind channel: %s", err)
}
@@ -392,7 +391,7 @@ func TestAutoReconnect(t *testing.T) {
}
log.Infof("reopent the connection")
_, err = clientAlice.OpenConn(ctx, ra, "bob", netip.Addr{})
_, err = clientAlice.OpenConn(ctx, RelayServer{Addr: ra}, "bob", false)
if err != nil {
t.Errorf("failed to open channel: %s", err)
}
@@ -454,7 +453,7 @@ func TestNotifierDoubleAdd(t *testing.T) {
t.Fatalf("failed to serve manager: %s", err)
}
conn1, err := clientAlice.OpenConn(ctx, clientAlice.ServerURLs()[0], "bob", netip.Addr{})
conn1, err := clientAlice.OpenConn(ctx, RelayServer{Addr: clientAlice.ServerURLs()[0]}, "bob", false)
if err != nil {
t.Fatalf("failed to bind channel: %s", err)
}