* [client] categorize root/system-mutating tests behind a privileged build tag
Tests that need root or mutate host state (nftables/iptables/DNS, TUN/WireGuard
interfaces, routes, eBPF, SSH/service install) are now gated behind a
//go:build privileged tag. The default `go test ./client/...` runs as a non-root
user with no sudo and leaves host networking untouched; mixed files were split so
pure-logic tests stay in the default suite.
A self-hosting ory/dockertest/v4 harness (client/testutil/privileged) runs the
privileged suite inside a --privileged --cap-add=NET_ADMIN container via
`make test-privileged`; a DOCKER_CI=true guard skips the spawn when already inside
the container. Added `make test-unit` for the host-safe run.
* [client] add PRIV_RUN/PRIV_PKGS filters to the privileged test harness
The dockertest harness now reads two optional env vars when building the
in-container `go test` command: PRIV_RUN adds a -run test-name filter and
PRIV_PKGS overrides the package list. Both empty reproduce the full privileged
suite, so CI and `make test-privileged` behave as before. Lets a developer run a
single privileged test in the container, e.g.:
PRIV_RUN=TestNftablesManager PRIV_PKGS=./client/firewall/nftables/... make test-privileged
* [client] fix unused-helper lint after the privileged test split
Splitting privileged tests into *_privileged_test.go left their shared helpers in
the untagged files, so in the default (no-tag) build they had no callers and
golangci-lint flagged them as unused.
Moved the privileged-only helpers into the privileged files next to their callers
(generateDummyHandler; createEngine/startSignal/startManagement/getConnectedPeers/
getPeers + kaep/kasp; (*mockDaemon).setJWTToken). Annotated the shared routing-test
fixtures that must stay untagged for cross-platform compilation with //nolint:unused
(systemops_bsd expected* vars, ensureIPv6DefaultRoute on bsd/windows,
loopbackIfaceWindows), matching the existing linux variant.
* [client] fix privileged test CI failures and run the harness on macOS
The host-safe unit run dropped sudo but two privileged test groups were
never tagged, and the Docker privileged job silently never ran the suite:
- Gate the ssh/server PrivilegeDropper command-construction tests behind
the privileged tag (they require root to target a different UID); split
them into executor_unix_privileged_test.go.
- Tag sharedsock raw-socket tests privileged (need CAP_NET_RAW).
- Fix the Docker job command: nested single quotes around the build tags
closed the sh -c wrapper early, dropping the go list package set and the
privileged tag, so go test ran on the empty repo root. Use double quotes.
Make the self-hosting harness usable from a dev Mac:
- Build it on darwin as well as linux; it only drives Docker.
- Resolve the active docker context endpoint into DOCKER_HOST when the
default /var/run/docker.sock is absent (Docker Desktop, Colima, OrbStack).
- Rename the misspelled containerGoModache constant to containerGoModCache.
* Update client/internal/engine_privileged_test.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update client/internal/routemanager/systemops/systemops_linux_test.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update client/internal/routemanager/systemops/systemops_windows_test.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update client/server/server_privileged_test.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* [ci] Run privileged-tagged tests on darwin, windows and freebsd
The privileged build tag split moved root/system-mutating tests behind
//go:build privileged, but only the linux docker job was given the tag.
The native darwin (sudo), windows (PsExec64 -s) and freebsd VM runners
already have the required privileges, so add the privileged tag there too
to keep CI running the same set of tests as before the split.
* [ci] Exclude dockertest harness from the darwin privileged run
The privileged tag now compiles client/testutil/privileged on darwin, whose
TestRunPrivilegedSuiteInDocker spawns a container the macOS runner has no
Docker for. Exclude the harness package from the darwin list, matching the
linux job, so the privileged tests run in place without a container spawn.
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* [client] Index peer tunnel IPs for O(1) PeerStateByIP lookup
Replace the linear scan over all peers with an ipToKey map maintained
by AddPeer/RemovePeer, covering both IPv4 and IPv6 tunnel addresses.
Offline peers are intentionally no longer resolvable by IP: only active
peers can carry traffic, so IdentityForIP and the DNS disconnected-peer
filter now treat them as unknown, same as foreign IPs.
Skip the DNS answer filter for single-record responses; dropping the
only answer was always restored by the empty-answer escape hatch, so
the fast path is behavior-neutral.
* Ensure `ipToKey` entries are only removed if they match the peer being deleted, preventing accidental removal of unrelated mappings.
Adds a new "private" service mode for the reverse proxy: services reachable exclusively over the embedded WireGuard tunnel, gated by per-peer group membership instead of operator auth schemes.
Wire contract
- ProxyMapping.private (field 13): the proxy MUST call ValidateTunnelPeer and fail closed; operator schemes are bypassed.
- ProxyCapabilities.private (4) + supports_private_service (5): capability gate. Management never streams private mappings to proxies that don't claim the capability; the broadcast path applies the same filter via filterMappingsForProxy.
- ValidateTunnelPeer RPC: resolves an inbound tunnel IP to a peer, checks the peer's groups against service.AccessGroups, and mints a session JWT on success. checkPeerGroupAccess fails closed when a private service has empty AccessGroups.
- ValidateSession/ValidateTunnelPeer responses now carry peer_group_ids + peer_group_names so the proxy can authorise policy-aware middlewares without an extra management round-trip.
- ProxyInboundListener + SendStatusUpdate.inbound_listener: per-account inbound listener state surfaced to dashboards.
- PathTargetOptions.direct_upstream (11): bypass the embedded NetBird client and dial the target via the proxy host's network stack for upstreams reachable without WireGuard.
Data model
- Service.Private (bool) + Service.AccessGroups ([]string, JSON- serialised). Validate() rejects bearer auth on private services. Copy() deep-copies AccessGroups. pgx getServices loads the columns.
- DomainConfig.Private threaded into the proxy auth middleware. Request handler routes private services through forwardWithTunnelPeer and returns 403 on validation failure.
- Account-level SynthesizePrivateServiceZones (synthetic DNS) and injectPrivateServicePolicies (synthetic ACL) gate on len(svc.AccessGroups) > 0.
Proxy
- /netbird proxy --private (embedded mode) flag; Config.Private in proxy/lifecycle.go.
- Per-account inbound listener (proxy/inbound.go) binding HTTP/HTTPS on the embedded NetBird client's WireGuard tunnel netstack.
- proxy/internal/auth/tunnel_cache: ValidateTunnelPeer response cache with single-flight de-duplication and per-account eviction.
- Local peerstore short-circuit: when the inbound IP isn't in the account roster, deny fast without an RPC.
- proxy/server.go reports SupportsPrivateService=true and redacts the full ProxyMapping JSON from info logs (auth_token + header-auth hashed values now only at debug level).
Identity forwarding
- ValidateSessionJWT returns user_id, email, method, groups, group_names. sessionkey.Claims carries Email + Groups + GroupNames so the proxy can stamp identity onto upstream requests without an extra management round-trip on every cookie-bearing request.
- CapturedData carries userEmail / userGroups / userGroupNames; the proxy stamps X-NetBird-User and X-NetBird-Groups on r.Out from the authenticated identity (strips client-supplied values first to prevent spoofing).
- AccessLog.UserGroups: access-log enrichment captures the user's group memberships at write time so the dashboard can render group context without reverse-resolving stale memberships.
OpenAPI/dashboard surface
- ReverseProxyService gains private + access_groups; ReverseProxyCluster gains private + supports_private. ReverseProxyTarget target_type enum gains "cluster". ServiceTargetOptions gains direct_upstream. ProxyAccessLog gains user_groups.
- Add GetSelectedClientRoutes() to the route manager that filters through FilterSelectedExitNodes, returning only active routes instead of all management routes
- Use GetSelectedClientRoutes() in the DNS route checker so deselected exit nodes' 0.0.0.0/0 no longer matches upstream DNS IPs — this prevented the resolver from switching
away from the utun-bound socket after exit node deselection
- Initialize iOS DNS server with host DNS fallback addresses (1.1.1.1:53, 1.0.0.1:53) and a permanent root zone handler, matching Android's behavior — without this, unmatched
DNS queries arriving via the 0.0.0.0/0 tunnel route had no handler and were silently dropped
Replace fmt.Sprintf("%s:%d", ip, port) with net.JoinHostPort() to
properly handle IPv6 addresses that need bracket wrapping (e.g.,
[2606:4700:4700::1111]:53 instead of 2606:4700:4700::1111:53).
Without this fix, configuring IPv6 nameservers causes "too many colons
in address" errors because Go's net.Dial cannot parse the malformed
address string.
Fixes#5601
Related to #4074
Co-authored-by: easonysliu <easonysliu@tencent.com>
* Fix DNS probe thread safety and avoid blocking engine sync
Refactor ProbeAvailability to prevent blocking the engine's sync mutex
during slow DNS probes. The probe now derives its context from the
server's own context (s.ctx) instead of accepting one from the caller,
and uses a mutex to ensure only one probe runs at a time — new calls
cancel the previous probe before starting. Also fixes a data race in
Stop() when accessing probeCancel without the probe mutex.
* Ensure DNS probe thread safety by locking critical sections
Add proper locking to prevent data races when accessing shared resources during DNS probe execution and Stop(). Update handlers snapshot logic to avoid conflicts with concurrent writers.
* Rename context and remove redundant cancellation
* Cancel first and lock
* Add locking to ensure thread safety when reactivating upstream servers
* [client] Batch macOS DNS domains across multiple scutil keys to avoid truncation
scutil has undocumented limits: 99-element cap on d.add arrays and ~2048
byte value buffer for SupplementalMatchDomains. Users with 60+ domains
hit silent domain loss. This applies the same batching approach used on
Windows (nrptMaxDomainsPerRule=50), splitting domains into indexed
resolver keys (NetBird-Match-0, NetBird-Match-1, etc.) with 50-element
and 1500-byte limits per key.
* check for all keys on getRemovableKeysWithDefaults
* use multi error
* Optimize Windows DNS performance with domain batching and batch mode
Implement two-layer optimization to reduce Windows NRPT registry operations:
1. Domain Batching (host_windows.go):
- Batch domains per NRPT
- Reduces NRPT rules by ~97% (e.g., 184 domains: 184 rules → 4 rules)
- Modified addDNSMatchPolicy() to create batched NRPT entries
- Added comprehensive tests in host_windows_test.go
2. Batch Mode (server.go):
- Added BeginBatch/EndBatch methods to defer DNS updates
- Modified RegisterHandler/DeregisterHandler to skip applyHostConfig in batch mode
- Protected all applyHostConfig() calls with batch mode checks
- Updated route manager to wrap route operations with batch calls
* Update tests
* Fix log line
* Fix NRPT rule index to ensure cleanup covers partially created rules
* Ensure NRPT entry count updates even on errors to improve cleanup reliability
* Switch DNS batch mode logging from Info to Debug level
* Fix batch mode to not suppress critical DNS config updates
Batch mode should only defer applyHostConfig() for RegisterHandler/
DeregisterHandler operations. Management updates and upstream nameserver
failures (deactivate/reactivate callbacks) need immediate DNS config
updates regardless of batch mode to ensure timely failover.
Without this fix, if a nameserver goes down during a route update,
the system DNS config won't be updated until EndBatch(), potentially
delaying failover by several seconds.
Or if you prefer a shorter version:
Fix batch mode to allow immediate DNS updates for critical paths
Batch mode now only affects RegisterHandler/DeregisterHandler.
Management updates and nameserver failures always trigger immediate
DNS config updates to ensure timely failover.
* Add DNS batch cancellation to rollback partial changes on errors
Introduces CancelBatch() method to the DNS server interface to handle error
scenarios during batch operations. When route updates fail partway through, the DNS
server can now discard accumulated changes instead of applying partial state. This
prevents leaving the DNS configuration in an inconsistent state when route manager
operations encounter errors.
The changes add error-aware batch handling to prevent partial DNS configuration
updates when route operations fail, which improves system reliability.
Per RFC 4592 section 2.2.1, wildcards should only match when the queried
name does not exist in the zone. Previously, if host.example.com had an
A record and *.example.com had an AAAA record, querying AAAA for
host.example.com would incorrectly return the wildcard AAAA instead of
NODATA.
Now the resolver checks if the domain exists (with any record type)
before falling back to wildcard matching, returning proper NODATA
responses for existing names without the requested record type.
* **New Features**
* Wildcard DNS fallback for eligible query types (excluding NS/SOA): attempts wildcard records when no exact match, rewrites wildcard names back to the original query, and rotates responses; preserves CNAME resolution.
* **Tests**
* Vastly expanded coverage for wildcard behaviors, precedence, multi-record round‑robin, multi-type chains, multi-hop and cross-zone scenarios, and edge cases (NXDOMAIN/NODATA, fallthrough).
* **Chores**
* CI lint config updated to ignore an additional codespell entry.
Adds a per-instance uint64 hash to DefaultServer to detect identical merged host DNS configs (including extra domains). applyHostConfig computes and compares the hash, skips applying if unchanged, treats hash errors as a fail-safe (proceed to apply), and updates the stored hash only after successful hashing and apply.
in some cases iOS and macOS may be locked when looking for management domains during network changes
This change introduce an additional timeout on top of the context call
Two new boolean flags, SearchDomainDisabled and SkipPTRProcess, are added to CustomZone and its protobuf; they are propagated through the engine to DNS host logic. Host matching now uses SearchDomainDisabled directly, and PTR collection skips zones with SkipPTRProcess; reverse zones are initialized with SearchDomainDisabled: true.
Upgrade Go toolchain and golang.org/x/* deps to 1.24.10, standardize GitHub Actions to derive Go version from go.mod and adjust checkout ordering, raise WASM size limit to 55 MB, update FreeBSD tarball and gomobile refs, fix a few format-string/logging calls, treat usernames ending with $ as system accounts, and add Windows tests.
