Each pass re-diffed the whole map against the peer store to find its next
batch: a key slice and set for removals, a parse of every peer's allowed
IPs plus an FQDN write for modifications, and a lookup per peer for
additions. On a 10k-peer map that is roughly 34 full walks for one update.
The diff now runs on the first pass and produces a plan, and every pass
takes up to the cap from it, so a pass costs its batch. The cap now bounds
removals, modifications and additions together, which is what bounds the
time syncMsgMux is held.
Every bounded pass re-ran updateConfig, which notifies the address
listener and logs at info level when SSH is disabled, and
FinishPeerListModifications, which snapshots every peer for the router
subscribers and wakes the UI. Both describe the map, not a batch: the
config now applies on the first pass and the notification goes out when
the map converges, or for what was applied when a pass fails.
Offline peers, SSH host keys, the SSH client config and the SSH server auth
ran on every bounded pass over the whole map. Each pass rewrote the SSH
client config file, and recording host keys for peers not added yet logged
a warning per missing peer. Offline peers now apply with the wholesale
sections on the first pass, and the SSH steps on the converging pass, once
every peer is in the store. The per-pass peer diff stays: it is how each
pass finds the next batch.
The doc comment said prev is ignored because updates are full maps, while
the body fills components the update omits from a pending prev. The inner
comment also repeated the same paragraph twice.
The later pass ran with no pending map, so it showed only that the update
was not decoded, not that the map decoded on the first pass gets applied.
It now starts from a pending map and checks the pass consumes it.
The first pass of a new target replaced syncTargetMap before the serial
check in updateNetworkMap ran. An outdated map arriving mid-convergence
took the place of the newer one, was then ignored as stale and reported as
converged, so the newer map's remaining peer batches were never applied
until the next update. A stale map now leaves the pending one in place.
Main moved applyInfoFlags into syncInfoFunc, which handles a refresh that
returned no info. The merge kept the old call in receiveManagementEvents,
so a refresh cut short by shutdown handed it a nil info and the client
panicked; Client Unit on Darwin and Windows caught it.
Every bounded pass published "Network map updated", so a large map pushed
dozens of identical events to the UI for a single update. The event now
fires from the converging pass, next to the lazy-exclude finalize. A map
ignored as outdated no longer publishes it either, since nothing was
applied.
Decoding a components envelope runs Calculate(), the most expensive step
of a sync, and it ran again on every bounded pass of the same target: about
34 full recalculations for a 10k-peer map at the current per-pass cap, all
under syncMsgMux. The decode and the other per-target steps now run on the
first pass, and the passes that drain the peer batches reuse the map.
Envelope syncs leave NetworkMap nil and ship the map in NetworkMapEnvelope,
so the nil-map checks read every one of them as config-only: they were never
persisted, and a config-only update coalescing over a pending envelope sync
dropped its map. The version, top-level peer config and checks travel with
the envelope, and a nil session deadline is "no change" like the other
optional fields, so the pending one is carried forward too.
The manager ran on the engine context, so after Sync returned it stayed
alive until the engine was cancelled. A caller waiting on shutdownWg before
cancelling, as TestEngine_FirstSyncInfoCarriesLoginChecks does, deadlocked.
The run loop also only looked at the context when idle, so a cancel landing
mid-convergence kept draining the remaining passes.
Main removed the ingress forward rules, so the forwardingRules stash that
carried them to the lazy-exclude finalize is gone with them. The envelope
decode and latestComponents retention move into applySyncPass; persisting
stays with the map-state manager.
* [client] Reproduce Signal reconnect listener shutdown deadlock
* [client] Invoke Signal reconnect listeners outside the client mutex
Capture the listener with the registration state, then release the mutex before calling it. Watcher shutdown can now unregister a listener while an in-flight callback waits for the watcher lock.
Keep callback invocation synchronous and document that unregistration does not drain an already captured callback. Record callback completion inside the regression test's external lock.
* [management] Return the reason when custom domain validation fails
The validate endpoint returned an empty 202 and ran the check in a
goroutine, so a failed verification only reached the management log.
Run the check in the request and return 204 or 412 with a classified
reason. Resolver errors are not forwarded.
* [management] Address review feedback on custom domain validation
Pass the request context to the lookup and store calls now that the
check runs in the request, compare the self-resolved name
case-insensitively, document the 412 like the other entries, and
check the returned error in the existing tests.
* [management] Match custom domain CNAME targets case-insensitively
DNS names are case-insensitive, so a CNAME pointing at the cluster in
different casing should validate instead of failing as a mismatch.
* [management] Add validation timeout
---------
Co-authored-by: DevBot NetBird <91386968+netbirddev@users.noreply.github.com>
StdNetBind.Send reused pooled *net.UDPAddr objects and copied the
destination into ua.IP before resizing the slice. After an IPv4 send the
slice was 4 bytes long, so the next IPv6 send copied only 4 bytes and
extended the slice with stale data, misdirecting datagrams to a truncated
or foreign IPv6 address. Userspace clients with both IPv4 and IPv6 P2P
peers saw intermittent loss and restarting handshakes.
The fork now carries the upstream fix (WireGuard/wireguard-go#135,
netbirdio/wireguard-go#24). Pick it up.
* [client] Keep interface discovery out of the bind open path
Since 0.81.0 a userspace-mode agent whose interface name is not covered
by the interface blacklist deadlocks while bringing the device up, and
the daemon, the GUI and the service stop all hang. wireguard-go holds
the device net lock across bind.Open, our bind builds the UDP mux there,
the mux enumerated interfaces in its constructor, and the interface
filter probed every unlisted name through wgctrl. For the agent's own
interface that probe connected to the daemon's own UAPI endpoint and
waited, with no deadline, for a reply the handler could only give after
taking the same lock. #7346 moved the first enumeration from the stdnet
constructor, which ran before the device existed, to this call site.
The mux no longer discovers interfaces when it is constructed. It
records whether its socket is bound to an unspecified address and
GetConn uses that instead of the length of the discovered list, while
GetListenAddresses keeps refreshing on demand, which pion calls during
candidate gathering outside the device locks. The WireGuard detector
also answers for the agent's own interface without probing, so no
filter asks the daemon about itself.
* [client] Take the agent's own interface name in the detector constructor
Every device implementation reports the configured name, so there is no
later name to follow and the setter was state without a reader that
needed it. The detector takes the name once and compares without a lock.
* [client] Create the default mux Net in the constructor
With discovery moved to GetListenAddresses, the nil Net fallback ran
there as well, and the first concurrent callers raced on writing it.
stdnet.NewNet does no discovery, so the constructor can create it before
the mux is shared and the address refresh only reads it.
* [client] Follow the created interface name in the detector
Android ignores the configured interface name and keeps the TUN name
returned by CreateUnmonitoredTUNFromFD, so the detector built from the
configuration would still probe the agent's real interface. The engine
updates the detector with wgInterface.Name() once the interface exists.
* [client] Keep the detector's own interface name immutable
On Android the kernel names the TUN and the name can change again on
renew, while the UAPI socket cannot be opened there at all, so neither
the probe nor a name set after creation can identify the agent's own
interface reliably. Excluding that interface on Android belongs to the
interface discovery the app provides. Everywhere else the configured
name is the created name, so the constructor argument is enough and the
detector needs no setter and no lock on the hot path.
* [proxy] Add --external-ip-map flag for the embedded client
The proxy behind a NAT with a forwarded fixed --wg-port could not advertise
its public address as an ICE candidate, so peers fell back to relay. Expose
the agent's external IP map through embed.Options and a proxy flag
(NB_PROXY_EXTERNAL_IP_MAP).
* [proxy] Validate --external-ip-map at startup
The engine drops every external IP mapping when one fails to parse, so a
typo or a space after a comma silently disabled the feature. Move the
agent's validator to shared/externalipmap, run it from the proxy at
startup, and trim entries first.
* [client,proxy] Require an external IP first in --external-ip-map
The engine reads the first component as the external IP, so a mapping like
iface/10.0.0.5 passed validation and then dropped every mapping.
The Linux network monitor skipped every route update whose Dst was not
nil, treating a nil Dst as the default route. Since the bump to
vishvananda/netlink v1.3 (v0.43.0) the default route arrives as an
explicit 0.0.0.0/0 or ::/0, so every default route change was filtered
out and the monitor never fired on Linux, no matter how the underlay
moved.
Treat both a nil Dst and a zero-length prefix as the default route.
* [client] Do not move an HA route on latency before both peers have a sample
A direct connection reports no latency until the first sample of its new ICE
agent, and a drop to relay clears the previous one. Route selection ranked
such a peer at unknownLatency, so a routing peer whose direct connection came
back within the relayed switch delay lost the route to an equivalent peer on
a "lower latency" switch right after recovering, resetting every masqueraded
flow the delay was meant to keep alive. The sample usually lands before the
evaluation, but nothing orders the two, and a pair with no round trip time
yet postpones it to the next sampling tick.
Latency-only switches now require a sample on both peers. Categorical
criteria (connected, metric, direct versus relayed) are unaffected.
* [client] Hold a never-direct HA routing peer before a direct peer takes over
The peers of an HA group come up over relay first and turn direct one after
the other, as when lazy connections wake the group or the client reconnects.
A routing peer that was relayed when it carried the route gave way at once
to the first equivalent peer that turned direct, so in a group of N peers
the route moved right after the session started in roughly (N-1)/N of the
cases, resetting every masqueraded flow, typically an SSH session opened on
the wake-up traffic.
Such a peer is now kept for the relayedSwitch delay, counted from the first
evaluation that finds a direct contender. If it turns direct within the
delay the route stays; otherwise it moves when the delay runs out, so the
route does not stay on a relay for longer than the delay. A peer that
dropped from direct keeps counting from the drop.
* [client] Keep allowed IPs when only the chosen routing peer's status changes
recalculateRoutes did not tell a status change of the chosen routing peer
(idle to connected when traffic wakes it, connected to relayed and back) from
a switch to another peer: it removed the allowed IPs and added them back on
the same peer. In between the prefix had no owner and packets for it were
dropped, and default routes published a spurious "disconnected due to high
availability change" event followed by two "connected" ones.
When the same route stays on the same peer with the same definition, only
the recorded status is updated now.
* [client] Limit the "never" relayed switch policy to peers that dropped from direct
NB_HA_RELAYED_SWITCH_DELAY=never keeps a routing peer whose direct
connection dropped to relay for as long as it stays connected. Once the
relayed delay also covered peers that were never direct, "never" pinned
those to the relay for good, although a direct equivalent was available
and the peer showed no sign of turning direct.
A peer that was never direct now uses the default delay under "never",
which keeps the policy's original scope.
* [client] Merge the unchanged-route checks of the HA route watcher
shouldSkipRecalculation and the status-only fast path in recalculateRoutes
both handled the same route staying on the same routing peer, one for an
unchanged status and one for a changed one. They are now a single
keepChosenRoute check.
Pure refactor, no behavior change.
* [client] Re-apply routed allowed IPs when a kept routing peer connects
A peer update can recreate a routing peer: modifyPeers removes it from the
interface, which drops its routed prefixes, and adds it back idle with its
overlay addresses only. The remove and re-add that recalculateRoutes used
to run on every status change of the chosen peer put the prefixes back as
a side effect once the peer connected. Without it, a watcher that sees the
same peer go idle and then connected leaves the routed subnet without an
owner until a route update or an HA switch.
When the chosen peer connects while keeping the route, the watcher now
re-applies its routed allowed IPs through the route manager's reconcile,
which is add-only and idempotent. Lazy connections already do the same when
they arm an idle peer.
* [client] Bound the HA route hold on a routing peer without a latency sample
A latency-only switch waited for a sample on both peers. If the current
peer never got one, because its candidate pair stats kept failing or
reporting no round trip time, it kept the route indefinitely against a
measured peer, however much faster that one was.
The hold on an unsampled current peer now lasts unmeasuredLatencyHold
(30s), long enough for a recovered direct connection to report its first
sample. After that the peer is ranked at unknownLatency again and the usual
switch margins apply. relayedHold is armed so the switch is reconsidered
when the hold runs out.
* [client] Restart the relayed delay of a never-direct peer when its contender goes
For a routing peer that was never direct, the relayed delay runs from the
first evaluation that finds a direct contender. It was only reset when the
current peer changed or turned direct, so if that contender went away and
another one showed up later, the delay counted from the first one and the
new contender took over at once.
The delay of such a peer now restarts once no direct contender is left.
The delay of a peer that dropped from direct keeps running from the drop.
* [client] Retrigger pull request synchronization
* [client] Retry a failed routed allowed IPs reconcile on the next evaluation
keepChosenRoute recorded the connected status before re-applying the routed
allowed IPs and only logged a failed reconcile. Later evaluations then saw
an unchanged status and never retried, so a transient AddAllowedIP failure
left the recreated routing peer without its routed prefixes until another
status transition or a route switch.
The status is now recorded only once the reconcile succeeds. A failure
leaves the previous status in place, so the next evaluation, at the latest
the periodic re-evaluation, sees the change again and retries.
* [infrastructure] Restrict the FreeBSD port job to stable releases
The port scripts picked the target version by scraping the GitHub tags
page and only dropped tags containing "rc". A canary tag such as
v0.81.0-canary.pr-8098.1 sorts above the latest stable release, so the
job tried to package a prerelease, and the page only lists ten tags, so
a run of prereleases could leave no candidate at all. Both scripts now
share one helper that reads the tags API, keeps MAJOR.MINOR.PATCH
versions only, and rejects an explicitly requested prerelease. Go proxy
downloads fail instead of hashing an error page.
The job moves out of the release workflow into its own, so it runs on
pushes to main, on release tags, and on pull requests that touch the
scripts or the workflow itself. Prerelease tags skip the job, and a
stable tag is packaged directly rather than whatever happens to be the
latest release at the time.
* [infrastructure] Keep the FreeBSD port job in the release workflow
A separate workflow would rename the check and detach the job from the
release run, so the job stays in release.yml behind a small trigger job
that decides when it runs: pushes to main, stable release tags, and pull
requests that touch the port scripts or the workflow. Prerelease tags
and other branches skip it.
Tags are now listed with git ls-remote, which returns every tag in one
call and needs neither a token nor pagination, and the Go module proxy
downloads are limited to HTTPS like the other fetches.
* [infrastructure] Address Sonar findings in the FreeBSD port helper
Read the version into a local and return explicitly from is_stable_version.
* [infrastructure] Address review findings in the FreeBSD port job
Only a full vMAJOR.MINOR.PATCH tag starts the port job, so a tag like
v1.2 skips it instead of failing in the resolver. The diff detection
steps use a glob array rather than parsing ls, and quote GITHUB_OUTPUT.
A failed tag listing now stops the issue body script instead of writing
an incomplete changelog, and a bare "v" argument is rejected rather than
treated as a request for the latest release.
Homebrew deprecated the postflight and uninstall_preflight cask stanzas
in favour of the declarative *_steps DSL, so every brew command that
evaluates netbirdio/tap now prints deprecation warnings. Once the
deprecation becomes a disable the generated cask stops loading and
netbird-ui can no longer be installed or upgraded through Homebrew.
The *_steps blocks take JSON-serialisable steps run in a sandbox rather
than arbitrary Ruby, so system_command is re-expressed as run/remove.
The two postflight blocks merge into one because a cask carries only a
single instance, preserving the original order. set_permissions moves
from a hardcoded /Applications to base: :appdir, matching what the
installer invocation already did. The launchctl fallbacks keep their
tolerant semantics through must_succeed: false, and remove is a no-op
when the plist is absent.
Conn.Close previously called onDisconnected only when evalStatus
reported StatusConnected. When a connection drops or closes from
a connecting or disconnecting state, the callback was skipped.
This left registered listeners such as Rosenpass unaware of the
teardown, keeping peer workers active and leaking sockets.
Invoke onDisconnected unconditionally on Conn.Close if registered.
* [client] Move the Signal registration wait into its own helper
Pure move, no behavior change. The wait for the registration header and
the check on its value leave connect() for awaitRegistration(), so the
next commit can bound that wait in one place.
* [client] Split the Signal test server helper from its listener setup
Pure move, no behavior change. serveTestSignalExchange serves any
SignalExchangeServer on a local listener; startTestSignalServer keeps
serving the real server through it. Lets a test wrap the real server.
* [client] Bound the wait for the Signal registration header
After an outage the Signal client could wait forever for the registration
header of a new stream. Keepalive cannot end that wait when whatever
answers the transport (a proxy in front of Signal, or Signal itself)
accepts the stream and never answers it, and the receive watchdog only
starts once registration succeeds. Signal then stayed down until the
daemon was restarted, while Management and Relay recovered.
Cancel the stream when the header does not arrive within 10 seconds, so
connect() fails and the existing retry loop opens a new stream. A timer
is used instead of a context deadline because the stream context must
outlive registration.
* [client] Report Signal unhealthy while no stream is registered
IsHealthy judged Signal by the gRPC connection state alone and marked it
connected whenever the connection was Ready or Idle. With the connection
up but no stream registered, the client cannot receive offers or answers,
yet netbird status showed Signal: Connected, hiding a stuck registration
for as long as it lasted.
Require a registered stream before reporting healthy. When there is none,
return false without touching the recorder: the disconnect that dropped
the stream already reported its reason.
* [client] Name the server that reconnected in the SR watcher log
Signal and Relay shared one reconnect callback, which logged "reconnected
to Signal or Relay server" for either. A Relay reconnect while the Signal
stream was down read as Signal having recovered, which misled diagnosis
of a stuck Signal registration.
Register one callback per server so the log names the one that actually
reconnected. Which listeners get notified, and when, is unchanged.
* [client] Trim comments in the Signal registration and health code
Comment-only change, no behavior change. Drop comments that restate the
code and keep only the reason a timer is used instead of a context
deadline.
* [client] Check Signal stream registration before the connection state
IsHealthy returned true for a Connecting or Shutdown channel before the
registration check ran, so a client with no registered stream could
still be reported healthy while the channel was reconnecting. Check the
stream first; the connection-state cases now only apply to a client
that has a registered stream.
* [client] Log a plain connect in the SR watcher
The Signal and Relay callbacks also fire on the first connection, not
only after a disconnect, so "reconnected" was wrong at startup. Log
"connected to <server> server" for both cases.
* [client] Say what the SR watcher does when it logs
The line was read as a per-server connect log, which Signal and Relay
already emit themselves. What it marks is the peer guards being told to
reset their reconnection backoff, which happens only once both are
ready. Log that, naming the server whose event triggered it.
* [management] Add a release-wired UBI image variant
OpenShift and other Red Hat environments expect UBI-based images that run
as an arbitrary non-root UID. The proxy, combined server and rootless
client already ship -ubi variants; this adds the same for management,
published as <version>-ubi and ubi-latest for amd64 and arm64.
The default --port of 80 cannot bind without root, so the image defaults
to 8080. For Let's Encrypt, public 443 forwards to --port and
--letsencrypt-listen-address is empty (#7706), so the main listener
answers the TLS-ALPN challenges.
* [management] Certify the management UBI image
Adds management to the Red Hat certification components. Its Partner
Connect component ID goes in the REDHAT_CERT_ID_MANAGEMENT repository
variable.
* [relay] Add a release-wired UBI image variant
* [relay] Align the UBI image with the shared UBI pattern
Main now ships UBI variants of the client, combined server and proxy that
share one license collector and one image layout, and the Red Hat
certification workflow expects every certified image to carry both amd64
and arm64. The relay variant predated that, kept its own copy of the
license script, built amd64 only, and ran as UID 65532 on the privileged
default port :443, so it would not start under OpenShift or rootless Podman.
Use release_files/collect-licenses.sh and build amd64 and arm64 like the
other UBI images. Run as 1000:0 with a group-0 writable /var/lib/netbird
for Let's Encrypt data, and default the listener to :8443 so an arbitrary
non-root UID can bind it; the relay's Let's Encrypt flow uses TLS-ALPN on
that same listener, so no separate challenge port is needed.
* [infrastructure] Certify the relay UBI image on release
Releases certify the UBI images through redhat-certify.yml, which only
lists the client, proxy and combined server, so a released relay UBI
image would never reach the Red Hat catalog.
Add relay to the component list and the manual dispatch options. Its
Partner Connect ID comes from REDHAT_CERT_ID_RELAY, which must exist
before this merges: the resolve step fails the whole run when any
component's variable is missing.
* [signal] Add an explicit non-root UBI image variant
Keep the existing signal image unchanged while making a separate UBI image available for local certification-readiness checks. Collect the linked Go dependency license terms as portable build inputs, and use SIGINT for the existing graceful stop handler.
* [signal] Wire the UBI variant into GoReleaser
Build the additive UBI signal variant through the existing release pipeline instead of relying on local Docker commands. Stage its licenses in the per-image build context and restrict the variant to validated AMD64 without changing existing image tags.
* [signal] Align the UBI image with the other UBI variants
Main now ships UBI images for the client, combined server and proxy with
a shared license collector and a common shape, so the signal variant
should look the same to reviewers and to Red Hat certification. Signal
also listens on port 80 by default, which an arbitrary non-root UID
cannot bind on OpenShift or Podman.
Use release_files/collect-licenses.sh instead of a signal-only copy,
build for amd64 and arm64 like the other UBI entries, and run as
1000:0 with a group-writable /var/lib/netbird that also holds Let's
Encrypt data. Default NB_PORT to the legacy gRPC port 10000 so the
image starts unprivileged and serves a single listener.
* [infrastructure] Certify the signal UBI image on release
The signal UBI image is published with every release, but only images
listed in the certification workflow reach the Red Hat catalog, so it
would otherwise need a manual run each time.
Add signal to the dispatch options and the component list, using the
same -ubi tag suffix as .goreleaser.yaml. The run reads its Partner
Connect ID from REDHAT_CERT_ID_SIGNAL.
* [signal] Answer Let's Encrypt challenges on the UBI image's port
The challenge listener defaults to :443, which a non-root UID cannot
bind on OpenShift or Podman, so Let's Encrypt could not work in the
UBI image. Now that the listener address is configurable, default it
to empty so the TLS listener on port 10000 answers the challenges
itself when public port 443 is forwarded to it.
The Windows installer shows LICENSE on its license page, where the
2022 copyright line reads as outdated. Extend the range to 2022-2026
in LICENSE, its REUSE copy, and the installer's LegalCopyright, keeping
2022 as the year of first publication.
* [client] Fix Windows UI exiting on first launch
Bump the Wails fork to pick up netbirdio/wails#6, which stops
SetBackgroundColour from dereferencing a WebView2 controller that does
not exist yet. On first launch the theme service re-tints the Welcome
window while it is still being created, and losing that race made the
UI exit with no window.
* [client] Point the Wails fork at the merged netbirdio/wails#6
Replace the temporary pin on emrcbrn/wails with the merge commit of
netbirdio/wails#6 on the fork's integration branch.
Reaching an internal service meant installing the agent, which needs root
for a TUN device. nblink runs the embedded client in netstack mode and
forwards local HTTP listeners to upstreams inside the tunnel, so it works on
managed machines and in rootless containers. Upstream names resolve
in-process.
Forwards bind loopback unless --allow-public-bind is given. The listener
refuses rebound and cross-site browser requests and bounds stalled request
bodies. It ships as release archives and a distroless image that runs under
an arbitrary UID.
An embedded client given no StatePath now keeps its state in memory rather
than falling back to the installed agent's state file, which every embedded
client in a proxy process also shared. The state manager gets an explicit
in-memory store for it.
With the embedded IdP, management.json can leave out the HttpConfig
section, and ApplyEmbeddedIdPConfig creates it later. Command-line
overrides run before that, so --letsencrypt-domain, or --cert-file with
--cert-key, dereferenced a nil HttpConfig and management crashed at
startup.
Create the section only when one of those flags writes to it, so a
config without TLS flags keeps HttpConfig unset.
Verify access decisions through real management and proxy containers, including upstream delivery and denial. Cover inherited auth, bypass/block, precedence, credential stripping, updates, restrictions, deletion, and private-service rejection.
Validation: container E2E and scoped lint passed locally. The workflow builds both components from the checkout.
Signal can report a successful start and then stop with exit code 0, or
continue without a listener, when a listener fails. Supervisors and
operators do not see the failure.
Stop at startup when the Let's Encrypt challenge listener cannot bind.
Keep a failure that occurs before the run loop waits, and return a
non-zero exit code after a server failure. A listener that is closed
during shutdown is not a failure.
riccardom