* [client] Move the Signal registration wait into its own helper
Pure move, no behavior change. The wait for the registration header and
the check on its value leave connect() for awaitRegistration(), so the
next commit can bound that wait in one place.
* [client] Split the Signal test server helper from its listener setup
Pure move, no behavior change. serveTestSignalExchange serves any
SignalExchangeServer on a local listener; startTestSignalServer keeps
serving the real server through it. Lets a test wrap the real server.
* [client] Bound the wait for the Signal registration header
After an outage the Signal client could wait forever for the registration
header of a new stream. Keepalive cannot end that wait when whatever
answers the transport (a proxy in front of Signal, or Signal itself)
accepts the stream and never answers it, and the receive watchdog only
starts once registration succeeds. Signal then stayed down until the
daemon was restarted, while Management and Relay recovered.
Cancel the stream when the header does not arrive within 10 seconds, so
connect() fails and the existing retry loop opens a new stream. A timer
is used instead of a context deadline because the stream context must
outlive registration.
* [client] Report Signal unhealthy while no stream is registered
IsHealthy judged Signal by the gRPC connection state alone and marked it
connected whenever the connection was Ready or Idle. With the connection
up but no stream registered, the client cannot receive offers or answers,
yet netbird status showed Signal: Connected, hiding a stuck registration
for as long as it lasted.
Require a registered stream before reporting healthy. When there is none,
return false without touching the recorder: the disconnect that dropped
the stream already reported its reason.
* [client] Name the server that reconnected in the SR watcher log
Signal and Relay shared one reconnect callback, which logged "reconnected
to Signal or Relay server" for either. A Relay reconnect while the Signal
stream was down read as Signal having recovered, which misled diagnosis
of a stuck Signal registration.
Register one callback per server so the log names the one that actually
reconnected. Which listeners get notified, and when, is unchanged.
* [client] Trim comments in the Signal registration and health code
Comment-only change, no behavior change. Drop comments that restate the
code and keep only the reason a timer is used instead of a context
deadline.
* [client] Check Signal stream registration before the connection state
IsHealthy returned true for a Connecting or Shutdown channel before the
registration check ran, so a client with no registered stream could
still be reported healthy while the channel was reconnecting. Check the
stream first; the connection-state cases now only apply to a client
that has a registered stream.
* [client] Log a plain connect in the SR watcher
The Signal and Relay callbacks also fire on the first connection, not
only after a disconnect, so "reconnected" was wrong at startup. Log
"connected to <server> server" for both cases.
* [client] Say what the SR watcher does when it logs
The line was read as a per-server connect log, which Signal and Relay
already emit themselves. What it marks is the peer guards being told to
reset their reconnection backoff, which happens only once both are
ready. Log that, naming the server whose event triggered it.
Losing the last network only flipped the availability state: the dead management, signal and relay sockets stayed silently connected until their own timeouts, so the client kept reporting Connected with no network at all.
Introduce client/netevents with a Manager that ties the availability state, the connection sweeper and the status recorder together, and move the netstate and netsweep packages under it (netsweep renamed to sweep). SetNetworkAvailable(false) now also sweeps the registered connections so their owners redial and the listener reaches the NoNetwork state.
The Android and iOS bindings own a Manager instance and inject it through the constructors; consumers hold the concrete *Manager whose nil zero value reports always-online and never sweeps, with interfaces kept only as parameter contracts. The relay guard settle wait moved into the Manager as WaitSettled, removing the netevents import from the relay package.
On network changes the client restarted the whole engine. That is heavy-handed and slow: it tears down working state to recover from a transition the engine could handle itself. This replaces the restart with proper network event handling.
Suspend the retry loops while no network is available. Instead of burning through backoff intervals against an unreachable network, the reconnection loops park until the OS reports a usable network again.
Reconnect immediately on a network switch. When the OS hands us a new network, connections bound to the old one are swept and re-dialed right away, rather than waiting for a timeout to notice they are dead.
* [client] Wait for signal receive watchdog to stop before reconnect
The per-stream watchReceiveStream goroutine was started fire-and-forget
and never joined. On reconnect a lingering watchdog could still flip
shared client state (receiveStalled, the disconnect notifier) on the
freshly established stream, since cancelStream only cancels its own
stream context.
Track the watchdog with a WaitGroup and wait for it to exit (after
cancelling its stream) before the operation returns, so each reconnect
starts with no stale watchdog.
* [client] Bind signal receive probe to the stream context
The watchdog probe reused the generic Send, which derives its per-attempt
timeouts from the long-lived client context, so cancelStream could not
interrupt an in-flight probe. After joining the watchdog on reconnect,
watchdogWg.Wait() could then block for the full send-attempt chain.
Split Send into a context-aware send and pass the stream context down
through sendReceiveProbe, so cancelStream aborts any in-flight probe and
the watchdog exits promptly.
* [client] always clean up on Engine.Start failure via defer
The rosenpass init paths (NewManager/Run) returned without calling
e.close(), leaking the WireGuard interface and other partially
initialized state on failure. Per-branch cleanup was easy to miss when
adding new early returns.
Convert Start to a named error return and tear down via a single defer
that calls e.close() whenever err != nil, removing the scattered
per-branch close() calls (including the redundant one in initFirewall).
* [client] make Engine single-use and guard against double Start
Create the run context once in NewEngine instead of in Start. This
keeps e.cancel valid for the engine's whole lifetime, so Stop can
cancel a Start that is blocked waiting on the network while holding
syncMsgMux: Stop now cancels before taking the lock, unblocking that
Start so it can release the mutex.
Reject re-entry into Start: a non-nil wgInterface means a prior Start
already ran (ErrEngineAlreadyStarted), and a cancelled run context
means the engine was stopped (ErrEngineAlreadyStopped). Both checks run
before the cleanup defer so a duplicate call cannot tear down the
running engine's state.
* [client] let engine context unblock WaitStreamConnected
WaitStreamConnected only watched the signal client's own context, which
derives from the parent engineCtx rather than the engine's run context.
A Start blocked here (signal stream not yet up) could therefore not be
released by Engine.Stop, since Stop only cancels the engine's run
context.
Pass a context into WaitStreamConnected and select on it too, and have
the engine pass e.ctx, so Stop cancelling e.ctx unblocks a parked Start.
Update the Client interface, the mock, and callers accordingly.
* [client] fix Start/Stop race by making the run loop own engine shutdown
ConnectClient.Stop stopped the engine directly while the run loop's
backoff cycle could still be starting an engine, so Engine.close raced
Engine.Start (e.g. firewall setup reading wgInterface while close nils
it). embed.Client.Start's rollback only avoided a deadlock by cancelling
before Stop; the race itself remained and was caught by -race.
Make the run loop the sole owner of engine shutdown: derive the run
context in NewConnectClient, and have Stop cancel it and wait for the
loop to exit (skipping the wait when the loop never ran) instead of
calling engine.Stop. The loop now always stops the engine on its way
out, dropping the unsynchronised wgInterface check it used to guard that
call. Self-calls from within the loop use runCancel to avoid waiting on
themselves.
embed keeps a defensive pre-Stop cancel(); the daemon's cleanupConnection
gets a TODO to adopt Stop() rather than stopping the engine in parallel.
* [client] init context state in engine tests
Engine tests built the engine context with context.WithCancel(
context.Background()), omitting CtxInitState. Now that the run context
is created in the constructor, the wgIfaceMonitor goroutine can reach
triggerClientRestart during teardown, which calls CtxGetState and
panics on the missing state. Real entry points (up, embed, service)
always CtxInitState; only the tests skipped it.
* [client] interrupt connect backoff on context cancel
The run loop retried with a raw ExponentialBackOff, so a backoff sleep
ignored context cancellation. Now that ConnectClient.Stop waits for the
run loop to exit, a cancel landing during a sleep would block Stop for
the full interval (up to MaxInterval). Wrap the backoff with the run
context so Retry returns promptly on cancel; the retry budget itself
(MaxElapsedTime) is unchanged.
* [client] bound WaitStreamConnected in signal client tests
The tests waited on WaitStreamConnected with context.Background() and the
client's own context was also Background, so a stream that never connects
would hang until the suite timeout. Pass a 5s timeout context and assert
StreamConnected afterwards so the tests fail fast with a clear reason.
* [client] fix WaitStreamConnected stale-channel race
The StreamConnected check and the wait-channel creation took the mutex
separately, so notifyStreamConnected could set the status and close/clear
connectedCh in between: the waiter then created a fresh channel nobody
would ever close and blocked forever. Also, the status read was unlocked
while notify wrote it under the mutex (a data race). Do the check and the
channel fetch in one locked section; drop the now-unused
getStreamStatusChan helper. Pre-existing bug, not introduced by this branch.
* [client] abort Start if context cancelled while waiting for signal stream
receiveSignalEvents blocks in WaitStreamConnected until the signal stream
connects or the context is cancelled. If Stop cancelled e.ctx while Start
was parked there, Start kept going: it started the remaining subsystems on
a cancelled context and marked a shutting-down engine as started. Return
the context error from receiveSignalEvents and propagate it from Start, so
the deferred cleanup runs and the cancellation reaches the caller.
* [client] clean up all started components on Start failure
Start's failure defer only called close(), which covers the wg interface,
firewall, rosenpass and port forwarding but leaves connMgr, srWatcher,
route/DNS/flow/state managers and the monitor goroutines running. A late
failure (e.g. the context-cancelled check after the signal stream) thus
leaked them.
Extract Stop's locked teardown into stopLocked (caller holds syncMsgMux,
does not wait on shutdownWg) and call it from both Stop and Start's defer.
The defer also cancels the run context first so goroutines started before
the failure unwind. Teardown order is unchanged.
Detecting shutdown by inspecting the gRPC status code conflates a local
context cancellation with a server- or proxy-sent codes.Canceled. When
the latter occurs (e.g. an intermediary proxy resets the stream), the
retry loop silently terminates and the client never reconnects.
Switch to ctx.Err() in the signal Receive loop and management Sync/Job
handlers, and stop matching gRPC Canceled/DeadlineExceeded in the flow
client's isContextDone helper. With this change, a server-sent Canceled
is treated as a transient error and the backoff retry loop continues.
Bump the IsHealthy() context timeout from 1s to 5s for both the
management and signal gRPC clients to reduce false negatives on
slower or congested connections.
This PR improves the NetBird client's status checking mechanism by implementing earlier detection of client state changes and better handling of connection lifecycle management. The key improvements focus on:
• Enhanced status detection - Added waitForReady option to StatusRequest for improved client status handling
• Better connection management - Improved context handling for signal and management gRPC connections• Reduced connection timeouts - Increased gRPC dial timeout from 3 to 10 seconds for better reliability
• Cleaner error handling - Enhanced error propagation and context cancellation in retry loops
Key Changes
Core Status Improvements:
- Added waitForReady optional field to StatusRequest proto (daemon.proto:190)
- Enhanced status checking logic to detect client state changes earlier in the connection process
- Improved handling of client permanent exit scenarios from retry loops
Connection & Context Management:
- Fixed context cancellation in management and signal client retry mechanisms
- Added proper context propagation for Login operations
- Enhanced gRPC connection handling with better timeout management
Error Handling & Cleanup:
- Moved feedback channels to upper layers for better separation of concerns
- Improved error handling patterns throughout the client server implementation
- Fixed synchronization issues and removed debug logging
- Move `util/grpc` and `util/net` to `client` so `internal` packages can be accessed
- Add methods to return the next best interface after the NetBird interface.
- Use `IP_UNICAST_IF` sock opt to force the outgoing interface for the NetBird `net.Dialer` and `net.ListenerConfig` to avoid routing loops. The interface is picked by the new route lookup method.
- Some refactoring to avoid import cycles
- Old behavior is available through `NB_USE_LEGACY_ROUTING=true` env var
In this PR, speed up the GRPC message processing, force the recreation of the ICE agent when getting a new, remote offer (do not wait for local STUN timeout).
Riccardo Manfrin