## Describe your changes
`recordConnectionMetrics` mapped only `conntype.Relay` to `relay` and
let a `default` branch
record everything else as `ice`. That silently included `ICETurn` — an
ICE connection through
a TURN server, which
[`conn.isRelayed`](https://github.com/netbirdio/netbird/blob/main/client/internal/peer/conn.go#L788-L795)
itself counts as relayed — and `None`, the transient state set when the
relay drops
([conn.go:632](https://github.com/netbirdio/netbird/blob/main/client/internal/peer/conn.go#L632))
or the peer state is reset
([conn.go:757](https://github.com/netbirdio/netbird/blob/main/client/internal/peer/conn.go#L757)).
Both were reported as direct peer-to-peer, so the `ice` share overstated
direct connections on
every platform.
The mapping now lists every priority explicitly and emits `ice_p2p`,
`ice_turn`, `relay` or
`unknown`. The new values deliberately do not reuse `ice` to avoid
ambuguity.
## Issue ticket number and link
No public issue. Found while reviewing the first production sample of
client metrics: 38% of iOS
connection events were tagged `ice` on a platform that forces relay by
default, which traced back
to the `default` branch at
[client/internal/peer/conn.go#L963-L968](https://github.com/netbirdio/netbird/blob/main/client/internal/peer/conn.go#L963-L968).
## Stack
<!-- branch-stack -->
### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [x] Created tests that fail without the change (if possible)
> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).
## Documentation
Select exactly one:
- [] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)
Internal metrics documentation only, in
`client/internal/metrics/infra/README.md`: the four
`connection_type` values with their derivation, and a note that pre-fix
`ice` samples are not
comparable with `ice_p2p`. No public API, CLI or configuration change,
so no netbirdio/docs PR.
### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:
N/A
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
* **New Features**
* Connection metrics now distinguish direct peer-to-peer, TURN-assisted,
relay, and unknown connection types.
* Metrics include clearer connection and peer identification details.
* **Documentation**
* Updated connection timing metric values, traffic semantics, priority
behavior, and historical data guidance.
* **Bug Fixes**
* Unset or unrecognized connection priorities are no longer incorrectly
classified as peer-to-peer.
* Unknown-transport metrics are skipped to prevent misleading connection
data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Tag each WG timeout event with the watcher's context and discard it at
dispatch time when that context is cancelled. This replaces the earlier
generation counter and the mailbox-level epoch filter: cancelling the
watcher is now the single act that both stops it and retires its pending
timeouts, so the two can no longer diverge, and the staleness check moves
to dispatch time so a cancel performed by an earlier event in the same
drained batch already suppresses a superseded watcher's timeout.
The rebase carried #6664's WGWatcher changes into our wg_watcher package
(single-shot, no enabled flag). Adapt conn.go to match: create a fresh
watcher per connection attempt in enableWgWatcherIfNeeded, drop it in
disableWgWatcherIfNeeded, nil-guard resetEndpoint.
Guard stale WG timeouts on the event loop instead of #6664's conn.mu
recheck: onWGDisconnected only checked watcherCtx on the watcher
goroutine, racing the loop that cancels it and processes the timeout. A
loop-owned wgWatcherGen tags evWGTimeout; handleWGTimeout drops events
from a superseded generation, so the check and the teardown happen
atomically on the single loop.
Move the signaling protocol out of the peer package into
client/internal/peer/signaling: OfferAnswer, IceCredentials, Handshaker
and Signaler. Break the peer<->signaling cycle by giving the Handshaker
a plain Config and an ICEWorker interface instead of *Conn/*WorkerICE,
and by passing the relay manager directly rather than the relay worker.
Combine the ICE worker's local-credentials and session-id accessors into
a single Credentials() returning a Credentials struct.
Move the ICE session id to the ice package as ice.SessionID, since it
identifies an ICE agent session and is minted there alongside
GenerateICECredentials; signaling only carries it.
Move the Status recorder and its state types out of the peer package
into client/internal/peer/status, split by struct across recorder.go,
peer_state.go, full_status.go, events.go, notifier.go and route.go
instead of one 1600-line file. Rename the type Status -> Recorder
(NewRecorder already implied it; avoids status.Status stutter). Split
conn_status.go: the ConnStatus type and its constants move to the status
package, connStatusInputs stays with the peer event loop.
The peer package references the status package directly; a transitional
status_alias.go re-exports the moved symbols for the ~50 external callers
still using peer.Status/State/ConnStatus, to be removed once they are
migrated.
Replace the mutex-guarded callback model of peer.Conn with a per-peer
event loop that exclusively owns all mutable connection state. External
callers and transport workers post typed events into a non-blocking,
coalescing mailbox instead of contending on conn.mu:
- offers/answers coalesce to the newest message, a new offer flushes
queued candidates of the superseded session
- candidates are applied in arrival order from a bounded FIFO
- transport state changes are never dropped
- the blocking relay dial runs on a helper goroutine with a single dial
in flight; signaling I/O (offer/answer sends) runs off the loop
conn.mu now only guards the open/close lifecycle. Close posts a close
event and waits for the loop teardown; the loop also tears down on
engine context cancellation and releases resources of unprocessed
events.
Delete the Handshaker listener machinery (Listen loop, unbuffered
drop-on-busy channels, AsyncOfferListener with its double-processing of
the first offer), the never-wired dispatcher package and the unused
ICEMonitor.ReconnectCh. Fix a goroutine leak in the WG watcher test
that raced with tests mutating the package-level check timing vars.
* Stick new watcher creation to actual existence of af the conn
and its removal to the removal of such same conn.
Avoid debouncing and cross lock dead locking
* Discriminate not updated from timeout handshakes
* [Recheck watcher ctx cancellation under conn.mu in onWGDisconnected
onWGDisconnected only checked conn.ctx (the engine-scoped context), never
the watcher's own context. disableWgWatcherIfNeeded cancels the wgWatcherCtx,
not conn.ctx, so a disabled watcher's timeout callback did not see the
cancellation.
handshakeCheck runs lock-free, so between the ctx check in periodicHandshakeCheck
and acquiring conn.mu a fast disconnect/reconnect can slip in: the stale watcher
then acquires the lock and tears down the *new*, healthy connection based on the
old timeout, forcing the guard into an unnecessary reconnect (flap).
Recheck watcherCtx.Err() under conn.mu so a superseded watcher exits without
touching the connection that replaced it.
* Remove verbose comments
* Fixup merge conflict leftovers
* Fixup context brought by onWGDisconnected
* [Client] Remove connection semaphore
Remove the semaphore and the initial random sleep time (300ms) from the connectivity logic to speed up the initial connection time.
Note: Implement limiter logic that can prioritize router peers and keep the fast connection option for the first few peers.
* Remove unused function
* [client] Add WGConfigurer interface
To allow Rosenpass to work both with kernel
WireGuard via wgctrl (default behavior) and
userspace WireGuard via IPC on Android/iOS
using WGUSPConfigurer
* [client] Remove Rosenpass debug logs
* [client] Return simpler peer configuration in outputKey method
ConfigureDevice, the method previously used in
outputKey via wgClient to update the device's
properties, is now defined in the WGConfigurer
interface and implemented both in kernel_unix and
usp configurers.
PresharedKey datatype was also changed from
boolean to [32]byte to compare it
to the original NetBird PSK, so that Rosenpass
may replace it with its own when necessary.
* [client] Remove unused field
* [client] Replace usage of WGConfigurer
Replaced with preshared key setter interface,
which only defines a method to set / update the preshared key.
Logic has been migrated from rosenpass/netbird_handler to client/iface.
* [client] Use same default peer keepalive value when setting preshared keys
* [client] Store PresharedKeySetter iface in rosenpass manager
To avoid no-op if SetInterface is called before generateConfig
* [client] Add mutex usage in rosenpass netbird handler
* [client] change implementation setting Rosenpass preshared key
Instead of providing a method to configure a device (device/interface.go),
it forwards the new parameters to the configurer (either
kernel_unix.go / usp.go).
This removes dependency on reading FullStats, and makes use of a common
method (buildPresharedKeyConfig in configurer/common.go) to build a
minimal WG config that only sets/updates the PSK.
netbird_handler.go now keeps s list of initializedPeers to choose whether
to set the value of "UpdateOnly" when calling iface.SetPresharedKey.
* [client] Address possible race condition
Between outputKey calls and peer removal; it
checks again if the peer still exists in the
peers map before inserting it in the
initializedPeers map.
* [client] Add psk Rosenpass-initialized check
On client/internal/peer/conn.go, the presharedKey
function would always return the current key
set in wgConfig.presharedKey.
This would eventually overwrite a key set
by Rosenpass if the feature is active.
The purpose here is to set a handler that will
check if a given peer has its psk initialized
by Rosenpass to skip updating the psk
via updatePeer (since it calls presharedKey
method in conn.go).
* Add missing updateOnly flag setup for usp peers
* Change common.go buildPresharedKeyConfig signature
PeerKey datatype changed from string to
wgTypes.Key. Callers are responsible for parsing
a peer key with string datatype.
When an ICE agent connection was in progress, new offers were being ignored. This was incorrect logic because the remote agent could be restarted at any time.
In this change, whenever a new session ID is received, the ongoing handshake is closed and a new one is started.
In this PR, speed up the GRPC message processing, force the recreation of the ICE agent when getting a new, remote offer (do not wait for local STUN timeout).
This will allow running netbird commands (including debugging) against the daemon and provide a flow similar to non-container usages.
It will by default both log to file and stderr so it can be handled more uniformly in container-native environments.
With the lazy connection feature, the peer will connect to target peers on-demand. The trigger can be any IP traffic.
This feature can be enabled with the NB_ENABLE_EXPERIMENTAL_LAZY_CONN environment variable.
When the engine receives a network map, it binds a free UDP port for every remote peer, and the system configures WireGuard endpoints for these ports. When traffic appears on a UDP socket, the system removes this listener and starts the peer connection procedure immediately.
Key changes
Fix slow netbird status -d command
Move from engine.go file to conn_mgr.go the peer connection related code
Refactor the iface interface usage and moved interface file next to the engine code
Add new command line flag and UI option to enable feature
The peer.Conn struct is reusable after it has been closed.
Change connection states
Connection states
Idle: The peer is not attempting to establish a connection. This typically means it's in a lazy state or the remote peer is expired.
Connecting: The peer is actively trying to establish a connection. This occurs when the peer has entered an active state and is continuously attempting to reach the remote peer.
Connected: A successful peer-to-peer connection has been established and communication is active.
fixes the Rosenpass preshared key handling to enable successful WireGuard handshakes when one side is in permissive mode. Key changes include:
Updating field accesses from RosenpassPubKey/RosenpassAddr to RosenpassConfig.PubKey/RosenpassConfig.Addr.
Modifying the preshared key computation logic to account for permissive mode.
Revising peer configuration in the Engine to use the new RosenpassConfig struct.
In the case of user space WireGuard mode, use in-memory proxy between the TURN/Relay connection and the WireGuard Bind. We keep the UDP proxy and eBPF proxy for kernel mode.
The key change is the new wgproxy/bind and the iface/bind/ice_bind changes. Everything else is just to fulfill the dependencies.
This update adds new relay integration for NetBird clients. The new relay is based on web sockets and listens on a single port.
- Adds new relay implementation with websocket with single port relaying mechanism
- refactor peer connection logic, allowing upgrade and downgrade from/to P2P connection
- peer connections are faster since it connects first to relay and then upgrades to P2P
- maintains compatibility with old clients by not using the new relay
- updates infrastructure scripts with new relay service
EBPF proxy between TURN (relay) and WireGuard to reduce number of used ports used by the NetBird agent.
- Separate the wg configuration from the proxy logic
- In case if eBPF type proxy has only one single proxy instance
- In case if the eBPF is not supported fallback to the original proxy Implementation
Between the signature of eBPF type proxy and original proxy has
differences so this is why the factory structure exists
This PR brings support of a shared port between stun (ICE agent) and
the kernel WireGuard
It implements a single port mode for execution with kernel WireGuard
interface using a raw socket listener.
BPF filters ensure that only STUN packets hit the NetBird userspace app
Removed a lot of the proxy logic and direct mode exchange.
Now we are doing an extra hole punch to the remote WireGuard
port for best-effort cases and support to old client's direct mode.
This PR adds supports for the WireGuard userspace implementation
using Bind interface from wireguard-go.
The newly introduced ICEBind struct implements Bind with UDPMux-based
structs from pion/ice to handle hole punching using ICE.
The core implementation was taken from StdBind of wireguard-go.
The result is a single WireGuard port that is used for host and server reflexive candidates.
Relay candidates are still handled separately and will be integrated in the following PRs.
ICEBind checks the incoming packets for being STUN or WireGuard ones
and routes them to UDPMux (to handle hole punching) or to WireGuard respectively.
On Android, because of the hard SELinux policies can not list the
interfaces of the ICE package. Without it can not generate a host type
candidate. In this pull request, the list of interfaces comes via the Java
interface.
Fix the status indication in the client service. The status of the
management server and the signal server was incorrect if the network
connection was broken. Basically the status update was not used by
the management and signal library.
Before defining if we will use direct or proxy connection we will exchange a
message with the other peer if the modes match we keep the decision
from the shouldUseProxy function otherwise we skip using direct connection.
Added a feature support message to the signal protocol
The ConnStatus is a custom type based on iota
like an enum. The problem was nowhere used to the
benefits of this implementation. All ConnStatus
instances has been compared with strings. I
suppose the reason to do it to avoid a circle
dependency. In this commit the separated status
package has been moved to peer package.
Remove unused, exported functions from engine
Added additional common blacklisted interfaces
Updated the signal protocol to pass the peer port and netbird version
Co-authored-by: braginini <bangvalo@gmail.com>
Zoltán Papp