sendnrw/netbird
2
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-09-29 10:09:07 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
c166fcbd556e3f351db39fb11126dcd63bc21d06
netbird/client/iface/configurer/allowedips_test.go
T
Riccardo Manfrin 164d92e78d [client] Stop dumping the whole device to clear one peer endpoint (#7632) 
* [client] Stop dumping the whole device to clear one peer endpoint

Clearing a peer's endpoint has to remove and re-add the peer, because neither the
netlink API nor the wireguard-go UAPI can clear an endpoint in place. To keep the
peer's allowed IPs across that dance, RemoveEndpointAddress read them back from the
device: a full wgctrl.Device() dump on the kernel path, a full IpcGet plus text parse
on the userspace one. Both cost a round trip proportional to the entire network map,
both run under the interface lock, and both run on every relay and ICE transition.
On a routing peer with ~15700 peers that is megabytes of netlink traffic per
transition, at a measured 713 transitions per minute, with every other configuration
operation queued behind it. RemoveAllowedIP paid the same price for the same reason.

The allowed IPs cannot come from the caller: peer.Conn knows the peer's own overlay
addresses, while the routed prefixes are attached separately by the route manager's
refcounter, so a caller-supplied set would silently drop every route behind the peer.

The configurer is the only writer of its device's peer set, so it can keep an
authoritative mirror of what it configured and answer from memory instead. The mirror
is fed by every operation that changes a peer's allowed IPs and reset by a device
reconfiguration that replaces the peer set. A peer the mirror has not seen, which is
what an out-of-band reconfiguration leaves behind, still falls back to reading the
device and seeds the mirror from it.

Prefixes are unmapped on the way in, so a v4-mapped address compares equal to the
plain v4 prefix for the same network rather than registering as a second entry.

Measured on a userspace device, allocations to clear one endpoint:

  peers      64     256    1024    4096
  before   1452       -   21617       -
  after      91      91      91      91

* [client] Keep update-only allowed IP adds out of the peer mirror

AddAllowedIP configures the device with update_only, which is a silent no-op when
the peer does not exist, so its success says nothing about whether the device took
the prefix. Recording it unconditionally let the mirror hold a peer the device had
dropped, and RemoveEndpointAddress re-adds a peer without update_only: clearing the
endpoint of such a peer recreated it, carrying allowed IPs the device never held.
Allowed IPs are unique per device, so the recreated peer takes those prefixes away
from the peer that legitimately holds them.

This is not a theoretical window. Under lazy connections a routing peer's device
entry is torn down and re-created on the idle transition, and a routed prefix
re-added during that window is lost exactly because of update_only (#6863).

Allowed IP adds now merge only onto a peer the store already knows, which mirrors
the device: the operations that can create a peer record it, the update-only ones
do not. A peer missing from the store still falls back to reading the device.

* [client] Hand a prefix over to its new owner in the peer mirror

An allowed IP belongs to exactly one peer: configuring a prefix on a peer takes it
away from whichever peer held it before, and the configurer leaves that handover to
the device rather than removing the prefix from the previous holder itself, which is
what UpdatePeer's "wg will handle duplicated peer IP" refers to. The mirror recorded
the prefix on the new peer while leaving it listed under the old one, so clearing the
old peer's endpoint rewrote its allowed IPs from that stale list and took the prefix
back from the peer that now owns it. Traffic for the routed prefix then went to the
wrong peer. Reading the device before each write used to rule this out.

The store now tracks the owner of each prefix and performs the same handover, so
rewriting one peer's list cannot reclaim a prefix another peer holds.

Prefixes are also masked on the way in. A device stores them masked, so a caller
passing host bits would otherwise fail to match what a device fallback seeded and
could never remove that prefix by value. Conversion back from the device now keys
the v4-mapped decision on the mask width as well, so a genuine v6 prefix inside the
mapped range stays v6 instead of being dropped as an invalid v4 prefix.

* [client] Keep a mapped v6 prefix below /96 out of the v4 form

normalizePrefix unmapped any v4-mapped address before masking it, keeping the
original prefix length. For a genuine v6 prefix inside the mapped range, such as
::ffff:0:0/64, that pairs a v4 address with a v6 sized mask: netip.PrefixFrom
returns an invalid prefix and Masked turns it into the zero prefix. The store then
held a prefix whose Bits is -1, which cannot reproduce the allowed IP the device
was given, so re-adding the peer after an endpoint removal could fail once the
peer had already been removed.

Masking now comes first, and it also decides the address family: only a prefix at
least 96 bits long keeps the mapped marker through the mask, so anything shorter
inside that range is v6 and stays v6.

* [client] Record a peer created by a preshared key write

Setting a preshared key without updateOnly creates the peer when it is absent, and
Rosenpass applies a peer's first key exactly that way, since applyKeyLocked passes
the peer's initialized flag. The store ignored that operation, so the peer could
exist on the device while the store treated it as unknown.

An update-only allowed IP add on such a peer then succeeded on the device, which
moved the prefix away from its previous holder, while the store skipped the peer
and left the previous holder still claiming it. Clearing that holder's endpoint
rewrote it from the stale claim and took the prefix back, leaving the peer that
owns the route with nothing.

Every device operation that can create a peer now records it, which is the same
rule the update-only operations already follow from the other side.

* [client] Match a peer on the parsed key instead of its base64 form

getPeer scanned the device comparing Key.String to the caller's key. wgtypes.Key
is a 32 byte array, so it compares directly, while String base64 encodes it into a
fresh allocation on every iteration. The scan therefore allocated once per peer on
the device to find a single peer, and on a large network that is tens of thousands
of allocations per lookup.

The key is parsed once up front and the arrays are compared. Behaviour is
unchanged: the callers already parse the same key before reaching here, so the new
parse error is unreachable in practice and only guards the helper on its own.

* [client] Normalize prefixes on their way to the device

Prefixes were normalized when recorded but not when written, so a caller's raw prefix
reached the device while a different form was kept for it. The conversion is also where
a mapped prefix goes wrong: net.IPNet prints a v4-mapped address as v4 but takes the
length from its 16 byte mask, so ::ffff:10.1.2.3/64 is handed to a userspace device as
10.1.2.3/0 — an allowed IP matching every v4 address, on a peer that was meant to carry
one /64.

prefixesToIPNets now normalizes, and the two hand-built conversions in AddAllowedIP go
through it, so there is a single place where a prefix is turned into something a device
is given and it cannot disagree with what is recorded for it.

* [client] Parse the endpoint before configuring the peer

The userspace UpdatePeer parsed the endpoint address after the device had already been
configured, and returned on a parse failure. The device was then left holding a peer
that neither the activity recorder nor the allowed IP store had been told about, so the
peer was invisible to the wake path and the prefix handover for its allowed IPs never
happened, leaving the previous holder still claiming them.

The parse now happens before anything is written, so the only failure left after the
device is touched is one the caller cannot cause.

* [client] Keep the record when a peer removal fails

The two configurers disagreed: the kernel one dropped its record only once the device
had accepted the removal, the userspace one dropped it either way. Removing a peer is a
single device write, so a failure leaves the peer exactly as it was, with the allowed IPs
the record still describes. Dropping it there asserts nothing useful and only sends the
next caller to read the whole device back for an answer it already had.

The userspace one now follows the kernel and returns early on failure.

* [client] Write down what the allowed IP store does not guarantee

Two properties were relied on without being stated. The store's lock covers its map and
not the device write beside it, so consistency between the two rests on callers being
serialized, which WGIface does with its mutex; anyone removing that would have no way to
learn it mattered. And the fallback to the device only covers a peer the store has never
seen, so a peer first recorded from empty while the device already held prefixes keeps
only what was recorded, and the next endpoint removal drops the rest.

* [client] Key the allowed IP store on the parsed peer key

The store keyed on the textual key, so a lookup compared 44 byte strings while the
callers all held the parsed key already and the configurer had to carry both forms.
wgtypes.Key is a 32 byte array and compares directly, which is what getPeer was changed
to do for the same reason.

The store and its helpers now take wgtypes.Key, the callers pass the key they parsed on
entry, and the textual form survives only where something outside speaks it: parseStatus
reports peers that way, so the userspace fallback converts once for its scan.

* [client] Document the configurer methods the store changed

The exported configurer methods now carry what the allowed IP store made true of them:
when the mirror is reset, that a peer update merges its prefixes and takes them from
their previous owner, that an update-only add on an absent peer does nothing, and what
each side does with its record when a device write fails — where the two configurers
differ, since the userspace one reports a prefix it does not have and the kernel one
treats it as a no-op. mergeLocked states the lock its callers must already hold.

Docstrings that only restated the name of a test are left out; the tests explain the
scenario they set up in the body, where the explanation belongs.
2026-09-28 18:03:42 +02:00

264 lines
9.6 KiB
Go
Raw Blame History

package configurer
import (
"net"
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
)
// The store keys on the parsed key, so the tests use two distinct ones rather than names.
var (
testPeer = wgtypes.Key{1}
otherPeer = wgtypes.Key{2}
)
func TestAllowedIPStoreUnknownPeer(t *testing.T) {
s := newAllowedIPStore()
prefixes, ok := s.get(testPeer)
assert.False(t, ok, "an unconfigured peer must be reported as unknown, not as one without prefixes")
assert.Nil(t, prefixes, "an unknown peer has no prefixes")
}
func TestAllowedIPStoreAddUnions(t *testing.T) {
s := newAllowedIPStore()
overlay := netip.MustParsePrefix("100.64.0.1/32")
routed := netip.MustParsePrefix("10.20.0.0/16")
s.set(testPeer, []netip.Prefix{overlay})
// A peer update does not replace allowed IPs, and a repeated prefix must not be doubled.
s.add(testPeer, []netip.Prefix{overlay, routed})
prefixes, ok := s.get(testPeer)
require.True(t, ok, "peer must be known after set")
assert.Equal(t, []netip.Prefix{overlay, routed}, prefixes, "add must union rather than replace")
}
func TestAllowedIPStoreGetReturnsCopy(t *testing.T) {
s := newAllowedIPStore()
overlay := netip.MustParsePrefix("100.64.0.1/32")
s.set(testPeer, []netip.Prefix{overlay})
prefixes, ok := s.get(testPeer)
require.True(t, ok, "peer must be known after set")
prefixes[0] = netip.MustParsePrefix("0.0.0.0/0")
stored, _ := s.get(testPeer)
assert.Equal(t, []netip.Prefix{overlay}, stored, "a caller mutating the returned slice must not corrupt the store")
}
func TestAllowedIPStoreForgetAndReset(t *testing.T) {
s := newAllowedIPStore()
s.set(testPeer, []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")})
s.set(otherPeer, []netip.Prefix{netip.MustParsePrefix("100.64.0.2/32")})
s.forget(testPeer)
_, ok := s.get(testPeer)
assert.False(t, ok, "a forgotten peer must be unknown")
_, ok = s.get(otherPeer)
assert.True(t, ok, "forgetting one peer must not touch the others")
s.reset()
_, ok = s.get(otherPeer)
assert.False(t, ok, "reset must drop every peer")
}
func TestIPNetsToPrefixes(t *testing.T) {
tests := []struct {
name string
ipNet net.IPNet
want string
}{
{
name: "v4",
ipNet: net.IPNet{IP: net.IP{10, 20, 0, 0}, Mask: net.CIDRMask(16, 32)},
want: "10.20.0.0/16",
},
{
name: "v4 mapped under a 128 bit mask",
ipNet: net.IPNet{IP: net.ParseIP("10.20.0.0"), Mask: net.CIDRMask(112, 128)},
want: "10.20.0.0/16",
},
{
name: "v6",
ipNet: net.IPNet{IP: net.ParseIP("fd00::"), Mask: net.CIDRMask(64, 128)},
want: "fd00::/64",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := ipNetsToPrefixes([]net.IPNet{tc.ipNet})
require.Len(t, got, 1, "the address must be converted, not dropped")
assert.Equal(t, tc.want, got[0].String(), "converted prefix")
})
}
}
func TestIPNetsToPrefixesRoundTrip(t *testing.T) {
prefixes := []netip.Prefix{
netip.MustParsePrefix("100.64.0.1/32"),
netip.MustParsePrefix("10.20.0.0/16"),
netip.MustParsePrefix("fd00::/64"),
}
assert.Equal(t, prefixes, ipNetsToPrefixes(prefixesToIPNets(prefixes)),
"prefixes handed to a device must come back unchanged")
}
func TestAllowedIPStoreNormalizesMappedPrefixes(t *testing.T) {
s := newAllowedIPStore()
v4 := netip.MustParsePrefix("10.20.0.0/16")
mapped := netip.PrefixFrom(netip.AddrFrom16(v4.Addr().As16()), 112)
s.set(testPeer, []netip.Prefix{mapped})
// A v4 rule only matches a v4-mapped address once it has been unmapped, so the store must
// hold the plain form and recognise the two spellings as the same prefix.
s.add(testPeer, []netip.Prefix{v4})
prefixes, ok := s.get(testPeer)
require.True(t, ok, "peer must be known after set")
assert.Equal(t, []netip.Prefix{v4}, prefixes, "a mapped prefix must be stored unmapped and not duplicated")
}
func TestNormalizePrefix(t *testing.T) {
v4 := netip.MustParsePrefix("10.20.0.0/16")
v6 := netip.MustParsePrefix("fd00::/64")
assert.Equal(t, v4, normalizePrefix(v4), "a plain v4 prefix is unchanged")
assert.Equal(t, v6, normalizePrefix(v6), "a real v6 prefix is unchanged")
assert.Equal(t, v4, normalizePrefix(netip.PrefixFrom(netip.AddrFrom16(v4.Addr().As16()), 112)),
"a mapped prefix under a 128 bit mask becomes plain v4")
// A prefix shorter than /96 inside the mapped range is a genuine v6 prefix. Unmapping it
// would pair a v4 address with a v6 sized mask, which is invalid, and the store would then
// record a zero prefix that can never recreate the allowed IP.
for _, tc := range []string{"::ffff:0:0/64", "::ffff:1.2.3.4/80", "::ffff:1.2.3.4/95"} {
got := normalizePrefix(netip.MustParsePrefix(tc))
assert.True(t, got.IsValid(), "%s must normalize to a valid prefix", tc)
assert.False(t, got.Addr().Is4(), "%s must stay v6", tc)
}
}
func TestAllowedIPStoreAddExistingDoesNotCreate(t *testing.T) {
s := newAllowedIPStore()
routed := netip.MustParsePrefix("10.20.0.0/16")
// An update-only device operation on an absent peer is a silent no-op, so nothing may be
// recorded for a peer the store does not already know.
s.addExisting(testPeer, []netip.Prefix{routed})
_, ok := s.get(testPeer)
assert.False(t, ok, "addExisting must not record an unknown peer")
overlay := netip.MustParsePrefix("100.64.0.1/32")
s.set(testPeer, []netip.Prefix{overlay})
s.addExisting(testPeer, []netip.Prefix{routed})
prefixes, _ := s.get(testPeer)
assert.Equal(t, []netip.Prefix{overlay, routed}, prefixes, "addExisting must union onto a known peer")
}
func TestAllowedIPStoreHandsPrefixOverToTheNewOwner(t *testing.T) {
s := newAllowedIPStore()
routed := netip.MustParsePrefix("10.20.0.0/16")
other := otherPeer
s.set(testPeer, []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32"), routed})
s.set(other, []netip.Prefix{netip.MustParsePrefix("100.64.0.2/32")})
// The device takes an allowed IP away from its previous holder when it is configured on
// another peer, so the store must do the same rather than list it under both.
s.addExisting(other, []netip.Prefix{routed})
previous, _ := s.get(testPeer)
assert.NotContains(t, previous, routed, "the previous owner must lose the prefix")
current, _ := s.get(other)
assert.Contains(t, current, routed, "the new owner must hold the prefix")
}
func TestAllowedIPStoreForgetReleasesOwnership(t *testing.T) {
s := newAllowedIPStore()
routed := netip.MustParsePrefix("10.20.0.0/16")
s.set(testPeer, []netip.Prefix{routed})
s.forget(testPeer)
s.set(otherPeer, []netip.Prefix{routed})
// A forgotten peer must not be resurrected as a key in the peer map by a later claim.
_, ok := s.get(testPeer)
assert.False(t, ok, "the forgotten peer must stay unknown")
current, _ := s.get(otherPeer)
assert.Equal(t, []netip.Prefix{routed}, current, "the new owner must hold the prefix")
}
func TestNormalizePrefixClearsHostBits(t *testing.T) {
// A device stores a prefix masked, so a caller passing host bits must still match what a
// device fallback seeded, otherwise that prefix could never be removed by value.
assert.Equal(t, netip.MustParsePrefix("10.20.0.0/16"),
normalizePrefix(netip.MustParsePrefix("10.20.0.1/16")), "host bits must be cleared")
assert.Equal(t, netip.MustParsePrefix("fd00::/64"),
normalizePrefix(netip.MustParsePrefix("fd00::1/64")), "host bits must be cleared for v6")
}
func TestIPNetsToPrefixesKeepsV6InTheMappedRange(t *testing.T) {
// ::ffff:0:0/64 reads as v4-mapped but is a genuine v6 prefix: unmapping it would leave a
// v4 address under a 64 bit mask, which is invalid, and the allowed IP would be dropped.
got := ipNetsToPrefixes([]net.IPNet{{
IP: net.ParseIP("::ffff:0:0"),
Mask: net.CIDRMask(64, 128),
}})
require.Len(t, got, 1, "the prefix must be converted, not dropped")
assert.False(t, got[0].Addr().Is4(), "a v6 prefix in the mapped range must not become v4")
assert.Equal(t, 64, got[0].Bits(), "the prefix length must survive the conversion")
}
func TestPrefixesToIPNetsNormalizes(t *testing.T) {
// net.IPNet prints a v4-mapped address as v4 but takes the length from its 16 byte
// mask, so an unnormalized ::ffff:10.1.2.3/64 reaches a userspace device as 10.1.2.3/0,
// an allowed IP that matches every v4 address.
tests := []struct {
name string
given string
want string
}{
{name: "mapped below /96", given: "::ffff:10.1.2.3/64", want: "::/64"},
{name: "mapped at /112", given: "::ffff:10.1.2.3/112", want: "10.1.0.0/16"},
{name: "host bits are cleared", given: "10.20.0.1/16", want: "10.20.0.0/16"},
{name: "v6 is untouched", given: "fd00::1/64", want: "fd00::/64"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := prefixesToIPNets([]netip.Prefix{netip.MustParsePrefix(tc.given)})
require.Len(t, got, 1, "the prefix must be converted, not dropped")
assert.Equal(t, tc.want, got[0].String(), "what the device is given")
assert.NotEqual(t, 0, mustOnes(t, got[0]), "a device must never be given a zero length allowed IP")
})
}
}
func mustOnes(t *testing.T, ipNet net.IPNet) int {
t.Helper()
ones, _ := ipNet.Mask.Size()
return ones
}
// TestPrefixesToIPNetsAgreesWithTheStore pins the property the store depends on: what a
// device is given and what is recorded for it are the same prefix.
func TestPrefixesToIPNetsAgreesWithTheStore(t *testing.T) {
for _, given := range []string{"::ffff:10.1.2.3/64", "::ffff:10.1.2.3/112", "10.20.0.1/16", "fd00::1/64"} {
prefix := netip.MustParsePrefix(given)
toDevice := prefixesToIPNets([]netip.Prefix{prefix})
recorded := normalizePrefix(prefix)
assert.Equal(t, recorded.String(), toDevice[0].String(),
"%s must reach the device in the form the store records", given)
}
}
Reference in New Issue View Git Blame Copy Permalink