Compare commits

..

6 Commits

Author SHA1 Message Date
Zoltán Papp
adf62a8080 Merge remote-tracking branch 'origin/main' into feature/ios-ssh
# Conflicts:
#	client/server/server.go
2026-08-12 12:35:47 +02:00
evgeniyChepelev
223153993c [client] iOS SSH: gate JWT behind server detection, reuse jwtcache (#6759)
* feat(ios/ssh): add ConnectNetBirdPeer, JWT cache, and WireGuard dialer

- Add ConnectNetBirdPeer() method that skips banner detection and
  authenticates directly via JWT (mirrors NetBird web dashboard flow)
- Cache JWT token for 8 minutes to avoid OAuth re-auth on every reconnect
  (SSH servers reject tokens older than DefaultJWTMaxTokenAge = 10 min)
- Bind SSH sockets to the WireGuard interface via IP_BOUND_IF (Darwin
  syscall 25) so connections to 100.x.x.x peers route through the tunnel
  in the iOS Network Extension process
- Improve auth failure error message with actionable checklist
- Add base64 JWT claims debug logging for auth troubleshooting

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ios/ssh): never offer the JWT to an unverified SSH service

Remove ConnectNetBirdPeer. It skipped banner detection and sent the JWT
as a password to whatever sshd answered, under InsecureIgnoreHostKey.
WireGuard authenticates the peer machine, not the SSH service listening
on it, so that handed a valid bearer token to an unidentified listener —
a plain OpenSSH + PAM setup may even log the password attempt.

Connect is now the only entry point. Detection is what proves the
listener is a NetBird SSH service before the token is offered, and the
JWT path already verifies the host key against the management-plane
registry via engineHostKeyVerifier. A peer whose NetBird SSH is disabled
is served by plain sshd and authenticates by key or password like any
other host, so it stays reachable without an OAuth round trip.

Move the NetBird-specific auth-failure checklist into annotateAuthError,
gated on serverType.RequiresJWT(), so a regular sshd rejection no longer
tells the user to check dashboard SSH access. Drop base64DecodeJWT,
which was left unused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-11 15:14:31 +02:00
Zoltán Papp
081f2d153b [client] Extract SSH JWT cache into shared jwtcache package
Move the daemon's memguard-backed JWT token cache from client/server
into client/ssh/jwtcache so it can be reused outside the daemon
process. The TTL derivation from Config.SSHJWTCacheTTL (nil=default,
0=disabled) moves into jwtcache.ResolveTTL.

Wire the cache into the iOS SDK's SSH client: iOS has no daemon to
delegate caching to, so without it every reconnect forced the user
through the browser OAuth device-code flow. The cache lives on the
long-lived Client (the app creates a new SSHClient per session) and
uses the same config-driven TTL semantics as the daemon.
2026-08-02 10:04:12 +02:00
Zoltán Papp
90bb5e7c0f Merge remote-tracking branch 'origin/main' into feature/ios-ssh 2026-08-02 09:56:00 +02:00
Zoltán Papp
1381dcf919 Merge tag 'v0.73.2' into feature/ios-ssh
Resolve conflict in client/ios/NetBirdSDK/client.go: keep v0.73.2's
thread-safe state handling (stateMu, setState, stateSnapshot) and layer
the SSH client's sshState() helper on top of stateSnapshot().

Add missing imports (strconv, internal/auth, internal/profilemanager) to
ssh_client.go that the SSH feature branch referenced but never imported.
2026-06-26 14:16:03 +02:00
Zoltan Papp
7ea0882975 [ios] add SSHClient gomobile binding for in-app terminal
Exposes SSHClient + SSHTerminalListener to the iOS app, mirroring the
Android binding. Connect() auto-detects the server type via banner
inspection and selects the auth path: NetBird-SSH with JWT triggers the
device-code OAuth flow via the existing URLOpener; NetBird-SSH without
JWT uses the NetBird private key; regular SSH falls back to the NetBird
key then optional password. The client dials through the running tunnel
with a plain net.Dialer and streams PTY output back to Swift via the
gomobile-bound listener for rendering in the terminal view.

Adds a config field and sshState() accessor to the iOS Client so the
SSH client can reach the active config and engine.
2026-05-27 18:29:12 +02:00
26 changed files with 849 additions and 1786 deletions

View File

@@ -42,12 +42,11 @@ type aclManager struct {
optionalEntries map[string][]entry
ipsetStore *ipsetStore
v6 bool
ipsetSupport *ipsetSupport
stateManager *statemanager.Manager
}
func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper, ipsetSupport *ipsetSupport) (*aclManager, error) {
func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper) (*aclManager, error) {
return &aclManager{
iptablesClient: iptablesClient,
wgIface: wgIface,
@@ -55,7 +54,6 @@ func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper, ipset
optionalEntries: make(map[string][]entry),
ipsetStore: newIpsetStore(),
v6: iptablesClient.Proto() == iptables.ProtocolIPv6,
ipsetSupport: ipsetSupport,
}, nil
}
@@ -86,78 +84,13 @@ func (m *aclManager) AddPeerFiltering(
dPort *firewall.Port,
action firewall.Action,
ipsetName string,
) ([]firewall.Rule, error) {
ipsetName = m.resolveIPSetName(ipsetName, sPort, dPort, action)
if ipsetName == "" {
return m.addPeerRule(ip, protocol, sPort, dPort, action, "")
}
// A set that is already in the store backs rules installed earlier, so it must
// survive this call's failure.
_, preexisting := m.ipsetStore.ipset(ipsetName)
rules, err := m.addPeerRule(ip, protocol, sPort, dPort, action, ipsetName)
if err == nil {
return rules, nil
}
var unusable *ipsetUnusableError
if !errors.As(err, &unusable) {
return nil, err
}
// The set could not be created or matched. Drop the one this call created and
// retry the rule matching the IP directly; only if that succeeds do we know
// ipset was to blame and latch it off for subsequent rules.
if !preexisting {
m.discardIPSet(ipsetName)
}
rules, retryErr := m.addPeerRule(ip, protocol, sPort, dPort, action, "")
if retryErr != nil {
return nil, fmt.Errorf("add peer rule (ipset: %w): %w", unusable.cause, retryErr)
}
m.ipsetSupport.markUnsupported(unusable.cause)
return rules, nil
}
// resolveIPSetName derives the ipset name for a rule, returning "" when the rule
// must match the IP directly: either no set was requested or ipset is unusable.
func (m *aclManager) resolveIPSetName(ipsetName string, sPort, dPort *firewall.Port, action firewall.Action) string {
ipsetName = transformIPsetName(ipsetName, sPort, dPort, action)
if ipsetName == "" || !m.ipsetSupport.supported() {
return ""
}
if m.v6 {
ipsetName += "-v6"
}
return ipsetName
}
// discardIPSet removes a set that turned out to be unusable, so a later rule
// does not find it in the store and assume it works.
func (m *aclManager) discardIPSet(ipsetName string) {
m.ipsetStore.deleteIpset(ipsetName)
if err := m.destroyIPSet(ipsetName); err != nil {
log.Debugf("destroy unusable ipset %s: %v", ipsetName, err)
}
}
func (m *aclManager) addPeerRule(
ip net.IP,
protocol firewall.Protocol,
sPort *firewall.Port,
dPort *firewall.Port,
action firewall.Action,
ipsetName string,
) ([]firewall.Rule, error) {
chain := chainNameInputRules
ipsetName = transformIPsetName(ipsetName, sPort, dPort, action)
if m.v6 && ipsetName != "" {
ipsetName += "-v6"
}
proto := protoForFamily(protocol, m.v6)
specs := filterRuleSpecs(ip, proto, sPort, dPort, action, ipsetName)
@@ -172,7 +105,7 @@ func (m *aclManager) addPeerRule(
if ipsetName != "" {
if ipList, ipsetExists := m.ipsetStore.ipset(ipsetName); ipsetExists {
if err := m.addToIPSet(ipsetName, ip); err != nil {
return nil, ipsetUnusable(fmt.Errorf("add IP to ipset: %w", err))
return nil, fmt.Errorf("add IP to ipset: %w", err)
}
// if ruleset already exists it means we already have the firewall rule
// so we need to update IPs in the ruleset and return new fw.Rule object for ACL manager.
@@ -195,10 +128,10 @@ func (m *aclManager) addPeerRule(
}
}
if err := m.createIPSet(ipsetName); err != nil {
return nil, ipsetUnusable(fmt.Errorf("create ipset: %w", err))
return nil, fmt.Errorf("create ipset: %w", err)
}
if err := m.addToIPSet(ipsetName, ip); err != nil {
return nil, ipsetUnusable(fmt.Errorf("add IP to ipset: %w", err))
return nil, fmt.Errorf("add IP to ipset: %w", err)
}
ipList := newIpList(ip.String())
@@ -207,7 +140,7 @@ func (m *aclManager) addPeerRule(
ok, err := m.iptablesClient.Exists(tableFilter, chain, specs...)
if err != nil {
return nil, maybeIPSetUnusable(ipsetName, fmt.Errorf("check rule: %w", err))
return nil, fmt.Errorf("failed to check rule: %w", err)
}
if ok {
return nil, fmt.Errorf("rule already exists")
@@ -221,7 +154,7 @@ func (m *aclManager) addPeerRule(
err = m.iptablesClient.Append(tableFilter, chain, specs...)
}
if err != nil {
return nil, maybeIPSetUnusable(ipsetName, err)
return nil, err
}
if err := m.iptablesClient.Append(tableMangle, chainRTPRE, mangleSpecs...); err != nil {

View File

@@ -1,240 +0,0 @@
//go:build privileged
package iptables
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
fw "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/iface/wgaddr"
)
func iptRefcountIfaceV4() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("10.20.0.1"),
Network: netip.MustParsePrefix("10.20.0.0/24"),
}
},
}
}
func iptRefcountIfaceDual() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("10.20.0.1"),
Network: netip.MustParsePrefix("10.20.0.0/24"),
IPv6: netip.MustParseAddr("fd00::1"),
IPv6Net: netip.MustParsePrefix("fd00::/64"),
}
},
}
}
func newIptRefcountManager(t *testing.T, dual bool) *Manager {
t.Helper()
var ifMock *iFaceMock
if dual {
ifMock = iptRefcountIfaceDual()
} else {
ifMock = iptRefcountIfaceV4()
}
m, err := Create(ifMock, iface.DefaultMTU)
require.NoError(t, err, "create manager")
require.NoError(t, m.Init(nil), "init manager")
t.Cleanup(func() {
require.NoError(t, m.Close(nil), "close manager")
})
return m
}
func iptDnatV4(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("10.20.0.2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
func iptDnatV6(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("fd00::2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
// TestIptablesRouting_RepeatedEnableSingleReference verifies that EnableRouting
// (called on every network-map update) holds at most one reference per family
// and a single DisableRouting drops both back to zero.
func TestIptablesRouting_RepeatedEnableSingleReference(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
require.NoError(t, m.EnableRouting(), "first enable")
require.NoError(t, m.EnableRouting(), "second enable")
require.NoError(t, m.EnableRouting(), "third enable")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "repeated enable holds a single v4 reference")
assert.Equal(t, 1, v6, "repeated enable holds a single v6 reference")
require.NoError(t, m.DisableRouting(), "disable")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "single disable releases the v4 reference")
assert.Equal(t, 0, v6, "single disable releases the v6 reference")
}
// TestIptablesRouting_DisableKeepsDNATReference verifies that an unpaired
// DisableRouting does not release references held by active DNAT rules.
func TestIptablesRouting_DisableKeepsDNATReference(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9095))
require.NoError(t, err, "add v6 dnat")
require.NoError(t, m.DisableRouting(), "unpaired disable")
_, v6 := state.Counts()
assert.Equal(t, 1, v6, "DNAT-held reference survives unpaired DisableRouting")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "delete releases the DNAT reference")
}
// TestIptablesDNAT_RefcountBalancedV4 covers a Balanced Add/Delete pair on v4.
func TestIptablesDNAT_RefcountBalancedV4(t *testing.T) {
m := newIptRefcountManager(t, false)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV4(7081))
require.NoError(t, err, "add v4 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
r2, err := m.AddDNATRule(iptDnatV4(7082))
require.NoError(t, err, "add v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 2, v4, "v4 refcount after second add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r1))
v4, v6 = state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r2))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount after second delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
}
// TestIptablesDNAT_RefcountBalancedV6 checks the v6 path increments v6 only and
// decrements back to zero.
func TestIptablesDNAT_RefcountBalancedV6(t *testing.T) {
m := newIptRefcountManager(t, true)
require.NotNil(t, m.router6, "v6 router")
require.Same(t, m.router.ipFwdState, m.router6.ipFwdState, "shared state")
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9081))
require.NoError(t, err, "add v6 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 1, v6, "v6 refcount after first add")
r2, err := m.AddDNATRule(iptDnatV6(9082))
require.NoError(t, err, "add v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 2, v6, "v6 refcount after second add")
require.NoError(t, m.DeleteDNATRule(r1))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first delete")
require.NoError(t, m.DeleteDNATRule(r2))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount after second delete")
}
// TestIptablesDNAT_DuplicateAddNoLeak verifies the duplicate-rule path returns
// without bumping the refcount.
func TestIptablesDNAT_DuplicateAddNoLeak(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
rule := iptDnatV4(7083)
r1, err := m.AddDNATRule(rule)
require.NoError(t, err)
v4, _ := state.Counts()
assert.Equal(t, 1, v4)
_, err = m.AddDNATRule(rule)
require.NoError(t, err, "duplicate add")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "duplicate add must not increment")
require.NoError(t, m.DeleteDNATRule(r1))
v4, _ = state.Counts()
assert.Equal(t, 0, v4, "single delete must drop to zero")
}
// TestIptablesDNAT_DeleteMissingNoUnderflow verifies Delete on an unknown rule
// neither errors nor releases the refcount.
func TestIptablesDNAT_DeleteMissingNoUnderflow(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
phantom := iptDnatV4(7099)
require.NoError(t, m.DeleteDNATRule(&phantom), "delete missing v4")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6)
phantom6 := iptDnatV6(9099)
require.NoError(t, m.DeleteDNATRule(&phantom6), "delete missing v6")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6)
r1, err := m.AddDNATRule(iptDnatV4(7100))
require.NoError(t, err)
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "real add still increments after phantom delete")
require.NoError(t, m.DeleteDNATRule(r1))
}
// TestIptablesDNAT_DoubleDeleteNoUnderflow verifies a second Delete on the same
// rule is a no-op.
func TestIptablesDNAT_DoubleDeleteNoUnderflow(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9083))
require.NoError(t, err)
_, v6 := state.Counts()
assert.Equal(t, 1, v6)
require.NoError(t, m.DeleteDNATRule(r1), "first delete")
_, v6 = state.Counts()
assert.Equal(t, 0, v6)
require.NoError(t, m.DeleteDNATRule(r1), "second delete must be no-op")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "double delete must not underflow")
}

View File

@@ -1,75 +0,0 @@
package iptables
import (
"sync"
log "github.com/sirupsen/logrus"
)
// ipsetSupport tracks whether ipset-backed firewall rules can be installed.
//
// It starts optimistic and latches to unsupported the first time the kernel
// proves otherwise: either the hash:net set type is missing (ip_set_hash_net) or
// iptables cannot match against a set (xt_set). Callers then emit per-IP and
// per-prefix rules instead. Without the fallback, a rule referencing an unusable
// set is never installed and the catch-all DROP silently blocks traffic the
// policy permits.
//
// One instance is shared by the ACL managers and routers of both address
// families, because ipset availability is a property of the kernel rather than
// of any single table.
type ipsetSupport struct {
mu sync.RWMutex
unsupported bool
}
func newIPSetSupport() *ipsetSupport {
return &ipsetSupport{}
}
func (s *ipsetSupport) supported() bool {
s.mu.RLock()
defer s.mu.RUnlock()
return !s.unsupported
}
// markUnsupported records that ipset cannot be used, logging the reason once.
func (s *ipsetSupport) markUnsupported(cause error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.unsupported {
return
}
s.unsupported = true
log.Warnf("ipset is unavailable (%v); falling back to per-IP firewall rules. "+
"Ensure the kernel provides ip_set_hash_net and xt_set; without them rule "+
"sets are larger and slower to converge on networks with many peers", cause)
}
// ipsetUnusableError marks a failure attributable to ipset, so the caller can
// retry the same rule in its per-IP form before latching the capability off.
type ipsetUnusableError struct {
cause error
}
func (e *ipsetUnusableError) Error() string { return e.cause.Error() }
func (e *ipsetUnusableError) Unwrap() error { return e.cause }
func ipsetUnusable(cause error) error {
return &ipsetUnusableError{cause: cause}
}
// maybeIPSetUnusable marks an iptables failure as ipset-attributable only when the
// rule actually carried a set match, since the same call can fail for unrelated
// reasons on a rule that matches addresses directly.
func maybeIPSetUnusable(ipsetName string, err error) error {
if ipsetName == "" {
return err
}
return ipsetUnusable(err)
}

View File

@@ -33,10 +33,6 @@ type Manager struct {
router *router
rawSupported bool
// ipsetSupport is shared by the ACL managers and routers of both families,
// so a kernel without usable ipset support degrades them together.
ipsetSupport *ipsetSupport
// IPv6 counterparts, nil when no v6 overlay
ipv6Client *iptables.IPTables
aclMgr6 *aclManager
@@ -57,17 +53,16 @@ func Create(wgIface iFaceMapper, mtu uint16) (*Manager, error) {
}
m := &Manager{
wgIface: wgIface,
ipv4Client: iptablesClient,
ipsetSupport: newIPSetSupport(),
wgIface: wgIface,
ipv4Client: iptablesClient,
}
m.router, err = newRouter(iptablesClient, wgIface, mtu, m.ipsetSupport)
m.router, err = newRouter(iptablesClient, wgIface, mtu)
if err != nil {
return nil, fmt.Errorf("create router: %w", err)
}
m.aclMgr, err = newAclManager(iptablesClient, wgIface, m.ipsetSupport)
m.aclMgr, err = newAclManager(iptablesClient, wgIface)
if err != nil {
return nil, fmt.Errorf("create acl manager: %w", err)
}
@@ -88,16 +83,16 @@ func (m *Manager) createIPv6Components(wgIface iFaceMapper, mtu uint16) error {
}
m.ipv6Client = ip6Client
m.router6, err = newRouter(ip6Client, wgIface, mtu, m.ipsetSupport)
m.router6, err = newRouter(ip6Client, wgIface, mtu)
if err != nil {
return fmt.Errorf("create v6 router: %w", err)
}
// Share the same IP forwarding state with the v4 router, since
// Forwarding refcounter is per-family but shared between v4 and v6 routers.
// EnableIPForwarding controls both v4 and v6 sysctls.
m.router6.ipFwdState = m.router.ipFwdState
m.aclMgr6, err = newAclManager(ip6Client, wgIface, m.ipsetSupport)
m.aclMgr6, err = newAclManager(ip6Client, wgIface)
if err != nil {
return fmt.Errorf("create v6 acl manager: %w", err)
}
@@ -407,12 +402,17 @@ func (m *Manager) SetLogLevel(log.Level) {
}
func (m *Manager) EnableRouting() error {
// v6 only when the overlay actually has v6.
return m.router.ipFwdState.RequestRouting(m.router6 != nil)
if err := m.router.ipFwdState.RequestForwarding(); err != nil {
return fmt.Errorf("enable IP forwarding: %w", err)
}
return nil
}
func (m *Manager) DisableRouting() error {
return m.router.ipFwdState.ReleaseRouting()
if err := m.router.ipFwdState.ReleaseForwarding(); err != nil {
return fmt.Errorf("disable IP forwarding: %w", err)
}
return nil
}
// AddDNATRule adds a DNAT rule

View File

@@ -291,98 +291,3 @@ func TestIptablesCreatePerformance(t *testing.T) {
})
}
}
// newACLTestManager returns a started manager. Create()/Init() is used so the
// router-owned chains (chainRTFWDIN/OUT) exist before the ACL manager's
// createDefaultChains() references them.
func newACLTestManager(t *testing.T) *Manager {
t.Helper()
manager, err := Create(ifaceMock, iface.DefaultMTU)
require.NoError(t, err)
require.NoError(t, manager.Init(nil))
t.Cleanup(func() {
require.NoError(t, manager.Close(nil))
})
return manager
}
// TestIptablesACLUsesIPSetOnHealthyKernel guards the default: on a kernel that
// does have ipset, rules must keep matching a set. A regression that reported
// ipset as unusable would silently move every Linux client to per-IP rules.
func TestIptablesACLUsesIPSetOnHealthyKernel(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
manager := newACLTestManager(t)
ip := netip.MustParseAddr("10.20.0.42")
port := &fw.Port{Values: []uint16{22}}
rules, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err)
require.NotEmpty(t, rules)
rule := rules[0].(*Rule)
require.Equal(t, "nb0000001-dport", rule.ipsetName, "healthy kernel must use an ipset")
require.Contains(t, rule.specs, "--match-set")
require.True(t, manager.ipsetSupport.supported(), "ipset must not be latched off on a healthy kernel")
checkRuleSpecs(t, ipv4Client, rule.chain, true, rule.specs...)
}
// TestIptablesACLFallsBackWhenIPSetUnusable drives the real failure path: an
// oversized set name is rejected by the kernel, which stands in for a kernel
// without ip_set_hash_net or xt_set. The rule must still land in the chain,
// matching the IP directly, and the capability must latch off so later rules skip
// ipset. Before the fallback existed, the rule was dropped and the catch-all DROP
// silently blocked traffic the policy permits.
func TestIptablesACLFallsBackWhenIPSetUnusable(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
manager := newACLTestManager(t)
// ipset names are limited to 31 characters, so creating this set fails.
unusableName := strings.Repeat("a", 40)
ip := netip.MustParseAddr("10.20.0.42")
port := &fw.Port{Values: []uint16{22}}
rules, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, unusableName)
require.NoError(t, err, "AddPeerFiltering must succeed by falling back")
require.NotEmpty(t, rules)
rule := rules[0].(*Rule)
require.Empty(t, rule.ipsetName, "fallback rule must not reference an ipset")
require.Contains(t, strings.Join(rule.specs, " "), "-s 10.20.0.42", "fallback rule must match the source IP")
require.NotContains(t, strings.Join(rule.specs, " "), "--match-set")
// The rule must actually be present, not silently missing.
checkRuleSpecs(t, ipv4Client, rule.chain, true, rule.specs...)
require.False(t, manager.ipsetSupport.supported(), "failure must latch ipset off")
// A subsequent rule with a perfectly valid set name now skips ipset too.
next, err := manager.aclMgr.AddPeerFiltering(nil, netip.MustParseAddr("10.20.0.43").AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err)
require.NotEmpty(t, next)
require.Empty(t, next[0].(*Rule).ipsetName, "later rules must skip ipset once latched")
}
// TestIptablesACLLeavesNoIPSetAfterFallback verifies the set created before the
// failure is destroyed, so a later rule does not find a half-built set and assume
// ipset works.
func TestIptablesACLLeavesNoIPSetAfterFallback(t *testing.T) {
manager := newACLTestManager(t)
port := &fw.Port{Values: []uint16{22}}
ip := netip.MustParseAddr("10.20.0.42")
_, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, strings.Repeat("a", 40))
require.NoError(t, err)
_, exists := manager.aclMgr.ipsetStore.ipset(strings.Repeat("a", 40) + "-dport")
require.False(t, exists, "unusable set must not stay in the store")
}

View File

@@ -3,7 +3,6 @@
package iptables
import (
"errors"
"fmt"
"maps"
"net/netip"
@@ -52,10 +51,6 @@ const (
markManglePost = "mark-mangle-post"
matchSet = "--match-set"
// routeSourceSuffix names the extra rules a route ACL needs when ipset is
// unusable and each source prefix has to be matched by its own rule.
routeSourceSuffix = "_src"
dnatSuffix = "_dnat"
snatSuffix = "_snat"
fwdSuffix = "_fwd"
@@ -73,6 +68,7 @@ type ruleInfo struct {
}
type routeFilteringRuleParams struct {
Source firewall.Network
Destination firewall.Network
Proto firewall.Protocol
SPort *firewall.Port
@@ -94,21 +90,19 @@ type router struct {
legacyManagement bool
mtu uint16
v6 bool
ipsetSupport *ipsetSupport
stateManager *statemanager.Manager
ipFwdState *ipfwdstate.IPForwardingState
}
func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint16, ipsetSupport *ipsetSupport) (*router, error) {
func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint16) (*router, error) {
r := &router{
iptablesClient: iptablesClient,
rules: make(map[string][]string),
wgIface: wgIface,
mtu: mtu,
v6: iptablesClient.Proto() == iptables.ProtocolIPv6,
ipFwdState: ipfwdstate.NewIPForwardingState(wgIface.Name()),
ipsetSupport: ipsetSupport,
ipFwdState: ipfwdstate.NewIPForwardingState(),
}
r.ipsetCounter = refcounter.New(
@@ -157,113 +151,6 @@ func (r *router) AddRouteFiltering(
return ruleKey, nil
}
params := routeFilteringRuleParams{
Destination: destination,
Proto: proto,
SPort: sPort,
DPort: dPort,
Action: action,
}
err := r.installRouteRules(string(ruleKey), params, sources, r.ipsetSupport.supported())
var unusable *ipsetUnusableError
if errors.As(err, &unusable) {
// The set could not be created or matched. Retry matching each source
// prefix on its own; only if that works do we know ipset was to blame.
r.removeRouteRules(string(ruleKey))
if retryErr := r.installRouteRules(string(ruleKey), params, sources, false); retryErr != nil {
r.removeRouteRules(string(ruleKey))
return nil, fmt.Errorf("add route rule (ipset: %w): %w", unusable.cause, retryErr)
}
r.ipsetSupport.markUnsupported(unusable.cause)
err = nil
}
if err != nil {
// Leave nothing half-installed: a later call finding the rule key would
// report success while some sources were never installed, which for a
// drop rule would leave them unblocked.
r.removeRouteRules(string(ruleKey))
return nil, fmt.Errorf("add route rule: %w", err)
}
r.updateState()
return ruleKey, nil
}
// installRouteRules installs every rule needed for one route ACL and records them
// under ruleKey. It is more than one rule only when useIPSet is false and the
// sources have to be matched one prefix at a time.
func (r *router) installRouteRules(ruleKey string, params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool) error {
specs, err := r.genRouteRuleSpecs(params, sources, useIPSet)
if err != nil {
return fmt.Errorf("generate route rule spec: %w", err)
}
for i, spec := range specs {
if err := r.insertRouteRule(params.Action, spec); err != nil {
if len(r.findSets(spec)) > 0 {
return ipsetUnusable(err)
}
return err
}
r.rules[routeRuleKey(ruleKey, i)] = spec
}
return nil
}
// genRouteRuleSpecs builds the rules for one route ACL. With ipset available that
// is a single rule matching a set of sources; without it, one rule per source
// prefix, which is the only form a stripped kernel can express.
func (r *router) genRouteRuleSpecs(params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool) ([][]string, error) {
destExp, err := r.applyNetwork("-d", params.Destination, nil)
if err != nil {
return nil, fmt.Errorf("apply network -d: %w", err)
}
specs, err := r.genSourceRules(params, sources, useIPSet, destExp)
if err != nil {
// The destination match may have taken a set reference already.
if decErr := r.decrementSetCounter(destExp); decErr != nil {
log.Debugf("release destination set after failed rule generation: %v", decErr)
}
return nil, err
}
return specs, nil
}
func (r *router) genSourceRules(params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool, destExp []string) ([][]string, error) {
if useIPSet || len(sources) <= 1 {
sourceExp, err := r.applyNetwork("-s", sourceNetwork(sources), sources)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
return [][]string{assembleRouteRule(sourceExp, destExp, params, r.v6)}, nil
}
specs := make([][]string, 0, len(sources))
for _, source := range sources {
sourceExp, err := r.applyNetwork("-s", firewall.Network{Prefix: source}, nil)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
specs = append(specs, assembleRouteRule(sourceExp, destExp, params, r.v6))
}
return specs, nil
}
func sourceNetwork(sources []netip.Prefix) firewall.Network {
var source firewall.Network
if len(sources) > 1 {
source.Set = firewall.NewPrefixSet(sources)
@@ -271,48 +158,37 @@ func sourceNetwork(sources []netip.Prefix) firewall.Network {
source.Prefix = sources[0]
}
return source
}
params := routeFilteringRuleParams{
Source: source,
Destination: destination,
Proto: proto,
SPort: sPort,
DPort: dPort,
Action: action,
}
rule, err := r.genRouteRuleSpec(params, sources)
if err != nil {
return nil, fmt.Errorf("generate route rule spec: %w", err)
}
func (r *router) insertRouteRule(action firewall.Action, spec []string) error {
// Insert DROP rules at the beginning, append ACCEPT rules at the end
if action == firewall.ActionDrop {
// after the established rule
return r.iptablesClient.Insert(tableFilter, chainRTFWDIN, 2, spec...)
err = r.iptablesClient.Insert(tableFilter, chainRTFWDIN, 2, rule...)
} else {
err = r.iptablesClient.Append(tableFilter, chainRTFWDIN, rule...)
}
return r.iptablesClient.Append(tableFilter, chainRTFWDIN, spec...)
}
// removeRouteRules deletes the rules recorded for ruleKey, used to undo a partial
// install before retrying without ipset.
func (r *router) removeRouteRules(ruleKey string) {
for i := 0; ; i++ {
key := routeRuleKey(ruleKey, i)
spec, exists := r.rules[key]
if !exists {
return
}
if err := r.iptablesClient.DeleteIfExists(tableFilter, chainRTFWDIN, spec...); err != nil {
log.Debugf("delete partial route rule %s: %v", key, err)
}
delete(r.rules, key)
if err := r.decrementSetCounter(spec); err != nil {
log.Debugf("decrement ipset counter for %s: %v", key, err)
}
}
}
// routeRuleKey names the i-th rule of a route ACL. The first keeps the plain rule
// key so single-rule ACLs, which is every ACL when ipset works, are unaffected.
func routeRuleKey(ruleKey string, i int) string {
if i == 0 {
return ruleKey
if err != nil {
return nil, fmt.Errorf("add route rule: %v", err)
}
return fmt.Sprintf("%s%s%d", ruleKey, routeSourceSuffix, i)
r.rules[string(ruleKey)] = rule
r.updateState()
return ruleKey, nil
}
func (r *router) hasRule(id string) bool {
@@ -323,29 +199,17 @@ func (r *router) hasRule(id string) bool {
func (r *router) DeleteRouteRule(rule firewall.Rule) error {
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey]; !exists {
log.Debugf("route rule %s not found", ruleKey)
r.updateState()
return nil
}
// In the ipset fallback one ACL is several rules, one per source prefix.
for i := 0; ; i++ {
key := routeRuleKey(ruleKey, i)
rule, exists := r.rules[key]
if !exists {
break
}
if rule, exists := r.rules[ruleKey]; exists {
if err := r.iptablesClient.Delete(tableFilter, chainRTFWDIN, rule...); err != nil {
return fmt.Errorf("delete route rule: %v", err)
}
delete(r.rules, key)
delete(r.rules, ruleKey)
if err := r.decrementSetCounter(rule); err != nil {
return fmt.Errorf("decrement ipset counter: %w", err)
}
} else {
log.Debugf("route rule %s not found", ruleKey)
}
r.updateState()
@@ -906,6 +770,10 @@ func (r *router) updateState() {
}
func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
if err := r.ipFwdState.RequestForwarding(); err != nil {
return nil, err
}
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey+dnatSuffix]; exists {
return rule, nil
@@ -972,34 +840,18 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
for key, ruleInfo := range rules {
if err := r.iptablesClient.Append(ruleInfo.table, ruleInfo.chain, ruleInfo.rule...); err != nil {
r.cleanupFailedDNATAdd(rules)
if rollbackErr := r.rollbackRules(rules); rollbackErr != nil {
log.Errorf("rollback failed: %v", rollbackErr)
}
return nil, fmt.Errorf("add rule %s: %w", key, err)
}
r.rules[key] = ruleInfo.rule
}
if err := r.ipFwdState.RequestForwarding(r.v6); err != nil {
r.cleanupFailedDNATAdd(rules)
return nil, fmt.Errorf("enable forwarding: %w", err)
}
r.updateState()
return rule, nil
}
// cleanupFailedDNATAdd removes the bookkeeping written by a partially applied
// AddDNATRule before rolling back the kernel rules, so no entries remain that
// never got a forwarding refcount. rollbackRules re-adds entries it failed to
// remove from the kernel.
func (r *router) cleanupFailedDNATAdd(rules map[string]ruleInfo) {
for key := range rules {
delete(r.rules, key)
}
if err := r.rollbackRules(rules); err != nil {
log.Errorf("rollback failed: %v", err)
}
}
func (r *router) rollbackRules(rules map[string]ruleInfo) error {
var merr *multierror.Error
for key, ruleInfo := range rules {
@@ -1016,70 +868,63 @@ func (r *router) rollbackRules(rules map[string]ruleInfo) error {
}
func (r *router) DeleteDNATRule(rule firewall.Rule) error {
ruleKey := rule.ID()
_, hadDNAT := r.rules[ruleKey+dnatSuffix]
_, hadSNAT := r.rules[ruleKey+snatSuffix]
_, hadFWD := r.rules[ruleKey+fwdSuffix]
if !hadDNAT && !hadSNAT && !hadFWD {
return nil
if err := r.ipFwdState.ReleaseForwarding(); err != nil {
log.Errorf("%v", err)
}
ruleKey := rule.ID()
var merr *multierror.Error
if dnatRule, exists := r.rules[ruleKey+dnatSuffix]; exists {
if err := r.iptablesClient.Delete(tableNat, chainRTRDR, dnatRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete DNAT rule: %w", err))
} else {
delete(r.rules, ruleKey+dnatSuffix)
}
delete(r.rules, ruleKey+dnatSuffix)
}
if snatRule, exists := r.rules[ruleKey+snatSuffix]; exists {
if err := r.iptablesClient.Delete(tableNat, chainRTNAT, snatRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete SNAT rule: %w", err))
} else {
delete(r.rules, ruleKey+snatSuffix)
}
delete(r.rules, ruleKey+snatSuffix)
}
if fwdRule, exists := r.rules[ruleKey+fwdSuffix]; exists {
if err := r.iptablesClient.Delete(tableFilter, chainRTFWDOUT, fwdRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete forward rule: %w", err))
} else {
delete(r.rules, ruleKey+fwdSuffix)
}
}
// Release the refcount only once all rules are gone from the kernel. On
// partial failure the failed entries stay in r.rules so a retry can remove
// them and release then.
if merr == nil {
if err := r.ipFwdState.ReleaseForwarding(r.v6); err != nil {
log.Errorf("%v", err)
}
delete(r.rules, ruleKey+fwdSuffix)
}
r.updateState()
return nberrors.FormatErrorOrNil(merr)
}
// assembleRouteRule joins the pre-built source and destination matches with the
// protocol, ports and target of a route ACL.
func assembleRouteRule(sourceExp, destExp []string, params routeFilteringRuleParams, v6 bool) []string {
func (r *router) genRouteRuleSpec(params routeFilteringRuleParams, sources []netip.Prefix) ([]string, error) {
var rule []string
sourceExp, err := r.applyNetwork("-s", params.Source, sources)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
destExp, err := r.applyNetwork("-d", params.Destination, nil)
if err != nil {
return nil, fmt.Errorf("apply network -d: %w", err)
}
rule = append(rule, sourceExp...)
rule = append(rule, destExp...)
if params.Proto != firewall.ProtocolALL {
rule = append(rule, "-p", strings.ToLower(protoForFamily(params.Proto, v6)))
rule = append(rule, "-p", strings.ToLower(protoForFamily(params.Proto, r.v6)))
rule = append(rule, applyPort("--sport", params.SPort)...)
rule = append(rule, applyPort("--dport", params.DPort)...)
}
rule = append(rule, "-j", actionToStr(params.Action))
return rule
return rule, nil
}
func (r *router) applyNetwork(flag string, network firewall.Network, prefixes []netip.Prefix) ([]string, error) {
@@ -1089,17 +934,9 @@ func (r *router) applyNetwork(flag string, network firewall.Network, prefixes []
}
if network.IsSet() {
// A destination set is populated later from DNS results, so unlike a
// source set it cannot be expanded into per-prefix rules here. Without
// ipset such a rule is not expressible; report it instead of installing
// something broader than the policy allows.
if flag == "-d" && !r.ipsetSupport.supported() {
return nil, fmt.Errorf("destination set %s requires ipset (ip_set_hash_net and xt_set)", network.Set.HashedName())
}
name := r.ipsetName(network.Set.HashedName())
if _, err := r.ipsetCounter.Increment(name, prefixes); err != nil {
return nil, ipsetUnusable(fmt.Errorf("create or get ipset: %w", err))
return nil, fmt.Errorf("create or get ipset: %w", err)
}
return []string{"-m", "set", matchSet, name, direction}, nil

View File

@@ -3,11 +3,9 @@
package iptables
import (
"errors"
"fmt"
"net/netip"
"os/exec"
"strings"
"testing"
"github.com/coreos/go-iptables/iptables"
@@ -17,9 +15,7 @@ import (
firewall "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/firewall/test"
"github.com/netbirdio/netbird/client/iface"
nbid "github.com/netbirdio/netbird/client/internal/acl/id"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/shared/management/domain"
)
func isIptablesSupported() bool {
@@ -35,7 +31,7 @@ func TestIptablesManager_RestoreOrCreateContainers(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "failed to init iptables client")
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "should return a valid iptables manager")
require.NoError(t, manager.init(nil))
@@ -88,7 +84,7 @@ func TestIptablesManager_AddNatRule(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "failed to init iptables client")
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "shouldn't return error")
require.NoError(t, manager.init(nil))
@@ -161,7 +157,7 @@ func TestIptablesManager_RemoveNatRule(t *testing.T) {
t.Run(testCase.Name, func(t *testing.T) {
iptablesClient, _ := iptables.NewWithProtocol(iptables.ProtocolIPv4)
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "shouldn't return error")
require.NoError(t, manager.init(nil))
defer func() {
@@ -223,7 +219,7 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "Failed to create iptables client")
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "Failed to create router manager")
require.NoError(t, r.init(nil))
@@ -341,26 +337,27 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
ruleKey, err := r.AddRouteFiltering(nil, tt.sources, firewall.Network{Prefix: tt.destination}, tt.proto, tt.sPort, tt.dPort, tt.action)
require.NoError(t, err, "AddRouteFiltering failed")
// A kernel without usable ipset splits a multi-source ACL into one
// rule per source, so compare against whichever form is in effect.
useIPSet := r.ipsetSupport.supported()
// Check if the rule is in the internal map
rule, ok := r.rules[ruleKey.ID()]
assert.True(t, ok, "Rule not found in internal map")
// Check if the rules are in the internal map
rules := routeRuleSpecs(t, r, ruleKey.ID())
require.NotEmpty(t, rules, "Rule not found in internal map")
// Log the internal rule
t.Logf("Internal rule: %v", rule)
// Log the internal rules
t.Logf("Internal rules: %v", rules)
// Check if the rule exists in iptables
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, rule...)
assert.NoError(t, err, "Failed to check rule existence")
assert.True(t, exists, "Rule not found in iptables")
// Check if the rules exist in iptables
for _, rule := range rules {
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, rule...)
assert.NoError(t, err, "Failed to check rule existence")
assert.True(t, exists, "Rule not found in iptables")
var source firewall.Network
if len(tt.sources) > 1 {
source.Set = firewall.NewPrefixSet(tt.sources)
} else if len(tt.sources) > 0 {
source.Prefix = tt.sources[0]
}
// Verify rule content
params := routeFilteringRuleParams{
Source: source,
Destination: firewall.Network{Prefix: tt.destination},
Proto: tt.proto,
SPort: tt.sPort,
@@ -368,18 +365,20 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
Action: tt.action,
}
expectedRules, err := r.genRouteRuleSpecs(params, tt.sources, useIPSet)
expectedRule, err := r.genRouteRuleSpec(params, nil)
require.NoError(t, err, "Failed to generate expected rule spec")
if tt.expectSet && useIPSet {
if tt.expectSet {
setName := firewall.NewPrefixSet(tt.sources).HashedName()
expectedRule, err = r.genRouteRuleSpec(params, nil)
require.NoError(t, err, "Failed to generate expected rule spec with set")
// Check if the set was created
_, exists := r.ipsetCounter.Get(setName)
assert.True(t, exists, "IPSet not created")
}
assert.Equal(t, expectedRules, rules, "Rule content mismatch")
assert.Equal(t, expectedRule, rule, "Rule content mismatch")
// Clean up
err = r.DeleteRouteRule(ruleKey)
@@ -446,145 +445,3 @@ func TestFindSetNameInRule(t *testing.T) {
})
}
}
// TestRouter_AddRouteFilteringIPSetFallback covers a kernel that cannot use ipset:
// a multi-source route ACL must become one rule per source prefix, all present in
// the chain, and deleting the ACL must remove every one of them. Without the
// fallback the rule was never installed and the interface-wide DROP in FORWARD
// silently dropped routed traffic.
func TestRouter_AddRouteFilteringIPSetFallback(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
sources := []netip.Prefix{
netip.MustParsePrefix("172.16.0.0/16"),
netip.MustParsePrefix("192.168.0.0/16"),
}
destination := firewall.Network{Prefix: netip.MustParsePrefix("10.0.0.0/8")}
rule, err := r.AddRouteFiltering(nil, sources, destination, firewall.ProtocolTCP, nil,
&firewall.Port{Values: []uint16{443}}, firewall.ActionAccept)
require.NoError(t, err, "route ACL must install without ipset")
specs := routeRuleSpecs(t, r, rule.ID())
require.Len(t, specs, len(sources), "each source prefix needs its own rule")
for i, spec := range specs {
joined := strings.Join(spec, " ")
require.Contains(t, joined, "-s "+sources[i].String(), "rule must match the source prefix directly")
require.NotContains(t, joined, matchSet, "fallback rule must not reference a set")
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, spec...)
require.NoError(t, err)
require.True(t, exists, "rule %d must be present in %s", i, chainRTFWDIN)
}
require.NoError(t, r.DeleteRouteRule(rule))
for i, spec := range specs {
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, spec...)
require.NoError(t, err)
require.False(t, exists, "rule %d must be removed", i)
}
require.Empty(t, routeRuleSpecs(t, r, rule.ID()), "no rule may be left recorded")
}
// TestRouter_DestinationSetRequiresIPSet documents that a dynamic (domain)
// destination cannot be expressed without ipset: its prefixes are only known
// after DNS resolution, so there is nothing to expand into per-prefix rules. The
// call must report that rather than install a broader rule than the policy allows.
func TestRouter_DestinationSetRequiresIPSet(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
destination := firewall.Network{Set: firewall.NewDomainSet(domain.List{"example.com"})}
_, err = r.AddRouteFiltering(nil, []netip.Prefix{netip.MustParsePrefix("172.16.0.0/16")},
destination, firewall.ProtocolALL, nil, nil, firewall.ActionAccept)
require.Error(t, err, "a domain destination is not expressible without ipset")
require.ErrorContains(t, err, "requires ipset")
}
// TestRouter_RouteFilteringRollsBackPartialInstall covers a fallback ACL whose
// second rule cannot be installed. Nothing may be left behind: if the rule key
// survived, a later call would short-circuit on it and report success while some
// sources were never installed, leaving them unblocked for a drop rule.
func TestRouter_RouteFilteringRollsBackPartialInstall(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
// The v6 prefix is rejected by the v4 iptables binary, so the second rule of
// the expansion fails after the first has been installed.
good := netip.MustParsePrefix("172.16.0.0/16")
sources := []netip.Prefix{good, netip.MustParsePrefix("2001:db8::/32")}
destination := firewall.Network{Prefix: netip.MustParsePrefix("10.0.0.0/8")}
_, err = r.AddRouteFiltering(nil, sources, destination, firewall.ProtocolALL, nil, nil, firewall.ActionDrop)
require.Error(t, err, "a source that iptables rejects must fail the whole ACL")
ruleKey := nbid.GenerateRouteRuleKey(sources, destination, firewall.ProtocolALL, nil, nil, firewall.ActionDrop)
require.Empty(t, routeRuleSpecs(t, r, string(ruleKey)), "no rule may stay recorded")
// The rule that did get installed must be gone from the chain.
installed := []string{"-s", good.String(), "-d", "10.0.0.0/8", "-j", "DROP"}
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, installed...)
require.NoError(t, err)
require.False(t, exists, "the already-installed rule must be rolled back")
}
// routeRuleSpecs collects the rules recorded for one route ACL, which is more than
// one when the ipset fallback splits it per source prefix.
func routeRuleSpecs(t *testing.T, r *router, ruleKey string) [][]string {
t.Helper()
var specs [][]string
for i := 0; ; i++ {
spec, exists := r.rules[routeRuleKey(ruleKey, i)]
if !exists {
return specs
}
specs = append(specs, spec)
}
}

View File

@@ -1,249 +0,0 @@
//go:build privileged
package nftables
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
fw "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/iface/wgaddr"
)
func nftRefcountIfaceV4() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("100.96.0.1"),
Network: netip.MustParsePrefix("100.96.0.0/16"),
}
},
}
}
func nftRefcountIfaceDual() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("100.96.0.1"),
Network: netip.MustParsePrefix("100.96.0.0/16"),
IPv6: netip.MustParseAddr("fd00::1"),
IPv6Net: netip.MustParsePrefix("fd00::/64"),
}
},
}
}
func newNftRefcountManager(t *testing.T, dual bool) *Manager {
t.Helper()
if check() != NFTABLES {
t.Skip("nftables not supported on this system")
}
var ifMock *iFaceMock
if dual {
ifMock = nftRefcountIfaceDual()
} else {
ifMock = nftRefcountIfaceV4()
}
m, err := Create(ifMock, iface.DefaultMTU)
require.NoError(t, err, "create manager")
require.NoError(t, m.Init(nil), "init manager")
t.Cleanup(func() {
require.NoError(t, m.Close(nil), "close manager")
})
return m
}
func dnatV4(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("100.96.0.2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
func dnatV6(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("fd00::2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
// TestNftablesDNAT_RefcountBalancedV4 verifies that Add/Delete pairs leave the
// v4 refcount at zero.
func TestNftablesDNAT_RefcountBalancedV4(t *testing.T) {
m := newNftRefcountManager(t, false)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV4(8081))
require.NoError(t, err, "add v4 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
r2, err := m.AddDNATRule(dnatV4(8082))
require.NoError(t, err, "add v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 2, v4, "v4 refcount after second add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r1), "delete v4 dnat 1")
v4, v6 = state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r2), "delete v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount after second delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
}
// TestNftablesDNAT_RefcountBalancedV6 verifies the v6 path increments v6 only
// and decrements back to zero on Delete.
func TestNftablesDNAT_RefcountBalancedV6(t *testing.T) {
m := newNftRefcountManager(t, true)
require.NotNil(t, m.router6, "v6 router")
require.Same(t, m.router.ipFwdState, m.router6.ipFwdState, "shared state")
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9091))
require.NoError(t, err, "add v6 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first add")
r2, err := m.AddDNATRule(dnatV6(9092))
require.NoError(t, err, "add v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 2, v6, "v6 refcount after second add")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat 1")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first delete")
require.NoError(t, m.DeleteDNATRule(r2), "delete v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount after second delete")
}
// TestNftablesDNAT_DuplicateAddNoLeak verifies that a duplicate Add (same
// ForwardRule) does not double-increment the refcount.
func TestNftablesDNAT_DuplicateAddNoLeak(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
rule := dnatV4(8083)
r1, err := m.AddDNATRule(rule)
require.NoError(t, err, "add v4 dnat")
v4, _ := state.Counts()
assert.Equal(t, 1, v4)
// duplicate add: same rule ID, must be a no-op for the refcount.
_, err = m.AddDNATRule(rule)
require.NoError(t, err, "duplicate add")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "duplicate add must not increment")
require.NoError(t, m.DeleteDNATRule(r1), "delete v4 dnat")
v4, _ = state.Counts()
assert.Equal(t, 0, v4, "single delete must drop to zero")
}
// TestNftablesDNAT_DeleteMissingNoUnderflow verifies deleting a rule that was
// never added does not underflow the refcount.
func TestNftablesDNAT_DeleteMissingNoUnderflow(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
// Construct a Rule reference for something never added. The router stores
// rules by ID(), and DeleteDNATRule looks them up in r.rules; a missing
// entry must be a no-op rather than calling Release.
phantom := dnatV4(8099)
require.NoError(t, m.DeleteDNATRule(&phantom), "delete missing v4 dnat")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unaffected by missing delete")
assert.Equal(t, 0, v6, "v6 refcount unaffected")
phantom6 := dnatV6(9099)
require.NoError(t, m.DeleteDNATRule(&phantom6), "delete missing v6 dnat")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount unaffected by missing delete")
// And after a phantom delete, a real add still results in count=1.
r1, err := m.AddDNATRule(dnatV4(8100))
require.NoError(t, err, "add v4 dnat after phantom delete")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "real add still increments after phantom delete")
require.NoError(t, m.DeleteDNATRule(r1))
}
// TestNftablesRouting_RepeatedEnableSingleReference verifies that EnableRouting
// (called on every network-map update) holds at most one reference per family
// and a single DisableRouting drops both back to zero.
func TestNftablesRouting_RepeatedEnableSingleReference(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
require.NoError(t, m.EnableRouting(), "first enable")
require.NoError(t, m.EnableRouting(), "second enable")
require.NoError(t, m.EnableRouting(), "third enable")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "repeated enable holds a single v4 reference")
assert.Equal(t, 1, v6, "repeated enable holds a single v6 reference")
require.NoError(t, m.DisableRouting(), "disable")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "single disable releases the v4 reference")
assert.Equal(t, 0, v6, "single disable releases the v6 reference")
}
// TestNftablesRouting_DisableKeepsDNATReference verifies that an unpaired
// DisableRouting does not release references held by active DNAT rules.
func TestNftablesRouting_DisableKeepsDNATReference(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9095))
require.NoError(t, err, "add v6 dnat")
require.NoError(t, m.DisableRouting(), "unpaired disable")
_, v6 := state.Counts()
assert.Equal(t, 1, v6, "DNAT-held reference survives unpaired DisableRouting")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "delete releases the DNAT reference")
}
// TestNftablesDNAT_DoubleDeleteNoUnderflow verifies that deleting the same rule
// twice does not underflow the refcount (the second delete is a no-op).
func TestNftablesDNAT_DoubleDeleteNoUnderflow(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9093))
require.NoError(t, err)
_, v6 := state.Counts()
assert.Equal(t, 1, v6)
require.NoError(t, m.DeleteDNATRule(r1), "first delete")
_, v6 = state.Counts()
assert.Equal(t, 0, v6)
require.NoError(t, m.DeleteDNATRule(r1), "second delete must be no-op")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "double delete must not underflow")
}

View File

@@ -105,8 +105,8 @@ func (m *Manager) createIPv6Components(tableName string, wgIface iFaceMapper, mt
return fmt.Errorf("create v6 router: %w", err)
}
// Share the per-family forwarding refcounter with the v4 router so a v4
// rule and a v6 rule against the same state machine cooperate cleanly.
// Share the same IP forwarding state with the v4 router, since
// EnableIPForwarding controls both v4 and v6 sysctls.
m.router6.ipFwdState = m.router.ipFwdState
m.aclManager6, err = newAclManager(workTable6, wgIface, chainNameRoutingFw)
@@ -530,12 +530,17 @@ func (m *Manager) SetLogLevel(log.Level) {
}
func (m *Manager) EnableRouting() error {
// v6 only when the overlay actually has v6.
return m.router.ipFwdState.RequestRouting(m.router6 != nil)
if err := m.router.ipFwdState.RequestForwarding(); err != nil {
return fmt.Errorf("enable IP forwarding: %w", err)
}
return nil
}
func (m *Manager) DisableRouting() error {
return m.router.ipFwdState.ReleaseRouting()
if err := m.router.ipFwdState.ReleaseForwarding(); err != nil {
return fmt.Errorf("disable IP forwarding: %w", err)
}
return nil
}
// Flush rule/chain/set operations from the buffer

View File

@@ -93,7 +93,7 @@ func newRouter(workTable *nftables.Table, wgIface iFaceMapper, mtu uint16) (*rou
rules: make(map[string]*nftables.Rule),
af: familyForAddr(workTable.Family == nftables.TableFamilyIPv4),
wgIface: wgIface,
ipFwdState: ipfwdstate.NewIPForwardingState(wgIface.Name()),
ipFwdState: ipfwdstate.NewIPForwardingState(),
mtu: mtu,
}
@@ -1553,6 +1553,10 @@ func (r *router) refreshRulesMap() error {
}
func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
if err := r.ipFwdState.RequestForwarding(); err != nil {
return nil, err
}
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey+dnatSuffix]; exists {
return rule, nil
@@ -1563,18 +1567,7 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
return nil, fmt.Errorf("convert protocol to number: %w", err)
}
// Request forwarding before queueing rules: addDnatRedirect/addDnatMasq
// buffer netlink messages on r.conn that the next caller's Flush would
// commit if we returned without flushing them ourselves.
v6 := r.af.tableFamily == nftables.TableFamilyIPv6
if err := r.ipFwdState.RequestForwarding(v6); err != nil {
return nil, fmt.Errorf("enable forwarding: %w", err)
}
if err := r.addDnatRedirect(rule, protoNum, ruleKey); err != nil {
if rerr := r.ipFwdState.ReleaseForwarding(v6); rerr != nil {
log.Warnf("rollback forwarding refcount: %v", rerr)
}
return nil, err
}
@@ -1586,11 +1579,6 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
// TODO: find chains with drop policies and add rules there
if err := r.conn.Flush(); err != nil {
if rerr := r.ipFwdState.ReleaseForwarding(v6); rerr != nil {
log.Warnf("rollback forwarding refcount: %v", rerr)
}
delete(r.rules, ruleKey+dnatSuffix)
delete(r.rules, ruleKey+snatSuffix)
return nil, fmt.Errorf("flush rules: %w", err)
}
@@ -1793,18 +1781,16 @@ func (r *router) addDnatMasq(rule firewall.ForwardRule, protoNum uint8, ruleKey
}
func (r *router) DeleteDNATRule(rule firewall.Rule) error {
if err := r.ipFwdState.ReleaseForwarding(); err != nil {
log.Errorf("%v", err)
}
ruleKey := rule.ID()
if err := r.refreshRulesMap(); err != nil {
return fmt.Errorf(refreshRulesMapError, err)
}
_, hadDNAT := r.rules[ruleKey+dnatSuffix]
_, hadSNAT := r.rules[ruleKey+snatSuffix]
if !hadDNAT && !hadSNAT {
return nil
}
var merr *multierror.Error
var needsFlush bool
@@ -1836,16 +1822,9 @@ func (r *router) DeleteDNATRule(rule firewall.Rule) error {
}
}
// Release the refcount only once the rules are gone from the kernel. On
// failure (including the refreshRulesMap error above) the rules and their
// map entries remain, keeping forwarding on until a retry removes them.
if merr == nil {
delete(r.rules, ruleKey+dnatSuffix)
delete(r.rules, ruleKey+snatSuffix)
if err := r.ipFwdState.ReleaseForwarding(r.af.tableFamily == nftables.TableFamilyIPv6); err != nil {
log.Errorf("%v", err)
}
}
return nberrors.FormatErrorOrNil(merr)

View File

@@ -844,10 +844,6 @@ func collectSysctls() string {
[]string{"net.ipv4.conf.all.src_valid_mark", "net.ipv4.conf.default.src_valid_mark"},
listInterfaceSysctls("ipv4", "src_valid_mark")...,
))
writeSysctlGroup(&builder, "accept_ra", append(
[]string{"net.ipv6.conf.all.accept_ra", "net.ipv6.conf.default.accept_ra"},
listInterfaceSysctls("ipv6", "accept_ra")...,
))
writeSysctlGroup(&builder, "conntrack", []string{
"net.netfilter.nf_conntrack_acct",
"net.netfilter.nf_conntrack_tcp_loose",

View File

@@ -267,38 +267,18 @@ func (s *systemConfigurator) getSystemDNSSettings() (SystemDNSSettings, error) {
return SystemDNSSettings{}, fmt.Errorf("sending the command: %w", err)
}
dnsSettings, serverAddresses, err := parseSystemDNSSettings(b)
if err != nil {
return dnsSettings, err
}
s.mu.Lock()
s.origNameservers = serverAddresses
s.mu.Unlock()
return dnsSettings, nil
}
// parseSystemDNSSettings parses the output of `scutil show State:/Network/Service/<id>/DNS`.
// Lines that don't match the expected "index : value" shape are skipped: hosts with unusual
// network services (e.g. orphaned hardware ports) can produce entries without a value.
func parseSystemDNSSettings(out []byte) (SystemDNSSettings, []netip.Addr, error) {
// port is not exposed by scutil, default to 53
dnsSettings := SystemDNSSettings{ServerPort: DefaultPort}
var dnsSettings SystemDNSSettings
var serverAddresses []netip.Addr
inSearchDomainsArray := false
inServerAddressesArray := false
scanner := bufio.NewScanner(bytes.NewReader(out))
scanner := bufio.NewScanner(bytes.NewReader(b))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
switch {
case strings.HasPrefix(line, "DomainName :"):
domainName := strings.TrimSpace(strings.TrimPrefix(line, "DomainName :"))
if domainName != "" {
dnsSettings.Domains = append(dnsSettings.Domains, domainName)
}
continue
domainName := strings.TrimSpace(strings.Split(line, ":")[1])
dnsSettings.Domains = append(dnsSettings.Domains, domainName)
case line == "SearchDomains : <array> {":
inSearchDomainsArray = true
continue
@@ -308,45 +288,36 @@ func parseSystemDNSSettings(out []byte) (SystemDNSSettings, []netip.Addr, error)
case line == "}":
inSearchDomainsArray = false
inServerAddressesArray = false
continue
}
if !inSearchDomainsArray && !inServerAddressesArray {
continue
}
parts := strings.SplitN(line, " : ", 2)
if len(parts) != 2 {
log.Debugf("skipping unexpected scutil DNS line %q", line)
continue
}
value := strings.TrimSpace(parts[1])
if value == "" {
continue
}
if inSearchDomainsArray {
dnsSettings.Domains = append(dnsSettings.Domains, value)
continue
}
ip, err := netip.ParseAddr(value)
if err != nil || ip.IsUnspecified() {
continue
}
ip = ip.Unmap()
serverAddresses = append(serverAddresses, ip)
// Prefer the first IPv4 server as ServerIP since our DNS listener is IPv4.
if !dnsSettings.ServerIP.IsValid() && ip.Is4() {
dnsSettings.ServerIP = ip
searchDomain := strings.Split(line, " : ")[1]
dnsSettings.Domains = append(dnsSettings.Domains, searchDomain)
} else if inServerAddressesArray {
address := strings.Split(line, " : ")[1]
if ip, err := netip.ParseAddr(address); err == nil && !ip.IsUnspecified() {
ip = ip.Unmap()
serverAddresses = append(serverAddresses, ip)
// Prefer the first IPv4 server as ServerIP since our DNS listener is IPv4.
if !dnsSettings.ServerIP.IsValid() && ip.Is4() {
dnsSettings.ServerIP = ip
}
}
}
}
if err := scanner.Err(); err != nil {
return dnsSettings, serverAddresses, err
return dnsSettings, err
}
return dnsSettings, serverAddresses, nil
// default to 53 port
dnsSettings.ServerPort = DefaultPort
s.mu.Lock()
s.origNameservers = serverAddresses
s.mu.Unlock()
return dnsSettings, nil
}
func (s *systemConfigurator) getOriginalNameservers() []netip.Addr {
@@ -464,15 +435,11 @@ func (s *systemConfigurator) getPrimaryService() (string, string, error) {
router := ""
for scanner.Scan() {
text := scanner.Text()
parts := strings.SplitN(text, ":", 2)
if len(parts) != 2 {
continue
}
if strings.Contains(text, "PrimaryService") {
primaryService = strings.TrimSpace(parts[1])
primaryService = strings.TrimSpace(strings.Split(text, ":")[1])
}
if strings.Contains(text, "Router") {
router = strings.TrimSpace(parts[1])
router = strings.TrimSpace(strings.Split(text, ":")[1])
}
}
if err := scanner.Err(); err != nil && err != io.EOF {

View File

@@ -328,120 +328,6 @@ func removeTestDNSKey(key string) error {
return err
}
func TestParseSystemDNSSettings(t *testing.T) {
tests := []struct {
name string
output string
expectedDomains []string
expectedServers []netip.Addr
expectedIP netip.Addr
}{
{
name: "well_formed",
output: `<dictionary> {
DomainName : example.com
SearchDomains : <array> {
0 : example.com
1 : corp.example.com
}
ServerAddresses : <array> {
0 : 192.168.1.1
1 : fd00::53
}
}
`,
expectedDomains: []string{"example.com", "example.com", "corp.example.com"},
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1"), netip.MustParseAddr("fd00::53")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
// entries without a value after the separator used to panic with
// "index out of range [1] with length 1"
name: "malformed_array_entries_skipped",
output: `<dictionary> {
SearchDomains : <array> {
0 :
(null)
1 : corp.example.com
}
ServerAddresses : <array> {
0 :
1 : 192.168.1.1
}
}
`,
expectedDomains: []string{"corp.example.com"},
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "domain_name_without_value_skipped",
output: `<dictionary> {
DomainName :
ServerAddresses : <array> {
0 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "ipv6_first_prefers_ipv4_server_ip",
output: `<dictionary> {
ServerAddresses : <array> {
0 : fd00::53
1 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("fd00::53"), netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "invalid_and_unspecified_addresses_skipped",
output: `<dictionary> {
ServerAddresses : <array> {
0 : (null)
1 : 0.0.0.0
2 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "v4_mapped_address_unmapped",
output: `<dictionary> {
ServerAddresses : <array> {
0 : ::ffff:192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "empty_output",
output: "",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
settings, servers, err := parseSystemDNSSettings([]byte(tc.output))
require.NoError(t, err, "parsing should not fail")
assert.Equal(t, tc.expectedDomains, settings.Domains, "domains should match")
assert.Equal(t, tc.expectedServers, servers, "server addresses should match")
assert.Equal(t, tc.expectedIP, settings.ServerIP, "server IP should match")
assert.Equal(t, DefaultPort, settings.ServerPort, "server port should default to 53")
})
}
}
func TestGetOriginalNameservers(t *testing.T) {
configurator := &systemConfigurator{
createdKeys: make(map[string]struct{}),

View File

@@ -2,183 +2,54 @@ package ipfwdstate
import (
"fmt"
"sync"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/routemanager/systemops"
)
// IPForwardingState tracks v4 and v6 IP-forwarding sysctl enables with
// independent refcounts so a v4-only routing setup doesn't flip v6 sysctls.
// IPForwardingState is a struct that keeps track of the IP forwarding state.
// todo: read initial state of the IP forwarding from the system and reset the state based on it.
// todo: separate v4/v6 forwarding state, since the sysctls are independent
// (net.ipv4.ip_forward vs net.ipv6.conf.all.forwarding). Currently the nftables
// manager shares one instance between both routers, which works only because
// EnableIPForwarding enables both sysctls in a single call.
type IPForwardingState struct {
mu sync.Mutex
v4Count int
v6Count int
// routingV4/routingV6 track whether the routing path currently holds a
// reference, so repeated EnableRouting calls (one per network-map update)
// hold at most one reference per family and an unpaired DisableRouting
// can't release references held by DNAT rules.
routingV4 bool
routingV6 bool
wgIfaceName string
v6Saved map[string]int
enabledCounter int
}
// NewIPForwardingState returns a state tracker for the IP-forwarding sysctls.
// wgIfaceName is excluded from the per-interface accept_ra handling.
func NewIPForwardingState(wgIfaceName string) *IPForwardingState {
return &IPForwardingState{wgIfaceName: wgIfaceName}
func NewIPForwardingState() *IPForwardingState {
return &IPForwardingState{}
}
// Counts returns the current v4 and v6 refcounts. Intended for diagnostics
// and tests.
func (f *IPForwardingState) Counts() (v4, v6 int) {
f.mu.Lock()
defer f.mu.Unlock()
return f.v4Count, f.v6Count
}
// RequestRouting takes the forwarding references for the routing path. It is
// idempotent: while routing already holds a reference, further calls don't
// increment the refcounts, and a v4-only request releases a previously held v6
// reference. A v6 sysctl failure is logged and not returned so it can't take
// down v4 routing (the sysctl may be unwritable, e.g. read-only /proc/sys or
// IPv6 disabled on the kernel command line); v6 is retried on the next call.
func (f *IPForwardingState) RequestRouting(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if !f.routingV4 {
if err := f.requestV4(); err != nil {
return err
}
f.routingV4 = true
}
if !v6 {
if !f.routingV6 {
return nil
}
f.routingV6 = false
return f.releaseV6()
}
if f.routingV6 {
return nil
}
if err := f.requestV6(); err != nil {
log.Warnf("enable IPv6 forwarding for routing: %v", err)
return nil
}
f.routingV6 = true
return nil
}
// ReleaseRouting releases the references RequestRouting holds. Calls without a
// held reference are no-ops.
func (f *IPForwardingState) ReleaseRouting() error {
f.mu.Lock()
defer f.mu.Unlock()
if f.routingV4 {
f.routingV4 = false
f.releaseV4()
}
if f.routingV6 {
f.routingV6 = false
return f.releaseV6()
}
return nil
}
// RequestForwarding enables the family's forwarding sysctl on first request.
func (f *IPForwardingState) RequestForwarding(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if v6 {
return f.requestV6()
}
return f.requestV4()
}
// ReleaseForwarding decrements the family counter. The last v6 release restores
// what enable captured. v4 stays on: net.ipv4.ip_forward is co-owned by other
// tooling (docker, k8s, libvirt).
func (f *IPForwardingState) ReleaseForwarding(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if v6 {
return f.releaseV6()
}
f.releaseV4()
return nil
}
func (f *IPForwardingState) requestV4() error {
if f.v4Count == 0 {
if err := systemops.EnableV4IPForwarding(); err != nil {
return fmt.Errorf("enable IPv4 forwarding: %w", err)
}
log.Info("IPv4 forwarding enabled")
}
f.v4Count++
return nil
}
func (f *IPForwardingState) releaseV4() {
if f.v4Count > 0 {
f.v4Count--
}
}
func (f *IPForwardingState) requestV6() error {
if f.v6Count == 0 {
saved, err := systemops.EnableV6IPForwarding(f.wgIfaceName)
if err != nil {
if rerr := systemops.DisableV6IPForwarding(saved); rerr != nil {
log.Warnf("rollback partial v6 sysctls: %v", rerr)
}
return fmt.Errorf("enable IPv6 forwarding: %w", err)
}
// A failed restore on a previous release keeps its saved values; those
// are the true originals, so keep them over what this enable captured.
if f.v6Saved == nil {
f.v6Saved = saved
} else {
for k, v := range saved {
if _, ok := f.v6Saved[k]; !ok {
f.v6Saved[k] = v
}
}
}
log.Info("IPv6 forwarding enabled")
}
f.v6Count++
return nil
}
func (f *IPForwardingState) releaseV6() error {
if f.v6Count == 0 {
return nil
}
f.v6Count--
if f.v6Count > 0 {
func (f *IPForwardingState) RequestForwarding() error {
if f.enabledCounter != 0 {
f.enabledCounter++
return nil
}
// Keep the saved values on failure so a later release or enable/release
// cycle can still restore them; re-restoring an already-restored key is a
// no-op since the sysctl already holds the desired value.
if err := systemops.DisableV6IPForwarding(f.v6Saved); err != nil {
return fmt.Errorf("disable IPv6 forwarding: %w", err)
if err := systemops.EnableIPForwarding(); err != nil {
return fmt.Errorf("failed to enable IP forwarding with sysctl: %w", err)
}
f.v6Saved = nil
log.Info("IPv6 forwarding disabled")
f.enabledCounter = 1
log.Info("IP forwarding enabled")
return nil
}
func (f *IPForwardingState) ReleaseForwarding() error {
if f.enabledCounter == 0 {
return nil
}
if f.enabledCounter > 1 {
f.enabledCounter--
return nil
}
// if failed to disable IP forwarding we anyway decrement the counter
f.enabledCounter = 0
// todo call systemops.DisableIPForwarding()
return nil
}

View File

@@ -1,39 +0,0 @@
//go:build privileged
package ipfwdstate
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestRequestRoutingV6ToV4Transition verifies that a v4-only routing request
// releases a previously held routing-owned v6 reference without touching
// references held by DNAT rules.
func TestRequestRoutingV6ToV4Transition(t *testing.T) {
f := NewIPForwardingState("wt-fwd-test")
require.NoError(t, f.RequestRouting(true), "request routing with v6")
v4, v6 := f.Counts()
assert.Equal(t, 1, v4, "v4 reference held")
assert.Equal(t, 1, v6, "v6 reference held")
require.NoError(t, f.RequestRouting(false), "request routing v4-only")
v4, v6 = f.Counts()
assert.Equal(t, 1, v4, "v4 reference kept")
assert.Equal(t, 0, v6, "routing-owned v6 reference released")
// A DNAT-held reference survives a v4-only routing request.
require.NoError(t, f.RequestForwarding(true), "dnat v6 reference")
require.NoError(t, f.RequestRouting(false), "repeat v4-only request")
_, v6 = f.Counts()
assert.Equal(t, 1, v6, "dnat-held v6 reference survives")
require.NoError(t, f.ReleaseForwarding(true), "release dnat v6 reference")
require.NoError(t, f.ReleaseRouting(), "release routing")
v4, v6 = f.Counts()
assert.Equal(t, 0, v4, "all v4 references released")
assert.Equal(t, 0, v6, "all v6 references released")
}

View File

@@ -58,7 +58,11 @@ func Setup(wgIface iface) (map[string]int, error) {
continue
}
i := fmt.Sprintf(rpFilterInterfacePath, EscapeInterfaceName(intf.Name))
// Escape '%' and '.' so they survive the dot-to-slash conversion in Set()
safeName := strings.ReplaceAll(intf.Name, "%", percentEscape)
safeName = strings.ReplaceAll(safeName, ".", dotEscape)
i := fmt.Sprintf(rpFilterInterfacePath, safeName)
oldVal, err := Set(i, 2, true)
if err != nil {
result = multierror.Append(result, err)
@@ -70,13 +74,6 @@ func Setup(wgIface iface) (map[string]int, error) {
return keys, nberrors.FormatErrorOrNil(result)
}
// EscapeInterfaceName escapes '%' and '.' in an interface name (e.g. VLANs
// like eth0.100) so the name survives the dot-to-slash conversion in Set.
func EscapeInterfaceName(name string) string {
safe := strings.ReplaceAll(name, "%", percentEscape)
return strings.ReplaceAll(safe, ".", dotEscape)
}
// Set sets a sysctl configuration, if onlyIfOne is true it will only set the new value if it's set to 1
func Set(key string, desiredValue int, onlyIfOne bool) (int, error) {
path := strings.ReplaceAll(key, ".", "/")

View File

@@ -32,17 +32,8 @@ func (r *SysOps) removeFromRouteTable(netip.Prefix, Nexthop) error {
return nil
}
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
return nil
}

View File

@@ -58,17 +58,8 @@ func (r *SysOps) removeFromRouteTable(netip.Prefix, Nexthop) error {
return nil
}
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
return nil
}

View File

@@ -763,10 +763,13 @@ func flushRoutes(tableID, family int) error {
return nberrors.FormatErrorOrNil(result)
}
func EnableV4IPForwarding() error {
func EnableIPForwarding() error {
if _, err := sysctl.Set(ipv4ForwardingPath, 1, false); err != nil {
return err
}
if _, err := sysctl.Set(ipv6ForwardingPath, 1, false); err != nil {
log.Warnf("failed to enable IPv6 forwarding: %v", err)
}
return nil
}

View File

@@ -43,17 +43,8 @@ func (r *SysOps) RemoveVPNRoute(prefix netip.Prefix, intf *net.Interface) error
return r.genericRemoveVPNRoute(prefix, intf)
}
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
return nil
}

View File

@@ -1,92 +0,0 @@
//go:build !android
package systemops
import (
"fmt"
"net"
"os"
"github.com/hashicorp/go-multierror"
log "github.com/sirupsen/logrus"
nberrors "github.com/netbirdio/netbird/client/errors"
"github.com/netbirdio/netbird/client/internal/routemanager/sysctl"
)
const (
// 1 (default) accepts RAs only while forwarding is off; 2 keeps RA
// acceptance on regardless, so RA-installed host defaults survive our
// v6 forwarding flip.
acceptRAInterfacePath = "net.ipv6.conf.%s.accept_ra"
acceptRADefaultPath = "net.ipv6.conf.default.accept_ra"
acceptRAProcPathFormat = "/proc/sys/net/ipv6/conf/%s/accept_ra"
)
// EnableV6IPForwarding bumps accept_ra=2 on host v6 interfaces before flipping
// forwarding=1, so RA-installed host defaults survive. Returns the prior values
// of sysctls we actually changed; entries already at the target are omitted.
func EnableV6IPForwarding(wgIfaceName string) (map[string]int, error) {
saved := map[string]int{}
bumpAcceptRA(saved, wgIfaceName)
oldVal, err := sysctl.Set(ipv6ForwardingPath, 1, false)
if err != nil {
return saved, err
}
if oldVal != 1 {
saved[ipv6ForwardingPath] = oldVal
}
return saved, nil
}
// DisableV6IPForwarding restores what EnableV6IPForwarding captured.
func DisableV6IPForwarding(saved map[string]int) error {
var result *multierror.Error
for key, value := range saved {
if _, err := sysctl.Set(key, value, false); err != nil {
result = multierror.Append(result, fmt.Errorf("restore %s: %w", key, err))
}
}
return nberrors.FormatErrorOrNil(result)
}
func bumpAcceptRA(saved map[string]int, wgIfaceName string) {
// Also bump conf.default so interfaces created while forwarding is on
// (hotplug, new Wi-Fi/dock) inherit accept_ra=2 and keep accepting RAs.
bumpAcceptRAKey(saved, acceptRADefaultPath)
interfaces, err := net.Interfaces()
if err != nil {
log.Warnf("list interfaces for accept_ra: %v", err)
return
}
for _, intf := range interfaces {
if intf.Name == "lo" || intf.Name == wgIfaceName {
continue
}
bumpAcceptRAForInterface(saved, intf.Name)
}
}
func bumpAcceptRAForInterface(saved map[string]int, name string) {
// Build procfs path from name, not the dotted key: VLAN names like eth0.100.
if _, err := os.Stat(fmt.Sprintf(acceptRAProcPathFormat, name)); err != nil {
return
}
bumpAcceptRAKey(saved, fmt.Sprintf(acceptRAInterfacePath, sysctl.EscapeInterfaceName(name)))
}
func bumpAcceptRAKey(saved map[string]int, key string) {
// onlyIfOne=true: leave admin overrides (0, 2) alone.
oldVal, err := sysctl.Set(key, 2, true)
if err != nil {
log.Warnf("bump %s: %v", key, err)
return
}
// With onlyIfOne, a write only happened when the old value was 1; values
// left untouched (0, 2) must not be recorded for restore.
if oldVal == 1 {
saved[key] = oldVal
}
}

View File

@@ -22,6 +22,7 @@ import (
"github.com/netbirdio/netbird/client/internal/listener"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/ssh/jwtcache"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/formatter"
"github.com/netbirdio/netbird/route"
@@ -92,7 +93,14 @@ type Client struct {
stateMu sync.RWMutex
connectClient *internal.ConnectClient
config *profilemanager.Config
// config holds the active configuration once Run has loaded it. Consumed by
// the in-app SSH client for the NetBird SSH key and the OAuth flow.
config *profilemanager.Config
// sshJWTCache keeps the SSH JWT token between reconnects so the user is not
// forced through the browser OAuth flow on every session. Lives on Client
// (not SSHClient) because the app creates a new SSHClient per session.
sshJWTCache *jwtcache.Cache
}
// NewClient instantiate a new Client
@@ -109,6 +117,7 @@ func NewClient(cfgFile, stateFile, cacheDir, logFilePath, deviceName string, osV
ctxCancelLock: &sync.Mutex{},
networkChangeListener: networkChangeListener,
dnsManager: dnsManager,
sshJWTCache: jwtcache.New(),
}
}
@@ -183,6 +192,7 @@ func (c *Client) Run(fd int32, interfaceName string, envList *EnvList) error {
ctx = internal.CtxInitState(ctx)
c.onHostDnsFn = func([]string) {}
cfg.WgIface = interfaceName
c.config = cfg
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
c.setState(cfg, connectClient)
@@ -708,6 +718,13 @@ func (c *Client) stateSnapshot() (*profilemanager.Config, *internal.ConnectClien
return c.config, c.connectClient
}
// sshState returns the active config and the running connect client for the
// in-app SSH client. Both are nil until Run has loaded the config and started
// the tunnel.
func (c *Client) sshState() (*profilemanager.Config, *internal.ConnectClient) {
return c.stateSnapshot()
}
func formatDuration(d time.Duration) string {
ds := d.String()
dotIndex := strings.Index(ds, ".")

View File

@@ -0,0 +1,512 @@
//go:build ios
package NetBirdSDK
import (
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"strings"
"sync"
"syscall"
"time"
log "github.com/sirupsen/logrus"
gossh "golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/profilemanager"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/ssh/detection"
"github.com/netbirdio/netbird/client/ssh/jwtcache"
)
const (
sshDialTimeout = 30 * time.Second
sshDetectionTimeout = 5 * time.Second
)
// SSHTerminalListener receives SSH session events. It is implemented in Swift.
//
// All callbacks are invoked from goroutines and may run concurrently with each
// other; the implementation must be safe to call from any thread.
type SSHTerminalListener interface {
OnConnected()
OnData(data []byte)
OnClose(reason string)
OnError(message string)
}
// SSHClient is a NetBird-aware SSH client exposed to Swift via gomobile.
//
// It dials through the running NetBird tunnel and runs a standard SSH session
// on top with PTY enabled. Host-key verification uses the NetBird-provided
// peer SSH host keys, identical to the desktop client.
type SSHClient struct {
nb *Client
mu sync.Mutex
listener SSHTerminalListener
urlOpener URLOpener
sshClient *gossh.Client
session *gossh.Session
stdin io.WriteCloser
closed bool
}
// NewSSHClient creates a new SSH client bound to the running NetBird Client.
func NewSSHClient(c *Client) *SSHClient {
return &SSHClient{nb: c}
}
// SetListener registers the Swift listener. Must be called before Connect to
// receive any events.
func (s *SSHClient) SetListener(l SSHTerminalListener) {
s.mu.Lock()
s.listener = l
s.mu.Unlock()
}
// SetURLOpener registers the Swift URL opener used to display the device-code
// authorization page in an in-app browser when the target peer requires JWT
// authentication. Must be set before Connect to be effective.
func (s *SSHClient) SetURLOpener(opener URLOpener) {
s.mu.Lock()
s.urlOpener = opener
s.mu.Unlock()
}
// Connect dials the SSH server through the NetBird tunnel and performs the
// SSH handshake. It auto-detects the server type via SSH banner inspection
// and selects the appropriate authentication path:
//
// - NetBird-SSH server requiring JWT: launches the OAuth 2.0 device-code
// flow, opens the verification URL through the registered URLOpener, and
// uses the resulting token as the SSH password. Host-key verification
// uses the NetBird peer registry.
// - NetBird-SSH server without JWT: authenticates with the NetBird SSH
// private key. Host-key verification uses the NetBird peer registry.
// - Regular SSH server (e.g. OpenSSH): authenticates with the NetBird key
// first (so a user-installed NetBird public key works), then falls back
// to the supplied password if non-empty. Host-key verification is
// disabled (TOFU pending).
//
// The password parameter is only consulted for regular SSH servers.
//
// This is the only way to open a session, deliberately so: the JWT is a bearer
// token, and detection is what proves the listener is a NetBird SSH service
// before the token is offered to it. A peer whose NetBird SSH is disabled is
// served by plain sshd and authenticates by key or password like any other
// host, so it stays reachable without an OAuth round trip.
func (s *SSHClient) Connect(host string, port int, user, password string) error {
cfg, cc := s.nb.sshState()
if cc == nil {
return errors.New("netbird client not running")
}
if cfg == nil {
return errors.New("netbird config not loaded")
}
engine := cc.Engine()
if engine == nil {
return errors.New("netbird engine not available")
}
wgDialer := makeWGDialer(cfg.WgIface, sshDialTimeout)
serverType := detectServerType(host, port, wgDialer)
log.Infof("SSH server type for %s:%d: %s", host, port, serverType)
authMethods, hostKeyCallback, err := s.buildAuth(cfg, engine, serverType, password)
if err != nil {
return err
}
clientConfig := &gossh.ClientConfig{
User: user,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: sshDialTimeout,
}
if err := s.dialAndHandshake(host, port, clientConfig, wgDialer); err != nil {
return annotateAuthError(err, serverType, user)
}
return nil
}
// annotateAuthError adds NetBird-specific guidance to an authentication
// failure, but only for a server that detection identified as NetBird-SSH.
// There a rejection nearly always means the peer's dashboard SSH access is not
// configured for this account, which the raw gossh error does not convey.
func annotateAuthError(err error, serverType detection.ServerType, user string) error {
if !serverType.RequiresJWT() {
return err
}
msg := err.Error()
if !strings.Contains(msg, "no supported methods remain") &&
!strings.Contains(msg, "unable to authenticate") {
return err
}
return fmt.Errorf("NetBird SSH authentication rejected.\n\n"+
"Checklist:\n"+
" 1. SSH is enabled for this peer in the NetBird dashboard\n"+
" 2. Your account is listed under SSH access for this peer\n"+
" 3. The OS username (%q) is mapped to your account\n\n"+
"If SSH access is not configured, connect with a password instead.\n\n"+
"Original: %w", user, err)
}
// StartSession requests a PTY and starts an interactive shell. Output from
// the session is forwarded to the listener via OnData.
func (s *SSHClient) StartSession(cols, rows int) error {
log.Debugf("SSH: starting session %dx%d", cols, rows)
s.mu.Lock()
sshClient := s.sshClient
s.mu.Unlock()
if sshClient == nil {
return errors.New("ssh client not connected")
}
session, err := sshClient.NewSession()
if err != nil {
return fmt.Errorf("new session: %w", err)
}
modes := gossh.TerminalModes{
gossh.ECHO: 1,
gossh.TTY_OP_ISPEED: 14400,
gossh.TTY_OP_OSPEED: 14400,
gossh.VINTR: 3,
gossh.VQUIT: 28,
gossh.VERASE: 127,
}
if err := session.RequestPty("xterm-256color", rows, cols, modes); err != nil {
closeQuiet(session, "session after pty error")
return fmt.Errorf("request pty: %w", err)
}
stdin, err := session.StdinPipe()
if err != nil {
closeQuiet(session, "session after stdin error")
return fmt.Errorf("stdin pipe: %w", err)
}
stdout, err := session.StdoutPipe()
if err != nil {
closeQuiet(session, "session after stdout error")
return fmt.Errorf("stdout pipe: %w", err)
}
stderr, err := session.StderrPipe()
if err != nil {
closeQuiet(session, "session after stderr error")
return fmt.Errorf("stderr pipe: %w", err)
}
if err := session.Shell(); err != nil {
closeQuiet(session, "session after shell error")
return fmt.Errorf("start shell: %w", err)
}
s.mu.Lock()
s.session = session
s.stdin = stdin
s.mu.Unlock()
go s.readLoop(stdout, "stdout")
go s.readLoop(stderr, "stderr")
log.Debug("SSH: session started, shell running")
return nil
}
// Write sends data to the SSH session stdin.
func (s *SSHClient) Write(data []byte) error {
s.mu.Lock()
stdin := s.stdin
s.mu.Unlock()
if stdin == nil {
return errors.New("ssh session not started")
}
if _, err := stdin.Write(data); err != nil {
return fmt.Errorf("write stdin: %w", err)
}
return nil
}
// Resize updates the PTY window size.
func (s *SSHClient) Resize(cols, rows int) error {
s.mu.Lock()
session := s.session
s.mu.Unlock()
if session == nil {
return errors.New("ssh session not started")
}
return session.WindowChange(rows, cols)
}
// Close terminates the SSH session and underlying connection. Safe to call
// multiple times.
func (s *SSHClient) Close() error {
s.mu.Lock()
sshClient := s.sshClient
session := s.session
stdin := s.stdin
s.sshClient = nil
s.session = nil
s.stdin = nil
s.mu.Unlock()
if stdin != nil {
if err := stdin.Close(); err != nil {
log.Debugf("ssh: stdin close: %v", err)
}
}
if session != nil {
if err := session.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: session close: %v", err)
}
}
var firstErr error
if sshClient != nil {
if err := sshClient.Close(); err != nil {
firstErr = err
}
}
s.notifyClose("closed by client")
return firstErr
}
func (s *SSHClient) buildAuth(cfg *profilemanager.Config, engine *internal.Engine,
serverType detection.ServerType, password string) ([]gossh.AuthMethod, gossh.HostKeyCallback, error) {
switch serverType {
case detection.ServerTypeNetBirdJWT:
token, err := s.requestJWTToken(cfg)
if err != nil {
return nil, nil, fmt.Errorf("jwt: %w", err)
}
auths := []gossh.AuthMethod{gossh.Password(token)}
return auths, nbssh.CreateHostKeyCallback(&engineHostKeyVerifier{engine: engine}), nil
case detection.ServerTypeNetBirdNoJWT:
if cfg.SSHKey == "" {
return nil, nil, errors.New("no NetBird SSH key available")
}
signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey))
if err != nil {
return nil, nil, fmt.Errorf("parse netbird ssh key: %w", err)
}
auths := []gossh.AuthMethod{gossh.PublicKeys(signer)}
return auths, nbssh.CreateHostKeyCallback(&engineHostKeyVerifier{engine: engine}), nil
default: // regular SSH
var auths []gossh.AuthMethod
if cfg.SSHKey != "" {
if signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey)); err == nil {
auths = append(auths, gossh.PublicKeys(signer))
} else {
log.Debugf("ssh: parse netbird key for regular auth: %v", err)
}
}
if password != "" {
pw := password
auths = append(auths, gossh.Password(pw))
auths = append(auths, gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range questions {
answers[i] = pw
}
return answers, nil
}))
}
if len(auths) == 0 {
return nil, nil, errors.New("no auth method available: provide a password or configure NetBird SSH key")
}
return auths, gossh.InsecureIgnoreHostKey(), nil // nolint:gosec // TOFU not yet implemented
}
}
func (s *SSHClient) requestJWTToken(cfg *profilemanager.Config) (string, error) {
// Reuse a cached token so the user is not forced through the browser OAuth
// flow on every reconnect. TTL comes from cfg.SSHJWTCacheTTL, same as the
// daemon's cache; unset/0 disables caching.
if token, ok := s.nb.sshJWTCache.Get(); ok {
log.Debug("SSH: reusing cached JWT token")
return token, nil
}
s.mu.Lock()
urlOpener := s.urlOpener
s.mu.Unlock()
if urlOpener == nil {
return "", errors.New("URL opener not configured for JWT auth")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
flow, err := auth.NewOAuthFlow(ctx, cfg, false, true, profilemanager.GetLoginHint())
if err != nil {
return "", fmt.Errorf("create oauth flow: %w", err)
}
flowInfo, err := flow.RequestAuthInfo(ctx)
if err != nil {
return "", fmt.Errorf("request auth info: %w", err)
}
go urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
tokenInfo, err := flow.WaitToken(ctx, flowInfo)
if err != nil {
return "", fmt.Errorf("wait for token: %w", err)
}
token := tokenInfo.GetTokenToUse()
if token == "" {
return "", errors.New("empty token returned by IdP")
}
if ttl := jwtcache.ResolveTTL(cfg.SSHJWTCacheTTL); ttl > 0 {
s.nb.sshJWTCache.Store(token, ttl)
}
return token, nil
}
func (s *SSHClient) dialAndHandshake(host string, port int, clientConfig *gossh.ClientConfig, dialer *net.Dialer) error {
addr := net.JoinHostPort(host, strconv.Itoa(port))
log.Infof("SSH: connecting to %s as %s", addr, clientConfig.User)
ctx, cancel := context.WithTimeout(context.Background(), sshDialTimeout)
defer cancel()
conn, err := dialer.DialContext(ctx, "tcp", addr)
if err != nil {
return fmt.Errorf("dial %s: %w", addr, err)
}
sshConn, chans, reqs, err := gossh.NewClientConn(conn, addr, clientConfig)
if err != nil {
if cerr := conn.Close(); cerr != nil {
log.Debugf("ssh: close after handshake error: %v", cerr)
}
return fmt.Errorf("ssh handshake: %w", err)
}
s.mu.Lock()
s.sshClient = gossh.NewClient(sshConn, chans, reqs)
listener := s.listener
s.mu.Unlock()
log.Infof("SSH: connected to %s", addr)
if listener != nil {
listener.OnConnected()
}
return nil
}
func (s *SSHClient) readLoop(r io.Reader, name string) {
buf := make([]byte, 4096)
for {
n, err := r.Read(buf)
if n > 0 {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
chunk := make([]byte, n)
copy(chunk, buf[:n])
listener.OnData(chunk)
}
}
if err != nil {
if !errors.Is(err, io.EOF) {
log.Debugf("ssh %s read: %v", name, err)
}
s.notifyClose(err.Error())
return
}
}
}
func (s *SSHClient) notifyClose(reason string) {
s.mu.Lock()
if s.closed {
s.mu.Unlock()
return
}
s.closed = true
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnClose(reason)
}
}
// engineHostKeyVerifier adapts *internal.Engine to nbssh.HostKeyVerifier.
type engineHostKeyVerifier struct {
engine *internal.Engine
}
func (v *engineHostKeyVerifier) VerifySSHHostKey(peerAddress string, presented []byte) error {
storedKey, found := v.engine.GetPeerSSHKey(peerAddress)
if !found {
return nbssh.ErrPeerNotFound
}
return nbssh.VerifyHostKey(storedKey, presented, peerAddress)
}
func detectServerType(host string, port int, dialer *net.Dialer) detection.ServerType {
ctx, cancel := context.WithTimeout(context.Background(), sshDetectionTimeout)
defer cancel()
serverType, err := detection.DetectSSHServerType(ctx, dialer, host, port)
if err != nil {
log.Debugf("ssh: server detection for %s:%d failed: %v (assuming regular SSH)", host, port, err)
return detection.ServerTypeRegular
}
return serverType
}
// makeWGDialer returns a net.Dialer whose sockets are bound to the WireGuard
// interface (wgIface, e.g. "utun100"). This is required in the iOS Network
// Extension process, where the OS deliberately excludes the provider's own
// traffic from the VPN tunnel to prevent routing loops. Without binding to
// the WireGuard interface, TCP connections to NetBird peer IPs (100.x.x.x
// CGNAT space) would be sent over the physical network and fail with
// "network is unreachable". Falls back to an unbound dialer if the interface
// cannot be found (e.g. tunnel not yet up).
func makeWGDialer(wgIface string, timeout time.Duration) *net.Dialer {
return &net.Dialer{
Timeout: timeout,
Control: func(network, address string, c syscall.RawConn) error {
iface, err := net.InterfaceByName(wgIface)
if err != nil {
log.Debugf("ssh: WG interface %q not found, dialing without bind: %v", wgIface, err)
return nil
}
var innerErr error
if ctrlErr := c.Control(func(fd uintptr) {
// IP_BOUND_IF (Darwin) = 25: binds the socket to a specific interface index.
innerErr = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IP, 25, iface.Index)
}); ctrlErr != nil {
return ctrlErr
}
if innerErr != nil {
log.Debugf("ssh: IP_BOUND_IF bind to %q failed: %v", wgIface, innerErr)
}
return innerErr
},
}
}
func closeQuiet(c io.Closer, label string) {
if c == nil {
return
}
if err := c.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: close %s: %v", label, err)
}
}

View File

@@ -1,79 +0,0 @@
package server
import (
"sync"
"time"
"github.com/awnumar/memguard"
log "github.com/sirupsen/logrus"
)
type jwtCache struct {
mu sync.RWMutex
enclave *memguard.Enclave
expiresAt time.Time
timer *time.Timer
maxTokenSize int
}
func newJWTCache() *jwtCache {
return &jwtCache{
maxTokenSize: 8192,
}
}
func (c *jwtCache) store(token string, maxAge time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
c.cleanup()
if c.timer != nil {
c.timer.Stop()
}
tokenBytes := []byte(token)
c.enclave = memguard.NewEnclave(tokenBytes)
c.expiresAt = time.Now().Add(maxAge)
var timer *time.Timer
timer = time.AfterFunc(maxAge, func() {
c.mu.Lock()
defer c.mu.Unlock()
if c.timer != timer {
return
}
c.cleanup()
c.timer = nil
log.Debugf("JWT token cache expired after %v, securely wiped from memory", maxAge)
})
c.timer = timer
}
func (c *jwtCache) get() (string, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
if c.enclave == nil || time.Now().After(c.expiresAt) {
return "", false
}
buffer, err := c.enclave.Open()
if err != nil {
log.Debugf("Failed to open JWT token enclave: %v", err)
return "", false
}
defer buffer.Destroy()
token := string(buffer.Bytes())
return token, true
}
// cleanup destroys the secure enclave, must be called with lock held
func (c *jwtCache) cleanup() {
if c.enclave != nil {
c.enclave = nil
}
c.expiresAt = time.Time{}
}

View File

@@ -26,6 +26,7 @@ import (
"github.com/netbirdio/netbird/client/internal/profilemanager"
sleephandler "github.com/netbirdio/netbird/client/internal/sleep/handler"
"github.com/netbirdio/netbird/client/mdm"
"github.com/netbirdio/netbird/client/ssh/jwtcache"
"github.com/netbirdio/netbird/client/system"
mgm "github.com/netbirdio/netbird/shared/management/client"
"github.com/netbirdio/netbird/shared/management/domain"
@@ -50,9 +51,6 @@ const (
defaultMaxRetryTime = 14 * 24 * time.Hour
defaultRetryMultiplier = 1.7
// JWT token cache TTL for the client daemon (disabled by default)
defaultJWTCacheTTL = 0
errRestoreResidualState = "failed to restore residual state: %v"
errProfilesDisabled = "profiles are disabled, you cannot use this feature without profiles enabled"
errUpdateSettingsDisabled = "update settings are disabled, you cannot use this feature without update settings enabled"
@@ -134,7 +132,7 @@ type Server struct {
updateManager *updater.Manager
jwtCache *jwtCache
jwtCache *jwtcache.Cache
// loginAttemptFn stands in for the Management login round trip. Tests set
// it to drive the login outcomes that need a server on the other end;
@@ -163,7 +161,7 @@ func New(ctx context.Context, logFile string, configFile string, profilesDisable
updateSettingsDisabled: updateSettingsDisabled,
captureEnabled: captureEnabled,
networksDisabled: networksDisabled,
jwtCache: newJWTCache(),
jwtCache: jwtcache.New(),
extendAuthSessionFlow: auth.NewPendingFlow(),
probeThrottle: newProbeThrottle(probeThreshold),
}
@@ -1670,19 +1668,11 @@ func (s *Server) getJWTCacheTTL() time.Duration {
config := s.config
s.mutex.Unlock()
if config == nil || config.SSHJWTCacheTTL == nil {
return defaultJWTCacheTTL
if config == nil {
return jwtcache.DefaultTTL
}
seconds := *config.SSHJWTCacheTTL
if seconds == 0 {
log.Debug("SSH JWT cache disabled (configured to 0)")
return 0
}
ttl := time.Duration(seconds) * time.Second
log.Debugf("SSH JWT cache TTL set to %v from config", ttl)
return ttl
return jwtcache.ResolveTTL(config.SSHJWTCacheTTL)
}
// RequestJWTAuth initiates JWT authentication flow for SSH
@@ -1704,7 +1694,7 @@ func (s *Server) RequestJWTAuth(
jwtCacheTTL := s.getJWTCacheTTL()
if jwtCacheTTL > 0 {
if cachedToken, found := s.jwtCache.get(); found {
if cachedToken, found := s.jwtCache.Get(); found {
log.Debugf("JWT token found in cache, returning cached token for SSH authentication")
return &proto.RequestJWTAuthResponse{
@@ -1777,7 +1767,7 @@ func (s *Server) WaitJWTToken(
jwtCacheTTL := s.getJWTCacheTTL()
if jwtCacheTTL > 0 {
s.jwtCache.store(token, jwtCacheTTL)
s.jwtCache.Store(token, jwtCacheTTL)
log.Debugf("JWT token cached for SSH authentication, TTL: %v", jwtCacheTTL)
} else {
log.Debug("JWT caching disabled, not storing token")

View File

@@ -0,0 +1,109 @@
// Package jwtcache provides an in-memory, TTL-bound cache for SSH JWT tokens.
// The token is kept in a secure memguard enclave and wiped from memory when it
// expires. It is shared by the daemon gRPC server and the mobile SDKs, which
// have no daemon process to delegate caching to.
package jwtcache
import (
"sync"
"time"
"github.com/awnumar/memguard"
log "github.com/sirupsen/logrus"
)
// DefaultTTL is used when no TTL is configured: caching disabled.
const DefaultTTL = 0
// Cache stores a single JWT token in a secure enclave until it expires.
type Cache struct {
mu sync.RWMutex
enclave *memguard.Enclave
expiresAt time.Time
timer *time.Timer
maxTokenSize int
}
// New creates an empty Cache.
func New() *Cache {
return &Cache{
maxTokenSize: 8192,
}
}
// Store caches the token for maxAge. A previously stored token is wiped.
func (c *Cache) Store(token string, maxAge time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
c.cleanup()
if c.timer != nil {
c.timer.Stop()
}
tokenBytes := []byte(token)
c.enclave = memguard.NewEnclave(tokenBytes)
c.expiresAt = time.Now().Add(maxAge)
var timer *time.Timer
timer = time.AfterFunc(maxAge, func() {
c.mu.Lock()
defer c.mu.Unlock()
if c.timer != timer {
return
}
c.cleanup()
c.timer = nil
log.Debugf("JWT token cache expired after %v, securely wiped from memory", maxAge)
})
c.timer = timer
}
// Get returns the cached token, or false if none is stored or it has expired.
func (c *Cache) Get() (string, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
if c.enclave == nil || time.Now().After(c.expiresAt) {
return "", false
}
buffer, err := c.enclave.Open()
if err != nil {
log.Debugf("Failed to open JWT token enclave: %v", err)
return "", false
}
defer buffer.Destroy()
token := string(buffer.Bytes())
return token, true
}
// cleanup destroys the secure enclave, must be called with lock held
func (c *Cache) cleanup() {
if c.enclave != nil {
c.enclave = nil
}
c.expiresAt = time.Time{}
}
// ResolveTTL converts the configured TTL (seconds, from
// profilemanager.Config.SSHJWTCacheTTL) into a duration. Returns DefaultTTL
// when unset; 0 means caching is disabled.
func ResolveTTL(configuredSeconds *int) time.Duration {
if configuredSeconds == nil {
return DefaultTTL
}
seconds := *configuredSeconds
if seconds == 0 {
log.Debug("SSH JWT cache disabled (configured to 0)")
return 0
}
ttl := time.Duration(seconds) * time.Second
log.Debugf("SSH JWT cache TTL set to %v from config", ttl)
return ttl
}