Compare commits

..

12 Commits

Author SHA1 Message Date
Misha Bragin
93e97f4bf1 [doc] Agent network docs update (#7020)
* [docs] Update agent-network docs for management-owned pricing

  The docs still described the retired proxy-side pricing: pricing.Loader,
  pricing_path, MiddlewareDataDir, embedded defaults_pricing.yaml, and the
  symlink-safe Unix loader. Rewrite them for the current design — management
  synthesizes the whole table and ships it in cost_meter's ConfigJSON, so the
  proxy carries no price list and has nothing to reload.
2026-08-15 19:31:49 +02:00
Zoltan Papp
16544dbc58 [client] Pass stored email as login hint from UI and keep it on logout (#7199)
* [client] Pass stored email as login hint from UI and keep it on logout

Follow the CLI pattern: the Wails UI now reads the account email from the
user-owned profile state file and passes it as the OIDC login_hint on login
and session extend, since the daemon-side fallback runs as root and cannot
see the user's state file. Logout no longer deletes the stored email, so a
later login preselects the account at the IdP; profile removal remains the
operation that deletes it.

* [client] Log ignored profile lookup errors in extend-session hint fallback
2026-08-15 11:21:57 +02:00
Zoltan Papp
f458c1f265 [client] Skip IPv6 route tests when the default nexthop is unusable (#7212)
* [client] Skip IPv6 route tests when the default nexthop is unusable

ensureIPv6DefaultRoute treated a successful netlink RouteAdd as proof that
a usable IPv6 nexthop exists. Installing ::/0 via loopback can succeed while
the kernel still rejects that nexthop for a concrete prefix, which surfaced
on ubuntu22/20260810.260 runners as:

    add route to table: netlink add route: invalid argument

Probe the resolved nexthop by installing and removing a discard-prefix route
through the same code path the tests use, and skip when it fails. EEXIST
means the nexthop already carries a route, so it counts as usable.

* [client] Probe the IPv6 nexthop through raw netlink

addRoute swallows EAFNOSUPPORT and EOPNOTSUPP via isOpErr, so a nil return
did not prove the probe route was installed. Call netlink directly so an
unsupported operation skips the test instead of passing as usable.
2026-08-15 10:13:06 +02:00
Viktor Liu
ec6f1b8c27 [client] Rank Windows route candidates by combined route and interface metric (#7210) 2026-08-15 09:06:22 +02:00
Zoltan Papp
2cfe14d7ec [client] Keep account email on Android logout, drop it on profile removal (#7200)
Align Android logout semantics with the desktop UI and CLI: logging out no
longer deletes the stored account email, so the next login passes it as the
OIDC login_hint and the IdP preselects the account. Removing the profile is
now the operation that deletes the email; previously RemoveProfile left the
account file behind, which the fixed-name default profile would have
inherited on recreation.
2026-08-14 18:13:52 +02:00
Eduard Gert
85dd335836 [client] Add CI check for translation key parity (#6852)
English (en) is the source of truth for UI translation keys; the other
nine locales rely on runtime English fallback for any missing key, so a
gap never surfaces in CI. Add a dependency-free Node check that fails
when any locale declared in _index.json does not carry the exact same
key set as en (missing or orphaned keys), wired into a dedicated
UI Translations workflow that runs on locale changes.

Also close the one existing gap the check found: ja was missing
daemon.outdated.download ("Download Latest").

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-14 10:57:11 +02:00
Viktor Liu
5544761b47 [client] Add Windows DNS configuration to the debug bundle (#7196) 2026-08-13 20:07:37 +02:00
Kim Harre
1d372bb634 [infrastructure] Support non-interactive installation in getting-started.sh (#7168) 2026-08-13 18:58:03 +03:00
Viktor Liu
e290769df1 [client] Take the graphical session answer from the caller instead of the daemon environment (#7187) 2026-08-13 10:28:34 +02:00
Jack Carter
58c09ead21 [management] Document mutual exclusivity of policy rule ports and port_ranges (#7158) 2026-08-12 20:39:06 +02:00
Brad Ison
c5503fdc7f [misc] Build release branches, and don't mark releases latest before signing (#7171)
Prepares the repository for the release-branch process agreed internally:
one long-lived release-0.N branch per minor, with fixes backported by
cherry-pick and patch releases tagged from the branch.

Pushes to release-* branches now run the Release workflow and publish
immutable sha-* container images, the way pushes to main already do, so
a release branch can be tested before it is tagged. Release branches
never publish the floating "main" image tag. The push-to-main CI
workflows (Go tests on all platforms, frontend UI, install script,
mobile/wasm validation, infrastructure files, license check) also run
on release-* pushes; pull request triggers were already unfiltered, so
backport PRs were covered — this closes the post-merge gap.

Releases are no longer marked latest before signing: make_latest is
now false in all four goreleaser configs, so a release stays published
but not latest until the signing pipeline uploads the signed Windows
and macOS artifacts and marks it latest itself. Previously the release
became GitHub's "Latest release" at publish time, and the download
endpoints that resolve through the latest-release API could serve a
release whose signed installers did not exist yet. prerelease: auto
additionally labels rc tags as prereleases, so a release candidate can
never take the latest slot.

The trigger_sync_tag job is removed: it dispatched a downstream
image build on every v* tag (release candidates included), which would
race the deliberate release-branch build on every release. The android
and ios submodule bumps are unchanged.

Also sets perennial-regex = "^release-" so git-town never syncs or
ships a release branch into main.
2026-08-12 18:04:36 +02:00
Zoltan Papp
6b69f5c05d [client] Remove installer registry handlers for autostart Run keys (#7183)
The NSIS installer deleted HKLM/HKCU CurrentVersion\Run values it never
writes, which matches common AV heuristics for unwanted Run-key
manipulation and is suspected to contribute to Windows Defender and
third-party antivirus false positives on the installer.

Drop all autostart registry deletions from both the install and
uninstall sections so the installer only touches keys it creates
itself. Cleanup of the legacy machine-wide entry written by old
installers is left to documentation.

Extends the approach of the closed PR #6735, which only removed the
per-user deletion on uninstall.
2026-08-12 17:35:01 +02:00
71 changed files with 2420 additions and 890 deletions

View File

@@ -3,7 +3,7 @@
[branches]
main = "main"
perennials = []
perennial-regex = ""
perennial-regex = "^release-"
[create]
new-branch-type = "feature"

View File

@@ -2,7 +2,7 @@ name: Check License Dependencies
on:
push:
branches: [main]
branches: [main, "release-*"]
paths:
- "go.mod"
- "go.sum"

View File

@@ -10,6 +10,7 @@ on:
push:
branches:
- main
- "release-*"
paths:
- "client/ui/frontend/**"
- "client/ui/i18n/**"

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
env:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
paths:
- "release_files/install.sh"

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -6,6 +6,7 @@ on:
- "v*"
branches:
- main
- "release-*"
pull_request:
env:
@@ -254,15 +255,23 @@ jobs:
id: tag_and_push_images
if: |
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) ||
(github.event_name == 'push' && github.ref == 'refs/heads/main')
(github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/release-')))
run: |
set -euo pipefail
# $GITHUB_REF / $GITHUB_EVENT_NAME are read from the runner
# environment rather than substituted into this script with the
# workflow expression syntax: branch names may legally contain
# $(…), and interpolating github.ref would execute it.
resolve_tags() {
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
echo "pr-${{ github.event.pull_request.number }}"
else
elif [[ "$GITHUB_REF" == "refs/heads/main" ]]; then
echo "main sha-$(git rev-parse --short HEAD)"
else
# Release branches get an immutable sha-* tag only — the floating
# "main" tag must never move from a release branch.
echo "sha-$(git rev-parse --short HEAD)"
fi
}

View File

@@ -9,21 +9,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
# Receiving workflows (cloud sync-tag, mobile bump-netbird) expect the short
# tag form (e.g. v0.30.0), not refs/tags/v0.30.0 — github.ref_name, not github.ref.
# The receiving bump-netbird workflows expect the short tag form
# (e.g. v0.30.0), not refs/tags/v0.30.0 — github.ref_name, not github.ref.
jobs:
trigger_sync_tag:
runs-on: ubuntu-latest
steps:
- name: Trigger release tag sync
uses: benc-uk/workflow-dispatch@31e2b3319479a63f0ab15bf800eff9e913504e26 # v1.3.2
with:
workflow: sync-tag.yml
ref: main
repo: ${{ secrets.UPSTREAM_REPO }}
token: ${{ secrets.NC_GITHUB_TOKEN }}
inputs: '{ "tag": "${{ github.ref_name }}" }'
trigger_android_bump:
runs-on: ubuntu-latest
if: github.event.created && !github.event.deleted && startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, '-')

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
paths:
- "infrastructure_files/**"

42
.github/workflows/ui-translations.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: UI Translations
on:
pull_request:
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
- ".github/workflows/ui-translations.yml"
push:
branches:
- main
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
jobs:
check-translations:
name: Check translation key parity
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "22"
# English (en) is the source of truth for translation keys; every other
# locale declared in _index.json must carry the exact same key set.
- name: Check translation key parity
run: node client/ui/i18n/check-translations.mjs

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -468,6 +468,13 @@ checksum:
- glob: ./infrastructure_files/migrate-to-enterprise.sh
release:
# The signing pipeline (netbirdio/sign-pipelines, dispatched by
# trigger_signer) marks the release latest once the Windows and macOS
# artifacts are signed. Without this override goreleaser marks it latest
# at publish time, while those artifacts are still unsigned.
make_latest: false
# Mark x.y.z-rc.* and other prerelease tags as prereleases on GitHub.
prerelease: auto
extra_files:
- glob: ./infrastructure_files/getting-started-with-zitadel.sh
- glob: ./release_files/install.sh

View File

@@ -144,3 +144,11 @@ uploads:
target: https://pkgs.wiretrustee.com/yum/{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}
username: dev@wiretrustee.com
method: PUT
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -43,3 +43,11 @@ checksum:
name_template: "{{ .ProjectName }}_darwin_checksums.txt"
changelog:
disable: true
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -134,3 +134,11 @@ uploads:
target: https://pkgs.wiretrustee.com/yum/{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}
username: dev@wiretrustee.com
method: PUT
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -1,6 +1,6 @@
# NetBird Agent Guidelines
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
network with a control plane. The **agent** (`client/`) runs on user machines as
a privileged daemon and manages the WireGuard interface, routing, firewall, and
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,

View File

@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
## Checklist before submitting a PR
As a critical network service and open-source project, we must enforce a few
As a critical network service and open source project, we must enforce a few
things before submitting a pull request. The
[pull request template](/.github/pull_request_template.md) mirrors this list —
fill it in rather than deleting it.

View File

@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
![CISPA_Logo_BLACK_EN_RZ_RGB (1)](https://user-images.githubusercontent.com/700848/203091324-c6d311a0-22b5-4b05-a288-91cbc6cdcc46.png)
### Acknowledgements
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
### Legal
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.

View File

@@ -14,7 +14,7 @@ Report security issues one of these two ways:
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
- **Email** — `security@netbird.io`.
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
filing a repository report.
### What to include

View File

@@ -204,8 +204,9 @@ func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}
// An empty hint is deliberate, not a fallback: a fresh or logged-out profile
// leaves the choice to the IdP, which is how accounts get switched.
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
if a.cfgPath != "" {
if hint := readProfileEmail(a.cfgPath); hint != "" {
if setter, ok := oAuthFlow.(loginHintSetter); ok {

View File

@@ -22,7 +22,8 @@ type Profile struct {
ID string
Name string
// Email is the account this profile last logged in with, "" if it never
// completed an SSO login or was logged out. See profile_state.go.
// completed an SSO login. Kept across logouts; cleared when the profile is
// removed. See profile_state.go.
Email string
IsActive bool
}
@@ -200,11 +201,9 @@ func (pm *ProfileManager) LogoutProfile(id string) error {
return fmt.Errorf("failed to save config: %w", err)
}
// Not fatal: a stale hint costs an account switch, not the logout itself.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to clear stored account email for profile %s: %v", id, err)
}
// The stored account email is kept on purpose, matching the desktop and CLI
// logout semantics: the next login passes it as the login_hint so the IdP
// preselects the account. Removing the profile is what deletes it.
log.Infof("logged out from profile: %s", id)
return nil
}
@@ -224,11 +223,24 @@ func (pm *ProfileManager) RenameProfile(id string, newName string) error {
// RemoveProfile deletes a profile
func (pm *ProfileManager) RemoveProfile(id string) error {
configPath, err := pm.getProfileConfigPath(id)
if err != nil {
return err
}
// Use ServiceManager (removes profile from profiles/ directory)
if err := pm.serviceMgr.RemoveProfile(profilemanager.ID(id), androidUsername); err != nil {
return fmt.Errorf("failed to remove profile: %w", err)
}
// The account file is this package's, not the ServiceManager's, so it must
// go here. The default profile has a fixed filename, so a recreated one
// would otherwise inherit the deleted profile's email as its login_hint.
// Not fatal: the profile itself is gone.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to remove stored account email for profile %s: %v", id, err)
}
log.Infof("removed profile: %s", id)
return nil
}

View File

@@ -90,10 +90,10 @@ func writeProfileEmail(configPath string, email string) error {
return nil
}
// removeProfileEmail drops the stored account email. Called on logout: while the
// email is on disk it goes out as a login_hint, which would steer the next login
// straight back into the account just logged out of. Mirrors the desktop UI's
// RemoveProfileState call.
// removeProfileEmail drops the stored account email. Called on profile removal,
// not on logout: a logged-out profile keeps its email so the next login passes
// it as the login_hint, matching the desktop and CLI semantics. Mirrors the
// desktop UI's RemoveProfileState call.
func removeProfileEmail(configPath string) error {
accountPath, err := profileAccountPathFor(configPath)
if err != nil {

View File

@@ -127,10 +127,10 @@ func TestWriteThenReadProfileEmail(t *testing.T) {
t.Fatalf("remove: %v", err)
}
if got := readProfileEmail(configPath); got != "" {
t.Errorf("expected no email after logout, got %q", got)
t.Errorf("expected no email after removal, got %q", got)
}
// Logout may run on a never-logged-in profile, so a second remove must pass.
// Removal may run on a never-logged-in profile, so a second remove must pass.
if err := removeProfileEmail(configPath); err != nil {
t.Fatalf("second remove should be a no-op: %v", err)
}

View File

@@ -305,6 +305,12 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
return domain
}
// A reverse zone names an address prefix, so it follows the address rules,
// which also keeps its digit labels intact.
if zone, ok := a.anonymizeReverseZone(baseDomain); ok {
return withTrailingDot(zone, hasDot)
}
if suffix := protectedSuffix(baseDomain); suffix != "" {
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
return domain
@@ -405,6 +411,10 @@ func (a *Anonymizer) AnonymizeString(str string) string {
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
// Reverse zones go first and are then held out of the passes below: their
// labels are digits, which the address patterns would otherwise consume.
str, restoreZones := a.replaceReverseZones(str)
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
@@ -425,7 +435,7 @@ func (a *Anonymizer) AnonymizeString(str string) string {
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
}
return str
return restoreZones(str)
}
// sortedDomains returns the domain mappings longest-first, so a full-FQDN

View File

@@ -0,0 +1,174 @@
package anonymize
import (
"encoding/hex"
"net/netip"
"regexp"
"strconv"
"strings"
)
const (
reverseZoneSuffixV4 = ".in-addr.arpa"
reverseZoneSuffixV6 = ".ip6.arpa"
v6Nibbles = 32
v4Octets = 4
)
// reverseZoneRegexes match a reverse zone or a full reverse name in free text.
// They are applied before the address passes of AnonymizeString, whose IPv4
// pattern would otherwise consume the digit labels of a zone and replace parts
// of it with unrelated addresses.
var reverseZoneRegexes = []*regexp.Regexp{
regexp.MustCompile(`(?:[0-9]{1,3}\.){1,4}in-addr\.arpa\b`),
regexp.MustCompile(`(?:[0-9a-fA-F]\.){1,32}ip6\.arpa\b`),
}
// anonymizeReverseZone maps a reverse zone to the zone of the anonymized form
// of the prefix it encodes, so it follows the address rules rather than the
// domain ones: the zone of an address that is preserved is preserved too, and
// the zone of one that is replaced names the replacement. This keeps a reverse
// zone recognizable as such, and consistent with the addresses it belongs to
// elsewhere in the same output. It reports false for anything that is not a
// reverse zone.
func (a *Anonymizer) anonymizeReverseZone(domain string) (string, bool) {
prefix, labelCount, suffix, ok := parseReverseZone(domain)
if !ok {
return "", false
}
anonymized := a.AnonymizeIP(prefix)
if anonymized == prefix {
return domain, true
}
return reverseZoneName(anonymized, labelCount) + suffix, true
}
// replaceReverseZones anonymizes every reverse zone in str and swaps each one
// for a placeholder, returning a function that puts the anonymized zones back.
// The placeholders carry no dots, digits or colons, so no later pass matches
// them.
func (a *Anonymizer) replaceReverseZones(str string) (string, func(string) string) {
var zones []string
for _, re := range reverseZoneRegexes {
str = re.ReplaceAllStringFunc(str, func(match string) string {
zone, ok := a.anonymizeReverseZone(match)
if !ok {
return match
}
zones = append(zones, zone)
return reverseZonePlaceholder(len(zones) - 1)
})
}
if len(zones) == 0 {
return str, func(s string) string { return s }
}
return str, func(s string) string {
for i, zone := range zones {
s = strings.ReplaceAll(s, reverseZonePlaceholder(i), zone)
}
return s
}
}
func reverseZonePlaceholder(index int) string {
return "\x00reversezone" + strconv.Itoa(index) + "\x00"
}
// parseReverseZone turns a reverse zone into the address of the prefix its
// labels spell backwards, padding the absent low-order part with zeroes, and
// returns the label count and zone suffix so the name can be rebuilt.
func parseReverseZone(domain string) (netip.Addr, int, string, bool) {
lower := strings.ToLower(domain)
switch {
case strings.HasSuffix(lower, reverseZoneSuffixV4):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV4), ".")
addr, ok := reverseZoneAddrV4(labels)
return addr, len(labels), reverseZoneSuffixV4, ok
case strings.HasSuffix(lower, reverseZoneSuffixV6):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV6), ".")
addr, ok := reverseZoneAddrV6(labels)
return addr, len(labels), reverseZoneSuffixV6, ok
default:
return netip.Addr{}, 0, "", false
}
}
func reverseZoneAddrV4(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v4Octets {
return netip.Addr{}, false
}
var octets [v4Octets]byte
for i, label := range labels {
octet, err := strconv.ParseUint(label, 10, 8)
if err != nil {
return netip.Addr{}, false
}
octets[len(labels)-1-i] = byte(octet)
}
return netip.AddrFrom4(octets), true
}
func reverseZoneAddrV6(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v6Nibbles {
return netip.Addr{}, false
}
nibbles := make([]byte, 0, v6Nibbles)
for i := len(labels) - 1; i >= 0; i-- {
if len(labels[i]) != 1 || !isHexDigit(labels[i][0]) {
return netip.Addr{}, false
}
nibbles = append(nibbles, labels[i][0])
}
for len(nibbles) < v6Nibbles {
nibbles = append(nibbles, '0')
}
var groups []string
for i := 0; i < len(nibbles); i += 4 {
groups = append(groups, string(nibbles[i:i+4]))
}
addr, err := netip.ParseAddr(strings.Join(groups, ":"))
if err != nil {
return netip.Addr{}, false
}
return addr, true
}
// reverseZoneName spells the first labelCount labels of addr backwards, the
// inverse of parseReverseZone, without the zone suffix.
func reverseZoneName(addr netip.Addr, labelCount int) string {
labels := make([]string, 0, labelCount)
if addr.Is4() {
octets := addr.As4()
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, strconv.Itoa(int(octets[i])))
}
return strings.Join(labels, ".")
}
address := addr.As16()
nibbles := hex.EncodeToString(address[:])
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, string(nibbles[i]))
}
return strings.Join(labels, ".")
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}

View File

@@ -0,0 +1,171 @@
package anonymize
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newLeveledAnonymizer(level Level) *Anonymizer {
a := NewAnonymizer(DefaultAddresses())
a.SetLevel(level)
return a
}
// TestAnonymizeDomainReverseZone covers reverse zones going through the address
// rules instead of the domain ones, so a zone stays a zone and an address that
// is preserved keeps the zone that names it.
func TestAnonymizeDomainReverseZone(t *testing.T) {
// 100.64.0.0/10 is the overlay range, which is CGNAT: preserved at the
// default level and replaced from the internal pool at the strict one
const overlayZone = "64.100.in-addr.arpa"
t.Run("overlay zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, overlayZone, a.AnonymizeDomain(overlayZone), "should keep the zone of a preserved address")
})
t.Run("private zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "168.192.in-addr.arpa", a.AnonymizeDomain("168.192.in-addr.arpa"), "should keep the zone of a private address")
})
t.Run("overlay zone replaced at the strict level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelStrict)
got := a.AnonymizeDomain(overlayZone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, overlayZone, got, "should replace the encoded prefix")
assert.Len(t, strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV4), "."), 2,
"should keep the label count, got %q", got)
})
t.Run("public zone replaced at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("113.0.203.in-addr.arpa")
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, "113.0.203.in-addr.arpa", got, "should replace a public prefix")
})
t.Run("zone of an address keeps that address mapping", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
anonymizedAddr := a.AnonymizeIPString("203.0.113.7")
got := a.AnonymizeDomain("7.113.0.203.in-addr.arpa")
octets := strings.Split(anonymizedAddr, ".")
want := octets[3] + "." + octets[2] + "." + octets[1] + "." + octets[0] + reverseZoneSuffixV4
assert.Equal(t, want, got, "should name the same replacement as the address itself")
})
t.Run("ipv6 nibble labels stay single digits", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeDomain(zone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV6), "should stay a reverse zone, got %q", got)
labels := strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV6), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
for _, label := range labels {
assert.Len(t, label, 1, "nibble label %q should stay a single digit", label)
}
})
t.Run("trailing dot is kept", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "64.100.in-addr.arpa.", a.AnonymizeDomain("64.100.in-addr.arpa."), "should keep the trailing dot")
})
t.Run("a domain that only looks like a zone is anonymized as a domain", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("not-a-zone.in-addr.arpa")
assert.NotContains(t, got, "in-addr.arpa", "should fall back to domain anonymization")
})
}
// TestAnonymizeStringReverseZone verifies that a zone inside free text, such as
// a DNS log line, is not chewed up by the address passes. The IPv4 pattern
// matches any run of dotted digits, which a reverse zone is made of.
func TestAnonymizeStringReverseZone(t *testing.T) {
t.Run("ipv6 zone survives the address passes", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeString("question: domain=" + zone + " type=PTR")
assert.Contains(t, got, "type=PTR", "should keep the rest of the line")
assert.NotContains(t, got, "198.51.100", "should not rewrite nibble labels as an address")
labels := strings.Split(strings.TrimSuffix(strings.TrimPrefix(got, "question: domain="), reverseZoneSuffixV6+" type=PTR"), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
})
t.Run("preserved ipv4 zone is untouched", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
line := "reverse zone 64.100.in-addr.arpa registered"
assert.Equal(t, line, a.AnonymizeString(line), "should keep the zone of a preserved address")
})
t.Run("public ipv4 zone is replaced consistently", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeString("zone 113.0.203.in-addr.arpa and address 203.0.113.7")
assert.NotContains(t, got, "113.0.203.in-addr.arpa", "should replace the zone")
assert.NotContains(t, got, "203.0.113.7", "should replace the address")
assert.Contains(t, got, reverseZoneSuffixV4, "should keep the zone suffix")
})
}
func TestParseReverseZone(t *testing.T) {
tests := []struct {
name string
zone string
addr string
labels int
}{
{name: "v4 two labels", zone: "0.100" + reverseZoneSuffixV4, addr: "100.0.0.0", labels: 2},
{name: "v4 three labels", zone: "1.168.192" + reverseZoneSuffixV4, addr: "192.168.1.0", labels: 3},
{name: "v4 full address", zone: "7.113.0.203" + reverseZoneSuffixV4, addr: "203.0.113.7", labels: 4},
{
name: "v6 prefix",
zone: "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6,
addr: "2::",
labels: 28,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
addr, labels, suffix, ok := parseReverseZone(tc.zone)
require.True(t, ok, "should decode the reverse zone")
assert.Equal(t, tc.addr, addr.String(), "should decode to the encoded prefix")
assert.Equal(t, tc.labels, labels, "should count the labels")
assert.Equal(t, tc.zone, reverseZoneName(addr, labels)+suffix, "should re-encode to the original zone")
})
}
}
func TestParseReverseZoneRejectsNonZones(t *testing.T) {
tests := []string{
"example.com",
"in-addr.arpa",
"x.100" + reverseZoneSuffixV4,
"256" + reverseZoneSuffixV4,
"1.2.3.4.5" + reverseZoneSuffixV4,
"ab" + reverseZoneSuffixV6,
"g" + reverseZoneSuffixV6,
}
for _, zone := range tests {
t.Run(zone, func(t *testing.T) {
_, _, _, ok := parseReverseZone(zone)
assert.False(t, ok, "should reject %q", zone)
})
}
}

View File

@@ -5,7 +5,6 @@ import (
"fmt"
"os"
"os/user"
"runtime"
"strings"
log "github.com/sirupsen/logrus"
@@ -121,7 +120,7 @@ func doDaemonLogin(ctx context.Context, cmd *cobra.Command, providedSetupKey str
loginRequest := proto.LoginRequest{
SetupKey: providedSetupKey,
ManagementUrl: managementURL,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
DnsLabels: dnsLabelsReq,
ProfileName: &handle,
@@ -189,7 +188,8 @@ func doExtendSession(ctx context.Context, cmd *cobra.Command) error {
client := proto.NewDaemonServiceClient(conn)
req := &proto.RequestExtendAuthSessionRequest{}
// the CLI runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: util.HasGraphicalSession()}
// Pre-fill the IdP login hint from the active profile so the user
// doesn't have to retype their email. Best-effort: we still proceed
// without a hint if the lookup fails.
@@ -408,7 +408,7 @@ func foregroundGetTokenInfo(ctx context.Context, cmd *cobra.Command, config *pro
hint = profileState.Email
}
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isUnixRunningDesktop(), false, hint)
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, util.HasGraphicalSession(), false, hint)
if err != nil {
return nil, err
}
@@ -458,14 +458,6 @@ func openURL(cmd *cobra.Command, verificationURIComplete, userCode string, noBro
}
}
// isUnixRunningDesktop checks if a Linux OS is running desktop environment
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func setEnvAndFlags(cmd *cobra.Command) error {
SetFlagsFromEnvVars(rootCmd)

View File

@@ -21,8 +21,8 @@ import (
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
@@ -626,7 +626,7 @@ func setupLoginRequest(providedSetupKey string, customDNSAddressConverted []byte
NatExternalIPs: natExternalIPs,
CleanNATExternalIPs: natExternalIPs != nil && len(natExternalIPs) == 0,
CustomDNSAddress: customDNSAddressConverted,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
ExtraIFaceBlacklist: extraIFaceBlackList,
DnsLabels: dnsLabels,

View File

@@ -42,12 +42,12 @@ type aclManager struct {
optionalEntries map[string][]entry
ipsetStore *ipsetStore
v6 bool
ipsetSupport *ipsetSupport
ipsetSupported bool
stateManager *statemanager.Manager
}
func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper, ipsetSupport *ipsetSupport) (*aclManager, error) {
func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper) (*aclManager, error) {
return &aclManager{
iptablesClient: iptablesClient,
wgIface: wgIface,
@@ -55,13 +55,14 @@ func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper, ipset
optionalEntries: make(map[string][]entry),
ipsetStore: newIpsetStore(),
v6: iptablesClient.Proto() == iptables.ProtocolIPv6,
ipsetSupport: ipsetSupport,
}, nil
}
func (m *aclManager) init(stateManager *statemanager.Manager) error {
m.stateManager = stateManager
m.ipsetSupported = m.probeIPSetSupport()
m.seedInitialEntries()
m.seedInitialOptionalEntries()
@@ -86,78 +87,19 @@ func (m *aclManager) AddPeerFiltering(
dPort *firewall.Port,
action firewall.Action,
ipsetName string,
) ([]firewall.Rule, error) {
ipsetName = m.resolveIPSetName(ipsetName, sPort, dPort, action)
if ipsetName == "" {
return m.addPeerRule(ip, protocol, sPort, dPort, action, "")
}
// A set that is already in the store backs rules installed earlier, so it must
// survive this call's failure.
_, preexisting := m.ipsetStore.ipset(ipsetName)
rules, err := m.addPeerRule(ip, protocol, sPort, dPort, action, ipsetName)
if err == nil {
return rules, nil
}
var unusable *ipsetUnusableError
if !errors.As(err, &unusable) {
return nil, err
}
// The set could not be created or matched. Drop the one this call created and
// retry the rule matching the IP directly; only if that succeeds do we know
// ipset was to blame and latch it off for subsequent rules.
if !preexisting {
m.discardIPSet(ipsetName)
}
rules, retryErr := m.addPeerRule(ip, protocol, sPort, dPort, action, "")
if retryErr != nil {
return nil, fmt.Errorf("add peer rule (ipset: %w): %w", unusable.cause, retryErr)
}
m.ipsetSupport.markUnsupported(unusable.cause)
return rules, nil
}
// resolveIPSetName derives the ipset name for a rule, returning "" when the rule
// must match the IP directly: either no set was requested or ipset is unusable.
func (m *aclManager) resolveIPSetName(ipsetName string, sPort, dPort *firewall.Port, action firewall.Action) string {
ipsetName = transformIPsetName(ipsetName, sPort, dPort, action)
if ipsetName == "" || !m.ipsetSupport.supported() {
return ""
}
if m.v6 {
ipsetName += "-v6"
}
return ipsetName
}
// discardIPSet removes a set that turned out to be unusable, so a later rule
// does not find it in the store and assume it works.
func (m *aclManager) discardIPSet(ipsetName string) {
m.ipsetStore.deleteIpset(ipsetName)
if err := m.destroyIPSet(ipsetName); err != nil {
log.Debugf("destroy unusable ipset %s: %v", ipsetName, err)
}
}
func (m *aclManager) addPeerRule(
ip net.IP,
protocol firewall.Protocol,
sPort *firewall.Port,
dPort *firewall.Port,
action firewall.Action,
ipsetName string,
) ([]firewall.Rule, error) {
chain := chainNameInputRules
ipsetName = transformIPsetName(ipsetName, sPort, dPort, action)
if m.v6 && ipsetName != "" {
ipsetName += "-v6"
}
// When the kernel lacks the required ipset hash module, fall back to
// per-IP iptables rules (pre-0.68 behavior) so ACLs keep working instead
// of silently leaving the chain empty.
if ipsetName != "" && !m.ipsetSupported {
ipsetName = ""
}
proto := protoForFamily(protocol, m.v6)
specs := filterRuleSpecs(ip, proto, sPort, dPort, action, ipsetName)
@@ -172,7 +114,7 @@ func (m *aclManager) addPeerRule(
if ipsetName != "" {
if ipList, ipsetExists := m.ipsetStore.ipset(ipsetName); ipsetExists {
if err := m.addToIPSet(ipsetName, ip); err != nil {
return nil, ipsetUnusable(fmt.Errorf("add IP to ipset: %w", err))
return nil, fmt.Errorf("add IP to ipset: %w", err)
}
// if ruleset already exists it means we already have the firewall rule
// so we need to update IPs in the ruleset and return new fw.Rule object for ACL manager.
@@ -195,10 +137,10 @@ func (m *aclManager) addPeerRule(
}
}
if err := m.createIPSet(ipsetName); err != nil {
return nil, ipsetUnusable(fmt.Errorf("create ipset: %w", err))
return nil, fmt.Errorf("create ipset: %w", err)
}
if err := m.addToIPSet(ipsetName, ip); err != nil {
return nil, ipsetUnusable(fmt.Errorf("add IP to ipset: %w", err))
return nil, fmt.Errorf("add IP to ipset: %w", err)
}
ipList := newIpList(ip.String())
@@ -207,7 +149,7 @@ func (m *aclManager) addPeerRule(
ok, err := m.iptablesClient.Exists(tableFilter, chain, specs...)
if err != nil {
return nil, maybeIPSetUnusable(ipsetName, fmt.Errorf("check rule: %w", err))
return nil, fmt.Errorf("failed to check rule: %w", err)
}
if ok {
return nil, fmt.Errorf("rule already exists")
@@ -221,7 +163,7 @@ func (m *aclManager) addPeerRule(
err = m.iptablesClient.Append(tableFilter, chain, specs...)
}
if err != nil {
return nil, maybeIPSetUnusable(ipsetName, err)
return nil, err
}
if err := m.iptablesClient.Append(tableMangle, chainRTPRE, mangleSpecs...); err != nil {
@@ -565,6 +507,40 @@ func transformIPsetName(ipsetName string, sPort, dPort *firewall.Port, action fi
}
}
// probeIPSetSupport checks whether the kernel can create the ipset type used for
// ACL rules. On kernels lacking the required ipset hash module, ipset creation
// fails (e.g. "invalid argument"), which would otherwise leave the ACL chain
// empty and silently drop all policy-permitted inbound traffic. When unsupported,
// the manager falls back to per-IP iptables rules.
func (m *aclManager) probeIPSetSupport() bool {
// Use a unique name so concurrent processes don't collide and we only ever
// destroy the set we created ourselves. ipset names are limited to 31 chars,
// so use a short random suffix.
probeName := "nb-probe-" + uuid.New().String()[:8]
opts := ipset.CreateOptions{
Replace: true,
}
if m.v6 {
opts.Family = ipset.FamilyIPV6
}
if err := ipset.Create(probeName, ipset.TypeHashNet, opts); err != nil {
log.Warnf("ipset is not available (failed to create probe set: %v); "+
"falling back to per-IP iptables ACL rules. Ensure the kernel provides "+
"the ipset hash:net module (ip_set_hash_net) for better performance with large rule sets", err)
return false
}
defer func() {
if err := ipset.Destroy(probeName); err != nil {
log.Debugf("destroy ipset probe set %q: %v", probeName, err)
}
}()
return true
}
func (m *aclManager) createIPSet(name string) error {
opts := ipset.CreateOptions{
Replace: true,

View File

@@ -1,75 +0,0 @@
package iptables
import (
"sync"
log "github.com/sirupsen/logrus"
)
// ipsetSupport tracks whether ipset-backed firewall rules can be installed.
//
// It starts optimistic and latches to unsupported the first time the kernel
// proves otherwise: either the hash:net set type is missing (ip_set_hash_net) or
// iptables cannot match against a set (xt_set). Callers then emit per-IP and
// per-prefix rules instead. Without the fallback, a rule referencing an unusable
// set is never installed and the catch-all DROP silently blocks traffic the
// policy permits.
//
// One instance is shared by the ACL managers and routers of both address
// families, because ipset availability is a property of the kernel rather than
// of any single table.
type ipsetSupport struct {
mu sync.RWMutex
unsupported bool
}
func newIPSetSupport() *ipsetSupport {
return &ipsetSupport{}
}
func (s *ipsetSupport) supported() bool {
s.mu.RLock()
defer s.mu.RUnlock()
return !s.unsupported
}
// markUnsupported records that ipset cannot be used, logging the reason once.
func (s *ipsetSupport) markUnsupported(cause error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.unsupported {
return
}
s.unsupported = true
log.Warnf("ipset is unavailable (%v); falling back to per-IP firewall rules. "+
"Ensure the kernel provides ip_set_hash_net and xt_set; without them rule "+
"sets are larger and slower to converge on networks with many peers", cause)
}
// ipsetUnusableError marks a failure attributable to ipset, so the caller can
// retry the same rule in its per-IP form before latching the capability off.
type ipsetUnusableError struct {
cause error
}
func (e *ipsetUnusableError) Error() string { return e.cause.Error() }
func (e *ipsetUnusableError) Unwrap() error { return e.cause }
func ipsetUnusable(cause error) error {
return &ipsetUnusableError{cause: cause}
}
// maybeIPSetUnusable marks an iptables failure as ipset-attributable only when the
// rule actually carried a set match, since the same call can fail for unrelated
// reasons on a rule that matches addresses directly.
func maybeIPSetUnusable(ipsetName string, err error) error {
if ipsetName == "" {
return err
}
return ipsetUnusable(err)
}

View File

@@ -33,10 +33,6 @@ type Manager struct {
router *router
rawSupported bool
// ipsetSupport is shared by the ACL managers and routers of both families,
// so a kernel without usable ipset support degrades them together.
ipsetSupport *ipsetSupport
// IPv6 counterparts, nil when no v6 overlay
ipv6Client *iptables.IPTables
aclMgr6 *aclManager
@@ -57,17 +53,16 @@ func Create(wgIface iFaceMapper, mtu uint16) (*Manager, error) {
}
m := &Manager{
wgIface: wgIface,
ipv4Client: iptablesClient,
ipsetSupport: newIPSetSupport(),
wgIface: wgIface,
ipv4Client: iptablesClient,
}
m.router, err = newRouter(iptablesClient, wgIface, mtu, m.ipsetSupport)
m.router, err = newRouter(iptablesClient, wgIface, mtu)
if err != nil {
return nil, fmt.Errorf("create router: %w", err)
}
m.aclMgr, err = newAclManager(iptablesClient, wgIface, m.ipsetSupport)
m.aclMgr, err = newAclManager(iptablesClient, wgIface)
if err != nil {
return nil, fmt.Errorf("create acl manager: %w", err)
}
@@ -88,7 +83,7 @@ func (m *Manager) createIPv6Components(wgIface iFaceMapper, mtu uint16) error {
}
m.ipv6Client = ip6Client
m.router6, err = newRouter(ip6Client, wgIface, mtu, m.ipsetSupport)
m.router6, err = newRouter(ip6Client, wgIface, mtu)
if err != nil {
return fmt.Errorf("create v6 router: %w", err)
}
@@ -97,7 +92,7 @@ func (m *Manager) createIPv6Components(wgIface iFaceMapper, mtu uint16) error {
// Forwarding refcounter is per-family but shared between v4 and v6 routers.
m.router6.ipFwdState = m.router.ipFwdState
m.aclMgr6, err = newAclManager(ip6Client, wgIface, m.ipsetSupport)
m.aclMgr6, err = newAclManager(ip6Client, wgIface)
if err != nil {
return fmt.Errorf("create v6 acl manager: %w", err)
}

View File

@@ -292,97 +292,39 @@ func TestIptablesCreatePerformance(t *testing.T) {
}
}
// newACLTestManager returns a started manager. Create()/Init() is used so the
// router-owned chains (chainRTFWDIN/OUT) exist before the ACL manager's
// createDefaultChains() references them.
func newACLTestManager(t *testing.T) *Manager {
t.Helper()
// TestIptablesACLIPSetFallback verifies that when the kernel lacks ipset support,
// the ACL manager falls back to per-IP iptables rules (-s <ip>) instead of
// silently leaving the chain empty. See discussion #6125.
func TestIptablesACLIPSetFallback(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
// Use Create()/Init() so the router-owned chains (chainRTFWDIN/OUT) are
// created before the ACL manager's createDefaultChains() references them.
manager, err := Create(ifaceMock, iface.DefaultMTU)
require.NoError(t, err)
require.NoError(t, manager.Init(nil))
t.Cleanup(func() {
aclMgr := manager.aclMgr
// Simulate a kernel without the ipset hash module.
aclMgr.ipsetSupported = false
defer func() {
require.NoError(t, manager.Close(nil))
})
return manager
}
// TestIptablesACLUsesIPSetOnHealthyKernel guards the default: on a kernel that
// does have ipset, rules must keep matching a set. A regression that reported
// ipset as unusable would silently move every Linux client to per-IP rules.
func TestIptablesACLUsesIPSetOnHealthyKernel(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
manager := newACLTestManager(t)
}()
ip := netip.MustParseAddr("10.20.0.42")
port := &fw.Port{Values: []uint16{22}}
rules, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err)
require.NotEmpty(t, rules)
rule := rules[0].(*Rule)
require.Equal(t, "nb0000001-dport", rule.ipsetName, "healthy kernel must use an ipset")
require.Contains(t, rule.specs, "--match-set")
require.True(t, manager.ipsetSupport.supported(), "ipset must not be latched off on a healthy kernel")
checkRuleSpecs(t, ipv4Client, rule.chain, true, rule.specs...)
}
// TestIptablesACLFallsBackWhenIPSetUnusable drives the real failure path: an
// oversized set name is rejected by the kernel, which stands in for a kernel
// without ip_set_hash_net or xt_set. The rule must still land in the chain,
// matching the IP directly, and the capability must latch off so later rules skip
// ipset. Before the fallback existed, the rule was dropped and the catch-all DROP
// silently blocked traffic the policy permits.
func TestIptablesACLFallsBackWhenIPSetUnusable(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
manager := newACLTestManager(t)
// ipset names are limited to 31 characters, so creating this set fails.
unusableName := strings.Repeat("a", 40)
ip := netip.MustParseAddr("10.20.0.42")
port := &fw.Port{Values: []uint16{22}}
rules, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, unusableName)
require.NoError(t, err, "AddPeerFiltering must succeed by falling back")
rules, err := aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err, "AddPeerFiltering should succeed via fallback")
require.NotEmpty(t, rules)
rule := rules[0].(*Rule)
require.Empty(t, rule.ipsetName, "fallback rule must not reference an ipset")
require.Contains(t, strings.Join(rule.specs, " "), "-s 10.20.0.42", "fallback rule must match the source IP")
require.NotContains(t, strings.Join(rule.specs, " "), "--match-set")
require.Contains(t, strings.Join(rule.specs, " "), "-s 10.20.0.42", "fallback rule must match by source IP")
require.NotContains(t, strings.Join(rule.specs, " "), "--match-set", "fallback rule must not use ipset matching")
// The rule must actually be present, not silently missing.
// The rule must actually be present in the ACL chain (not silently dropped).
checkRuleSpecs(t, ipv4Client, rule.chain, true, rule.specs...)
require.False(t, manager.ipsetSupport.supported(), "failure must latch ipset off")
// A subsequent rule with a perfectly valid set name now skips ipset too.
next, err := manager.aclMgr.AddPeerFiltering(nil, netip.MustParseAddr("10.20.0.43").AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err)
require.NotEmpty(t, next)
require.Empty(t, next[0].(*Rule).ipsetName, "later rules must skip ipset once latched")
}
// TestIptablesACLLeavesNoIPSetAfterFallback verifies the set created before the
// failure is destroyed, so a later rule does not find a half-built set and assume
// ipset works.
func TestIptablesACLLeavesNoIPSetAfterFallback(t *testing.T) {
manager := newACLTestManager(t)
port := &fw.Port{Values: []uint16{22}}
ip := netip.MustParseAddr("10.20.0.42")
_, err := manager.aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, strings.Repeat("a", 40))
require.NoError(t, err)
_, exists := manager.aclMgr.ipsetStore.ipset(strings.Repeat("a", 40) + "-dport")
require.False(t, exists, "unusable set must not stay in the store")
}

View File

@@ -3,7 +3,6 @@
package iptables
import (
"errors"
"fmt"
"maps"
"net/netip"
@@ -52,10 +51,6 @@ const (
markManglePost = "mark-mangle-post"
matchSet = "--match-set"
// routeSourceSuffix names the extra rules a route ACL needs when ipset is
// unusable and each source prefix has to be matched by its own rule.
routeSourceSuffix = "_src"
dnatSuffix = "_dnat"
snatSuffix = "_snat"
fwdSuffix = "_fwd"
@@ -73,6 +68,7 @@ type ruleInfo struct {
}
type routeFilteringRuleParams struct {
Source firewall.Network
Destination firewall.Network
Proto firewall.Protocol
SPort *firewall.Port
@@ -94,13 +90,12 @@ type router struct {
legacyManagement bool
mtu uint16
v6 bool
ipsetSupport *ipsetSupport
stateManager *statemanager.Manager
ipFwdState *ipfwdstate.IPForwardingState
}
func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint16, ipsetSupport *ipsetSupport) (*router, error) {
func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint16) (*router, error) {
r := &router{
iptablesClient: iptablesClient,
rules: make(map[string][]string),
@@ -108,7 +103,6 @@ func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint1
mtu: mtu,
v6: iptablesClient.Proto() == iptables.ProtocolIPv6,
ipFwdState: ipfwdstate.NewIPForwardingState(wgIface.Name()),
ipsetSupport: ipsetSupport,
}
r.ipsetCounter = refcounter.New(
@@ -157,113 +151,6 @@ func (r *router) AddRouteFiltering(
return ruleKey, nil
}
params := routeFilteringRuleParams{
Destination: destination,
Proto: proto,
SPort: sPort,
DPort: dPort,
Action: action,
}
err := r.installRouteRules(string(ruleKey), params, sources, r.ipsetSupport.supported())
var unusable *ipsetUnusableError
if errors.As(err, &unusable) {
// The set could not be created or matched. Retry matching each source
// prefix on its own; only if that works do we know ipset was to blame.
r.removeRouteRules(string(ruleKey))
if retryErr := r.installRouteRules(string(ruleKey), params, sources, false); retryErr != nil {
r.removeRouteRules(string(ruleKey))
return nil, fmt.Errorf("add route rule (ipset: %w): %w", unusable.cause, retryErr)
}
r.ipsetSupport.markUnsupported(unusable.cause)
err = nil
}
if err != nil {
// Leave nothing half-installed: a later call finding the rule key would
// report success while some sources were never installed, which for a
// drop rule would leave them unblocked.
r.removeRouteRules(string(ruleKey))
return nil, fmt.Errorf("add route rule: %w", err)
}
r.updateState()
return ruleKey, nil
}
// installRouteRules installs every rule needed for one route ACL and records them
// under ruleKey. It is more than one rule only when useIPSet is false and the
// sources have to be matched one prefix at a time.
func (r *router) installRouteRules(ruleKey string, params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool) error {
specs, err := r.genRouteRuleSpecs(params, sources, useIPSet)
if err != nil {
return fmt.Errorf("generate route rule spec: %w", err)
}
for i, spec := range specs {
if err := r.insertRouteRule(params.Action, spec); err != nil {
if len(r.findSets(spec)) > 0 {
return ipsetUnusable(err)
}
return err
}
r.rules[routeRuleKey(ruleKey, i)] = spec
}
return nil
}
// genRouteRuleSpecs builds the rules for one route ACL. With ipset available that
// is a single rule matching a set of sources; without it, one rule per source
// prefix, which is the only form a stripped kernel can express.
func (r *router) genRouteRuleSpecs(params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool) ([][]string, error) {
destExp, err := r.applyNetwork("-d", params.Destination, nil)
if err != nil {
return nil, fmt.Errorf("apply network -d: %w", err)
}
specs, err := r.genSourceRules(params, sources, useIPSet, destExp)
if err != nil {
// The destination match may have taken a set reference already.
if decErr := r.decrementSetCounter(destExp); decErr != nil {
log.Debugf("release destination set after failed rule generation: %v", decErr)
}
return nil, err
}
return specs, nil
}
func (r *router) genSourceRules(params routeFilteringRuleParams, sources []netip.Prefix, useIPSet bool, destExp []string) ([][]string, error) {
if useIPSet || len(sources) <= 1 {
sourceExp, err := r.applyNetwork("-s", sourceNetwork(sources), sources)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
return [][]string{assembleRouteRule(sourceExp, destExp, params, r.v6)}, nil
}
specs := make([][]string, 0, len(sources))
for _, source := range sources {
sourceExp, err := r.applyNetwork("-s", firewall.Network{Prefix: source}, nil)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
specs = append(specs, assembleRouteRule(sourceExp, destExp, params, r.v6))
}
return specs, nil
}
func sourceNetwork(sources []netip.Prefix) firewall.Network {
var source firewall.Network
if len(sources) > 1 {
source.Set = firewall.NewPrefixSet(sources)
@@ -271,48 +158,37 @@ func sourceNetwork(sources []netip.Prefix) firewall.Network {
source.Prefix = sources[0]
}
return source
}
params := routeFilteringRuleParams{
Source: source,
Destination: destination,
Proto: proto,
SPort: sPort,
DPort: dPort,
Action: action,
}
rule, err := r.genRouteRuleSpec(params, sources)
if err != nil {
return nil, fmt.Errorf("generate route rule spec: %w", err)
}
func (r *router) insertRouteRule(action firewall.Action, spec []string) error {
// Insert DROP rules at the beginning, append ACCEPT rules at the end
if action == firewall.ActionDrop {
// after the established rule
return r.iptablesClient.Insert(tableFilter, chainRTFWDIN, 2, spec...)
err = r.iptablesClient.Insert(tableFilter, chainRTFWDIN, 2, rule...)
} else {
err = r.iptablesClient.Append(tableFilter, chainRTFWDIN, rule...)
}
return r.iptablesClient.Append(tableFilter, chainRTFWDIN, spec...)
}
// removeRouteRules deletes the rules recorded for ruleKey, used to undo a partial
// install before retrying without ipset.
func (r *router) removeRouteRules(ruleKey string) {
for i := 0; ; i++ {
key := routeRuleKey(ruleKey, i)
spec, exists := r.rules[key]
if !exists {
return
}
if err := r.iptablesClient.DeleteIfExists(tableFilter, chainRTFWDIN, spec...); err != nil {
log.Debugf("delete partial route rule %s: %v", key, err)
}
delete(r.rules, key)
if err := r.decrementSetCounter(spec); err != nil {
log.Debugf("decrement ipset counter for %s: %v", key, err)
}
}
}
// routeRuleKey names the i-th rule of a route ACL. The first keeps the plain rule
// key so single-rule ACLs, which is every ACL when ipset works, are unaffected.
func routeRuleKey(ruleKey string, i int) string {
if i == 0 {
return ruleKey
if err != nil {
return nil, fmt.Errorf("add route rule: %v", err)
}
return fmt.Sprintf("%s%s%d", ruleKey, routeSourceSuffix, i)
r.rules[string(ruleKey)] = rule
r.updateState()
return ruleKey, nil
}
func (r *router) hasRule(id string) bool {
@@ -323,29 +199,17 @@ func (r *router) hasRule(id string) bool {
func (r *router) DeleteRouteRule(rule firewall.Rule) error {
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey]; !exists {
log.Debugf("route rule %s not found", ruleKey)
r.updateState()
return nil
}
// In the ipset fallback one ACL is several rules, one per source prefix.
for i := 0; ; i++ {
key := routeRuleKey(ruleKey, i)
rule, exists := r.rules[key]
if !exists {
break
}
if rule, exists := r.rules[ruleKey]; exists {
if err := r.iptablesClient.Delete(tableFilter, chainRTFWDIN, rule...); err != nil {
return fmt.Errorf("delete route rule: %v", err)
}
delete(r.rules, key)
delete(r.rules, ruleKey)
if err := r.decrementSetCounter(rule); err != nil {
return fmt.Errorf("decrement ipset counter: %w", err)
}
} else {
log.Debugf("route rule %s not found", ruleKey)
}
r.updateState()
@@ -1063,23 +927,31 @@ func (r *router) DeleteDNATRule(rule firewall.Rule) error {
return nberrors.FormatErrorOrNil(merr)
}
// assembleRouteRule joins the pre-built source and destination matches with the
// protocol, ports and target of a route ACL.
func assembleRouteRule(sourceExp, destExp []string, params routeFilteringRuleParams, v6 bool) []string {
func (r *router) genRouteRuleSpec(params routeFilteringRuleParams, sources []netip.Prefix) ([]string, error) {
var rule []string
sourceExp, err := r.applyNetwork("-s", params.Source, sources)
if err != nil {
return nil, fmt.Errorf("apply network -s: %w", err)
}
destExp, err := r.applyNetwork("-d", params.Destination, nil)
if err != nil {
return nil, fmt.Errorf("apply network -d: %w", err)
}
rule = append(rule, sourceExp...)
rule = append(rule, destExp...)
if params.Proto != firewall.ProtocolALL {
rule = append(rule, "-p", strings.ToLower(protoForFamily(params.Proto, v6)))
rule = append(rule, "-p", strings.ToLower(protoForFamily(params.Proto, r.v6)))
rule = append(rule, applyPort("--sport", params.SPort)...)
rule = append(rule, applyPort("--dport", params.DPort)...)
}
rule = append(rule, "-j", actionToStr(params.Action))
return rule
return rule, nil
}
func (r *router) applyNetwork(flag string, network firewall.Network, prefixes []netip.Prefix) ([]string, error) {
@@ -1089,17 +961,9 @@ func (r *router) applyNetwork(flag string, network firewall.Network, prefixes []
}
if network.IsSet() {
// A destination set is populated later from DNS results, so unlike a
// source set it cannot be expanded into per-prefix rules here. Without
// ipset such a rule is not expressible; report it instead of installing
// something broader than the policy allows.
if flag == "-d" && !r.ipsetSupport.supported() {
return nil, fmt.Errorf("destination set %s requires ipset (ip_set_hash_net and xt_set)", network.Set.HashedName())
}
name := r.ipsetName(network.Set.HashedName())
if _, err := r.ipsetCounter.Increment(name, prefixes); err != nil {
return nil, ipsetUnusable(fmt.Errorf("create or get ipset: %w", err))
return nil, fmt.Errorf("create or get ipset: %w", err)
}
return []string{"-m", "set", matchSet, name, direction}, nil

View File

@@ -3,11 +3,9 @@
package iptables
import (
"errors"
"fmt"
"net/netip"
"os/exec"
"strings"
"testing"
"github.com/coreos/go-iptables/iptables"
@@ -17,9 +15,7 @@ import (
firewall "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/firewall/test"
"github.com/netbirdio/netbird/client/iface"
nbid "github.com/netbirdio/netbird/client/internal/acl/id"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/shared/management/domain"
)
func isIptablesSupported() bool {
@@ -35,7 +31,7 @@ func TestIptablesManager_RestoreOrCreateContainers(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "failed to init iptables client")
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "should return a valid iptables manager")
require.NoError(t, manager.init(nil))
@@ -88,7 +84,7 @@ func TestIptablesManager_AddNatRule(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "failed to init iptables client")
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "shouldn't return error")
require.NoError(t, manager.init(nil))
@@ -161,7 +157,7 @@ func TestIptablesManager_RemoveNatRule(t *testing.T) {
t.Run(testCase.Name, func(t *testing.T) {
iptablesClient, _ := iptables.NewWithProtocol(iptables.ProtocolIPv4)
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
manager, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "shouldn't return error")
require.NoError(t, manager.init(nil))
defer func() {
@@ -223,7 +219,7 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "Failed to create iptables client")
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, newIPSetSupport())
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU)
require.NoError(t, err, "Failed to create router manager")
require.NoError(t, r.init(nil))
@@ -341,26 +337,27 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
ruleKey, err := r.AddRouteFiltering(nil, tt.sources, firewall.Network{Prefix: tt.destination}, tt.proto, tt.sPort, tt.dPort, tt.action)
require.NoError(t, err, "AddRouteFiltering failed")
// A kernel without usable ipset splits a multi-source ACL into one
// rule per source, so compare against whichever form is in effect.
useIPSet := r.ipsetSupport.supported()
// Check if the rule is in the internal map
rule, ok := r.rules[ruleKey.ID()]
assert.True(t, ok, "Rule not found in internal map")
// Check if the rules are in the internal map
rules := routeRuleSpecs(t, r, ruleKey.ID())
require.NotEmpty(t, rules, "Rule not found in internal map")
// Log the internal rule
t.Logf("Internal rule: %v", rule)
// Log the internal rules
t.Logf("Internal rules: %v", rules)
// Check if the rule exists in iptables
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, rule...)
assert.NoError(t, err, "Failed to check rule existence")
assert.True(t, exists, "Rule not found in iptables")
// Check if the rules exist in iptables
for _, rule := range rules {
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, rule...)
assert.NoError(t, err, "Failed to check rule existence")
assert.True(t, exists, "Rule not found in iptables")
var source firewall.Network
if len(tt.sources) > 1 {
source.Set = firewall.NewPrefixSet(tt.sources)
} else if len(tt.sources) > 0 {
source.Prefix = tt.sources[0]
}
// Verify rule content
params := routeFilteringRuleParams{
Source: source,
Destination: firewall.Network{Prefix: tt.destination},
Proto: tt.proto,
SPort: tt.sPort,
@@ -368,18 +365,20 @@ func TestRouter_AddRouteFiltering(t *testing.T) {
Action: tt.action,
}
expectedRules, err := r.genRouteRuleSpecs(params, tt.sources, useIPSet)
expectedRule, err := r.genRouteRuleSpec(params, nil)
require.NoError(t, err, "Failed to generate expected rule spec")
if tt.expectSet && useIPSet {
if tt.expectSet {
setName := firewall.NewPrefixSet(tt.sources).HashedName()
expectedRule, err = r.genRouteRuleSpec(params, nil)
require.NoError(t, err, "Failed to generate expected rule spec with set")
// Check if the set was created
_, exists := r.ipsetCounter.Get(setName)
assert.True(t, exists, "IPSet not created")
}
assert.Equal(t, expectedRules, rules, "Rule content mismatch")
assert.Equal(t, expectedRule, rule, "Rule content mismatch")
// Clean up
err = r.DeleteRouteRule(ruleKey)
@@ -446,145 +445,3 @@ func TestFindSetNameInRule(t *testing.T) {
})
}
}
// TestRouter_AddRouteFilteringIPSetFallback covers a kernel that cannot use ipset:
// a multi-source route ACL must become one rule per source prefix, all present in
// the chain, and deleting the ACL must remove every one of them. Without the
// fallback the rule was never installed and the interface-wide DROP in FORWARD
// silently dropped routed traffic.
func TestRouter_AddRouteFilteringIPSetFallback(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
sources := []netip.Prefix{
netip.MustParsePrefix("172.16.0.0/16"),
netip.MustParsePrefix("192.168.0.0/16"),
}
destination := firewall.Network{Prefix: netip.MustParsePrefix("10.0.0.0/8")}
rule, err := r.AddRouteFiltering(nil, sources, destination, firewall.ProtocolTCP, nil,
&firewall.Port{Values: []uint16{443}}, firewall.ActionAccept)
require.NoError(t, err, "route ACL must install without ipset")
specs := routeRuleSpecs(t, r, rule.ID())
require.Len(t, specs, len(sources), "each source prefix needs its own rule")
for i, spec := range specs {
joined := strings.Join(spec, " ")
require.Contains(t, joined, "-s "+sources[i].String(), "rule must match the source prefix directly")
require.NotContains(t, joined, matchSet, "fallback rule must not reference a set")
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, spec...)
require.NoError(t, err)
require.True(t, exists, "rule %d must be present in %s", i, chainRTFWDIN)
}
require.NoError(t, r.DeleteRouteRule(rule))
for i, spec := range specs {
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, spec...)
require.NoError(t, err)
require.False(t, exists, "rule %d must be removed", i)
}
require.Empty(t, routeRuleSpecs(t, r, rule.ID()), "no rule may be left recorded")
}
// TestRouter_DestinationSetRequiresIPSet documents that a dynamic (domain)
// destination cannot be expressed without ipset: its prefixes are only known
// after DNS resolution, so there is nothing to expand into per-prefix rules. The
// call must report that rather than install a broader rule than the policy allows.
func TestRouter_DestinationSetRequiresIPSet(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
destination := firewall.Network{Set: firewall.NewDomainSet(domain.List{"example.com"})}
_, err = r.AddRouteFiltering(nil, []netip.Prefix{netip.MustParsePrefix("172.16.0.0/16")},
destination, firewall.ProtocolALL, nil, nil, firewall.ActionAccept)
require.Error(t, err, "a domain destination is not expressible without ipset")
require.ErrorContains(t, err, "requires ipset")
}
// TestRouter_RouteFilteringRollsBackPartialInstall covers a fallback ACL whose
// second rule cannot be installed. Nothing may be left behind: if the rule key
// survived, a later call would short-circuit on it and report success while some
// sources were never installed, leaving them unblocked for a drop rule.
func TestRouter_RouteFilteringRollsBackPartialInstall(t *testing.T) {
if !isIptablesSupported() {
t.Skip("iptables not supported on this system")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
support := newIPSetSupport()
support.markUnsupported(errors.New("test: pretend the kernel has no ipset"))
r, err := newRouter(iptablesClient, ifaceMock, iface.DefaultMTU, support)
require.NoError(t, err)
require.NoError(t, r.init(nil))
t.Cleanup(func() {
require.NoError(t, r.Reset())
})
// The v6 prefix is rejected by the v4 iptables binary, so the second rule of
// the expansion fails after the first has been installed.
good := netip.MustParsePrefix("172.16.0.0/16")
sources := []netip.Prefix{good, netip.MustParsePrefix("2001:db8::/32")}
destination := firewall.Network{Prefix: netip.MustParsePrefix("10.0.0.0/8")}
_, err = r.AddRouteFiltering(nil, sources, destination, firewall.ProtocolALL, nil, nil, firewall.ActionDrop)
require.Error(t, err, "a source that iptables rejects must fail the whole ACL")
ruleKey := nbid.GenerateRouteRuleKey(sources, destination, firewall.ProtocolALL, nil, nil, firewall.ActionDrop)
require.Empty(t, routeRuleSpecs(t, r, string(ruleKey)), "no rule may stay recorded")
// The rule that did get installed must be gone from the chain.
installed := []string{"-s", good.String(), "-d", "10.0.0.0/8", "-j", "DROP"}
exists, err := iptablesClient.Exists(tableFilter, chainRTFWDIN, installed...)
require.NoError(t, err)
require.False(t, exists, "the already-installed rule must be rolled back")
}
// routeRuleSpecs collects the rules recorded for one route ACL, which is more than
// one when the ipset fallback splits it per source prefix.
func routeRuleSpecs(t *testing.T, r *router, ruleKey string) [][]string {
t.Helper()
var specs [][]string
for i := 0; ; i++ {
spec, exists := r.rules[routeRuleKey(ruleKey, i)]
if !exists {
return specs
}
specs = append(specs, spec)
}
}

View File

@@ -22,8 +22,6 @@
!define UI_REG_APP_PATH "Software\Microsoft\Windows\CurrentVersion\App Paths\${UI_APP_EXE}"
!define UI_UNINSTALL_PATH "Software\Microsoft\Windows\CurrentVersion\Uninstall\${UI_APP_NAME}"
!define AUTOSTART_REG_KEY "Software\Microsoft\Windows\CurrentVersion\Run"
!define NETBIRD_DATA_DIR "$COMMONPROGRAMDATA\Netbird"
Unicode True
@@ -228,13 +226,6 @@ WriteRegStr ${REG_ROOT} "${UNINSTALL_PATH}" "Publisher" "${COMP_NAME}"
WriteRegStr ${REG_ROOT} "${UI_REG_APP_PATH}" "" "$INSTDIR\${UI_APP_EXE}"
; Autostart is owned by the UI's per-user setting (HKCU\...\Run via Wails),
; not the installer. Drop the machine-wide entry older installers wrote so the
; toggle is the single source of truth. HKCU is left untouched -- it may hold
; the user's own toggle state, which must survive upgrades.
DetailPrint "Removing installer-managed autostart registry entry if present..."
DeleteRegValue HKLM "${AUTOSTART_REG_KEY}" "${APP_NAME}"
EnVar::SetHKLM
EnVar::AddValueEx "path" "$INSTDIR"
@@ -299,15 +290,6 @@ ExecWait '"$INSTDIR\${MAIN_APP_EXE}" service uninstall'
DetailPrint "Terminating Netbird UI process..."
ExecWait `taskkill /im ${UI_APP_EXE}.exe /f`
; Remove autostart registry entries
DetailPrint "Removing autostart registry entries if they exist..."
; Legacy machine-wide entry written by older installers.
DeleteRegValue HKLM "${AUTOSTART_REG_KEY}" "${APP_NAME}"
; Per-user entry the UI toggle writes via Wails (value name is the lowercase
; app-name slug). Uninstall removes the app, so drop it too.
DeleteRegValue HKCU "${AUTOSTART_REG_KEY}" "${APP_NAME}"
DeleteRegValue HKCU "${AUTOSTART_REG_KEY}" "netbird"
; Handle data deletion based on checkbox
DetailPrint "Checking if user requested data deletion..."
${If} $DeleteDataEnabled == "1"

View File

@@ -51,6 +51,7 @@ nftables.txt: Anonymized nftables rules with packet counters across all families
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
config.txt: Anonymized configuration information of the NetBird client.
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
@@ -237,6 +238,13 @@ scutil_dns.txt (macOS only):
- Shows DNS configuration for all network interfaces
- Includes search domains, nameservers, and DNS resolver settings
- All IP addresses and domain names are anonymized
dns_windows.txt (Windows only):
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
- All IP addresses and domain names are anonymized
`
const (

View File

@@ -1,4 +1,4 @@
//go:build !unix
//go:build !unix && !windows
package debug

View File

@@ -0,0 +1,443 @@
//go:build windows
package debug
import (
"encoding/hex"
"errors"
"fmt"
"net/netip"
"strings"
"unsafe"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
nbdns "github.com/netbirdio/netbird/client/internal/dns"
)
const dnsInfoFileName = "dns_windows.txt"
const (
gpoDNSClientRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient`
tcpipParamsPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters`
dnscacheParams = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters`
)
// interfaceDNSValues are the per-interface values that decide how a name is
// resolved and registered. Everything the DNS host manager writes is in here,
// so a bundle shows both what we set and what it replaced.
var interfaceDNSValues = []string{
"NameServer",
"DhcpNameServer",
"Domain",
"DhcpDomain",
"SearchList",
"RegistrationEnabled",
"DisableDynamicUpdate",
"MaxNumberOfAddressesToRegister",
"EnableDHCP",
}
// addDNSInfo collects and adds DNS configuration information to the archive
func (g *BundleGenerator) addDNSInfo() error {
if err := g.addFileToZip(strings.NewReader(g.collectDNSInfo()), dnsInfoFileName); err != nil {
return fmt.Errorf("add DNS info to zip: %w", err)
}
return nil
}
// collectDNSInfo renders the report. Everything below it reaches the platform
// through COM and through lazily resolved procedures, which panic when a
// procedure is missing rather than returning an error, and a debug bundle is not
// allowed to take the daemon down. The panic is contained here, and whatever was
// collected before it is kept and reported with it.
func (g *BundleGenerator) collectDNSInfo() (content string) {
var sb strings.Builder
defer func() {
if r := recover(); r != nil {
log.Errorf("collecting Windows DNS configuration panicked: %v", r)
fmt.Fprintf(&sb, "\nerror: collection stopped: %v\n", r)
}
content = sb.String()
}()
sb.WriteString("Windows DNS configuration\n")
sb.WriteString("=========================\n")
adapters, adaptersErr := adapterAddresses()
g.writeNRPTRules(&sb, "NRPT rules, local policy store", nbdns.DNSPolicyConfigRoot)
g.writeNRPTRules(&sb, "NRPT rules, group policy store", nbdns.GPODNSPolicyConfigRoot)
g.writeEffectiveNRPTPolicies(&sb)
g.writeRegistryKey(&sb, "DNS client group policy", gpoDNSClientRoot)
g.writeRegistryKey(&sb, "Global TCP/IP parameters", tcpipParamsPath)
g.writeRegistryKey(&sb, "Dnscache parameters", dnscacheParams)
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv4", nbdns.InterfaceConfigPath, adapterNames(adapters))
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv6", nbdns.InterfaceConfigPathV6, adapterNames(adapters))
g.writeAdapterDNS(&sb, adapters, adaptersErr)
return sb.String()
}
// writeNRPTRules lists every rule in a policy store, ours and any other
// product's, since a foreign rule for the same namespace decides resolution
// just as ours does. Rules the client wrote are marked.
func (g *BundleGenerator) writeNRPTRules(sb *strings.Builder, title, root string) {
writeSection(sb, title, root)
names, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(names) == 0 {
sb.WriteString("no rules\n")
return
}
for _, name := range names {
owner := ""
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(nbdns.NRPTKeyPrefix)) {
owner = " (netbird)"
}
fmt.Fprintf(sb, "%s%s\n", name, owner)
g.writeValues(sb, root+`\`+name, nil, " ")
}
}
// writeEffectiveNRPTPolicies reports the table the resolver answers from, which
// the registry cannot show: a rule is written before it is loaded, and it keeps
// being enforced after its key is gone until the resolver reloads its policy.
func (g *BundleGenerator) writeEffectiveNRPTPolicies(sb *strings.Builder) {
writeSection(sb, "NRPT policy table in effect", nrptPolicyClass+"."+nrptPolicyMethod+" in "+nrptPolicyNamespace)
entries, err := effectiveNRPTPolicies()
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(entries) == 0 {
sb.WriteString("no policies\n")
return
}
for _, entry := range entries {
fmt.Fprintf(sb, "%s\n", g.anonymizeValue("Namespace", entry.namespace))
for _, value := range entry.values {
fmt.Fprintf(sb, " %s: %s\n", value.name, g.anonymizeValue(value.name, value.value))
}
}
}
// writeInterfaceDNS reports the DNS values of every interface that has any, so
// the netbird interface can be compared against the physical ones. The registry
// keys the values by GUID, so each is named from the adapter list; a GUID with
// no adapter is a leftover key of an interface that no longer exists.
func (g *BundleGenerator) writeInterfaceDNS(sb *strings.Builder, title, root string, names map[string]string) {
writeSection(sb, title, root)
guids, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
var reported int
for _, guid := range guids {
var iface strings.Builder
g.writeValues(&iface, root+`\`+guid, interfaceDNSValues, " ")
if iface.Len() == 0 {
continue
}
name, ok := names[strings.ToLower(guid)]
if !ok {
name = "no adapter with this GUID"
}
reported++
fmt.Fprintf(sb, "%s (%s)\n%s", guid, name, iface.String())
}
if reported == 0 {
sb.WriteString("no interface holds DNS values\n")
}
}
// writeRegistryKey reports the values of a single key, without its subkeys.
func (g *BundleGenerator) writeRegistryKey(sb *strings.Builder, title, path string) {
writeSection(sb, title, path)
var values strings.Builder
g.writeValues(&values, path, nil, "")
if values.Len() == 0 {
sb.WriteString("no values\n")
return
}
sb.WriteString(values.String())
}
// writeValues renders the values of a key. A nil names list reports every
// value, otherwise only those named and present.
func (g *BundleGenerator) writeValues(sb *strings.Builder, path string, names []string, indent string) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, path, registry.QUERY_VALUE)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, windows.ERROR_PATH_NOT_FOUND):
// an absent key is the normal state for the GPO store and for
// interfaces without DNS settings
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", path)
return
case err != nil:
fmt.Fprintf(sb, "%serror: open HKEY_LOCAL_MACHINE\\%s: %v\n", indent, path, err)
return
}
defer closeKey(k)
if names == nil {
names, err = k.ReadValueNames(-1)
if err != nil {
fmt.Fprintf(sb, "%serror: read value names: %v\n", indent, err)
return
}
}
for _, name := range names {
value, err := readRegistryValue(k, name)
switch {
case errors.Is(err, registry.ErrNotExist):
// the caller asks for a fixed set of values, most of which a
// given interface does not carry
continue
case err != nil:
// report rather than omit: a value that is there but cannot be
// read reads as unset otherwise
fmt.Fprintf(sb, "%s%s: error: %v\n", indent, name, err)
continue
}
fmt.Fprintf(sb, "%s%s: %s\n", indent, name, g.anonymizeValue(name, value))
}
}
// anonymizeValue redacts a registry value according to what its name says it
// holds. Domains and addresses are handled per entry rather than by the string
// pass: the pass only replaces domains something else in the bundle already
// seeded, and its address regex would eat the digit labels of a reverse zone.
func (g *BundleGenerator) anonymizeValue(name, value string) string {
if !g.anonymize || value == "" {
return value
}
switch {
case holdsDomains(name):
return joinValueEntries(splitValueEntries(value), g.anonymizeDomain)
case holdsAddresses(name):
return joinValueEntries(splitValueEntries(value), g.anonymizer.AnonymizeIPString)
default:
return g.anonymizer.AnonymizeString(value)
}
}
// holdsDomains reports whether a value name holds domains: the domain list of
// an NRPT rule (Name) or of the policy table (Namespace), a search list, the
// DNS suffix values of the TCP/IP and policy keys, which all end in "Domain"
// (Domain, DhcpDomain, NV Domain, ICSDomain), and a proxy host name.
func holdsDomains(name string) bool {
lower := strings.ToLower(name)
return lower == "name" || lower == "namespace" || lower == "searchlist" ||
strings.HasSuffix(lower, "domain") || strings.HasSuffix(lower, "proxyname")
}
// holdsAddresses reports whether a value name holds DNS server addresses
// (NameServer, DhcpNameServer, GenericDNSServers, NameServers).
func holdsAddresses(name string) bool {
lower := strings.ToLower(name)
return strings.Contains(lower, "nameserver") || strings.Contains(lower, "dnsserver")
}
// adapterNames maps adapter GUIDs, as the registry keys the interfaces, to the
// names an operator sees.
func adapterNames(adapters []*windows.IpAdapterAddresses) map[string]string {
names := make(map[string]string, len(adapters))
for _, adapter := range adapters {
guid := windows.BytePtrToString(adapter.AdapterName)
names[strings.ToLower(guid)] = windows.UTF16PtrToString(adapter.FriendlyName)
}
return names
}
// writeAdapterDNS reports the resolver configuration in effect per adapter,
// which is what the resolver uses for a name no NRPT rule matches.
func (g *BundleGenerator) writeAdapterDNS(sb *strings.Builder, adapters []*windows.IpAdapterAddresses, err error) {
writeSection(sb, "Adapter DNS configuration", "GetAdaptersAddresses")
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
for _, adapter := range adapters {
name := windows.UTF16PtrToString(adapter.FriendlyName)
suffix := g.anonymizeDomain(windows.UTF16PtrToString(adapter.DnsSuffix))
fmt.Fprintf(sb, "%s (index %d, oper status %d)\n", name, adapter.IfIndex, adapter.OperStatus)
fmt.Fprintf(sb, " DNS suffix: %s\n", suffix)
var servers []string
for server := adapter.FirstDnsServerAddress; server != nil; server = server.Next {
addr, ok := netip.AddrFromSlice(server.Address.IP())
if !ok {
continue
}
addr = addr.Unmap()
if g.anonymize {
addr = g.anonymizer.AnonymizeIP(addr)
}
servers = append(servers, addr.String())
}
fmt.Fprintf(sb, " DNS servers: %s\n", strings.Join(servers, ", "))
}
}
// anonymizeDomain anonymizes a single domain, keeping the leading dot an NRPT
// match domain carries.
func (g *BundleGenerator) anonymizeDomain(entry string) string {
if !g.anonymize {
return entry
}
domain, dot := strings.CutPrefix(entry, ".")
if domain == "" {
return entry
}
anonymized := g.anonymizer.AnonymizeDomain(domain)
if dot {
anonymized = "." + anonymized
}
return anonymized
}
// splitValueEntries splits a registry value that holds a list. The separator
// differs per value: a REG_MULTI_SZ arrives joined with ", ", a SearchList is
// comma separated and a NameServer may use commas or spaces.
func splitValueEntries(value string) []string {
return strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == ';' || r == ' ' || r == '\t'
})
}
func joinValueEntries(entries []string, anonymize func(string) string) string {
for i, entry := range entries {
entries[i] = anonymize(entry)
}
return strings.Join(entries, ", ")
}
func writeSection(sb *strings.Builder, title, source string) {
fmt.Fprintf(sb, "\n%s\n%s\n%s\n", title, strings.Repeat("-", len(title)), source)
}
func subKeyNames(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
if err != nil {
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closeKey(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
return names, nil
}
// readRegistryValue renders a value as text regardless of its type, so an
// unexpected type in a policy key still shows up instead of being dropped.
func readRegistryValue(k registry.Key, name string) (string, error) {
_, valueType, err := k.GetValue(name, nil)
if err != nil {
return "", fmt.Errorf("get value %s: %w", name, err)
}
switch valueType {
case registry.SZ, registry.EXPAND_SZ:
value, _, err := k.GetStringValue(name)
if err != nil {
return "", fmt.Errorf("get string value %s: %w", name, err)
}
return value, nil
case registry.MULTI_SZ:
values, _, err := k.GetStringsValue(name)
if err != nil {
return "", fmt.Errorf("get strings value %s: %w", name, err)
}
return strings.Join(values, ", "), nil
case registry.DWORD, registry.QWORD:
value, _, err := k.GetIntegerValue(name)
if err != nil {
return "", fmt.Errorf("get integer value %s: %w", name, err)
}
return fmt.Sprintf("%d (0x%x)", value, value), nil
case registry.BINARY:
value, _, err := k.GetBinaryValue(name)
if err != nil {
return "", fmt.Errorf("get binary value %s: %w", name, err)
}
return hex.EncodeToString(value), nil
default:
return fmt.Sprintf("<unhandled registry type %d>", valueType), nil
}
}
// adapterAddresses returns the adapter list including DNS servers. The call
// reports the size it needs, so grow the buffer and retry until it fits.
func adapterAddresses() (adapters []*windows.IpAdapterAddresses, err error) {
// GetAdaptersAddresses is resolved on first use and panics when it is
// missing, so this reports it as an error and leaves the rest of the
// report intact.
defer func() {
if r := recover(); r != nil {
adapters, err = nil, fmt.Errorf("GetAdaptersAddresses: %v", r)
}
}()
const flags = windows.GAA_FLAG_SKIP_ANYCAST | windows.GAA_FLAG_SKIP_MULTICAST
size := uint32(15000)
for range 3 {
buf := make([]byte, size)
first := (*windows.IpAdapterAddresses)(unsafe.Pointer(&buf[0]))
err := windows.GetAdaptersAddresses(windows.AF_UNSPEC, flags, 0, first, &size)
if errors.Is(err, windows.ERROR_BUFFER_OVERFLOW) {
continue
}
if err != nil {
return nil, fmt.Errorf("GetAdaptersAddresses: %w", err)
}
for adapter := first; adapter != nil; adapter = adapter.Next {
adapters = append(adapters, adapter)
}
return adapters, nil
}
return nil, fmt.Errorf("GetAdaptersAddresses: buffer kept growing")
}
func closeKey(k registry.Key) {
if err := k.Close(); err != nil {
log.Debugf("close registry key: %v", err)
}
}

View File

@@ -0,0 +1,146 @@
//go:build windows
package debug
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/anonymize"
)
func newDNSValueGenerator(level anonymize.Level) *BundleGenerator {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(level)
return &BundleGenerator{
anonymize: true,
anonymizeLevel: level,
anonymizer: anonymizer,
}
}
// TestAnonymizeValueByName covers the value kinds of the DNS registry keys. The
// names decide the treatment, because the string pass alone replaces only
// domains another part of the bundle already seeded.
func TestAnonymizeValueByName(t *testing.T) {
tests := []struct {
name string
valueName string
value string
assert func(t *testing.T, got string)
}{
{
name: "NRPT match domains keep the leading dot",
valueName: "Name",
value: ".internal.example.com, .corp.example.org",
assert: func(t *testing.T, got string) {
t.Helper()
for _, entry := range strings.Split(got, ", ") {
assert.True(t, strings.HasPrefix(entry, "."), "entry %q should keep its leading dot", entry)
assert.NotContains(t, entry, "example", "entry %q should not keep the original domain", entry)
}
},
},
{
name: "any value name ending in Domain is treated as a domain",
valueName: "ICSDomain",
value: "mshome.net",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "mshome", "should anonymize a domain suffix value")
},
},
{
name: "search list is a comma separated domain list",
valueName: "SearchList",
value: "corp.example.com,branch.example.com",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "example", "should anonymize every search domain")
assert.Len(t, strings.Split(got, ", "), 2, "should keep both search domains")
},
},
{
name: "name servers are anonymized as addresses",
valueName: "DhcpNameServer",
value: "203.0.113.10 8.8.8.8",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "203.0.113.10", "should anonymize a public resolver address")
// well-known resolvers stay readable at every level
assert.Contains(t, got, "8.8.8.8", "should keep a well-known resolver address")
},
},
{
name: "opaque values are left to the string pass",
valueName: "DataBasePath",
value: `%SystemRoot%\System32\drivers\etc`,
assert: func(t *testing.T, got string) {
t.Helper()
assert.Equal(t, `%SystemRoot%\System32\drivers\etc`, got, "should not alter a path")
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
g := newDNSValueGenerator(anonymize.LevelDefault)
tc.assert(t, g.anonymizeValue(tc.valueName, tc.value))
})
}
}
// TestParseNRPTPolicyTable parses the MOF text of the policy table out
// parameters, as the provider on a client with one NRPT rule renders it.
func TestParseNRPTPolicyTable(t *testing.T) {
const text = `[abstract]
class __PARAMETERS
{
[Out, EmbeddedInstance("DnsClientPolicyConfiguration"): ToSubClass, ID(2): DisableOverride ToInstance] DnsClientPolicyConfiguration cmdletOutput[] = {
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
DirectAccessQueryIPsecRequired = FALSE;
NameEncoding = "Utf8WithoutMapping";
Namespace = ".0.100.in-addr.arpa";
},
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
NameEncoding = "Utf8WithoutMapping";
NameServers = {"100.0.255.254", "100.0.255.253"};
Namespace = ".nb.internal";
}};
[in] boolean Effective;
[out] uint32 ReturnValue = 0;
};
`
entries := parseNRPTPolicyTable(text)
require.Len(t, entries, 2, "should parse both embedded instances")
assert.Equal(t, ".0.100.in-addr.arpa", entries[0].namespace, "should read the namespace of the first instance")
assert.Equal(t, ".nb.internal", entries[1].namespace, "should read the namespace of the second instance")
assert.Equal(t, []registryValue{
{name: "DirectAccessProxyType", value: "NoProxy"},
{name: "DirectAccessQueryIPsecRequired", value: "FALSE"},
{name: "NameEncoding", value: "Utf8WithoutMapping"},
}, entries[0].values, "should keep the remaining values in order")
assert.Contains(t, entries[1].values, registryValue{name: "NameServers", value: "100.0.255.254, 100.0.255.253"},
"should flatten a MOF array")
for _, value := range entries[1].values {
assert.NotContains(t, value.name, "ReturnValue", "should not read the class level parameters as values")
}
}
func TestParseNRPTPolicyTableEmpty(t *testing.T) {
assert.Empty(t, parseNRPTPolicyTable(""), "should parse no entries from empty text")
assert.Empty(t, parseNRPTPolicyTable("class __PARAMETERS\n{\n};\n"), "should parse no entries from a table with no instances")
}

View File

@@ -0,0 +1,317 @@
//go:build windows
package debug
import (
"errors"
"fmt"
"runtime"
"strings"
"time"
"github.com/go-ole/go-ole"
"github.com/go-ole/go-ole/oleutil"
log "github.com/sirupsen/logrus"
)
const (
// The NRPT policy table is reachable through the CIM class that backs
// Get-DnsClientNrptPolicy. Unlike the rules in the registry, the table is
// what the resolver currently has loaded, which is the only way to tell an
// applied rule from one that is merely written, in either direction.
nrptPolicyNamespace = `root\Microsoft\Windows\DNS`
nrptPolicyClass = "PS_DnsClientNrptPolicy"
nrptPolicyMethod = "Get"
// The class has no instances, so the table comes from the out parameters
// of a static method call, rendered as MOF text: the embedded instances
// arrive as a safe array of objects, which cannot be read back through the
// COM bindings, and the text form carries all of them.
nrptPolicyInstanceKeyword = "instance of DnsClientPolicyConfiguration"
nrptPolicyTimeout = 15 * time.Second
)
// COM initialization results that leave the calling thread usable: S_FALSE for
// a thread this process already initialized, RPC_E_CHANGED_MODE for one that
// belongs to another apartment.
const (
sFalse = 0x00000001
rpcEChangedMode = 0x80010106
)
// nrptQueryInFlight admits one read of the policy table at a time. A provider
// that stops answering keeps its goroutine and the OS thread that goroutine
// pinned, so a later bundle reports that instead of pinning another one.
var nrptQueryInFlight = make(chan struct{}, 1)
// nrptPolicyEntry is one namespace of the effective policy table, holding the
// values of an embedded DnsClientPolicyConfiguration instance in the order the
// provider reported them.
type nrptPolicyEntry struct {
namespace string
values []registryValue
}
// registryValue is a name and its rendered value, shared by the registry and
// policy table readers so both anonymize by value name the same way.
type registryValue struct {
name string
value string
}
// effectiveNRPTPolicies reads the effective NRPT table. The call is bounded
// because a WMI provider can block indefinitely and a debug bundle must not.
func effectiveNRPTPolicies() ([]nrptPolicyEntry, error) {
type result struct {
text string
err error
}
select {
case nrptQueryInFlight <- struct{}{}:
default:
return nil, errors.New("an earlier read of the policy table has not returned")
}
done := make(chan result, 1)
go func() {
// the slot is released here rather than by the caller, so a read that
// outlives the timeout holds it until the provider answers
defer func() { <-nrptQueryInFlight }()
text, err := nrptPolicyTableText()
done <- result{text: text, err: err}
}()
select {
case res := <-done:
if res.err != nil {
return nil, res.err
}
return parseNRPTPolicyTable(res.text), nil
case <-time.After(nrptPolicyTimeout):
return nil, errors.New("read of the policy table timed out")
}
}
// nrptPolicyTableText calls the policy table method and returns the MOF text of
// its out parameters.
func nrptPolicyTableText() (text string, err error) {
// COM is per thread, and the collection is short lived, so the thread is
// pinned for the duration rather than initialized for the process.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
defer func() {
// The COM call chain is dynamically typed, so a provider that answers
// with an unexpected shape must not take the daemon down with it.
if r := recover(); r != nil {
err = fmt.Errorf("read NRPT policy table: %v", r)
}
}()
owns, err := coInitialize()
if err != nil {
return "", err
}
if owns {
defer ole.CoUninitialize()
}
locator, err := oleutil.CreateObject("WbemScripting.SWbemLocator")
if err != nil {
return "", fmt.Errorf("create WMI locator: %w", err)
}
defer locator.Release()
dispatch, err := locator.QueryInterface(ole.IID_IDispatch)
if err != nil {
return "", fmt.Errorf("query WMI locator interface: %w", err)
}
defer dispatch.Release()
service, err := dispatchCall(dispatch, "ConnectServer", nil, nrptPolicyNamespace)
if err != nil {
return "", fmt.Errorf("connect to %s: %w", nrptPolicyNamespace, err)
}
defer service.Release()
inParams, err := spawnMethodInParams(service)
if err != nil {
return "", err
}
defer inParams.Release()
// The effective table is the merge of the local and the group policy
// store, which is what the resolver answers from.
if _, err := oleutil.PutProperty(inParams, "Effective", true); err != nil {
return "", fmt.Errorf("set Effective parameter: %w", err)
}
outParams, err := dispatchCall(service, "ExecMethod", nrptPolicyClass, nrptPolicyMethod, inParams)
if err != nil {
return "", fmt.Errorf("call %s.%s: %w", nrptPolicyClass, nrptPolicyMethod, err)
}
defer outParams.Release()
textVariant, err := oleutil.CallMethod(outParams, "GetObjectText_")
if err != nil {
return "", fmt.Errorf("render policy table: %w", err)
}
defer func() {
if err := textVariant.Clear(); err != nil {
log.Debugf("clear policy table variant: %v", err)
}
}()
return textVariant.ToString(), nil
}
// spawnMethodInParams builds the in parameters instance the method needs. The
// provider rejects the call without one, even when every parameter is optional.
func spawnMethodInParams(service *ole.IDispatch) (*ole.IDispatch, error) {
class, err := dispatchCall(service, "Get", nrptPolicyClass)
if err != nil {
return nil, fmt.Errorf("get class %s: %w", nrptPolicyClass, err)
}
defer class.Release()
methods, err := dispatchProperty(class, "Methods_")
if err != nil {
return nil, fmt.Errorf("get class methods: %w", err)
}
defer methods.Release()
method, err := dispatchCall(methods, "Item", nrptPolicyMethod)
if err != nil {
return nil, fmt.Errorf("get method %s: %w", nrptPolicyMethod, err)
}
defer method.Release()
params, err := dispatchProperty(method, "InParameters")
if err != nil {
return nil, fmt.Errorf("get method parameters: %w", err)
}
defer params.Release()
inParams, err := dispatchCall(params, "SpawnInstance_")
if err != nil {
return nil, fmt.Errorf("spawn parameter instance: %w", err)
}
return inParams, nil
}
// parseNRPTPolicyTable pulls the embedded instances out of the MOF text. Each
// instance is a namespace of the table, with one name and value per line.
func parseNRPTPolicyTable(text string) []nrptPolicyEntry {
var entries []nrptPolicyEntry
var current *nrptPolicyEntry
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(line), ";"))
switch {
case strings.HasPrefix(line, nrptPolicyInstanceKeyword):
entries = append(entries, nrptPolicyEntry{})
current = &entries[len(entries)-1]
continue
case strings.HasPrefix(line, "}"):
// closes an instance, and the array with the last one, so the
// class level parameters that follow are not read as values
current = nil
continue
case current == nil, line == "{":
continue
}
name, value, ok := strings.Cut(line, " = ")
if !ok {
continue
}
value = unquoteMOFValue(value)
if name == "Namespace" {
current.namespace = value
continue
}
current.values = append(current.values, registryValue{name: name, value: value})
}
return entries
}
// unquoteMOFValue renders a MOF scalar or array as plain text: "a" becomes a,
// and {"a", "b"} becomes a, b.
func unquoteMOFValue(value string) string {
value = strings.TrimSpace(value)
if inner, ok := strings.CutPrefix(value, "{"); ok {
value = strings.TrimSuffix(inner, "}")
entries := strings.Split(value, ",")
for i, entry := range entries {
entries[i] = strings.Trim(strings.TrimSpace(entry), `"`)
}
return strings.Join(entries, ", ")
}
return strings.Trim(value, `"`)
}
// coInitialize prepares the calling thread for COM and reports whether this
// call owns the initialization, which decides whether it may be balanced with
// CoUninitialize. S_FALSE took a reference on a thread this process had already
// initialized and so has to be released, while RPC_E_CHANGED_MODE took none:
// the thread belongs to another apartment, which is usable but is not ours to
// uninitialize.
func coInitialize() (bool, error) {
err := ole.CoInitializeEx(0, ole.COINIT_MULTITHREADED)
if err == nil {
return true, nil
}
var oleErr *ole.OleError
if errors.As(err, &oleErr) {
switch oleErr.Code() {
case sFalse:
return true, nil
case rpcEChangedMode:
return false, nil
}
}
return false, fmt.Errorf("initialize COM: %w", err)
}
// dispatchCall calls a COM method that returns an object.
func dispatchCall(dispatch *ole.IDispatch, method string, params ...any) (*ole.IDispatch, error) {
variant, err := oleutil.CallMethod(dispatch, method, params...)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("%s returned no object", method)
}
return object, nil
}
// dispatchProperty reads a COM property that holds an object.
func dispatchProperty(dispatch *ole.IDispatch, property string) (*ole.IDispatch, error) {
variant, err := oleutil.GetProperty(dispatch, property)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("property %s holds no object", property)
}
return object, nil
}

View File

@@ -31,10 +31,28 @@ var (
dnsFlushResolverCacheFn = dnsapi.NewProc("DnsFlushResolverCache")
)
// Registry locations of the host DNS configuration this package programs,
// exported so a diagnostic reader reports the same locations that are written.
const (
dnsPolicyConfigMatchPath = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig\NetBird-Match`
gpoDnsPolicyRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
gpoDnsPolicyConfigMatchPath = gpoDnsPolicyRoot + `\NetBird-Match`
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
NRPTKeyPrefix = "NetBird-Match"
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
DNSPolicyConfigRoot = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig`
// GPODNSPolicyConfigRoot holds the NRPT rules of the group policy store,
// which takes precedence over the local one when it is present.
GPODNSPolicyConfigRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
// InterfaceConfigPath and InterfaceConfigPathV6 hold the per-interface DNS
// settings, keyed by interface GUID, in separate hives per address family.
InterfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
InterfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
)
const (
dnsPolicyConfigMatchPath = DNSPolicyConfigRoot + `\` + NRPTKeyPrefix
gpoDnsPolicyConfigMatchPath = GPODNSPolicyConfigRoot + `\` + NRPTKeyPrefix
dnsPolicyConfigVersionKey = "Version"
dnsPolicyConfigVersionValue = 2
@@ -45,8 +63,6 @@ const (
nrptMaxDomainsPerRule = 50
interfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
interfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
interfaceConfigNameServerKey = "NameServer"
interfaceConfigDhcpNameSrvKey = "DhcpNameServer"
interfaceConfigSearchListKey = "SearchList"
@@ -84,7 +100,7 @@ func newHostManager(wgInterface WGIface) (*registryConfigurator, error) {
}
var useGPO bool
k, err := registry.OpenKey(registry.LOCAL_MACHINE, gpoDnsPolicyRoot, registry.QUERY_VALUE)
k, err := registry.OpenKey(registry.LOCAL_MACHINE, GPODNSPolicyConfigRoot, registry.QUERY_VALUE)
if err != nil {
log.Debugf("failed to open GPO DNS policy root: %v", err)
} else {
@@ -123,7 +139,7 @@ func (r *registryConfigurator) captureOriginalNameservers() ([]netip.Addr, error
seen := make(map[netip.Addr]struct{})
var out []netip.Addr
var merr *multierror.Error
for _, root := range []string{interfaceConfigPath, interfaceConfigPathV6} {
for _, root := range []string{InterfaceConfigPath, InterfaceConfigPathV6} {
addrs, err := r.captureFromTcpipRoot(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("%s: %w", root, err))
@@ -496,7 +512,7 @@ func (r *registryConfigurator) deleteInterfaceRegistryKeyProperty(propertyKey st
}
func (r *registryConfigurator) getInterfaceRegistryKey() (registry.Key, error) {
regKeyPath := interfaceConfigPath + "\\" + r.guid
regKeyPath := InterfaceConfigPath + "\\" + r.guid
regKey, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.SET_VALUE)
if err != nil {
return regKey, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", regKeyPath, err)

View File

@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -0,0 +1,82 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
}
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
return candidates[i].routeMetric < candidates[j].routeMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest route metric
// 3. Lowest interface metric
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -5,6 +5,7 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -5628,9 +5628,13 @@ func (x *GetPeerSSHHostKeyResponse) GetFound() bool {
type RequestJWTAuthRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// hint for OIDC login_hint parameter (typically email address)
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestJWTAuthRequest) Reset() {
@@ -5670,6 +5674,13 @@ func (x *RequestJWTAuthRequest) GetHint() string {
return ""
}
func (x *RequestJWTAuthRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestJWTAuthResponse contains authentication flow information
type RequestJWTAuthResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
@@ -5894,9 +5905,13 @@ type RequestExtendAuthSessionRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestExtendAuthSessionRequest) Reset() {
@@ -5936,6 +5951,13 @@ func (x *RequestExtendAuthSessionRequest) GetHint() string {
return ""
}
func (x *RequestExtendAuthSessionRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestExtendAuthSessionResponse carries the verification URI the UI
// should open in a browser. The daemon retains the flow state and resolves
// it via WaitExtendAuthSession.
@@ -7503,9 +7525,10 @@ const file_daemon_proto_rawDesc = "" +
"sshHostKey\x12\x16\n" +
"\x06peerIP\x18\x02 \x01(\tR\x06peerIP\x12\x1a\n" +
"\bpeerFQDN\x18\x03 \x01(\tR\bpeerFQDN\x12\x14\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"9\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"k\n" +
"\x15RequestJWTAuthRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\x9a\x02\n" +
"\x16RequestJWTAuthResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +
@@ -7525,9 +7548,10 @@ const file_daemon_proto_rawDesc = "" +
"\x14WaitJWTTokenResponse\x12\x14\n" +
"\x05token\x18\x01 \x01(\tR\x05token\x12\x1c\n" +
"\ttokenType\x18\x02 \x01(\tR\ttokenType\x12\x1c\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"C\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"u\n" +
"\x1fRequestExtendAuthSessionRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\xe0\x01\n" +
" RequestExtendAuthSessionResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +

View File

@@ -894,6 +894,10 @@ message GetPeerSSHHostKeyResponse {
message RequestJWTAuthRequest {
// hint for OIDC login_hint parameter (typically email address)
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestJWTAuthResponse contains authentication flow information
@@ -937,6 +941,10 @@ message RequestExtendAuthSessionRequest {
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestExtendAuthSessionResponse carries the verification URI the UI

View File

@@ -1723,8 +1723,8 @@ func (s *Server) RequestJWTAuth(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -1827,8 +1827,8 @@ func (s *Server) RequestExtendAuthSession(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -2000,13 +2000,6 @@ func (s *Server) ExposeService(req *proto.ExposeServiceRequest, srv proto.Daemon
return nil
}
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func (s *Server) runProbes(ctx context.Context, waitForProbeResult bool) {
if s.connectClient == nil {
return

View File

@@ -13,6 +13,7 @@ import (
"golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/util"
)
const (
@@ -92,7 +93,8 @@ func printAuthInstructions(stderr io.Writer, authResponse *proto.RequestJWTAuthR
// RequestJWTToken requests or retrieves a JWT token for SSH authentication
func RequestJWTToken(ctx context.Context, client proto.DaemonServiceClient, stdout, stderr io.Writer, useCache bool, hint string, openBrowser func(string) error) (string, error) {
req := &proto.RequestJWTAuthRequest{}
// the ssh client runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestJWTAuthRequest{HasGraphicalSession: util.HasGraphicalSession()}
if hint != "" {
req.Hint = &hint
}
@@ -193,4 +195,3 @@ func buildAddressList(hostname string, remote net.Addr) []string {
}
return addresses
}

View File

@@ -6,9 +6,11 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -58,10 +60,21 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
return ExtendStartResult{}, err
}
req := &proto.RequestExtendAuthSessionRequest{}
if p.Hint != "" {
h := p.Hint
req.Hint = &h
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -15,7 +15,8 @@
"lint": "eslint \"src/**/*.{ts,tsx}\"",
"lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix",
"check": "pnpm lint && pnpm typecheck && pnpm format:check",
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck"
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck",
"i18n:check": "node ../i18n/check-translations.mjs"
},
"dependencies": {
"@radix-ui/react-dialog": "^1.1.15",

View File

@@ -0,0 +1,104 @@
#!/usr/bin/env node
// Validates that every shipped translation bundle carries exactly the same set
// of keys as the English source of truth. English (en) defines the keys; every
// other locale declared in _index.json must match it 1:1:
//
// - no missing keys — a missing key silently falls back to English at runtime
// (see i18n bundle fallback), so the gap never surfaces to users or CI
// without this check;
// - no orphaned keys — keys left behind after an English key is renamed or
// removed are dead weight and a sign the locale is drifting.
//
// Pure Node, no dependencies, so it runs without installing the frontend
// toolchain.
//
// Local: node client/ui/i18n/check-translations.mjs (or: pnpm i18n:check)
// CI: .github/workflows/ui-translations.yml
import { readdirSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const SOURCE = "en";
const localesDir = join(dirname(fileURLToPath(import.meta.url)), "locales");
const isCI = Boolean(process.env.GITHUB_ACTIONS);
function readJSON(path) {
return JSON.parse(readFileSync(path, "utf8"));
}
function keysOf(langCode) {
return Object.keys(readJSON(join(localesDir, langCode, "common.json")));
}
// Emit a GitHub Actions annotation so failures render inline on the PR diff.
function annotate(file, message) {
if (isCI) console.log(`::error file=${file}::${message}`);
}
const index = readJSON(join(localesDir, "_index.json"));
const declared = index.languages.map((l) => l.code);
if (!declared.includes(SOURCE)) {
console.error(`FATAL: source language "${SOURCE}" is not declared in _index.json`);
process.exit(1);
}
const sourceKeys = keysOf(SOURCE);
const sourceSet = new Set(sourceKeys);
console.log(`Source of truth: ${SOURCE}/common.json — ${sourceKeys.length} keys\n`);
let failed = false;
for (const code of declared) {
if (code === SOURCE) continue;
const file = `client/ui/i18n/locales/${code}/common.json`;
let keys;
try {
keys = keysOf(code);
} catch (e) {
failed = true;
const msg = `bundle is declared in _index.json but common.json is missing or invalid (${e.message})`;
console.error(`${code}: ${msg}`);
annotate("client/ui/i18n/locales/_index.json", `${code}: ${msg}`);
continue;
}
const set = new Set(keys);
const missing = sourceKeys.filter((k) => !set.has(k));
const extra = keys.filter((k) => !sourceSet.has(k));
if (missing.length === 0 && extra.length === 0) {
console.log(`${code}: ${keys.length} keys`);
continue;
}
failed = true;
console.error(`${code}: ${keys.length} keys (expected ${sourceKeys.length})`);
if (missing.length) {
console.error(` missing ${missing.length}: ${missing.join(", ")}`);
annotate(file, `Missing ${missing.length} key(s) present in ${SOURCE}: ${missing.join(", ")}`);
}
if (extra.length) {
console.error(` extra ${extra.length}: ${extra.join(", ")}`);
annotate(file, `Has ${extra.length} key(s) not present in ${SOURCE}: ${extra.join(", ")}`);
}
}
// Locale directories present on disk but not declared in _index.json are never
// loaded by the app — surface them so dead translation files don't rot silently.
const onDisk = readdirSync(localesDir, { withFileTypes: true })
.filter((e) => e.isDirectory())
.map((e) => e.name);
const undeclared = onDisk.filter((d) => !declared.includes(d));
if (undeclared.length) {
console.warn(`\n⚠ locale directories not declared in _index.json (not shipped): ${undeclared.join(", ")}`);
}
console.log();
if (failed) {
console.error("Translation check FAILED — every locale must match the English key set.");
process.exit(1);
}
console.log("Translation check passed — all locales match the English key set.");

View File

@@ -1312,6 +1312,9 @@
"daemon.outdated.description": {
"message": "このアプリを使用するには NetBird サービスを更新してください。"
},
"daemon.outdated.download": {
"message": "最新版をダウンロード"
},
"error.jwt_clock_skew": {
"message": "サインインに失敗しました: このデバイスの時計がサーバーと同期していません。システムの時計を同期してからもう一度お試しください。"
},

View File

@@ -108,10 +108,11 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
}
req := &proto.LoginRequest{
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
IsUnixDesktopClient: runtime.GOOS == "linux",
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
// a login driven by the UI always has a graphical session available
IsUnixDesktopClient: true,
}
if profileName != "" {
req.ProfileName = ptrStr(profileName)
@@ -122,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
}
resp, err := cli.Login(ctx, req)
@@ -227,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -260,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Logout below).
// root and the per-profile state file is user-owned (see Profiles.List).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -115,7 +115,7 @@ sequenceDiagram
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)

View File

@@ -15,6 +15,10 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
| ---- | ---- |
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
@@ -48,6 +52,8 @@ flowchart TD
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
K2 --> P
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
@@ -60,6 +66,84 @@ flowchart TD
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### LLM pricing (management is the sole authority)
**The proxy carries no price list.** Management synthesizes the entire pricing
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
table and there is nothing to reload on the proxy side.
```mermaid
flowchart TD
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
B --> C{AgentNetwork.PricingDefaultsFile}
C -- absent --> D[compiled-in table serves]
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
E --> F[mergedTable atomic.Pointer]
D --> G[DefaultTable]
F --> G
G --> H[buildCostMeterConfigJSON — pricing.defaults]
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
K --> L[pricing.providers keyed by provider record ID]
H --> M[cost_meter ConfigJSON]
L --> M
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
```
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
id → rates. The **full** default table ships to every account: it is small
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
models, so they claim every model) priced.
- `pricing.providers` — provider **record** id → normalized model id → rates,
matched against the `llm.resolved_provider_id` the router stamps. Entries are
**fully materialized here**, at synth time: `materializeEntry` starts from the
default entry for that model so cache rates the operator didn't state are
inherited, overlays operator `input`/`output` verbatim (**including an explicit
0**, which prices a self-hosted or internal endpoint as free rather than
silently reverting to list price), and overlays cache-rate **pointers only when
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
discount, bill this bucket at the input rate". The proxy therefore does two map
lookups and no merging.
Same orphan rule as the router: a provider no enabled policy authorises is
unreachable, so its prices aren't shipped. Model ids are normalized with the
**same** functions the request parser uses (`NormalizeBedrockModel` /
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
first-occurrence-wins, matching the routing dedup order.
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
replace default rates without a rebuild. Schema is `surface → model → rates`
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
- An **explicitly configured** path is *required to load*: a typo or malformed
file fails startup, because the operator believes those rates are live. The
conventional probe is optional — an absent file just serves compiled-in
defaults, and the path stays watched in case it appears later.
- File entries **replace** the compiled-in entry for the same (surface, model)
**whole** — they are not field-merged, so an entry must repeat the cache rates
it wants to keep. Everything the file doesn't mention keeps built-in rates.
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
finite and non-negative — the same constraints the HTTP API enforces on
operator per-provider prices.
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
a parse error keeps the previous table, a deleted file reverts to compiled-in
defaults. A mid-edit save can never take pricing down.
The live table feeds **both** consumers, which is what keeps them consistent: the
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
golden-tested, so operators start from a file matching the built-in rates exactly.
### Budget rule resolution (min-wins, group+user bound)
```mermaid
@@ -124,7 +208,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | |
| on_response | 6 | `cost_meter` | `{}` | |
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
@@ -139,6 +223,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
## Things to scrutinize
@@ -176,10 +266,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
@@ -188,6 +280,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
## Test coverage
@@ -198,6 +291,9 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |

View File

@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing catalog this chain delegates to.
adapters + pricing table and cost formula this chain delegates to.
---
@@ -34,7 +34,7 @@ rewrites.
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:2640](../../../proxy/internal/middleware/builtin/all_test.go)
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
@@ -168,12 +168,46 @@ token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`).
**Management owns pricing.** The proxy carries no embedded price list: the whole
table arrives in this middleware's `ConfigJSON` as
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
plus the operator's stored per-provider prices
([factory.go:1334](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
construction, so a non-finite or negative rate fails the chain build. A price
change is an ordinary mapping push — the chain rebuild yields a fresh instance
over a fresh immutable table, so there is no data dir, no pricing file, no
reload goroutine, and nothing to invalidate.
**Two-tier lookup**
([middleware.go:165183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
1. **Per-provider-record** — the operator's stored price for the route that
actually served the request, keyed by the `llm.resolved_provider_id` that
`llm_router` stamped on the allow path, then by normalized model id. Entries
arrive fully materialized (management folds default cache rates in at synth
time), so there is no merging here. Absent metadata — no router in the chain
— skips this tier.
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
providers, which enumerate no models and therefore get no per-record entry.
**Backward compatibility:** a config with no `pricing` block means management
predates config-delivered pricing. The factory logs one warning at build time
and the instance records `cost.skipped=unknown_model` ($0) for every request
rather than falling back to a stale built-in price list
([factory.go:5560](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
(sibling doc) and is selected by the **surface**, not by which tier the entry
came from — `cost_meter` stays provider-agnostic, and a per-record override on
an Anthropic route still bills its cache buckets additively.
### llm_limit_record
@@ -246,12 +280,14 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time.
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
block carrying a negative or non-finite rate fails the build too, rather than
mispricing live traffic.
## Invariants
@@ -320,10 +356,11 @@ non-object `metadata` field
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
middleware config and never mutated, so the lookup path needs no lock or atomic
swap — a price change replaces the whole instance. Every middleware is
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
@@ -349,13 +386,13 @@ counter accuracy.
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing loader.
+ SSE framer + pricing table and cost formula.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing loader.
— the 8 middlewares that consume this package's parsers + pricing table.
---
@@ -24,8 +24,9 @@ proxy-framework dependencies:
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/`embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `pricing/`immutable pricing table + the per-surface cost formula. The
rates themselves come from management inside `cost_meter`'s middleware
config; this package holds no price list and reads no files.
- `fixtures/` — captured request/response/stream bodies the tests replay.
The package carries zero proxy-framework dependencies so the same parsers can
@@ -47,12 +48,9 @@ be reused later by a WASM adapter
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream |
## Request body → parser dispatch
@@ -188,9 +186,11 @@ response leg, covering both Bedrock body shapes:
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
management ships, keyed by the **normalised** model id (region prefix + version
suffix stripped by the request parser; management normalises its keys the same
way at synth time so the two compare equal). `ParseResponse` returns
`ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
@@ -205,11 +205,34 @@ response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing catalog
### Pricing table
`Table.Cost`
([pricing.go:129174](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
**Management is the sole pricing authority.** The proxy carries no embedded
price list and reads no pricing file: the whole table arrives inside
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
is just another push — the chain rebuild constructs a fresh `Table`, so there
is nothing to reload
([pricing.go:17](../../../proxy/internal/llm/pricing/pricing.go)). The
management side of the contract (catalog defaults, the operator's stored
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
management-side module guide; `cost_meter`'s wire shape is in
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
`EntryJSON`
([pricing.go:3645](../../../proxy/internal/llm/pricing/pricing.go)) is the
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
direct struct conversion rather than field-by-field copying (a new rate can't
be silently dropped in transit).
`EntryCosts`
([pricing.go:183234](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module. The
**surface** (the `llm.provider` value the request parser stamped) selects the
formula, never the tier the entry came from: a per-provider-record override on
an Anthropic route still bills its cache buckets additively.
| Provider | Formula |
|---|---|
@@ -218,7 +241,7 @@ is the cost formula — most security-relevant math in this module:
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
([pricing.go:214229](../../../proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
@@ -226,15 +249,12 @@ to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Loader`
([pricing.go:212268](../../../proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
`Costs`
([pricing.go:143163](../../../proxy/internal/llm/pricing/pricing.go)) is the
per-request split. The four per-bucket fields are the base; `TotalUSD` and
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
the breakdown. `InputUSD` is always the non-cached input bucket on both
provider shapes, so input and cached-input never double-count.
## Public contracts
@@ -264,29 +284,38 @@ Order matters: `DetectFromURL` ties resolve by registration order.
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` lookup**
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
**`Pricing` construction + lookup**
([pricing.go:60130](../../../proxy/internal/llm/pricing/pricing.go)):
```go
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
func (t *Table) Lookup(provider, model string) (Entry, bool)
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
```
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130132](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
reject any non-finite or negative rate, so a corrupt config fails the chain
build rather than mispricing silently. Nil input yields an empty,
never-matching table.
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
([pricing.go:9699](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means the surface or model is absent from the table management sent;
the caller emits `cost.skipped=unknown_model`.
## Invariants
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:1416](../../../proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
1. **The pricing package is pure and platform-independent.** No file I/O, no
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
there is nothing platform-specific left to port. Anything reintroducing a
read-from-disk path here re-splits pricing authority between management and
the proxy, which is exactly what this design removed.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
@@ -298,38 +327,45 @@ emits `cost.skipped=unknown_model`.
usage rather than aborting
([streaming.go:6873, 144150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:2930](../../../proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:4249](../../../proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
3. **Management is the only source of rates.** `Table` has no constructor that
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
management sent. A missing or empty `pricing` block therefore means *no
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
never a stale built-in fallback that would silently bill list price.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370394](../../../proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
4. **Tables are immutable once built.** `Table.entries` is written only in
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
map is likewise build-time-only
([pricing.go:4752](../../../proxy/internal/llm/pricing/pricing.go)). This
is what makes the no-reload design safe: a price change arrives as a mapping
push that builds a new middleware instance over a new table, so concurrent
readers can't observe a half-updated price list and no atomic swap or lock
is needed on the hot path.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:2557](../../../proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
5. **Rate validation happens at chain-build time, not per request.**
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
([pricing.go:6083](../../../proxy/internal/llm/pricing/pricing.go)), naming
the offending surface/model/field in the error. Management enforces the same
constraints at its API boundary and in its YAML parser, so this is
defense-in-depth — but it means a corrupt push fails loudly at build instead
of producing negative costs on live traffic. Test:
`TestNewTable_ValidatesRates`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397398](../../../proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
`pricing.Entry` together.** `NewEntries` converts by direct struct
conversion `Entry(e)`
([pricing.go:7678](../../../proxy/internal/llm/pricing/pricing.go)), which
only compiles while the two structs stay field-identical — so the proxy half
is compiler-enforced. The management half is not: a rate added there but not
here unmarshals into nothing and prices that bucket at `InputPer1K`.
## Things to scrutinise
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147149](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
**Correctness.** Verify the OpenAI cached-prompt clamp at
[pricing.go:203206](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. Negative token counts are clamped to zero up
front ([pricing.go:186197](../../../proxy/internal/llm/pricing/pricing.go)) so
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
@@ -338,22 +374,27 @@ noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
oversize surface is gone entirely — the config channel (an authenticated
mapping push from management) is now the only way rates enter the proxy, and
`NewEntries` is the validation boundary on it. A new rate added to management's
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
path (see invariant 6).
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Concurrency.** Nothing in this package is shared mutable state: tables are
built once and never written again, so `cost_meter`'s hot path is lock-free by
construction rather than by atomic swap. Per-call `Scanner` instances mean no
shared state across concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
`ReadString('\n')` per line. No background goroutines and no per-request
allocation of pricing state.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
**Observability.** A config carrying no `pricing` block logs one warning at
chain-build time (`cost_meter` factory) and then records
`cost.skipped=unknown_model` per request, so an old-management deployment is
visible in both logs and the access log rather than quietly reporting $0.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
@@ -365,7 +406,7 @@ right `cost.skipped` reason.
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
@@ -373,14 +414,15 @@ right `cost.skipped` reason.
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
No pricing fixture: the table is config-delivered, so pricing tests construct
it in-process from a wire-shape map.
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewLoader`.
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -1,7 +1,7 @@
# proxy/runtime — translate + serve + log
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
## Module boundary
@@ -114,8 +114,7 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
## Public contracts touched
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).

View File

@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic``llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
parser, priced under the **`anthropic`** surface of the pricing table
management ships (the parser emits the standard Anthropic provider label, so
Vertex Claude reuses first-party Anthropic prices).
- `openai``llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
@@ -104,8 +104,9 @@ is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
> those regions read low. Operators who need exact regional billing set the
> affected models' prices on the provider record, or replace the default entries
> via management's `AgentNetwork.PricingDefaultsFile`.
## AWS Bedrock (`bedrock_api`)
@@ -211,15 +212,19 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
`model_prices_and_context_window.json`. The **catalog is the source of default
prices**: management's `pricing.DefaultTable` folds every catalog provider's
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
is structural rather than maintained in a parallel file
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
providers contribute the same (surface, model) at different rates. Bedrock
entries are keyed by the **normalised** id the request parser emits (region
prefix + version suffix stripped) — management applies the same normalisation to
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
surface.
## Things to scrutinise
@@ -232,16 +237,17 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
pricing key — a model that normalises to something absent from the shipped
pricing table meters at `cost.skipped=unknown_model` rather than failing the
request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
~10% premium not modelled by base pricing — flagged in the catalog comment.
Operators needing exact regional billing set per-provider prices on the model
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
## Cross-references

View File

@@ -6,7 +6,7 @@
"name": "NetBird GmbH",
"email": "hello@netbird.io",
"phone": "",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open-source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open-source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"webpageUrl": {
"url": "https://github.com/netbirdio"
}
@@ -15,7 +15,7 @@
{
"guid": "netbird",
"name": "NetBird",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open-source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"webpageUrl": {
"url": "https://github.com/netbirdio/netbird"
},
@@ -59,7 +59,7 @@
"guid": "support-yearly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Yearly",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "yearly",
@@ -72,7 +72,7 @@
"guid": "support-one-time-year",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Year",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "one-time",
@@ -85,7 +85,7 @@
"guid": "support-one-time-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Monthly",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",
@@ -98,7 +98,7 @@
"guid": "support-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Month",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",

2
go.mod
View File

@@ -57,6 +57,7 @@ require (
github.com/fsnotify/fsnotify v1.9.0
github.com/gliderlabs/ssh v0.3.8
github.com/go-jose/go-jose/v4 v4.1.4
github.com/go-ole/go-ole v1.3.0
github.com/gobwas/ws v1.4.0
github.com/goccy/go-yaml v1.18.0
github.com/godbus/dbus/v5 v5.2.2
@@ -199,7 +200,6 @@ require (
github.com/go-ldap/ldap/v3 v3.4.13 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.2 // indirect
github.com/go-openapi/jsonpointer v0.21.1 // indirect

View File

@@ -111,6 +111,59 @@ check_nb_domain() {
return 0
}
# Non-interactive configuration
# ------------------------------
# Every prompt below can be pre-answered with an environment variable, so the
# script runs unattended (cloud-init, CI, Terraform, curl | bash). resolve()
# is the single place that decides env var vs prompt vs default; the read_*
# helpers stay pure prompts.
#
# Supported env vars:
# NETBIRD_DOMAIN domain/FQDN (required)
# NETBIRD_LETSENCRYPT_EMAIL ACME email (required for built-in Traefik)
# NETBIRD_AGENT_NETWORK true enables the agent-network preset
# NETBIRD_REVERSE_PROXY_TYPE 0-5 (default 0 = built-in Traefik)
# NETBIRD_ENABLE_PROXY true/false (default false)
# NETBIRD_ENABLE_CROWDSEC true/false (default false)
# NETBIRD_TRAEFIK_EXTERNAL_NETWORK external-Traefik network (type 1)
# NETBIRD_TRAEFIK_ENTRYPOINT external-Traefik entrypoint (type 1, default websecure)
# NETBIRD_TRAEFIK_CERTRESOLVER external-Traefik cert resolver (type 1)
# NETBIRD_BIND_LOCALHOST_ONLY true/false (default true, types 2-5)
# NETBIRD_EXTERNAL_PROXY_NETWORK docker network to join (types 2-4)
# NETBIRD_NON_INTERACTIVE true forces unattended mode even with a TTY
# tty_available succeeds only when we may prompt: never when the operator has
# set NETBIRD_NON_INTERACTIVE=true, otherwise only when /dev/tty can actually
# be opened. A PTY can be attached in automation (CI runners, some
# provisioners), so the env override is the authoritative signal and the
# /dev/tty probe is the fallback. /dev/tty is a world-rw device node even with
# no terminal, so a permission test ([ -r ]) is not enough - we must open it.
tty_available() {
[[ "${NETBIRD_NON_INTERACTIVE:-}" == "true" ]] && return 1
{ true < /dev/tty; } 2>/dev/null
}
# resolve ENV_VAR_NAME DEFAULT PROMPT_FN [prompt args...]
# env var set and non-empty -> its value
# interactive -> PROMPT_FN "$@" (prompt behavior unchanged)
# otherwise -> DEFAULT, or abort when DEFAULT is "required"
resolve() {
local env_name="$1" default="$2" prompt_fn="$3"
shift 3
local env_value="${!env_name:-}"
if [[ -n "$env_value" ]]; then
echo "$env_value"
elif tty_available; then
"$prompt_fn" "$@"
elif [[ "$default" == "required" ]]; then
echo "$env_name is required for a non-interactive install." > /dev/stderr
exit 1
else
echo "$default"
fi
return 0
}
read_nb_domain() {
READ_NETBIRD_DOMAIN=""
echo -n "Enter the domain you want to use for NetBird (e.g. netbird.my-domain.com): " > /dev/stderr
@@ -383,7 +436,14 @@ initialize_default_values() {
}
configure_domain() {
# Domain is validated (not a free-form value), so it keeps its own guard
# rather than going through resolve(): a valid NETBIRD_DOMAIN is used as-is,
# otherwise we prompt, or abort when there is no terminal to prompt on.
if ! check_nb_domain "$NETBIRD_DOMAIN"; then
if ! tty_available; then
echo "NETBIRD_DOMAIN is required for a non-interactive install." > /dev/stderr
exit 1
fi
NETBIRD_DOMAIN=$(read_nb_domain)
fi
@@ -411,11 +471,7 @@ apply_agent_network_preset() {
ENABLE_PROXY="true"
ENABLE_CROWDSEC="false"
if [[ -n "${NETBIRD_LETSENCRYPT_EMAIL}" ]]; then
TRAEFIK_ACME_EMAIL="${NETBIRD_LETSENCRYPT_EMAIL}"
else
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
fi
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
echo "" > /dev/stderr
echo "Agent-network preset enabled (NETBIRD_AGENT_NETWORK=true):" > /dev/stderr
@@ -437,35 +493,35 @@ configure_reverse_proxy() {
return 0
fi
# Prompt for reverse proxy type
REVERSE_PROXY_TYPE=$(read_reverse_proxy_type)
# Reverse proxy type (env NETBIRD_REVERSE_PROXY_TYPE, else prompt, else 0)
REVERSE_PROXY_TYPE=$(resolve NETBIRD_REVERSE_PROXY_TYPE 0 read_reverse_proxy_type)
# Handle built-in Traefik prompts (option 0)
if [[ "$REVERSE_PROXY_TYPE" == "0" ]]; then
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
ENABLE_PROXY=$(read_enable_proxy)
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
ENABLE_PROXY=$(resolve NETBIRD_ENABLE_PROXY false read_enable_proxy)
if [[ "$ENABLE_PROXY" == "true" ]]; then
ENABLE_CROWDSEC=$(read_enable_crowdsec)
ENABLE_CROWDSEC=$(resolve NETBIRD_ENABLE_CROWDSEC false read_enable_crowdsec)
fi
fi
# Handle external Traefik-specific prompts (option 1)
if [[ "$REVERSE_PROXY_TYPE" == "1" ]]; then
TRAEFIK_EXTERNAL_NETWORK=$(read_traefik_network)
TRAEFIK_ENTRYPOINT=$(read_traefik_entrypoint)
TRAEFIK_CERTRESOLVER=$(read_traefik_certresolver)
TRAEFIK_EXTERNAL_NETWORK=$(resolve NETBIRD_TRAEFIK_EXTERNAL_NETWORK "" read_traefik_network)
TRAEFIK_ENTRYPOINT=$(resolve NETBIRD_TRAEFIK_ENTRYPOINT websecure read_traefik_entrypoint)
TRAEFIK_CERTRESOLVER=$(resolve NETBIRD_TRAEFIK_CERTRESOLVER "" read_traefik_certresolver)
fi
# Handle port binding for external proxy options (2-5)
if [[ "$REVERSE_PROXY_TYPE" -ge 2 ]]; then
BIND_LOCALHOST_ONLY=$(read_port_binding_preference)
BIND_LOCALHOST_ONLY=$(resolve NETBIRD_BIND_LOCALHOST_ONLY true read_port_binding_preference)
fi
# Handle Docker network prompts for external proxies (options 2-4)
case "$REVERSE_PROXY_TYPE" in
2) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx") ;;
3) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx Proxy Manager") ;;
4) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Caddy") ;;
2) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx") ;;
3) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx Proxy Manager") ;;
4) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Caddy") ;;
*) ;; # No network prompt for other options
esac
return 0
@@ -643,8 +699,13 @@ start_services_and_show_instructions() {
print_post_setup_instructions
echo ""
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
read -r < /dev/tty
if tty_available; then
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
read -r < /dev/tty
else
echo "Non-interactive mode: starting NetBird containers now. Finish configuring"
echo "your reverse proxy using the instructions above so it can reach them."
fi
echo -e "$MSG_STARTING_SERVICES"
$DOCKER_COMPOSE_COMMAND up -d

View File

@@ -1433,13 +1433,14 @@ components:
enum: [ "all", "tcp", "udp", "icmp", "netbird-ssh" ]
example: "tcp"
ports:
description: Policy rule affected ports
description: Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
x-omit-from-example: true
type: array
items:
type: string
example: "80"
port_ranges:
description: Policy rule affected ports ranges list
description: Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
type: array
items:
$ref: '#/components/schemas/RulePortRange'
@@ -1459,7 +1460,7 @@ components:
- action
RulePortRange:
description: Policy rule affected ports range
description: Policy rule affected ports range. A range with identical start and end values represents a single port.
type: object
properties:
start:
@@ -4607,7 +4608,7 @@ components:
FleetDMMatchAttributes:
type: object
description: Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
description: Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
additionalProperties: false
properties:
disk_encryption_enabled:

View File

@@ -2876,7 +2876,7 @@ type EDRFleetDMRequest struct {
// LastSyncedInterval The devices last sync requirement interval in hours. Minimum value is 24 hours
LastSyncedInterval int `json:"last_synced_interval"`
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
MatchAttributes FleetDMMatchAttributes `json:"match_attributes"`
}
@@ -2909,7 +2909,7 @@ type EDRFleetDMResponse struct {
// LastSyncedInterval The devices last sync requirement interval in hours.
LastSyncedInterval int `json:"last_synced_interval"`
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
// MatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
MatchAttributes FleetDMMatchAttributes `json:"match_attributes"`
// UpdatedAt Timestamp of when the integration was last updated.
@@ -3129,7 +3129,7 @@ type Event struct {
// EventActivityCode The string code of the activity that occurred during the event
type EventActivityCode string
// FleetDMMatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open-source version. Premium-only attributes are marked accordingly
// FleetDMMatchAttributes Attribute conditions to match when approving FleetDM hosts. Most attributes work with FleetDM's free/open source version. Premium-only attributes are marked accordingly
type FleetDMMatchAttributes struct {
// DiskEncryptionEnabled Whether disk encryption (FileVault/BitLocker) must be enabled on the host
DiskEncryptionEnabled *bool `json:"disk_encryption_enabled,omitempty"`
@@ -4468,10 +4468,10 @@ type PolicyRule struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4508,10 +4508,10 @@ type PolicyRuleMinimum struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4551,10 +4551,10 @@ type PolicyRuleUpdate struct {
// Name Policy rule name identifier
Name string `json:"name"`
// PortRanges Policy rule affected ports ranges list
// PortRanges Policy rule affected ports ranges list. Mutually exclusive with `ports`. To mix individual ports with ranges in one rule, express each single port as a range with identical start and end values (for example, start 443, end 443).
PortRanges *[]RulePortRange `json:"port_ranges,omitempty"`
// Ports Policy rule affected ports
// Ports Policy rule affected ports. Mutually exclusive with `port_ranges`. A rule accepts either individual ports or port ranges, not both.
Ports *[]string `json:"ports,omitempty"`
// Protocol Policy rule type of the traffic
@@ -4962,7 +4962,7 @@ type RouteRequest struct {
SkipAutoApply *bool `json:"skip_auto_apply,omitempty"`
}
// RulePortRange Policy rule affected ports range
// RulePortRange Policy rule affected ports range. A range with identical start and end values represents a single port.
type RulePortRange struct {
// End The ending port of the range
End int `json:"end"`

View File

@@ -3,18 +3,69 @@ package util
import (
"os"
"os/exec"
"runtime"
"slices"
"github.com/skratchdot/open-golang/open"
)
const (
// envBrowser overrides the browser OpenBrowser launches
envBrowser = "BROWSER"
// envDesktopSession and envXDGCurrentDesktop are what xdg-open uses to pick a handler
envDesktopSession = "DESKTOP_SESSION"
envXDGCurrentDesktop = "XDG_CURRENT_DESKTOP"
// envDisplay and envWaylandDisplay are what a graphical browser needs to reach a display
envDisplay = "DISPLAY"
envWaylandDisplay = "WAYLAND_DISPLAY"
// envXDGSessionType names the session kind, e.g. tty, x11 or wayland
envXDGSessionType = "XDG_SESSION_TYPE"
)
// OpenBrowser opens the URL in a browser, respecting the BROWSER environment variable.
func OpenBrowser(url string) error {
if browser := os.Getenv("BROWSER"); browser != "" {
if browser := os.Getenv(envBrowser); browser != "" {
return exec.Command(browser, url).Start()
}
return open.Run(url)
}
// browserSessionEnvVars returns the variables that decide whether OpenBrowser can open a URL.
// DISPLAY and WAYLAND_DISPLAY are exactly what xdg-open's own has_display() checks, and without
// them it degrades to terminal browsers. BROWSER is the explicit override both xdg-open and
// OpenBrowser honor first. DESKTOP_SESSION and XDG_CURRENT_DESKTOP only tell xdg-open which
// desktop-specific opener to prefer, so they are weaker evidence, kept because the previous
// detection relied on them alone and dropping them would demote sessions that work today.
func browserSessionEnvVars() []string {
return []string{envDisplay, envWaylandDisplay, envBrowser, envDesktopSession, envXDGCurrentDesktop}
}
// graphicalXDGSessionTypes are the systemd-logind session types that come with a display. The
// other documented values are "tty" and "unspecified"; anything unrecognized is treated as no
// display, so an unknown value picks the device code flow, which works without a browser.
func graphicalXDGSessionTypes() []string {
return []string{"x11", "wayland", "mir"}
}
// HasGraphicalSession reports whether this process can open a browser and serve a loopback
// redirect back to it. Windows and macOS always can. On Linux and FreeBSD the answer is env
// based, so it only holds for a process started from the graphical session itself: a service
// does not inherit those variables and always reports false, which is why callers running in
// the user's session pass their own answer to the daemon.
func HasGraphicalSession() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return true
}
for _, env := range browserSessionEnvVars() {
if os.Getenv(env) != "" {
return true
}
}
return slices.Contains(graphicalXDGSessionTypes(), os.Getenv(envXDGSessionType))
}
// SliceDiff returns the elements in slice `x` that are not in slice `y`
func SliceDiff(x, y []string) []string {
mapY := make(map[string]struct{}, len(y))

50
util/session_test.go Normal file
View File

@@ -0,0 +1,50 @@
package util
import (
"os"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
)
func TestHasGraphicalSession(t *testing.T) {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
assert.True(t, HasGraphicalSession(), "%s always has a graphical session", runtime.GOOS)
return
}
// clear anything inherited from the session running the test, restored on cleanup
for _, env := range append(browserSessionEnvVars(), envXDGSessionType) {
t.Setenv(env, "")
os.Unsetenv(env)
}
assert.False(t, HasGraphicalSession(), "no session variables means no graphical session")
tests := []struct {
env string
value string
expected bool
}{
{env: envDisplay, value: ":0", expected: true},
{env: envWaylandDisplay, value: "wayland-0", expected: true},
{env: envDesktopSession, value: "gnome", expected: true},
{env: envXDGCurrentDesktop, value: "KDE", expected: true},
{env: envBrowser, value: "firefox", expected: true},
{env: envXDGSessionType, value: "wayland", expected: true},
{env: envXDGSessionType, value: "x11", expected: true},
{env: envXDGSessionType, value: "mir", expected: true},
{env: envXDGSessionType, value: "tty", expected: false},
{env: envXDGSessionType, value: "unspecified", expected: false},
// an unrecognized type must not be read as a display: the device code flow works anyway
{env: envXDGSessionType, value: "something-new", expected: false},
}
for _, tt := range tests {
t.Run(tt.env+"="+tt.value, func(t *testing.T) {
t.Setenv(tt.env, tt.value)
assert.Equal(t, tt.expected, HasGraphicalSession(), "%s=%s", tt.env, tt.value)
})
}
}