Compare commits

...

23 Commits

Author SHA1 Message Date
mlsmaycon
b10e18fe42 [e2e] Close the two gaps the first spike run left on Vertex
The publisher listing answered only under v1beta1, and returned two
models — fewer than the catalog ships. That is what a publisher-global
list looks like rather than what a given project has enabled, and
per-project availability is most of why live discovery beats a static
catalogue. The project-scoped form was only tried under v1, which 404s
like every other v1 path here, so try it under the version that answers.

Also ask for all versions. A publisher model is addressed as
'<id>@<version>' on the rawPredict path while the plain listing reports
one versionId per model, so a model with several live versions would
have the rest silently hidden from a picker.

Report the version in the extracted id for the same reason: Vertex
splits the wire id across name and versionId, and a listing that showed
only the name would look usable without being so.
2026-08-19 06:43:27 +00:00
mlsmaycon
47cb96c010 [e2e] Spike the vendors' own model-discovery endpoints
Management could populate the provider-config model picker from live
vendor data instead of the hand-curated catalog, which today carries
comments tracking which models a vendor retired on which date and which
cannot know what a particular account may actually invoke.

Whether that is a small feature or a large one turns on one unknown:
ListInferenceProfiles is a control-plane operation on
bedrock.<region>.amazonaws.com, while a provider record's upstream must
be bedrock-runtime.<region> for InvokeModel. Nothing documents whether a
Bedrock API key authorises the control plane, and if it does not, live
discovery for Bedrock needs SigV4 signing in management.

Probe each vendor directly — no proxy, no tunnel, no containers — under
the credential an operator would actually supply, and print a verdict
table. Bedrock gets three probes so the control-plane answer is read
against the runtime host we already know 404s, rather than as a lone
data point; Vertex gets three because which path serves the publisher
listing is itself part of the question. Vertex reuses the same JWT mint
and cloud-platform scope llm_router uses in production, so a result here
transfers.

No status code is asserted: the status is the finding. A probe fails
only when its credential is present and the request could not be made at
all, which is a broken probe rather than an answer.

Add a test_pattern dispatch input so this can run on its own in about a
minute instead of behind the sixteen-minute container suite. It reaches
the shell through an env var, since a dispatch input interpolated into a
run script is a script-injection seam.
2026-08-19 06:31:30 +00:00
Zoltan Papp
ecfbd686b8 [client, android] Expose ssh functionality for Android (#7156)
Adds an SSHClient gomobile binding so the Android app can run an SSH session over the tunnel with a PTY, exposed through a listener interface for the in-app terminal.

Server type is auto-detected from the SSH banner, which selects the auth path: JWT device-code flow, NetBird key, or a regular server (NetBird key first, then password). Host keys are verified against the peer registry for NetBird servers and trust-on-first-use for regular ones.
2026-08-18 18:49:13 +02:00
Maycon Santos
d5b283dca8 [management] Refuse a usage limit a one-off setup key cannot honour (#7220)
refuse creating one-off keys without limits set to 1
2026-08-18 18:36:42 +02:00
Viktor Liu
6210399e65 [client] Declare multi-buffer support for the loopback XDP program (#7230) 2026-08-17 13:10:12 +02:00
Viktor Liu
939b686d05 [client] Delete NRPT rules by enumerating the registry instead of a rule count (#7195) 2026-08-17 12:52:17 +02:00
Zoltan Papp
70f192344b [client] Update golang.org/x/mobile to v0.0.0-20260816165457-f98cc9b3c733 (#7229) 2026-08-17 12:10:19 +02:00
Misha Bragin
4e5b632490 [infrastructure] Don't override the dashboard image on enterprise migration (#7206) 2026-08-16 16:40:20 +02:00
Misha Bragin
93e97f4bf1 [doc] Agent network docs update (#7020)
* [docs] Update agent-network docs for management-owned pricing

  The docs still described the retired proxy-side pricing: pricing.Loader,
  pricing_path, MiddlewareDataDir, embedded defaults_pricing.yaml, and the
  symlink-safe Unix loader. Rewrite them for the current design — management
  synthesizes the whole table and ships it in cost_meter's ConfigJSON, so the
  proxy carries no price list and has nothing to reload.
2026-08-15 19:31:49 +02:00
Zoltan Papp
16544dbc58 [client] Pass stored email as login hint from UI and keep it on logout (#7199)
* [client] Pass stored email as login hint from UI and keep it on logout

Follow the CLI pattern: the Wails UI now reads the account email from the
user-owned profile state file and passes it as the OIDC login_hint on login
and session extend, since the daemon-side fallback runs as root and cannot
see the user's state file. Logout no longer deletes the stored email, so a
later login preselects the account at the IdP; profile removal remains the
operation that deletes it.

* [client] Log ignored profile lookup errors in extend-session hint fallback
2026-08-15 11:21:57 +02:00
Zoltan Papp
f458c1f265 [client] Skip IPv6 route tests when the default nexthop is unusable (#7212)
* [client] Skip IPv6 route tests when the default nexthop is unusable

ensureIPv6DefaultRoute treated a successful netlink RouteAdd as proof that
a usable IPv6 nexthop exists. Installing ::/0 via loopback can succeed while
the kernel still rejects that nexthop for a concrete prefix, which surfaced
on ubuntu22/20260810.260 runners as:

    add route to table: netlink add route: invalid argument

Probe the resolved nexthop by installing and removing a discard-prefix route
through the same code path the tests use, and skip when it fails. EEXIST
means the nexthop already carries a route, so it counts as usable.

* [client] Probe the IPv6 nexthop through raw netlink

addRoute swallows EAFNOSUPPORT and EOPNOTSUPP via isOpErr, so a nil return
did not prove the probe route was installed. Call netlink directly so an
unsupported operation skips the test instead of passing as usable.
2026-08-15 10:13:06 +02:00
Viktor Liu
ec6f1b8c27 [client] Rank Windows route candidates by combined route and interface metric (#7210) 2026-08-15 09:06:22 +02:00
Zoltan Papp
2cfe14d7ec [client] Keep account email on Android logout, drop it on profile removal (#7200)
Align Android logout semantics with the desktop UI and CLI: logging out no
longer deletes the stored account email, so the next login passes it as the
OIDC login_hint and the IdP preselects the account. Removing the profile is
now the operation that deletes the email; previously RemoveProfile left the
account file behind, which the fixed-name default profile would have
inherited on recreation.
2026-08-14 18:13:52 +02:00
Eduard Gert
85dd335836 [client] Add CI check for translation key parity (#6852)
English (en) is the source of truth for UI translation keys; the other
nine locales rely on runtime English fallback for any missing key, so a
gap never surfaces in CI. Add a dependency-free Node check that fails
when any locale declared in _index.json does not carry the exact same
key set as en (missing or orphaned keys), wired into a dedicated
UI Translations workflow that runs on locale changes.

Also close the one existing gap the check found: ja was missing
daemon.outdated.download ("Download Latest").

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-14 10:57:11 +02:00
Viktor Liu
5544761b47 [client] Add Windows DNS configuration to the debug bundle (#7196) 2026-08-13 20:07:37 +02:00
Kim Harre
1d372bb634 [infrastructure] Support non-interactive installation in getting-started.sh (#7168) 2026-08-13 18:58:03 +03:00
Viktor Liu
e290769df1 [client] Take the graphical session answer from the caller instead of the daemon environment (#7187) 2026-08-13 10:28:34 +02:00
Jack Carter
58c09ead21 [management] Document mutual exclusivity of policy rule ports and port_ranges (#7158) 2026-08-12 20:39:06 +02:00
Brad Ison
c5503fdc7f [misc] Build release branches, and don't mark releases latest before signing (#7171)
Prepares the repository for the release-branch process agreed internally:
one long-lived release-0.N branch per minor, with fixes backported by
cherry-pick and patch releases tagged from the branch.

Pushes to release-* branches now run the Release workflow and publish
immutable sha-* container images, the way pushes to main already do, so
a release branch can be tested before it is tagged. Release branches
never publish the floating "main" image tag. The push-to-main CI
workflows (Go tests on all platforms, frontend UI, install script,
mobile/wasm validation, infrastructure files, license check) also run
on release-* pushes; pull request triggers were already unfiltered, so
backport PRs were covered — this closes the post-merge gap.

Releases are no longer marked latest before signing: make_latest is
now false in all four goreleaser configs, so a release stays published
but not latest until the signing pipeline uploads the signed Windows
and macOS artifacts and marks it latest itself. Previously the release
became GitHub's "Latest release" at publish time, and the download
endpoints that resolve through the latest-release API could serve a
release whose signed installers did not exist yet. prerelease: auto
additionally labels rc tags as prereleases, so a release candidate can
never take the latest slot.

The trigger_sync_tag job is removed: it dispatched a downstream
image build on every v* tag (release candidates included), which would
race the deliberate release-branch build on every release. The android
and ios submodule bumps are unchanged.

Also sets perennial-regex = "^release-" so git-town never syncs or
ships a release branch into main.
2026-08-12 18:04:36 +02:00
Zoltan Papp
6b69f5c05d [client] Remove installer registry handlers for autostart Run keys (#7183)
The NSIS installer deleted HKLM/HKCU CurrentVersion\Run values it never
writes, which matches common AV heuristics for unwanted Run-key
manipulation and is suspected to contribute to Windows Defender and
third-party antivirus false positives on the installer.

Drop all autostart registry deletions from both the install and
uninstall sections so the installer only touches keys it creates
itself. Cleanup of the legacy machine-wide entry written by old
installers is left to documentation.

Extends the approach of the closed PR #6735, which only removed the
per-user deletion on uninstall.
2026-08-12 17:35:01 +02:00
Viktor Liu
db9fcf39ef [client] Gate IPv6 forwarding on overlay v6 and preserve host RA acceptance (#6221) 2026-08-12 16:07:00 +02:00
Lamera
52faa202b2 [client] fall back to per-IP ACL rules when ipset is unavailable (#6332) 2026-08-12 14:37:48 +02:00
Viktor Liu
f5ce0bc65a [client] Fix macOS DNS panic on malformed scutil output (#7180) 2026-08-12 13:25:12 +02:00
112 changed files with 5554 additions and 668 deletions

View File

@@ -3,7 +3,7 @@
[branches]
main = "main"
perennials = []
perennial-regex = ""
perennial-regex = "^release-"
[create]
new-branch-type = "feature"

View File

@@ -12,6 +12,13 @@ on:
AWS issues it. Leave empty for the Sonnet 4.6 default.
required: false
default: ""
test_pattern:
description: >-
Package pattern to run. Defaults to the whole suite; narrow it to one
package (e.g. ./e2e/providerdiscovery/...) when a run only needs that
package's answer and not the sixteen minutes the container suite costs.
required: false
default: "./e2e/..."
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -77,4 +84,8 @@ jobs:
GOOGLE_VERTEX_PROJECT: ${{ secrets.E2E_GOOGLE_VERTEX_PROJECT }}
GOOGLE_VERTEX_REGION: ${{ secrets.E2E_GOOGLE_VERTEX_REGION }}
GOOGLE_VERTEX_MODEL: ${{ secrets.E2E_GOOGLE_VERTEX_MODEL }}
run: go test -tags e2e -timeout 40m -v ./e2e/...
# Read through an env var rather than interpolated into the run
# script: a dispatch input reaching a shell command directly is a
# script-injection seam, however trusted the dispatcher.
TEST_PATTERN: ${{ inputs.test_pattern || './e2e/...' }}
run: go test -tags e2e -timeout 40m -v "$TEST_PATTERN"

View File

@@ -2,7 +2,7 @@ name: Check License Dependencies
on:
push:
branches: [main]
branches: [main, "release-*"]
paths:
- "go.mod"
- "go.sum"

View File

@@ -10,6 +10,7 @@ on:
push:
branches:
- main
- "release-*"
paths:
- "client/ui/frontend/**"
- "client/ui/i18n/**"

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
env:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
paths:
- "release_files/install.sh"

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -6,6 +6,7 @@ on:
- "v*"
branches:
- main
- "release-*"
pull_request:
env:
@@ -254,15 +255,23 @@ jobs:
id: tag_and_push_images
if: |
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) ||
(github.event_name == 'push' && github.ref == 'refs/heads/main')
(github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/release-')))
run: |
set -euo pipefail
# $GITHUB_REF / $GITHUB_EVENT_NAME are read from the runner
# environment rather than substituted into this script with the
# workflow expression syntax: branch names may legally contain
# $(…), and interpolating github.ref would execute it.
resolve_tags() {
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
echo "pr-${{ github.event.pull_request.number }}"
else
elif [[ "$GITHUB_REF" == "refs/heads/main" ]]; then
echo "main sha-$(git rev-parse --short HEAD)"
else
# Release branches get an immutable sha-* tag only — the floating
# "main" tag must never move from a release branch.
echo "sha-$(git rev-parse --short HEAD)"
fi
}

View File

@@ -9,21 +9,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
# Receiving workflows (cloud sync-tag, mobile bump-netbird) expect the short
# tag form (e.g. v0.30.0), not refs/tags/v0.30.0 — github.ref_name, not github.ref.
# The receiving bump-netbird workflows expect the short tag form
# (e.g. v0.30.0), not refs/tags/v0.30.0 — github.ref_name, not github.ref.
jobs:
trigger_sync_tag:
runs-on: ubuntu-latest
steps:
- name: Trigger release tag sync
uses: benc-uk/workflow-dispatch@31e2b3319479a63f0ab15bf800eff9e913504e26 # v1.3.2
with:
workflow: sync-tag.yml
ref: main
repo: ${{ secrets.UPSTREAM_REPO }}
token: ${{ secrets.NC_GITHUB_TOKEN }}
inputs: '{ "tag": "${{ github.ref_name }}" }'
trigger_android_bump:
runs-on: ubuntu-latest
if: github.event.created && !github.event.deleted && startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, '-')

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
paths:
- "infrastructure_files/**"

42
.github/workflows/ui-translations.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: UI Translations
on:
pull_request:
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
- ".github/workflows/ui-translations.yml"
push:
branches:
- main
paths:
- "client/ui/i18n/locales/**"
- "client/ui/i18n/check-translations.mjs"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
cancel-in-progress: true
jobs:
check-translations:
name: Check translation key parity
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "22"
# English (en) is the source of truth for translation keys; every other
# locale declared in _index.json must carry the exact same key set.
- name: Check translation key parity
run: node client/ui/i18n/check-translations.mjs

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- "release-*"
pull_request:
concurrency:

View File

@@ -468,6 +468,13 @@ checksum:
- glob: ./infrastructure_files/migrate-to-enterprise.sh
release:
# The signing pipeline (netbirdio/sign-pipelines, dispatched by
# trigger_signer) marks the release latest once the Windows and macOS
# artifacts are signed. Without this override goreleaser marks it latest
# at publish time, while those artifacts are still unsigned.
make_latest: false
# Mark x.y.z-rc.* and other prerelease tags as prereleases on GitHub.
prerelease: auto
extra_files:
- glob: ./infrastructure_files/getting-started-with-zitadel.sh
- glob: ./release_files/install.sh

View File

@@ -144,3 +144,11 @@ uploads:
target: https://pkgs.wiretrustee.com/yum/{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}
username: dev@wiretrustee.com
method: PUT
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -43,3 +43,11 @@ checksum:
name_template: "{{ .ProjectName }}_darwin_checksums.txt"
changelog:
disable: true
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -134,3 +134,11 @@ uploads:
target: https://pkgs.wiretrustee.com/yum/{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}
username: dev@wiretrustee.com
method: PUT
release:
# Uploads into the release created by the main .goreleaser.yaml run.
# make_latest stays false everywhere: the signing pipeline
# (netbirdio/sign-pipelines) marks the release latest after the Windows
# and macOS artifacts are signed.
make_latest: false
prerelease: auto

View File

@@ -1,6 +1,6 @@
# NetBird Agent Guidelines
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
network with a control plane. The **agent** (`client/`) runs on user machines as
a privileged daemon and manages the WireGuard interface, routing, firewall, and
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,

View File

@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
## Checklist before submitting a PR
As a critical network service and open-source project, we must enforce a few
As a critical network service and open source project, we must enforce a few
things before submitting a pull request. The
[pull request template](/.github/pull_request_template.md) mirrors this list —
fill it in rather than deleting it.

View File

@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
![CISPA_Logo_BLACK_EN_RZ_RGB (1)](https://user-images.githubusercontent.com/700848/203091324-c6d311a0-22b5-4b05-a288-91cbc6cdcc46.png)
### Acknowledgements
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
### Legal
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.

View File

@@ -14,7 +14,7 @@ Report security issues one of these two ways:
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
- **Email** — `security@netbird.io`.
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
filing a repository report.
### What to include

View File

@@ -191,39 +191,49 @@ func (a *Auth) login(urlOpener URLOpener, isAndroidTV bool) error {
return nil
}
// loginHintSetter is implemented by both concrete flows (PKCE and device code)
// but absent from the OAuthFlow interface, hence the assertion below — the same
// way internal/auth wires it in authenticateWithPKCEFlow.
type loginHintSetter interface {
SetLoginHint(hint string)
}
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, isAndroidTV bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, isAndroidTV, profileLoginHint(a.cfgPath))
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}
// An empty hint is deliberate, not a fallback: a fresh or logged-out profile
// leaves the choice to the IdP, which is how accounts get switched.
if a.cfgPath != "" {
if hint := readProfileEmail(a.cfgPath); hint != "" {
if setter, ok := oAuthFlow.(loginHintSetter); ok {
setter.SetLoginHint(hint)
}
}
return runOAuthFlow(a.ctx, oAuthFlow, urlOpener, nil)
}
// profileLoginHint returns the stored account email for the profile at cfgPath.
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
// choice to the IdP. Switching accounts is done by switching or removing
// profiles, not by logging out — logout keeps the email.
func profileLoginHint(cfgPath string) string {
if cfgPath == "" {
return ""
}
return readProfileEmail(cfgPath)
}
// runOAuthFlow drives an already acquired OAuth flow to a token: requests the
// flow info, presents the verification URL through the opener and waits for
// the browser round-trip. Open is called synchronously — it is what marks the
// surface as opened on the client side, and a fast token's OnLoginSuccess is
// a no-op until it has, so the dismissal would be dropped rather than
// delayed. Openers must therefore not block: they post their UI work and
// return. onWaiting, when set, runs after the URL is shown, right before the
// blocking wait.
func runOAuthFlow(ctx context.Context, flow auth.OAuthFlow, urlOpener URLOpener, onWaiting func()) (*auth.TokenInfo, error) {
flowInfo, err := flow.RequestAuthInfo(ctx)
if err != nil {
return nil, fmt.Errorf("request auth info: %w", err)
}
flowInfo, err := oAuthFlow.RequestAuthInfo(context.TODO())
if err != nil {
return nil, fmt.Errorf("getting a request OAuth flow info failed: %v", err)
urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
if onWaiting != nil {
onWaiting()
}
go urlOpener.Open(flowInfo.VerificationURIComplete, flowInfo.UserCode)
tokenInfo, err := oAuthFlow.WaitToken(a.ctx, flowInfo)
tokenInfo, err := flow.WaitToken(ctx, flowInfo)
if err != nil {
return nil, fmt.Errorf("waiting for browser login failed: %v", err)
return nil, fmt.Errorf("wait for token: %w", err)
}
return &tokenInfo, nil

View File

@@ -22,7 +22,8 @@ type Profile struct {
ID string
Name string
// Email is the account this profile last logged in with, "" if it never
// completed an SSO login or was logged out. See profile_state.go.
// completed an SSO login. Kept across logouts; cleared when the profile is
// removed. See profile_state.go.
Email string
IsActive bool
}
@@ -200,11 +201,9 @@ func (pm *ProfileManager) LogoutProfile(id string) error {
return fmt.Errorf("failed to save config: %w", err)
}
// Not fatal: a stale hint costs an account switch, not the logout itself.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to clear stored account email for profile %s: %v", id, err)
}
// The stored account email is kept on purpose, matching the desktop and CLI
// logout semantics: the next login passes it as the login_hint so the IdP
// preselects the account. Removing the profile is what deletes it.
log.Infof("logged out from profile: %s", id)
return nil
}
@@ -224,11 +223,24 @@ func (pm *ProfileManager) RenameProfile(id string, newName string) error {
// RemoveProfile deletes a profile
func (pm *ProfileManager) RemoveProfile(id string) error {
configPath, err := pm.getProfileConfigPath(id)
if err != nil {
return err
}
// Use ServiceManager (removes profile from profiles/ directory)
if err := pm.serviceMgr.RemoveProfile(profilemanager.ID(id), androidUsername); err != nil {
return fmt.Errorf("failed to remove profile: %w", err)
}
// The account file is this package's, not the ServiceManager's, so it must
// go here. The default profile has a fixed filename, so a recreated one
// would otherwise inherit the deleted profile's email as its login_hint.
// Not fatal: the profile itself is gone.
if err := removeProfileEmail(configPath); err != nil {
log.Warnf("failed to remove stored account email for profile %s: %v", id, err)
}
log.Infof("removed profile: %s", id)
return nil
}

View File

@@ -0,0 +1,38 @@
//go:build android
package android
import (
"fmt"
"github.com/netbirdio/netbird/client/internal/profilemanager"
)
type prefsStore interface {
Get(namespace string, v any) (bool, error)
Put(namespace string, v any) error
}
type profilePrefs struct {
prefs *profilemanager.Prefs
}
func newProfilePrefs(configDir, profileID string) (*profilePrefs, error) {
if configDir == "" || profileID == "" {
return nil, fmt.Errorf("profile prefs require a config dir and profile ID")
}
pm := NewProfileManager(configDir)
prefs, err := pm.serviceMgr.ProfilePrefs(profilemanager.ID(profileID), androidUsername)
if err != nil {
return nil, fmt.Errorf("resolve profile prefs: %w", err)
}
return &profilePrefs{prefs: prefs}, nil
}
func (p *profilePrefs) Get(namespace string, v any) (bool, error) {
return p.prefs.Get(namespace, v)
}
func (p *profilePrefs) Put(namespace string, v any) error {
return p.prefs.Put(namespace, v)
}

View File

@@ -90,10 +90,10 @@ func writeProfileEmail(configPath string, email string) error {
return nil
}
// removeProfileEmail drops the stored account email. Called on logout: while the
// email is on disk it goes out as a login_hint, which would steer the next login
// straight back into the account just logged out of. Mirrors the desktop UI's
// RemoveProfileState call.
// removeProfileEmail drops the stored account email. Called on profile removal,
// not on logout: a logged-out profile keeps its email so the next login passes
// it as the login_hint, matching the desktop and CLI semantics. Mirrors the
// desktop UI's RemoveProfileState call.
func removeProfileEmail(configPath string) error {
accountPath, err := profileAccountPathFor(configPath)
if err != nil {

View File

@@ -127,10 +127,10 @@ func TestWriteThenReadProfileEmail(t *testing.T) {
t.Fatalf("remove: %v", err)
}
if got := readProfileEmail(configPath); got != "" {
t.Errorf("expected no email after logout, got %q", got)
t.Errorf("expected no email after removal, got %q", got)
}
// Logout may run on a never-logged-in profile, so a second remove must pass.
// Removal may run on a never-logged-in profile, so a second remove must pass.
if err := removeProfileEmail(configPath); err != nil {
t.Fatalf("second remove should be a no-op: %v", err)
}

View File

@@ -0,0 +1,649 @@
//go:build android
package android
import (
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
gossh "golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/profilemanager"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/ssh/detection"
)
const (
sshDialTimeout = 30 * time.Second
sshDetectionTimeout = 5 * time.Second
)
// PasswordRequiredMarker tells Java to prompt for a password and retry. It is
// a string because gomobile flattens errors to their message, so a sentinel
// value would not survive the binding.
const PasswordRequiredMarker = "netbird-ssh-password-required"
// HostKeyUnknownMarker tells Java to show the fingerprint and, on confirmation,
// retry with TrustHostKey set. The presented fingerprint is appended after the
// marker so the prompt can display it and the retry can guard against a key
// that changed between the two connects. Only regular (non-NetBird) servers
// reach this: NetBird peers verify against the registry.
const HostKeyUnknownMarker = "netbird-ssh-hostkey-unknown"
var (
errPasswordRequired = errors.New(PasswordRequiredMarker)
errClientClosed = errors.New("ssh client closed")
)
// errHostKeyUnknown carries the presented fingerprint so Connect can build the
// marker message the Java side parses.
type errHostKeyUnknown struct {
fingerprint string
}
func (e *errHostKeyUnknown) Error() string {
return HostKeyUnknownMarker + ":" + e.fingerprint
}
// SSHTerminalListener receives SSH session events. It is implemented in Java.
//
// All callbacks are invoked from goroutines and may run concurrently with each
// other; the implementation must be safe to call from any thread.
type SSHTerminalListener interface {
OnConnected()
OnData(data []byte)
OnClose(reason string)
OnError(message string)
}
// SSHClient is a NetBird-aware SSH client exposed to Java via gomobile.
//
// It dials through the running NetBird tunnel and runs a standard SSH session
// on top with PTY enabled. Host-key verification uses the NetBird-provided
// peer SSH host keys, identical to the desktop client.
type SSHClient struct {
nb *Client
mu sync.Mutex
listener SSHTerminalListener
urlOpener URLOpener
sshClient *gossh.Client
session *gossh.Session
stdin io.WriteCloser
closed bool
// gen identifies the current connection attempt. Connect and Close bump it,
// so an in-flight dial or a reader left over from a previous connection
// finds itself stale and stays silent instead of publishing OnConnected or
// OnClose for a connection the caller already abandoned.
gen uint64
dialCancel context.CancelFunc
// knownHostsConfigDir and knownHostsProfile locate the TOFU store for
// regular SSH servers in the profile's preferences. Java supplies them,
// since an overlay IP is a different host under a different profile. Empty
// until set: without them a regular server cannot be verified and Connect
// refuses one.
knownHostsConfigDir string
knownHostsProfile string
// trustHostKey carries the fingerprint the user confirmed on a previous
// attempt, so the retry accepts exactly that key and persists it.
trustHostKey string
}
// NewSSHClient creates a new SSH client bound to the running NetBird Client.
func NewSSHClient(c *Client) *SSHClient {
return &SSHClient{nb: c}
}
// SetListener registers the Java listener. Must be called before Connect to
// receive any events.
func (s *SSHClient) SetListener(l SSHTerminalListener) {
s.mu.Lock()
s.listener = l
s.mu.Unlock()
}
// SetURLOpener registers the Java URL opener used to display the device-code
// authorization page in a Custom Tabs window when the target peer requires
// JWT authentication. Must be set before Connect to be effective.
func (s *SSHClient) SetURLOpener(opener URLOpener) {
s.mu.Lock()
s.urlOpener = opener
s.mu.Unlock()
}
// SetKnownHostsStore points the TOFU host-key store at a profile's preferences.
// Must be set before connecting to a regular SSH server; without it such a
// server cannot be verified and Connect refuses one.
func (s *SSHClient) SetKnownHostsStore(configDir, profileID string) {
s.mu.Lock()
s.knownHostsConfigDir = configDir
s.knownHostsProfile = profileID
s.mu.Unlock()
}
// TrustHostKey records the fingerprint the user confirmed for a regular server,
// so the next Connect accepts that exact key and adds it to the known-hosts
// store. Passing a fingerprint that no longer matches makes the connect fail
// rather than trust a key that changed since the prompt.
func (s *SSHClient) TrustHostKey(fingerprint string) {
s.mu.Lock()
s.trustHostKey = fingerprint
s.mu.Unlock()
}
// Connect dials the SSH server through the NetBird tunnel and performs the
// SSH handshake. It auto-detects the server type via SSH banner inspection
// and selects the appropriate authentication path:
//
// - NetBird-SSH server requiring JWT: launches the OAuth 2.0 device-code
// flow, opens the verification URL through the registered URLOpener, and
// uses the resulting token as the SSH password. Host-key verification
// uses the NetBird peer registry.
// - NetBird-SSH server without JWT: authenticates with the NetBird SSH
// private key. Host-key verification uses the NetBird peer registry.
// - Regular SSH server (e.g. OpenSSH): authenticates with the NetBird key
// first (so a user-installed NetBird public key works), then falls back
// to the supplied password if non-empty. Host-key verification is
// trust-on-first-use against the per-profile known-hosts store.
//
// The password parameter is only consulted for regular SSH servers.
func (s *SSHClient) Connect(host string, port int, user, password string) error {
if port < 1 || port > 65535 {
return fmt.Errorf("invalid port: %d", port)
}
cfg, cfgPath, cc := s.nb.authSnapshot()
if cc == nil {
return errors.New("netbird client not running")
}
if cfg == nil {
return errors.New("netbird config not loaded")
}
engine := cc.Engine()
if engine == nil {
return errors.New("netbird engine not available")
}
s.mu.Lock()
s.gen++
gen := s.gen
s.mu.Unlock()
serverType := detectServerType(host, port)
log.Debugf("SSH server type: %s", serverType)
authMethods, hostKeyCallback, err := s.buildAuth(cfg, cfgPath, engine, serverType, password)
if err != nil {
return err
}
clientConfig := &gossh.ClientConfig{
User: user,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: sshDialTimeout,
}
err = s.dialAndHandshake(gen, host, port, clientConfig)
// An unknown host key is a prompt, not a failure: return the marker intact
// (rootCause would unwrap it) so Java can show the fingerprint and retry.
var unknownHost *errHostKeyUnknown
if errors.As(err, &unknownHost) {
return errors.New(unknownHost.Error())
}
// A regular server may still accept a password, so let the caller ask for
// one instead of failing. NetBird servers never use a password, so a
// failure there is genuine.
if err != nil && serverType != detection.ServerTypeNetBirdJWT &&
serverType != detection.ServerTypeNetBirdNoJWT && isAuthFailure(err) &&
passwordCouldHelp(err, password != "") {
return errPasswordRequired
}
if err != nil {
return rootCause(err)
}
return nil
}
// StartSession requests a PTY and starts an interactive shell. Output from
// the session is forwarded to the listener via OnData.
func (s *SSHClient) StartSession(cols, rows int) error {
err := s.startSession(cols, rows)
if err != nil {
log.Infof("SSH: start session failed: %v", err)
return rootCause(err)
}
return nil
}
// Write sends data to the SSH session stdin.
func (s *SSHClient) Write(data []byte) error {
s.mu.Lock()
stdin := s.stdin
s.mu.Unlock()
if stdin == nil {
return errors.New("ssh session not started")
}
if _, err := stdin.Write(data); err != nil {
return fmt.Errorf("write stdin: %w", err)
}
return nil
}
// Resize updates the PTY window size.
func (s *SSHClient) Resize(cols, rows int) error {
s.mu.Lock()
session := s.session
s.mu.Unlock()
if session == nil {
return errors.New("ssh session not started")
}
return session.WindowChange(rows, cols)
}
// Reset makes a closed client usable for another Connect: Close leaves the
// one-shot guard set, and clearing it lets the same client back a reconnect.
func (s *SSHClient) Reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.closed = false
}
// Close terminates the SSH session and underlying connection. Safe to call
// multiple times.
func (s *SSHClient) Close() error {
s.mu.Lock()
s.gen++
if s.dialCancel != nil {
s.dialCancel()
s.dialCancel = nil
}
sshClient := s.sshClient
session := s.session
stdin := s.stdin
s.sshClient = nil
s.session = nil
s.stdin = nil
notify := !s.closed
s.closed = true
listener := s.listener
s.mu.Unlock()
if stdin != nil {
if err := stdin.Close(); err != nil {
log.Debugf("ssh: stdin close: %v", err)
}
}
if session != nil {
if err := session.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: session close: %v", err)
}
}
var firstErr error
if sshClient != nil {
if err := sshClient.Close(); err != nil {
firstErr = err
}
}
if notify && listener != nil {
listener.OnClose("closed by client")
}
return firstErr
}
func (s *SSHClient) startSession(cols, rows int) error {
log.Debugf("SSH: starting session %dx%d", cols, rows)
s.mu.Lock()
sshClient := s.sshClient
gen := s.gen
s.mu.Unlock()
if sshClient == nil {
return errors.New("ssh client not connected")
}
pty, err := nbssh.StartPTYSession(sshClient, cols, rows)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(pty.Session, "stale session")
return errClientClosed
}
s.session = pty.Session
s.stdin = pty.Stdin
s.mu.Unlock()
readerDone := make(chan string, 2)
go func() { readerDone <- s.readLoop(pty.Stdout, "stdout") }()
go func() { readerDone <- s.readLoop(pty.Stderr, "stderr") }()
go func() {
reason := <-readerDone
if second := <-readerDone; reason == "" {
reason = second
}
s.notifyClose(gen, reason)
}()
log.Debug("SSH: session started, shell running")
return nil
}
func (s *SSHClient) buildAuth(cfg *profilemanager.Config, cfgPath string, engine *internal.Engine,
serverType detection.ServerType, password string) ([]gossh.AuthMethod, gossh.HostKeyCallback, error) {
switch serverType {
case detection.ServerTypeNetBirdJWT:
token, err := s.requestJWTToken(cfg, cfgPath)
if err != nil {
return nil, nil, fmt.Errorf("jwt: %w", err)
}
auths := []gossh.AuthMethod{gossh.Password(token)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeNetBirdNoJWT:
if cfg.SSHKey == "" {
return nil, nil, errors.New("no NetBird SSH key available")
}
signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey))
if err != nil {
return nil, nil, fmt.Errorf("parse netbird ssh key: %w", err)
}
auths := []gossh.AuthMethod{gossh.PublicKeys(signer)}
return auths, nbssh.CreateHostKeyCallback(nbssh.PeerKeyLookup(engine.GetPeerSSHKey)), nil
case detection.ServerTypeRegular:
var auths []gossh.AuthMethod
if cfg.SSHKey != "" {
if signer, err := gossh.ParsePrivateKey([]byte(cfg.SSHKey)); err == nil {
auths = append(auths, gossh.PublicKeys(signer))
} else {
log.Debugf("ssh: parse netbird key for regular auth: %v", err)
}
}
if password != "" {
pw := password
auths = append(auths, gossh.Password(pw))
auths = append(auths, gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range questions {
answers[i] = pw
}
return answers, nil
}))
}
if len(auths) == 0 {
// Nothing to offer at all: ask for a password rather than failing,
// so the caller can retry once the user supplies one.
return nil, nil, errPasswordRequired
}
callback, err := s.tofuHostKeyCallback()
if err != nil {
return nil, nil, err
}
return auths, callback, nil
default:
return nil, nil, fmt.Errorf("unsupported SSH server type: %v", serverType)
}
}
// tofuHostKeyCallback verifies a regular server's host key against the
// per-profile known-hosts store. An unknown host returns errHostKeyUnknown so
// Java can show the fingerprint and, once confirmed, retry with the key
// trusted; a changed key is rejected outright, as OpenSSH does. When the user
// has confirmed a fingerprint, the callback accepts exactly that key and
// appends it to the store.
func (s *SSHClient) tofuHostKeyCallback() (gossh.HostKeyCallback, error) {
s.mu.Lock()
configDir := s.knownHostsConfigDir
profileID := s.knownHostsProfile
trusted := s.trustHostKey
s.mu.Unlock()
if configDir == "" || profileID == "" {
return nil, errors.New("no known-hosts store configured for regular SSH")
}
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return nil, fmt.Errorf("load known-hosts store: %w", err)
}
return func(hostname string, remote net.Addr, key gossh.PublicKey) error {
verdict, err := store.verify(hostname, remote, key)
if err != nil {
return err
}
if verdict == hostKeyMatched {
return nil
}
if verdict == hostKeyChanged {
return fmt.Errorf("SSH host key changed for %s (possible attack)", hostname)
}
fingerprint := gossh.FingerprintSHA256(key)
if trusted == "" {
return &errHostKeyUnknown{fingerprint: fingerprint}
}
if trusted != fingerprint {
return fmt.Errorf("SSH host key changed since it was confirmed for %s", hostname)
}
if err := store.append(hostname, remote, key); err != nil {
return fmt.Errorf("persist trusted host key: %w", err)
}
// The confirmation is spent: now that the key is stored, a later
// reconnect must verify against the file, not re-accept this fingerprint.
s.mu.Lock()
s.trustHostKey = ""
s.mu.Unlock()
return nil
}, nil
}
func (s *SSHClient) requestJWTToken(cfg *profilemanager.Config, cfgPath string) (string, error) {
s.mu.Lock()
urlOpener := s.urlOpener
s.mu.Unlock()
if urlOpener == nil {
return "", errors.New("URL opener not configured for JWT auth")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
flow, err := auth.NewOAuthFlow(ctx, cfg, false, true, profileLoginHint(cfgPath))
if err != nil {
return "", fmt.Errorf("create oauth flow: %w", err)
}
// The status callback covers the browser round-trip, which would
// otherwise leave the terminal blank.
tokenInfo, err := runOAuthFlow(ctx, flow, urlOpener, func() {
s.notifyStatus("Waiting for browser authentication...")
})
if err != nil {
return "", err
}
token := tokenInfo.GetTokenToUse()
if token == "" {
return "", errors.New("empty token returned by IdP")
}
// Tells the client the browser round-trip is over so it can dismiss the
// surface it opened, the same way the login and session-extend flows do.
// Without it the Custom Tab stays in front of the terminal even though the
// token has already been collected.
urlOpener.OnLoginSuccess()
return token, nil
}
func (s *SSHClient) dialAndHandshake(gen uint64, host string, port int, clientConfig *gossh.ClientConfig) error {
addr := net.JoinHostPort(host, strconv.Itoa(port))
ctx, cancel := context.WithTimeout(context.Background(), sshDialTimeout)
defer cancel()
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
return errClientClosed
}
s.dialCancel = cancel
s.mu.Unlock()
var dialer net.Dialer
conn, err := dialer.DialContext(ctx, "tcp", addr)
if err != nil {
return fmt.Errorf("dial %s: %w", addr, err)
}
client, err := nbssh.Handshake(ctx, conn, addr, clientConfig)
if err != nil {
return err
}
s.mu.Lock()
if gen != s.gen {
s.mu.Unlock()
closeQuiet(client, "stale ssh client")
return errClientClosed
}
s.sshClient = client
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnConnected()
}
return nil
}
func (s *SSHClient) readLoop(r io.Reader, name string) string {
buf := make([]byte, 4096)
for {
n, err := r.Read(buf)
if n > 0 {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
chunk := make([]byte, n)
copy(chunk, buf[:n])
listener.OnData(chunk)
}
}
if err != nil {
// EOF is a normal shell exit, so report it without a reason.
if errors.Is(err, io.EOF) {
return ""
}
log.Debugf("ssh %s read: %v", name, err)
return rootCause(err).Error()
}
}
}
// notifyStatus writes a progress line to the terminal through the normal
// output path, so long steps are visible while nothing else is arriving.
func (s *SSHClient) notifyStatus(text string) {
s.mu.Lock()
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnData([]byte("\r\n\x1b[33m" + text + "\x1b[0m\r\n"))
}
}
func (s *SSHClient) notifyClose(gen uint64, reason string) {
s.mu.Lock()
if gen != s.gen || s.closed {
s.mu.Unlock()
return
}
s.closed = true
listener := s.listener
s.mu.Unlock()
if listener != nil {
listener.OnClose(reason)
}
}
func closeQuiet(c io.Closer, label string) {
if c == nil {
return
}
if err := c.Close(); err != nil && !errors.Is(err, io.EOF) {
log.Debugf("ssh: close %s: %v", label, err)
}
}
func detectServerType(host string, port int) detection.ServerType {
ctx, cancel := context.WithTimeout(context.Background(), sshDetectionTimeout)
defer cancel()
dialer := &net.Dialer{}
serverType, err := detection.DetectSSHServerType(ctx, dialer, host, port)
if err != nil {
log.Debugf("ssh: server detection failed: %v (assuming regular SSH)", err)
return detection.ServerTypeRegular
}
return serverType
}
// rootCause returns the innermost error of a %w chain, so the terminal shows
// "i/o timeout" rather than every layer that added context on the way up.
func rootCause(err error) error {
for {
// A joined error has no single root, so keep it as-is.
if _, ok := err.(interface{ Unwrap() []error }); ok {
return err
}
next := errors.Unwrap(err)
if next == nil {
return err
}
err = next
}
}
// isAuthFailure distinguishes credential rejection from dial, timeout and
// host-key errors, which retrying with a password would not fix.
func isAuthFailure(err error) bool {
if errors.Is(err, errPasswordRequired) {
return true
}
var partial *gossh.PartialSuccessError
if errors.As(err, &partial) {
return true
}
return strings.Contains(err.Error(), "unable to authenticate")
}
// passwordCouldHelp reports whether prompting for a password again can change
// the outcome. gossh lists a method under "attempted methods" only when the
// server offered it, so a supplied password that was never attempted means the
// server does not accept passwords and the real error should surface instead.
func passwordCouldHelp(err error, passwordOffered bool) bool {
if !passwordOffered {
return true
}
msg := err.Error()
return strings.Contains(msg, "password") || strings.Contains(msg, "keyboard-interactive")
}

View File

@@ -0,0 +1,168 @@
//go:build android
package android
import (
"bytes"
"net"
"strconv"
"strings"
"sync"
gossh "golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
const knownHostsNamespace = "ssh"
const (
hostKeyUnknown hostKeyVerdict = iota
hostKeyMatched
hostKeyChanged
)
var knownHostsMu sync.Mutex
type hostKeyVerdict uint8
type knownHostsSection struct {
KnownHosts []string `json:"knownHosts"`
}
type knownHostsStore struct {
prefs prefsStore
}
// RemoveKnownHost deletes every known-hosts entry for host:port from the
// profile's store, so a host trusted for a session that is being deleted does
// not linger. Java calls this only once no session targets that host, so a
// shared host stays trusted. A missing entry is not an error: the goal state
// is "absent".
func RemoveKnownHost(configDir, profileID, host string, port int) error {
store, err := openKnownHostsStore(configDir, profileID)
if err != nil {
return err
}
return store.removeHost(host, port)
}
func openKnownHostsStore(configDir, profileID string) (*knownHostsStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &knownHostsStore{prefs: prefs}, nil
}
func (st *knownHostsStore) verify(hostname string, remote net.Addr, key gossh.PublicKey) (hostKeyVerdict, error) {
lines, err := st.lines()
if err != nil {
return hostKeyUnknown, err
}
targets := knownHostsTargets(hostname, remote)
verdict := hostKeyUnknown
for _, line := range lines {
pubKey, ok := knownHostsLineKey(line, targets)
if !ok {
continue
}
if pubKey.Type() == key.Type() && bytes.Equal(pubKey.Marshal(), key.Marshal()) {
return hostKeyMatched, nil
}
verdict = hostKeyChanged
}
return verdict, nil
}
func (st *knownHostsStore) append(hostname string, remote net.Addr, key gossh.PublicKey) error {
line := knownhosts.Line(knownHostsTargets(hostname, remote), key)
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: append(lines, line)})
}
func (st *knownHostsStore) removeHost(host string, port int) error {
target := knownhosts.Normalize(net.JoinHostPort(host, strconv.Itoa(port)))
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
lines, err := st.lines()
if err != nil {
return err
}
kept := make([]string, 0, len(lines))
for _, line := range lines {
if knownHostsLineMatches(line, target) {
continue
}
kept = append(kept, line)
}
if len(kept) == len(lines) {
return nil
}
return st.prefs.Put(knownHostsNamespace, knownHostsSection{KnownHosts: kept})
}
func (st *knownHostsStore) lines() ([]string, error) {
var section knownHostsSection
if _, err := st.prefs.Get(knownHostsNamespace, &section); err != nil {
return nil, err
}
return section.KnownHosts, nil
}
func knownHostsTargets(hostname string, remote net.Addr) []string {
targets := []string{knownhosts.Normalize(hostname)}
if remote != nil {
if normalized := knownhosts.Normalize(remote.String()); normalized != targets[0] {
targets = append(targets, normalized)
}
}
return targets
}
func knownHostsLineKey(line string, targets []string) (gossh.PublicKey, bool) {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return nil, false
}
_, hosts, pubKey, _, _, err := gossh.ParseKnownHosts([]byte(trimmed))
if err != nil {
return nil, false
}
for _, host := range hosts {
for _, target := range targets {
if host == target {
return pubKey, true
}
}
}
return nil, false
}
// knownHostsLineMatches reports whether a known-hosts line's address list
// contains the normalized target. Comment and blank lines never match.
func knownHostsLineMatches(line, target string) bool {
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return false
}
fields := strings.Fields(trimmed)
if len(fields) == 0 {
return false
}
for _, addr := range strings.Split(fields[0], ",") {
if addr == target {
return true
}
}
return false
}

View File

@@ -0,0 +1,104 @@
//go:build android
package android
const (
sshSessionsNamespace = "ssh-sessions"
maxStoredSSHSessions = 50
)
type sshSessionRecord struct {
ID string `json:"id"`
Host string `json:"host"`
Port int `json:"port"`
User string `json:"user"`
}
type sshSessionsSection struct {
Sessions []sshSessionRecord `json:"sessions"`
}
// SSHSessionEntry is one stored SSH session, without any credential.
type SSHSessionEntry struct {
ID string
Host string
Port int
User string
}
// SSHSessionArray wraps stored SSH sessions for gomobile compatibility.
type SSHSessionArray struct {
items []*SSHSessionEntry
}
// NewSSHSessionArray creates an empty session array to fill via Add.
func NewSSHSessionArray() *SSHSessionArray {
return &SSHSessionArray{}
}
// Add appends a session entry, oldest first.
func (a *SSHSessionArray) Add(id, host string, port int, user string) {
a.items = append(a.items, &SSHSessionEntry{ID: id, Host: host, Port: port, User: user})
}
// Length returns the number of entries.
func (a *SSHSessionArray) Length() int {
return len(a.items)
}
// Get returns the entry at index i, or nil when out of range.
func (a *SSHSessionArray) Get(i int) *SSHSessionEntry {
if i < 0 || i >= len(a.items) {
return nil
}
return a.items[i]
}
// SSHSessionStore reads and writes a profile's stored SSH sessions.
type SSHSessionStore struct {
prefs prefsStore
}
// NewSSHSessionStore opens the session store of the given profile.
func NewSSHSessionStore(configDir, profileID string) (*SSHSessionStore, error) {
prefs, err := newProfilePrefs(configDir, profileID)
if err != nil {
return nil, err
}
return &SSHSessionStore{prefs: prefs}, nil
}
// Load returns the stored sessions, oldest first.
func (s *SSHSessionStore) Load() (*SSHSessionArray, error) {
var section sshSessionsSection
if _, err := s.prefs.Get(sshSessionsNamespace, &section); err != nil {
return nil, err
}
out := NewSSHSessionArray()
for _, record := range section.Sessions {
if record.ID == "" || record.Host == "" {
continue
}
out.Add(record.ID, record.Host, record.Port, record.User)
}
return out, nil
}
// Save replaces the stored sessions, keeping only the newest entries when the
// list exceeds the storage cap.
func (s *SSHSessionStore) Save(sessions *SSHSessionArray) error {
var items []*SSHSessionEntry
if sessions != nil {
items = sessions.items
}
if len(items) > maxStoredSSHSessions {
items = items[len(items)-maxStoredSSHSessions:]
}
records := make([]sshSessionRecord, 0, len(items))
for _, item := range items {
records = append(records, sshSessionRecord{ID: item.ID, Host: item.Host, Port: item.Port, User: item.User})
}
return s.prefs.Put(sshSessionsNamespace, sshSessionsSection{Sessions: records})
}

View File

@@ -305,6 +305,12 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
return domain
}
// A reverse zone names an address prefix, so it follows the address rules,
// which also keeps its digit labels intact.
if zone, ok := a.anonymizeReverseZone(baseDomain); ok {
return withTrailingDot(zone, hasDot)
}
if suffix := protectedSuffix(baseDomain); suffix != "" {
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
return domain
@@ -405,6 +411,10 @@ func (a *Anonymizer) AnonymizeString(str string) string {
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
// Reverse zones go first and are then held out of the passes below: their
// labels are digits, which the address patterns would otherwise consume.
str, restoreZones := a.replaceReverseZones(str)
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
@@ -425,7 +435,7 @@ func (a *Anonymizer) AnonymizeString(str string) string {
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
}
return str
return restoreZones(str)
}
// sortedDomains returns the domain mappings longest-first, so a full-FQDN

View File

@@ -0,0 +1,174 @@
package anonymize
import (
"encoding/hex"
"net/netip"
"regexp"
"strconv"
"strings"
)
const (
reverseZoneSuffixV4 = ".in-addr.arpa"
reverseZoneSuffixV6 = ".ip6.arpa"
v6Nibbles = 32
v4Octets = 4
)
// reverseZoneRegexes match a reverse zone or a full reverse name in free text.
// They are applied before the address passes of AnonymizeString, whose IPv4
// pattern would otherwise consume the digit labels of a zone and replace parts
// of it with unrelated addresses.
var reverseZoneRegexes = []*regexp.Regexp{
regexp.MustCompile(`(?:[0-9]{1,3}\.){1,4}in-addr\.arpa\b`),
regexp.MustCompile(`(?:[0-9a-fA-F]\.){1,32}ip6\.arpa\b`),
}
// anonymizeReverseZone maps a reverse zone to the zone of the anonymized form
// of the prefix it encodes, so it follows the address rules rather than the
// domain ones: the zone of an address that is preserved is preserved too, and
// the zone of one that is replaced names the replacement. This keeps a reverse
// zone recognizable as such, and consistent with the addresses it belongs to
// elsewhere in the same output. It reports false for anything that is not a
// reverse zone.
func (a *Anonymizer) anonymizeReverseZone(domain string) (string, bool) {
prefix, labelCount, suffix, ok := parseReverseZone(domain)
if !ok {
return "", false
}
anonymized := a.AnonymizeIP(prefix)
if anonymized == prefix {
return domain, true
}
return reverseZoneName(anonymized, labelCount) + suffix, true
}
// replaceReverseZones anonymizes every reverse zone in str and swaps each one
// for a placeholder, returning a function that puts the anonymized zones back.
// The placeholders carry no dots, digits or colons, so no later pass matches
// them.
func (a *Anonymizer) replaceReverseZones(str string) (string, func(string) string) {
var zones []string
for _, re := range reverseZoneRegexes {
str = re.ReplaceAllStringFunc(str, func(match string) string {
zone, ok := a.anonymizeReverseZone(match)
if !ok {
return match
}
zones = append(zones, zone)
return reverseZonePlaceholder(len(zones) - 1)
})
}
if len(zones) == 0 {
return str, func(s string) string { return s }
}
return str, func(s string) string {
for i, zone := range zones {
s = strings.ReplaceAll(s, reverseZonePlaceholder(i), zone)
}
return s
}
}
func reverseZonePlaceholder(index int) string {
return "\x00reversezone" + strconv.Itoa(index) + "\x00"
}
// parseReverseZone turns a reverse zone into the address of the prefix its
// labels spell backwards, padding the absent low-order part with zeroes, and
// returns the label count and zone suffix so the name can be rebuilt.
func parseReverseZone(domain string) (netip.Addr, int, string, bool) {
lower := strings.ToLower(domain)
switch {
case strings.HasSuffix(lower, reverseZoneSuffixV4):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV4), ".")
addr, ok := reverseZoneAddrV4(labels)
return addr, len(labels), reverseZoneSuffixV4, ok
case strings.HasSuffix(lower, reverseZoneSuffixV6):
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV6), ".")
addr, ok := reverseZoneAddrV6(labels)
return addr, len(labels), reverseZoneSuffixV6, ok
default:
return netip.Addr{}, 0, "", false
}
}
func reverseZoneAddrV4(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v4Octets {
return netip.Addr{}, false
}
var octets [v4Octets]byte
for i, label := range labels {
octet, err := strconv.ParseUint(label, 10, 8)
if err != nil {
return netip.Addr{}, false
}
octets[len(labels)-1-i] = byte(octet)
}
return netip.AddrFrom4(octets), true
}
func reverseZoneAddrV6(labels []string) (netip.Addr, bool) {
if len(labels) == 0 || len(labels) > v6Nibbles {
return netip.Addr{}, false
}
nibbles := make([]byte, 0, v6Nibbles)
for i := len(labels) - 1; i >= 0; i-- {
if len(labels[i]) != 1 || !isHexDigit(labels[i][0]) {
return netip.Addr{}, false
}
nibbles = append(nibbles, labels[i][0])
}
for len(nibbles) < v6Nibbles {
nibbles = append(nibbles, '0')
}
var groups []string
for i := 0; i < len(nibbles); i += 4 {
groups = append(groups, string(nibbles[i:i+4]))
}
addr, err := netip.ParseAddr(strings.Join(groups, ":"))
if err != nil {
return netip.Addr{}, false
}
return addr, true
}
// reverseZoneName spells the first labelCount labels of addr backwards, the
// inverse of parseReverseZone, without the zone suffix.
func reverseZoneName(addr netip.Addr, labelCount int) string {
labels := make([]string, 0, labelCount)
if addr.Is4() {
octets := addr.As4()
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, strconv.Itoa(int(octets[i])))
}
return strings.Join(labels, ".")
}
address := addr.As16()
nibbles := hex.EncodeToString(address[:])
for i := labelCount - 1; i >= 0; i-- {
labels = append(labels, string(nibbles[i]))
}
return strings.Join(labels, ".")
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}

View File

@@ -0,0 +1,171 @@
package anonymize
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newLeveledAnonymizer(level Level) *Anonymizer {
a := NewAnonymizer(DefaultAddresses())
a.SetLevel(level)
return a
}
// TestAnonymizeDomainReverseZone covers reverse zones going through the address
// rules instead of the domain ones, so a zone stays a zone and an address that
// is preserved keeps the zone that names it.
func TestAnonymizeDomainReverseZone(t *testing.T) {
// 100.64.0.0/10 is the overlay range, which is CGNAT: preserved at the
// default level and replaced from the internal pool at the strict one
const overlayZone = "64.100.in-addr.arpa"
t.Run("overlay zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, overlayZone, a.AnonymizeDomain(overlayZone), "should keep the zone of a preserved address")
})
t.Run("private zone preserved at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "168.192.in-addr.arpa", a.AnonymizeDomain("168.192.in-addr.arpa"), "should keep the zone of a private address")
})
t.Run("overlay zone replaced at the strict level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelStrict)
got := a.AnonymizeDomain(overlayZone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, overlayZone, got, "should replace the encoded prefix")
assert.Len(t, strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV4), "."), 2,
"should keep the label count, got %q", got)
})
t.Run("public zone replaced at the default level", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("113.0.203.in-addr.arpa")
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
assert.NotEqual(t, "113.0.203.in-addr.arpa", got, "should replace a public prefix")
})
t.Run("zone of an address keeps that address mapping", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
anonymizedAddr := a.AnonymizeIPString("203.0.113.7")
got := a.AnonymizeDomain("7.113.0.203.in-addr.arpa")
octets := strings.Split(anonymizedAddr, ".")
want := octets[3] + "." + octets[2] + "." + octets[1] + "." + octets[0] + reverseZoneSuffixV4
assert.Equal(t, want, got, "should name the same replacement as the address itself")
})
t.Run("ipv6 nibble labels stay single digits", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeDomain(zone)
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV6), "should stay a reverse zone, got %q", got)
labels := strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV6), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
for _, label := range labels {
assert.Len(t, label, 1, "nibble label %q should stay a single digit", label)
}
})
t.Run("trailing dot is kept", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
assert.Equal(t, "64.100.in-addr.arpa.", a.AnonymizeDomain("64.100.in-addr.arpa."), "should keep the trailing dot")
})
t.Run("a domain that only looks like a zone is anonymized as a domain", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeDomain("not-a-zone.in-addr.arpa")
assert.NotContains(t, got, "in-addr.arpa", "should fall back to domain anonymization")
})
}
// TestAnonymizeStringReverseZone verifies that a zone inside free text, such as
// a DNS log line, is not chewed up by the address passes. The IPv4 pattern
// matches any run of dotted digits, which a reverse zone is made of.
func TestAnonymizeStringReverseZone(t *testing.T) {
t.Run("ipv6 zone survives the address passes", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
got := a.AnonymizeString("question: domain=" + zone + " type=PTR")
assert.Contains(t, got, "type=PTR", "should keep the rest of the line")
assert.NotContains(t, got, "198.51.100", "should not rewrite nibble labels as an address")
labels := strings.Split(strings.TrimSuffix(strings.TrimPrefix(got, "question: domain="), reverseZoneSuffixV6+" type=PTR"), ".")
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
})
t.Run("preserved ipv4 zone is untouched", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
line := "reverse zone 64.100.in-addr.arpa registered"
assert.Equal(t, line, a.AnonymizeString(line), "should keep the zone of a preserved address")
})
t.Run("public ipv4 zone is replaced consistently", func(t *testing.T) {
a := newLeveledAnonymizer(LevelDefault)
got := a.AnonymizeString("zone 113.0.203.in-addr.arpa and address 203.0.113.7")
assert.NotContains(t, got, "113.0.203.in-addr.arpa", "should replace the zone")
assert.NotContains(t, got, "203.0.113.7", "should replace the address")
assert.Contains(t, got, reverseZoneSuffixV4, "should keep the zone suffix")
})
}
func TestParseReverseZone(t *testing.T) {
tests := []struct {
name string
zone string
addr string
labels int
}{
{name: "v4 two labels", zone: "0.100" + reverseZoneSuffixV4, addr: "100.0.0.0", labels: 2},
{name: "v4 three labels", zone: "1.168.192" + reverseZoneSuffixV4, addr: "192.168.1.0", labels: 3},
{name: "v4 full address", zone: "7.113.0.203" + reverseZoneSuffixV4, addr: "203.0.113.7", labels: 4},
{
name: "v6 prefix",
zone: "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6,
addr: "2::",
labels: 28,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
addr, labels, suffix, ok := parseReverseZone(tc.zone)
require.True(t, ok, "should decode the reverse zone")
assert.Equal(t, tc.addr, addr.String(), "should decode to the encoded prefix")
assert.Equal(t, tc.labels, labels, "should count the labels")
assert.Equal(t, tc.zone, reverseZoneName(addr, labels)+suffix, "should re-encode to the original zone")
})
}
}
func TestParseReverseZoneRejectsNonZones(t *testing.T) {
tests := []string{
"example.com",
"in-addr.arpa",
"x.100" + reverseZoneSuffixV4,
"256" + reverseZoneSuffixV4,
"1.2.3.4.5" + reverseZoneSuffixV4,
"ab" + reverseZoneSuffixV6,
"g" + reverseZoneSuffixV6,
}
for _, zone := range tests {
t.Run(zone, func(t *testing.T) {
_, _, _, ok := parseReverseZone(zone)
assert.False(t, ok, "should reject %q", zone)
})
}
}

View File

@@ -5,7 +5,6 @@ import (
"fmt"
"os"
"os/user"
"runtime"
"strings"
log "github.com/sirupsen/logrus"
@@ -121,7 +120,7 @@ func doDaemonLogin(ctx context.Context, cmd *cobra.Command, providedSetupKey str
loginRequest := proto.LoginRequest{
SetupKey: providedSetupKey,
ManagementUrl: managementURL,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
DnsLabels: dnsLabelsReq,
ProfileName: &handle,
@@ -189,7 +188,8 @@ func doExtendSession(ctx context.Context, cmd *cobra.Command) error {
client := proto.NewDaemonServiceClient(conn)
req := &proto.RequestExtendAuthSessionRequest{}
// the CLI runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: util.HasGraphicalSession()}
// Pre-fill the IdP login hint from the active profile so the user
// doesn't have to retype their email. Best-effort: we still proceed
// without a hint if the lookup fails.
@@ -408,7 +408,7 @@ func foregroundGetTokenInfo(ctx context.Context, cmd *cobra.Command, config *pro
hint = profileState.Email
}
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isUnixRunningDesktop(), false, hint)
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, util.HasGraphicalSession(), false, hint)
if err != nil {
return nil, err
}
@@ -458,14 +458,6 @@ func openURL(cmd *cobra.Command, verificationURIComplete, userCode string, noBro
}
}
// isUnixRunningDesktop checks if a Linux OS is running desktop environment
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func setEnvAndFlags(cmd *cobra.Command) error {
SetFlagsFromEnvVars(rootCmd)

View File

@@ -21,8 +21,8 @@ import (
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
@@ -626,7 +626,7 @@ func setupLoginRequest(providedSetupKey string, customDNSAddressConverted []byte
NatExternalIPs: natExternalIPs,
CleanNATExternalIPs: natExternalIPs != nil && len(natExternalIPs) == 0,
CustomDNSAddress: customDNSAddressConverted,
IsUnixDesktopClient: isUnixRunningDesktop(),
IsUnixDesktopClient: util.HasGraphicalSession(),
Hostname: hostName,
ExtraIFaceBlacklist: extraIFaceBlackList,
DnsLabels: dnsLabels,

View File

@@ -21,7 +21,7 @@ import (
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
sshcommon "github.com/netbirdio/netbird/client/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
@@ -521,12 +521,7 @@ func (c *Client) VerifySSHHostKey(peerAddress string, key []byte) error {
return err
}
storedKey, found := engine.GetPeerSSHKey(peerAddress)
if !found {
return sshcommon.ErrPeerNotFound
}
return sshcommon.VerifyHostKey(storedKey, key, peerAddress)
return nbssh.PeerKeyLookup(engine.GetPeerSSHKey).VerifySSHHostKey(peerAddress, key)
}
// SetPerformance retunes a running Client. Only PreallocatedBuffersPerPool

View File

@@ -42,6 +42,7 @@ type aclManager struct {
optionalEntries map[string][]entry
ipsetStore *ipsetStore
v6 bool
ipsetSupported bool
stateManager *statemanager.Manager
}
@@ -60,6 +61,8 @@ func newAclManager(iptablesClient *iptables.IPTables, wgIface iFaceMapper) (*acl
func (m *aclManager) init(stateManager *statemanager.Manager) error {
m.stateManager = stateManager
m.ipsetSupported = m.probeIPSetSupport()
m.seedInitialEntries()
m.seedInitialOptionalEntries()
@@ -91,6 +94,12 @@ func (m *aclManager) AddPeerFiltering(
if m.v6 && ipsetName != "" {
ipsetName += "-v6"
}
// When the kernel lacks the required ipset hash module, fall back to
// per-IP iptables rules (pre-0.68 behavior) so ACLs keep working instead
// of silently leaving the chain empty.
if ipsetName != "" && !m.ipsetSupported {
ipsetName = ""
}
proto := protoForFamily(protocol, m.v6)
specs := filterRuleSpecs(ip, proto, sPort, dPort, action, ipsetName)
@@ -498,6 +507,40 @@ func transformIPsetName(ipsetName string, sPort, dPort *firewall.Port, action fi
}
}
// probeIPSetSupport checks whether the kernel can create the ipset type used for
// ACL rules. On kernels lacking the required ipset hash module, ipset creation
// fails (e.g. "invalid argument"), which would otherwise leave the ACL chain
// empty and silently drop all policy-permitted inbound traffic. When unsupported,
// the manager falls back to per-IP iptables rules.
func (m *aclManager) probeIPSetSupport() bool {
// Use a unique name so concurrent processes don't collide and we only ever
// destroy the set we created ourselves. ipset names are limited to 31 chars,
// so use a short random suffix.
probeName := "nb-probe-" + uuid.New().String()[:8]
opts := ipset.CreateOptions{
Replace: true,
}
if m.v6 {
opts.Family = ipset.FamilyIPV6
}
if err := ipset.Create(probeName, ipset.TypeHashNet, opts); err != nil {
log.Warnf("ipset is not available (failed to create probe set: %v); "+
"falling back to per-IP iptables ACL rules. Ensure the kernel provides "+
"the ipset hash:net module (ip_set_hash_net) for better performance with large rule sets", err)
return false
}
defer func() {
if err := ipset.Destroy(probeName); err != nil {
log.Debugf("destroy ipset probe set %q: %v", probeName, err)
}
}()
return true
}
func (m *aclManager) createIPSet(name string) error {
opts := ipset.CreateOptions{
Replace: true,

View File

@@ -0,0 +1,240 @@
//go:build privileged
package iptables
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
fw "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/iface/wgaddr"
)
func iptRefcountIfaceV4() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("10.20.0.1"),
Network: netip.MustParsePrefix("10.20.0.0/24"),
}
},
}
}
func iptRefcountIfaceDual() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("10.20.0.1"),
Network: netip.MustParsePrefix("10.20.0.0/24"),
IPv6: netip.MustParseAddr("fd00::1"),
IPv6Net: netip.MustParsePrefix("fd00::/64"),
}
},
}
}
func newIptRefcountManager(t *testing.T, dual bool) *Manager {
t.Helper()
var ifMock *iFaceMock
if dual {
ifMock = iptRefcountIfaceDual()
} else {
ifMock = iptRefcountIfaceV4()
}
m, err := Create(ifMock, iface.DefaultMTU)
require.NoError(t, err, "create manager")
require.NoError(t, m.Init(nil), "init manager")
t.Cleanup(func() {
require.NoError(t, m.Close(nil), "close manager")
})
return m
}
func iptDnatV4(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("10.20.0.2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
func iptDnatV6(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("fd00::2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
// TestIptablesRouting_RepeatedEnableSingleReference verifies that EnableRouting
// (called on every network-map update) holds at most one reference per family
// and a single DisableRouting drops both back to zero.
func TestIptablesRouting_RepeatedEnableSingleReference(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
require.NoError(t, m.EnableRouting(), "first enable")
require.NoError(t, m.EnableRouting(), "second enable")
require.NoError(t, m.EnableRouting(), "third enable")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "repeated enable holds a single v4 reference")
assert.Equal(t, 1, v6, "repeated enable holds a single v6 reference")
require.NoError(t, m.DisableRouting(), "disable")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "single disable releases the v4 reference")
assert.Equal(t, 0, v6, "single disable releases the v6 reference")
}
// TestIptablesRouting_DisableKeepsDNATReference verifies that an unpaired
// DisableRouting does not release references held by active DNAT rules.
func TestIptablesRouting_DisableKeepsDNATReference(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9095))
require.NoError(t, err, "add v6 dnat")
require.NoError(t, m.DisableRouting(), "unpaired disable")
_, v6 := state.Counts()
assert.Equal(t, 1, v6, "DNAT-held reference survives unpaired DisableRouting")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "delete releases the DNAT reference")
}
// TestIptablesDNAT_RefcountBalancedV4 covers a Balanced Add/Delete pair on v4.
func TestIptablesDNAT_RefcountBalancedV4(t *testing.T) {
m := newIptRefcountManager(t, false)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV4(7081))
require.NoError(t, err, "add v4 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
r2, err := m.AddDNATRule(iptDnatV4(7082))
require.NoError(t, err, "add v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 2, v4, "v4 refcount after second add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r1))
v4, v6 = state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r2))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount after second delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
}
// TestIptablesDNAT_RefcountBalancedV6 checks the v6 path increments v6 only and
// decrements back to zero.
func TestIptablesDNAT_RefcountBalancedV6(t *testing.T) {
m := newIptRefcountManager(t, true)
require.NotNil(t, m.router6, "v6 router")
require.Same(t, m.router.ipFwdState, m.router6.ipFwdState, "shared state")
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9081))
require.NoError(t, err, "add v6 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 1, v6, "v6 refcount after first add")
r2, err := m.AddDNATRule(iptDnatV6(9082))
require.NoError(t, err, "add v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 2, v6, "v6 refcount after second add")
require.NoError(t, m.DeleteDNATRule(r1))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first delete")
require.NoError(t, m.DeleteDNATRule(r2))
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount after second delete")
}
// TestIptablesDNAT_DuplicateAddNoLeak verifies the duplicate-rule path returns
// without bumping the refcount.
func TestIptablesDNAT_DuplicateAddNoLeak(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
rule := iptDnatV4(7083)
r1, err := m.AddDNATRule(rule)
require.NoError(t, err)
v4, _ := state.Counts()
assert.Equal(t, 1, v4)
_, err = m.AddDNATRule(rule)
require.NoError(t, err, "duplicate add")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "duplicate add must not increment")
require.NoError(t, m.DeleteDNATRule(r1))
v4, _ = state.Counts()
assert.Equal(t, 0, v4, "single delete must drop to zero")
}
// TestIptablesDNAT_DeleteMissingNoUnderflow verifies Delete on an unknown rule
// neither errors nor releases the refcount.
func TestIptablesDNAT_DeleteMissingNoUnderflow(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
phantom := iptDnatV4(7099)
require.NoError(t, m.DeleteDNATRule(&phantom), "delete missing v4")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6)
phantom6 := iptDnatV6(9099)
require.NoError(t, m.DeleteDNATRule(&phantom6), "delete missing v6")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6)
r1, err := m.AddDNATRule(iptDnatV4(7100))
require.NoError(t, err)
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "real add still increments after phantom delete")
require.NoError(t, m.DeleteDNATRule(r1))
}
// TestIptablesDNAT_DoubleDeleteNoUnderflow verifies a second Delete on the same
// rule is a no-op.
func TestIptablesDNAT_DoubleDeleteNoUnderflow(t *testing.T) {
m := newIptRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(iptDnatV6(9083))
require.NoError(t, err)
_, v6 := state.Counts()
assert.Equal(t, 1, v6)
require.NoError(t, m.DeleteDNATRule(r1), "first delete")
_, v6 = state.Counts()
assert.Equal(t, 0, v6)
require.NoError(t, m.DeleteDNATRule(r1), "second delete must be no-op")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "double delete must not underflow")
}

View File

@@ -89,7 +89,7 @@ func (m *Manager) createIPv6Components(wgIface iFaceMapper, mtu uint16) error {
}
// Share the same IP forwarding state with the v4 router, since
// EnableIPForwarding controls both v4 and v6 sysctls.
// Forwarding refcounter is per-family but shared between v4 and v6 routers.
m.router6.ipFwdState = m.router.ipFwdState
m.aclMgr6, err = newAclManager(ip6Client, wgIface)
@@ -402,17 +402,12 @@ func (m *Manager) SetLogLevel(log.Level) {
}
func (m *Manager) EnableRouting() error {
if err := m.router.ipFwdState.RequestForwarding(); err != nil {
return fmt.Errorf("enable IP forwarding: %w", err)
}
return nil
// v6 only when the overlay actually has v6.
return m.router.ipFwdState.RequestRouting(m.router6 != nil)
}
func (m *Manager) DisableRouting() error {
if err := m.router.ipFwdState.ReleaseForwarding(); err != nil {
return fmt.Errorf("disable IP forwarding: %w", err)
}
return nil
return m.router.ipFwdState.ReleaseRouting()
}
// AddDNATRule adds a DNAT rule

View File

@@ -291,3 +291,40 @@ func TestIptablesCreatePerformance(t *testing.T) {
})
}
}
// TestIptablesACLIPSetFallback verifies that when the kernel lacks ipset support,
// the ACL manager falls back to per-IP iptables rules (-s <ip>) instead of
// silently leaving the chain empty. See discussion #6125.
func TestIptablesACLIPSetFallback(t *testing.T) {
ipv4Client, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err)
// Use Create()/Init() so the router-owned chains (chainRTFWDIN/OUT) are
// created before the ACL manager's createDefaultChains() references them.
manager, err := Create(ifaceMock, iface.DefaultMTU)
require.NoError(t, err)
require.NoError(t, manager.Init(nil))
aclMgr := manager.aclMgr
// Simulate a kernel without the ipset hash module.
aclMgr.ipsetSupported = false
defer func() {
require.NoError(t, manager.Close(nil))
}()
ip := netip.MustParseAddr("10.20.0.42")
port := &fw.Port{Values: []uint16{22}}
rules, err := aclMgr.AddPeerFiltering(nil, ip.AsSlice(), "tcp", nil, port, fw.ActionAccept, "nb0000001")
require.NoError(t, err, "AddPeerFiltering should succeed via fallback")
require.NotEmpty(t, rules)
rule := rules[0].(*Rule)
require.Empty(t, rule.ipsetName, "fallback rule must not reference an ipset")
require.Contains(t, strings.Join(rule.specs, " "), "-s 10.20.0.42", "fallback rule must match by source IP")
require.NotContains(t, strings.Join(rule.specs, " "), "--match-set", "fallback rule must not use ipset matching")
// The rule must actually be present in the ACL chain (not silently dropped).
checkRuleSpecs(t, ipv4Client, rule.chain, true, rule.specs...)
}

View File

@@ -102,7 +102,7 @@ func newRouter(iptablesClient *iptables.IPTables, wgIface iFaceMapper, mtu uint1
wgIface: wgIface,
mtu: mtu,
v6: iptablesClient.Proto() == iptables.ProtocolIPv6,
ipFwdState: ipfwdstate.NewIPForwardingState(),
ipFwdState: ipfwdstate.NewIPForwardingState(wgIface.Name()),
}
r.ipsetCounter = refcounter.New(
@@ -770,10 +770,6 @@ func (r *router) updateState() {
}
func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
if err := r.ipFwdState.RequestForwarding(); err != nil {
return nil, err
}
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey+dnatSuffix]; exists {
return rule, nil
@@ -840,18 +836,34 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
for key, ruleInfo := range rules {
if err := r.iptablesClient.Append(ruleInfo.table, ruleInfo.chain, ruleInfo.rule...); err != nil {
if rollbackErr := r.rollbackRules(rules); rollbackErr != nil {
log.Errorf("rollback failed: %v", rollbackErr)
}
r.cleanupFailedDNATAdd(rules)
return nil, fmt.Errorf("add rule %s: %w", key, err)
}
r.rules[key] = ruleInfo.rule
}
if err := r.ipFwdState.RequestForwarding(r.v6); err != nil {
r.cleanupFailedDNATAdd(rules)
return nil, fmt.Errorf("enable forwarding: %w", err)
}
r.updateState()
return rule, nil
}
// cleanupFailedDNATAdd removes the bookkeeping written by a partially applied
// AddDNATRule before rolling back the kernel rules, so no entries remain that
// never got a forwarding refcount. rollbackRules re-adds entries it failed to
// remove from the kernel.
func (r *router) cleanupFailedDNATAdd(rules map[string]ruleInfo) {
for key := range rules {
delete(r.rules, key)
}
if err := r.rollbackRules(rules); err != nil {
log.Errorf("rollback failed: %v", err)
}
}
func (r *router) rollbackRules(rules map[string]ruleInfo) error {
var merr *multierror.Error
for key, ruleInfo := range rules {
@@ -868,32 +880,47 @@ func (r *router) rollbackRules(rules map[string]ruleInfo) error {
}
func (r *router) DeleteDNATRule(rule firewall.Rule) error {
if err := r.ipFwdState.ReleaseForwarding(); err != nil {
log.Errorf("%v", err)
}
ruleKey := rule.ID()
_, hadDNAT := r.rules[ruleKey+dnatSuffix]
_, hadSNAT := r.rules[ruleKey+snatSuffix]
_, hadFWD := r.rules[ruleKey+fwdSuffix]
if !hadDNAT && !hadSNAT && !hadFWD {
return nil
}
var merr *multierror.Error
if dnatRule, exists := r.rules[ruleKey+dnatSuffix]; exists {
if err := r.iptablesClient.Delete(tableNat, chainRTRDR, dnatRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete DNAT rule: %w", err))
} else {
delete(r.rules, ruleKey+dnatSuffix)
}
delete(r.rules, ruleKey+dnatSuffix)
}
if snatRule, exists := r.rules[ruleKey+snatSuffix]; exists {
if err := r.iptablesClient.Delete(tableNat, chainRTNAT, snatRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete SNAT rule: %w", err))
} else {
delete(r.rules, ruleKey+snatSuffix)
}
delete(r.rules, ruleKey+snatSuffix)
}
if fwdRule, exists := r.rules[ruleKey+fwdSuffix]; exists {
if err := r.iptablesClient.Delete(tableFilter, chainRTFWDOUT, fwdRule...); err != nil {
merr = multierror.Append(merr, fmt.Errorf("delete forward rule: %w", err))
} else {
delete(r.rules, ruleKey+fwdSuffix)
}
}
// Release the refcount only once all rules are gone from the kernel. On
// partial failure the failed entries stay in r.rules so a retry can remove
// them and release then.
if merr == nil {
if err := r.ipFwdState.ReleaseForwarding(r.v6); err != nil {
log.Errorf("%v", err)
}
delete(r.rules, ruleKey+fwdSuffix)
}
r.updateState()

View File

@@ -0,0 +1,249 @@
//go:build privileged
package nftables
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
fw "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/iface"
"github.com/netbirdio/netbird/client/iface/wgaddr"
)
func nftRefcountIfaceV4() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("100.96.0.1"),
Network: netip.MustParsePrefix("100.96.0.0/16"),
}
},
}
}
func nftRefcountIfaceDual() *iFaceMock {
return &iFaceMock{
NameFunc: func() string { return "wt-refcount" },
AddressFunc: func() wgaddr.Address {
return wgaddr.Address{
IP: netip.MustParseAddr("100.96.0.1"),
Network: netip.MustParsePrefix("100.96.0.0/16"),
IPv6: netip.MustParseAddr("fd00::1"),
IPv6Net: netip.MustParsePrefix("fd00::/64"),
}
},
}
}
func newNftRefcountManager(t *testing.T, dual bool) *Manager {
t.Helper()
if check() != NFTABLES {
t.Skip("nftables not supported on this system")
}
var ifMock *iFaceMock
if dual {
ifMock = nftRefcountIfaceDual()
} else {
ifMock = nftRefcountIfaceV4()
}
m, err := Create(ifMock, iface.DefaultMTU)
require.NoError(t, err, "create manager")
require.NoError(t, m.Init(nil), "init manager")
t.Cleanup(func() {
require.NoError(t, m.Close(nil), "close manager")
})
return m
}
func dnatV4(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("100.96.0.2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
func dnatV6(port uint16) fw.ForwardRule {
return fw.ForwardRule{
Protocol: fw.ProtocolTCP,
DestinationPort: fw.Port{Values: []uint16{port}},
TranslatedAddress: netip.MustParseAddr("fd00::2"),
TranslatedPort: fw.Port{Values: []uint16{80}},
}
}
// TestNftablesDNAT_RefcountBalancedV4 verifies that Add/Delete pairs leave the
// v4 refcount at zero.
func TestNftablesDNAT_RefcountBalancedV4(t *testing.T) {
m := newNftRefcountManager(t, false)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV4(8081))
require.NoError(t, err, "add v4 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
r2, err := m.AddDNATRule(dnatV4(8082))
require.NoError(t, err, "add v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 2, v4, "v4 refcount after second add")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r1), "delete v4 dnat 1")
v4, v6 = state.Counts()
assert.Equal(t, 1, v4, "v4 refcount after first delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
require.NoError(t, m.DeleteDNATRule(r2), "delete v4 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount after second delete")
assert.Equal(t, 0, v6, "v6 refcount unchanged")
}
// TestNftablesDNAT_RefcountBalancedV6 verifies the v6 path increments v6 only
// and decrements back to zero on Delete.
func TestNftablesDNAT_RefcountBalancedV6(t *testing.T) {
m := newNftRefcountManager(t, true)
require.NotNil(t, m.router6, "v6 router")
require.Same(t, m.router.ipFwdState, m.router6.ipFwdState, "shared state")
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9091))
require.NoError(t, err, "add v6 dnat 1")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first add")
r2, err := m.AddDNATRule(dnatV6(9092))
require.NoError(t, err, "add v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 2, v6, "v6 refcount after second add")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat 1")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unchanged")
assert.Equal(t, 1, v6, "v6 refcount after first delete")
require.NoError(t, m.DeleteDNATRule(r2), "delete v6 dnat 2")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount after second delete")
}
// TestNftablesDNAT_DuplicateAddNoLeak verifies that a duplicate Add (same
// ForwardRule) does not double-increment the refcount.
func TestNftablesDNAT_DuplicateAddNoLeak(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
rule := dnatV4(8083)
r1, err := m.AddDNATRule(rule)
require.NoError(t, err, "add v4 dnat")
v4, _ := state.Counts()
assert.Equal(t, 1, v4)
// duplicate add: same rule ID, must be a no-op for the refcount.
_, err = m.AddDNATRule(rule)
require.NoError(t, err, "duplicate add")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "duplicate add must not increment")
require.NoError(t, m.DeleteDNATRule(r1), "delete v4 dnat")
v4, _ = state.Counts()
assert.Equal(t, 0, v4, "single delete must drop to zero")
}
// TestNftablesDNAT_DeleteMissingNoUnderflow verifies deleting a rule that was
// never added does not underflow the refcount.
func TestNftablesDNAT_DeleteMissingNoUnderflow(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
// Construct a Rule reference for something never added. The router stores
// rules by ID(), and DeleteDNATRule looks them up in r.rules; a missing
// entry must be a no-op rather than calling Release.
phantom := dnatV4(8099)
require.NoError(t, m.DeleteDNATRule(&phantom), "delete missing v4 dnat")
v4, v6 := state.Counts()
assert.Equal(t, 0, v4, "v4 refcount unaffected by missing delete")
assert.Equal(t, 0, v6, "v6 refcount unaffected")
phantom6 := dnatV6(9099)
require.NoError(t, m.DeleteDNATRule(&phantom6), "delete missing v6 dnat")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4)
assert.Equal(t, 0, v6, "v6 refcount unaffected by missing delete")
// And after a phantom delete, a real add still results in count=1.
r1, err := m.AddDNATRule(dnatV4(8100))
require.NoError(t, err, "add v4 dnat after phantom delete")
v4, _ = state.Counts()
assert.Equal(t, 1, v4, "real add still increments after phantom delete")
require.NoError(t, m.DeleteDNATRule(r1))
}
// TestNftablesRouting_RepeatedEnableSingleReference verifies that EnableRouting
// (called on every network-map update) holds at most one reference per family
// and a single DisableRouting drops both back to zero.
func TestNftablesRouting_RepeatedEnableSingleReference(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
require.NoError(t, m.EnableRouting(), "first enable")
require.NoError(t, m.EnableRouting(), "second enable")
require.NoError(t, m.EnableRouting(), "third enable")
v4, v6 := state.Counts()
assert.Equal(t, 1, v4, "repeated enable holds a single v4 reference")
assert.Equal(t, 1, v6, "repeated enable holds a single v6 reference")
require.NoError(t, m.DisableRouting(), "disable")
v4, v6 = state.Counts()
assert.Equal(t, 0, v4, "single disable releases the v4 reference")
assert.Equal(t, 0, v6, "single disable releases the v6 reference")
}
// TestNftablesRouting_DisableKeepsDNATReference verifies that an unpaired
// DisableRouting does not release references held by active DNAT rules.
func TestNftablesRouting_DisableKeepsDNATReference(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9095))
require.NoError(t, err, "add v6 dnat")
require.NoError(t, m.DisableRouting(), "unpaired disable")
_, v6 := state.Counts()
assert.Equal(t, 1, v6, "DNAT-held reference survives unpaired DisableRouting")
require.NoError(t, m.DeleteDNATRule(r1), "delete v6 dnat")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "delete releases the DNAT reference")
}
// TestNftablesDNAT_DoubleDeleteNoUnderflow verifies that deleting the same rule
// twice does not underflow the refcount (the second delete is a no-op).
func TestNftablesDNAT_DoubleDeleteNoUnderflow(t *testing.T) {
m := newNftRefcountManager(t, true)
state := m.router.ipFwdState
r1, err := m.AddDNATRule(dnatV6(9093))
require.NoError(t, err)
_, v6 := state.Counts()
assert.Equal(t, 1, v6)
require.NoError(t, m.DeleteDNATRule(r1), "first delete")
_, v6 = state.Counts()
assert.Equal(t, 0, v6)
require.NoError(t, m.DeleteDNATRule(r1), "second delete must be no-op")
_, v6 = state.Counts()
assert.Equal(t, 0, v6, "double delete must not underflow")
}

View File

@@ -105,8 +105,8 @@ func (m *Manager) createIPv6Components(tableName string, wgIface iFaceMapper, mt
return fmt.Errorf("create v6 router: %w", err)
}
// Share the same IP forwarding state with the v4 router, since
// EnableIPForwarding controls both v4 and v6 sysctls.
// Share the per-family forwarding refcounter with the v4 router so a v4
// rule and a v6 rule against the same state machine cooperate cleanly.
m.router6.ipFwdState = m.router.ipFwdState
m.aclManager6, err = newAclManager(workTable6, wgIface, chainNameRoutingFw)
@@ -530,17 +530,12 @@ func (m *Manager) SetLogLevel(log.Level) {
}
func (m *Manager) EnableRouting() error {
if err := m.router.ipFwdState.RequestForwarding(); err != nil {
return fmt.Errorf("enable IP forwarding: %w", err)
}
return nil
// v6 only when the overlay actually has v6.
return m.router.ipFwdState.RequestRouting(m.router6 != nil)
}
func (m *Manager) DisableRouting() error {
if err := m.router.ipFwdState.ReleaseForwarding(); err != nil {
return fmt.Errorf("disable IP forwarding: %w", err)
}
return nil
return m.router.ipFwdState.ReleaseRouting()
}
// Flush rule/chain/set operations from the buffer

View File

@@ -93,7 +93,7 @@ func newRouter(workTable *nftables.Table, wgIface iFaceMapper, mtu uint16) (*rou
rules: make(map[string]*nftables.Rule),
af: familyForAddr(workTable.Family == nftables.TableFamilyIPv4),
wgIface: wgIface,
ipFwdState: ipfwdstate.NewIPForwardingState(),
ipFwdState: ipfwdstate.NewIPForwardingState(wgIface.Name()),
mtu: mtu,
}
@@ -1553,10 +1553,6 @@ func (r *router) refreshRulesMap() error {
}
func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
if err := r.ipFwdState.RequestForwarding(); err != nil {
return nil, err
}
ruleKey := rule.ID()
if _, exists := r.rules[ruleKey+dnatSuffix]; exists {
return rule, nil
@@ -1567,7 +1563,18 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
return nil, fmt.Errorf("convert protocol to number: %w", err)
}
// Request forwarding before queueing rules: addDnatRedirect/addDnatMasq
// buffer netlink messages on r.conn that the next caller's Flush would
// commit if we returned without flushing them ourselves.
v6 := r.af.tableFamily == nftables.TableFamilyIPv6
if err := r.ipFwdState.RequestForwarding(v6); err != nil {
return nil, fmt.Errorf("enable forwarding: %w", err)
}
if err := r.addDnatRedirect(rule, protoNum, ruleKey); err != nil {
if rerr := r.ipFwdState.ReleaseForwarding(v6); rerr != nil {
log.Warnf("rollback forwarding refcount: %v", rerr)
}
return nil, err
}
@@ -1579,6 +1586,11 @@ func (r *router) AddDNATRule(rule firewall.ForwardRule) (firewall.Rule, error) {
// TODO: find chains with drop policies and add rules there
if err := r.conn.Flush(); err != nil {
if rerr := r.ipFwdState.ReleaseForwarding(v6); rerr != nil {
log.Warnf("rollback forwarding refcount: %v", rerr)
}
delete(r.rules, ruleKey+dnatSuffix)
delete(r.rules, ruleKey+snatSuffix)
return nil, fmt.Errorf("flush rules: %w", err)
}
@@ -1781,16 +1793,18 @@ func (r *router) addDnatMasq(rule firewall.ForwardRule, protoNum uint8, ruleKey
}
func (r *router) DeleteDNATRule(rule firewall.Rule) error {
if err := r.ipFwdState.ReleaseForwarding(); err != nil {
log.Errorf("%v", err)
}
ruleKey := rule.ID()
if err := r.refreshRulesMap(); err != nil {
return fmt.Errorf(refreshRulesMapError, err)
}
_, hadDNAT := r.rules[ruleKey+dnatSuffix]
_, hadSNAT := r.rules[ruleKey+snatSuffix]
if !hadDNAT && !hadSNAT {
return nil
}
var merr *multierror.Error
var needsFlush bool
@@ -1822,9 +1836,16 @@ func (r *router) DeleteDNATRule(rule firewall.Rule) error {
}
}
// Release the refcount only once the rules are gone from the kernel. On
// failure (including the refreshRulesMap error above) the rules and their
// map entries remain, keeping forwarding on until a retry removes them.
if merr == nil {
delete(r.rules, ruleKey+dnatSuffix)
delete(r.rules, ruleKey+snatSuffix)
if err := r.ipFwdState.ReleaseForwarding(r.af.tableFamily == nftables.TableFamilyIPv6); err != nil {
log.Errorf("%v", err)
}
}
return nberrors.FormatErrorOrNil(merr)

View File

@@ -22,8 +22,6 @@
!define UI_REG_APP_PATH "Software\Microsoft\Windows\CurrentVersion\App Paths\${UI_APP_EXE}"
!define UI_UNINSTALL_PATH "Software\Microsoft\Windows\CurrentVersion\Uninstall\${UI_APP_NAME}"
!define AUTOSTART_REG_KEY "Software\Microsoft\Windows\CurrentVersion\Run"
!define NETBIRD_DATA_DIR "$COMMONPROGRAMDATA\Netbird"
Unicode True
@@ -228,13 +226,6 @@ WriteRegStr ${REG_ROOT} "${UNINSTALL_PATH}" "Publisher" "${COMP_NAME}"
WriteRegStr ${REG_ROOT} "${UI_REG_APP_PATH}" "" "$INSTDIR\${UI_APP_EXE}"
; Autostart is owned by the UI's per-user setting (HKCU\...\Run via Wails),
; not the installer. Drop the machine-wide entry older installers wrote so the
; toggle is the single source of truth. HKCU is left untouched -- it may hold
; the user's own toggle state, which must survive upgrades.
DetailPrint "Removing installer-managed autostart registry entry if present..."
DeleteRegValue HKLM "${AUTOSTART_REG_KEY}" "${APP_NAME}"
EnVar::SetHKLM
EnVar::AddValueEx "path" "$INSTDIR"
@@ -299,15 +290,6 @@ ExecWait '"$INSTDIR\${MAIN_APP_EXE}" service uninstall'
DetailPrint "Terminating Netbird UI process..."
ExecWait `taskkill /im ${UI_APP_EXE}.exe /f`
; Remove autostart registry entries
DetailPrint "Removing autostart registry entries if they exist..."
; Legacy machine-wide entry written by older installers.
DeleteRegValue HKLM "${AUTOSTART_REG_KEY}" "${APP_NAME}"
; Per-user entry the UI toggle writes via Wails (value name is the lowercase
; app-name slug). Uninstall removes the app, so drop it too.
DeleteRegValue HKCU "${AUTOSTART_REG_KEY}" "${APP_NAME}"
DeleteRegValue HKCU "${AUTOSTART_REG_KEY}" "netbird"
; Handle data deletion based on checkbox
DetailPrint "Checking if user requested data deletion..."
${If} $DeleteDataEnabled == "1"

View File

@@ -138,26 +138,37 @@ func (a *Auth) IsSSOSupported(ctx context.Context) (bool, error) {
// GetOAuthFlow returns an OAuth flow (PKCE or Device) using the existing management connection
// This avoids creating a new connection to the management server
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool) (OAuthFlow, error) {
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool, hint string) (OAuthFlow, error) {
var flow OAuthFlow
var err error
err = a.withRetry(ctx, func(client *mgm.GrpcClient) error {
err := a.withRetry(ctx, func(client *mgm.GrpcClient) error {
if forceDeviceAuth {
flow, err = a.getDeviceFlow(client)
return err
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
// Try PKCE flow first
flow, err = a.getPKCEFlow(client)
pkceFlow, err := a.getPKCEFlow(client)
if err != nil {
// If PKCE not supported, try Device flow
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
flow, err = a.getDeviceFlow(client)
return err
deviceFlow, err := a.getDeviceFlow(client)
if err != nil {
return err
}
deviceFlow.SetLoginHint(hint)
flow = deviceFlow
return nil
}
return err
}
pkceFlow.SetLoginHint(hint)
flow = pkceFlow
return nil
})

View File

@@ -97,9 +97,7 @@ func authenticateWithPKCEFlow(ctx context.Context, config *profilemanager.Config
return nil, fmt.Errorf("getting pkce authorization flow info failed with error: %v", err)
}
if hint != "" {
pkceFlowInfo.SetLoginHint(hint)
}
pkceFlowInfo.SetLoginHint(hint)
return pkceFlowInfo, nil
}
@@ -127,9 +125,7 @@ func authenticateWithDeviceCodeFlow(ctx context.Context, config *profilemanager.
}
}
if hint != "" {
deviceFlowInfo.SetLoginHint(hint)
}
deviceFlowInfo.SetLoginHint(hint)
return deviceFlowInfo, nil
}

View File

@@ -51,6 +51,7 @@ nftables.txt: Anonymized nftables rules with packet counters across all families
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
config.txt: Anonymized configuration information of the NetBird client.
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
@@ -237,6 +238,13 @@ scutil_dns.txt (macOS only):
- Shows DNS configuration for all network interfaces
- Includes search domains, nameservers, and DNS resolver settings
- All IP addresses and domain names are anonymized
dns_windows.txt (Windows only):
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
- All IP addresses and domain names are anonymized
`
const (

View File

@@ -844,6 +844,10 @@ func collectSysctls() string {
[]string{"net.ipv4.conf.all.src_valid_mark", "net.ipv4.conf.default.src_valid_mark"},
listInterfaceSysctls("ipv4", "src_valid_mark")...,
))
writeSysctlGroup(&builder, "accept_ra", append(
[]string{"net.ipv6.conf.all.accept_ra", "net.ipv6.conf.default.accept_ra"},
listInterfaceSysctls("ipv6", "accept_ra")...,
))
writeSysctlGroup(&builder, "conntrack", []string{
"net.netfilter.nf_conntrack_acct",
"net.netfilter.nf_conntrack_tcp_loose",

View File

@@ -1,4 +1,4 @@
//go:build !unix
//go:build !unix && !windows
package debug

View File

@@ -0,0 +1,443 @@
//go:build windows
package debug
import (
"encoding/hex"
"errors"
"fmt"
"net/netip"
"strings"
"unsafe"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
nbdns "github.com/netbirdio/netbird/client/internal/dns"
)
const dnsInfoFileName = "dns_windows.txt"
const (
gpoDNSClientRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient`
tcpipParamsPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters`
dnscacheParams = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters`
)
// interfaceDNSValues are the per-interface values that decide how a name is
// resolved and registered. Everything the DNS host manager writes is in here,
// so a bundle shows both what we set and what it replaced.
var interfaceDNSValues = []string{
"NameServer",
"DhcpNameServer",
"Domain",
"DhcpDomain",
"SearchList",
"RegistrationEnabled",
"DisableDynamicUpdate",
"MaxNumberOfAddressesToRegister",
"EnableDHCP",
}
// addDNSInfo collects and adds DNS configuration information to the archive
func (g *BundleGenerator) addDNSInfo() error {
if err := g.addFileToZip(strings.NewReader(g.collectDNSInfo()), dnsInfoFileName); err != nil {
return fmt.Errorf("add DNS info to zip: %w", err)
}
return nil
}
// collectDNSInfo renders the report. Everything below it reaches the platform
// through COM and through lazily resolved procedures, which panic when a
// procedure is missing rather than returning an error, and a debug bundle is not
// allowed to take the daemon down. The panic is contained here, and whatever was
// collected before it is kept and reported with it.
func (g *BundleGenerator) collectDNSInfo() (content string) {
var sb strings.Builder
defer func() {
if r := recover(); r != nil {
log.Errorf("collecting Windows DNS configuration panicked: %v", r)
fmt.Fprintf(&sb, "\nerror: collection stopped: %v\n", r)
}
content = sb.String()
}()
sb.WriteString("Windows DNS configuration\n")
sb.WriteString("=========================\n")
adapters, adaptersErr := adapterAddresses()
g.writeNRPTRules(&sb, "NRPT rules, local policy store", nbdns.DNSPolicyConfigRoot)
g.writeNRPTRules(&sb, "NRPT rules, group policy store", nbdns.GPODNSPolicyConfigRoot)
g.writeEffectiveNRPTPolicies(&sb)
g.writeRegistryKey(&sb, "DNS client group policy", gpoDNSClientRoot)
g.writeRegistryKey(&sb, "Global TCP/IP parameters", tcpipParamsPath)
g.writeRegistryKey(&sb, "Dnscache parameters", dnscacheParams)
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv4", nbdns.InterfaceConfigPath, adapterNames(adapters))
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv6", nbdns.InterfaceConfigPathV6, adapterNames(adapters))
g.writeAdapterDNS(&sb, adapters, adaptersErr)
return sb.String()
}
// writeNRPTRules lists every rule in a policy store, ours and any other
// product's, since a foreign rule for the same namespace decides resolution
// just as ours does. Rules the client wrote are marked.
func (g *BundleGenerator) writeNRPTRules(sb *strings.Builder, title, root string) {
writeSection(sb, title, root)
names, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(names) == 0 {
sb.WriteString("no rules\n")
return
}
for _, name := range names {
owner := ""
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(nbdns.NRPTKeyPrefix)) {
owner = " (netbird)"
}
fmt.Fprintf(sb, "%s%s\n", name, owner)
g.writeValues(sb, root+`\`+name, nil, " ")
}
}
// writeEffectiveNRPTPolicies reports the table the resolver answers from, which
// the registry cannot show: a rule is written before it is loaded, and it keeps
// being enforced after its key is gone until the resolver reloads its policy.
func (g *BundleGenerator) writeEffectiveNRPTPolicies(sb *strings.Builder) {
writeSection(sb, "NRPT policy table in effect", nrptPolicyClass+"."+nrptPolicyMethod+" in "+nrptPolicyNamespace)
entries, err := effectiveNRPTPolicies()
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
if len(entries) == 0 {
sb.WriteString("no policies\n")
return
}
for _, entry := range entries {
fmt.Fprintf(sb, "%s\n", g.anonymizeValue("Namespace", entry.namespace))
for _, value := range entry.values {
fmt.Fprintf(sb, " %s: %s\n", value.name, g.anonymizeValue(value.name, value.value))
}
}
}
// writeInterfaceDNS reports the DNS values of every interface that has any, so
// the netbird interface can be compared against the physical ones. The registry
// keys the values by GUID, so each is named from the adapter list; a GUID with
// no adapter is a leftover key of an interface that no longer exists.
func (g *BundleGenerator) writeInterfaceDNS(sb *strings.Builder, title, root string, names map[string]string) {
writeSection(sb, title, root)
guids, err := subKeyNames(root)
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
var reported int
for _, guid := range guids {
var iface strings.Builder
g.writeValues(&iface, root+`\`+guid, interfaceDNSValues, " ")
if iface.Len() == 0 {
continue
}
name, ok := names[strings.ToLower(guid)]
if !ok {
name = "no adapter with this GUID"
}
reported++
fmt.Fprintf(sb, "%s (%s)\n%s", guid, name, iface.String())
}
if reported == 0 {
sb.WriteString("no interface holds DNS values\n")
}
}
// writeRegistryKey reports the values of a single key, without its subkeys.
func (g *BundleGenerator) writeRegistryKey(sb *strings.Builder, title, path string) {
writeSection(sb, title, path)
var values strings.Builder
g.writeValues(&values, path, nil, "")
if values.Len() == 0 {
sb.WriteString("no values\n")
return
}
sb.WriteString(values.String())
}
// writeValues renders the values of a key. A nil names list reports every
// value, otherwise only those named and present.
func (g *BundleGenerator) writeValues(sb *strings.Builder, path string, names []string, indent string) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, path, registry.QUERY_VALUE)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, windows.ERROR_PATH_NOT_FOUND):
// an absent key is the normal state for the GPO store and for
// interfaces without DNS settings
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", path)
return
case err != nil:
fmt.Fprintf(sb, "%serror: open HKEY_LOCAL_MACHINE\\%s: %v\n", indent, path, err)
return
}
defer closeKey(k)
if names == nil {
names, err = k.ReadValueNames(-1)
if err != nil {
fmt.Fprintf(sb, "%serror: read value names: %v\n", indent, err)
return
}
}
for _, name := range names {
value, err := readRegistryValue(k, name)
switch {
case errors.Is(err, registry.ErrNotExist):
// the caller asks for a fixed set of values, most of which a
// given interface does not carry
continue
case err != nil:
// report rather than omit: a value that is there but cannot be
// read reads as unset otherwise
fmt.Fprintf(sb, "%s%s: error: %v\n", indent, name, err)
continue
}
fmt.Fprintf(sb, "%s%s: %s\n", indent, name, g.anonymizeValue(name, value))
}
}
// anonymizeValue redacts a registry value according to what its name says it
// holds. Domains and addresses are handled per entry rather than by the string
// pass: the pass only replaces domains something else in the bundle already
// seeded, and its address regex would eat the digit labels of a reverse zone.
func (g *BundleGenerator) anonymizeValue(name, value string) string {
if !g.anonymize || value == "" {
return value
}
switch {
case holdsDomains(name):
return joinValueEntries(splitValueEntries(value), g.anonymizeDomain)
case holdsAddresses(name):
return joinValueEntries(splitValueEntries(value), g.anonymizer.AnonymizeIPString)
default:
return g.anonymizer.AnonymizeString(value)
}
}
// holdsDomains reports whether a value name holds domains: the domain list of
// an NRPT rule (Name) or of the policy table (Namespace), a search list, the
// DNS suffix values of the TCP/IP and policy keys, which all end in "Domain"
// (Domain, DhcpDomain, NV Domain, ICSDomain), and a proxy host name.
func holdsDomains(name string) bool {
lower := strings.ToLower(name)
return lower == "name" || lower == "namespace" || lower == "searchlist" ||
strings.HasSuffix(lower, "domain") || strings.HasSuffix(lower, "proxyname")
}
// holdsAddresses reports whether a value name holds DNS server addresses
// (NameServer, DhcpNameServer, GenericDNSServers, NameServers).
func holdsAddresses(name string) bool {
lower := strings.ToLower(name)
return strings.Contains(lower, "nameserver") || strings.Contains(lower, "dnsserver")
}
// adapterNames maps adapter GUIDs, as the registry keys the interfaces, to the
// names an operator sees.
func adapterNames(adapters []*windows.IpAdapterAddresses) map[string]string {
names := make(map[string]string, len(adapters))
for _, adapter := range adapters {
guid := windows.BytePtrToString(adapter.AdapterName)
names[strings.ToLower(guid)] = windows.UTF16PtrToString(adapter.FriendlyName)
}
return names
}
// writeAdapterDNS reports the resolver configuration in effect per adapter,
// which is what the resolver uses for a name no NRPT rule matches.
func (g *BundleGenerator) writeAdapterDNS(sb *strings.Builder, adapters []*windows.IpAdapterAddresses, err error) {
writeSection(sb, "Adapter DNS configuration", "GetAdaptersAddresses")
if err != nil {
fmt.Fprintf(sb, "error: %v\n", err)
return
}
for _, adapter := range adapters {
name := windows.UTF16PtrToString(adapter.FriendlyName)
suffix := g.anonymizeDomain(windows.UTF16PtrToString(adapter.DnsSuffix))
fmt.Fprintf(sb, "%s (index %d, oper status %d)\n", name, adapter.IfIndex, adapter.OperStatus)
fmt.Fprintf(sb, " DNS suffix: %s\n", suffix)
var servers []string
for server := adapter.FirstDnsServerAddress; server != nil; server = server.Next {
addr, ok := netip.AddrFromSlice(server.Address.IP())
if !ok {
continue
}
addr = addr.Unmap()
if g.anonymize {
addr = g.anonymizer.AnonymizeIP(addr)
}
servers = append(servers, addr.String())
}
fmt.Fprintf(sb, " DNS servers: %s\n", strings.Join(servers, ", "))
}
}
// anonymizeDomain anonymizes a single domain, keeping the leading dot an NRPT
// match domain carries.
func (g *BundleGenerator) anonymizeDomain(entry string) string {
if !g.anonymize {
return entry
}
domain, dot := strings.CutPrefix(entry, ".")
if domain == "" {
return entry
}
anonymized := g.anonymizer.AnonymizeDomain(domain)
if dot {
anonymized = "." + anonymized
}
return anonymized
}
// splitValueEntries splits a registry value that holds a list. The separator
// differs per value: a REG_MULTI_SZ arrives joined with ", ", a SearchList is
// comma separated and a NameServer may use commas or spaces.
func splitValueEntries(value string) []string {
return strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == ';' || r == ' ' || r == '\t'
})
}
func joinValueEntries(entries []string, anonymize func(string) string) string {
for i, entry := range entries {
entries[i] = anonymize(entry)
}
return strings.Join(entries, ", ")
}
func writeSection(sb *strings.Builder, title, source string) {
fmt.Fprintf(sb, "\n%s\n%s\n%s\n", title, strings.Repeat("-", len(title)), source)
}
func subKeyNames(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
if err != nil {
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closeKey(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
return names, nil
}
// readRegistryValue renders a value as text regardless of its type, so an
// unexpected type in a policy key still shows up instead of being dropped.
func readRegistryValue(k registry.Key, name string) (string, error) {
_, valueType, err := k.GetValue(name, nil)
if err != nil {
return "", fmt.Errorf("get value %s: %w", name, err)
}
switch valueType {
case registry.SZ, registry.EXPAND_SZ:
value, _, err := k.GetStringValue(name)
if err != nil {
return "", fmt.Errorf("get string value %s: %w", name, err)
}
return value, nil
case registry.MULTI_SZ:
values, _, err := k.GetStringsValue(name)
if err != nil {
return "", fmt.Errorf("get strings value %s: %w", name, err)
}
return strings.Join(values, ", "), nil
case registry.DWORD, registry.QWORD:
value, _, err := k.GetIntegerValue(name)
if err != nil {
return "", fmt.Errorf("get integer value %s: %w", name, err)
}
return fmt.Sprintf("%d (0x%x)", value, value), nil
case registry.BINARY:
value, _, err := k.GetBinaryValue(name)
if err != nil {
return "", fmt.Errorf("get binary value %s: %w", name, err)
}
return hex.EncodeToString(value), nil
default:
return fmt.Sprintf("<unhandled registry type %d>", valueType), nil
}
}
// adapterAddresses returns the adapter list including DNS servers. The call
// reports the size it needs, so grow the buffer and retry until it fits.
func adapterAddresses() (adapters []*windows.IpAdapterAddresses, err error) {
// GetAdaptersAddresses is resolved on first use and panics when it is
// missing, so this reports it as an error and leaves the rest of the
// report intact.
defer func() {
if r := recover(); r != nil {
adapters, err = nil, fmt.Errorf("GetAdaptersAddresses: %v", r)
}
}()
const flags = windows.GAA_FLAG_SKIP_ANYCAST | windows.GAA_FLAG_SKIP_MULTICAST
size := uint32(15000)
for range 3 {
buf := make([]byte, size)
first := (*windows.IpAdapterAddresses)(unsafe.Pointer(&buf[0]))
err := windows.GetAdaptersAddresses(windows.AF_UNSPEC, flags, 0, first, &size)
if errors.Is(err, windows.ERROR_BUFFER_OVERFLOW) {
continue
}
if err != nil {
return nil, fmt.Errorf("GetAdaptersAddresses: %w", err)
}
for adapter := first; adapter != nil; adapter = adapter.Next {
adapters = append(adapters, adapter)
}
return adapters, nil
}
return nil, fmt.Errorf("GetAdaptersAddresses: buffer kept growing")
}
func closeKey(k registry.Key) {
if err := k.Close(); err != nil {
log.Debugf("close registry key: %v", err)
}
}

View File

@@ -0,0 +1,146 @@
//go:build windows
package debug
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/anonymize"
)
func newDNSValueGenerator(level anonymize.Level) *BundleGenerator {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(level)
return &BundleGenerator{
anonymize: true,
anonymizeLevel: level,
anonymizer: anonymizer,
}
}
// TestAnonymizeValueByName covers the value kinds of the DNS registry keys. The
// names decide the treatment, because the string pass alone replaces only
// domains another part of the bundle already seeded.
func TestAnonymizeValueByName(t *testing.T) {
tests := []struct {
name string
valueName string
value string
assert func(t *testing.T, got string)
}{
{
name: "NRPT match domains keep the leading dot",
valueName: "Name",
value: ".internal.example.com, .corp.example.org",
assert: func(t *testing.T, got string) {
t.Helper()
for _, entry := range strings.Split(got, ", ") {
assert.True(t, strings.HasPrefix(entry, "."), "entry %q should keep its leading dot", entry)
assert.NotContains(t, entry, "example", "entry %q should not keep the original domain", entry)
}
},
},
{
name: "any value name ending in Domain is treated as a domain",
valueName: "ICSDomain",
value: "mshome.net",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "mshome", "should anonymize a domain suffix value")
},
},
{
name: "search list is a comma separated domain list",
valueName: "SearchList",
value: "corp.example.com,branch.example.com",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "example", "should anonymize every search domain")
assert.Len(t, strings.Split(got, ", "), 2, "should keep both search domains")
},
},
{
name: "name servers are anonymized as addresses",
valueName: "DhcpNameServer",
value: "203.0.113.10 8.8.8.8",
assert: func(t *testing.T, got string) {
t.Helper()
assert.NotContains(t, got, "203.0.113.10", "should anonymize a public resolver address")
// well-known resolvers stay readable at every level
assert.Contains(t, got, "8.8.8.8", "should keep a well-known resolver address")
},
},
{
name: "opaque values are left to the string pass",
valueName: "DataBasePath",
value: `%SystemRoot%\System32\drivers\etc`,
assert: func(t *testing.T, got string) {
t.Helper()
assert.Equal(t, `%SystemRoot%\System32\drivers\etc`, got, "should not alter a path")
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
g := newDNSValueGenerator(anonymize.LevelDefault)
tc.assert(t, g.anonymizeValue(tc.valueName, tc.value))
})
}
}
// TestParseNRPTPolicyTable parses the MOF text of the policy table out
// parameters, as the provider on a client with one NRPT rule renders it.
func TestParseNRPTPolicyTable(t *testing.T) {
const text = `[abstract]
class __PARAMETERS
{
[Out, EmbeddedInstance("DnsClientPolicyConfiguration"): ToSubClass, ID(2): DisableOverride ToInstance] DnsClientPolicyConfiguration cmdletOutput[] = {
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
DirectAccessQueryIPsecRequired = FALSE;
NameEncoding = "Utf8WithoutMapping";
Namespace = ".0.100.in-addr.arpa";
},
instance of DnsClientPolicyConfiguration
{
DirectAccessProxyType = "NoProxy";
NameEncoding = "Utf8WithoutMapping";
NameServers = {"100.0.255.254", "100.0.255.253"};
Namespace = ".nb.internal";
}};
[in] boolean Effective;
[out] uint32 ReturnValue = 0;
};
`
entries := parseNRPTPolicyTable(text)
require.Len(t, entries, 2, "should parse both embedded instances")
assert.Equal(t, ".0.100.in-addr.arpa", entries[0].namespace, "should read the namespace of the first instance")
assert.Equal(t, ".nb.internal", entries[1].namespace, "should read the namespace of the second instance")
assert.Equal(t, []registryValue{
{name: "DirectAccessProxyType", value: "NoProxy"},
{name: "DirectAccessQueryIPsecRequired", value: "FALSE"},
{name: "NameEncoding", value: "Utf8WithoutMapping"},
}, entries[0].values, "should keep the remaining values in order")
assert.Contains(t, entries[1].values, registryValue{name: "NameServers", value: "100.0.255.254, 100.0.255.253"},
"should flatten a MOF array")
for _, value := range entries[1].values {
assert.NotContains(t, value.name, "ReturnValue", "should not read the class level parameters as values")
}
}
func TestParseNRPTPolicyTableEmpty(t *testing.T) {
assert.Empty(t, parseNRPTPolicyTable(""), "should parse no entries from empty text")
assert.Empty(t, parseNRPTPolicyTable("class __PARAMETERS\n{\n};\n"), "should parse no entries from a table with no instances")
}

View File

@@ -0,0 +1,317 @@
//go:build windows
package debug
import (
"errors"
"fmt"
"runtime"
"strings"
"time"
"github.com/go-ole/go-ole"
"github.com/go-ole/go-ole/oleutil"
log "github.com/sirupsen/logrus"
)
const (
// The NRPT policy table is reachable through the CIM class that backs
// Get-DnsClientNrptPolicy. Unlike the rules in the registry, the table is
// what the resolver currently has loaded, which is the only way to tell an
// applied rule from one that is merely written, in either direction.
nrptPolicyNamespace = `root\Microsoft\Windows\DNS`
nrptPolicyClass = "PS_DnsClientNrptPolicy"
nrptPolicyMethod = "Get"
// The class has no instances, so the table comes from the out parameters
// of a static method call, rendered as MOF text: the embedded instances
// arrive as a safe array of objects, which cannot be read back through the
// COM bindings, and the text form carries all of them.
nrptPolicyInstanceKeyword = "instance of DnsClientPolicyConfiguration"
nrptPolicyTimeout = 15 * time.Second
)
// COM initialization results that leave the calling thread usable: S_FALSE for
// a thread this process already initialized, RPC_E_CHANGED_MODE for one that
// belongs to another apartment.
const (
sFalse = 0x00000001
rpcEChangedMode = 0x80010106
)
// nrptQueryInFlight admits one read of the policy table at a time. A provider
// that stops answering keeps its goroutine and the OS thread that goroutine
// pinned, so a later bundle reports that instead of pinning another one.
var nrptQueryInFlight = make(chan struct{}, 1)
// nrptPolicyEntry is one namespace of the effective policy table, holding the
// values of an embedded DnsClientPolicyConfiguration instance in the order the
// provider reported them.
type nrptPolicyEntry struct {
namespace string
values []registryValue
}
// registryValue is a name and its rendered value, shared by the registry and
// policy table readers so both anonymize by value name the same way.
type registryValue struct {
name string
value string
}
// effectiveNRPTPolicies reads the effective NRPT table. The call is bounded
// because a WMI provider can block indefinitely and a debug bundle must not.
func effectiveNRPTPolicies() ([]nrptPolicyEntry, error) {
type result struct {
text string
err error
}
select {
case nrptQueryInFlight <- struct{}{}:
default:
return nil, errors.New("an earlier read of the policy table has not returned")
}
done := make(chan result, 1)
go func() {
// the slot is released here rather than by the caller, so a read that
// outlives the timeout holds it until the provider answers
defer func() { <-nrptQueryInFlight }()
text, err := nrptPolicyTableText()
done <- result{text: text, err: err}
}()
select {
case res := <-done:
if res.err != nil {
return nil, res.err
}
return parseNRPTPolicyTable(res.text), nil
case <-time.After(nrptPolicyTimeout):
return nil, errors.New("read of the policy table timed out")
}
}
// nrptPolicyTableText calls the policy table method and returns the MOF text of
// its out parameters.
func nrptPolicyTableText() (text string, err error) {
// COM is per thread, and the collection is short lived, so the thread is
// pinned for the duration rather than initialized for the process.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
defer func() {
// The COM call chain is dynamically typed, so a provider that answers
// with an unexpected shape must not take the daemon down with it.
if r := recover(); r != nil {
err = fmt.Errorf("read NRPT policy table: %v", r)
}
}()
owns, err := coInitialize()
if err != nil {
return "", err
}
if owns {
defer ole.CoUninitialize()
}
locator, err := oleutil.CreateObject("WbemScripting.SWbemLocator")
if err != nil {
return "", fmt.Errorf("create WMI locator: %w", err)
}
defer locator.Release()
dispatch, err := locator.QueryInterface(ole.IID_IDispatch)
if err != nil {
return "", fmt.Errorf("query WMI locator interface: %w", err)
}
defer dispatch.Release()
service, err := dispatchCall(dispatch, "ConnectServer", nil, nrptPolicyNamespace)
if err != nil {
return "", fmt.Errorf("connect to %s: %w", nrptPolicyNamespace, err)
}
defer service.Release()
inParams, err := spawnMethodInParams(service)
if err != nil {
return "", err
}
defer inParams.Release()
// The effective table is the merge of the local and the group policy
// store, which is what the resolver answers from.
if _, err := oleutil.PutProperty(inParams, "Effective", true); err != nil {
return "", fmt.Errorf("set Effective parameter: %w", err)
}
outParams, err := dispatchCall(service, "ExecMethod", nrptPolicyClass, nrptPolicyMethod, inParams)
if err != nil {
return "", fmt.Errorf("call %s.%s: %w", nrptPolicyClass, nrptPolicyMethod, err)
}
defer outParams.Release()
textVariant, err := oleutil.CallMethod(outParams, "GetObjectText_")
if err != nil {
return "", fmt.Errorf("render policy table: %w", err)
}
defer func() {
if err := textVariant.Clear(); err != nil {
log.Debugf("clear policy table variant: %v", err)
}
}()
return textVariant.ToString(), nil
}
// spawnMethodInParams builds the in parameters instance the method needs. The
// provider rejects the call without one, even when every parameter is optional.
func spawnMethodInParams(service *ole.IDispatch) (*ole.IDispatch, error) {
class, err := dispatchCall(service, "Get", nrptPolicyClass)
if err != nil {
return nil, fmt.Errorf("get class %s: %w", nrptPolicyClass, err)
}
defer class.Release()
methods, err := dispatchProperty(class, "Methods_")
if err != nil {
return nil, fmt.Errorf("get class methods: %w", err)
}
defer methods.Release()
method, err := dispatchCall(methods, "Item", nrptPolicyMethod)
if err != nil {
return nil, fmt.Errorf("get method %s: %w", nrptPolicyMethod, err)
}
defer method.Release()
params, err := dispatchProperty(method, "InParameters")
if err != nil {
return nil, fmt.Errorf("get method parameters: %w", err)
}
defer params.Release()
inParams, err := dispatchCall(params, "SpawnInstance_")
if err != nil {
return nil, fmt.Errorf("spawn parameter instance: %w", err)
}
return inParams, nil
}
// parseNRPTPolicyTable pulls the embedded instances out of the MOF text. Each
// instance is a namespace of the table, with one name and value per line.
func parseNRPTPolicyTable(text string) []nrptPolicyEntry {
var entries []nrptPolicyEntry
var current *nrptPolicyEntry
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(line), ";"))
switch {
case strings.HasPrefix(line, nrptPolicyInstanceKeyword):
entries = append(entries, nrptPolicyEntry{})
current = &entries[len(entries)-1]
continue
case strings.HasPrefix(line, "}"):
// closes an instance, and the array with the last one, so the
// class level parameters that follow are not read as values
current = nil
continue
case current == nil, line == "{":
continue
}
name, value, ok := strings.Cut(line, " = ")
if !ok {
continue
}
value = unquoteMOFValue(value)
if name == "Namespace" {
current.namespace = value
continue
}
current.values = append(current.values, registryValue{name: name, value: value})
}
return entries
}
// unquoteMOFValue renders a MOF scalar or array as plain text: "a" becomes a,
// and {"a", "b"} becomes a, b.
func unquoteMOFValue(value string) string {
value = strings.TrimSpace(value)
if inner, ok := strings.CutPrefix(value, "{"); ok {
value = strings.TrimSuffix(inner, "}")
entries := strings.Split(value, ",")
for i, entry := range entries {
entries[i] = strings.Trim(strings.TrimSpace(entry), `"`)
}
return strings.Join(entries, ", ")
}
return strings.Trim(value, `"`)
}
// coInitialize prepares the calling thread for COM and reports whether this
// call owns the initialization, which decides whether it may be balanced with
// CoUninitialize. S_FALSE took a reference on a thread this process had already
// initialized and so has to be released, while RPC_E_CHANGED_MODE took none:
// the thread belongs to another apartment, which is usable but is not ours to
// uninitialize.
func coInitialize() (bool, error) {
err := ole.CoInitializeEx(0, ole.COINIT_MULTITHREADED)
if err == nil {
return true, nil
}
var oleErr *ole.OleError
if errors.As(err, &oleErr) {
switch oleErr.Code() {
case sFalse:
return true, nil
case rpcEChangedMode:
return false, nil
}
}
return false, fmt.Errorf("initialize COM: %w", err)
}
// dispatchCall calls a COM method that returns an object.
func dispatchCall(dispatch *ole.IDispatch, method string, params ...any) (*ole.IDispatch, error) {
variant, err := oleutil.CallMethod(dispatch, method, params...)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("%s returned no object", method)
}
return object, nil
}
// dispatchProperty reads a COM property that holds an object.
func dispatchProperty(dispatch *ole.IDispatch, property string) (*ole.IDispatch, error) {
variant, err := oleutil.GetProperty(dispatch, property)
if err != nil {
return nil, err
}
object := variant.ToIDispatch()
if object == nil {
return nil, fmt.Errorf("property %s holds no object", property)
}
return object, nil
}

View File

@@ -267,18 +267,38 @@ func (s *systemConfigurator) getSystemDNSSettings() (SystemDNSSettings, error) {
return SystemDNSSettings{}, fmt.Errorf("sending the command: %w", err)
}
var dnsSettings SystemDNSSettings
dnsSettings, serverAddresses, err := parseSystemDNSSettings(b)
if err != nil {
return dnsSettings, err
}
s.mu.Lock()
s.origNameservers = serverAddresses
s.mu.Unlock()
return dnsSettings, nil
}
// parseSystemDNSSettings parses the output of `scutil show State:/Network/Service/<id>/DNS`.
// Lines that don't match the expected "index : value" shape are skipped: hosts with unusual
// network services (e.g. orphaned hardware ports) can produce entries without a value.
func parseSystemDNSSettings(out []byte) (SystemDNSSettings, []netip.Addr, error) {
// port is not exposed by scutil, default to 53
dnsSettings := SystemDNSSettings{ServerPort: DefaultPort}
var serverAddresses []netip.Addr
inSearchDomainsArray := false
inServerAddressesArray := false
scanner := bufio.NewScanner(bytes.NewReader(b))
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
switch {
case strings.HasPrefix(line, "DomainName :"):
domainName := strings.TrimSpace(strings.Split(line, ":")[1])
dnsSettings.Domains = append(dnsSettings.Domains, domainName)
domainName := strings.TrimSpace(strings.TrimPrefix(line, "DomainName :"))
if domainName != "" {
dnsSettings.Domains = append(dnsSettings.Domains, domainName)
}
continue
case line == "SearchDomains : <array> {":
inSearchDomainsArray = true
continue
@@ -288,36 +308,45 @@ func (s *systemConfigurator) getSystemDNSSettings() (SystemDNSSettings, error) {
case line == "}":
inSearchDomainsArray = false
inServerAddressesArray = false
continue
}
if !inSearchDomainsArray && !inServerAddressesArray {
continue
}
parts := strings.SplitN(line, " : ", 2)
if len(parts) != 2 {
log.Debugf("skipping unexpected scutil DNS line %q", line)
continue
}
value := strings.TrimSpace(parts[1])
if value == "" {
continue
}
if inSearchDomainsArray {
searchDomain := strings.Split(line, " : ")[1]
dnsSettings.Domains = append(dnsSettings.Domains, searchDomain)
} else if inServerAddressesArray {
address := strings.Split(line, " : ")[1]
if ip, err := netip.ParseAddr(address); err == nil && !ip.IsUnspecified() {
ip = ip.Unmap()
serverAddresses = append(serverAddresses, ip)
// Prefer the first IPv4 server as ServerIP since our DNS listener is IPv4.
if !dnsSettings.ServerIP.IsValid() && ip.Is4() {
dnsSettings.ServerIP = ip
}
}
dnsSettings.Domains = append(dnsSettings.Domains, value)
continue
}
ip, err := netip.ParseAddr(value)
if err != nil || ip.IsUnspecified() {
continue
}
ip = ip.Unmap()
serverAddresses = append(serverAddresses, ip)
// Prefer the first IPv4 server as ServerIP since our DNS listener is IPv4.
if !dnsSettings.ServerIP.IsValid() && ip.Is4() {
dnsSettings.ServerIP = ip
}
}
if err := scanner.Err(); err != nil {
return dnsSettings, err
return dnsSettings, serverAddresses, err
}
// default to 53 port
dnsSettings.ServerPort = DefaultPort
s.mu.Lock()
s.origNameservers = serverAddresses
s.mu.Unlock()
return dnsSettings, nil
return dnsSettings, serverAddresses, nil
}
func (s *systemConfigurator) getOriginalNameservers() []netip.Addr {
@@ -435,11 +464,15 @@ func (s *systemConfigurator) getPrimaryService() (string, string, error) {
router := ""
for scanner.Scan() {
text := scanner.Text()
parts := strings.SplitN(text, ":", 2)
if len(parts) != 2 {
continue
}
if strings.Contains(text, "PrimaryService") {
primaryService = strings.TrimSpace(strings.Split(text, ":")[1])
primaryService = strings.TrimSpace(parts[1])
}
if strings.Contains(text, "Router") {
router = strings.TrimSpace(strings.Split(text, ":")[1])
router = strings.TrimSpace(parts[1])
}
}
if err := scanner.Err(); err != nil && err != io.EOF {

View File

@@ -328,6 +328,120 @@ func removeTestDNSKey(key string) error {
return err
}
func TestParseSystemDNSSettings(t *testing.T) {
tests := []struct {
name string
output string
expectedDomains []string
expectedServers []netip.Addr
expectedIP netip.Addr
}{
{
name: "well_formed",
output: `<dictionary> {
DomainName : example.com
SearchDomains : <array> {
0 : example.com
1 : corp.example.com
}
ServerAddresses : <array> {
0 : 192.168.1.1
1 : fd00::53
}
}
`,
expectedDomains: []string{"example.com", "example.com", "corp.example.com"},
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1"), netip.MustParseAddr("fd00::53")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
// entries without a value after the separator used to panic with
// "index out of range [1] with length 1"
name: "malformed_array_entries_skipped",
output: `<dictionary> {
SearchDomains : <array> {
0 :
(null)
1 : corp.example.com
}
ServerAddresses : <array> {
0 :
1 : 192.168.1.1
}
}
`,
expectedDomains: []string{"corp.example.com"},
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "domain_name_without_value_skipped",
output: `<dictionary> {
DomainName :
ServerAddresses : <array> {
0 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "ipv6_first_prefers_ipv4_server_ip",
output: `<dictionary> {
ServerAddresses : <array> {
0 : fd00::53
1 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("fd00::53"), netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "invalid_and_unspecified_addresses_skipped",
output: `<dictionary> {
ServerAddresses : <array> {
0 : (null)
1 : 0.0.0.0
2 : 192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "v4_mapped_address_unmapped",
output: `<dictionary> {
ServerAddresses : <array> {
0 : ::ffff:192.168.1.1
}
}
`,
expectedServers: []netip.Addr{netip.MustParseAddr("192.168.1.1")},
expectedIP: netip.MustParseAddr("192.168.1.1"),
},
{
name: "empty_output",
output: "",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
settings, servers, err := parseSystemDNSSettings([]byte(tc.output))
require.NoError(t, err, "parsing should not fail")
assert.Equal(t, tc.expectedDomains, settings.Domains, "domains should match")
assert.Equal(t, tc.expectedServers, servers, "server addresses should match")
assert.Equal(t, tc.expectedIP, settings.ServerIP, "server IP should match")
assert.Equal(t, DefaultPort, settings.ServerPort, "server port should default to 53")
})
}
}
func TestGetOriginalNameservers(t *testing.T) {
configurator := &systemConfigurator{
createdKeys: make(map[string]struct{}),

View File

@@ -31,10 +31,30 @@ var (
dnsFlushResolverCacheFn = dnsapi.NewProc("DnsFlushResolverCache")
)
// Registry locations of the host DNS configuration this package programs,
// exported so a diagnostic reader reports the same locations that are written.
const (
dnsPolicyConfigMatchPath = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig\NetBird-Match`
gpoDnsPolicyRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
gpoDnsPolicyConfigMatchPath = gpoDnsPolicyRoot + `\NetBird-Match`
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
// Older versions used different layouts under the same prefix: a single
// unsuffixed key, then one key per domain, now one key per batch of domains.
NRPTKeyPrefix = "NetBird-Match"
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
DNSPolicyConfigRoot = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig`
// GPODNSPolicyConfigRoot holds the NRPT rules of the group policy store,
// which takes precedence over the local one when it is present.
GPODNSPolicyConfigRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
// InterfaceConfigPath and InterfaceConfigPathV6 hold the per-interface DNS
// settings, keyed by interface GUID, in separate hives per address family.
InterfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
InterfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
)
const (
dnsPolicyConfigMatchPath = DNSPolicyConfigRoot + `\` + NRPTKeyPrefix
gpoDnsPolicyConfigMatchPath = GPODNSPolicyConfigRoot + `\` + NRPTKeyPrefix
dnsPolicyConfigVersionKey = "Version"
dnsPolicyConfigVersionValue = 2
@@ -45,8 +65,6 @@ const (
nrptMaxDomainsPerRule = 50
interfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
interfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
interfaceConfigNameServerKey = "NameServer"
interfaceConfigDhcpNameSrvKey = "DhcpNameServer"
interfaceConfigSearchListKey = "SearchList"
@@ -73,7 +91,6 @@ type registryConfigurator struct {
guid string
routingAll bool
gpo bool
nrptEntryCount int
origNameservers []netip.Addr
}
@@ -84,7 +101,7 @@ func newHostManager(wgInterface WGIface) (*registryConfigurator, error) {
}
var useGPO bool
k, err := registry.OpenKey(registry.LOCAL_MACHINE, gpoDnsPolicyRoot, registry.QUERY_VALUE)
k, err := registry.OpenKey(registry.LOCAL_MACHINE, GPODNSPolicyConfigRoot, registry.QUERY_VALUE)
if err != nil {
log.Debugf("failed to open GPO DNS policy root: %v", err)
} else {
@@ -123,7 +140,7 @@ func (r *registryConfigurator) captureOriginalNameservers() ([]netip.Addr, error
seen := make(map[netip.Addr]struct{})
var out []netip.Addr
var merr *multierror.Error
for _, root := range []string{interfaceConfigPath, interfaceConfigPathV6} {
for _, root := range []string{InterfaceConfigPath, InterfaceConfigPathV6} {
addrs, err := r.captureFromTcpipRoot(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("%s: %w", root, err))
@@ -306,14 +323,9 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
}
if len(matchDomains) != 0 {
count, err := r.addDNSMatchPolicy(matchDomains, config.ServerIP)
// Update count even on error to ensure cleanup covers partially created rules
r.nrptEntryCount = count
if err != nil {
if err := r.addDNSMatchPolicy(matchDomains, config.ServerIP); err != nil {
return fmt.Errorf("add dns match policy: %w", err)
}
} else {
r.nrptEntryCount = 0
}
r.updateState(stateManager)
@@ -329,9 +341,8 @@ func (r *registryConfigurator) applyDNSConfig(config HostDNSConfig, stateManager
func (r *registryConfigurator) updateState(stateManager *statemanager.Manager) {
if err := stateManager.UpdateState(&ShutdownState{
Guid: r.guid,
GPO: r.gpo,
NRPTEntryCount: r.nrptEntryCount,
Guid: r.guid,
GPO: r.gpo,
}); err != nil {
log.Errorf("failed to update shutdown state: %s", err)
}
@@ -346,7 +357,7 @@ func (r *registryConfigurator) addDNSSetupForAll(ip netip.Addr) error {
return nil
}
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) (int, error) {
func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr) error {
// if the gpo key is present, we need to put our DNS settings there, otherwise our config might be ignored
// see https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-gpnrpt/8cc31cb9-20cb-4140-9e85-3e08703b4745
@@ -363,19 +374,17 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, ruleIndex)
if err := r.configureDNSPolicy(localPath, batchDomains, ip); err != nil {
return ruleIndex, fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
return fmt.Errorf("configure DNS Local policy for rule %d: %w", ruleIndex, err)
}
// Increment immediately so the caller's cleanup path knows about this rule
ruleIndex++
if r.gpo {
if err := r.configureDNSPolicy(gpoPath, batchDomains, ip); err != nil {
return ruleIndex, fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex-1, err)
return fmt.Errorf("configure gpo DNS policy for rule %d: %w", ruleIndex, err)
}
}
log.Debugf("added NRPT rule %d with %d domains", ruleIndex-1, len(batchDomains))
log.Debugf("added NRPT rule %d with %d domains", ruleIndex, len(batchDomains))
ruleIndex++
}
if r.gpo {
@@ -385,7 +394,7 @@ func (r *registryConfigurator) addDNSMatchPolicy(domains []string, ip netip.Addr
}
log.Infof("added %d NRPT rules for %d domains", ruleIndex, len(domains))
return ruleIndex, nil
return nil
}
func (r *registryConfigurator) configureDNSPolicy(policyPath string, domains []string, ip netip.Addr) error {
@@ -496,7 +505,7 @@ func (r *registryConfigurator) deleteInterfaceRegistryKeyProperty(propertyKey st
}
func (r *registryConfigurator) getInterfaceRegistryKey() (registry.Key, error) {
regKeyPath := interfaceConfigPath + "\\" + r.guid
regKeyPath := InterfaceConfigPath + "\\" + r.guid
regKey, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.SET_VALUE)
if err != nil {
return regKey, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", regKeyPath, err)
@@ -518,28 +527,28 @@ func (r *registryConfigurator) restoreHostDNS() error {
return nil
}
// removeDNSMatchPolicies deletes every NRPT rule this client may have created,
// from the local and the GPO policy store. The rules are found by enumerating
// the registry, the only authoritative record of what was written. Cleanup must
// not depend on a rule count: the in-memory one is scoped to a single
// registryConfigurator and the persisted one is deleted on every clean
// disconnect, and a rule left behind keeps resolving names over an interface
// that is gone, until reboot discards the volatile key.
func (r *registryConfigurator) removeDNSMatchPolicies() error {
var merr *multierror.Error
// Try to remove the base entries (for backward compatibility)
if err := removeRegistryKeyFromDNSPolicyConfig(dnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local base entry: %w", err))
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoDnsPolicyConfigMatchPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO base entry: %w", err))
}
for i := 0; i < r.nrptEntryCount; i++ {
localPath := fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i)
gpoPath := fmt.Sprintf("%s-%d", gpoDnsPolicyConfigMatchPath, i)
if err := removeRegistryKeyFromDNSPolicyConfig(localPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove local entry %d: %w", i, err))
for _, root := range []string{DNSPolicyConfigRoot, GPODNSPolicyConfigRoot} {
names, err := listNRPTRuleKeys(root)
if err != nil {
merr = multierror.Append(merr, fmt.Errorf("list rule keys under %s: %w", root, err))
continue
}
if err := removeRegistryKeyFromDNSPolicyConfig(gpoPath); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove GPO entry %d: %w", i, err))
for _, name := range names {
path := root + `\` + name
if err := removeRegistryKeyFromDNSPolicyConfig(path); err != nil {
merr = multierror.Append(merr, fmt.Errorf("remove entry %s: %w", path, err))
}
}
}
@@ -554,6 +563,39 @@ func (r *registryConfigurator) restoreUncleanShutdownDNS() error {
return r.restoreHostDNS()
}
// listNRPTRuleKeys returns the names of our NRPT rule keys under a policy store
// root. An absent root holds nothing to clean up, which is the normal state of
// the GPO store on a machine without DNS Client policy.
func listNRPTRuleKeys(root string) ([]string, error) {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
switch {
case errors.Is(err, registry.ErrNotExist), errors.Is(err, syscall.ERROR_PATH_NOT_FOUND):
// the GPO store is absent on a machine without DNS client policy
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", root)
return nil, nil
case err != nil:
// any other failure has to reach the caller: reporting no rules would
// report a successful cleanup while leaving the rules in place
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
}
defer closer(k)
names, err := k.ReadSubKeyNames(-1)
if err != nil {
return nil, fmt.Errorf("read subkey names: %w", err)
}
var ruleKeys []string
for _, name := range names {
// registry key names are case insensitive
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(NRPTKeyPrefix)) {
ruleKeys = append(ruleKeys, name)
}
}
return ruleKeys, nil
}
func removeRegistryKeyFromDNSPolicyConfig(regKeyPath string) error {
k, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.QUERY_VALUE)
if err != nil {

View File

@@ -25,7 +25,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
interfacePath := InterfaceConfigPath + `\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -56,7 +56,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify 3 NRPT rules exist
assert.Equal(t, 3, cfg.nrptEntryCount, "Should create 3 NRPT rules for 125 domains")
assert.Equal(t, 3, countNRPTRuleKeys(t), "Should create 3 NRPT rules for 125 domains")
for i := 0; i < 3; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -81,7 +81,7 @@ func TestNRPTEntriesCleanupOnConfigChange(t *testing.T) {
require.NoError(t, err)
// Verify first 2 NRPT rules exist
assert.Equal(t, 2, cfg.nrptEntryCount, "Should create 2 NRPT rules for 75 domains")
assert.Equal(t, 2, countNRPTRuleKeys(t), "Should create 2 NRPT rules for 75 domains")
for i := 0; i < 2; i++ {
exists, err := registryKeyExists(fmt.Sprintf("%s-%d", dnsPolicyConfigMatchPath, i))
require.NoError(t, err)
@@ -106,9 +106,65 @@ func registryKeyExists(path string) (bool, error) {
return true, nil
}
// TestNRPTCleanupWithoutRuleCount verifies that rules written by a previous run
// are removed by a configurator that has no record of how many there are: an
// unclean exit loses the in-memory count and a clean disconnect deletes the
// persisted one, so cleanup cannot depend on either.
func TestNRPTCleanupWithoutRuleCount(t *testing.T) {
if testing.Short() {
t.Skip("skipping registry integration test in short mode")
}
defer cleanupRegistryKeys(t)
cleanupRegistryKeys(t)
testIP := netip.MustParseAddr("100.64.0.1")
// 75 domains produce two indexed rules, as the current layout does
domains := make([]string, 75)
for i := range domains {
domains[i] = fmt.Sprintf(".domain%d.com", i+1)
}
previousRun := &registryConfigurator{}
require.NoError(t, previousRun.addDNSMatchPolicy(domains, testIP))
// the unsuffixed key an older version would have written
require.NoError(t, previousRun.configureDNSPolicy(dnsPolicyConfigMatchPath, []string{".legacy.example.com"}, testIP))
// a policy owned by someone else, which cleanup must not touch
foreignPath := DNSPolicyConfigRoot + `\DnsPolicyConfigTestForeign`
foreignKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, foreignPath, registry.SET_VALUE)
require.NoError(t, err, "Should create foreign policy key")
foreignKey.Close()
defer func() {
_ = registry.DeleteKey(registry.LOCAL_MACHINE, foreignPath)
}()
require.Equal(t, 3, countNRPTRuleKeys(t), "Should have two indexed rules and the legacy one")
// a configurator that never applied a DNS config, as one built after a
// restart or from a shutdown state without a count is
freshRun := &registryConfigurator{}
require.NoError(t, freshRun.removeDNSMatchPolicies())
assert.Equal(t, 0, countNRPTRuleKeys(t), "Should remove every rule left by the previous run")
exists, err := registryKeyExists(foreignPath)
require.NoError(t, err)
assert.True(t, exists, "Should not remove a policy that is not ours")
}
func countNRPTRuleKeys(t *testing.T) int {
t.Helper()
names, err := listNRPTRuleKeys(DNSPolicyConfigRoot)
require.NoError(t, err, "Should list NRPT rule keys")
return len(names)
}
func cleanupRegistryKeys(*testing.T) {
// Clean up more entries to account for batching tests with many domains
cfg := &registryConfigurator{nrptEntryCount: 20}
cfg := &registryConfigurator{}
_ = cfg.removeDNSMatchPolicies()
}
@@ -125,7 +181,7 @@ func TestNRPTDomainBatching(t *testing.T) {
// Create a test interface registry key so updateSearchDomains doesn't fail
testGUID := "{12345678-1234-1234-1234-123456789ABC}"
interfacePath := `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\` + testGUID
interfacePath := InterfaceConfigPath + `\` + testGUID
testKey, _, err := registry.CreateKey(registry.LOCAL_MACHINE, interfacePath, registry.SET_VALUE)
require.NoError(t, err, "Should create test interface registry key")
testKey.Close()
@@ -193,7 +249,7 @@ func TestNRPTDomainBatching(t *testing.T) {
require.NoError(t, err)
// Verify that exactly expectedRuleCount rules were created
assert.Equal(t, tc.expectedRuleCount, cfg.nrptEntryCount,
assert.Equal(t, tc.expectedRuleCount, countNRPTRuleKeys(t),
"Should create %d NRPT rules for %d domains", tc.expectedRuleCount, tc.domainCount)
// Verify all expected rules exist

View File

@@ -5,9 +5,8 @@ import (
)
type ShutdownState struct {
Guid string
GPO bool
NRPTEntryCount int
Guid string
GPO bool
}
func (s *ShutdownState) Name() string {
@@ -16,9 +15,8 @@ func (s *ShutdownState) Name() string {
func (s *ShutdownState) Cleanup() error {
manager := &registryConfigurator{
guid: s.Guid,
gpo: s.GPO,
nrptEntryCount: s.NRPTEntryCount,
guid: s.Guid,
gpo: s.GPO,
}
if err := manager.restoreUncleanShutdownDNS(); err != nil {

View File

@@ -2,17 +2,21 @@ package ebpf
import (
_ "embed"
"fmt"
"net"
"sync"
"github.com/cilium/ebpf/link"
"github.com/cilium/ebpf/rlimit"
log "github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
"github.com/netbirdio/netbird/client/internal/ebpf/manager"
)
const (
xdpProgName = "nb_xdp_prog"
mapKeyFeatures uint32 = 0
featureFlagWGProxy = 0b00000001
@@ -68,21 +72,50 @@ func (tf *GeneralManager) loadXdp() error {
return err
}
// load pre-compiled programs into the kernel.
err = loadBpfObjects(&tf.bpfObjs, nil)
// lo has no native XDP, so the program runs in generic mode. Unless it
// declares multi-buffer support the kernel must linearize every non-linear
// skb before running it. Loopback packets are up to 64 KB, so that is a
// contiguous GFP_ATOMIC allocation per packet, and when it fails the packet
// is dropped before the program runs, stalling local TCP connections.
// Multi-buffer XDP in generic mode requires kernel 6.3, so fall back to a
// plain attach when the kernel rejects it.
err = tf.attachXdp(iFace.Index, true)
if err == nil {
return nil
}
log.Debugf("failed to attach multi-buffer xdp program, retrying without it: %s", err)
return tf.attachXdp(iFace.Index, false)
}
func (tf *GeneralManager) attachXdp(iFaceIndex int, multiBuffer bool) error {
spec, err := loadBpf()
if err != nil {
return err
return fmt.Errorf("load bpf spec: %w", err)
}
if multiBuffer {
prog, ok := spec.Programs[xdpProgName]
if !ok {
return fmt.Errorf("program %s not found in bpf spec", xdpProgName)
}
prog.Flags |= unix.BPF_F_XDP_HAS_FRAGS
}
if err := spec.LoadAndAssign(&tf.bpfObjs, nil); err != nil {
return fmt.Errorf("load bpf objects: %w", err)
}
tf.link, err = link.AttachXDP(link.XDPOptions{
Program: tf.bpfObjs.NbXdpProg,
Interface: iFace.Index,
Interface: iFaceIndex,
})
if err != nil {
_ = tf.bpfObjs.Close()
if closeErr := tf.bpfObjs.Close(); closeErr != nil {
log.Debugf("failed to close bpf objects after xdp attach error: %s", closeErr)
}
tf.link = nil
return err
return fmt.Errorf("attach xdp: %w", err)
}
return nil
}

View File

@@ -0,0 +1,130 @@
package profilemanager
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"github.com/netbirdio/netbird/util"
)
const prefsFileSuffix = ".prefs.json"
var prefsMu sync.Mutex
// Prefs is a namespaced per-profile preference store backed by a single JSON
// file next to the profile config; it is deleted together with the profile.
type Prefs struct {
path string
}
// ProfilePrefs returns the preference store of the profile identified by id.
func (s *ServiceManager) ProfilePrefs(id ID, username string) (*Prefs, error) {
if !IsValidProfileFilenameStem(id) {
return nil, fmt.Errorf("invalid profile ID: %q", id)
}
if id == defaultProfileName {
return &Prefs{path: filepath.Join(filepath.Dir(DefaultConfigPath), id.String()+prefsFileSuffix)}, nil
}
configDir, err := s.getConfigDir(username)
if err != nil {
return nil, fmt.Errorf("get config directory for user %s: %w", username, err)
}
return &Prefs{path: filepath.Join(configDir, id.String()+prefsFileSuffix)}, nil
}
// Get unmarshals the namespace section into v and reports whether it exists.
func (p *Prefs) Get(namespace string, v any) (bool, error) {
if namespace == "" {
return false, fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return false, err
}
raw, ok := sections[namespace]
if !ok {
return false, nil
}
if err := json.Unmarshal(raw, v); err != nil {
return false, fmt.Errorf("decode prefs namespace %q: %w", namespace, err)
}
return true, nil
}
// Put stores v as the namespace section, replacing any previous value.
func (p *Prefs) Put(namespace string, v any) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
raw, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("encode prefs namespace %q: %w", namespace, err)
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
sections[namespace] = raw
return writePrefsFile(p.path, sections)
}
// Remove deletes the namespace section; a missing one is not an error.
func (p *Prefs) Remove(namespace string) error {
if namespace == "" {
return fmt.Errorf("empty prefs namespace")
}
prefsMu.Lock()
defer prefsMu.Unlock()
sections, err := readPrefsFile(p.path)
if err != nil {
return err
}
if _, ok := sections[namespace]; !ok {
return nil
}
delete(sections, namespace)
return writePrefsFile(p.path, sections)
}
func removePrefsFile(path string) error {
prefsMu.Lock()
defer prefsMu.Unlock()
return os.Remove(path)
}
func readPrefsFile(path string) (map[string]json.RawMessage, error) {
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return map[string]json.RawMessage{}, nil
}
if err != nil {
return nil, fmt.Errorf("read prefs: %w", err)
}
sections := map[string]json.RawMessage{}
if err := json.Unmarshal(data, &sections); err != nil {
return nil, fmt.Errorf("decode prefs: %w", err)
}
return sections, nil
}
func writePrefsFile(path string, sections map[string]json.RawMessage) error {
if err := util.WriteJsonWithRestrictedPermission(context.Background(), path, sections); err != nil {
return fmt.Errorf("write prefs: %w", err)
}
return nil
}

View File

@@ -0,0 +1,138 @@
package profilemanager
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type testPrefsSection struct {
Mode uint8 `json:"mode"`
Dest string `json:"dest"`
}
func TestProfilePrefs_RoundTrip(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2, Dest: "/tmp/x"}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, testPrefsSection{Mode: 2, Dest: "/tmp/x"}, got)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_GetMissingNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
})
}
func TestProfilePrefs_RemoveNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
require.NoError(t, prefs.Put("other", map[string]int{"n": 1}))
require.NoError(t, prefs.Remove("filedrop"))
require.NoError(t, prefs.Remove("missing"))
var got testPrefsSection
found, err := prefs.Get("filedrop", &got)
require.NoError(t, err)
assert.False(t, found)
var other map[string]int
found, err = prefs.Get("other", &other)
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, map[string]int{"n": 1}, other)
})
}
func TestProfilePrefs_RejectsInvalidID(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
_, err := sm.ProfilePrefs("../escape", username)
assert.Error(t, err)
})
}
func TestProfilePrefs_RejectsEmptyNamespace(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
_, err = prefs.Get("", &testPrefsSection{})
assert.Error(t, err)
assert.Error(t, prefs.Put("", testPrefsSection{}))
assert.Error(t, prefs.Remove(""))
})
}
func TestProfilePrefs_DefaultProfile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
prefs, err := sm.ProfilePrefs(defaultProfileName, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 1}))
expected := filepath.Join(filepath.Dir(DefaultConfigPath), "default"+prefsFileSuffix)
_, err = os.Stat(expected)
require.NoError(t, err)
})
}
func TestRemoveProfile_DeletesPrefsFile(t *testing.T) {
withTestSM(t, func(sm *ServiceManager, username string) {
created, err := sm.AddProfile("work", username)
require.NoError(t, err)
prefs, err := sm.ProfilePrefs(created.ID, username)
require.NoError(t, err)
require.NoError(t, prefs.Put("filedrop", testPrefsSection{Mode: 2}))
configDir, err := sm.getConfigDir(username)
require.NoError(t, err)
prefsPath := filepath.Join(configDir, created.ID.String()+prefsFileSuffix)
_, err = os.Stat(prefsPath)
require.NoError(t, err)
require.NoError(t, sm.RemoveProfile(created.ID, username))
_, err = os.Stat(prefsPath)
assert.True(t, errors.Is(err, os.ErrNotExist), "prefs file should be removed")
})
}

View File

@@ -420,6 +420,11 @@ func (s *ServiceManager) RemoveProfile(id ID, username string) error {
log.Warnf("failed to remove profile state file %s: %v", stateFile, err)
}
prefsFile := filepath.Join(filepath.Dir(target.Path), id.String()+prefsFileSuffix)
if err := removePrefsFile(prefsFile); err != nil && !os.IsNotExist(err) {
log.Warnf("failed to remove profile prefs file %s: %v", prefsFile, err)
}
return nil
}

View File

@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -2,54 +2,183 @@ package ipfwdstate
import (
"fmt"
"sync"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/routemanager/systemops"
)
// IPForwardingState is a struct that keeps track of the IP forwarding state.
// todo: read initial state of the IP forwarding from the system and reset the state based on it.
// todo: separate v4/v6 forwarding state, since the sysctls are independent
// (net.ipv4.ip_forward vs net.ipv6.conf.all.forwarding). Currently the nftables
// manager shares one instance between both routers, which works only because
// EnableIPForwarding enables both sysctls in a single call.
// IPForwardingState tracks v4 and v6 IP-forwarding sysctl enables with
// independent refcounts so a v4-only routing setup doesn't flip v6 sysctls.
type IPForwardingState struct {
enabledCounter int
mu sync.Mutex
v4Count int
v6Count int
// routingV4/routingV6 track whether the routing path currently holds a
// reference, so repeated EnableRouting calls (one per network-map update)
// hold at most one reference per family and an unpaired DisableRouting
// can't release references held by DNAT rules.
routingV4 bool
routingV6 bool
wgIfaceName string
v6Saved map[string]int
}
func NewIPForwardingState() *IPForwardingState {
return &IPForwardingState{}
// NewIPForwardingState returns a state tracker for the IP-forwarding sysctls.
// wgIfaceName is excluded from the per-interface accept_ra handling.
func NewIPForwardingState(wgIfaceName string) *IPForwardingState {
return &IPForwardingState{wgIfaceName: wgIfaceName}
}
func (f *IPForwardingState) RequestForwarding() error {
if f.enabledCounter != 0 {
f.enabledCounter++
// Counts returns the current v4 and v6 refcounts. Intended for diagnostics
// and tests.
func (f *IPForwardingState) Counts() (v4, v6 int) {
f.mu.Lock()
defer f.mu.Unlock()
return f.v4Count, f.v6Count
}
// RequestRouting takes the forwarding references for the routing path. It is
// idempotent: while routing already holds a reference, further calls don't
// increment the refcounts, and a v4-only request releases a previously held v6
// reference. A v6 sysctl failure is logged and not returned so it can't take
// down v4 routing (the sysctl may be unwritable, e.g. read-only /proc/sys or
// IPv6 disabled on the kernel command line); v6 is retried on the next call.
func (f *IPForwardingState) RequestRouting(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if !f.routingV4 {
if err := f.requestV4(); err != nil {
return err
}
f.routingV4 = true
}
if !v6 {
if !f.routingV6 {
return nil
}
f.routingV6 = false
return f.releaseV6()
}
if f.routingV6 {
return nil
}
if err := systemops.EnableIPForwarding(); err != nil {
return fmt.Errorf("failed to enable IP forwarding with sysctl: %w", err)
if err := f.requestV6(); err != nil {
log.Warnf("enable IPv6 forwarding for routing: %v", err)
return nil
}
f.enabledCounter = 1
log.Info("IP forwarding enabled")
f.routingV6 = true
return nil
}
func (f *IPForwardingState) ReleaseForwarding() error {
if f.enabledCounter == 0 {
return nil
// ReleaseRouting releases the references RequestRouting holds. Calls without a
// held reference are no-ops.
func (f *IPForwardingState) ReleaseRouting() error {
f.mu.Lock()
defer f.mu.Unlock()
if f.routingV4 {
f.routingV4 = false
f.releaseV4()
}
if f.enabledCounter > 1 {
f.enabledCounter--
return nil
if f.routingV6 {
f.routingV6 = false
return f.releaseV6()
}
// if failed to disable IP forwarding we anyway decrement the counter
f.enabledCounter = 0
// todo call systemops.DisableIPForwarding()
return nil
}
// RequestForwarding enables the family's forwarding sysctl on first request.
func (f *IPForwardingState) RequestForwarding(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if v6 {
return f.requestV6()
}
return f.requestV4()
}
// ReleaseForwarding decrements the family counter. The last v6 release restores
// what enable captured. v4 stays on: net.ipv4.ip_forward is co-owned by other
// tooling (docker, k8s, libvirt).
func (f *IPForwardingState) ReleaseForwarding(v6 bool) error {
f.mu.Lock()
defer f.mu.Unlock()
if v6 {
return f.releaseV6()
}
f.releaseV4()
return nil
}
func (f *IPForwardingState) requestV4() error {
if f.v4Count == 0 {
if err := systemops.EnableV4IPForwarding(); err != nil {
return fmt.Errorf("enable IPv4 forwarding: %w", err)
}
log.Info("IPv4 forwarding enabled")
}
f.v4Count++
return nil
}
func (f *IPForwardingState) releaseV4() {
if f.v4Count > 0 {
f.v4Count--
}
}
func (f *IPForwardingState) requestV6() error {
if f.v6Count == 0 {
saved, err := systemops.EnableV6IPForwarding(f.wgIfaceName)
if err != nil {
if rerr := systemops.DisableV6IPForwarding(saved); rerr != nil {
log.Warnf("rollback partial v6 sysctls: %v", rerr)
}
return fmt.Errorf("enable IPv6 forwarding: %w", err)
}
// A failed restore on a previous release keeps its saved values; those
// are the true originals, so keep them over what this enable captured.
if f.v6Saved == nil {
f.v6Saved = saved
} else {
for k, v := range saved {
if _, ok := f.v6Saved[k]; !ok {
f.v6Saved[k] = v
}
}
}
log.Info("IPv6 forwarding enabled")
}
f.v6Count++
return nil
}
func (f *IPForwardingState) releaseV6() error {
if f.v6Count == 0 {
return nil
}
f.v6Count--
if f.v6Count > 0 {
return nil
}
// Keep the saved values on failure so a later release or enable/release
// cycle can still restore them; re-restoring an already-restored key is a
// no-op since the sysctl already holds the desired value.
if err := systemops.DisableV6IPForwarding(f.v6Saved); err != nil {
return fmt.Errorf("disable IPv6 forwarding: %w", err)
}
f.v6Saved = nil
log.Info("IPv6 forwarding disabled")
return nil
}

View File

@@ -0,0 +1,39 @@
//go:build privileged
package ipfwdstate
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestRequestRoutingV6ToV4Transition verifies that a v4-only routing request
// releases a previously held routing-owned v6 reference without touching
// references held by DNAT rules.
func TestRequestRoutingV6ToV4Transition(t *testing.T) {
f := NewIPForwardingState("wt-fwd-test")
require.NoError(t, f.RequestRouting(true), "request routing with v6")
v4, v6 := f.Counts()
assert.Equal(t, 1, v4, "v4 reference held")
assert.Equal(t, 1, v6, "v6 reference held")
require.NoError(t, f.RequestRouting(false), "request routing v4-only")
v4, v6 = f.Counts()
assert.Equal(t, 1, v4, "v4 reference kept")
assert.Equal(t, 0, v6, "routing-owned v6 reference released")
// A DNAT-held reference survives a v4-only routing request.
require.NoError(t, f.RequestForwarding(true), "dnat v6 reference")
require.NoError(t, f.RequestRouting(false), "repeat v4-only request")
_, v6 = f.Counts()
assert.Equal(t, 1, v6, "dnat-held v6 reference survives")
require.NoError(t, f.ReleaseForwarding(true), "release dnat v6 reference")
require.NoError(t, f.ReleaseRouting(), "release routing")
v4, v6 = f.Counts()
assert.Equal(t, 0, v4, "all v4 references released")
assert.Equal(t, 0, v6, "all v6 references released")
}

View File

@@ -58,11 +58,7 @@ func Setup(wgIface iface) (map[string]int, error) {
continue
}
// Escape '%' and '.' so they survive the dot-to-slash conversion in Set()
safeName := strings.ReplaceAll(intf.Name, "%", percentEscape)
safeName = strings.ReplaceAll(safeName, ".", dotEscape)
i := fmt.Sprintf(rpFilterInterfacePath, safeName)
i := fmt.Sprintf(rpFilterInterfacePath, EscapeInterfaceName(intf.Name))
oldVal, err := Set(i, 2, true)
if err != nil {
result = multierror.Append(result, err)
@@ -74,6 +70,13 @@ func Setup(wgIface iface) (map[string]int, error) {
return keys, nberrors.FormatErrorOrNil(result)
}
// EscapeInterfaceName escapes '%' and '.' in an interface name (e.g. VLANs
// like eth0.100) so the name survives the dot-to-slash conversion in Set.
func EscapeInterfaceName(name string) string {
safe := strings.ReplaceAll(name, "%", percentEscape)
return strings.ReplaceAll(safe, ".", dotEscape)
}
// Set sets a sysctl configuration, if onlyIfOne is true it will only set the new value if it's set to 1
func Set(key string, desiredValue int, onlyIfOne bool) (int, error) {
path := strings.ReplaceAll(key, ".", "/")

View File

@@ -0,0 +1,82 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -32,8 +32,17 @@ func (r *SysOps) removeFromRouteTable(netip.Prefix, Nexthop) error {
return nil
}
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
return nil
}

View File

@@ -58,8 +58,17 @@ func (r *SysOps) removeFromRouteTable(netip.Prefix, Nexthop) error {
return nil
}
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
return nil
}

View File

@@ -763,13 +763,10 @@ func flushRoutes(tableID, family int) error {
return nberrors.FormatErrorOrNil(result)
}
func EnableIPForwarding() error {
func EnableV4IPForwarding() error {
if _, err := sysctl.Set(ipv4ForwardingPath, 1, false); err != nil {
return err
}
if _, err := sysctl.Set(ipv6ForwardingPath, 1, false); err != nil {
log.Warnf("failed to enable IPv6 forwarding: %v", err)
}
return nil
}

View File

@@ -43,8 +43,17 @@ func (r *SysOps) RemoveVPNRoute(prefix netip.Prefix, intf *net.Interface) error
return r.genericRemoveVPNRoute(prefix, intf)
}
func EnableIPForwarding() error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
func EnableV4IPForwarding() error {
log.Infof("Enable IPv4 forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func EnableV6IPForwarding(string) (map[string]int, error) {
log.Infof("Enable IPv6 forwarding is not implemented on %s", runtime.GOOS)
return map[string]int{}, nil
}
func DisableV6IPForwarding(map[string]int) error {
return nil
}

View File

@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
}
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
return candidates[i].routeMetric < candidates[j].routeMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest route metric
// 3. Lowest interface metric
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -0,0 +1,92 @@
//go:build !android
package systemops
import (
"fmt"
"net"
"os"
"github.com/hashicorp/go-multierror"
log "github.com/sirupsen/logrus"
nberrors "github.com/netbirdio/netbird/client/errors"
"github.com/netbirdio/netbird/client/internal/routemanager/sysctl"
)
const (
// 1 (default) accepts RAs only while forwarding is off; 2 keeps RA
// acceptance on regardless, so RA-installed host defaults survive our
// v6 forwarding flip.
acceptRAInterfacePath = "net.ipv6.conf.%s.accept_ra"
acceptRADefaultPath = "net.ipv6.conf.default.accept_ra"
acceptRAProcPathFormat = "/proc/sys/net/ipv6/conf/%s/accept_ra"
)
// EnableV6IPForwarding bumps accept_ra=2 on host v6 interfaces before flipping
// forwarding=1, so RA-installed host defaults survive. Returns the prior values
// of sysctls we actually changed; entries already at the target are omitted.
func EnableV6IPForwarding(wgIfaceName string) (map[string]int, error) {
saved := map[string]int{}
bumpAcceptRA(saved, wgIfaceName)
oldVal, err := sysctl.Set(ipv6ForwardingPath, 1, false)
if err != nil {
return saved, err
}
if oldVal != 1 {
saved[ipv6ForwardingPath] = oldVal
}
return saved, nil
}
// DisableV6IPForwarding restores what EnableV6IPForwarding captured.
func DisableV6IPForwarding(saved map[string]int) error {
var result *multierror.Error
for key, value := range saved {
if _, err := sysctl.Set(key, value, false); err != nil {
result = multierror.Append(result, fmt.Errorf("restore %s: %w", key, err))
}
}
return nberrors.FormatErrorOrNil(result)
}
func bumpAcceptRA(saved map[string]int, wgIfaceName string) {
// Also bump conf.default so interfaces created while forwarding is on
// (hotplug, new Wi-Fi/dock) inherit accept_ra=2 and keep accepting RAs.
bumpAcceptRAKey(saved, acceptRADefaultPath)
interfaces, err := net.Interfaces()
if err != nil {
log.Warnf("list interfaces for accept_ra: %v", err)
return
}
for _, intf := range interfaces {
if intf.Name == "lo" || intf.Name == wgIfaceName {
continue
}
bumpAcceptRAForInterface(saved, intf.Name)
}
}
func bumpAcceptRAForInterface(saved map[string]int, name string) {
// Build procfs path from name, not the dotted key: VLAN names like eth0.100.
if _, err := os.Stat(fmt.Sprintf(acceptRAProcPathFormat, name)); err != nil {
return
}
bumpAcceptRAKey(saved, fmt.Sprintf(acceptRAInterfacePath, sysctl.EscapeInterfaceName(name)))
}
func bumpAcceptRAKey(saved map[string]int, key string) {
// onlyIfOne=true: leave admin overrides (0, 2) alone.
oldVal, err := sysctl.Set(key, 2, true)
if err != nil {
log.Warnf("bump %s: %v", key, err)
return
}
// With onlyIfOne, a write only happened when the old value was 1; values
// left untouched (0, 2) must not be recorded for restore.
if oldVal == 1 {
saved[key] = oldVal
}
}

View File

@@ -5,6 +5,7 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -323,7 +323,7 @@ func (a *Auth) login(urlOpener URLOpener, forceDeviceAuth bool, deviceName strin
const authInfoRequestTimeout = 30 * time.Second
func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener, forceDeviceAuth bool) (*auth.TokenInfo, error) {
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth)
oAuthFlow, err := authClient.GetOAuthFlow(a.ctx, forceDeviceAuth, "")
if err != nil {
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
}

View File

@@ -5628,9 +5628,13 @@ func (x *GetPeerSSHHostKeyResponse) GetFound() bool {
type RequestJWTAuthRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// hint for OIDC login_hint parameter (typically email address)
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestJWTAuthRequest) Reset() {
@@ -5670,6 +5674,13 @@ func (x *RequestJWTAuthRequest) GetHint() string {
return ""
}
func (x *RequestJWTAuthRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestJWTAuthResponse contains authentication flow information
type RequestJWTAuthResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
@@ -5894,9 +5905,13 @@ type RequestExtendAuthSessionRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
Hint *string `protobuf:"bytes,1,opt,name=hint,proto3,oneof" json:"hint,omitempty"`
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
HasGraphicalSession bool `protobuf:"varint,2,opt,name=hasGraphicalSession,proto3" json:"hasGraphicalSession,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RequestExtendAuthSessionRequest) Reset() {
@@ -5936,6 +5951,13 @@ func (x *RequestExtendAuthSessionRequest) GetHint() string {
return ""
}
func (x *RequestExtendAuthSessionRequest) GetHasGraphicalSession() bool {
if x != nil {
return x.HasGraphicalSession
}
return false
}
// RequestExtendAuthSessionResponse carries the verification URI the UI
// should open in a browser. The daemon retains the flow state and resolves
// it via WaitExtendAuthSession.
@@ -7503,9 +7525,10 @@ const file_daemon_proto_rawDesc = "" +
"sshHostKey\x12\x16\n" +
"\x06peerIP\x18\x02 \x01(\tR\x06peerIP\x12\x1a\n" +
"\bpeerFQDN\x18\x03 \x01(\tR\bpeerFQDN\x12\x14\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"9\n" +
"\x05found\x18\x04 \x01(\bR\x05found\"k\n" +
"\x15RequestJWTAuthRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\x9a\x02\n" +
"\x16RequestJWTAuthResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +
@@ -7525,9 +7548,10 @@ const file_daemon_proto_rawDesc = "" +
"\x14WaitJWTTokenResponse\x12\x14\n" +
"\x05token\x18\x01 \x01(\tR\x05token\x12\x1c\n" +
"\ttokenType\x18\x02 \x01(\tR\ttokenType\x12\x1c\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"C\n" +
"\texpiresIn\x18\x03 \x01(\x03R\texpiresIn\"u\n" +
"\x1fRequestExtendAuthSessionRequest\x12\x17\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01B\a\n" +
"\x04hint\x18\x01 \x01(\tH\x00R\x04hint\x88\x01\x01\x120\n" +
"\x13hasGraphicalSession\x18\x02 \x01(\bR\x13hasGraphicalSessionB\a\n" +
"\x05_hint\"\xe0\x01\n" +
" RequestExtendAuthSessionResponse\x12(\n" +
"\x0fverificationURI\x18\x01 \x01(\tR\x0fverificationURI\x128\n" +

View File

@@ -894,6 +894,10 @@ message GetPeerSSHHostKeyResponse {
message RequestJWTAuthRequest {
// hint for OIDC login_hint parameter (typically email address)
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestJWTAuthResponse contains authentication flow information
@@ -937,6 +941,10 @@ message RequestExtendAuthSessionRequest {
// Optional OIDC login_hint (typically the user's email) to pre-fill the
// IdP login form.
optional string hint = 1;
// hasGraphicalSession tells the daemon that the caller has a graphical session,
// which decides whether PKCE or the device code flow is preferred. The daemon
// cannot detect this itself: it does not inherit the session environment.
bool hasGraphicalSession = 2;
}
// RequestExtendAuthSessionResponse carries the verification URI the UI

View File

@@ -1723,8 +1723,8 @@ func (s *Server) RequestJWTAuth(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -1827,8 +1827,8 @@ func (s *Server) RequestExtendAuthSession(
hint = profilemanager.GetLoginHint()
}
isDesktop := isUnixRunningDesktop()
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, isDesktop, false, hint)
// the daemon has no graphical session of its own, only the caller can answer this
oAuthFlow, err := auth.NewOAuthFlow(ctx, config, msg.GetHasGraphicalSession(), false, hint)
if err != nil {
return nil, gstatus.Errorf(codes.Internal, "failed to create OAuth flow: %v", err)
}
@@ -2000,13 +2000,6 @@ func (s *Server) ExposeService(req *proto.ExposeServiceRequest, srv proto.Daemon
return nil
}
func isUnixRunningDesktop() bool {
if runtime.GOOS != "linux" && runtime.GOOS != "freebsd" {
return false
}
return os.Getenv("DESKTOP_SESSION") != "" || os.Getenv("XDG_CURRENT_DESKTOP") != ""
}
func (s *Server) runProbes(ctx context.Context, waitForProbeResult bool) {
if s.connectClient == nil {
return

View File

@@ -313,21 +313,23 @@ func Dial(ctx context.Context, addr, user string, opts DialOptions) (*Client, er
// dialSSH establishes an SSH connection without JWT authentication
func dialSSH(ctx context.Context, network, addr string, config *ssh.ClientConfig) (*Client, error) {
if config.Timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, config.Timeout)
defer cancel()
}
dialer := &net.Dialer{}
conn, err := dialer.DialContext(ctx, network, addr)
if err != nil {
return nil, fmt.Errorf("dial %s: %w", addr, err)
}
clientConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
client, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
if closeErr := conn.Close(); closeErr != nil {
log.Debugf("connection close after handshake failure: %v", closeErr)
}
return nil, fmt.Errorf("ssh handshake: %w", err)
return nil, err
}
client := ssh.NewClient(clientConn, chans, reqs)
return &Client{
client: client,
}, nil

View File

@@ -12,6 +12,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
"golang.org/x/term"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
func (c *Client) setupTerminalMode(ctx context.Context, session *ssh.Session) error {
@@ -82,37 +84,7 @@ func (c *Client) setupTerminal(session *ssh.Session, fd int) error {
return fmt.Errorf("get terminal size: %w", err)
}
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
// Ctrl+C
ssh.VINTR: 3,
// Ctrl+\
ssh.VQUIT: 28,
// Backspace
ssh.VERASE: 127,
// Ctrl+U
ssh.VKILL: 21,
// Ctrl+D
ssh.VEOF: 4,
ssh.VEOL: 0,
ssh.VEOL2: 0,
// Ctrl+Q
ssh.VSTART: 17,
// Ctrl+S
ssh.VSTOP: 19,
// Ctrl+Z
ssh.VSUSP: 26,
// Ctrl+O
ssh.VDISCARD: 15,
// Ctrl+R
ssh.VREPRINT: 18,
// Ctrl+W
ssh.VWERASE: 23,
// Ctrl+V
ssh.VLNEXT: 22,
}
modes := nbssh.DefaultTerminalModes
terminal := os.Getenv("TERM")
if terminal == "" {

View File

@@ -10,6 +10,8 @@ import (
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
nbssh "github.com/netbirdio/netbird/client/ssh"
)
const (
@@ -80,28 +82,14 @@ func (c *Client) setupTerminalMode(_ context.Context, session *ssh.Session) erro
w, h := c.getWindowsConsoleSize()
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
ssh.VREPRINT: 18, // Ctrl+R
ssh.ICRNL: 1,
ssh.OPOST: 1,
ssh.ONLCR: 1,
ssh.ISIG: 1,
ssh.ICANON: 1,
}
for mode, value := range nbssh.DefaultTerminalModes {
modes[mode] = value
}
if err := session.RequestPty("xterm-256color", h, w, modes); err != nil {

View File

@@ -13,6 +13,7 @@ import (
"golang.org/x/crypto/ssh"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/util"
)
const (
@@ -34,6 +35,19 @@ type HostKeyVerifier interface {
VerifySSHHostKey(peerAddress string, key []byte) error
}
// PeerKeyLookup returns the stored SSH host key for a peer address.
type PeerKeyLookup func(peerAddress string) ([]byte, bool)
// VerifySSHHostKey implements HostKeyVerifier by looking up the stored key
// and comparing it against the presented key.
func (l PeerKeyLookup) VerifySSHHostKey(peerAddress string, presentedKey []byte) error {
storedKey, found := l(peerAddress)
if !found {
return ErrPeerNotFound
}
return VerifyHostKey(storedKey, presentedKey, peerAddress)
}
// DaemonHostKeyVerifier implements HostKeyVerifier using the NetBird daemon
type DaemonHostKeyVerifier struct {
client proto.DaemonServiceClient
@@ -92,7 +106,8 @@ func printAuthInstructions(stderr io.Writer, authResponse *proto.RequestJWTAuthR
// RequestJWTToken requests or retrieves a JWT token for SSH authentication
func RequestJWTToken(ctx context.Context, client proto.DaemonServiceClient, stdout, stderr io.Writer, useCache bool, hint string, openBrowser func(string) error) (string, error) {
req := &proto.RequestJWTAuthRequest{}
// the ssh client runs in the user's session, the daemon does not: tell it what we can see
req := &proto.RequestJWTAuthRequest{HasGraphicalSession: util.HasGraphicalSession()}
if hint != "" {
req.Hint = &hint
}
@@ -193,4 +208,3 @@ func buildAddressList(hostname string, remote net.Addr) []string {
}
return addresses
}

45
client/ssh/handshake.go Normal file
View File

@@ -0,0 +1,45 @@
package ssh
import (
"context"
"fmt"
"io"
"net"
"time"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// Handshake runs the SSH client handshake on an already dialed conn and
// returns the resulting client. Dialing bounds only the TCP establishment;
// without a deadline on the socket a peer that accepts and then goes silent
// blocks the handshake forever, so the context deadline is applied to conn
// for the duration of the handshake. conn is closed on any error.
func Handshake(ctx context.Context, conn net.Conn, addr string, config *ssh.ClientConfig) (*ssh.Client, error) {
if deadline, ok := ctx.Deadline(); ok {
if err := conn.SetDeadline(deadline); err != nil {
closeHandshake(conn, "conn after deadline error")
return nil, fmt.Errorf("set handshake deadline: %w", err)
}
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
if err != nil {
closeHandshake(conn, "conn after handshake error")
return nil, fmt.Errorf("ssh handshake: %w", err)
}
if err := conn.SetDeadline(time.Time{}); err != nil {
closeHandshake(sshConn, "ssh conn after deadline clear error")
return nil, fmt.Errorf("clear handshake deadline: %w", err)
}
return ssh.NewClient(sshConn, chans, reqs), nil
}
func closeHandshake(c io.Closer, label string) {
if err := c.Close(); err != nil {
log.Debugf("ssh: close %s: %v", label, err)
}
}

View File

@@ -610,13 +610,10 @@ func (p *SSHProxy) dialBackend(ctx context.Context, addr, user, jwtToken string)
return nil, fmt.Errorf("connect to server: %w", err)
}
clientConn, chans, reqs, err := cryptossh.NewClientConn(conn, addr, config)
if err != nil {
_ = conn.Close()
return nil, fmt.Errorf("SSH handshake: %w", err)
}
handshakeCtx, cancel := context.WithTimeout(ctx, sshHandshakeTimeout)
defer cancel()
return cryptossh.NewClient(clientConn, chans, reqs), nil
return nbssh.Handshake(handshakeCtx, conn, addr, config)
}
func (p *SSHProxy) verifyHostKey(hostname string, remote net.Addr, key cryptossh.PublicKey) error {

84
client/ssh/session.go Normal file
View File

@@ -0,0 +1,84 @@
package ssh
import (
"fmt"
"io"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh"
)
// DefaultTerminalModes are the PTY modes used by the interactive terminal clients.
var DefaultTerminalModes = ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3, // Ctrl+C
ssh.VQUIT: 28, // Ctrl+\
ssh.VERASE: 127, // Backspace
ssh.VKILL: 21, // Ctrl+U
ssh.VEOF: 4, // Ctrl+D
ssh.VEOL: 0,
ssh.VEOL2: 0,
ssh.VSTART: 17, // Ctrl+Q
ssh.VSTOP: 19, // Ctrl+S
ssh.VSUSP: 26, // Ctrl+Z
ssh.VDISCARD: 15, // Ctrl+O
ssh.VREPRINT: 18, // Ctrl+R
ssh.VWERASE: 23, // Ctrl+W
ssh.VLNEXT: 22, // Ctrl+V
}
// PTYSession is an interactive shell session with a PTY and its I/O pipes.
type PTYSession struct {
Session *ssh.Session
Stdin io.WriteCloser
Stdout io.Reader
Stderr io.Reader
}
// StartPTYSession opens a session on the client, requests an xterm-256color PTY
// with the default terminal modes, wires up the I/O pipes and starts a shell.
// The session is closed on any error.
func StartPTYSession(client *ssh.Client, cols, rows int) (*PTYSession, error) {
session, err := client.NewSession()
if err != nil {
return nil, fmt.Errorf("new session: %w", err)
}
pty, err := setupPTYSession(session, cols, rows)
if err != nil {
if closeErr := session.Close(); closeErr != nil {
log.Debugf("ssh: session close after setup error: %v", closeErr)
}
return nil, err
}
return pty, nil
}
// setupPTYSession requests the PTY, opens the pipes and starts the shell on an
// already created session.
func setupPTYSession(session *ssh.Session, cols, rows int) (*PTYSession, error) {
if err := session.RequestPty("xterm-256color", rows, cols, DefaultTerminalModes); err != nil {
return nil, fmt.Errorf("request pty: %w", err)
}
stdin, err := session.StdinPipe()
if err != nil {
return nil, fmt.Errorf("stdin pipe: %w", err)
}
stdout, err := session.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("stdout pipe: %w", err)
}
stderr, err := session.StderrPipe()
if err != nil {
return nil, fmt.Errorf("stderr pipe: %w", err)
}
if err := session.Shell(); err != nil {
return nil, fmt.Errorf("start shell: %w", err)
}
return &PTYSession{Session: session, Stdin: stdin, Stdout: stdout, Stderr: stderr}, nil
}

View File

@@ -6,9 +6,11 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -58,10 +60,21 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
return ExtendStartResult{}, err
}
req := &proto.RequestExtendAuthSessionRequest{}
if p.Hint != "" {
h := p.Hint
req.Hint = &h
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -15,7 +15,8 @@
"lint": "eslint \"src/**/*.{ts,tsx}\"",
"lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix",
"check": "pnpm lint && pnpm typecheck && pnpm format:check",
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck"
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck",
"i18n:check": "node ../i18n/check-translations.mjs"
},
"dependencies": {
"@radix-ui/react-dialog": "^1.1.15",

View File

@@ -0,0 +1,104 @@
#!/usr/bin/env node
// Validates that every shipped translation bundle carries exactly the same set
// of keys as the English source of truth. English (en) defines the keys; every
// other locale declared in _index.json must match it 1:1:
//
// - no missing keys — a missing key silently falls back to English at runtime
// (see i18n bundle fallback), so the gap never surfaces to users or CI
// without this check;
// - no orphaned keys — keys left behind after an English key is renamed or
// removed are dead weight and a sign the locale is drifting.
//
// Pure Node, no dependencies, so it runs without installing the frontend
// toolchain.
//
// Local: node client/ui/i18n/check-translations.mjs (or: pnpm i18n:check)
// CI: .github/workflows/ui-translations.yml
import { readdirSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const SOURCE = "en";
const localesDir = join(dirname(fileURLToPath(import.meta.url)), "locales");
const isCI = Boolean(process.env.GITHUB_ACTIONS);
function readJSON(path) {
return JSON.parse(readFileSync(path, "utf8"));
}
function keysOf(langCode) {
return Object.keys(readJSON(join(localesDir, langCode, "common.json")));
}
// Emit a GitHub Actions annotation so failures render inline on the PR diff.
function annotate(file, message) {
if (isCI) console.log(`::error file=${file}::${message}`);
}
const index = readJSON(join(localesDir, "_index.json"));
const declared = index.languages.map((l) => l.code);
if (!declared.includes(SOURCE)) {
console.error(`FATAL: source language "${SOURCE}" is not declared in _index.json`);
process.exit(1);
}
const sourceKeys = keysOf(SOURCE);
const sourceSet = new Set(sourceKeys);
console.log(`Source of truth: ${SOURCE}/common.json — ${sourceKeys.length} keys\n`);
let failed = false;
for (const code of declared) {
if (code === SOURCE) continue;
const file = `client/ui/i18n/locales/${code}/common.json`;
let keys;
try {
keys = keysOf(code);
} catch (e) {
failed = true;
const msg = `bundle is declared in _index.json but common.json is missing or invalid (${e.message})`;
console.error(`${code}: ${msg}`);
annotate("client/ui/i18n/locales/_index.json", `${code}: ${msg}`);
continue;
}
const set = new Set(keys);
const missing = sourceKeys.filter((k) => !set.has(k));
const extra = keys.filter((k) => !sourceSet.has(k));
if (missing.length === 0 && extra.length === 0) {
console.log(`${code}: ${keys.length} keys`);
continue;
}
failed = true;
console.error(`${code}: ${keys.length} keys (expected ${sourceKeys.length})`);
if (missing.length) {
console.error(` missing ${missing.length}: ${missing.join(", ")}`);
annotate(file, `Missing ${missing.length} key(s) present in ${SOURCE}: ${missing.join(", ")}`);
}
if (extra.length) {
console.error(` extra ${extra.length}: ${extra.join(", ")}`);
annotate(file, `Has ${extra.length} key(s) not present in ${SOURCE}: ${extra.join(", ")}`);
}
}
// Locale directories present on disk but not declared in _index.json are never
// loaded by the app — surface them so dead translation files don't rot silently.
const onDisk = readdirSync(localesDir, { withFileTypes: true })
.filter((e) => e.isDirectory())
.map((e) => e.name);
const undeclared = onDisk.filter((d) => !declared.includes(d));
if (undeclared.length) {
console.warn(`\n⚠ locale directories not declared in _index.json (not shipped): ${undeclared.join(", ")}`);
}
console.log();
if (failed) {
console.error("Translation check FAILED — every locale must match the English key set.");
process.exit(1);
}
console.log("Translation check passed — all locales match the English key set.");

View File

@@ -1312,6 +1312,9 @@
"daemon.outdated.description": {
"message": "このアプリを使用するには NetBird サービスを更新してください。"
},
"daemon.outdated.download": {
"message": "最新版をダウンロード"
},
"error.jwt_clock_skew": {
"message": "サインインに失敗しました: このデバイスの時計がサーバーと同期していません。システムの時計を同期してからもう一度お試しください。"
},

View File

@@ -108,10 +108,11 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
}
req := &proto.LoginRequest{
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
IsUnixDesktopClient: runtime.GOOS == "linux",
ManagementUrl: p.ManagementURL,
SetupKey: p.SetupKey,
Hostname: p.Hostname,
// a login driven by the UI always has a graphical session available
IsUnixDesktopClient: true,
}
if profileName != "" {
req.ProfileName = ptrStr(profileName)
@@ -122,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
}
resp, err := cli.Login(ctx, req)
@@ -227,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -260,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Logout below).
// root and the per-profile state file is user-owned (see Profiles.List).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -80,13 +80,12 @@ func (c *Client) Connect(host string, port int, username, jwtToken string, ipVer
return fmt.Errorf("dial %s: %w", addr, err)
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
sshClient, err := nbssh.Handshake(ctx, conn, addr, config)
if err != nil {
closeWithLog(conn, "connection after handshake error")
return fmt.Errorf("SSH handshake: %w", err)
return err
}
c.sshClient = ssh.NewClient(sshConn, chans, reqs)
c.sshClient = sshClient
logrus.Infof("SSH: Connected to %s", addr)
return nil
@@ -119,57 +118,26 @@ func (c *Client) getAuthMethods(jwtToken string) ([]ssh.AuthMethod, error) {
return []ssh.AuthMethod{ssh.PublicKeys(signer)}, nil
}
// StartSession starts an SSH session with PTY
// StartSession starts an SSH session with PTY. It holds the client lock for
// the whole startup so Close cannot tear the client down mid-setup and the
// new session cannot be installed into an already closed client.
func (c *Client) StartSession(cols, rows int) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.sshClient == nil {
return fmt.Errorf("SSH client not connected")
}
session, err := c.sshClient.NewSession()
pty, err := nbssh.StartPTYSession(c.sshClient, cols, rows)
if err != nil {
return fmt.Errorf("create session: %w", err)
return err
}
c.mu.Lock()
defer c.mu.Unlock()
c.session = session
modes := ssh.TerminalModes{
ssh.ECHO: 1,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
ssh.VINTR: 3,
ssh.VQUIT: 28,
ssh.VERASE: 127,
}
if err := session.RequestPty("xterm-256color", rows, cols, modes); err != nil {
closeWithLog(session, "session after PTY error")
return fmt.Errorf("PTY request: %w", err)
}
c.stdin, err = session.StdinPipe()
if err != nil {
closeWithLog(session, "session after stdin error")
return fmt.Errorf("get stdin: %w", err)
}
c.stdout, err = session.StdoutPipe()
if err != nil {
closeWithLog(session, "session after stdout error")
return fmt.Errorf("get stdout: %w", err)
}
c.stderr, err = session.StderrPipe()
if err != nil {
closeWithLog(session, "session after stderr error")
return fmt.Errorf("get stderr: %w", err)
}
if err := session.Shell(); err != nil {
closeWithLog(session, "session after shell error")
return fmt.Errorf("start shell: %w", err)
}
c.session = pty.Session
c.stdin = pty.Stdin
c.stdout = pty.Stdout
c.stderr = pty.Stderr
logrus.Info("SSH: Session started with PTY")
return nil

View File

@@ -115,7 +115,7 @@ sequenceDiagram
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)

View File

@@ -15,6 +15,10 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
| ---- | ---- |
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
@@ -48,6 +52,8 @@ flowchart TD
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
K2 --> P
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
@@ -60,6 +66,84 @@ flowchart TD
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### LLM pricing (management is the sole authority)
**The proxy carries no price list.** Management synthesizes the entire pricing
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
table and there is nothing to reload on the proxy side.
```mermaid
flowchart TD
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
B --> C{AgentNetwork.PricingDefaultsFile}
C -- absent --> D[compiled-in table serves]
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
E --> F[mergedTable atomic.Pointer]
D --> G[DefaultTable]
F --> G
G --> H[buildCostMeterConfigJSON — pricing.defaults]
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
K --> L[pricing.providers keyed by provider record ID]
H --> M[cost_meter ConfigJSON]
L --> M
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
```
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
id → rates. The **full** default table ships to every account: it is small
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
models, so they claim every model) priced.
- `pricing.providers` — provider **record** id → normalized model id → rates,
matched against the `llm.resolved_provider_id` the router stamps. Entries are
**fully materialized here**, at synth time: `materializeEntry` starts from the
default entry for that model so cache rates the operator didn't state are
inherited, overlays operator `input`/`output` verbatim (**including an explicit
0**, which prices a self-hosted or internal endpoint as free rather than
silently reverting to list price), and overlays cache-rate **pointers only when
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
discount, bill this bucket at the input rate". The proxy therefore does two map
lookups and no merging.
Same orphan rule as the router: a provider no enabled policy authorises is
unreachable, so its prices aren't shipped. Model ids are normalized with the
**same** functions the request parser uses (`NormalizeBedrockModel` /
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
first-occurrence-wins, matching the routing dedup order.
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
replace default rates without a rebuild. Schema is `surface → model → rates`
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
- An **explicitly configured** path is *required to load*: a typo or malformed
file fails startup, because the operator believes those rates are live. The
conventional probe is optional — an absent file just serves compiled-in
defaults, and the path stays watched in case it appears later.
- File entries **replace** the compiled-in entry for the same (surface, model)
**whole** — they are not field-merged, so an entry must repeat the cache rates
it wants to keep. Everything the file doesn't mention keeps built-in rates.
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
finite and non-negative — the same constraints the HTTP API enforces on
operator per-provider prices.
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
a parse error keeps the previous table, a deleted file reverts to compiled-in
defaults. A mid-edit save can never take pricing down.
The live table feeds **both** consumers, which is what keeps them consistent: the
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
golden-tested, so operators start from a file matching the built-in rates exactly.
### Budget rule resolution (min-wins, group+user bound)
```mermaid
@@ -124,7 +208,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | |
| on_response | 6 | `cost_meter` | `{}` | |
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
@@ -139,6 +223,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
## Things to scrutinize
@@ -176,10 +266,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
@@ -188,6 +280,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
## Test coverage
@@ -198,6 +291,9 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |

View File

@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing catalog this chain delegates to.
adapters + pricing table and cost formula this chain delegates to.
---
@@ -34,7 +34,7 @@ rewrites.
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:2640](../../../proxy/internal/middleware/builtin/all_test.go)
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
@@ -168,12 +168,46 @@ token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`).
**Management owns pricing.** The proxy carries no embedded price list: the whole
table arrives in this middleware's `ConfigJSON` as
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
plus the operator's stored per-provider prices
([factory.go:1334](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
construction, so a non-finite or negative rate fails the chain build. A price
change is an ordinary mapping push — the chain rebuild yields a fresh instance
over a fresh immutable table, so there is no data dir, no pricing file, no
reload goroutine, and nothing to invalidate.
**Two-tier lookup**
([middleware.go:165183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
1. **Per-provider-record** — the operator's stored price for the route that
actually served the request, keyed by the `llm.resolved_provider_id` that
`llm_router` stamped on the allow path, then by normalized model id. Entries
arrive fully materialized (management folds default cache rates in at synth
time), so there is no merging here. Absent metadata — no router in the chain
— skips this tier.
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
providers, which enumerate no models and therefore get no per-record entry.
**Backward compatibility:** a config with no `pricing` block means management
predates config-delivered pricing. The factory logs one warning at build time
and the instance records `cost.skipped=unknown_model` ($0) for every request
rather than falling back to a stale built-in price list
([factory.go:5560](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
(sibling doc) and is selected by the **surface**, not by which tier the entry
came from — `cost_meter` stays provider-agnostic, and a per-record override on
an Anthropic route still bills its cache buckets additively.
### llm_limit_record
@@ -246,12 +280,14 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time.
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
block carrying a negative or non-finite rate fails the build too, rather than
mispricing live traffic.
## Invariants
@@ -320,10 +356,11 @@ non-object `metadata` field
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
middleware config and never mutated, so the lookup path needs no lock or atomic
swap — a price change replaces the whole instance. Every middleware is
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
@@ -349,13 +386,13 @@ counter accuracy.
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing loader.
+ SSE framer + pricing table and cost formula.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing loader.
— the 8 middlewares that consume this package's parsers + pricing table.
---
@@ -24,8 +24,9 @@ proxy-framework dependencies:
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/`embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `pricing/`immutable pricing table + the per-surface cost formula. The
rates themselves come from management inside `cost_meter`'s middleware
config; this package holds no price list and reads no files.
- `fixtures/` — captured request/response/stream bodies the tests replay.
The package carries zero proxy-framework dependencies so the same parsers can
@@ -47,12 +48,9 @@ be reused later by a WASM adapter
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream |
## Request body → parser dispatch
@@ -188,9 +186,11 @@ response leg, covering both Bedrock body shapes:
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
management ships, keyed by the **normalised** model id (region prefix + version
suffix stripped by the request parser; management normalises its keys the same
way at synth time so the two compare equal). `ParseResponse` returns
`ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
@@ -205,11 +205,34 @@ response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing catalog
### Pricing table
`Table.Cost`
([pricing.go:129174](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
**Management is the sole pricing authority.** The proxy carries no embedded
price list and reads no pricing file: the whole table arrives inside
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
is just another push — the chain rebuild constructs a fresh `Table`, so there
is nothing to reload
([pricing.go:17](../../../proxy/internal/llm/pricing/pricing.go)). The
management side of the contract (catalog defaults, the operator's stored
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
management-side module guide; `cost_meter`'s wire shape is in
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
`EntryJSON`
([pricing.go:3645](../../../proxy/internal/llm/pricing/pricing.go)) is the
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
direct struct conversion rather than field-by-field copying (a new rate can't
be silently dropped in transit).
`EntryCosts`
([pricing.go:183234](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module. The
**surface** (the `llm.provider` value the request parser stamped) selects the
formula, never the tier the entry came from: a per-provider-record override on
an Anthropic route still bills its cache buckets additively.
| Provider | Formula |
|---|---|
@@ -218,7 +241,7 @@ is the cost formula — most security-relevant math in this module:
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
([pricing.go:214229](../../../proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
@@ -226,15 +249,12 @@ to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Loader`
([pricing.go:212268](../../../proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
`Costs`
([pricing.go:143163](../../../proxy/internal/llm/pricing/pricing.go)) is the
per-request split. The four per-bucket fields are the base; `TotalUSD` and
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
the breakdown. `InputUSD` is always the non-cached input bucket on both
provider shapes, so input and cached-input never double-count.
## Public contracts
@@ -264,29 +284,38 @@ Order matters: `DetectFromURL` ties resolve by registration order.
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` lookup**
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
**`Pricing` construction + lookup**
([pricing.go:60130](../../../proxy/internal/llm/pricing/pricing.go)):
```go
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
func (t *Table) Lookup(provider, model string) (Entry, bool)
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
```
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130132](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
reject any non-finite or negative rate, so a corrupt config fails the chain
build rather than mispricing silently. Nil input yields an empty,
never-matching table.
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
([pricing.go:9699](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means the surface or model is absent from the table management sent;
the caller emits `cost.skipped=unknown_model`.
## Invariants
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:1416](../../../proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
1. **The pricing package is pure and platform-independent.** No file I/O, no
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
there is nothing platform-specific left to port. Anything reintroducing a
read-from-disk path here re-splits pricing authority between management and
the proxy, which is exactly what this design removed.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
@@ -298,38 +327,45 @@ emits `cost.skipped=unknown_model`.
usage rather than aborting
([streaming.go:6873, 144150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:2930](../../../proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:4249](../../../proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
3. **Management is the only source of rates.** `Table` has no constructor that
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
management sent. A missing or empty `pricing` block therefore means *no
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
never a stale built-in fallback that would silently bill list price.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370394](../../../proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
4. **Tables are immutable once built.** `Table.entries` is written only in
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
map is likewise build-time-only
([pricing.go:4752](../../../proxy/internal/llm/pricing/pricing.go)). This
is what makes the no-reload design safe: a price change arrives as a mapping
push that builds a new middleware instance over a new table, so concurrent
readers can't observe a half-updated price list and no atomic swap or lock
is needed on the hot path.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:2557](../../../proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
5. **Rate validation happens at chain-build time, not per request.**
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
([pricing.go:6083](../../../proxy/internal/llm/pricing/pricing.go)), naming
the offending surface/model/field in the error. Management enforces the same
constraints at its API boundary and in its YAML parser, so this is
defense-in-depth — but it means a corrupt push fails loudly at build instead
of producing negative costs on live traffic. Test:
`TestNewTable_ValidatesRates`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397398](../../../proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
`pricing.Entry` together.** `NewEntries` converts by direct struct
conversion `Entry(e)`
([pricing.go:7678](../../../proxy/internal/llm/pricing/pricing.go)), which
only compiles while the two structs stay field-identical — so the proxy half
is compiler-enforced. The management half is not: a rate added there but not
here unmarshals into nothing and prices that bucket at `InputPer1K`.
## Things to scrutinise
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147149](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
**Correctness.** Verify the OpenAI cached-prompt clamp at
[pricing.go:203206](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. Negative token counts are clamped to zero up
front ([pricing.go:186197](../../../proxy/internal/llm/pricing/pricing.go)) so
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
@@ -338,22 +374,27 @@ noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
oversize surface is gone entirely — the config channel (an authenticated
mapping push from management) is now the only way rates enter the proxy, and
`NewEntries` is the validation boundary on it. A new rate added to management's
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
path (see invariant 6).
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Concurrency.** Nothing in this package is shared mutable state: tables are
built once and never written again, so `cost_meter`'s hot path is lock-free by
construction rather than by atomic swap. Per-call `Scanner` instances mean no
shared state across concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
`ReadString('\n')` per line. No background goroutines and no per-request
allocation of pricing state.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
**Observability.** A config carrying no `pricing` block logs one warning at
chain-build time (`cost_meter` factory) and then records
`cost.skipped=unknown_model` per request, so an old-management deployment is
visible in both logs and the access log rather than quietly reporting $0.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
@@ -365,7 +406,7 @@ right `cost.skipped` reason.
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
@@ -373,14 +414,15 @@ right `cost.skipped` reason.
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
No pricing fixture: the table is config-delivered, so pricing tests construct
it in-process from a wire-shape map.
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewLoader`.
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -1,7 +1,7 @@
# proxy/runtime — translate + serve + log
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
## Module boundary
@@ -114,8 +114,7 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
## Public contracts touched
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).

View File

@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic``llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
parser, priced under the **`anthropic`** surface of the pricing table
management ships (the parser emits the standard Anthropic provider label, so
Vertex Claude reuses first-party Anthropic prices).
- `openai``llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
@@ -104,8 +104,9 @@ is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
> those regions read low. Operators who need exact regional billing set the
> affected models' prices on the provider record, or replace the default entries
> via management's `AgentNetwork.PricingDefaultsFile`.
## AWS Bedrock (`bedrock_api`)
@@ -211,15 +212,19 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
`model_prices_and_context_window.json`. The **catalog is the source of default
prices**: management's `pricing.DefaultTable` folds every catalog provider's
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
is structural rather than maintained in a parallel file
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
providers contribute the same (surface, model) at different rates. Bedrock
entries are keyed by the **normalised** id the request parser emits (region
prefix + version suffix stripped) — management applies the same normalisation to
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
surface.
## Things to scrutinise
@@ -232,16 +237,17 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
pricing key — a model that normalises to something absent from the shipped
pricing table meters at `cost.skipped=unknown_model` rather than failing the
request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
~10% premium not modelled by base pricing — flagged in the catalog comment.
Operators needing exact regional billing set per-provider prices on the model
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
## Cross-references

View File

@@ -0,0 +1,403 @@
//go:build e2e
// Package providerdiscovery is a credential-driven spike, not a regression
// suite. It calls each vendor's model-discovery endpoint DIRECTLY — no proxy,
// no tunnel, no containers — to answer a question the mock upstream cannot:
// what does each vendor return to a caller holding only the credential an
// operator gave us, and in what shape?
//
// That is the call management would have to make to populate the provider-config
// model picker from live data instead of the hand-maintained catalog in
// management/internals/modules/agentnetwork/catalog. Today that catalog is
// curated by hand — it carries comments tracking which models a vendor retired
// on which date — and it cannot know what a particular account may actually
// invoke.
//
// The load-bearing unknown is Bedrock. ListInferenceProfiles is a CONTROL PLANE
// operation on bedrock.<region>.amazonaws.com, while a provider record's
// upstream is bedrock-runtime.<region> because that is what InvokeModel needs.
// Whether a Bedrock API key (AWS_BEARER_TOKEN_BEDROCK) authorises the control
// plane at all is undocumented, and the answer decides whether live discovery
// for Bedrock is a small feature or needs SigV4 signing in management. The
// other three surfaces are here to corroborate the shape.
//
// NOTHING HERE ASSERTS A STATUS CODE, deliberately: the status is the finding.
// A probe fails the test only when its credential is present and the request
// could not be made at all, which is a harness problem rather than an answer.
package providerdiscovery
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strings"
"testing"
"time"
"golang.org/x/oauth2/google"
"github.com/stretchr/testify/require"
)
// probeTimeout bounds a single vendor call. Generous for a discovery GET, and
// short enough that a hanging endpoint reports rather than stalls the job.
const probeTimeout = 20 * time.Second
// maxLoggedBody bounds what a probe echoes into the job log. A full model
// listing runs to tens of kilobytes and the useful part is the front; an error
// body is short and is reproduced whole.
const maxLoggedBody = 1500
// gcpScope matches the scope llm_router mints Vertex tokens under, so this
// probe exercises the same credential the proxy already uses in production
// rather than a differently-scoped one that might succeed where it fails.
const gcpScope = "https://www.googleapis.com/auth/cloud-platform"
// probe is one discovery endpoint to try.
type probe struct {
// surface is the provider-config entry this informs (openai_api, …).
surface string
// variant distinguishes several candidate endpoints for one surface,
// because part of the spike is finding out WHICH path answers.
variant string
url string
headers map[string]string
}
// result is what came back, including the failure cases — those are answers too.
type result struct {
probe
status int
// shape names the JSON envelope the ids were found under, so the eventual
// implementation knows which parser each surface needs.
shape string
ids []string
body string
err error
}
// TestProviderModelDiscoverySpike probes every configured vendor and prints a
// verdict table. Read the log, not the pass/fail.
func TestProviderModelDiscoverySpike(t *testing.T) {
probes := configuredProbes(t)
if len(probes) == 0 {
t.Skip("no provider credentials set; source ~/.llm-keys to run the discovery spike")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
results := make([]result, 0, len(probes))
for _, p := range probes {
t.Run(p.surface+"/"+p.variant, func(t *testing.T) {
res := runProbe(ctx, p)
results = append(results, res)
// A transport error means we never got an answer — that is a broken
// probe, not a finding about the vendor.
require.NoError(t, res.err, "%s %s: request could not be made", p.surface, p.variant)
t.Logf("[spike] %s %s -> %d", p.surface, p.variant, res.status)
t.Logf("[spike] %s %s url: %s", p.surface, p.variant, p.url)
if res.shape != "" {
t.Logf("[spike] %s %s shape=%s ids=%d: %s",
p.surface, p.variant, res.shape, len(res.ids), strings.Join(sample(res.ids, 12), ", "))
}
t.Logf("[spike] %s %s body: %s", p.surface, p.variant, truncate(res.body, maxLoggedBody))
})
}
t.Log("[spike] ==================== VERDICT ====================")
for _, r := range results {
t.Logf("[spike] %-28s %-42s %3d %s",
r.surface+"/"+r.variant, verdict(r), r.status, shapeNote(r))
}
t.Log("[spike] =================================================")
}
// configuredProbes assembles the probe list from whichever credentials are
// present, mirroring the env-var gating the rest of the e2e suite uses.
func configuredProbes(t *testing.T) []probe {
t.Helper()
var ps []probe
if k := os.Getenv("OPENAI_TOKEN"); k != "" {
ps = append(ps, probe{
surface: "openai_api", variant: "v1-models",
url: "https://api.openai.com/v1/models",
headers: map[string]string{"Authorization": "Bearer " + k},
})
}
if k := os.Getenv("ANTHROPIC_TOKEN"); k != "" {
// limit=1000 because the default page is small and a picker wants the
// whole catalogue in one call if it can get it.
ps = append(ps, probe{
surface: "anthropic_api", variant: "v1-models",
url: "https://api.anthropic.com/v1/models?limit=1000",
headers: map[string]string{
"x-api-key": k,
"anthropic-version": "2023-06-01",
},
})
}
ps = append(ps, bedrockProbes()...)
ps = append(ps, vertexProbes(t)...)
return ps
}
// bedrockProbes covers the three candidate hosts/paths. The runtime probe is
// the control: we already know it 404s, and having it in the same table makes
// the control-plane result unambiguous rather than a lone data point.
func bedrockProbes() []probe {
k := os.Getenv("AWS_BEARER_TOKEN_BEDROCK")
if k == "" {
return nil
}
region := os.Getenv("AWS_REGION")
if region == "" {
region = "eu-central-1"
}
auth := map[string]string{"Authorization": "Bearer " + k}
return []probe{
{
surface: "bedrock_api", variant: "control-inference-profiles",
url: "https://bedrock." + region + ".amazonaws.com/inference-profiles",
headers: auth,
},
{
surface: "bedrock_api", variant: "control-foundation-models",
url: "https://bedrock." + region + ".amazonaws.com/foundation-models",
headers: auth,
},
{
// The control: the record's real upstream, which we expect to
// answer <UnknownOperationException/>.
surface: "bedrock_api", variant: "runtime-inference-profiles",
url: "https://bedrock-runtime." + region + ".amazonaws.com/inference-profiles",
headers: auth,
},
}
}
// vertexProbes covers the publisher-model listing under both API versions and
// both the global and project-scoped forms, because which one answers is itself
// part of what the spike is for.
func vertexProbes(t *testing.T) []probe {
t.Helper()
sa := os.Getenv("GOOGLE_VERTEX_SA_BASE64")
project := os.Getenv("GOOGLE_VERTEX_PROJECT")
if sa == "" || project == "" {
return nil
}
region := os.Getenv("GOOGLE_VERTEX_REGION")
if region == "" {
region = "global"
}
host := "aiplatform.googleapis.com"
if region != "global" {
host = region + "-aiplatform.googleapis.com"
}
token, err := mintGCPToken(sa)
if err != nil {
// Report rather than fail: a credential we cannot mint from is a
// finding about the credential, and the other surfaces still have
// something to say.
t.Logf("[spike] vertex_ai_api: could not mint an OAuth token from the service-account key: %v", err)
return nil
}
auth := map[string]string{"Authorization": "Bearer " + token}
return []probe{
{
surface: "vertex_ai_api", variant: "v1-publishers",
url: "https://" + host + "/v1/publishers/anthropic/models",
headers: auth,
},
{
surface: "vertex_ai_api", variant: "v1beta1-publishers",
url: "https://" + host + "/v1beta1/publishers/anthropic/models",
headers: auth,
},
{
surface: "vertex_ai_api", variant: "v1-project-scoped",
url: "https://" + host + "/v1/projects/" + project +
"/locations/" + region + "/publishers/anthropic/models",
headers: auth,
},
{
// The project-scoped list under the version that actually answers.
// The first run's publisher list returned only two models, fewer
// than the catalog ships, which is what a publisher-global list
// looks like rather than what THIS project has enabled — and
// per-project availability is most of why live discovery beats a
// static catalogue. The v1 form 404s, so v1beta1 is the one form
// left that could carry it.
surface: "vertex_ai_api", variant: "v1beta1-project-scoped",
url: "https://" + host + "/v1beta1/projects/" + project +
"/locations/" + region + "/publishers/anthropic/models",
headers: auth,
},
{
// A publisher model is addressed as "<id>@<version>" on the
// rawPredict path, and the plain listing reports one versionId per
// model. If a model has several live versions, a picker that only
// ever saw one would silently hide the rest.
surface: "vertex_ai_api", variant: "v1beta1-all-versions",
url: "https://" + host + "/v1beta1/publishers/anthropic/models?listAllVersions=true",
headers: auth,
},
}
}
// mintGCPToken builds an access token from the base64 service-account key,
// the same way llm_router does at request time.
func mintGCPToken(saKeyB64 string) (string, error) {
jsonKey, err := base64.StdEncoding.DecodeString(strings.TrimSpace(saKeyB64))
if err != nil {
return "", fmt.Errorf("decode service-account key: %w", err)
}
conf, err := google.JWTConfigFromJSON(jsonKey, gcpScope)
if err != nil {
return "", fmt.Errorf("parse service-account key: %w", err)
}
ctx, cancel := context.WithTimeout(context.Background(), probeTimeout)
defer cancel()
tok, err := conf.TokenSource(ctx).Token()
if err != nil {
return "", fmt.Errorf("mint token: %w", err)
}
return tok.AccessToken, nil
}
// runProbe issues one request and extracts whatever model ids it can find.
func runProbe(ctx context.Context, p probe) result {
res := result{probe: p}
reqCtx, cancel := context.WithTimeout(ctx, probeTimeout)
defer cancel()
req, err := http.NewRequestWithContext(reqCtx, http.MethodGet, p.url, nil)
if err != nil {
res.err = err
return res
}
for k, v := range p.headers {
req.Header.Set(k, v)
}
req.Header.Set("Accept", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
res.err = err
return res
}
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if err != nil {
res.err = err
return res
}
res.status = resp.StatusCode
res.body = string(body)
res.shape, res.ids = extractIDs(body)
return res
}
// listingShapes maps a response envelope to the field naming the model id
// inside it. Each vendor invented its own; a picker has to read all of them.
var listingShapes = []struct {
envelope string
idField string
}{
{"data", "id"}, // OpenAI, Anthropic
{"inferenceProfileSummaries", "inferenceProfileId"}, // Bedrock control plane
{"modelSummaries", "modelId"}, // Bedrock foundation models
{"publisherModels", "name"}, // Vertex publisher models
{"models", "name"}, // Vertex, older shape
}
// extractIDs returns the envelope that matched and the ids under it. An empty
// shape means the body is not a listing this spike recognises — which for an
// error response is the expected outcome.
func extractIDs(body []byte) (string, []string) {
var doc map[string]json.RawMessage
if err := json.Unmarshal(body, &doc); err != nil {
return "", nil
}
for _, shape := range listingShapes {
raw, ok := doc[shape.envelope]
if !ok {
continue
}
var entries []map[string]json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
continue
}
ids := make([]string, 0, len(entries))
for _, entry := range entries {
var id string
if err := json.Unmarshal(entry[shape.idField], &id); err != nil || id == "" {
continue
}
// Vertex splits the wire id across two fields: a publisher model is
// addressed as "<id>@<version>" on rawPredict, so a listing that
// reported only the name would look usable and not be.
var version string
if raw, ok := entry["versionId"]; ok {
if err := json.Unmarshal(raw, &version); err == nil && version != "" {
id += "@" + version
}
}
ids = append(ids, id)
}
return shape.envelope, ids
}
return "", nil
}
// verdict renders the one-line answer for the summary table.
func verdict(r result) string {
switch {
case r.err != nil:
return "REQUEST FAILED"
case r.status == http.StatusOK && len(r.ids) > 0:
return "USABLE — listing returned"
case r.status == http.StatusOK:
return "200 but no ids parsed"
case r.status == http.StatusUnauthorized, r.status == http.StatusForbidden:
return "CREDENTIAL REJECTED"
case r.status == http.StatusNotFound:
return "NOT SERVED HERE"
default:
return "UNEXPECTED"
}
}
func shapeNote(r result) string {
if r.shape == "" {
return ""
}
return fmt.Sprintf("shape=%s ids=%d", r.shape, len(r.ids))
}
func sample(ids []string, n int) []string {
if len(ids) <= n {
return ids
}
return append(append([]string{}, ids[:n]...), fmt.Sprintf("… +%d more", len(ids)-n))
}
func truncate(s string, limit int) string {
if len(s) <= limit {
return s
}
return s[:limit] + fmt.Sprintf("… (%d more bytes)", len(s)-limit)
}

Some files were not shown because too many files have changed in this diff Show More