mirror of
https://github.com/netbirdio/netbird.git
synced 2026-08-16 20:51:28 +02:00
Compare commits
10 Commits
revert/com
...
agent-netw
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e2a09a648b | ||
|
|
a2be755b7d | ||
|
|
93e97f4bf1 | ||
|
|
16544dbc58 | ||
|
|
f458c1f265 | ||
|
|
ec6f1b8c27 | ||
|
|
2cfe14d7ec | ||
|
|
85dd335836 | ||
|
|
5544761b47 | ||
|
|
1d372bb634 |
9
.github/workflows/golang-test-linux.yml
vendored
9
.github/workflows/golang-test-linux.yml
vendored
@@ -730,11 +730,6 @@ jobs:
|
||||
- name: Install modules
|
||||
run: go mod tidy
|
||||
|
||||
- name: Run Mage
|
||||
uses: magefile/mage-action@a662bd8c29d8106879588cfff83b2faf6e6f59db # v4.0.0
|
||||
with:
|
||||
install-only: true
|
||||
|
||||
- name: check git status
|
||||
run: git --no-pager diff --exit-code
|
||||
|
||||
@@ -743,7 +738,9 @@ jobs:
|
||||
CGO_ENABLED=1 GOARCH=${{ matrix.arch }} \
|
||||
NETBIRD_STORE_ENGINE=${{ matrix.store }} \
|
||||
CI=true \
|
||||
mage integrationtest:all -gotestflags="-coverprofile=coverage.txt"
|
||||
go test -tags=integration -coverprofile=coverage.txt \
|
||||
-exec 'sudo --preserve-env=CI,NETBIRD_STORE_ENGINE' \
|
||||
-timeout 20m ./management/server/http/...
|
||||
|
||||
- name: Upload coverage reports to Codecov
|
||||
if: matrix.arch == 'amd64'
|
||||
|
||||
42
.github/workflows/ui-translations.yml
vendored
Normal file
42
.github/workflows/ui-translations.yml
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
name: UI Translations
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "client/ui/i18n/locales/**"
|
||||
- "client/ui/i18n/check-translations.mjs"
|
||||
- ".github/workflows/ui-translations.yml"
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "client/ui/i18n/locales/**"
|
||||
- "client/ui/i18n/check-translations.mjs"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || github.actor_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
check-translations:
|
||||
name: Check translation key parity
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22"
|
||||
|
||||
# English (en) is the source of truth for translation keys; every other
|
||||
# locale declared in _index.json must carry the exact same key set.
|
||||
- name: Check translation key parity
|
||||
run: node client/ui/i18n/check-translations.mjs
|
||||
@@ -1,6 +1,6 @@
|
||||
# NetBird Agent Guidelines
|
||||
|
||||
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
|
||||
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
|
||||
network with a control plane. The **agent** (`client/`) runs on user machines as
|
||||
a privileged daemon and manages the WireGuard interface, routing, firewall, and
|
||||
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,
|
||||
|
||||
@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
|
||||
|
||||
## Checklist before submitting a PR
|
||||
|
||||
As a critical network service and open-source project, we must enforce a few
|
||||
As a critical network service and open source project, we must enforce a few
|
||||
things before submitting a pull request. The
|
||||
[pull request template](/.github/pull_request_template.md) mirrors this list —
|
||||
fill it in rather than deleting it.
|
||||
|
||||
@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
|
||||

|
||||
|
||||
### Acknowledgements
|
||||
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
|
||||
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
|
||||
|
||||
### Legal
|
||||
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.
|
||||
|
||||
@@ -14,7 +14,7 @@ Report security issues one of these two ways:
|
||||
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
|
||||
- **Email** — `security@netbird.io`.
|
||||
|
||||
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
|
||||
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
|
||||
filing a repository report.
|
||||
|
||||
### What to include
|
||||
|
||||
@@ -67,6 +67,30 @@ components:
|
||||
— the management-side control plane: providers, policies, guardrails, limits, routing,
|
||||
and usage/access logs.
|
||||
|
||||
## Access roles
|
||||
|
||||
Agent Network permissions build on the account permission matrix
|
||||
([`management/server/permissions/`](../management/server/permissions)). The
|
||||
`agent_network` area is split into dotted submodules (`agent_network.providers`,
|
||||
`.policies`, `.guardrails`, `.budgets`, `.usage`, `.logs`, `.settings`); a role may
|
||||
grant a single submodule or the parent, which cascades to all of them.
|
||||
|
||||
Two roles delegate Agent Network access without account-admin rights:
|
||||
|
||||
- **`agent_network_admin`** — full control over the whole `agent_network` area plus
|
||||
read-only users, groups, peers, and account info (needed to build policies).
|
||||
Nothing else in the account.
|
||||
- **`usage_viewer`** — the regular User baseline plus read on
|
||||
`agent_network.usage` (the aggregated usage and cost overview). No provider
|
||||
configuration, no policies, no request-level access logs.
|
||||
|
||||
Every authenticated user, regardless of role, can read the caller-scoped
|
||||
self-service endpoints: `GET /api/agent-network/me/setup` (the endpoint, providers,
|
||||
and models the caller's own policies allow — what a local AI tool needs and nothing
|
||||
more) and `GET /api/agent-network/me/consumption` (the caller's own token and cost
|
||||
counters). Role definitions live in
|
||||
[`management/server/permissions/roles/`](../management/server/permissions/roles).
|
||||
|
||||
## Documentation
|
||||
|
||||
Full documentation, architecture, and quickstart:
|
||||
|
||||
@@ -204,8 +204,9 @@ func (a *Auth) foregroundGetTokenInfo(authClient *auth.Auth, urlOpener URLOpener
|
||||
return nil, fmt.Errorf("failed to get OAuth flow: %v", err)
|
||||
}
|
||||
|
||||
// An empty hint is deliberate, not a fallback: a fresh or logged-out profile
|
||||
// leaves the choice to the IdP, which is how accounts get switched.
|
||||
// An empty hint is deliberate, not a fallback: a fresh profile leaves the
|
||||
// choice to the IdP. Switching accounts is done by switching or removing
|
||||
// profiles, not by logging out — logout keeps the email.
|
||||
if a.cfgPath != "" {
|
||||
if hint := readProfileEmail(a.cfgPath); hint != "" {
|
||||
if setter, ok := oAuthFlow.(loginHintSetter); ok {
|
||||
|
||||
@@ -22,7 +22,8 @@ type Profile struct {
|
||||
ID string
|
||||
Name string
|
||||
// Email is the account this profile last logged in with, "" if it never
|
||||
// completed an SSO login or was logged out. See profile_state.go.
|
||||
// completed an SSO login. Kept across logouts; cleared when the profile is
|
||||
// removed. See profile_state.go.
|
||||
Email string
|
||||
IsActive bool
|
||||
}
|
||||
@@ -200,11 +201,9 @@ func (pm *ProfileManager) LogoutProfile(id string) error {
|
||||
return fmt.Errorf("failed to save config: %w", err)
|
||||
}
|
||||
|
||||
// Not fatal: a stale hint costs an account switch, not the logout itself.
|
||||
if err := removeProfileEmail(configPath); err != nil {
|
||||
log.Warnf("failed to clear stored account email for profile %s: %v", id, err)
|
||||
}
|
||||
|
||||
// The stored account email is kept on purpose, matching the desktop and CLI
|
||||
// logout semantics: the next login passes it as the login_hint so the IdP
|
||||
// preselects the account. Removing the profile is what deletes it.
|
||||
log.Infof("logged out from profile: %s", id)
|
||||
return nil
|
||||
}
|
||||
@@ -224,11 +223,24 @@ func (pm *ProfileManager) RenameProfile(id string, newName string) error {
|
||||
|
||||
// RemoveProfile deletes a profile
|
||||
func (pm *ProfileManager) RemoveProfile(id string) error {
|
||||
configPath, err := pm.getProfileConfigPath(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Use ServiceManager (removes profile from profiles/ directory)
|
||||
if err := pm.serviceMgr.RemoveProfile(profilemanager.ID(id), androidUsername); err != nil {
|
||||
return fmt.Errorf("failed to remove profile: %w", err)
|
||||
}
|
||||
|
||||
// The account file is this package's, not the ServiceManager's, so it must
|
||||
// go here. The default profile has a fixed filename, so a recreated one
|
||||
// would otherwise inherit the deleted profile's email as its login_hint.
|
||||
// Not fatal: the profile itself is gone.
|
||||
if err := removeProfileEmail(configPath); err != nil {
|
||||
log.Warnf("failed to remove stored account email for profile %s: %v", id, err)
|
||||
}
|
||||
|
||||
log.Infof("removed profile: %s", id)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -90,10 +90,10 @@ func writeProfileEmail(configPath string, email string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// removeProfileEmail drops the stored account email. Called on logout: while the
|
||||
// email is on disk it goes out as a login_hint, which would steer the next login
|
||||
// straight back into the account just logged out of. Mirrors the desktop UI's
|
||||
// RemoveProfileState call.
|
||||
// removeProfileEmail drops the stored account email. Called on profile removal,
|
||||
// not on logout: a logged-out profile keeps its email so the next login passes
|
||||
// it as the login_hint, matching the desktop and CLI semantics. Mirrors the
|
||||
// desktop UI's RemoveProfileState call.
|
||||
func removeProfileEmail(configPath string) error {
|
||||
accountPath, err := profileAccountPathFor(configPath)
|
||||
if err != nil {
|
||||
|
||||
@@ -127,10 +127,10 @@ func TestWriteThenReadProfileEmail(t *testing.T) {
|
||||
t.Fatalf("remove: %v", err)
|
||||
}
|
||||
if got := readProfileEmail(configPath); got != "" {
|
||||
t.Errorf("expected no email after logout, got %q", got)
|
||||
t.Errorf("expected no email after removal, got %q", got)
|
||||
}
|
||||
|
||||
// Logout may run on a never-logged-in profile, so a second remove must pass.
|
||||
// Removal may run on a never-logged-in profile, so a second remove must pass.
|
||||
if err := removeProfileEmail(configPath); err != nil {
|
||||
t.Fatalf("second remove should be a no-op: %v", err)
|
||||
}
|
||||
|
||||
@@ -305,6 +305,12 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
|
||||
return domain
|
||||
}
|
||||
|
||||
// A reverse zone names an address prefix, so it follows the address rules,
|
||||
// which also keeps its digit labels intact.
|
||||
if zone, ok := a.anonymizeReverseZone(baseDomain); ok {
|
||||
return withTrailingDot(zone, hasDot)
|
||||
}
|
||||
|
||||
if suffix := protectedSuffix(baseDomain); suffix != "" {
|
||||
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
|
||||
return domain
|
||||
@@ -405,6 +411,10 @@ func (a *Anonymizer) AnonymizeString(str string) string {
|
||||
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
|
||||
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
|
||||
|
||||
// Reverse zones go first and are then held out of the passes below: their
|
||||
// labels are digits, which the address patterns would otherwise consume.
|
||||
str, restoreZones := a.replaceReverseZones(str)
|
||||
|
||||
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
|
||||
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
|
||||
|
||||
@@ -425,7 +435,7 @@ func (a *Anonymizer) AnonymizeString(str string) string {
|
||||
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
|
||||
}
|
||||
|
||||
return str
|
||||
return restoreZones(str)
|
||||
}
|
||||
|
||||
// sortedDomains returns the domain mappings longest-first, so a full-FQDN
|
||||
|
||||
174
client/anonymize/reverse_zone.go
Normal file
174
client/anonymize/reverse_zone.go
Normal file
@@ -0,0 +1,174 @@
|
||||
package anonymize
|
||||
|
||||
import (
|
||||
"encoding/hex"
|
||||
"net/netip"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
reverseZoneSuffixV4 = ".in-addr.arpa"
|
||||
reverseZoneSuffixV6 = ".ip6.arpa"
|
||||
|
||||
v6Nibbles = 32
|
||||
v4Octets = 4
|
||||
)
|
||||
|
||||
// reverseZoneRegexes match a reverse zone or a full reverse name in free text.
|
||||
// They are applied before the address passes of AnonymizeString, whose IPv4
|
||||
// pattern would otherwise consume the digit labels of a zone and replace parts
|
||||
// of it with unrelated addresses.
|
||||
var reverseZoneRegexes = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(?:[0-9]{1,3}\.){1,4}in-addr\.arpa\b`),
|
||||
regexp.MustCompile(`(?:[0-9a-fA-F]\.){1,32}ip6\.arpa\b`),
|
||||
}
|
||||
|
||||
// anonymizeReverseZone maps a reverse zone to the zone of the anonymized form
|
||||
// of the prefix it encodes, so it follows the address rules rather than the
|
||||
// domain ones: the zone of an address that is preserved is preserved too, and
|
||||
// the zone of one that is replaced names the replacement. This keeps a reverse
|
||||
// zone recognizable as such, and consistent with the addresses it belongs to
|
||||
// elsewhere in the same output. It reports false for anything that is not a
|
||||
// reverse zone.
|
||||
func (a *Anonymizer) anonymizeReverseZone(domain string) (string, bool) {
|
||||
prefix, labelCount, suffix, ok := parseReverseZone(domain)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
|
||||
anonymized := a.AnonymizeIP(prefix)
|
||||
if anonymized == prefix {
|
||||
return domain, true
|
||||
}
|
||||
|
||||
return reverseZoneName(anonymized, labelCount) + suffix, true
|
||||
}
|
||||
|
||||
// replaceReverseZones anonymizes every reverse zone in str and swaps each one
|
||||
// for a placeholder, returning a function that puts the anonymized zones back.
|
||||
// The placeholders carry no dots, digits or colons, so no later pass matches
|
||||
// them.
|
||||
func (a *Anonymizer) replaceReverseZones(str string) (string, func(string) string) {
|
||||
var zones []string
|
||||
|
||||
for _, re := range reverseZoneRegexes {
|
||||
str = re.ReplaceAllStringFunc(str, func(match string) string {
|
||||
zone, ok := a.anonymizeReverseZone(match)
|
||||
if !ok {
|
||||
return match
|
||||
}
|
||||
|
||||
zones = append(zones, zone)
|
||||
return reverseZonePlaceholder(len(zones) - 1)
|
||||
})
|
||||
}
|
||||
|
||||
if len(zones) == 0 {
|
||||
return str, func(s string) string { return s }
|
||||
}
|
||||
|
||||
return str, func(s string) string {
|
||||
for i, zone := range zones {
|
||||
s = strings.ReplaceAll(s, reverseZonePlaceholder(i), zone)
|
||||
}
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
func reverseZonePlaceholder(index int) string {
|
||||
return "\x00reversezone" + strconv.Itoa(index) + "\x00"
|
||||
}
|
||||
|
||||
// parseReverseZone turns a reverse zone into the address of the prefix its
|
||||
// labels spell backwards, padding the absent low-order part with zeroes, and
|
||||
// returns the label count and zone suffix so the name can be rebuilt.
|
||||
func parseReverseZone(domain string) (netip.Addr, int, string, bool) {
|
||||
lower := strings.ToLower(domain)
|
||||
|
||||
switch {
|
||||
case strings.HasSuffix(lower, reverseZoneSuffixV4):
|
||||
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV4), ".")
|
||||
addr, ok := reverseZoneAddrV4(labels)
|
||||
return addr, len(labels), reverseZoneSuffixV4, ok
|
||||
case strings.HasSuffix(lower, reverseZoneSuffixV6):
|
||||
labels := strings.Split(strings.TrimSuffix(lower, reverseZoneSuffixV6), ".")
|
||||
addr, ok := reverseZoneAddrV6(labels)
|
||||
return addr, len(labels), reverseZoneSuffixV6, ok
|
||||
default:
|
||||
return netip.Addr{}, 0, "", false
|
||||
}
|
||||
}
|
||||
|
||||
func reverseZoneAddrV4(labels []string) (netip.Addr, bool) {
|
||||
if len(labels) == 0 || len(labels) > v4Octets {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
|
||||
var octets [v4Octets]byte
|
||||
for i, label := range labels {
|
||||
octet, err := strconv.ParseUint(label, 10, 8)
|
||||
if err != nil {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
octets[len(labels)-1-i] = byte(octet)
|
||||
}
|
||||
|
||||
return netip.AddrFrom4(octets), true
|
||||
}
|
||||
|
||||
func reverseZoneAddrV6(labels []string) (netip.Addr, bool) {
|
||||
if len(labels) == 0 || len(labels) > v6Nibbles {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
|
||||
nibbles := make([]byte, 0, v6Nibbles)
|
||||
for i := len(labels) - 1; i >= 0; i-- {
|
||||
if len(labels[i]) != 1 || !isHexDigit(labels[i][0]) {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
nibbles = append(nibbles, labels[i][0])
|
||||
}
|
||||
for len(nibbles) < v6Nibbles {
|
||||
nibbles = append(nibbles, '0')
|
||||
}
|
||||
|
||||
var groups []string
|
||||
for i := 0; i < len(nibbles); i += 4 {
|
||||
groups = append(groups, string(nibbles[i:i+4]))
|
||||
}
|
||||
|
||||
addr, err := netip.ParseAddr(strings.Join(groups, ":"))
|
||||
if err != nil {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
|
||||
return addr, true
|
||||
}
|
||||
|
||||
// reverseZoneName spells the first labelCount labels of addr backwards, the
|
||||
// inverse of parseReverseZone, without the zone suffix.
|
||||
func reverseZoneName(addr netip.Addr, labelCount int) string {
|
||||
labels := make([]string, 0, labelCount)
|
||||
|
||||
if addr.Is4() {
|
||||
octets := addr.As4()
|
||||
for i := labelCount - 1; i >= 0; i-- {
|
||||
labels = append(labels, strconv.Itoa(int(octets[i])))
|
||||
}
|
||||
return strings.Join(labels, ".")
|
||||
}
|
||||
|
||||
address := addr.As16()
|
||||
nibbles := hex.EncodeToString(address[:])
|
||||
for i := labelCount - 1; i >= 0; i-- {
|
||||
labels = append(labels, string(nibbles[i]))
|
||||
}
|
||||
|
||||
return strings.Join(labels, ".")
|
||||
}
|
||||
|
||||
func isHexDigit(c byte) bool {
|
||||
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
|
||||
}
|
||||
171
client/anonymize/reverse_zone_test.go
Normal file
171
client/anonymize/reverse_zone_test.go
Normal file
@@ -0,0 +1,171 @@
|
||||
package anonymize
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func newLeveledAnonymizer(level Level) *Anonymizer {
|
||||
a := NewAnonymizer(DefaultAddresses())
|
||||
a.SetLevel(level)
|
||||
return a
|
||||
}
|
||||
|
||||
// TestAnonymizeDomainReverseZone covers reverse zones going through the address
|
||||
// rules instead of the domain ones, so a zone stays a zone and an address that
|
||||
// is preserved keeps the zone that names it.
|
||||
func TestAnonymizeDomainReverseZone(t *testing.T) {
|
||||
// 100.64.0.0/10 is the overlay range, which is CGNAT: preserved at the
|
||||
// default level and replaced from the internal pool at the strict one
|
||||
const overlayZone = "64.100.in-addr.arpa"
|
||||
|
||||
t.Run("overlay zone preserved at the default level", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
assert.Equal(t, overlayZone, a.AnonymizeDomain(overlayZone), "should keep the zone of a preserved address")
|
||||
})
|
||||
|
||||
t.Run("private zone preserved at the default level", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
assert.Equal(t, "168.192.in-addr.arpa", a.AnonymizeDomain("168.192.in-addr.arpa"), "should keep the zone of a private address")
|
||||
})
|
||||
|
||||
t.Run("overlay zone replaced at the strict level", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelStrict)
|
||||
|
||||
got := a.AnonymizeDomain(overlayZone)
|
||||
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
|
||||
assert.NotEqual(t, overlayZone, got, "should replace the encoded prefix")
|
||||
assert.Len(t, strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV4), "."), 2,
|
||||
"should keep the label count, got %q", got)
|
||||
})
|
||||
|
||||
t.Run("public zone replaced at the default level", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
got := a.AnonymizeDomain("113.0.203.in-addr.arpa")
|
||||
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV4), "should stay a reverse zone, got %q", got)
|
||||
assert.NotEqual(t, "113.0.203.in-addr.arpa", got, "should replace a public prefix")
|
||||
})
|
||||
|
||||
t.Run("zone of an address keeps that address mapping", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
anonymizedAddr := a.AnonymizeIPString("203.0.113.7")
|
||||
got := a.AnonymizeDomain("7.113.0.203.in-addr.arpa")
|
||||
|
||||
octets := strings.Split(anonymizedAddr, ".")
|
||||
want := octets[3] + "." + octets[2] + "." + octets[1] + "." + octets[0] + reverseZoneSuffixV4
|
||||
assert.Equal(t, want, got, "should name the same replacement as the address itself")
|
||||
})
|
||||
|
||||
t.Run("ipv6 nibble labels stay single digits", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
|
||||
got := a.AnonymizeDomain(zone)
|
||||
|
||||
require.True(t, strings.HasSuffix(got, reverseZoneSuffixV6), "should stay a reverse zone, got %q", got)
|
||||
labels := strings.Split(strings.TrimSuffix(got, reverseZoneSuffixV6), ".")
|
||||
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
|
||||
for _, label := range labels {
|
||||
assert.Len(t, label, 1, "nibble label %q should stay a single digit", label)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("trailing dot is kept", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
assert.Equal(t, "64.100.in-addr.arpa.", a.AnonymizeDomain("64.100.in-addr.arpa."), "should keep the trailing dot")
|
||||
})
|
||||
|
||||
t.Run("a domain that only looks like a zone is anonymized as a domain", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
got := a.AnonymizeDomain("not-a-zone.in-addr.arpa")
|
||||
assert.NotContains(t, got, "in-addr.arpa", "should fall back to domain anonymization")
|
||||
})
|
||||
}
|
||||
|
||||
// TestAnonymizeStringReverseZone verifies that a zone inside free text, such as
|
||||
// a DNS log line, is not chewed up by the address passes. The IPv4 pattern
|
||||
// matches any run of dotted digits, which a reverse zone is made of.
|
||||
func TestAnonymizeStringReverseZone(t *testing.T) {
|
||||
t.Run("ipv6 zone survives the address passes", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
zone := "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6
|
||||
got := a.AnonymizeString("question: domain=" + zone + " type=PTR")
|
||||
|
||||
assert.Contains(t, got, "type=PTR", "should keep the rest of the line")
|
||||
assert.NotContains(t, got, "198.51.100", "should not rewrite nibble labels as an address")
|
||||
|
||||
labels := strings.Split(strings.TrimSuffix(strings.TrimPrefix(got, "question: domain="), reverseZoneSuffixV6+" type=PTR"), ".")
|
||||
assert.Len(t, labels, 28, "should keep every nibble label, got %q", got)
|
||||
})
|
||||
|
||||
t.Run("preserved ipv4 zone is untouched", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
line := "reverse zone 64.100.in-addr.arpa registered"
|
||||
assert.Equal(t, line, a.AnonymizeString(line), "should keep the zone of a preserved address")
|
||||
})
|
||||
|
||||
t.Run("public ipv4 zone is replaced consistently", func(t *testing.T) {
|
||||
a := newLeveledAnonymizer(LevelDefault)
|
||||
|
||||
got := a.AnonymizeString("zone 113.0.203.in-addr.arpa and address 203.0.113.7")
|
||||
assert.NotContains(t, got, "113.0.203.in-addr.arpa", "should replace the zone")
|
||||
assert.NotContains(t, got, "203.0.113.7", "should replace the address")
|
||||
assert.Contains(t, got, reverseZoneSuffixV4, "should keep the zone suffix")
|
||||
})
|
||||
}
|
||||
|
||||
func TestParseReverseZone(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
zone string
|
||||
addr string
|
||||
labels int
|
||||
}{
|
||||
{name: "v4 two labels", zone: "0.100" + reverseZoneSuffixV4, addr: "100.0.0.0", labels: 2},
|
||||
{name: "v4 three labels", zone: "1.168.192" + reverseZoneSuffixV4, addr: "192.168.1.0", labels: 3},
|
||||
{name: "v4 full address", zone: "7.113.0.203" + reverseZoneSuffixV4, addr: "203.0.113.7", labels: 4},
|
||||
{
|
||||
name: "v6 prefix",
|
||||
zone: "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.0" + reverseZoneSuffixV6,
|
||||
addr: "2::",
|
||||
labels: 28,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
addr, labels, suffix, ok := parseReverseZone(tc.zone)
|
||||
require.True(t, ok, "should decode the reverse zone")
|
||||
assert.Equal(t, tc.addr, addr.String(), "should decode to the encoded prefix")
|
||||
assert.Equal(t, tc.labels, labels, "should count the labels")
|
||||
assert.Equal(t, tc.zone, reverseZoneName(addr, labels)+suffix, "should re-encode to the original zone")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseReverseZoneRejectsNonZones(t *testing.T) {
|
||||
tests := []string{
|
||||
"example.com",
|
||||
"in-addr.arpa",
|
||||
"x.100" + reverseZoneSuffixV4,
|
||||
"256" + reverseZoneSuffixV4,
|
||||
"1.2.3.4.5" + reverseZoneSuffixV4,
|
||||
"ab" + reverseZoneSuffixV6,
|
||||
"g" + reverseZoneSuffixV6,
|
||||
}
|
||||
|
||||
for _, zone := range tests {
|
||||
t.Run(zone, func(t *testing.T) {
|
||||
_, _, _, ok := parseReverseZone(zone)
|
||||
assert.False(t, ok, "should reject %q", zone)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -124,7 +124,7 @@ func startManagement(t *testing.T, config *config.Config, testFile string) (*grp
|
||||
|
||||
updateManager := update_channel.NewPeersUpdateManager(metrics)
|
||||
requestBuffer := mgmt.NewAccountRequestBuffer(ctx, store)
|
||||
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config, nil)
|
||||
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config)
|
||||
|
||||
accountManager, err := mgmt.BuildManager(ctx, config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
|
||||
if err != nil {
|
||||
|
||||
@@ -146,7 +146,7 @@ func startManagement(t *testing.T, signalAddr string) string {
|
||||
|
||||
updateManager := update_channel.NewPeersUpdateManager(metrics)
|
||||
requestBuffer := mgmt.NewAccountRequestBuffer(context.Background(), testStore)
|
||||
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg, nil)
|
||||
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg)
|
||||
accountManager, err := mgmt.BuildManager(context.Background(), cfg, testStore, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
|
||||
require.NoError(t, err)
|
||||
|
||||
|
||||
@@ -51,6 +51,7 @@ nftables.txt: Anonymized nftables rules with packet counters across all families
|
||||
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
|
||||
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
|
||||
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
|
||||
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
|
||||
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
|
||||
config.txt: Anonymized configuration information of the NetBird client.
|
||||
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
|
||||
@@ -237,6 +238,13 @@ scutil_dns.txt (macOS only):
|
||||
- Shows DNS configuration for all network interfaces
|
||||
- Includes search domains, nameservers, and DNS resolver settings
|
||||
- All IP addresses and domain names are anonymized
|
||||
|
||||
dns_windows.txt (Windows only):
|
||||
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
|
||||
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
|
||||
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
|
||||
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
|
||||
- All IP addresses and domain names are anonymized
|
||||
`
|
||||
|
||||
const (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//go:build !unix
|
||||
//go:build !unix && !windows
|
||||
|
||||
package debug
|
||||
|
||||
|
||||
443
client/internal/debug/debug_windows.go
Normal file
443
client/internal/debug/debug_windows.go
Normal file
@@ -0,0 +1,443 @@
|
||||
//go:build windows
|
||||
|
||||
package debug
|
||||
|
||||
import (
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"unsafe"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
"golang.org/x/sys/windows"
|
||||
"golang.org/x/sys/windows/registry"
|
||||
|
||||
nbdns "github.com/netbirdio/netbird/client/internal/dns"
|
||||
)
|
||||
|
||||
const dnsInfoFileName = "dns_windows.txt"
|
||||
|
||||
const (
|
||||
gpoDNSClientRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient`
|
||||
tcpipParamsPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters`
|
||||
dnscacheParams = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters`
|
||||
)
|
||||
|
||||
// interfaceDNSValues are the per-interface values that decide how a name is
|
||||
// resolved and registered. Everything the DNS host manager writes is in here,
|
||||
// so a bundle shows both what we set and what it replaced.
|
||||
var interfaceDNSValues = []string{
|
||||
"NameServer",
|
||||
"DhcpNameServer",
|
||||
"Domain",
|
||||
"DhcpDomain",
|
||||
"SearchList",
|
||||
"RegistrationEnabled",
|
||||
"DisableDynamicUpdate",
|
||||
"MaxNumberOfAddressesToRegister",
|
||||
"EnableDHCP",
|
||||
}
|
||||
|
||||
// addDNSInfo collects and adds DNS configuration information to the archive
|
||||
func (g *BundleGenerator) addDNSInfo() error {
|
||||
if err := g.addFileToZip(strings.NewReader(g.collectDNSInfo()), dnsInfoFileName); err != nil {
|
||||
return fmt.Errorf("add DNS info to zip: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// collectDNSInfo renders the report. Everything below it reaches the platform
|
||||
// through COM and through lazily resolved procedures, which panic when a
|
||||
// procedure is missing rather than returning an error, and a debug bundle is not
|
||||
// allowed to take the daemon down. The panic is contained here, and whatever was
|
||||
// collected before it is kept and reported with it.
|
||||
func (g *BundleGenerator) collectDNSInfo() (content string) {
|
||||
var sb strings.Builder
|
||||
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
log.Errorf("collecting Windows DNS configuration panicked: %v", r)
|
||||
fmt.Fprintf(&sb, "\nerror: collection stopped: %v\n", r)
|
||||
}
|
||||
content = sb.String()
|
||||
}()
|
||||
|
||||
sb.WriteString("Windows DNS configuration\n")
|
||||
sb.WriteString("=========================\n")
|
||||
|
||||
adapters, adaptersErr := adapterAddresses()
|
||||
|
||||
g.writeNRPTRules(&sb, "NRPT rules, local policy store", nbdns.DNSPolicyConfigRoot)
|
||||
g.writeNRPTRules(&sb, "NRPT rules, group policy store", nbdns.GPODNSPolicyConfigRoot)
|
||||
g.writeEffectiveNRPTPolicies(&sb)
|
||||
g.writeRegistryKey(&sb, "DNS client group policy", gpoDNSClientRoot)
|
||||
g.writeRegistryKey(&sb, "Global TCP/IP parameters", tcpipParamsPath)
|
||||
g.writeRegistryKey(&sb, "Dnscache parameters", dnscacheParams)
|
||||
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv4", nbdns.InterfaceConfigPath, adapterNames(adapters))
|
||||
g.writeInterfaceDNS(&sb, "Per-interface DNS, IPv6", nbdns.InterfaceConfigPathV6, adapterNames(adapters))
|
||||
g.writeAdapterDNS(&sb, adapters, adaptersErr)
|
||||
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// writeNRPTRules lists every rule in a policy store, ours and any other
|
||||
// product's, since a foreign rule for the same namespace decides resolution
|
||||
// just as ours does. Rules the client wrote are marked.
|
||||
func (g *BundleGenerator) writeNRPTRules(sb *strings.Builder, title, root string) {
|
||||
writeSection(sb, title, root)
|
||||
|
||||
names, err := subKeyNames(root)
|
||||
if err != nil {
|
||||
fmt.Fprintf(sb, "error: %v\n", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(names) == 0 {
|
||||
sb.WriteString("no rules\n")
|
||||
return
|
||||
}
|
||||
|
||||
for _, name := range names {
|
||||
owner := ""
|
||||
if strings.HasPrefix(strings.ToLower(name), strings.ToLower(nbdns.NRPTKeyPrefix)) {
|
||||
owner = " (netbird)"
|
||||
}
|
||||
fmt.Fprintf(sb, "%s%s\n", name, owner)
|
||||
g.writeValues(sb, root+`\`+name, nil, " ")
|
||||
}
|
||||
}
|
||||
|
||||
// writeEffectiveNRPTPolicies reports the table the resolver answers from, which
|
||||
// the registry cannot show: a rule is written before it is loaded, and it keeps
|
||||
// being enforced after its key is gone until the resolver reloads its policy.
|
||||
func (g *BundleGenerator) writeEffectiveNRPTPolicies(sb *strings.Builder) {
|
||||
writeSection(sb, "NRPT policy table in effect", nrptPolicyClass+"."+nrptPolicyMethod+" in "+nrptPolicyNamespace)
|
||||
|
||||
entries, err := effectiveNRPTPolicies()
|
||||
if err != nil {
|
||||
fmt.Fprintf(sb, "error: %v\n", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(entries) == 0 {
|
||||
sb.WriteString("no policies\n")
|
||||
return
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
fmt.Fprintf(sb, "%s\n", g.anonymizeValue("Namespace", entry.namespace))
|
||||
for _, value := range entry.values {
|
||||
fmt.Fprintf(sb, " %s: %s\n", value.name, g.anonymizeValue(value.name, value.value))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// writeInterfaceDNS reports the DNS values of every interface that has any, so
|
||||
// the netbird interface can be compared against the physical ones. The registry
|
||||
// keys the values by GUID, so each is named from the adapter list; a GUID with
|
||||
// no adapter is a leftover key of an interface that no longer exists.
|
||||
func (g *BundleGenerator) writeInterfaceDNS(sb *strings.Builder, title, root string, names map[string]string) {
|
||||
writeSection(sb, title, root)
|
||||
|
||||
guids, err := subKeyNames(root)
|
||||
if err != nil {
|
||||
fmt.Fprintf(sb, "error: %v\n", err)
|
||||
return
|
||||
}
|
||||
|
||||
var reported int
|
||||
for _, guid := range guids {
|
||||
var iface strings.Builder
|
||||
g.writeValues(&iface, root+`\`+guid, interfaceDNSValues, " ")
|
||||
if iface.Len() == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
name, ok := names[strings.ToLower(guid)]
|
||||
if !ok {
|
||||
name = "no adapter with this GUID"
|
||||
}
|
||||
|
||||
reported++
|
||||
fmt.Fprintf(sb, "%s (%s)\n%s", guid, name, iface.String())
|
||||
}
|
||||
|
||||
if reported == 0 {
|
||||
sb.WriteString("no interface holds DNS values\n")
|
||||
}
|
||||
}
|
||||
|
||||
// writeRegistryKey reports the values of a single key, without its subkeys.
|
||||
func (g *BundleGenerator) writeRegistryKey(sb *strings.Builder, title, path string) {
|
||||
writeSection(sb, title, path)
|
||||
|
||||
var values strings.Builder
|
||||
g.writeValues(&values, path, nil, "")
|
||||
if values.Len() == 0 {
|
||||
sb.WriteString("no values\n")
|
||||
return
|
||||
}
|
||||
|
||||
sb.WriteString(values.String())
|
||||
}
|
||||
|
||||
// writeValues renders the values of a key. A nil names list reports every
|
||||
// value, otherwise only those named and present.
|
||||
func (g *BundleGenerator) writeValues(sb *strings.Builder, path string, names []string, indent string) {
|
||||
k, err := registry.OpenKey(registry.LOCAL_MACHINE, path, registry.QUERY_VALUE)
|
||||
switch {
|
||||
case errors.Is(err, registry.ErrNotExist), errors.Is(err, windows.ERROR_PATH_NOT_FOUND):
|
||||
// an absent key is the normal state for the GPO store and for
|
||||
// interfaces without DNS settings
|
||||
log.Debugf("HKEY_LOCAL_MACHINE\\%s does not exist", path)
|
||||
return
|
||||
case err != nil:
|
||||
fmt.Fprintf(sb, "%serror: open HKEY_LOCAL_MACHINE\\%s: %v\n", indent, path, err)
|
||||
return
|
||||
}
|
||||
defer closeKey(k)
|
||||
|
||||
if names == nil {
|
||||
names, err = k.ReadValueNames(-1)
|
||||
if err != nil {
|
||||
fmt.Fprintf(sb, "%serror: read value names: %v\n", indent, err)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
for _, name := range names {
|
||||
value, err := readRegistryValue(k, name)
|
||||
switch {
|
||||
case errors.Is(err, registry.ErrNotExist):
|
||||
// the caller asks for a fixed set of values, most of which a
|
||||
// given interface does not carry
|
||||
continue
|
||||
case err != nil:
|
||||
// report rather than omit: a value that is there but cannot be
|
||||
// read reads as unset otherwise
|
||||
fmt.Fprintf(sb, "%s%s: error: %v\n", indent, name, err)
|
||||
continue
|
||||
}
|
||||
|
||||
fmt.Fprintf(sb, "%s%s: %s\n", indent, name, g.anonymizeValue(name, value))
|
||||
}
|
||||
}
|
||||
|
||||
// anonymizeValue redacts a registry value according to what its name says it
|
||||
// holds. Domains and addresses are handled per entry rather than by the string
|
||||
// pass: the pass only replaces domains something else in the bundle already
|
||||
// seeded, and its address regex would eat the digit labels of a reverse zone.
|
||||
func (g *BundleGenerator) anonymizeValue(name, value string) string {
|
||||
if !g.anonymize || value == "" {
|
||||
return value
|
||||
}
|
||||
|
||||
switch {
|
||||
case holdsDomains(name):
|
||||
return joinValueEntries(splitValueEntries(value), g.anonymizeDomain)
|
||||
case holdsAddresses(name):
|
||||
return joinValueEntries(splitValueEntries(value), g.anonymizer.AnonymizeIPString)
|
||||
default:
|
||||
return g.anonymizer.AnonymizeString(value)
|
||||
}
|
||||
}
|
||||
|
||||
// holdsDomains reports whether a value name holds domains: the domain list of
|
||||
// an NRPT rule (Name) or of the policy table (Namespace), a search list, the
|
||||
// DNS suffix values of the TCP/IP and policy keys, which all end in "Domain"
|
||||
// (Domain, DhcpDomain, NV Domain, ICSDomain), and a proxy host name.
|
||||
func holdsDomains(name string) bool {
|
||||
lower := strings.ToLower(name)
|
||||
return lower == "name" || lower == "namespace" || lower == "searchlist" ||
|
||||
strings.HasSuffix(lower, "domain") || strings.HasSuffix(lower, "proxyname")
|
||||
}
|
||||
|
||||
// holdsAddresses reports whether a value name holds DNS server addresses
|
||||
// (NameServer, DhcpNameServer, GenericDNSServers, NameServers).
|
||||
func holdsAddresses(name string) bool {
|
||||
lower := strings.ToLower(name)
|
||||
return strings.Contains(lower, "nameserver") || strings.Contains(lower, "dnsserver")
|
||||
}
|
||||
|
||||
// adapterNames maps adapter GUIDs, as the registry keys the interfaces, to the
|
||||
// names an operator sees.
|
||||
func adapterNames(adapters []*windows.IpAdapterAddresses) map[string]string {
|
||||
names := make(map[string]string, len(adapters))
|
||||
for _, adapter := range adapters {
|
||||
guid := windows.BytePtrToString(adapter.AdapterName)
|
||||
names[strings.ToLower(guid)] = windows.UTF16PtrToString(adapter.FriendlyName)
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// writeAdapterDNS reports the resolver configuration in effect per adapter,
|
||||
// which is what the resolver uses for a name no NRPT rule matches.
|
||||
func (g *BundleGenerator) writeAdapterDNS(sb *strings.Builder, adapters []*windows.IpAdapterAddresses, err error) {
|
||||
writeSection(sb, "Adapter DNS configuration", "GetAdaptersAddresses")
|
||||
|
||||
if err != nil {
|
||||
fmt.Fprintf(sb, "error: %v\n", err)
|
||||
return
|
||||
}
|
||||
|
||||
for _, adapter := range adapters {
|
||||
name := windows.UTF16PtrToString(adapter.FriendlyName)
|
||||
suffix := g.anonymizeDomain(windows.UTF16PtrToString(adapter.DnsSuffix))
|
||||
|
||||
fmt.Fprintf(sb, "%s (index %d, oper status %d)\n", name, adapter.IfIndex, adapter.OperStatus)
|
||||
fmt.Fprintf(sb, " DNS suffix: %s\n", suffix)
|
||||
|
||||
var servers []string
|
||||
for server := adapter.FirstDnsServerAddress; server != nil; server = server.Next {
|
||||
addr, ok := netip.AddrFromSlice(server.Address.IP())
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
|
||||
addr = addr.Unmap()
|
||||
if g.anonymize {
|
||||
addr = g.anonymizer.AnonymizeIP(addr)
|
||||
}
|
||||
servers = append(servers, addr.String())
|
||||
}
|
||||
|
||||
fmt.Fprintf(sb, " DNS servers: %s\n", strings.Join(servers, ", "))
|
||||
}
|
||||
}
|
||||
|
||||
// anonymizeDomain anonymizes a single domain, keeping the leading dot an NRPT
|
||||
// match domain carries.
|
||||
func (g *BundleGenerator) anonymizeDomain(entry string) string {
|
||||
if !g.anonymize {
|
||||
return entry
|
||||
}
|
||||
|
||||
domain, dot := strings.CutPrefix(entry, ".")
|
||||
if domain == "" {
|
||||
return entry
|
||||
}
|
||||
|
||||
anonymized := g.anonymizer.AnonymizeDomain(domain)
|
||||
if dot {
|
||||
anonymized = "." + anonymized
|
||||
}
|
||||
return anonymized
|
||||
}
|
||||
|
||||
// splitValueEntries splits a registry value that holds a list. The separator
|
||||
// differs per value: a REG_MULTI_SZ arrives joined with ", ", a SearchList is
|
||||
// comma separated and a NameServer may use commas or spaces.
|
||||
func splitValueEntries(value string) []string {
|
||||
return strings.FieldsFunc(value, func(r rune) bool {
|
||||
return r == ',' || r == ';' || r == ' ' || r == '\t'
|
||||
})
|
||||
}
|
||||
|
||||
func joinValueEntries(entries []string, anonymize func(string) string) string {
|
||||
for i, entry := range entries {
|
||||
entries[i] = anonymize(entry)
|
||||
}
|
||||
return strings.Join(entries, ", ")
|
||||
}
|
||||
|
||||
func writeSection(sb *strings.Builder, title, source string) {
|
||||
fmt.Fprintf(sb, "\n%s\n%s\n%s\n", title, strings.Repeat("-", len(title)), source)
|
||||
}
|
||||
|
||||
func subKeyNames(root string) ([]string, error) {
|
||||
k, err := registry.OpenKey(registry.LOCAL_MACHINE, root, registry.ENUMERATE_SUB_KEYS)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", root, err)
|
||||
}
|
||||
defer closeKey(k)
|
||||
|
||||
names, err := k.ReadSubKeyNames(-1)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read subkey names: %w", err)
|
||||
}
|
||||
|
||||
return names, nil
|
||||
}
|
||||
|
||||
// readRegistryValue renders a value as text regardless of its type, so an
|
||||
// unexpected type in a policy key still shows up instead of being dropped.
|
||||
func readRegistryValue(k registry.Key, name string) (string, error) {
|
||||
_, valueType, err := k.GetValue(name, nil)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get value %s: %w", name, err)
|
||||
}
|
||||
|
||||
switch valueType {
|
||||
case registry.SZ, registry.EXPAND_SZ:
|
||||
value, _, err := k.GetStringValue(name)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get string value %s: %w", name, err)
|
||||
}
|
||||
return value, nil
|
||||
case registry.MULTI_SZ:
|
||||
values, _, err := k.GetStringsValue(name)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get strings value %s: %w", name, err)
|
||||
}
|
||||
return strings.Join(values, ", "), nil
|
||||
case registry.DWORD, registry.QWORD:
|
||||
value, _, err := k.GetIntegerValue(name)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get integer value %s: %w", name, err)
|
||||
}
|
||||
return fmt.Sprintf("%d (0x%x)", value, value), nil
|
||||
case registry.BINARY:
|
||||
value, _, err := k.GetBinaryValue(name)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get binary value %s: %w", name, err)
|
||||
}
|
||||
return hex.EncodeToString(value), nil
|
||||
default:
|
||||
return fmt.Sprintf("<unhandled registry type %d>", valueType), nil
|
||||
}
|
||||
}
|
||||
|
||||
// adapterAddresses returns the adapter list including DNS servers. The call
|
||||
// reports the size it needs, so grow the buffer and retry until it fits.
|
||||
func adapterAddresses() (adapters []*windows.IpAdapterAddresses, err error) {
|
||||
// GetAdaptersAddresses is resolved on first use and panics when it is
|
||||
// missing, so this reports it as an error and leaves the rest of the
|
||||
// report intact.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
adapters, err = nil, fmt.Errorf("GetAdaptersAddresses: %v", r)
|
||||
}
|
||||
}()
|
||||
|
||||
const flags = windows.GAA_FLAG_SKIP_ANYCAST | windows.GAA_FLAG_SKIP_MULTICAST
|
||||
|
||||
size := uint32(15000)
|
||||
for range 3 {
|
||||
buf := make([]byte, size)
|
||||
first := (*windows.IpAdapterAddresses)(unsafe.Pointer(&buf[0]))
|
||||
|
||||
err := windows.GetAdaptersAddresses(windows.AF_UNSPEC, flags, 0, first, &size)
|
||||
if errors.Is(err, windows.ERROR_BUFFER_OVERFLOW) {
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("GetAdaptersAddresses: %w", err)
|
||||
}
|
||||
|
||||
for adapter := first; adapter != nil; adapter = adapter.Next {
|
||||
adapters = append(adapters, adapter)
|
||||
}
|
||||
return adapters, nil
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("GetAdaptersAddresses: buffer kept growing")
|
||||
}
|
||||
|
||||
func closeKey(k registry.Key) {
|
||||
if err := k.Close(); err != nil {
|
||||
log.Debugf("close registry key: %v", err)
|
||||
}
|
||||
}
|
||||
146
client/internal/debug/debug_windows_test.go
Normal file
146
client/internal/debug/debug_windows_test.go
Normal file
@@ -0,0 +1,146 @@
|
||||
//go:build windows
|
||||
|
||||
package debug
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/netbirdio/netbird/client/anonymize"
|
||||
)
|
||||
|
||||
func newDNSValueGenerator(level anonymize.Level) *BundleGenerator {
|
||||
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
|
||||
anonymizer.SetLevel(level)
|
||||
|
||||
return &BundleGenerator{
|
||||
anonymize: true,
|
||||
anonymizeLevel: level,
|
||||
anonymizer: anonymizer,
|
||||
}
|
||||
}
|
||||
|
||||
// TestAnonymizeValueByName covers the value kinds of the DNS registry keys. The
|
||||
// names decide the treatment, because the string pass alone replaces only
|
||||
// domains another part of the bundle already seeded.
|
||||
func TestAnonymizeValueByName(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
valueName string
|
||||
value string
|
||||
assert func(t *testing.T, got string)
|
||||
}{
|
||||
{
|
||||
name: "NRPT match domains keep the leading dot",
|
||||
valueName: "Name",
|
||||
value: ".internal.example.com, .corp.example.org",
|
||||
assert: func(t *testing.T, got string) {
|
||||
t.Helper()
|
||||
for _, entry := range strings.Split(got, ", ") {
|
||||
assert.True(t, strings.HasPrefix(entry, "."), "entry %q should keep its leading dot", entry)
|
||||
assert.NotContains(t, entry, "example", "entry %q should not keep the original domain", entry)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "any value name ending in Domain is treated as a domain",
|
||||
valueName: "ICSDomain",
|
||||
value: "mshome.net",
|
||||
assert: func(t *testing.T, got string) {
|
||||
t.Helper()
|
||||
assert.NotContains(t, got, "mshome", "should anonymize a domain suffix value")
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "search list is a comma separated domain list",
|
||||
valueName: "SearchList",
|
||||
value: "corp.example.com,branch.example.com",
|
||||
assert: func(t *testing.T, got string) {
|
||||
t.Helper()
|
||||
assert.NotContains(t, got, "example", "should anonymize every search domain")
|
||||
assert.Len(t, strings.Split(got, ", "), 2, "should keep both search domains")
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "name servers are anonymized as addresses",
|
||||
valueName: "DhcpNameServer",
|
||||
value: "203.0.113.10 8.8.8.8",
|
||||
assert: func(t *testing.T, got string) {
|
||||
t.Helper()
|
||||
assert.NotContains(t, got, "203.0.113.10", "should anonymize a public resolver address")
|
||||
// well-known resolvers stay readable at every level
|
||||
assert.Contains(t, got, "8.8.8.8", "should keep a well-known resolver address")
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "opaque values are left to the string pass",
|
||||
valueName: "DataBasePath",
|
||||
value: `%SystemRoot%\System32\drivers\etc`,
|
||||
assert: func(t *testing.T, got string) {
|
||||
t.Helper()
|
||||
assert.Equal(t, `%SystemRoot%\System32\drivers\etc`, got, "should not alter a path")
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
g := newDNSValueGenerator(anonymize.LevelDefault)
|
||||
tc.assert(t, g.anonymizeValue(tc.valueName, tc.value))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestParseNRPTPolicyTable parses the MOF text of the policy table out
|
||||
// parameters, as the provider on a client with one NRPT rule renders it.
|
||||
func TestParseNRPTPolicyTable(t *testing.T) {
|
||||
const text = `[abstract]
|
||||
class __PARAMETERS
|
||||
{
|
||||
[Out, EmbeddedInstance("DnsClientPolicyConfiguration"): ToSubClass, ID(2): DisableOverride ToInstance] DnsClientPolicyConfiguration cmdletOutput[] = {
|
||||
instance of DnsClientPolicyConfiguration
|
||||
{
|
||||
DirectAccessProxyType = "NoProxy";
|
||||
DirectAccessQueryIPsecRequired = FALSE;
|
||||
NameEncoding = "Utf8WithoutMapping";
|
||||
Namespace = ".0.100.in-addr.arpa";
|
||||
},
|
||||
instance of DnsClientPolicyConfiguration
|
||||
{
|
||||
DirectAccessProxyType = "NoProxy";
|
||||
NameEncoding = "Utf8WithoutMapping";
|
||||
NameServers = {"100.0.255.254", "100.0.255.253"};
|
||||
Namespace = ".nb.internal";
|
||||
}};
|
||||
[in] boolean Effective;
|
||||
[out] uint32 ReturnValue = 0;
|
||||
};
|
||||
`
|
||||
|
||||
entries := parseNRPTPolicyTable(text)
|
||||
require.Len(t, entries, 2, "should parse both embedded instances")
|
||||
|
||||
assert.Equal(t, ".0.100.in-addr.arpa", entries[0].namespace, "should read the namespace of the first instance")
|
||||
assert.Equal(t, ".nb.internal", entries[1].namespace, "should read the namespace of the second instance")
|
||||
|
||||
assert.Equal(t, []registryValue{
|
||||
{name: "DirectAccessProxyType", value: "NoProxy"},
|
||||
{name: "DirectAccessQueryIPsecRequired", value: "FALSE"},
|
||||
{name: "NameEncoding", value: "Utf8WithoutMapping"},
|
||||
}, entries[0].values, "should keep the remaining values in order")
|
||||
|
||||
assert.Contains(t, entries[1].values, registryValue{name: "NameServers", value: "100.0.255.254, 100.0.255.253"},
|
||||
"should flatten a MOF array")
|
||||
|
||||
for _, value := range entries[1].values {
|
||||
assert.NotContains(t, value.name, "ReturnValue", "should not read the class level parameters as values")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseNRPTPolicyTableEmpty(t *testing.T) {
|
||||
assert.Empty(t, parseNRPTPolicyTable(""), "should parse no entries from empty text")
|
||||
assert.Empty(t, parseNRPTPolicyTable("class __PARAMETERS\n{\n};\n"), "should parse no entries from a table with no instances")
|
||||
}
|
||||
317
client/internal/debug/nrpt_windows.go
Normal file
317
client/internal/debug/nrpt_windows.go
Normal file
@@ -0,0 +1,317 @@
|
||||
//go:build windows
|
||||
|
||||
package debug
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-ole/go-ole"
|
||||
"github.com/go-ole/go-ole/oleutil"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
// The NRPT policy table is reachable through the CIM class that backs
|
||||
// Get-DnsClientNrptPolicy. Unlike the rules in the registry, the table is
|
||||
// what the resolver currently has loaded, which is the only way to tell an
|
||||
// applied rule from one that is merely written, in either direction.
|
||||
nrptPolicyNamespace = `root\Microsoft\Windows\DNS`
|
||||
nrptPolicyClass = "PS_DnsClientNrptPolicy"
|
||||
nrptPolicyMethod = "Get"
|
||||
|
||||
// The class has no instances, so the table comes from the out parameters
|
||||
// of a static method call, rendered as MOF text: the embedded instances
|
||||
// arrive as a safe array of objects, which cannot be read back through the
|
||||
// COM bindings, and the text form carries all of them.
|
||||
nrptPolicyInstanceKeyword = "instance of DnsClientPolicyConfiguration"
|
||||
|
||||
nrptPolicyTimeout = 15 * time.Second
|
||||
)
|
||||
|
||||
// COM initialization results that leave the calling thread usable: S_FALSE for
|
||||
// a thread this process already initialized, RPC_E_CHANGED_MODE for one that
|
||||
// belongs to another apartment.
|
||||
const (
|
||||
sFalse = 0x00000001
|
||||
rpcEChangedMode = 0x80010106
|
||||
)
|
||||
|
||||
// nrptQueryInFlight admits one read of the policy table at a time. A provider
|
||||
// that stops answering keeps its goroutine and the OS thread that goroutine
|
||||
// pinned, so a later bundle reports that instead of pinning another one.
|
||||
var nrptQueryInFlight = make(chan struct{}, 1)
|
||||
|
||||
// nrptPolicyEntry is one namespace of the effective policy table, holding the
|
||||
// values of an embedded DnsClientPolicyConfiguration instance in the order the
|
||||
// provider reported them.
|
||||
type nrptPolicyEntry struct {
|
||||
namespace string
|
||||
values []registryValue
|
||||
}
|
||||
|
||||
// registryValue is a name and its rendered value, shared by the registry and
|
||||
// policy table readers so both anonymize by value name the same way.
|
||||
type registryValue struct {
|
||||
name string
|
||||
value string
|
||||
}
|
||||
|
||||
// effectiveNRPTPolicies reads the effective NRPT table. The call is bounded
|
||||
// because a WMI provider can block indefinitely and a debug bundle must not.
|
||||
func effectiveNRPTPolicies() ([]nrptPolicyEntry, error) {
|
||||
type result struct {
|
||||
text string
|
||||
err error
|
||||
}
|
||||
|
||||
select {
|
||||
case nrptQueryInFlight <- struct{}{}:
|
||||
default:
|
||||
return nil, errors.New("an earlier read of the policy table has not returned")
|
||||
}
|
||||
|
||||
done := make(chan result, 1)
|
||||
go func() {
|
||||
// the slot is released here rather than by the caller, so a read that
|
||||
// outlives the timeout holds it until the provider answers
|
||||
defer func() { <-nrptQueryInFlight }()
|
||||
|
||||
text, err := nrptPolicyTableText()
|
||||
done <- result{text: text, err: err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case res := <-done:
|
||||
if res.err != nil {
|
||||
return nil, res.err
|
||||
}
|
||||
return parseNRPTPolicyTable(res.text), nil
|
||||
case <-time.After(nrptPolicyTimeout):
|
||||
return nil, errors.New("read of the policy table timed out")
|
||||
}
|
||||
}
|
||||
|
||||
// nrptPolicyTableText calls the policy table method and returns the MOF text of
|
||||
// its out parameters.
|
||||
func nrptPolicyTableText() (text string, err error) {
|
||||
// COM is per thread, and the collection is short lived, so the thread is
|
||||
// pinned for the duration rather than initialized for the process.
|
||||
runtime.LockOSThread()
|
||||
defer runtime.UnlockOSThread()
|
||||
|
||||
defer func() {
|
||||
// The COM call chain is dynamically typed, so a provider that answers
|
||||
// with an unexpected shape must not take the daemon down with it.
|
||||
if r := recover(); r != nil {
|
||||
err = fmt.Errorf("read NRPT policy table: %v", r)
|
||||
}
|
||||
}()
|
||||
|
||||
owns, err := coInitialize()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if owns {
|
||||
defer ole.CoUninitialize()
|
||||
}
|
||||
|
||||
locator, err := oleutil.CreateObject("WbemScripting.SWbemLocator")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create WMI locator: %w", err)
|
||||
}
|
||||
defer locator.Release()
|
||||
|
||||
dispatch, err := locator.QueryInterface(ole.IID_IDispatch)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("query WMI locator interface: %w", err)
|
||||
}
|
||||
defer dispatch.Release()
|
||||
|
||||
service, err := dispatchCall(dispatch, "ConnectServer", nil, nrptPolicyNamespace)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("connect to %s: %w", nrptPolicyNamespace, err)
|
||||
}
|
||||
defer service.Release()
|
||||
|
||||
inParams, err := spawnMethodInParams(service)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer inParams.Release()
|
||||
|
||||
// The effective table is the merge of the local and the group policy
|
||||
// store, which is what the resolver answers from.
|
||||
if _, err := oleutil.PutProperty(inParams, "Effective", true); err != nil {
|
||||
return "", fmt.Errorf("set Effective parameter: %w", err)
|
||||
}
|
||||
|
||||
outParams, err := dispatchCall(service, "ExecMethod", nrptPolicyClass, nrptPolicyMethod, inParams)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("call %s.%s: %w", nrptPolicyClass, nrptPolicyMethod, err)
|
||||
}
|
||||
defer outParams.Release()
|
||||
|
||||
textVariant, err := oleutil.CallMethod(outParams, "GetObjectText_")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("render policy table: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err := textVariant.Clear(); err != nil {
|
||||
log.Debugf("clear policy table variant: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
return textVariant.ToString(), nil
|
||||
}
|
||||
|
||||
// spawnMethodInParams builds the in parameters instance the method needs. The
|
||||
// provider rejects the call without one, even when every parameter is optional.
|
||||
func spawnMethodInParams(service *ole.IDispatch) (*ole.IDispatch, error) {
|
||||
class, err := dispatchCall(service, "Get", nrptPolicyClass)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get class %s: %w", nrptPolicyClass, err)
|
||||
}
|
||||
defer class.Release()
|
||||
|
||||
methods, err := dispatchProperty(class, "Methods_")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get class methods: %w", err)
|
||||
}
|
||||
defer methods.Release()
|
||||
|
||||
method, err := dispatchCall(methods, "Item", nrptPolicyMethod)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get method %s: %w", nrptPolicyMethod, err)
|
||||
}
|
||||
defer method.Release()
|
||||
|
||||
params, err := dispatchProperty(method, "InParameters")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get method parameters: %w", err)
|
||||
}
|
||||
defer params.Release()
|
||||
|
||||
inParams, err := dispatchCall(params, "SpawnInstance_")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("spawn parameter instance: %w", err)
|
||||
}
|
||||
|
||||
return inParams, nil
|
||||
}
|
||||
|
||||
// parseNRPTPolicyTable pulls the embedded instances out of the MOF text. Each
|
||||
// instance is a namespace of the table, with one name and value per line.
|
||||
func parseNRPTPolicyTable(text string) []nrptPolicyEntry {
|
||||
var entries []nrptPolicyEntry
|
||||
var current *nrptPolicyEntry
|
||||
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
line = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(line), ";"))
|
||||
|
||||
switch {
|
||||
case strings.HasPrefix(line, nrptPolicyInstanceKeyword):
|
||||
entries = append(entries, nrptPolicyEntry{})
|
||||
current = &entries[len(entries)-1]
|
||||
continue
|
||||
case strings.HasPrefix(line, "}"):
|
||||
// closes an instance, and the array with the last one, so the
|
||||
// class level parameters that follow are not read as values
|
||||
current = nil
|
||||
continue
|
||||
case current == nil, line == "{":
|
||||
continue
|
||||
}
|
||||
|
||||
name, value, ok := strings.Cut(line, " = ")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
|
||||
value = unquoteMOFValue(value)
|
||||
if name == "Namespace" {
|
||||
current.namespace = value
|
||||
continue
|
||||
}
|
||||
|
||||
current.values = append(current.values, registryValue{name: name, value: value})
|
||||
}
|
||||
|
||||
return entries
|
||||
}
|
||||
|
||||
// unquoteMOFValue renders a MOF scalar or array as plain text: "a" becomes a,
|
||||
// and {"a", "b"} becomes a, b.
|
||||
func unquoteMOFValue(value string) string {
|
||||
value = strings.TrimSpace(value)
|
||||
|
||||
if inner, ok := strings.CutPrefix(value, "{"); ok {
|
||||
value = strings.TrimSuffix(inner, "}")
|
||||
|
||||
entries := strings.Split(value, ",")
|
||||
for i, entry := range entries {
|
||||
entries[i] = strings.Trim(strings.TrimSpace(entry), `"`)
|
||||
}
|
||||
return strings.Join(entries, ", ")
|
||||
}
|
||||
|
||||
return strings.Trim(value, `"`)
|
||||
}
|
||||
|
||||
// coInitialize prepares the calling thread for COM and reports whether this
|
||||
// call owns the initialization, which decides whether it may be balanced with
|
||||
// CoUninitialize. S_FALSE took a reference on a thread this process had already
|
||||
// initialized and so has to be released, while RPC_E_CHANGED_MODE took none:
|
||||
// the thread belongs to another apartment, which is usable but is not ours to
|
||||
// uninitialize.
|
||||
func coInitialize() (bool, error) {
|
||||
err := ole.CoInitializeEx(0, ole.COINIT_MULTITHREADED)
|
||||
if err == nil {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
var oleErr *ole.OleError
|
||||
if errors.As(err, &oleErr) {
|
||||
switch oleErr.Code() {
|
||||
case sFalse:
|
||||
return true, nil
|
||||
case rpcEChangedMode:
|
||||
return false, nil
|
||||
}
|
||||
}
|
||||
|
||||
return false, fmt.Errorf("initialize COM: %w", err)
|
||||
}
|
||||
|
||||
// dispatchCall calls a COM method that returns an object.
|
||||
func dispatchCall(dispatch *ole.IDispatch, method string, params ...any) (*ole.IDispatch, error) {
|
||||
variant, err := oleutil.CallMethod(dispatch, method, params...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
object := variant.ToIDispatch()
|
||||
if object == nil {
|
||||
return nil, fmt.Errorf("%s returned no object", method)
|
||||
}
|
||||
|
||||
return object, nil
|
||||
}
|
||||
|
||||
// dispatchProperty reads a COM property that holds an object.
|
||||
func dispatchProperty(dispatch *ole.IDispatch, property string) (*ole.IDispatch, error) {
|
||||
variant, err := oleutil.GetProperty(dispatch, property)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
object := variant.ToIDispatch()
|
||||
if object == nil {
|
||||
return nil, fmt.Errorf("property %s holds no object", property)
|
||||
}
|
||||
|
||||
return object, nil
|
||||
}
|
||||
@@ -31,10 +31,28 @@ var (
|
||||
dnsFlushResolverCacheFn = dnsapi.NewProc("DnsFlushResolverCache")
|
||||
)
|
||||
|
||||
// Registry locations of the host DNS configuration this package programs,
|
||||
// exported so a diagnostic reader reports the same locations that are written.
|
||||
const (
|
||||
dnsPolicyConfigMatchPath = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig\NetBird-Match`
|
||||
gpoDnsPolicyRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
|
||||
gpoDnsPolicyConfigMatchPath = gpoDnsPolicyRoot + `\NetBird-Match`
|
||||
// NRPTKeyPrefix starts the name of every NRPT rule key this client creates.
|
||||
NRPTKeyPrefix = "NetBird-Match"
|
||||
|
||||
// DNSPolicyConfigRoot holds the NRPT rules of the local policy store.
|
||||
DNSPolicyConfigRoot = `SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig`
|
||||
|
||||
// GPODNSPolicyConfigRoot holds the NRPT rules of the group policy store,
|
||||
// which takes precedence over the local one when it is present.
|
||||
GPODNSPolicyConfigRoot = `SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig`
|
||||
|
||||
// InterfaceConfigPath and InterfaceConfigPathV6 hold the per-interface DNS
|
||||
// settings, keyed by interface GUID, in separate hives per address family.
|
||||
InterfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
|
||||
InterfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
|
||||
)
|
||||
|
||||
const (
|
||||
dnsPolicyConfigMatchPath = DNSPolicyConfigRoot + `\` + NRPTKeyPrefix
|
||||
gpoDnsPolicyConfigMatchPath = GPODNSPolicyConfigRoot + `\` + NRPTKeyPrefix
|
||||
|
||||
dnsPolicyConfigVersionKey = "Version"
|
||||
dnsPolicyConfigVersionValue = 2
|
||||
@@ -45,8 +63,6 @@ const (
|
||||
|
||||
nrptMaxDomainsPerRule = 50
|
||||
|
||||
interfaceConfigPath = `SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces`
|
||||
interfaceConfigPathV6 = `SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces`
|
||||
interfaceConfigNameServerKey = "NameServer"
|
||||
interfaceConfigDhcpNameSrvKey = "DhcpNameServer"
|
||||
interfaceConfigSearchListKey = "SearchList"
|
||||
@@ -84,7 +100,7 @@ func newHostManager(wgInterface WGIface) (*registryConfigurator, error) {
|
||||
}
|
||||
|
||||
var useGPO bool
|
||||
k, err := registry.OpenKey(registry.LOCAL_MACHINE, gpoDnsPolicyRoot, registry.QUERY_VALUE)
|
||||
k, err := registry.OpenKey(registry.LOCAL_MACHINE, GPODNSPolicyConfigRoot, registry.QUERY_VALUE)
|
||||
if err != nil {
|
||||
log.Debugf("failed to open GPO DNS policy root: %v", err)
|
||||
} else {
|
||||
@@ -123,7 +139,7 @@ func (r *registryConfigurator) captureOriginalNameservers() ([]netip.Addr, error
|
||||
seen := make(map[netip.Addr]struct{})
|
||||
var out []netip.Addr
|
||||
var merr *multierror.Error
|
||||
for _, root := range []string{interfaceConfigPath, interfaceConfigPathV6} {
|
||||
for _, root := range []string{InterfaceConfigPath, InterfaceConfigPathV6} {
|
||||
addrs, err := r.captureFromTcpipRoot(root)
|
||||
if err != nil {
|
||||
merr = multierror.Append(merr, fmt.Errorf("%s: %w", root, err))
|
||||
@@ -496,7 +512,7 @@ func (r *registryConfigurator) deleteInterfaceRegistryKeyProperty(propertyKey st
|
||||
}
|
||||
|
||||
func (r *registryConfigurator) getInterfaceRegistryKey() (registry.Key, error) {
|
||||
regKeyPath := interfaceConfigPath + "\\" + r.guid
|
||||
regKeyPath := InterfaceConfigPath + "\\" + r.guid
|
||||
regKey, err := registry.OpenKey(registry.LOCAL_MACHINE, regKeyPath, registry.SET_VALUE)
|
||||
if err != nil {
|
||||
return regKey, fmt.Errorf("open HKEY_LOCAL_MACHINE\\%s: %w", regKeyPath, err)
|
||||
|
||||
@@ -519,7 +519,7 @@ func startManagement(t *testing.T, dataDir, testFile string) (*grpc.Server, stri
|
||||
|
||||
updateManager := update_channel.NewPeersUpdateManager(metrics)
|
||||
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
|
||||
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
|
||||
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
|
||||
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
|
||||
@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
|
||||
|
||||
// RemoveProfileState deletes the per-profile state file (which holds the
|
||||
// account email used for the SSO login hint and the UI display). Called after
|
||||
// a successful logout so a logged-out profile no longer shows a stale account
|
||||
// email. The state file only stores the email, so deleting it is equivalent to
|
||||
// clearing it; the next SSO login recreates it. A missing file is not an error.
|
||||
// profile removal; logout keeps the file so the next login can pass the email
|
||||
// as the login_hint. The state file only stores the email, so deleting it is
|
||||
// equivalent to clearing it; the next SSO login recreates it. A missing file
|
||||
// is not an error.
|
||||
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
|
||||
configDir, err := getConfigDir()
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
//go:build windows
|
||||
|
||||
package systemops
|
||||
|
||||
import (
|
||||
"math"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestSortRouteCandidates(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
candidates []candidateRoute
|
||||
wantOrder []uint32
|
||||
}{
|
||||
{
|
||||
name: "longest prefix wins over metrics",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
|
||||
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
|
||||
},
|
||||
wantOrder: []uint32{2, 1},
|
||||
},
|
||||
{
|
||||
// Windows ranks equal-length prefixes by route metric + interface metric,
|
||||
// so a higher route metric on a low metric interface can still win.
|
||||
name: "combined metric beats route metric alone",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
|
||||
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
|
||||
},
|
||||
wantOrder: []uint32{5, 8},
|
||||
},
|
||||
{
|
||||
name: "lower combined metric wins",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
|
||||
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
|
||||
},
|
||||
wantOrder: []uint32{8, 5},
|
||||
},
|
||||
{
|
||||
name: "equal combined metric falls back to route metric",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
|
||||
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
|
||||
},
|
||||
wantOrder: []uint32{2, 1},
|
||||
},
|
||||
{
|
||||
// The metrics are uint32 on the Windows side, so the sum must not wrap.
|
||||
name: "combined metric beyond the uint32 range",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
|
||||
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
|
||||
},
|
||||
wantOrder: []uint32{2, 1},
|
||||
},
|
||||
{
|
||||
name: "unknown interface metric ranks on route metric only",
|
||||
candidates: []candidateRoute{
|
||||
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
|
||||
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
|
||||
},
|
||||
wantOrder: []uint32{2, 1},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
sortRouteCandidates(tt.candidates)
|
||||
|
||||
got := make([]uint32, 0, len(tt.candidates))
|
||||
for _, c := range tt.candidates {
|
||||
got = append(got, c.interfaceIndex)
|
||||
}
|
||||
assert.Equal(t, tt.wantOrder, got)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
|
||||
return int(ipInterfaceRow.Metric)
|
||||
}
|
||||
|
||||
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
|
||||
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
|
||||
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
|
||||
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
|
||||
func sortRouteCandidates(candidates []candidateRoute) {
|
||||
sort.Slice(candidates, func(i, j int) bool {
|
||||
if candidates[i].prefixLength != candidates[j].prefixLength {
|
||||
return candidates[i].prefixLength > candidates[j].prefixLength
|
||||
}
|
||||
if candidates[i].routeMetric != candidates[j].routeMetric {
|
||||
return candidates[i].routeMetric < candidates[j].routeMetric
|
||||
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
|
||||
if mi != mj {
|
||||
return mi < mj
|
||||
}
|
||||
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
|
||||
return candidates[i].routeMetric < candidates[j].routeMetric
|
||||
})
|
||||
}
|
||||
|
||||
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
|
||||
// the sum of the route metric and the metric of the interface the route is on, see
|
||||
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
|
||||
// An unknown interface metric contributes nothing.
|
||||
func combinedMetric(candidate candidateRoute) uint64 {
|
||||
if candidate.interfaceMetric < 0 {
|
||||
return uint64(candidate.routeMetric)
|
||||
}
|
||||
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
|
||||
}
|
||||
|
||||
// GetBestInterface finds the best interface for reaching a destination,
|
||||
// excluding the VPN interface to avoid routing loops.
|
||||
//
|
||||
// Route selection priority:
|
||||
// 1. Longest prefix match (most specific route)
|
||||
// 2. Lowest route metric
|
||||
// 3. Lowest interface metric
|
||||
// 2. Lowest combined metric (route metric + interface metric)
|
||||
// 3. Lowest route metric.
|
||||
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
|
||||
var skipInterfaceIndex int
|
||||
if vpnIntf != "" {
|
||||
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
|
||||
return nil, fmt.Errorf("no route to %s", dest)
|
||||
}
|
||||
|
||||
// Sort routes: prefix length -> route metric -> interface metric
|
||||
sortRouteCandidates(candidates)
|
||||
|
||||
for _, candidate := range candidates {
|
||||
|
||||
@@ -5,6 +5,7 @@ package systemops
|
||||
import (
|
||||
"errors"
|
||||
"net"
|
||||
"net/netip"
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
|
||||
}
|
||||
if err := netlink.RouteAdd(route); err != nil {
|
||||
if errors.Is(err, syscall.EEXIST) {
|
||||
requireUsableIPv6Nexthop(t)
|
||||
return
|
||||
}
|
||||
t.Skipf("install IPv6 fallback default route: %v", err)
|
||||
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
|
||||
t.Logf("delete IPv6 fallback default route: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
requireUsableIPv6Nexthop(t)
|
||||
}
|
||||
|
||||
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
|
||||
// nexthop can actually carry a route. Installing the default route succeeding
|
||||
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
|
||||
func requireUsableIPv6Nexthop(t *testing.T) {
|
||||
t.Helper()
|
||||
|
||||
nexthop, err := GetNextHop(netip.IPv6Unspecified())
|
||||
if err != nil {
|
||||
t.Skipf("resolve IPv6 default nexthop: %v", err)
|
||||
}
|
||||
|
||||
probe := &netlink.Route{
|
||||
Scope: netlink.SCOPE_UNIVERSE,
|
||||
Table: syscall.RT_TABLE_MAIN,
|
||||
Family: netlink.FAMILY_V6,
|
||||
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
|
||||
}
|
||||
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
|
||||
|
||||
switch err := netlink.RouteAdd(probe); {
|
||||
case err == nil:
|
||||
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
|
||||
t.Logf("delete IPv6 probe route: %v", err)
|
||||
}
|
||||
case errors.Is(err, syscall.EEXIST):
|
||||
default:
|
||||
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -232,3 +232,4 @@ func toNetIDs(routes []string) []route.NetID {
|
||||
}
|
||||
return netIDs
|
||||
}
|
||||
|
||||
|
||||
@@ -200,7 +200,7 @@ func startManagement(t *testing.T, signalAddr string, counter *int) (*grpc.Serve
|
||||
|
||||
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
|
||||
peersUpdateManager := update_channel.NewPeersUpdateManager(metrics)
|
||||
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
|
||||
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
|
||||
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
|
||||
@@ -6,9 +6,11 @@ import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
"google.golang.org/grpc/codes"
|
||||
gstatus "google.golang.org/grpc/status"
|
||||
|
||||
"github.com/netbirdio/netbird/client/internal/profilemanager"
|
||||
"github.com/netbirdio/netbird/client/proto"
|
||||
)
|
||||
|
||||
@@ -60,9 +62,19 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
|
||||
|
||||
// a request from the UI implies a graphical session, which the daemon cannot detect itself
|
||||
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
|
||||
if p.Hint != "" {
|
||||
h := p.Hint
|
||||
req.Hint = &h
|
||||
hint := p.Hint
|
||||
if hint == "" {
|
||||
pm := profilemanager.NewProfileManager()
|
||||
if active, perr := pm.GetActiveProfile(); perr != nil {
|
||||
log.Debugf("failed to get active profile for login hint: %v", perr)
|
||||
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
|
||||
log.Debugf("failed to get profile state for login hint: %v", serr)
|
||||
} else {
|
||||
hint = state.Email
|
||||
}
|
||||
}
|
||||
if hint != "" {
|
||||
req.Hint = &hint
|
||||
}
|
||||
|
||||
resp, err := cli.RequestExtendAuthSession(ctx, req)
|
||||
|
||||
@@ -15,7 +15,8 @@
|
||||
"lint": "eslint \"src/**/*.{ts,tsx}\"",
|
||||
"lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix",
|
||||
"check": "pnpm lint && pnpm typecheck && pnpm format:check",
|
||||
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck"
|
||||
"check:fix": "pnpm lint:fix && pnpm format && pnpm typecheck",
|
||||
"i18n:check": "node ../i18n/check-translations.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@radix-ui/react-dialog": "^1.1.15",
|
||||
|
||||
104
client/ui/i18n/check-translations.mjs
Normal file
104
client/ui/i18n/check-translations.mjs
Normal file
@@ -0,0 +1,104 @@
|
||||
#!/usr/bin/env node
|
||||
// Validates that every shipped translation bundle carries exactly the same set
|
||||
// of keys as the English source of truth. English (en) defines the keys; every
|
||||
// other locale declared in _index.json must match it 1:1:
|
||||
//
|
||||
// - no missing keys — a missing key silently falls back to English at runtime
|
||||
// (see i18n bundle fallback), so the gap never surfaces to users or CI
|
||||
// without this check;
|
||||
// - no orphaned keys — keys left behind after an English key is renamed or
|
||||
// removed are dead weight and a sign the locale is drifting.
|
||||
//
|
||||
// Pure Node, no dependencies, so it runs without installing the frontend
|
||||
// toolchain.
|
||||
//
|
||||
// Local: node client/ui/i18n/check-translations.mjs (or: pnpm i18n:check)
|
||||
// CI: .github/workflows/ui-translations.yml
|
||||
|
||||
import { readdirSync, readFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const SOURCE = "en";
|
||||
const localesDir = join(dirname(fileURLToPath(import.meta.url)), "locales");
|
||||
const isCI = Boolean(process.env.GITHUB_ACTIONS);
|
||||
|
||||
function readJSON(path) {
|
||||
return JSON.parse(readFileSync(path, "utf8"));
|
||||
}
|
||||
|
||||
function keysOf(langCode) {
|
||||
return Object.keys(readJSON(join(localesDir, langCode, "common.json")));
|
||||
}
|
||||
|
||||
// Emit a GitHub Actions annotation so failures render inline on the PR diff.
|
||||
function annotate(file, message) {
|
||||
if (isCI) console.log(`::error file=${file}::${message}`);
|
||||
}
|
||||
|
||||
const index = readJSON(join(localesDir, "_index.json"));
|
||||
const declared = index.languages.map((l) => l.code);
|
||||
|
||||
if (!declared.includes(SOURCE)) {
|
||||
console.error(`FATAL: source language "${SOURCE}" is not declared in _index.json`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const sourceKeys = keysOf(SOURCE);
|
||||
const sourceSet = new Set(sourceKeys);
|
||||
console.log(`Source of truth: ${SOURCE}/common.json — ${sourceKeys.length} keys\n`);
|
||||
|
||||
let failed = false;
|
||||
|
||||
for (const code of declared) {
|
||||
if (code === SOURCE) continue;
|
||||
const file = `client/ui/i18n/locales/${code}/common.json`;
|
||||
|
||||
let keys;
|
||||
try {
|
||||
keys = keysOf(code);
|
||||
} catch (e) {
|
||||
failed = true;
|
||||
const msg = `bundle is declared in _index.json but common.json is missing or invalid (${e.message})`;
|
||||
console.error(`✗ ${code}: ${msg}`);
|
||||
annotate("client/ui/i18n/locales/_index.json", `${code}: ${msg}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const set = new Set(keys);
|
||||
const missing = sourceKeys.filter((k) => !set.has(k));
|
||||
const extra = keys.filter((k) => !sourceSet.has(k));
|
||||
|
||||
if (missing.length === 0 && extra.length === 0) {
|
||||
console.log(`✓ ${code}: ${keys.length} keys`);
|
||||
continue;
|
||||
}
|
||||
|
||||
failed = true;
|
||||
console.error(`✗ ${code}: ${keys.length} keys (expected ${sourceKeys.length})`);
|
||||
if (missing.length) {
|
||||
console.error(` missing ${missing.length}: ${missing.join(", ")}`);
|
||||
annotate(file, `Missing ${missing.length} key(s) present in ${SOURCE}: ${missing.join(", ")}`);
|
||||
}
|
||||
if (extra.length) {
|
||||
console.error(` extra ${extra.length}: ${extra.join(", ")}`);
|
||||
annotate(file, `Has ${extra.length} key(s) not present in ${SOURCE}: ${extra.join(", ")}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Locale directories present on disk but not declared in _index.json are never
|
||||
// loaded by the app — surface them so dead translation files don't rot silently.
|
||||
const onDisk = readdirSync(localesDir, { withFileTypes: true })
|
||||
.filter((e) => e.isDirectory())
|
||||
.map((e) => e.name);
|
||||
const undeclared = onDisk.filter((d) => !declared.includes(d));
|
||||
if (undeclared.length) {
|
||||
console.warn(`\n⚠ locale directories not declared in _index.json (not shipped): ${undeclared.join(", ")}`);
|
||||
}
|
||||
|
||||
console.log();
|
||||
if (failed) {
|
||||
console.error("Translation check FAILED — every locale must match the English key set.");
|
||||
process.exit(1);
|
||||
}
|
||||
console.log("Translation check passed — all locales match the English key set.");
|
||||
@@ -1312,6 +1312,9 @@
|
||||
"daemon.outdated.description": {
|
||||
"message": "このアプリを使用するには NetBird サービスを更新してください。"
|
||||
},
|
||||
"daemon.outdated.download": {
|
||||
"message": "最新版をダウンロード"
|
||||
},
|
||||
"error.jwt_clock_skew": {
|
||||
"message": "サインインに失敗しました: このデバイスの時計がサーバーと同期していません。システムの時計を同期してからもう一度お試しください。"
|
||||
},
|
||||
|
||||
@@ -123,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
|
||||
if p.PreSharedKey != "" {
|
||||
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
|
||||
}
|
||||
if p.Hint != "" {
|
||||
req.Hint = ptrStr(p.Hint)
|
||||
hint := p.Hint
|
||||
if hint == "" && profileID != "" {
|
||||
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
|
||||
hint = state.Email
|
||||
} else {
|
||||
log.Debugf("failed to get profile state for login hint: %v", serr)
|
||||
}
|
||||
}
|
||||
if hint != "" {
|
||||
req.Hint = ptrStr(hint)
|
||||
}
|
||||
|
||||
resp, err := cli.Login(ctx, req)
|
||||
@@ -228,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
|
||||
return s.classifyDaemonError(err)
|
||||
}
|
||||
|
||||
// The daemon runs as root and can't reach the user-owned per-profile state
|
||||
// file holding the account email (see Profiles.List), so clear the stale
|
||||
// email here; the next SSO login recreates it.
|
||||
if p.ProfileName != "" {
|
||||
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
|
||||
// Non-fatal: the logout itself succeeded.
|
||||
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -261,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
|
||||
|
||||
// Persist the account email the same way the CLI does after its own
|
||||
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
|
||||
// root and the per-profile state file is user-owned (see Logout below).
|
||||
// root and the per-profile state file is user-owned (see Profiles.List).
|
||||
// Without this the profile has no email, so Profiles.List shows no account
|
||||
// and later logins and session extends go out without a login_hint —
|
||||
// leaving the IdP to guess which account was meant.
|
||||
|
||||
@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
|
||||
}
|
||||
|
||||
// The daemon deletes what it owns but runs as root, so it leaves the
|
||||
// user-owned state file holding the account email behind (same split as
|
||||
// Connection.Logout). Legacy profiles are keyed by name rather than by a
|
||||
// user-owned state file holding the account email behind. Logout keeps the
|
||||
// email on purpose so later logins can pass it as the login_hint; profile
|
||||
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
|
||||
// generated ID, so a recreated profile of the same name would inherit the
|
||||
// deleted one's email and offer it as the login_hint.
|
||||
//
|
||||
|
||||
@@ -115,7 +115,7 @@ sequenceDiagram
|
||||
Resp->>Resp: parse usage tokens, completion
|
||||
Note over Resp: capture_completion gates raw<br/>completion capture
|
||||
Resp->>Cost: tokens
|
||||
Cost->>Cost: lookup pricing.yaml + compute cost
|
||||
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
|
||||
Cost->>Rec: tokens + cost
|
||||
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
|
||||
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)
|
||||
|
||||
@@ -15,6 +15,10 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
|
||||
| ---- | ---- |
|
||||
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
|
||||
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
|
||||
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
|
||||
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
|
||||
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
|
||||
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
|
||||
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
|
||||
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
|
||||
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
|
||||
@@ -48,6 +52,8 @@ flowchart TD
|
||||
I --> J[indexProviderGroups: providerID -> sorted source groups]
|
||||
J --> K[buildRouterConfigJSON drops orphan providers]
|
||||
J --> L[buildIdentityInjectConfigJSON per catalog entry]
|
||||
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
|
||||
K2 --> P
|
||||
H --> M[mergeGuardrails: union allowlist, OR redact]
|
||||
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
|
||||
N --> O[marshalGuardrailConfig]
|
||||
@@ -60,6 +66,84 @@ flowchart TD
|
||||
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
|
||||
```
|
||||
|
||||
### LLM pricing (management is the sole authority)
|
||||
|
||||
**The proxy carries no price list.** Management synthesizes the entire pricing
|
||||
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
|
||||
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
|
||||
table and there is nothing to reload on the proxy side.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
|
||||
B --> C{AgentNetwork.PricingDefaultsFile}
|
||||
C -- absent --> D[compiled-in table serves]
|
||||
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
|
||||
E --> F[mergedTable atomic.Pointer]
|
||||
D --> G[DefaultTable]
|
||||
F --> G
|
||||
G --> H[buildCostMeterConfigJSON — pricing.defaults]
|
||||
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
|
||||
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
|
||||
K --> L[pricing.providers keyed by provider record ID]
|
||||
H --> M[cost_meter ConfigJSON]
|
||||
L --> M
|
||||
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
|
||||
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
|
||||
```
|
||||
|
||||
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
|
||||
|
||||
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
|
||||
id → rates. The **full** default table ships to every account: it is small
|
||||
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
|
||||
models, so they claim every model) priced.
|
||||
- `pricing.providers` — provider **record** id → normalized model id → rates,
|
||||
matched against the `llm.resolved_provider_id` the router stamps. Entries are
|
||||
**fully materialized here**, at synth time: `materializeEntry` starts from the
|
||||
default entry for that model so cache rates the operator didn't state are
|
||||
inherited, overlays operator `input`/`output` verbatim (**including an explicit
|
||||
0**, which prices a self-hosted or internal endpoint as free rather than
|
||||
silently reverting to list price), and overlays cache-rate **pointers only when
|
||||
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
|
||||
discount, bill this bucket at the input rate". The proxy therefore does two map
|
||||
lookups and no merging.
|
||||
|
||||
Same orphan rule as the router: a provider no enabled policy authorises is
|
||||
unreachable, so its prices aren't shipped. Model ids are normalized with the
|
||||
**same** functions the request parser uses (`NormalizeBedrockModel` /
|
||||
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
|
||||
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
|
||||
first-occurrence-wins, matching the routing dedup order.
|
||||
|
||||
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
|
||||
replace default rates without a rebuild. Schema is `surface → model → rates`
|
||||
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
|
||||
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
|
||||
|
||||
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
|
||||
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
|
||||
- An **explicitly configured** path is *required to load*: a typo or malformed
|
||||
file fails startup, because the operator believes those rates are live. The
|
||||
conventional probe is optional — an absent file just serves compiled-in
|
||||
defaults, and the path stays watched in case it appears later.
|
||||
- File entries **replace** the compiled-in entry for the same (surface, model)
|
||||
**whole** — they are not field-merged, so an entry must repeat the cache rates
|
||||
it wants to keep. Everything the file doesn't mention keeps built-in rates.
|
||||
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
|
||||
finite and non-negative — the same constraints the HTTP API enforces on
|
||||
operator per-provider prices.
|
||||
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
|
||||
a parse error keeps the previous table, a deleted file reverts to compiled-in
|
||||
defaults. A mid-edit save can never take pricing down.
|
||||
|
||||
The live table feeds **both** consumers, which is what keeps them consistent: the
|
||||
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
|
||||
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
|
||||
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
|
||||
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
|
||||
golden-tested, so operators start from a file matching the built-in rates exactly.
|
||||
|
||||
### Budget rule resolution (min-wins, group+user bound)
|
||||
|
||||
```mermaid
|
||||
@@ -124,7 +208,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
|
||||
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
|
||||
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | – |
|
||||
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | – |
|
||||
| on_response | 6 | `cost_meter` | `{}` | – |
|
||||
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | – |
|
||||
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | – |
|
||||
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
|
||||
|
||||
@@ -139,6 +223,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
|
||||
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
|
||||
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
|
||||
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
|
||||
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
|
||||
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
|
||||
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
|
||||
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
|
||||
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
|
||||
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
|
||||
|
||||
## Things to scrutinize
|
||||
|
||||
@@ -176,10 +266,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
|
||||
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
|
||||
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
|
||||
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
|
||||
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
|
||||
|
||||
### Performance
|
||||
|
||||
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
|
||||
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
|
||||
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
|
||||
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
|
||||
|
||||
@@ -188,6 +280,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
|
||||
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
|
||||
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
|
||||
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
|
||||
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
|
||||
|
||||
## Test coverage
|
||||
|
||||
@@ -198,6 +291,9 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
|
||||
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
|
||||
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
|
||||
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
|
||||
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
|
||||
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
|
||||
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
|
||||
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
|
||||
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
|
||||
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |
|
||||
|
||||
@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
|
||||
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
|
||||
|
||||
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
|
||||
adapters + pricing catalog this chain delegates to.
|
||||
adapters + pricing table and cost formula this chain delegates to.
|
||||
|
||||
---
|
||||
|
||||
@@ -34,7 +34,7 @@ rewrites.
|
||||
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
|
||||
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
|
||||
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
|
||||
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
|
||||
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
|
||||
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
|
||||
|
||||
[all_test.go:26–40](../../../proxy/internal/middleware/builtin/all_test.go)
|
||||
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
|
||||
|
||||
| File | LOC | Notes |
|
||||
|---|---:|---|
|
||||
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
|
||||
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
|
||||
| `all_test.go` | 41 | Locks the 8-ID registry surface |
|
||||
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
|
||||
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
|
||||
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
|
||||
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
|
||||
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
|
||||
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
|
||||
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
|
||||
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
|
||||
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
|
||||
|
||||
## Per-middleware
|
||||
@@ -168,12 +168,46 @@ token schema.
|
||||
|
||||
### cost_meter
|
||||
|
||||
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
|
||||
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
|
||||
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
|
||||
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
|
||||
context via `startReloader`. **Key invariant:** provider-shape switch lives
|
||||
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
|
||||
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
|
||||
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
|
||||
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
|
||||
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
|
||||
`unknown_model`).
|
||||
|
||||
**Management owns pricing.** The proxy carries no embedded price list: the whole
|
||||
table arrives in this middleware's `ConfigJSON` as
|
||||
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
|
||||
plus the operator's stored per-provider prices
|
||||
([factory.go:13–34](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
|
||||
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
|
||||
construction, so a non-finite or negative rate fails the chain build. A price
|
||||
change is an ordinary mapping push — the chain rebuild yields a fresh instance
|
||||
over a fresh immutable table, so there is no data dir, no pricing file, no
|
||||
reload goroutine, and nothing to invalidate.
|
||||
|
||||
**Two-tier lookup**
|
||||
([middleware.go:165–183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
|
||||
|
||||
1. **Per-provider-record** — the operator's stored price for the route that
|
||||
actually served the request, keyed by the `llm.resolved_provider_id` that
|
||||
`llm_router` stamped on the allow path, then by normalized model id. Entries
|
||||
arrive fully materialized (management folds default cache rates in at synth
|
||||
time), so there is no merging here. Absent metadata — no router in the chain
|
||||
— skips this tier.
|
||||
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
|
||||
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
|
||||
providers, which enumerate no models and therefore get no per-record entry.
|
||||
|
||||
**Backward compatibility:** a config with no `pricing` block means management
|
||||
predates config-delivered pricing. The factory logs one warning at build time
|
||||
and the instance records `cost.skipped=unknown_model` ($0) for every request
|
||||
rather than falling back to a stale built-in price list
|
||||
([factory.go:55–60](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
|
||||
|
||||
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
|
||||
(sibling doc) and is selected by the **surface**, not by which tier the entry
|
||||
came from — `cost_meter` stays provider-agnostic, and a per-record override on
|
||||
an Anthropic route still bills its cache buckets additively.
|
||||
|
||||
### llm_limit_record
|
||||
|
||||
@@ -246,12 +280,14 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
|
||||
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
|
||||
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
|
||||
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
|
||||
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
|
||||
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
|
||||
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
|
||||
|
||||
All factories accept empty / null / `{}` / whitespace as zero-value config;
|
||||
only structurally invalid JSON is rejected so misconfig surfaces at chain
|
||||
build time.
|
||||
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
|
||||
block carrying a negative or non-finite rate fails the build too, rather than
|
||||
mispricing live traffic.
|
||||
|
||||
## Invariants
|
||||
|
||||
@@ -320,10 +356,11 @@ non-object `metadata` field
|
||||
— header path still attributes, but body-level tag-budget enforcement
|
||||
doesn't run for that request.
|
||||
|
||||
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
|
||||
`atomic.Pointer[Table]`; readers always see a consistent table. Every
|
||||
middleware is a stateless value receiver. Integration test uses real bufconn
|
||||
gRPC — race detector is the meaningful bar.
|
||||
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
|
||||
middleware config and never mutated, so the lookup path needs no lock or atomic
|
||||
swap — a price change replaces the whole instance. Every middleware is
|
||||
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
|
||||
race detector is the meaningful bar.
|
||||
|
||||
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
|
||||
is O(1); SSE accumulation is single-pass. No map allocation per call.
|
||||
@@ -349,13 +386,13 @@ counter accuracy.
|
||||
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
|
||||
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
|
||||
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
|
||||
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
|
||||
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
|
||||
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
|
||||
|
||||
## Cross-references
|
||||
|
||||
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
|
||||
+ SSE framer + pricing loader.
|
||||
+ SSE framer + pricing table and cost formula.
|
||||
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
|
||||
token minting, `/bedrock` prefix:
|
||||
[50-path-routed-providers.md](./50-path-routed-providers.md).
|
||||
|
||||
@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
|
||||
small bug would silently mis-bill operators.
|
||||
|
||||
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
|
||||
— the 8 middlewares that consume this package's parsers + pricing loader.
|
||||
— the 8 middlewares that consume this package's parsers + pricing table.
|
||||
|
||||
---
|
||||
|
||||
@@ -24,8 +24,9 @@ proxy-framework dependencies:
|
||||
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
|
||||
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
|
||||
- `errors.go` — sentinels callers branch on with `errors.Is`.
|
||||
- `pricing/` — embedded-default + hot-reload override table with
|
||||
symlink-safe Unix loader (build-tagged stub elsewhere).
|
||||
- `pricing/` — immutable pricing table + the per-surface cost formula. The
|
||||
rates themselves come from management inside `cost_meter`'s middleware
|
||||
config; this package holds no price list and reads no files.
|
||||
- `fixtures/` — captured request/response/stream bodies the tests replay.
|
||||
|
||||
The package carries zero proxy-framework dependencies so the same parsers can
|
||||
@@ -47,12 +48,9 @@ be reused later by a WASM adapter
|
||||
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
|
||||
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
|
||||
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
|
||||
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
|
||||
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
|
||||
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
|
||||
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
|
||||
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
|
||||
| `fixtures/*` | 21–59 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
|
||||
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
|
||||
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
|
||||
| `fixtures/*` | 21–59 | OAI chat/responses/stream + Anthro messages/stream |
|
||||
|
||||
## Request body → parser dispatch
|
||||
|
||||
@@ -188,9 +186,11 @@ response leg, covering both Bedrock body shapes:
|
||||
`totalTokens`). `firstNonZero` folds the two naming conventions into one
|
||||
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
|
||||
|
||||
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
|
||||
keyed by the **normalised** model id (region prefix + version suffix stripped by
|
||||
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
|
||||
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
|
||||
management ships, keyed by the **normalised** model id (region prefix + version
|
||||
suffix stripped by the request parser; management normalises its keys the same
|
||||
way at synth time so the two compare equal). `ParseResponse` returns
|
||||
`ErrStreamingUnsupported` for an
|
||||
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
|
||||
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
|
||||
|
||||
@@ -205,11 +205,34 @@ response body. Streaming accumulators live in the middleware package
|
||||
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
|
||||
but use `llm.NewScanner` so the framing contract stays here.
|
||||
|
||||
### Pricing catalog
|
||||
### Pricing table
|
||||
|
||||
`Table.Cost`
|
||||
([pricing.go:129–174](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
is the cost formula — most security-relevant math in this module:
|
||||
**Management is the sole pricing authority.** The proxy carries no embedded
|
||||
price list and reads no pricing file: the whole table arrives inside
|
||||
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
|
||||
is just another push — the chain rebuild constructs a fresh `Table`, so there
|
||||
is nothing to reload
|
||||
([pricing.go:1–7](../../../proxy/internal/llm/pricing/pricing.go)). The
|
||||
management side of the contract (catalog defaults, the operator's stored
|
||||
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
|
||||
management-side module guide; `cost_meter`'s wire shape is in
|
||||
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
|
||||
|
||||
`EntryJSON`
|
||||
([pricing.go:36–45](../../../proxy/internal/llm/pricing/pricing.go)) is the
|
||||
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
|
||||
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
|
||||
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
|
||||
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
|
||||
direct struct conversion rather than field-by-field copying (a new rate can't
|
||||
be silently dropped in transit).
|
||||
|
||||
`EntryCosts`
|
||||
([pricing.go:183–234](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
is the cost formula — most security-relevant math in this module. The
|
||||
**surface** (the `llm.provider` value the request parser stamped) selects the
|
||||
formula, never the tier the entry came from: a per-provider-record override on
|
||||
an Anthropic route still bills its cache buckets additively.
|
||||
|
||||
| Provider | Formula |
|
||||
|---|---|
|
||||
@@ -218,7 +241,7 @@ is the cost formula — most security-relevant math in this module:
|
||||
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
|
||||
|
||||
`bedrock` shares the Anthropic additive-cache formula
|
||||
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
|
||||
([pricing.go:214–229](../../../proxy/internal/llm/pricing/pricing.go)):
|
||||
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
|
||||
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
|
||||
to `input + output`.
|
||||
@@ -226,15 +249,12 @@ to `input + output`.
|
||||
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
|
||||
to discounts by setting the field.
|
||||
|
||||
`Loader`
|
||||
([pricing.go:212–268](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
|
||||
defaults. Atomic pointer swap means readers never observe a partial update.
|
||||
The mtime-poll reloader (30s default cadence) keeps the previous table on
|
||||
parse failure so cost annotation never goes blank during a botched edit.
|
||||
|
||||
`defaults_pricing.yaml` is the source of truth for built-in pricing.
|
||||
Operator overrides only carry the entries they want to change.
|
||||
`Costs`
|
||||
([pricing.go:143–163](../../../proxy/internal/llm/pricing/pricing.go)) is the
|
||||
per-request split. The four per-bucket fields are the base; `TotalUSD` and
|
||||
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
|
||||
the breakdown. `InputUSD` is always the non-cached input bucket on both
|
||||
provider shapes, so input and cached-input never double-count.
|
||||
|
||||
## Public contracts
|
||||
|
||||
@@ -264,29 +284,38 @@ Order matters: `DetectFromURL` ties resolve by registration order.
|
||||
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
|
||||
them as wire-stable — new providers must take fresh numbers.
|
||||
|
||||
**`Pricing` lookup**
|
||||
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
|
||||
**`Pricing` construction + lookup**
|
||||
([pricing.go:60–130](../../../proxy/internal/llm/pricing/pricing.go)):
|
||||
|
||||
```go
|
||||
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
|
||||
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
|
||||
|
||||
func (t *Table) Lookup(provider, model string) (Entry, bool)
|
||||
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
|
||||
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
|
||||
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
|
||||
```
|
||||
|
||||
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
|
||||
([pricing.go:130–132](../../../proxy/internal/llm/pricing/pricing.go)).
|
||||
`ok=false` means provider or model is absent from the loaded table; the caller
|
||||
emits `cost.skipped=unknown_model`.
|
||||
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
|
||||
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
|
||||
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
|
||||
reject any non-finite or negative rate, so a corrupt config fails the chain
|
||||
build rather than mispricing silently. Nil input yields an empty,
|
||||
never-matching table.
|
||||
|
||||
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
|
||||
([pricing.go:96–99](../../../proxy/internal/llm/pricing/pricing.go)).
|
||||
`ok=false` means the surface or model is absent from the table management sent;
|
||||
the caller emits `cost.skipped=unknown_model`.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
|
||||
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
|
||||
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
|
||||
fallback that returns `"not supported on this platform"`
|
||||
([pricing_other.go:14–16](../../../proxy/internal/llm/pricing/pricing_other.go)).
|
||||
The proxy is Linux-only in production today; a Windows port needs an
|
||||
equivalent path-as-handle implementation. Reviewers building on Windows
|
||||
should expect this surface to return an error at startup if an override
|
||||
file is configured.
|
||||
1. **The pricing package is pure and platform-independent.** No file I/O, no
|
||||
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
|
||||
there is nothing platform-specific left to port. Anything reintroducing a
|
||||
read-from-disk path here re-splits pricing authority between management and
|
||||
the proxy, which is exactly what this design removed.
|
||||
|
||||
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
|
||||
in `\n\n` still yields its accumulated event before `io.EOF`
|
||||
@@ -298,38 +327,45 @@ emits `cost.skipped=unknown_model`.
|
||||
usage rather than aborting
|
||||
([streaming.go:68–73, 144–150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
|
||||
|
||||
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
|
||||
binary via `//go:embed`
|
||||
([pricing.go:29–30](../../../proxy/internal/llm/pricing/pricing.go)).
|
||||
`DefaultTable()` parses once and panics on parse failure
|
||||
([pricing.go:42–49](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
— by design: a broken embedded YAML must not ship to production.
|
||||
3. **Management is the only source of rates.** `Table` has no constructor that
|
||||
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
|
||||
management sent. A missing or empty `pricing` block therefore means *no
|
||||
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
|
||||
never a stale built-in fallback that would silently bill list price.
|
||||
|
||||
4. **Loader path validation.** `resolveMiddlewareDataPath`
|
||||
([pricing.go:370–394](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
rejects absolute paths, traversal segments, and basenames that fail
|
||||
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
|
||||
inside `baseDir` even after `filepath.Clean`. Tests:
|
||||
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
|
||||
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
|
||||
4. **Tables are immutable once built.** `Table.entries` is written only in
|
||||
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
|
||||
map is likewise build-time-only
|
||||
([pricing.go:47–52](../../../proxy/internal/llm/pricing/pricing.go)). This
|
||||
is what makes the no-reload design safe: a price change arrives as a mapping
|
||||
push that builds a new middleware instance over a new table, so concurrent
|
||||
readers can't observe a half-updated price list and no atomic swap or lock
|
||||
is needed on the hot path.
|
||||
|
||||
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
|
||||
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
|
||||
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
|
||||
([pricing_unix.go:25–57](../../../proxy/internal/llm/pricing/pricing_unix.go)).
|
||||
A mid-read symlink swap is detected because the fstat is on the original
|
||||
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
|
||||
5. **Rate validation happens at chain-build time, not per request.**
|
||||
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
|
||||
([pricing.go:60–83](../../../proxy/internal/llm/pricing/pricing.go)), naming
|
||||
the offending surface/model/field in the error. Management enforces the same
|
||||
constraints at its API boundary and in its YAML parser, so this is
|
||||
defense-in-depth — but it means a corrupt push fails loudly at build instead
|
||||
of producing negative costs on live traffic. Test:
|
||||
`TestNewTable_ValidatesRates`.
|
||||
|
||||
6. **`yaml.NewDecoder(...).KnownFields(true)`**
|
||||
([pricing.go:397–398](../../../proxy/internal/llm/pricing/pricing.go))
|
||||
rejects YAML files that carry fields not in the schema. A typo in an
|
||||
operator override file fails loud instead of silently zeroing rates.
|
||||
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
|
||||
`pricing.Entry` together.** `NewEntries` converts by direct struct
|
||||
conversion `Entry(e)`
|
||||
([pricing.go:76–78](../../../proxy/internal/llm/pricing/pricing.go)), which
|
||||
only compiles while the two structs stay field-identical — so the proxy half
|
||||
is compiler-enforced. The management half is not: a rate added there but not
|
||||
here unmarshals into nothing and prices that bucket at `InputPer1K`.
|
||||
|
||||
## Things to scrutinise
|
||||
|
||||
**Correctness.** Verify OpenAI cached-prompt clamp at
|
||||
[pricing.go:147–149](../../../proxy/internal/llm/pricing/pricing.go)
|
||||
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
|
||||
**Correctness.** Verify the OpenAI cached-prompt clamp at
|
||||
[pricing.go:203–206](../../../proxy/internal/llm/pricing/pricing.go)
|
||||
short-circuits before subtraction. Negative token counts are clamped to zero up
|
||||
front ([pricing.go:186–197](../../../proxy/internal/llm/pricing/pricing.go)) so
|
||||
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
|
||||
buckets (in + out + cache_read + cache_creation) — downstream dashboards
|
||||
need to know this differs from `input + output`.
|
||||
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
|
||||
@@ -338,22 +374,27 @@ noting).
|
||||
|
||||
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
|
||||
errors from `Scanner.Next` and both accumulators stop with partial usage.
|
||||
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
|
||||
new schema additions are mirrored in both `pricingFile` and `Entry`;
|
||||
`KnownFields(true)` will reject silently-typo'd operator overrides
|
||||
otherwise.
|
||||
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
|
||||
oversize surface is gone entirely — the config channel (an authenticated
|
||||
mapping push from management) is now the only way rates enter the proxy, and
|
||||
`NewEntries` is the validation boundary on it. A new rate added to management's
|
||||
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
|
||||
path (see invariant 6).
|
||||
|
||||
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
|
||||
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
|
||||
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
|
||||
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
|
||||
concurrent response-parser calls.
|
||||
**Concurrency.** Nothing in this package is shared mutable state: tables are
|
||||
built once and never written again, so `cost_meter`'s hot path is lock-free by
|
||||
construction rather than by atomic swap. Per-call `Scanner` instances mean no
|
||||
shared state across concurrent response-parser calls.
|
||||
|
||||
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
|
||||
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
|
||||
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
|
||||
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
|
||||
`ReadString('\n')` per line. No background goroutines and no per-request
|
||||
allocation of pricing state.
|
||||
|
||||
**Observability.** Reload failures count via `metric.Int64Counter` keyed
|
||||
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
|
||||
**Observability.** A config carrying no `pricing` block logs one warning at
|
||||
chain-build time (`cost_meter` factory) and then records
|
||||
`cost.skipped=unknown_model` per request, so an old-management deployment is
|
||||
visible in both logs and the access log rather than quietly reporting $0.
|
||||
Parser errors return sentinels — middleware uses `errors.Is` to map to the
|
||||
right `cost.skipped` reason.
|
||||
|
||||
@@ -365,7 +406,7 @@ right `cost.skipped` reason.
|
||||
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
|
||||
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
|
||||
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
|
||||
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
|
||||
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
|
||||
|
||||
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
|
||||
`openai_chat_completion.json` (chat.completions with usage),
|
||||
@@ -373,14 +414,15 @@ right `cost.skipped` reason.
|
||||
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
|
||||
`anthropic_messages.json` (Messages API non-streaming),
|
||||
`anthropic_stream.txt` (full 7-event sequence: message_start →
|
||||
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
|
||||
`pricing.yaml` (realistic-pricing starter for operator overrides).
|
||||
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
|
||||
No pricing fixture: the table is config-delivered, so pricing tests construct
|
||||
it in-process from a wire-shape map.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
|
||||
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
|
||||
`llm.NewScanner`, `pricing.NewLoader`.
|
||||
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
|
||||
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
|
||||
Bedrock AWS event-stream accumulator:
|
||||
[50-path-routed-providers.md](./50-path-routed-providers.md).
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# proxy/runtime — translate + serve + log
|
||||
|
||||
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
|
||||
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
|
||||
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
|
||||
|
||||
## Module boundary
|
||||
|
||||
@@ -114,8 +114,7 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
|
||||
|
||||
## Public contracts touched
|
||||
|
||||
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
|
||||
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
|
||||
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
|
||||
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
|
||||
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
|
||||
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).
|
||||
|
||||
@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
|
||||
surface via `vertexPublisherVendor`:
|
||||
|
||||
- `anthropic` → `llm.provider="anthropic"` → metered through the Anthropic
|
||||
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
|
||||
(the parser emits the standard Anthropic provider label, so Vertex Claude
|
||||
reuses first-party Anthropic prices).
|
||||
parser, priced under the **`anthropic`** surface of the pricing table
|
||||
management ships (the parser emits the standard Anthropic provider label, so
|
||||
Vertex Claude reuses first-party Anthropic prices).
|
||||
- `openai` → `llm.provider="openai"` (reserved; not in the catalog lineup
|
||||
today).
|
||||
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
|
||||
@@ -104,8 +104,9 @@ is omitted from the catalog.
|
||||
|
||||
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
|
||||
> premium that the base per-token rates do **not** model — cost annotations for
|
||||
> those regions read low. Operators who need exact regional billing override
|
||||
> the affected entries in `pricing.yaml`.
|
||||
> those regions read low. Operators who need exact regional billing set the
|
||||
> affected models' prices on the provider record, or replace the default entries
|
||||
> via management's `AgentNetwork.PricingDefaultsFile`.
|
||||
|
||||
## AWS Bedrock (`bedrock_api`)
|
||||
|
||||
@@ -211,15 +212,19 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
|
||||
## Catalog ↔ pricing cross-check
|
||||
|
||||
Catalog prices and context windows are cross-checked against LiteLLM's
|
||||
`model_prices_and_context_window.json`. The proxy's embedded
|
||||
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
|
||||
enumerates — guarded by
|
||||
`TestDefaultTable_FirstPartyModelCoverage`
|
||||
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
|
||||
which fails if a catalog model has no embedded price. Bedrock entries are keyed
|
||||
by the **normalised** id the request parser emits (region prefix + version
|
||||
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
|
||||
straight off the `anthropic` block.
|
||||
`model_prices_and_context_window.json`. The **catalog is the source of default
|
||||
prices**: management's `pricing.DefaultTable` folds every catalog provider's
|
||||
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
|
||||
is structural rather than maintained in a parallel file
|
||||
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
|
||||
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
|
||||
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
|
||||
providers contribute the same (surface, model) at different rates. Bedrock
|
||||
entries are keyed by the **normalised** id the request parser emits (region
|
||||
prefix + version suffix stripped) — management applies the same normalisation to
|
||||
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
|
||||
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
|
||||
surface.
|
||||
|
||||
## Things to scrutinise
|
||||
|
||||
@@ -232,16 +237,17 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
|
||||
publishers).
|
||||
|
||||
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
|
||||
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
|
||||
meters at `cost.skipped=unknown_model` rather than failing the request. The
|
||||
pricing key — a model that normalises to something absent from the shipped
|
||||
pricing table meters at `cost.skipped=unknown_model` rather than failing the
|
||||
request. The
|
||||
`/bedrock` prefix strip must run on both the parser side (so the model is
|
||||
extracted) and the router side (so the upstream path is native); a regression in
|
||||
either silently breaks the other.
|
||||
|
||||
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
|
||||
~10% premium not modelled by base pricing — flagged in both the catalog comment
|
||||
and `defaults_pricing.yaml`. Operators needing exact regional billing override
|
||||
the relevant entries.
|
||||
~10% premium not modelled by base pricing — flagged in the catalog comment.
|
||||
Operators needing exact regional billing set per-provider prices on the model
|
||||
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
|
||||
|
||||
## Cross-references
|
||||
|
||||
|
||||
12
funding.json
12
funding.json
@@ -6,7 +6,7 @@
|
||||
"name": "NetBird GmbH",
|
||||
"email": "hello@netbird.io",
|
||||
"phone": "",
|
||||
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open-source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open-source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
|
||||
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
|
||||
"webpageUrl": {
|
||||
"url": "https://github.com/netbirdio"
|
||||
}
|
||||
@@ -15,7 +15,7 @@
|
||||
{
|
||||
"guid": "netbird",
|
||||
"name": "NetBird",
|
||||
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open-source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
|
||||
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
|
||||
"webpageUrl": {
|
||||
"url": "https://github.com/netbirdio/netbird"
|
||||
},
|
||||
@@ -59,7 +59,7 @@
|
||||
"guid": "support-yearly",
|
||||
"status": "active",
|
||||
"name": "Support Open Source Development and Maintenance - Yearly",
|
||||
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
|
||||
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
|
||||
"amount": 100000,
|
||||
"currency": "USD",
|
||||
"frequency": "yearly",
|
||||
@@ -72,7 +72,7 @@
|
||||
"guid": "support-one-time-year",
|
||||
"status": "active",
|
||||
"name": "Support Open Source Development and Maintenance - One Year",
|
||||
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
|
||||
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
|
||||
"amount": 100000,
|
||||
"currency": "USD",
|
||||
"frequency": "one-time",
|
||||
@@ -85,7 +85,7 @@
|
||||
"guid": "support-one-time-monthly",
|
||||
"status": "active",
|
||||
"name": "Support Open Source Development and Maintenance - Monthly",
|
||||
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
|
||||
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
|
||||
"amount": 10000,
|
||||
"currency": "USD",
|
||||
"frequency": "monthly",
|
||||
@@ -98,7 +98,7 @@
|
||||
"guid": "support-monthly",
|
||||
"status": "active",
|
||||
"name": "Support Open Source Development and Maintenance - One Month",
|
||||
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
|
||||
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
|
||||
"amount": 10000,
|
||||
"currency": "USD",
|
||||
"frequency": "monthly",
|
||||
|
||||
11
go.mod
11
go.mod
@@ -57,6 +57,7 @@ require (
|
||||
github.com/fsnotify/fsnotify v1.9.0
|
||||
github.com/gliderlabs/ssh v0.3.8
|
||||
github.com/go-jose/go-jose/v4 v4.1.4
|
||||
github.com/go-ole/go-ole v1.3.0
|
||||
github.com/gobwas/ws v1.4.0
|
||||
github.com/goccy/go-yaml v1.18.0
|
||||
github.com/godbus/dbus/v5 v5.2.2
|
||||
@@ -71,18 +72,17 @@ require (
|
||||
github.com/hashicorp/go-multierror v1.1.1
|
||||
github.com/hashicorp/go-secure-stdlib/base62 v0.1.2
|
||||
github.com/hashicorp/go-version v1.7.0
|
||||
github.com/jackc/pgx/v5 v5.10.0
|
||||
github.com/jackc/pgx/v5 v5.5.5
|
||||
github.com/libdns/route53 v1.5.0
|
||||
github.com/libp2p/go-nat v0.2.0
|
||||
github.com/libp2p/go-netroute v0.4.0
|
||||
github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81
|
||||
github.com/magefile/mage v1.17.2
|
||||
github.com/mdlayher/socket v0.5.1
|
||||
github.com/mdp/qrterminal/v3 v3.2.1
|
||||
github.com/miekg/dns v1.1.72
|
||||
github.com/mitchellh/hashstructure/v2 v2.0.2
|
||||
github.com/moby/moby/api v1.54.1
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42
|
||||
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45
|
||||
github.com/oapi-codegen/runtime v1.1.2
|
||||
github.com/okta/okta-sdk-golang/v2 v2.18.0
|
||||
@@ -200,7 +200,6 @@ require (
|
||||
github.com/go-ldap/ldap/v3 v3.4.13 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-ole/go-ole v1.3.0 // indirect
|
||||
github.com/go-openapi/analysis v0.23.0 // indirect
|
||||
github.com/go-openapi/errors v0.22.2 // indirect
|
||||
github.com/go-openapi/jsonpointer v0.21.1 // indirect
|
||||
@@ -237,8 +236,8 @@ require (
|
||||
github.com/huin/goupnp v1.2.0 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.1 // indirect
|
||||
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
|
||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
|
||||
18
go.sum
18
go.sum
@@ -341,12 +341,12 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
|
||||
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
|
||||
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
||||
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
||||
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
|
||||
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk=
|
||||
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
||||
github.com/jackc/pgx/v5 v5.5.5 h1:amBjrZVmksIdNjxGW/IiIMzxMKZFelXbUoPNb+8sjQw=
|
||||
github.com/jackc/pgx/v5 v5.5.5/go.mod h1:ez9gk+OAat140fv9ErkZDYFWmXLfV+++K0uAOiwgm1A=
|
||||
github.com/jackc/puddle/v2 v2.2.1 h1:RhxXJtFG022u4ibrCSMSiu5aOq1i77R3OHKNJj77OAk=
|
||||
github.com/jackc/puddle/v2 v2.2.1/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=
|
||||
github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=
|
||||
github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
|
||||
@@ -415,8 +415,6 @@ github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81 h1:J56rFEfUTFT9j9
|
||||
github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81/go.mod h1:RD8ML/YdXctQ7qbcizZkw5mZ6l8Ogrl1dodBzVJduwI=
|
||||
github.com/lufia/plan9stats v0.0.0-20240513124658-fba389f38bae h1:dIZY4ULFcto4tAFlj1FYZl8ztUZ13bdq+PLY+NOfbyI=
|
||||
github.com/lufia/plan9stats v0.0.0-20240513124658-fba389f38bae/go.mod h1:ilwx/Dta8jXAgpFYFvSWEMwxmbWXyiUHkd5FwyKhb5k=
|
||||
github.com/magefile/mage v1.17.2 h1:fyXVu1eadI8Ap1HCCNgEhJ5McIWiYhLR8uol64ZZc40=
|
||||
github.com/magefile/mage v1.17.2/go.mod h1:Yj51kqllmsgFpvvSzgrZPK9WtluG3kUhFaBUVLo4feA=
|
||||
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
|
||||
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
|
||||
github.com/matryer/is v1.4.1 h1:55ehd8zaGABKLXQUe2awZ99BD/PTc2ls+KV/dXphgEQ=
|
||||
@@ -484,8 +482,8 @@ github.com/netbirdio/easyjson v0.9.0 h1:6Nw2lghSVuy8RSkAYDhDv1thBVEmfVbKZnV7T7Z6
|
||||
github.com/netbirdio/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
|
||||
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51 h1:Ov4qdafATOgGMB1wbSuh+0aAHcwz9hdvB6VZjh1mVMI=
|
||||
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51/go.mod h1:ZSIbPdBn5hePO8CpF1PekH2SfpTxg1PDhEwtbqZS7R8=
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87 h1:iJeUvSMC0BTpkw7u4JyWcY4/3dl7fEL9DR/TpKf2+1w=
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87/go.mod h1:pmsCPx1S0nuZRxCextGpc9AV4hLgGSuTsc4NMuwGeCo=
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42 h1:F3zS5fT9xzD1OFLfcdAE+3FfyiwjGukF1hvj0jErgs8=
|
||||
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42/go.mod h1:n47r67ZSPgwSmT/Z1o48JjZQW9YJ6m/6Bd/uAXkL3Pg=
|
||||
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502 h1:3tHlFmhTdX9axERMVN63dqyFqnvuD+EMJHzM7mNGON8=
|
||||
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502/go.mod h1:CIMRFEJVL+0DS1a3Nx06NaMn4Dz63Ng6O7dl0qH0zVM=
|
||||
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45 h1:ujgviVYmx243Ksy7NdSwrdGPSRNE3pb8kEDSpH0QuAQ=
|
||||
|
||||
@@ -111,6 +111,59 @@ check_nb_domain() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# Non-interactive configuration
|
||||
# ------------------------------
|
||||
# Every prompt below can be pre-answered with an environment variable, so the
|
||||
# script runs unattended (cloud-init, CI, Terraform, curl | bash). resolve()
|
||||
# is the single place that decides env var vs prompt vs default; the read_*
|
||||
# helpers stay pure prompts.
|
||||
#
|
||||
# Supported env vars:
|
||||
# NETBIRD_DOMAIN domain/FQDN (required)
|
||||
# NETBIRD_LETSENCRYPT_EMAIL ACME email (required for built-in Traefik)
|
||||
# NETBIRD_AGENT_NETWORK true enables the agent-network preset
|
||||
# NETBIRD_REVERSE_PROXY_TYPE 0-5 (default 0 = built-in Traefik)
|
||||
# NETBIRD_ENABLE_PROXY true/false (default false)
|
||||
# NETBIRD_ENABLE_CROWDSEC true/false (default false)
|
||||
# NETBIRD_TRAEFIK_EXTERNAL_NETWORK external-Traefik network (type 1)
|
||||
# NETBIRD_TRAEFIK_ENTRYPOINT external-Traefik entrypoint (type 1, default websecure)
|
||||
# NETBIRD_TRAEFIK_CERTRESOLVER external-Traefik cert resolver (type 1)
|
||||
# NETBIRD_BIND_LOCALHOST_ONLY true/false (default true, types 2-5)
|
||||
# NETBIRD_EXTERNAL_PROXY_NETWORK docker network to join (types 2-4)
|
||||
# NETBIRD_NON_INTERACTIVE true forces unattended mode even with a TTY
|
||||
|
||||
# tty_available succeeds only when we may prompt: never when the operator has
|
||||
# set NETBIRD_NON_INTERACTIVE=true, otherwise only when /dev/tty can actually
|
||||
# be opened. A PTY can be attached in automation (CI runners, some
|
||||
# provisioners), so the env override is the authoritative signal and the
|
||||
# /dev/tty probe is the fallback. /dev/tty is a world-rw device node even with
|
||||
# no terminal, so a permission test ([ -r ]) is not enough - we must open it.
|
||||
tty_available() {
|
||||
[[ "${NETBIRD_NON_INTERACTIVE:-}" == "true" ]] && return 1
|
||||
{ true < /dev/tty; } 2>/dev/null
|
||||
}
|
||||
|
||||
# resolve ENV_VAR_NAME DEFAULT PROMPT_FN [prompt args...]
|
||||
# env var set and non-empty -> its value
|
||||
# interactive -> PROMPT_FN "$@" (prompt behavior unchanged)
|
||||
# otherwise -> DEFAULT, or abort when DEFAULT is "required"
|
||||
resolve() {
|
||||
local env_name="$1" default="$2" prompt_fn="$3"
|
||||
shift 3
|
||||
local env_value="${!env_name:-}"
|
||||
if [[ -n "$env_value" ]]; then
|
||||
echo "$env_value"
|
||||
elif tty_available; then
|
||||
"$prompt_fn" "$@"
|
||||
elif [[ "$default" == "required" ]]; then
|
||||
echo "$env_name is required for a non-interactive install." > /dev/stderr
|
||||
exit 1
|
||||
else
|
||||
echo "$default"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
read_nb_domain() {
|
||||
READ_NETBIRD_DOMAIN=""
|
||||
echo -n "Enter the domain you want to use for NetBird (e.g. netbird.my-domain.com): " > /dev/stderr
|
||||
@@ -383,7 +436,14 @@ initialize_default_values() {
|
||||
}
|
||||
|
||||
configure_domain() {
|
||||
# Domain is validated (not a free-form value), so it keeps its own guard
|
||||
# rather than going through resolve(): a valid NETBIRD_DOMAIN is used as-is,
|
||||
# otherwise we prompt, or abort when there is no terminal to prompt on.
|
||||
if ! check_nb_domain "$NETBIRD_DOMAIN"; then
|
||||
if ! tty_available; then
|
||||
echo "NETBIRD_DOMAIN is required for a non-interactive install." > /dev/stderr
|
||||
exit 1
|
||||
fi
|
||||
NETBIRD_DOMAIN=$(read_nb_domain)
|
||||
fi
|
||||
|
||||
@@ -411,11 +471,7 @@ apply_agent_network_preset() {
|
||||
ENABLE_PROXY="true"
|
||||
ENABLE_CROWDSEC="false"
|
||||
|
||||
if [[ -n "${NETBIRD_LETSENCRYPT_EMAIL}" ]]; then
|
||||
TRAEFIK_ACME_EMAIL="${NETBIRD_LETSENCRYPT_EMAIL}"
|
||||
else
|
||||
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
|
||||
fi
|
||||
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
|
||||
|
||||
echo "" > /dev/stderr
|
||||
echo "Agent-network preset enabled (NETBIRD_AGENT_NETWORK=true):" > /dev/stderr
|
||||
@@ -437,35 +493,35 @@ configure_reverse_proxy() {
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Prompt for reverse proxy type
|
||||
REVERSE_PROXY_TYPE=$(read_reverse_proxy_type)
|
||||
# Reverse proxy type (env NETBIRD_REVERSE_PROXY_TYPE, else prompt, else 0)
|
||||
REVERSE_PROXY_TYPE=$(resolve NETBIRD_REVERSE_PROXY_TYPE 0 read_reverse_proxy_type)
|
||||
|
||||
# Handle built-in Traefik prompts (option 0)
|
||||
if [[ "$REVERSE_PROXY_TYPE" == "0" ]]; then
|
||||
TRAEFIK_ACME_EMAIL=$(read_traefik_acme_email)
|
||||
ENABLE_PROXY=$(read_enable_proxy)
|
||||
TRAEFIK_ACME_EMAIL=$(resolve NETBIRD_LETSENCRYPT_EMAIL required read_traefik_acme_email)
|
||||
ENABLE_PROXY=$(resolve NETBIRD_ENABLE_PROXY false read_enable_proxy)
|
||||
if [[ "$ENABLE_PROXY" == "true" ]]; then
|
||||
ENABLE_CROWDSEC=$(read_enable_crowdsec)
|
||||
ENABLE_CROWDSEC=$(resolve NETBIRD_ENABLE_CROWDSEC false read_enable_crowdsec)
|
||||
fi
|
||||
fi
|
||||
|
||||
# Handle external Traefik-specific prompts (option 1)
|
||||
if [[ "$REVERSE_PROXY_TYPE" == "1" ]]; then
|
||||
TRAEFIK_EXTERNAL_NETWORK=$(read_traefik_network)
|
||||
TRAEFIK_ENTRYPOINT=$(read_traefik_entrypoint)
|
||||
TRAEFIK_CERTRESOLVER=$(read_traefik_certresolver)
|
||||
TRAEFIK_EXTERNAL_NETWORK=$(resolve NETBIRD_TRAEFIK_EXTERNAL_NETWORK "" read_traefik_network)
|
||||
TRAEFIK_ENTRYPOINT=$(resolve NETBIRD_TRAEFIK_ENTRYPOINT websecure read_traefik_entrypoint)
|
||||
TRAEFIK_CERTRESOLVER=$(resolve NETBIRD_TRAEFIK_CERTRESOLVER "" read_traefik_certresolver)
|
||||
fi
|
||||
|
||||
# Handle port binding for external proxy options (2-5)
|
||||
if [[ "$REVERSE_PROXY_TYPE" -ge 2 ]]; then
|
||||
BIND_LOCALHOST_ONLY=$(read_port_binding_preference)
|
||||
BIND_LOCALHOST_ONLY=$(resolve NETBIRD_BIND_LOCALHOST_ONLY true read_port_binding_preference)
|
||||
fi
|
||||
|
||||
# Handle Docker network prompts for external proxies (options 2-4)
|
||||
case "$REVERSE_PROXY_TYPE" in
|
||||
2) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx") ;;
|
||||
3) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Nginx Proxy Manager") ;;
|
||||
4) EXTERNAL_PROXY_NETWORK=$(read_proxy_docker_network "Caddy") ;;
|
||||
2) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx") ;;
|
||||
3) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Nginx Proxy Manager") ;;
|
||||
4) EXTERNAL_PROXY_NETWORK=$(resolve NETBIRD_EXTERNAL_PROXY_NETWORK "" read_proxy_docker_network "Caddy") ;;
|
||||
*) ;; # No network prompt for other options
|
||||
esac
|
||||
return 0
|
||||
@@ -643,8 +699,13 @@ start_services_and_show_instructions() {
|
||||
print_post_setup_instructions
|
||||
|
||||
echo ""
|
||||
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
|
||||
read -r < /dev/tty
|
||||
if tty_available; then
|
||||
echo -n "Press Enter when your reverse proxy is configured (or Ctrl+C to exit)... "
|
||||
read -r < /dev/tty
|
||||
else
|
||||
echo "Non-interactive mode: starting NetBird containers now. Finish configuring"
|
||||
echo "your reverse proxy using the instructions above so it can reach them."
|
||||
fi
|
||||
|
||||
echo -e "$MSG_STARTING_SERVICES"
|
||||
$DOCKER_COMPOSE_COMMAND up -d
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetAccountSettings(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into accounts (id, settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
|
||||
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
|
||||
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
|
||||
values('account-3',null,null,null,null,null,null,null,null,null,null,null)`)
|
||||
|
||||
accountSettings, err := conn(t, ctx).GetAccountSettings(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
|
||||
PeerLoginExpirationEnabled: true,
|
||||
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
|
||||
PeerInactivityExpirationEnabled: false,
|
||||
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
|
||||
DNSDomain: "",
|
||||
IPv6EnabledGroups: []string{"group-one-resource-id"},
|
||||
RoutingPeerDNSResolutionEnabled: false,
|
||||
LazyConnectionEnabled: false,
|
||||
AutoUpdateVersion: "disabled",
|
||||
AutoUpdateAlways: false,
|
||||
MetricsPushEnabled: false,
|
||||
})
|
||||
|
||||
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-2")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
|
||||
PeerLoginExpirationEnabled: true,
|
||||
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
|
||||
PeerInactivityExpirationEnabled: false,
|
||||
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
|
||||
DNSDomain: "",
|
||||
IPv6EnabledGroups: []string{"group-two-resources-id"},
|
||||
RoutingPeerDNSResolutionEnabled: false,
|
||||
LazyConnectionEnabled: false,
|
||||
AutoUpdateVersion: "disabled",
|
||||
AutoUpdateAlways: false,
|
||||
MetricsPushEnabled: false,
|
||||
})
|
||||
|
||||
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-3")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{})
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
|
||||
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
|
||||
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
|
||||
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
|
||||
VALUES('account-1','network-1','{"IP":"100.103.0.0","Mask":"//8AAA=="}','{"IP":"fdde:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',1,'["disabled-group-1","disabled-group-2"]',
|
||||
true, 86400000000000, false,
|
||||
86400000000000, null, '["group-one-resource-id"]', false,
|
||||
false, 'disabled', false, false);
|
||||
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
|
||||
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
|
||||
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
|
||||
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
|
||||
VALUES('account-2','network-2','{"IP":"110.0.0.0","Mask":"//8AAA=="}','{"IP":"fddf:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',2,null,
|
||||
true, 86400000000000, false,
|
||||
86400000000000, null, '["group-two-resources-id"]', false,
|
||||
false, 'disabled', false, false);
|
||||
insert into groups (id, account_id, name, resources, public_id) VALUES('group-one-resource-id','account-1','group-1-name', '[{"ID":"host-id-1","Type":"host"}]','group-one-resource-id-public');
|
||||
insert into groups (id, account_id, name, resources, public_id) VALUES('group-two-resources-id','account-1','group-2-name', '[{"ID":"subnet-id-1","Type":"subnet"}, {"ID":"host-id-2","Type":"host"}]','group-two-resources-id-public');
|
||||
insert into groups (id, account_id, name, resources, public_id) VALUES('group-no-resources-id','account-1','group-3-name', null,'group-no-resources-id-public');
|
||||
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-1','group-one-resource-id');
|
||||
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-2','group-two-resources-id');
|
||||
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-3','group-two-resources-id');
|
||||
@@ -1,25 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetDnsSettings(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
settings, err := conn(t, ctx).GetDnsSettings(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, settings, nmdata.DNSSettings{
|
||||
DisabledManagementGroups: []string{"disabled-group-1", "disabled-group-2"},
|
||||
})
|
||||
|
||||
settings, err = conn(t, ctx).GetDnsSettings(ctx, "account-2")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, settings, nmdata.DNSSettings{})
|
||||
}
|
||||
@@ -1,62 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/miekg/dns"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetAppliedZoneCandidatesViaPgxConnection(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
|
||||
VALUES('zone-1','account-1','test-1.com',true,'["group-one-resource-id"]')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
|
||||
VALUES('zone-2','account-1','test-2.com',false,'["group-two-resources-id"]')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into records (id, account_id, zone_id, name, type, ttl, content)
|
||||
VALUES('record-1','account-1','zone-1','test.test-1.com','A',1800,'1.1.1.1')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into records (id, account_id, zone_id, name, type, ttl, content)
|
||||
VALUES('record-2','account-1','zone-1','test2.test-1.com','A',1800,'1.1.1.2')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into records (id, account_id, zone_id, name, type, ttl, content)
|
||||
VALUES('record-3','account-1','zone-1','test3.test-1.com','CNAME',1800,'test4.test-1.com')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into records (id, account_id, zone_id, name, type, ttl, content)
|
||||
VALUES('record-4','account-1','zone-2','test2.test-2.com','CNAME',1800,'test3.test-2.com')`)
|
||||
|
||||
zoneCandidates, err := conn(t, ctx).GetAppliedZoneCandidates(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
|
||||
DistributionGroups: []string{"group-one-resource-id"},
|
||||
Zone: nmdata.CustomZone{
|
||||
Domain: "test-1.com",
|
||||
SearchDomainDisabled: false,
|
||||
Records: []nmdata.SimpleRecord{
|
||||
{Name: "test.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.1"},
|
||||
{Name: "test2.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.2"},
|
||||
{Name: "test3.test-1.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test4.test-1.com."},
|
||||
},
|
||||
},
|
||||
})
|
||||
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
|
||||
DistributionGroups: []string{"group-two-resources-id"},
|
||||
Zone: nmdata.CustomZone{
|
||||
Domain: "test-2.com",
|
||||
SearchDomainDisabled: true,
|
||||
Records: []nmdata.SimpleRecord{
|
||||
{Name: "test2.test-2.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test3.test-2.com."},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -1,39 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"testing"
|
||||
|
||||
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetDomains(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into domains (id, account_id, domain, target_cluster)
|
||||
VALUES('domain-1','account-1','test-1.com','target-1.cluster.local')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into domains (id, account_id, domain, target_cluster)
|
||||
VALUES('domain-2','account-1','test-2.com','target-2.cluster.local')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into domains (id, account_id, domain, target_cluster)
|
||||
VALUES('domain-3','account-1',null,null)`)
|
||||
|
||||
domains, err := conn(t, ctx).GetDomains(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, domains, 2)
|
||||
|
||||
assert.Contains(t, domains, networkmapdb.Domain{
|
||||
Domain: sql.NullString{String: "test-1.com", Valid: true},
|
||||
TargetCluster: sql.NullString{String: "target-1.cluster.local", Valid: true},
|
||||
})
|
||||
assert.Contains(t, domains, networkmapdb.Domain{
|
||||
Domain: sql.NullString{String: "test-2.com", Valid: true},
|
||||
TargetCluster: sql.NullString{String: "target-2.cluster.local", Valid: true},
|
||||
})
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/management/server/types"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestGetGroups(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
groups, resourceToGroupIdx, err := conn(t, ctx).GetGroups(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Contains(t,
|
||||
groups,
|
||||
nmdata.Group{ID: "group-one-resource-id", Name: "group-1-name", PublicID: "group-one-resource-id-public", Resources: []nmdata.Resource{{ID: "host-id-1", Type: "host"}}, Peers: []string{"peer-id-1"}},
|
||||
)
|
||||
assert.NotNil(t, resourceToGroupIdx["host-id-1"]["group-one-resource-id"])
|
||||
assert.Contains(t,
|
||||
groups,
|
||||
nmdata.Group{ID: "group-two-resources-id", Name: "group-2-name", PublicID: "group-two-resources-id-public",
|
||||
Resources: []nmdata.Resource{{ID: "subnet-id-1", Type: "subnet"}, {ID: "host-id-2", Type: "host"}},
|
||||
Peers: []string{"peer-id-2", "peer-id-3"}},
|
||||
)
|
||||
assert.NotNil(t, resourceToGroupIdx["host-id-2"]["group-two-resources-id"])
|
||||
assert.NotNil(t, resourceToGroupIdx["subnet-id-1"]["group-two-resources-id"])
|
||||
assert.Contains(t,
|
||||
groups,
|
||||
nmdata.Group{ID: "group-no-resources-id", Name: "group-3-name", PublicID: "group-no-resources-id-public"})
|
||||
}
|
||||
|
||||
// Verify handling of empty fields in groups table
|
||||
// Verify that group's PublicID gets populated on retrieval
|
||||
// TODO (dmitri) PublicID should not be populated with delta updates,
|
||||
// which require stable PublicIDs
|
||||
func TestGetGroupsWithoutExpectedFields(t *testing.T) {
|
||||
if engine == string(types.SqliteStoreEngine) {
|
||||
t.Skip()
|
||||
}
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
"insert into accounts (id) VALUES('random-id')")
|
||||
|
||||
execQuery(t, ctx,
|
||||
"insert into groups (id, account_id) VALUES('g2-test-group-id-1','random-id')")
|
||||
|
||||
groups, _, err := conn(t, ctx).GetGroups(ctx, "random-id")
|
||||
assert.NoError(t, err)
|
||||
require.Len(t, groups, 1)
|
||||
assert.NotEmpty(t, groups[0].PublicID)
|
||||
}
|
||||
@@ -1,105 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
_ "embed"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
|
||||
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
|
||||
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
|
||||
"github.com/netbirdio/netbird/management/server/types"
|
||||
)
|
||||
|
||||
//go:embed base_data.sql
|
||||
var baseData string
|
||||
|
||||
//go:embed pg_data.sql
|
||||
var pgData string
|
||||
|
||||
//go:embed sqlite_data.sql
|
||||
var sqliteData string
|
||||
|
||||
var (
|
||||
pgstore *networkmap_pgsql.PgStore
|
||||
sqlitestore *networkmap_sqlite.SqliteStore
|
||||
engine string
|
||||
)
|
||||
|
||||
func TestMain(m *testing.M) {
|
||||
var cleanup func()
|
||||
kind, _ := os.LookupEnv("NETBIRD_STORE_ENGINE")
|
||||
switch kind {
|
||||
case string(types.PostgresStoreEngine):
|
||||
engine = string(types.PostgresStoreEngine)
|
||||
pgstore, cleanup = createPGTestStore(baseData, pgData)
|
||||
pgstore.UsingTimeZone(time.UTC)
|
||||
case "", string(types.SqliteStoreEngine):
|
||||
engine = string(types.SqliteStoreEngine)
|
||||
sqlitestore, cleanup = createSqliteTestStore(baseData, sqliteData)
|
||||
default:
|
||||
log.Fatalf("unsupported db '%s' in NETBIRD_STORE_ENGINE env var", kind)
|
||||
}
|
||||
|
||||
code := m.Run()
|
||||
|
||||
cleanup()
|
||||
os.Exit(code)
|
||||
}
|
||||
|
||||
func conn(t *testing.T, ctx context.Context) networkmapdb.NetworkMapDBStoreConn {
|
||||
t.Helper()
|
||||
switch engine {
|
||||
case string(types.PostgresStoreEngine):
|
||||
c, err := pgstore.Pool.Acquire(ctx)
|
||||
assert.NoError(t, err)
|
||||
return pgstore.UsingConnection(c.Conn())
|
||||
case string(types.SqliteStoreEngine):
|
||||
return sqlitestore.UsingConn()
|
||||
}
|
||||
log.Fatalf("unknown db engine kind %s", engine)
|
||||
return nil
|
||||
}
|
||||
|
||||
func store(t *testing.T) networkmapdb.NetworkMapDBStore {
|
||||
t.Helper()
|
||||
switch engine {
|
||||
case string(types.PostgresStoreEngine):
|
||||
return pgstore
|
||||
case string(types.SqliteStoreEngine):
|
||||
return sqlitestore
|
||||
}
|
||||
log.Fatalf("unknown db engine kind %s", engine)
|
||||
return nil
|
||||
}
|
||||
|
||||
func execQuery(t *testing.T, ctx context.Context, q string) {
|
||||
t.Helper()
|
||||
switch engine {
|
||||
case string(types.PostgresStoreEngine):
|
||||
_, err := pgstore.Pool.Exec(ctx, q)
|
||||
assert.NoError(t, err)
|
||||
case string(types.SqliteStoreEngine):
|
||||
_, err := sqlitestore.Db.ExecContext(ctx, q)
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
}
|
||||
|
||||
// use to parse time in time.RFC3339Nano format
|
||||
// returns the time in the UTC time zone
|
||||
func mustParseTime(t string) *time.Time {
|
||||
tt, err := time.Parse(time.RFC3339Nano, t)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
utc := tt.UTC()
|
||||
return &utc
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetNameServerGroups(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled, "primary", account_id)
|
||||
VALUES('nsgroup-1','nsgroup-1-public','nsgroup-1','nsgroup-1','[{"IP":"192.168.31.2","NSType":1,"Port":53}]','["group-one-resource-id"]','["test-1.com"]',TRUE,FALSE,TRUE,'account-1')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
|
||||
VALUES('nsgroup-2','nsgroup-2-public','nsgroup-2','nsgroup-2','[{"IP":"192.168.32.3","NSType":1,"Port":53}]','["group-one-resource-id","group-no-resources-id"]','["test-1.com","test-2.com"]',TRUE,FALSE,TRUE,'account-1')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
|
||||
VALUES('nsgroup-3','nsgroup-3-public',null,null,null,null,null,TRUE,FALSE,FALSE,'account-1')`)
|
||||
|
||||
nsgroups, err := conn(t, ctx).GetNameServerGroups(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
|
||||
ID: "nsgroup-1",
|
||||
PublicID: "nsgroup-1-public",
|
||||
Name: "nsgroup-1",
|
||||
Description: "nsgroup-1",
|
||||
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.31.2"), NSType: 1, Port: 53}},
|
||||
Groups: []string{"group-one-resource-id"},
|
||||
Domains: []string{"test-1.com"},
|
||||
Primary: true,
|
||||
SearchDomainsEnabled: false,
|
||||
Enabled: true,
|
||||
})
|
||||
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
|
||||
ID: "nsgroup-2",
|
||||
PublicID: "nsgroup-2-public",
|
||||
Name: "nsgroup-2",
|
||||
Description: "nsgroup-2",
|
||||
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.32.3"), NSType: 1, Port: 53}},
|
||||
Groups: []string{"group-one-resource-id", "group-no-resources-id"},
|
||||
Domains: []string{"test-1.com", "test-2.com"},
|
||||
Primary: true,
|
||||
SearchDomainsEnabled: false,
|
||||
Enabled: true,
|
||||
})
|
||||
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
|
||||
ID: "nsgroup-3",
|
||||
PublicID: "nsgroup-3-public",
|
||||
Primary: false,
|
||||
SearchDomainsEnabled: false,
|
||||
Enabled: true,
|
||||
})
|
||||
}
|
||||
@@ -1,372 +0,0 @@
|
||||
{
|
||||
"Peers": {
|
||||
"peer-id-1": {
|
||||
"ID": "peer-id-1",
|
||||
"Key": "key-1",
|
||||
"SSHKey": "ssh-key-1",
|
||||
"DNSLabel": "peer-1",
|
||||
"UserID": "user-id-1",
|
||||
"SSHEnabled": true,
|
||||
"LoginExpirationEnabled": true,
|
||||
"LastLogin": "2026-08-06T13:25:59.12999Z",
|
||||
"IP": "10.10.10.1",
|
||||
"IPv6": "fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940",
|
||||
"RequiresApproval": false,
|
||||
"ExtraDNSLabels": [
|
||||
"extra-peer-1"
|
||||
],
|
||||
"Meta": {
|
||||
"WtVersion": "0.76.0",
|
||||
"GoOS": "linux",
|
||||
"OSVersion": "26.4.1",
|
||||
"KernelVersion": "6.8.0-134-generic",
|
||||
"NetworkAddresses": [
|
||||
{
|
||||
"NetIP": "fe80::8b4c:973f:a76b:3771/64"
|
||||
},
|
||||
{
|
||||
"NetIP": "192.168.16.1/20"
|
||||
}
|
||||
],
|
||||
"Files": [
|
||||
{
|
||||
"Path": "/usr/bin/netbird",
|
||||
"ProcessIsRunning": false
|
||||
}
|
||||
],
|
||||
"Capabilities": [
|
||||
1,
|
||||
2
|
||||
],
|
||||
"Flags": {
|
||||
"ServerSSHAllowed": true,
|
||||
"DisableIPv6": false
|
||||
},
|
||||
"SyncMessageVersion": 1
|
||||
},
|
||||
"ProxyMeta": {
|
||||
"Embedded": true
|
||||
},
|
||||
"Location": {
|
||||
"CountryCode": "DE",
|
||||
"CityName": "Berlin",
|
||||
"ConnectionIP": "46.201.148.187"
|
||||
}
|
||||
},
|
||||
"peer-id-2": {
|
||||
"ID": "peer-id-2",
|
||||
"Key": "key-2",
|
||||
"SSHKey": "ssh-key-2",
|
||||
"DNSLabel": "peer-2",
|
||||
"UserID": "user-id-2",
|
||||
"SSHEnabled": true,
|
||||
"LoginExpirationEnabled": true,
|
||||
"LastLogin": "2026-08-06T14:25:59.12999Z",
|
||||
"IP": "10.10.100.1",
|
||||
"IPv6": "fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940",
|
||||
"RequiresApproval": false,
|
||||
"ExtraDNSLabels": [
|
||||
"extra-peer-2"
|
||||
],
|
||||
"Meta": {
|
||||
"WtVersion": "0.76.1",
|
||||
"GoOS": "linux",
|
||||
"OSVersion": "26.4.2",
|
||||
"KernelVersion": "6.8.0-135-generic",
|
||||
"NetworkAddresses": [
|
||||
{
|
||||
"NetIP": "fe81::8b4c:973f:a76b:3771/64"
|
||||
},
|
||||
{
|
||||
"NetIP": "192.168.17.1/20"
|
||||
}
|
||||
],
|
||||
"Files": [
|
||||
{
|
||||
"Path": "/usr/bin/netbird",
|
||||
"ProcessIsRunning": false
|
||||
}
|
||||
],
|
||||
"Capabilities": [
|
||||
1,
|
||||
2
|
||||
],
|
||||
"Flags": {
|
||||
"ServerSSHAllowed": true,
|
||||
"DisableIPv6": false
|
||||
},
|
||||
"SyncMessageVersion": 0
|
||||
},
|
||||
"ProxyMeta": {
|
||||
"Embedded": true
|
||||
},
|
||||
"Location": {
|
||||
"CountryCode": "DE",
|
||||
"CityName": "Berlin",
|
||||
"ConnectionIP": "46.201.149.187"
|
||||
}
|
||||
},
|
||||
"peer-id-3": {
|
||||
"ID": "peer-id-3",
|
||||
"Key": "key-3",
|
||||
"SSHKey": "ssh-key-3",
|
||||
"DNSLabel": "peer-3",
|
||||
"UserID": "user-id-3",
|
||||
"SSHEnabled": true,
|
||||
"LoginExpirationEnabled": true,
|
||||
"LastLogin": "2026-08-06T12:25:59.12999Z",
|
||||
"IP": "10.10.200.1",
|
||||
"IPv6": "fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940",
|
||||
"RequiresApproval": false,
|
||||
"ExtraDNSLabels": [
|
||||
"extra-peer-3"
|
||||
],
|
||||
"Meta": {
|
||||
"WtVersion": "0.76.2",
|
||||
"GoOS": "linux",
|
||||
"OSVersion": "26.4.3",
|
||||
"KernelVersion": "6.8.0-136-generic",
|
||||
"NetworkAddresses": [
|
||||
{
|
||||
"NetIP": "fe82::8b4c:973f:a76b:3771/64"
|
||||
},
|
||||
{
|
||||
"NetIP": "192.168.18.1/20"
|
||||
}
|
||||
],
|
||||
"Files": [
|
||||
{
|
||||
"Path": "/usr/bin/netbird",
|
||||
"ProcessIsRunning": false
|
||||
}
|
||||
],
|
||||
"Capabilities": [
|
||||
1,
|
||||
2
|
||||
],
|
||||
"Flags": {
|
||||
"ServerSSHAllowed": true,
|
||||
"DisableIPv6": false
|
||||
},
|
||||
"SyncMessageVersion": 1
|
||||
},
|
||||
"ProxyMeta": {
|
||||
"Embedded": true
|
||||
},
|
||||
"Location": {
|
||||
"CountryCode": "DE",
|
||||
"CityName": "Berlin",
|
||||
"ConnectionIP": "46.201.150.187"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Groups": {
|
||||
"group-no-resources-id": {
|
||||
"ID": "group-no-resources-id",
|
||||
"Name": "group-3-name",
|
||||
"PublicID": "group-no-resources-id-public",
|
||||
"Peers": null,
|
||||
"Resources": null
|
||||
},
|
||||
"group-one-resource-id": {
|
||||
"ID": "group-one-resource-id",
|
||||
"Name": "group-1-name",
|
||||
"PublicID": "group-one-resource-id-public",
|
||||
"Peers": [
|
||||
"peer-id-1"
|
||||
],
|
||||
"Resources": [
|
||||
{
|
||||
"ID": "host-id-1",
|
||||
"Type": "host"
|
||||
}
|
||||
]
|
||||
},
|
||||
"group-two-resources-id": {
|
||||
"ID": "group-two-resources-id",
|
||||
"Name": "group-2-name",
|
||||
"PublicID": "group-two-resources-id-public",
|
||||
"Peers": [
|
||||
"peer-id-2",
|
||||
"peer-id-3"
|
||||
],
|
||||
"Resources": [
|
||||
{
|
||||
"ID": "subnet-id-1",
|
||||
"Type": "subnet"
|
||||
},
|
||||
{
|
||||
"ID": "host-id-2",
|
||||
"Type": "host"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Policies": [],
|
||||
"Routes": [],
|
||||
"NameServerGroups": [
|
||||
{
|
||||
"ID": "nsgroup-1",
|
||||
"PublicID": "nsgroup-1-public",
|
||||
"Name": "nsgroup-1",
|
||||
"Description": "nsgroup-1",
|
||||
"NameServers": [
|
||||
{
|
||||
"IP": "192.168.31.2",
|
||||
"NSType": 1,
|
||||
"Port": 53
|
||||
}
|
||||
],
|
||||
"Groups": [
|
||||
"group-one-resource-id"
|
||||
],
|
||||
"Primary": true,
|
||||
"Domains": [
|
||||
"test-1.com"
|
||||
],
|
||||
"Enabled": true,
|
||||
"SearchDomainsEnabled": false
|
||||
},
|
||||
{
|
||||
"ID": "nsgroup-2",
|
||||
"PublicID": "nsgroup-2-public",
|
||||
"Name": "nsgroup-2",
|
||||
"Description": "nsgroup-2",
|
||||
"NameServers": [
|
||||
{
|
||||
"IP": "192.168.32.3",
|
||||
"NSType": 1,
|
||||
"Port": 53
|
||||
}
|
||||
],
|
||||
"Groups": [
|
||||
"group-one-resource-id",
|
||||
"group-no-resources-id"
|
||||
],
|
||||
"Primary": true,
|
||||
"Domains": [
|
||||
"test-1.com",
|
||||
"test-2.com"
|
||||
],
|
||||
"Enabled": true,
|
||||
"SearchDomainsEnabled": false
|
||||
},
|
||||
{
|
||||
"ID": "nsgroup-3",
|
||||
"PublicID": "nsgroup-3-public",
|
||||
"Name": "",
|
||||
"Description": "",
|
||||
"NameServers": null,
|
||||
"Groups": null,
|
||||
"Primary": false,
|
||||
"Domains": null,
|
||||
"Enabled": true,
|
||||
"SearchDomainsEnabled": false
|
||||
}
|
||||
],
|
||||
"NetworkResources": [],
|
||||
"Network": {
|
||||
"Identifier": "network-1",
|
||||
"Net": {
|
||||
"IP": "100.103.0.0",
|
||||
"Mask": "//8AAA=="
|
||||
},
|
||||
"NetV6": {
|
||||
"IP": "fdde:e995:fd38:a465::",
|
||||
"Mask": "//////////8AAAAAAAAAAA=="
|
||||
},
|
||||
"Dns": "",
|
||||
"Serial": 1
|
||||
},
|
||||
"DNSSettings": {
|
||||
"DisabledManagementGroups": [
|
||||
"disabled-group-1",
|
||||
"disabled-group-2"
|
||||
]
|
||||
},
|
||||
"AccountSettings": {
|
||||
"PeerLoginExpirationEnabled": true,
|
||||
"PeerLoginExpiration": 86400000000000,
|
||||
"PeerInactivityExpirationEnabled": false,
|
||||
"PeerInactivityExpiration": 86400000000000,
|
||||
"DNSDomain": "",
|
||||
"IPv6EnabledGroups": [
|
||||
"group-one-resource-id"
|
||||
],
|
||||
"RoutingPeerDNSResolutionEnabled": false,
|
||||
"LazyConnectionEnabled": false,
|
||||
"AutoUpdateVersion": "disabled",
|
||||
"AutoUpdateAlways": false,
|
||||
"MetricsPushEnabled": false
|
||||
},
|
||||
"PostureChecks": {},
|
||||
"PostureValidation": null,
|
||||
"AllowedUserIDs": {},
|
||||
"NetworkXIDToPublicID": {},
|
||||
"PostureCheckXIDToPublicID": {},
|
||||
"ValidatedPeers": {
|
||||
"peer-id-1": {},
|
||||
"peer-id-2": {},
|
||||
"peer-id-3": {}
|
||||
},
|
||||
"ResourcePolicies": {},
|
||||
"Routers": {},
|
||||
"GroupIDToUserIDs": {},
|
||||
"DNSDomain": "",
|
||||
"ProxyTargetedDomainResourceIDs": {},
|
||||
"AppliedZoneCandidates": [
|
||||
{
|
||||
"DistributionGroups": [
|
||||
"group-one-resource-id"
|
||||
],
|
||||
"Zone": {
|
||||
"Domain": "test-1.com",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "test.test-1.com",
|
||||
"Type": 1,
|
||||
"Class": "IN",
|
||||
"TTL": 1800,
|
||||
"RData": "1.1.1.1"
|
||||
},
|
||||
{
|
||||
"Name": "test2.test-1.com",
|
||||
"Type": 1,
|
||||
"Class": "IN",
|
||||
"TTL": 1800,
|
||||
"RData": "1.1.1.2"
|
||||
},
|
||||
{
|
||||
"Name": "test3.test-1.com",
|
||||
"Type": 5,
|
||||
"Class": "IN",
|
||||
"TTL": 1800,
|
||||
"RData": "test4.test-1.com."
|
||||
}
|
||||
],
|
||||
"SearchDomainDisabled": false,
|
||||
"NonAuthoritative": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"DistributionGroups": [
|
||||
"group-two-resources-id"
|
||||
],
|
||||
"Zone": {
|
||||
"Domain": "test-2.com",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "test2.test-2.com",
|
||||
"Type": 5,
|
||||
"Class": "IN",
|
||||
"TTL": 1800,
|
||||
"RData": "test3.test-2.com."
|
||||
}
|
||||
],
|
||||
"SearchDomainDisabled": true,
|
||||
"NonAuthoritative": false
|
||||
}
|
||||
}
|
||||
],
|
||||
"PrivateServiceCandidates": null
|
||||
}
|
||||
@@ -1,103 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"net/netip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
|
||||
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
|
||||
"github.com/netbirdio/netbird/management/server/integrations/extra_settings"
|
||||
nbpeer "github.com/netbirdio/netbird/management/server/peer"
|
||||
"github.com/netbirdio/netbird/management/server/types"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/netbirdio/netbird/shared/management/proto"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
//go:embed network_map_data_golden.json
|
||||
var goldenNMap string
|
||||
|
||||
const EnvUpdateGoldenData = "NMAP_UPDATE_GOLDEN_DATA"
|
||||
|
||||
func TestGetNetworkMapData(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
storeImpl := networkmapdb.NetworkMapDBStoreImpl{
|
||||
Store: store(t),
|
||||
ExtraSettingsManager: &extraSettingsManagerForTesting{},
|
||||
IntegratedPeerValidator: &peerValidatorForTesting{},
|
||||
}
|
||||
|
||||
nmap, err := storeImpl.GetNetworkMapData(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
serializedNMap, err := json.MarshalIndent(nmap, "", " ")
|
||||
assert.NoError(t, err)
|
||||
|
||||
if _, ok := os.LookupEnv(EnvUpdateGoldenData); ok {
|
||||
_, filename, _, _ := runtime.Caller(0)
|
||||
tosavepath := filepath.Join(filepath.Dir(filename), "network_map_data_golden.json")
|
||||
err = os.WriteFile(tosavepath, serializedNMap, 0644)
|
||||
assert.NoError(t, err)
|
||||
goldenNMap = string(serializedNMap)
|
||||
}
|
||||
assert.Equal(t, goldenNMap, string(serializedNMap))
|
||||
}
|
||||
|
||||
// need these calls for the test
|
||||
func (*extraSettingsManagerForTesting) GetExtraSettings(ctx context.Context, accountID string) (*types.ExtraSettings, error) {
|
||||
return &types.ExtraSettings{}, nil
|
||||
}
|
||||
func (*peerValidatorForTesting) GetValidatedPeers(ctx context.Context, accountID string, groups []*nmdata.Group, peers []*nmdata.Peer, extraSettings *types.ExtraSettings) (map[string]struct{}, error) {
|
||||
return map[string]struct{}{
|
||||
"peer-id-1": {},
|
||||
"peer-id-2": {},
|
||||
"peer-id-3": {},
|
||||
}, nil
|
||||
}
|
||||
|
||||
type extraSettingsManagerForTesting struct{}
|
||||
|
||||
func (*extraSettingsManagerForTesting) GetExtraSettingsManager() extra_settings.Manager { return nil }
|
||||
func (*extraSettingsManagerForTesting) GetSettings(ctx context.Context, accountID string, userID string) (*types.Settings, error) {
|
||||
return nil, nil
|
||||
}
|
||||
func (*extraSettingsManagerForTesting) UpdateExtraSettings(ctx context.Context, accountID, userID string, extraSettings *types.ExtraSettings) (bool, error) {
|
||||
return false, nil
|
||||
}
|
||||
func (*extraSettingsManagerForTesting) GetEffectiveNetworkRanges(ctx context.Context, accountID string) (v4, v6 netip.Prefix, err error) {
|
||||
return netip.Prefix{}, netip.Prefix{}, nil
|
||||
}
|
||||
|
||||
type peerValidatorForTesting struct{}
|
||||
|
||||
func (*peerValidatorForTesting) ValidateExtraSettings(ctx context.Context, newExtraSettings *types.ExtraSettings, oldExtraSettings *types.ExtraSettings, userID string, accountID string) error {
|
||||
return nil
|
||||
}
|
||||
func (*peerValidatorForTesting) ValidatePeer(ctx context.Context, update *nbpeer.Peer, peer *nbpeer.Peer, userID string, accountID string, dnsDomain string, peersGroup []string, extraSettings *types.ExtraSettings) (*nbpeer.Peer, bool, error) {
|
||||
return nil, false, nil
|
||||
}
|
||||
func (*peerValidatorForTesting) PreparePeer(ctx context.Context, accountID string, peer *nbpeer.Peer, peersGroup []string, extraSettings *types.ExtraSettings, temporary bool) *nbpeer.Peer {
|
||||
return nil
|
||||
}
|
||||
func (*peerValidatorForTesting) IsNotValidPeer(ctx context.Context, accountID string, peer *nbpeer.Peer, peersGroup []string, extraSettings *types.ExtraSettings) (bool, bool, error) {
|
||||
return false, false, nil
|
||||
}
|
||||
func (*peerValidatorForTesting) GetInvalidPeers(ctx context.Context, accountID string, extraSettings *types.ExtraSettings) (map[string]string, error) {
|
||||
return nil, nil
|
||||
}
|
||||
func (*peerValidatorForTesting) PeerDeleted(ctx context.Context, accountID, peerID string, extraSettings *types.ExtraSettings) error {
|
||||
return nil
|
||||
}
|
||||
func (*peerValidatorForTesting) SetPeerInvalidationListener(fn func(accountID string, peerIDs []string)) {
|
||||
}
|
||||
func (*peerValidatorForTesting) Stop(ctx context.Context) {}
|
||||
func (*peerValidatorForTesting) ValidateFlowResponse(ctx context.Context, peerKey string, flowResponse *proto.PKCEAuthorizationFlow) *proto.PKCEAuthorizationFlow {
|
||||
return nil
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetNetworkResources(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
|
||||
VALUES('net-resource-1','account-1','network-1','net-resource-public-1','network-resource-1','network-resource-1','subnet','','"10.0.0.0/16"',TRUE)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
|
||||
VALUES('net-resource-2','account-1','network-2','net-resource-public-2','network-resource-2','network-resource-2','domain','test.com','',TRUE)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
|
||||
VALUES('net-resource-3','account-1','network-3','net-resource-public-3','network-resource-3','network-resource-3','host','','"10.0.0.1/32"',TRUE)`)
|
||||
|
||||
resources, err := conn(t, ctx).GetNetworkResources(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
assert.Contains(t, resources, nmdata.NetworkResource{
|
||||
ID: "net-resource-1",
|
||||
AccountID: "account-1",
|
||||
NetworkID: "network-1",
|
||||
PublicID: "net-resource-public-1",
|
||||
Name: "network-resource-1",
|
||||
Description: "network-resource-1",
|
||||
Type: "subnet",
|
||||
Domain: "",
|
||||
Prefix: netip.MustParsePrefix("10.0.0.0/16"),
|
||||
Enabled: true,
|
||||
})
|
||||
assert.Contains(t, resources, nmdata.NetworkResource{
|
||||
ID: "net-resource-2",
|
||||
AccountID: "account-1",
|
||||
NetworkID: "network-2",
|
||||
PublicID: "net-resource-public-2",
|
||||
Name: "network-resource-2",
|
||||
Description: "network-resource-2",
|
||||
Type: "domain",
|
||||
Domain: "test.com",
|
||||
Enabled: true,
|
||||
})
|
||||
assert.Contains(t, resources, nmdata.NetworkResource{
|
||||
ID: "net-resource-3",
|
||||
AccountID: "account-1",
|
||||
NetworkID: "network-3",
|
||||
PublicID: "net-resource-public-3",
|
||||
Name: "network-resource-3",
|
||||
Description: "network-resource-3",
|
||||
Type: "host",
|
||||
Domain: "",
|
||||
Prefix: netip.MustParsePrefix("10.0.0.1/32"),
|
||||
Enabled: true,
|
||||
})
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetNetworkRouters(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
|
||||
VALUES('test-nr-id-1','account-1','public-id-1','peer-id-1','network-id-1',TRUE,999,TRUE,'["group-one-resource-id"]')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
|
||||
VALUES('test-nr-id-2','account-1','public-id-2','','network-id-2',TRUE,333,TRUE,'["group-two-resources-id","group-no-resources-id"]')`)
|
||||
|
||||
routers, err := conn(t, ctx).GetNetworkRouters(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.NotEmpty(t, routers)
|
||||
|
||||
assert.Equal(t, routers["network-id-1"],
|
||||
map[string]*nmdata.NetworkRouter{"peer-id-1": {PublicID: "public-id-1", Masquerade: true, Metric: 999, Enabled: true, PeerGroups: []string{"group-one-resource-id"}}})
|
||||
assert.Equal(t, routers["network-id-2"],
|
||||
map[string]*nmdata.NetworkRouter{
|
||||
"peer-id-2": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}},
|
||||
"peer-id-3": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}}})
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetNetwork(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
network, err := conn(t, ctx).GetNetwork(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, network, nmdata.Network{
|
||||
Identifier: "network-1",
|
||||
Net: mustParseCIDR("100.103.0.0/16"),
|
||||
NetV6: mustParseCIDR("fdde:e995:fd38:a465::/64"),
|
||||
Serial: 1,
|
||||
})
|
||||
|
||||
network, err = conn(t, ctx).GetNetwork(ctx, "account-2")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, network, nmdata.Network{
|
||||
Identifier: "network-2",
|
||||
Net: mustParseCIDR("110.0.0.0/16"),
|
||||
NetV6: mustParseCIDR("fddf:e995:fd38:a465::/64"),
|
||||
Serial: 2,
|
||||
})
|
||||
}
|
||||
|
||||
func mustParseCIDR(s string) net.IPNet {
|
||||
var toret net.IPNet
|
||||
|
||||
_, net, err := net.ParseCIDR(s)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
jn, err := json.Marshal(net)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
err = json.Unmarshal(jn, &toret)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
return toret
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetNetworks(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into networks (id, account_id, public_id) VALUES('network-1','account-1','network-1-public')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into networks (id, account_id, public_id) VALUES('network-2','account-1','network-2-public')`)
|
||||
|
||||
networksIdx, err := conn(t, ctx).GetNetworkXIDToPublicIdMap(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, networksIdx, map[string]string{
|
||||
"network-1": "network-1-public",
|
||||
"network-2": "network-2-public",
|
||||
})
|
||||
}
|
||||
@@ -1,163 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetPeers(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
peers, clusterToPeersIdx, err := conn(t, ctx).GetPeers(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
// shouldn't be returned in the index, as it's not connected
|
||||
execQuery(t, ctx,
|
||||
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
|
||||
values('peer-4','account-1','key-4','ssh-key-4',true,false)`)
|
||||
// shouldn't be returned in the index as it doesn't have cluster set
|
||||
execQuery(t, ctx,
|
||||
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
|
||||
values('peer-5','account-1','key-5','ssh-key-5',false,true)`)
|
||||
|
||||
peer1 := nmdata.Peer{
|
||||
ID: "peer-id-1",
|
||||
Key: "key-1",
|
||||
SSHKey: "ssh-key-1",
|
||||
DNSLabel: "peer-1",
|
||||
ExtraDNSLabels: []string{"extra-peer-1"},
|
||||
UserID: "user-id-1",
|
||||
SSHEnabled: true,
|
||||
LoginExpirationEnabled: true,
|
||||
LastLogin: mustParseTime("2026-08-06T13:25:59.12999+00:00"),
|
||||
IP: netip.MustParseAddr("10.10.10.1"),
|
||||
IPv6: netip.MustParseAddr("fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"),
|
||||
RequiresApproval: false,
|
||||
Meta: nmdata.PeerSystemMeta{
|
||||
WtVersion: "0.76.0",
|
||||
GoOS: "linux",
|
||||
OSVersion: "26.4.1",
|
||||
KernelVersion: "6.8.0-134-generic",
|
||||
NetworkAddresses: []nmdata.NetworkAddress{
|
||||
{NetIP: netip.MustParsePrefix("fe80::8b4c:973f:a76b:3771/64")},
|
||||
{NetIP: netip.MustParsePrefix("192.168.16.1/20")},
|
||||
},
|
||||
Files: []nmdata.File{
|
||||
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
|
||||
},
|
||||
Capabilities: []int32{1, 2},
|
||||
Flags: nmdata.Flags{
|
||||
ServerSSHAllowed: true,
|
||||
DisableIPv6: false,
|
||||
},
|
||||
SyncMessageVersion: 1,
|
||||
},
|
||||
ProxyMeta: nmdata.ProxyMeta{
|
||||
Embedded: true,
|
||||
},
|
||||
Location: nmdata.PeerLocation{
|
||||
CountryCode: "DE",
|
||||
CityName: "Berlin",
|
||||
ConnectionIP: net.ParseIP("46.201.148.187"),
|
||||
},
|
||||
}
|
||||
peer2 := nmdata.Peer{
|
||||
ID: "peer-id-2",
|
||||
Key: "key-2",
|
||||
SSHKey: "ssh-key-2",
|
||||
DNSLabel: "peer-2",
|
||||
ExtraDNSLabels: []string{"extra-peer-2"},
|
||||
UserID: "user-id-2",
|
||||
SSHEnabled: true,
|
||||
LoginExpirationEnabled: true,
|
||||
LastLogin: mustParseTime("2026-08-06T14:25:59.12999+00:00"),
|
||||
IP: netip.MustParseAddr("10.10.100.1"),
|
||||
IPv6: netip.MustParseAddr("fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"),
|
||||
RequiresApproval: false,
|
||||
Meta: nmdata.PeerSystemMeta{
|
||||
WtVersion: "0.76.1",
|
||||
GoOS: "linux",
|
||||
OSVersion: "26.4.2",
|
||||
KernelVersion: "6.8.0-135-generic",
|
||||
NetworkAddresses: []nmdata.NetworkAddress{
|
||||
{NetIP: netip.MustParsePrefix("fe81::8b4c:973f:a76b:3771/64")},
|
||||
{NetIP: netip.MustParsePrefix("192.168.17.1/20")},
|
||||
},
|
||||
Files: []nmdata.File{
|
||||
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
|
||||
},
|
||||
Capabilities: []int32{1, 2},
|
||||
Flags: nmdata.Flags{
|
||||
ServerSSHAllowed: true,
|
||||
DisableIPv6: false,
|
||||
},
|
||||
SyncMessageVersion: 0,
|
||||
},
|
||||
ProxyMeta: nmdata.ProxyMeta{
|
||||
Embedded: true,
|
||||
},
|
||||
Location: nmdata.PeerLocation{
|
||||
CountryCode: "DE",
|
||||
CityName: "Berlin",
|
||||
ConnectionIP: net.ParseIP("46.201.149.187"),
|
||||
},
|
||||
}
|
||||
peer3 := nmdata.Peer{
|
||||
ID: "peer-id-3",
|
||||
Key: "key-3",
|
||||
SSHKey: "ssh-key-3",
|
||||
DNSLabel: "peer-3",
|
||||
ExtraDNSLabels: []string{"extra-peer-3"},
|
||||
UserID: "user-id-3",
|
||||
SSHEnabled: true,
|
||||
LoginExpirationEnabled: true,
|
||||
LastLogin: mustParseTime("2026-08-06T12:25:59.12999+00:00"),
|
||||
IP: netip.MustParseAddr("10.10.200.1"),
|
||||
IPv6: netip.MustParseAddr("fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"),
|
||||
RequiresApproval: false,
|
||||
Meta: nmdata.PeerSystemMeta{
|
||||
WtVersion: "0.76.2",
|
||||
GoOS: "linux",
|
||||
OSVersion: "26.4.3",
|
||||
KernelVersion: "6.8.0-136-generic",
|
||||
NetworkAddresses: []nmdata.NetworkAddress{
|
||||
{NetIP: netip.MustParsePrefix("fe82::8b4c:973f:a76b:3771/64")},
|
||||
{NetIP: netip.MustParsePrefix("192.168.18.1/20")},
|
||||
},
|
||||
Files: []nmdata.File{
|
||||
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
|
||||
},
|
||||
Capabilities: []int32{1, 2},
|
||||
Flags: nmdata.Flags{
|
||||
ServerSSHAllowed: true,
|
||||
DisableIPv6: false,
|
||||
},
|
||||
SyncMessageVersion: 1,
|
||||
},
|
||||
ProxyMeta: nmdata.ProxyMeta{
|
||||
Embedded: true,
|
||||
},
|
||||
Location: nmdata.PeerLocation{
|
||||
CountryCode: "DE",
|
||||
CityName: "Berlin",
|
||||
ConnectionIP: net.ParseIP("46.201.150.187"),
|
||||
},
|
||||
}
|
||||
|
||||
assert.Contains(t, peers, peer1)
|
||||
assert.Contains(t, peers, peer2)
|
||||
assert.Contains(t, peers, peer3)
|
||||
|
||||
assert.Equal(t, clusterToPeersIdx, map[string][]*nmdata.Peer{
|
||||
"cluster-1.netbird.services": {&peer1},
|
||||
"cluster-2.netbird.services": {&peer2},
|
||||
"cluster-3.netbird.services": {&peer3},
|
||||
})
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-1','account-1','key-1','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999+00','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-1.netbird.services',
|
||||
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
|
||||
'DE','Berlin','"46.201.148.187"');
|
||||
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-2','account-1','key-2','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999+00','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-2.netbird.services',
|
||||
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
|
||||
'DE','Berlin','"46.201.149.187"');
|
||||
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-3','account-1','key-3','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999+00','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-3.netbird.services',
|
||||
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
|
||||
'DE','Berlin','"46.201.150.187"');
|
||||
@@ -1,122 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
|
||||
"github.com/google/uuid"
|
||||
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
|
||||
gormstore "github.com/netbirdio/netbird/management/server/store"
|
||||
"github.com/netbirdio/netbird/management/server/testutil"
|
||||
"gorm.io/driver/postgres"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func createPGTestStore(baseData, pgData string) (*networkmap_pgsql.PgStore, func()) {
|
||||
_, tmpdsn, err := testutil.CreatePostgresTestContainer()
|
||||
if err != nil {
|
||||
log.Fatalf("error starting postres container %v", err)
|
||||
}
|
||||
|
||||
var db *gorm.DB
|
||||
for i := range 5 {
|
||||
db, err = gorm.Open(postgres.Open(tmpdsn), &gorm.Config{})
|
||||
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
|
||||
if i < 5 {
|
||||
waitTime := time.Duration(100*(i+1)) * time.Millisecond
|
||||
time.Sleep(waitTime)
|
||||
continue
|
||||
}
|
||||
|
||||
log.Fatalf("error connecting to postres db %v", err)
|
||||
}
|
||||
|
||||
var cleanup func()
|
||||
dsn, cleanup, err := createRandomDB(tmpdsn, db)
|
||||
sqlDB, _ := db.DB()
|
||||
if sqlDB != nil {
|
||||
sqlDB.Close()
|
||||
}
|
||||
if err != nil {
|
||||
log.Fatalf("error creating postres db %v", err)
|
||||
}
|
||||
|
||||
_, err = gormstore.NewPostgresqlStoreForTests(context.TODO(), dsn, nil, false)
|
||||
if err != nil {
|
||||
log.Fatalf("error running migrations %v", err)
|
||||
}
|
||||
|
||||
ctx := context.TODO()
|
||||
pgstore, err := networkmap_pgsql.NewPostgresqlStore(ctx, dsn)
|
||||
if err != nil {
|
||||
log.Fatal("error creating postgres store %w", err)
|
||||
}
|
||||
|
||||
for _, query := range slices.Concat(strings.Split(baseData, ";"), strings.Split(pgData, ";")) {
|
||||
if _, err := pgstore.Pool.Exec(ctx, query); err != nil {
|
||||
log.Fatalf("error initializing db: %s", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
return pgstore, cleanup
|
||||
}
|
||||
|
||||
func createRandomDB(dsn string, db *gorm.DB) (string, func(), error) {
|
||||
dbName := fmt.Sprintf("test_db_%s", strings.ReplaceAll(uuid.New().String(), "-", "_"))
|
||||
|
||||
if err := db.Exec(fmt.Sprintf("CREATE DATABASE %s", dbName)).Error; err != nil {
|
||||
return "", nil, fmt.Errorf("failed to create database: %v", err)
|
||||
}
|
||||
|
||||
originalDSN := dsn
|
||||
|
||||
cleanup := func() {
|
||||
var dropDB *gorm.DB
|
||||
var err error
|
||||
|
||||
dropDB, err = gorm.Open(postgres.Open(originalDSN), &gorm.Config{
|
||||
SkipDefaultTransaction: true,
|
||||
PrepareStmt: false,
|
||||
})
|
||||
if err != nil {
|
||||
log.Errorf("failed to connect for dropping database %s: %v", dbName, err)
|
||||
return
|
||||
}
|
||||
defer func() {
|
||||
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
|
||||
sqlDB.Close()
|
||||
}
|
||||
}()
|
||||
|
||||
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
sqlDB.SetMaxIdleConns(0)
|
||||
sqlDB.SetConnMaxLifetime(time.Second)
|
||||
}
|
||||
|
||||
err = dropDB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s WITH (FORCE)", dbName)).Error
|
||||
|
||||
if err != nil {
|
||||
log.Errorf("failed to drop database %s: %v", dbName, err)
|
||||
}
|
||||
}
|
||||
|
||||
return replaceDBName(dsn, dbName), cleanup, nil
|
||||
}
|
||||
|
||||
func replaceDBName(dsn, newDBName string) string {
|
||||
re := regexp.MustCompile(`(?P<pre>[:/@])(?P<dbname>[^/?]+)(?P<post>\?|$)`)
|
||||
return re.ReplaceAllString(dsn, `${pre}`+newDBName+`${post}`)
|
||||
}
|
||||
@@ -1,146 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetPolicies(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
|
||||
values('policy-1','policy-1-public','account-1',true,'["posture-checks-1","posture-checks-2"]')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
|
||||
source_resource, destination_resource, ports, port_ranges,
|
||||
authorized_groups, authorized_user)
|
||||
values('policy-1-rule-1','policy-1',true,'accept','tcp',true,'["group-one-resource-id","group-two-resources-id"]','["group-one-resource-id","group-two-resources-id"]',
|
||||
'{"ID":"host-id-1","Type":"host"}','{"ID":"domain-1","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
|
||||
'{"group-one-resource-id":["user-1", "user-2"]}','user-3')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
|
||||
values('policy-2','policy-2-public','account-1',true,'["posture-checks-3","posture-checks-4"]')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
|
||||
source_resource, destination_resource, ports, port_ranges,
|
||||
authorized_groups, authorized_user)
|
||||
values('policy-2-rule-1','policy-2',true,'accept','tcp',true,'["group-one-resource-id"]','["group-two-resources-id"]',
|
||||
'{"ID":"host-id-3","Type":"host"}','{"ID":"domain-3","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
|
||||
'{"group-one-resource-id":["user-6", "user-7"]}','user-8')`)
|
||||
// policy with a rule with null fields
|
||||
execQuery(t, ctx,
|
||||
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
|
||||
values('policy-3','policy-3-public','account-1',true,null)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
|
||||
source_resource, destination_resource, ports, port_ranges,
|
||||
authorized_groups, authorized_user)
|
||||
values('policy-3-rule-1','policy-3',true,null,null,null,null,null,null,null,null,null,null,null)`)
|
||||
// policy with a disabled rule, destination resource and groups should not be in indexes
|
||||
execQuery(t, ctx,
|
||||
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
|
||||
values('policy-4','policy-4-public','account-1',true,null)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
|
||||
source_resource, destination_resource, ports, port_ranges,
|
||||
authorized_groups, authorized_user)
|
||||
values('policy-4-rule-1','policy-4',false,null,null,null,null,'["group-two-resources-id"]',
|
||||
null,'{"ID":"domain-3","Type":"domain"}',null,null,null,null)`)
|
||||
|
||||
policies, policyToDestinationResourceIdx, policyToDestinationGroupIdx, err := conn(t, ctx).GetPolicies(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
assert.Contains(t, policies, nmdata.Policy{
|
||||
ID: "policy-1",
|
||||
PublicID: "policy-1-public",
|
||||
Enabled: true,
|
||||
SourcePostureChecks: []string{"posture-checks-1", "posture-checks-2"},
|
||||
Rules: []*nmdata.PolicyRule{
|
||||
{
|
||||
ID: "policy-1",
|
||||
PolicyID: "policy-1",
|
||||
Enabled: true,
|
||||
Action: "accept",
|
||||
Protocol: "tcp",
|
||||
Bidirectional: true,
|
||||
Sources: []string{"group-one-resource-id", "group-two-resources-id"},
|
||||
Destinations: []string{"group-one-resource-id", "group-two-resources-id"},
|
||||
SourceResource: nmdata.Resource{ID: "host-id-1", Type: "host"},
|
||||
DestinationResource: nmdata.Resource{ID: "domain-1", Type: "domain"},
|
||||
Ports: []string{"8080", "8443"},
|
||||
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
|
||||
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-1", "user-2"}},
|
||||
AuthorizedUser: "user-3",
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
assert.Contains(t, policies, nmdata.Policy{
|
||||
ID: "policy-2",
|
||||
PublicID: "policy-2-public",
|
||||
Enabled: true,
|
||||
SourcePostureChecks: []string{"posture-checks-3", "posture-checks-4"},
|
||||
Rules: []*nmdata.PolicyRule{
|
||||
{
|
||||
ID: "policy-2",
|
||||
PolicyID: "policy-2",
|
||||
Enabled: true,
|
||||
Action: "accept",
|
||||
Protocol: "tcp",
|
||||
Bidirectional: true,
|
||||
Sources: []string{"group-one-resource-id"},
|
||||
Destinations: []string{"group-two-resources-id"},
|
||||
SourceResource: nmdata.Resource{ID: "host-id-3", Type: "host"},
|
||||
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
|
||||
Ports: []string{"8080", "8443"},
|
||||
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
|
||||
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-6", "user-7"}},
|
||||
AuthorizedUser: "user-8",
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
assert.Contains(t, policies, nmdata.Policy{
|
||||
ID: "policy-3",
|
||||
PublicID: "policy-3-public",
|
||||
Enabled: true,
|
||||
SourcePostureChecks: nil,
|
||||
Rules: []*nmdata.PolicyRule{
|
||||
{
|
||||
ID: "policy-3",
|
||||
PolicyID: "policy-3",
|
||||
Enabled: true,
|
||||
},
|
||||
},
|
||||
})
|
||||
assert.Contains(t, policies, nmdata.Policy{
|
||||
ID: "policy-4",
|
||||
PublicID: "policy-4-public",
|
||||
Enabled: true,
|
||||
SourcePostureChecks: nil,
|
||||
Rules: []*nmdata.PolicyRule{
|
||||
{
|
||||
ID: "policy-4",
|
||||
PolicyID: "policy-4",
|
||||
Enabled: false,
|
||||
Destinations: []string{"group-two-resources-id"},
|
||||
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
assert.Equal(t, policyToDestinationGroupIdx, map[string]map[string]any{
|
||||
"policy-1": {"group-one-resource-id": struct{}{}, "group-two-resources-id": struct{}{}},
|
||||
"policy-2": {"group-two-resources-id": struct{}{}},
|
||||
})
|
||||
assert.Equal(t, policyToDestinationResourceIdx, map[string]map[string]any{
|
||||
"policy-1": {"domain-1": struct{}{}},
|
||||
"policy-2": {"domain-3": struct{}{}},
|
||||
})
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetPostureChecks(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into posture_checks (id, account_id, public_id, checks)
|
||||
VALUES('posturecheck-1','account-1','posturecheck-1-public',
|
||||
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
|
||||
"OSVersionCheck":{"Darwin":{"MinVersion":"12.0"}},
|
||||
"GeoLocationCheck":{"Locations":[{"CountryCode":"FI","CityName":""}],"Action":"allow"},
|
||||
"PeerNetworkRangeCheck":{"Action":"deny","Ranges":["192.168.0.1/24"]}}')`)
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into posture_checks (id, account_id, public_id, checks)
|
||||
VALUES('posturecheck-2','account-1','posturecheck-2-public',
|
||||
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
|
||||
"OSVersionCheck":{"Android":{"MinVersion":"0"}},
|
||||
"GeoLocationCheck":{"Locations":[{"CountryCode":"US","CityName":"Harker Heights"}],"Action":"allow"},
|
||||
"PeerNetworkRangeCheck":{"Action":"allow","Ranges":["0.0.0.0/0"]}}')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into posture_checks (id, account_id, public_id, checks)
|
||||
VALUES('posturecheck-3','account-1','posturecheck-3-public', null)`)
|
||||
|
||||
postureChecks, idToPublicIDIdx, err := conn(t, ctx).GetPostureChecks(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, idToPublicIDIdx, map[string]string{
|
||||
"posturecheck-1": "posturecheck-1-public",
|
||||
"posturecheck-2": "posturecheck-2-public",
|
||||
"posturecheck-3": "posturecheck-3-public",
|
||||
})
|
||||
assert.Contains(t, postureChecks, nmdata.PostureChecks{
|
||||
ID: "posturecheck-1",
|
||||
Checks: nmdata.ChecksDefinition{
|
||||
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
|
||||
OSVersionCheck: &nmdata.OSVersionCheck{Darwin: &nmdata.MinVersionCheck{MinVersion: "12.0"}},
|
||||
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "FI"}}, Action: "allow"},
|
||||
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "deny", Ranges: []netip.Prefix{netip.MustParsePrefix("192.168.0.1/24")}},
|
||||
}})
|
||||
assert.Contains(t, postureChecks, nmdata.PostureChecks{
|
||||
ID: "posturecheck-2",
|
||||
Checks: nmdata.ChecksDefinition{
|
||||
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
|
||||
OSVersionCheck: &nmdata.OSVersionCheck{Android: &nmdata.MinVersionCheck{MinVersion: "0"}},
|
||||
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "US", CityName: "Harker Heights"}}, Action: "allow"},
|
||||
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "allow", Ranges: []netip.Prefix{netip.MustParsePrefix("0.0.0.0/0")}},
|
||||
}})
|
||||
assert.Contains(t, postureChecks, nmdata.PostureChecks{
|
||||
ID: "posturecheck-3"})
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/domain"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetRoutes(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
|
||||
peer, peer_groups, network_type, masquerade, metric, enabled,
|
||||
groups, access_control_groups, skip_auto_apply)
|
||||
VALUES('route-1','account-1','route-1-public','"172.0.0.0/16"','["test-1.com"]',true,'route-1-net-id','route-1',
|
||||
'peer-id-1','["group-one-resource-id"]',1,true,9999,true,
|
||||
'["group-one-resource-id"]','["group-one-resource-id"]',false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
|
||||
peer, peer_groups, network_type, masquerade, metric, enabled,
|
||||
groups, access_control_groups, skip_auto_apply)
|
||||
VALUES('route-2','account-1','route-2-public','"172.10.0.0/16"','["test-1.com","test-2.com"]',true,'route-2-net-id','route-2',
|
||||
'peer-id-2','["group-two-resources-id"]',1,true,9999,true,
|
||||
'["group-two-resources-id"]','["group-two-resources-id"]',false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
|
||||
peer, peer_groups, network_type, masquerade, metric, enabled,
|
||||
groups, access_control_groups, skip_auto_apply)
|
||||
VALUES('route-3','account-1','route-3-public',null,null,null,null,'route-3',
|
||||
null,null,null,null,null,null,null,null,null)`)
|
||||
|
||||
routes, err := conn(t, ctx).GetRoutes(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Contains(t, routes, nmdata.Route{
|
||||
ID: "route-1",
|
||||
AccountID: "account-1",
|
||||
PublicID: "route-1-public",
|
||||
Network: netip.MustParsePrefix("172.0.0.0/16"),
|
||||
Domains: domain.List{"test-1.com"},
|
||||
KeepRoute: true,
|
||||
NetID: "route-1-net-id",
|
||||
Description: "route-1",
|
||||
Peer: "peer-id-1",
|
||||
PeerID: "peer-id-1",
|
||||
PeerGroups: []string{"group-one-resource-id"},
|
||||
NetworkType: 1,
|
||||
Masquerade: true,
|
||||
Metric: 9999,
|
||||
Enabled: true,
|
||||
Groups: []string{"group-one-resource-id"},
|
||||
AccessControlGroups: []string{"group-one-resource-id"},
|
||||
SkipAutoApply: false,
|
||||
})
|
||||
assert.Contains(t, routes, nmdata.Route{
|
||||
ID: "route-2",
|
||||
AccountID: "account-1",
|
||||
PublicID: "route-2-public",
|
||||
Network: netip.MustParsePrefix("172.10.0.0/16"),
|
||||
Domains: domain.List{"test-1.com", "test-2.com"},
|
||||
KeepRoute: true,
|
||||
NetID: "route-2-net-id",
|
||||
Description: "route-2",
|
||||
Peer: "peer-id-2",
|
||||
PeerID: "peer-id-2",
|
||||
PeerGroups: []string{"group-two-resources-id"},
|
||||
NetworkType: 1,
|
||||
Masquerade: true,
|
||||
Metric: 9999,
|
||||
Enabled: true,
|
||||
Groups: []string{"group-two-resources-id"},
|
||||
AccessControlGroups: []string{"group-two-resources-id"},
|
||||
SkipAutoApply: false,
|
||||
})
|
||||
assert.Contains(t, routes, nmdata.Route{
|
||||
ID: "route-3",
|
||||
AccountID: "account-1",
|
||||
PublicID: "route-3-public",
|
||||
Description: "route-3",
|
||||
})
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
|
||||
)
|
||||
|
||||
func TestGetPrivateServices(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
|
||||
values('service-1','account-1',true,true,'["group-one-resource-id"]','test-1.com','test-2.com')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
|
||||
values('service-2','account-1',true,true,'["group-one-resource-id","group-two-resources-id"]','test-3.com','test-4.com')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
|
||||
values('service-3','account-1',null,null,null,null,null)`)
|
||||
|
||||
services, err := conn(t, ctx).GetPrivateServices(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Contains(t, services, networkmapdb.Service{
|
||||
Enabled: sql.NullBool{Bool: true, Valid: true},
|
||||
Private: sql.NullBool{Bool: true, Valid: true},
|
||||
AccessGroups: []string{"group-one-resource-id"},
|
||||
ProxyCluster: sql.NullString{String: "test-1.com", Valid: true},
|
||||
Domain: sql.NullString{String: "test-2.com", Valid: true},
|
||||
})
|
||||
assert.Contains(t, services, networkmapdb.Service{
|
||||
Enabled: sql.NullBool{Bool: true, Valid: true},
|
||||
Private: sql.NullBool{Bool: true, Valid: true},
|
||||
AccessGroups: []string{"group-one-resource-id", "group-two-resources-id"},
|
||||
ProxyCluster: sql.NullString{String: "test-3.com", Valid: true},
|
||||
Domain: sql.NullString{String: "test-4.com", Valid: true},
|
||||
})
|
||||
assert.Contains(t, services, networkmapdb.Service{
|
||||
Enabled: sql.NullBool{Bool: false, Valid: false},
|
||||
Private: sql.NullBool{Bool: false, Valid: false},
|
||||
AccessGroups: []string{},
|
||||
ProxyCluster: sql.NullString{String: "", Valid: false},
|
||||
Domain: sql.NullString{String: "", Valid: false},
|
||||
})
|
||||
}
|
||||
|
||||
func TestGetProxyTargetedDomainResourceIDs(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, terminated)
|
||||
values('service-4','account-1',true,false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-1','account-1','service-4',true,'domain')`)
|
||||
// id shouldn't be returned as the taget_type is not "domain"
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-2','account-1','service-4',true,'cluster')`)
|
||||
// id shouldn't be included as the target is disabled
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-3','account-1','service-4',false,'domain')`)
|
||||
// id shouldn't be included as the service is disabled
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, terminated)
|
||||
values('service-5','account-1',false,false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-4','account-1','service-5',false,'domain')`)
|
||||
// id shouldn't be included as the service is terminated (explicitly)
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, terminated)
|
||||
values('service-6','account-1',true,true)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-5','account-1','service-6',true,'domain')`)
|
||||
// id shouldn't be included as the service is terminated (implicitly)
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, terminated)
|
||||
values('service-7','account-1',true,null)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-6','account-1','service-7',true,'domain')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into services (id, account_id, enabled, terminated)
|
||||
values('service-8','account-1',true,false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values('target-7','account-1','service-8',true,'domain')`)
|
||||
// id shouldn't be returned as the taget_id is null
|
||||
execQuery(t, ctx,
|
||||
`insert into targets (target_id, account_id, service_id, enabled, target_type)
|
||||
values(null,'account-1','service-4',true,'cluster')`)
|
||||
|
||||
servtargetedDomains, err := conn(t, ctx).GetProxyTargetedDomainResourceIDs(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, servtargetedDomains, map[string]struct{}{
|
||||
"target-1": {},
|
||||
"target-6": {},
|
||||
"target-7": {},
|
||||
})
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-1','account-1','key-1','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-1.netbird.services',
|
||||
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
|
||||
'DE','Berlin','"46.201.148.187"');
|
||||
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-2','account-1','key-2','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-2.netbird.services',
|
||||
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
|
||||
'DE','Berlin','"46.201.149.187"');
|
||||
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
|
||||
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
|
||||
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
|
||||
meta_capabilities, meta_flags, meta_sync_message_version,
|
||||
location_country_code, location_city_name, location_connection_ip)
|
||||
values('peer-id-3','account-1','key-3','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
|
||||
false,true,true,'cluster-3.netbird.services',
|
||||
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
|
||||
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
|
||||
'DE','Berlin','"46.201.150.187"');
|
||||
@@ -1,49 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"runtime"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
|
||||
gormstore "github.com/netbirdio/netbird/management/server/store"
|
||||
"github.com/netbirdio/netbird/management/server/types"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func createSqliteTestStore(baseData, sqliteData string) (*networkmap_sqlite.SqliteStore, func()) {
|
||||
storeSqliteFileName := ":memory:"
|
||||
storeStr := fmt.Sprintf("%s?cache=shared", storeSqliteFileName)
|
||||
if runtime.GOOS == "windows" {
|
||||
// Vo avoid `The process cannot access the file because it is being used by another process` on Windows
|
||||
storeStr = storeSqliteFileName
|
||||
}
|
||||
|
||||
db, err := gorm.Open(sqlite.Open(storeStr), &gorm.Config{})
|
||||
if err != nil {
|
||||
log.Fatalf("error initializing db: %s", err.Error())
|
||||
}
|
||||
_, err = gormstore.NewSqlStore(context.TODO(), db, types.SqliteStoreEngine, nil, false)
|
||||
if err != nil {
|
||||
log.Fatalf("error initializing db: %s", err.Error())
|
||||
}
|
||||
|
||||
sqldb, err := db.DB()
|
||||
if err != nil {
|
||||
log.Fatalf("error initializing db: %s", err.Error())
|
||||
|
||||
}
|
||||
for _, query := range slices.Concat(strings.Split(baseData, ";"), strings.Split(sqliteData, ";")) {
|
||||
if _, err := sqldb.Exec(query); err != nil {
|
||||
log.Fatalf("error initializing db: %s", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
return &networkmap_sqlite.SqliteStore{Db: sqldb}, func() {}
|
||||
}
|
||||
@@ -1,57 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package networkmap_pgsql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGetAllowedUsers(t *testing.T) {
|
||||
ctx := context.TODO()
|
||||
|
||||
execQuery(t, ctx,
|
||||
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
|
||||
VALUES('user-1','user-1','account-1','["group-one-resource-id"]',false,false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
|
||||
VALUES('user-2','user-2','account-1','["group-one-resource-id","group-two-resources-id"]',false,false)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
|
||||
VALUES('user-3','user-3','account-1','["group-two-resources-id"]',false,false)`)
|
||||
// shouldn't be included as it's blocked
|
||||
execQuery(t, ctx,
|
||||
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
|
||||
VALUES('user-4','user-4','account-1','["group-two-resources-id"]',true,false)`)
|
||||
// shouldn't be included as it's a service_user
|
||||
execQuery(t, ctx,
|
||||
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
|
||||
VALUES('user-5','user-5','account-1','["group-two-resources-id"]',false,true)`)
|
||||
execQuery(t, ctx,
|
||||
`insert into groups (id, name, account_id)
|
||||
VALUES('all-group-1','All','account-1')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into groups (id, name, account_id)
|
||||
VALUES('all-group-2','All','account-1')`)
|
||||
execQuery(t, ctx,
|
||||
`insert into groups (id, name, account_id)
|
||||
VALUES('all-group-3','All','account-1')`)
|
||||
|
||||
userIdx, groupIdToUserIds, err := conn(t, ctx).GetAllowedUsers(ctx, "account-1")
|
||||
assert.NoError(t, err)
|
||||
|
||||
assert.Equal(t, userIdx, map[string]struct{}{
|
||||
"user-1": {},
|
||||
"user-2": {},
|
||||
"user-3": {},
|
||||
})
|
||||
assert.Equal(t, groupIdToUserIds, map[string][]string{
|
||||
"group-one-resource-id": {"user-1", "user-2"},
|
||||
"group-two-resources-id": {"user-2", "user-3"},
|
||||
"all-group-1": {"user-1", "user-2", "user-3"},
|
||||
"all-group-2": {"user-1", "user-2", "user-3"},
|
||||
"all-group-3": {"user-1", "user-2", "user-3"},
|
||||
})
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
//mage:multiline
|
||||
|
||||
// Set the general description you want to have displayed with mage -l here.
|
||||
package main
|
||||
|
||||
// mg contains helpful utility functions, like Deps
|
||||
|
||||
// Default target to run when none is specified
|
||||
// If not set, running mage will list available targets
|
||||
//var Default = Integrationtest.All
|
||||
@@ -1,74 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"github.com/magefile/mage/mg"
|
||||
"github.com/magefile/mage/sh"
|
||||
)
|
||||
|
||||
var defaultcli = []string{"test", "-tags=integration", "-timeout=20m"}
|
||||
|
||||
type Integrationtest mg.Namespace
|
||||
|
||||
func (i Integrationtest) All(gotestflags *string) error {
|
||||
var errs []error
|
||||
if err := i.Api(gotestflags); err != nil {
|
||||
errs = append(errs, err)
|
||||
}
|
||||
if err := i.NmapDb(gotestflags); err != nil {
|
||||
errs = append(errs, err)
|
||||
}
|
||||
if len(errs) > 0 {
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (Integrationtest) NmapDb(gotestflags *string) error {
|
||||
cli := defaultcli
|
||||
if gotestflags != nil {
|
||||
cli = append(cli, strings.Split(*gotestflags, " ")...)
|
||||
}
|
||||
cli = append(cli, "./integration_tests/management/network_map_db/...")
|
||||
|
||||
return sh.RunV("go", cli...)
|
||||
}
|
||||
|
||||
func (Integrationtest) NmapDbPostgres(gotestflags *string) error {
|
||||
cli := defaultcli
|
||||
if gotestflags != nil {
|
||||
cli = append(cli, strings.Split(*gotestflags, " ")...)
|
||||
}
|
||||
cli = append(cli, "./integration_tests/management/network_map_db/...")
|
||||
|
||||
return sh.RunWithV(map[string]string{"NETBIRD_STORE_ENGINE": "postgres"}, "go", cli...)
|
||||
}
|
||||
|
||||
func (Integrationtest) NmapDbSqlite(gotestflags *string) error {
|
||||
cli := defaultcli
|
||||
if gotestflags != nil {
|
||||
cli = append(cli, strings.Split(*gotestflags, " ")...)
|
||||
}
|
||||
cli = append(cli, "./integration_tests/management/network_map_db/...")
|
||||
return sh.RunWithV(map[string]string{"NETBIRD_STORE_ENGINE": "sqlite"}, "go", cli...)
|
||||
}
|
||||
|
||||
func (Integrationtest) RegenerateNmapGoldenData(gotestflags *string) error {
|
||||
cli := defaultcli
|
||||
if gotestflags != nil {
|
||||
cli = append(cli, strings.Split(*gotestflags, " ")...)
|
||||
}
|
||||
cli = append(cli, "./integration_tests/management/network_map_db/...")
|
||||
return sh.RunWithV(map[string]string{"NMAP_UPDATE_GOLDEN_DATA": "true", "NETBIRD_STORE_ENGINE": "sqlite"}, "go", cli...)
|
||||
}
|
||||
|
||||
func (Integrationtest) Api(gotestflags *string) error {
|
||||
cli := defaultcli
|
||||
if gotestflags != nil {
|
||||
cli = append(cli, strings.Split(*gotestflags, " ")...)
|
||||
}
|
||||
cli = append(cli, "./management/server/http/...")
|
||||
return sh.RunV("go", cli...)
|
||||
}
|
||||
@@ -18,7 +18,6 @@ import (
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map"
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller/cache"
|
||||
"github.com/netbirdio/netbird/management/internals/modules/peers/ephemeral"
|
||||
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
|
||||
"github.com/netbirdio/netbird/management/internals/server/config"
|
||||
"github.com/netbirdio/netbird/management/internals/shared/grpc"
|
||||
"github.com/netbirdio/netbird/management/server/account"
|
||||
@@ -31,8 +30,6 @@ import (
|
||||
"github.com/netbirdio/netbird/management/server/telemetry"
|
||||
"github.com/netbirdio/netbird/management/server/types"
|
||||
sharedgrpc "github.com/netbirdio/netbird/shared/management/grpc"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/netbirdio/netbird/shared/management/proto"
|
||||
"github.com/netbirdio/netbird/shared/management/status"
|
||||
"github.com/netbirdio/netbird/util"
|
||||
@@ -64,8 +61,6 @@ type Controller struct {
|
||||
serverSupportedSyncMessageVersion sharedgrpc.SyncMessageVersion
|
||||
|
||||
perAccountServerSupportedSyncMessageVersions map[string]sharedgrpc.SyncMessageVersion
|
||||
|
||||
nmdataStore *networkmapdb.NetworkMapDBStoreImpl
|
||||
}
|
||||
|
||||
type bufferUpdate struct {
|
||||
@@ -83,7 +78,7 @@ type bufferAffectedUpdate struct {
|
||||
|
||||
var _ network_map.Controller = (*Controller)(nil)
|
||||
|
||||
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config, nmdataStore *networkmapdb.NetworkMapDBStoreImpl) *Controller {
|
||||
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config) *Controller {
|
||||
nMetrics, err := newMetrics(metrics.UpdateChannelMetrics())
|
||||
if err != nil {
|
||||
log.Fatal(fmt.Errorf("error creating metrics: %w", err))
|
||||
@@ -104,7 +99,6 @@ func NewController(ctx context.Context, store store.Store, metrics telemetry.App
|
||||
EphemeralPeersManager: ephemeralPeersManager,
|
||||
serverSupportedSyncMessageVersion: sharedgrpc.SyncMessageVersionFromConfig(config.HighestSupportedSyncMessageVersion),
|
||||
perAccountServerSupportedSyncMessageVersions: sharedgrpc.SyncMessageVersionsFromMap(config.PerAccountHighestSupportedSyncMessageVersion),
|
||||
nmdataStore: nmdataStore,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -153,11 +147,6 @@ func (c *Controller) CountStreams() int {
|
||||
|
||||
func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID string, reason types.UpdateReason) error {
|
||||
log.WithContext(ctx).Tracef("updating peers for account %s from %s", accountID, util.GetCallerName())
|
||||
|
||||
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
|
||||
return c.sendUpdateAccountPeersFromData(ctx, accountID, reason, nmData)
|
||||
}
|
||||
|
||||
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get account: %v", err)
|
||||
@@ -178,7 +167,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
|
||||
return nil
|
||||
}
|
||||
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get validate peers: %v", err)
|
||||
}
|
||||
@@ -266,7 +255,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
|
||||
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
|
||||
// the client merges it into Calculate()'s output the same
|
||||
// way the legacy server did via NetworkMap.Merge.
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
@@ -287,7 +276,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
|
||||
}
|
||||
|
||||
start = time.Now()
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
c.metrics.CountToSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
@@ -305,261 +294,6 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
|
||||
return nil
|
||||
}
|
||||
|
||||
// sendUpdateAccountPeersFromData is the account-free variant of
|
||||
// sendUpdateAccountPeers: everything is computed from the network-map DB
|
||||
// store's twin data; only extra settings and validated peers are resolved at
|
||||
// runtime. Proxy network maps and policy injection, private-service zones,
|
||||
// group-to-user SSH mappings and forced routing-peer DNS resolution have no
|
||||
// DB-backed source yet and are omitted.
|
||||
func (c *Controller) sendUpdateAccountPeersFromData(ctx context.Context, accountID string, reason types.UpdateReason, nmData *networkmap.NetworkMapData) error {
|
||||
peersToUpdate := c.connectedPeersFromData(nmData, nil)
|
||||
if len(peersToUpdate) == 0 {
|
||||
return nil
|
||||
}
|
||||
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, &reason)
|
||||
}
|
||||
|
||||
// sendUpdateForAffectedPeersFromData is the account-free variant of
|
||||
// sendUpdateForAffectedPeers.
|
||||
func (c *Controller) sendUpdateForAffectedPeersFromData(ctx context.Context, accountID string, peerIDs []string, nmData *networkmap.NetworkMapData) error {
|
||||
if len(peerIDs) == 0 {
|
||||
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no affected peers")
|
||||
return nil
|
||||
}
|
||||
|
||||
peersToUpdate := c.connectedPeersFromData(nmData, peerIDs)
|
||||
if len(peersToUpdate) == 0 {
|
||||
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no peers to update (affected peers not found in data or no channels)")
|
||||
return nil
|
||||
}
|
||||
|
||||
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: sending network map to %d connected peers", len(peersToUpdate))
|
||||
|
||||
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, nil)
|
||||
}
|
||||
|
||||
// connectedPeersFromData returns the peers with an open update channel. An
|
||||
// empty affected list means all peers; a non-empty list restricts the result
|
||||
// to those peer IDs.
|
||||
func (c *Controller) connectedPeersFromData(nmData *networkmap.NetworkMapData, affected []string) []*nmdata.Peer {
|
||||
if len(affected) == 0 {
|
||||
result := make([]*nmdata.Peer, 0, len(nmData.Peers))
|
||||
for _, peer := range nmData.Peers {
|
||||
if c.peersUpdateManager.HasChannel(peer.ID) {
|
||||
result = append(result, peer)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
result := make([]*nmdata.Peer, 0, len(affected))
|
||||
for _, peerID := range affected {
|
||||
peer := nmData.Peers[peerID]
|
||||
if peer == nil {
|
||||
continue
|
||||
}
|
||||
if c.peersUpdateManager.HasChannel(peerID) {
|
||||
result = append(result, peer)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func (c *Controller) sendUpdatesFromData(ctx context.Context, accountID string, nmData *networkmap.NetworkMapData, peersToUpdate []*nmdata.Peer, reason *types.UpdateReason) error {
|
||||
globalStart := time.Now()
|
||||
|
||||
extraSettings, err := c.settingsManager.GetExtraSettings(ctx, accountID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get flow enabled status: %v", err)
|
||||
}
|
||||
|
||||
nmData.PrecomputePostureValidation()
|
||||
|
||||
dnsCache := &cache.DNSConfigCache{}
|
||||
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
|
||||
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
|
||||
|
||||
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
semaphore := make(chan struct{}, 10)
|
||||
|
||||
for _, peer := range peersToUpdate {
|
||||
if reason != nil && c.accountManagerMetrics != nil {
|
||||
c.accountManagerMetrics.CountNmapTriggered(string(reason.Resource), string(reason.Operation))
|
||||
}
|
||||
|
||||
wg.Add(1)
|
||||
semaphore <- struct{}{}
|
||||
go func(p *nmdata.Peer) {
|
||||
defer wg.Done()
|
||||
defer func() { <-semaphore }()
|
||||
|
||||
start := time.Now()
|
||||
|
||||
postureChecks := peerPostureChecksFromData(nmData, p.ID)
|
||||
|
||||
c.metrics.CountCalcPostureChecksDuration(time.Since(start))
|
||||
start = time.Now()
|
||||
|
||||
peerGroups := maps.Keys(nmData.GetPeerGroups(p.ID))
|
||||
var update *proto.SyncResponse
|
||||
|
||||
commonSyncMessageVersion := sharedgrpc.HighestCommonSyncMessageVersion(
|
||||
c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
|
||||
sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion))
|
||||
|
||||
log.WithContext(ctx).
|
||||
WithFields(log.Fields{
|
||||
"sync_message_version": commonSyncMessageVersion,
|
||||
"server_sync_message_version": c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
|
||||
"peer_sync_message_version": sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion),
|
||||
}).Debug("common highest sync message version")
|
||||
|
||||
if commonSyncMessageVersion == sharedgrpc.ComponentNetworkMap {
|
||||
components := nmData.GetPeerNetworkMapComponents(p.ID, peersCustomZone)
|
||||
|
||||
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
|
||||
|
||||
start = time.Now()
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, nil, dnsDomain, postureChecks, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
|
||||
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
Update: update,
|
||||
MessageType: network_map.MessageTypeNetworkMap,
|
||||
})
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
nmap := NetworkMapFromData(ctx, nmData, p.ID, peersCustomZone)
|
||||
|
||||
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
|
||||
|
||||
start = time.Now()
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
|
||||
c.metrics.CountToSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
Update: update,
|
||||
MessageType: network_map.MessageTypeNetworkMap,
|
||||
})
|
||||
}(peer)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
if c.accountManagerMetrics != nil {
|
||||
c.accountManagerMetrics.CountUpdateAccountPeersDuration(time.Since(globalStart))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Controller) getNetworkMapData(ctx context.Context, accountID string) *networkmap.NetworkMapData {
|
||||
if c.nmdataStore == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
nmData, err := c.nmdataStore.GetNetworkMapData(ctx, accountID)
|
||||
if err != nil {
|
||||
log.WithContext(ctx).Errorf("failed to get network map data for account %s, falling back to account-based computation: %v", accountID, err)
|
||||
return nil
|
||||
}
|
||||
|
||||
return nmData
|
||||
}
|
||||
|
||||
func (c *Controller) getDNSDomainFromData(settings *nmdata.AccountSettingsInfo) string {
|
||||
if settings == nil || settings.DNSDomain == "" {
|
||||
return c.dnsDomain
|
||||
}
|
||||
return settings.DNSDomain
|
||||
}
|
||||
|
||||
func IPv6AllowedPeersFromData(nmData *networkmap.NetworkMapData) map[string]struct{} {
|
||||
result := make(map[string]struct{})
|
||||
if nmData.AccountSettings != nil {
|
||||
for _, groupID := range nmData.AccountSettings.IPv6EnabledGroups {
|
||||
group := nmData.Groups[groupID]
|
||||
if group == nil {
|
||||
continue
|
||||
}
|
||||
for _, peerID := range group.Peers {
|
||||
result[peerID] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
for id, p := range nmData.Peers {
|
||||
if p != nil && p.ProxyMeta.Embedded {
|
||||
result[id] = struct{}{}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func NetworkMapFromData(ctx context.Context, nmData *networkmap.NetworkMapData, peerID string, peersCustomZone nmdata.CustomZone) *types.NetworkMap {
|
||||
components := nmData.GetPeerNetworkMapComponents(peerID, peersCustomZone)
|
||||
if components.IsEmpty() {
|
||||
return &types.NetworkMap{Network: components.Network}
|
||||
}
|
||||
return types.CalculateNetworkMapFromComponents(ctx, components)
|
||||
}
|
||||
|
||||
// peerPostureChecksFromData mirrors getPeerPostureChecks on the twin store. The
|
||||
// sync response only encodes process-check file paths, so only ProcessCheck is
|
||||
// converted back to the server posture type.
|
||||
func peerPostureChecksFromData(nmData *networkmap.NetworkMapData, peerID string) []*posture.Checks {
|
||||
if len(nmData.PostureChecks) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
peerPostureChecks := make(map[string]*posture.Checks)
|
||||
for _, policy := range nmData.Policies {
|
||||
if policy == nil || !policy.Enabled || len(policy.SourcePostureChecks) == 0 {
|
||||
continue
|
||||
}
|
||||
if !isPeerInPolicySourceGroupsFromData(nmData, peerID, policy) {
|
||||
continue
|
||||
}
|
||||
for _, checkID := range policy.SourcePostureChecks {
|
||||
twin := nmData.PostureChecks[checkID]
|
||||
if twin == nil {
|
||||
continue
|
||||
}
|
||||
peerPostureChecks[checkID] = postureChecksFromTwin(twin)
|
||||
}
|
||||
}
|
||||
|
||||
return maps.Values(peerPostureChecks)
|
||||
}
|
||||
|
||||
func isPeerInPolicySourceGroupsFromData(nmData *networkmap.NetworkMapData, peerID string, policy *nmdata.Policy) bool {
|
||||
for _, rule := range policy.Rules {
|
||||
if rule == nil || !rule.Enabled {
|
||||
continue
|
||||
}
|
||||
for _, groupID := range rule.Sources {
|
||||
if group := nmData.Groups[groupID]; group != nil && slices.Contains(group.Peers, peerID) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func postureChecksFromTwin(twin *nmdata.PostureChecks) *posture.Checks {
|
||||
checks := &posture.Checks{ID: twin.ID}
|
||||
if twin.Checks.ProcessCheck != nil {
|
||||
processes := make([]posture.Process, 0, len(twin.Checks.ProcessCheck.Processes))
|
||||
for _, p := range twin.Checks.ProcessCheck.Processes {
|
||||
processes = append(processes, posture.Process{LinuxPath: p.LinuxPath, MacPath: p.MacPath, WindowsPath: p.WindowsPath})
|
||||
}
|
||||
checks.Checks.ProcessCheck = &posture.ProcessCheck{Processes: processes}
|
||||
}
|
||||
return checks
|
||||
}
|
||||
|
||||
func (c *Controller) perAccountOrGlobalSupportedSyncMessageVersions(accountId string) sharedgrpc.SyncMessageVersion {
|
||||
if perAccount, ok := c.perAccountServerSupportedSyncMessageVersions[accountId]; ok {
|
||||
return perAccount
|
||||
@@ -592,10 +326,6 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
|
||||
return nil
|
||||
}
|
||||
|
||||
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
|
||||
return c.sendUpdateForAffectedPeersFromData(ctx, accountID, peerIDs, nmData)
|
||||
}
|
||||
|
||||
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get account: %v", err)
|
||||
@@ -611,7 +341,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
|
||||
|
||||
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeers: sending network map to %d connected peers", len(peersToUpdate))
|
||||
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get validate peers: %v", err)
|
||||
}
|
||||
@@ -698,7 +428,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
|
||||
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
|
||||
// the client merges it into Calculate()'s output the same
|
||||
// way the legacy server did via NetworkMap.Merge.
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
@@ -719,7 +449,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
|
||||
}
|
||||
|
||||
start = time.Now()
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
|
||||
c.metrics.CountToSyncResponseDuration(time.Since(start))
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
|
||||
@@ -776,7 +506,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
|
||||
return fmt.Errorf("peer %s doesn't exists in account %s", peerId, accountId)
|
||||
}
|
||||
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get validated peers: %v", err)
|
||||
}
|
||||
@@ -836,7 +566,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
|
||||
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
|
||||
// the client merges it into Calculate()'s output the same
|
||||
// way the legacy server did via NetworkMap.Merge.
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
|
||||
Update: update,
|
||||
@@ -853,7 +583,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
|
||||
nmap.Merge(proxyNetworkMap)
|
||||
}
|
||||
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
|
||||
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
|
||||
|
||||
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
|
||||
Update: update,
|
||||
@@ -913,11 +643,7 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
|
||||
if err != nil {
|
||||
return nil, nil, nil, nil, 0, err
|
||||
}
|
||||
return peer, &types.NetworkMapComponents{Network: types.TwinNetwork(network)}, nil, nil, 0, nil
|
||||
}
|
||||
|
||||
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
|
||||
return c.getValidatedPeerWithComponentsFromData(ctx, accountID, peer, nmData)
|
||||
return peer, &types.NetworkMapComponents{Network: network.Copy()}, nil, nil, 0, nil
|
||||
}
|
||||
|
||||
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
|
||||
@@ -927,7 +653,7 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
|
||||
|
||||
c.injectAllProxyPolicies(ctx, account)
|
||||
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return nil, nil, nil, nil, 0, err
|
||||
}
|
||||
@@ -964,21 +690,6 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
|
||||
return peer, components, proxyNetworkMaps[peer.ID], postureChecks, dnsFwdPort, nil
|
||||
}
|
||||
|
||||
// getValidatedPeerWithComponentsFromData is the account-free variant of
|
||||
// GetValidatedPeerWithComponents. The proxy network map fragment is omitted
|
||||
// like on the other nmdata paths.
|
||||
func (c *Controller) getValidatedPeerWithComponentsFromData(ctx context.Context, accountID string, peer *nbpeer.Peer, nmData *networkmap.NetworkMapData) (*nbpeer.Peer, *types.NetworkMapComponents, *types.NetworkMap, []*posture.Checks, int64, error) {
|
||||
postureChecks := peerPostureChecksFromData(nmData, peer.ID)
|
||||
|
||||
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
|
||||
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
|
||||
|
||||
components := nmData.GetPeerNetworkMapComponents(peer.ID, peersCustomZone)
|
||||
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
|
||||
|
||||
return peer, components, nil, postureChecks, dnsFwdPort, nil
|
||||
}
|
||||
|
||||
// BufferUpdateAffectedPeers accumulates peer IDs and flushes them after the buffer interval.
|
||||
func (c *Controller) BufferUpdateAffectedPeers(ctx context.Context, accountID string, peerIDs []string, reason types.UpdateReason) error {
|
||||
if len(peerIDs) == 0 {
|
||||
@@ -1085,15 +796,11 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
|
||||
}
|
||||
|
||||
emptyMap := &types.NetworkMap{
|
||||
Network: types.TwinNetwork(network),
|
||||
Network: network.Copy(),
|
||||
}
|
||||
return emptyMap, nil, 0, nil
|
||||
}
|
||||
|
||||
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
|
||||
return c.getValidatedPeerWithMapFromData(ctx, accountID, peerID, nmData)
|
||||
}
|
||||
|
||||
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
|
||||
if err != nil {
|
||||
return nil, nil, 0, err
|
||||
@@ -1101,7 +808,7 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
|
||||
|
||||
c.injectAllProxyPolicies(ctx, account)
|
||||
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return nil, nil, 0, err
|
||||
}
|
||||
@@ -1141,21 +848,6 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
|
||||
return networkMap, postureChecks, dnsFwdPort, nil
|
||||
}
|
||||
|
||||
// getValidatedPeerWithMapFromData is the account-free variant of
|
||||
// GetValidatedPeerWithMap. The proxy network map fragment is omitted like on
|
||||
// the other nmdata paths.
|
||||
func (c *Controller) getValidatedPeerWithMapFromData(ctx context.Context, accountID string, peerID string, nmData *networkmap.NetworkMapData) (*types.NetworkMap, []*posture.Checks, int64, error) {
|
||||
postureChecks := peerPostureChecksFromData(nmData, peerID)
|
||||
|
||||
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
|
||||
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
|
||||
|
||||
networkMap := NetworkMapFromData(ctx, nmData, peerID, peersCustomZone)
|
||||
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
|
||||
|
||||
return networkMap, postureChecks, dnsFwdPort, nil
|
||||
}
|
||||
|
||||
// GetDNSDomain returns the configured dnsDomain
|
||||
func (c *Controller) GetDNSDomain(settings *types.Settings) string {
|
||||
if settings == nil {
|
||||
@@ -1218,36 +910,20 @@ func (c *Controller) StartWarmup(ctx context.Context) {
|
||||
// computeForwarderPort checks if all peers in the account have updated to a specific version or newer.
|
||||
// If all peers have the required version, it returns the new well-known port (22054), otherwise returns 0.
|
||||
func computeForwarderPort(peers []*nbpeer.Peer, requiredVersion string) int64 {
|
||||
versions := make([]string, 0, len(peers))
|
||||
for _, peer := range peers {
|
||||
versions = append(versions, peer.Meta.WtVersion)
|
||||
}
|
||||
return computeForwarderPortFromVersions(versions, requiredVersion)
|
||||
}
|
||||
|
||||
func ComputeForwarderPortFromData(peers map[string]*nmdata.Peer, requiredVersion string) int64 {
|
||||
versions := make([]string, 0, len(peers))
|
||||
for _, peer := range peers {
|
||||
versions = append(versions, peer.Meta.WtVersion)
|
||||
}
|
||||
return computeForwarderPortFromVersions(versions, requiredVersion)
|
||||
}
|
||||
|
||||
func computeForwarderPortFromVersions(wtVersions []string, requiredVersion string) int64 {
|
||||
if len(wtVersions) == 0 {
|
||||
if len(peers) == 0 {
|
||||
return int64(network_map.OldForwarderPort)
|
||||
}
|
||||
|
||||
reqVer := semver.Canonical(requiredVersion)
|
||||
|
||||
// Check if all peers have the required version or newer
|
||||
for _, wtVersion := range wtVersions {
|
||||
for _, peer := range peers {
|
||||
|
||||
// Development version is always supported
|
||||
if version.IsDevelopmentVersion(wtVersion) {
|
||||
if version.IsDevelopmentVersion(peer.Meta.WtVersion) {
|
||||
continue
|
||||
}
|
||||
peerVersion := semver.Canonical("v" + wtVersion)
|
||||
peerVersion := semver.Canonical("v" + peer.Meta.WtVersion)
|
||||
if peerVersion == "" {
|
||||
// If any peer doesn't have version info, return 0
|
||||
return int64(network_map.OldForwarderPort)
|
||||
@@ -1381,7 +1057,7 @@ func (c *Controller) GetNetworkMap(ctx context.Context, peerID string) (*types.N
|
||||
groups[groupID] = group.Peers
|
||||
}
|
||||
|
||||
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
|
||||
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -1,380 +0,0 @@
|
||||
package nmaptest
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"cmp"
|
||||
"fmt"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/proto"
|
||||
)
|
||||
|
||||
// normalizeIDSpace replaces policy and route identifiers with positional
|
||||
// placeholders so a comparison can reach everything else.
|
||||
//
|
||||
// This exists only because the envelope round-trip currently substitutes each
|
||||
// internal xid with the object's public id, which is a tracked defect and not a
|
||||
// licence to differ: those identifiers reach the server again inside flow
|
||||
// events, which resolve them by internal id, so the substitution silently
|
||||
// breaks flow attribution for component-format peers. TestIDSpaceMatches
|
||||
// asserts the equality that must eventually hold; this erasure keeps the other
|
||||
// 40-odd cases reporting on semantics meanwhile. When the id space is unified,
|
||||
// delete this and the calls to it — every case should still pass.
|
||||
//
|
||||
// Cardinality and cross-references survive the erasure: two rules under one
|
||||
// policy still share a token and a route firewall rule still points at its
|
||||
// route, so a path that drops a policy, merges two policies, or misattributes a
|
||||
// rule to the wrong route still fails.
|
||||
func normalizeIDSpace(nm *proto.NetworkMap) {
|
||||
if nm == nil {
|
||||
return
|
||||
}
|
||||
policies := newTokenizer("policy")
|
||||
routes := newTokenizer("route")
|
||||
|
||||
for _, i := range orderBy(nm.Routes, routeKeyWithoutID) {
|
||||
nm.Routes[i].ID = routes.get(nm.Routes[i].ID)
|
||||
}
|
||||
for _, i := range orderBy(nm.FirewallRules, firewallKeyWithoutPolicy) {
|
||||
r := nm.FirewallRules[i]
|
||||
if len(r.PolicyID) > 0 {
|
||||
r.PolicyID = []byte(policies.get(string(r.PolicyID)))
|
||||
}
|
||||
}
|
||||
for _, i := range orderBy(nm.RoutesFirewallRules, routeFirewallKeyWithoutIDs) {
|
||||
r := nm.RoutesFirewallRules[i]
|
||||
if len(r.PolicyID) > 0 {
|
||||
r.PolicyID = []byte(policies.get(string(r.PolicyID)))
|
||||
}
|
||||
r.RouteID = routes.get(r.RouteID)
|
||||
}
|
||||
}
|
||||
|
||||
// tokenizer maps identifiers to positional placeholders in order of first use.
|
||||
type tokenizer struct {
|
||||
prefix string
|
||||
seen map[string]string
|
||||
}
|
||||
|
||||
func newTokenizer(prefix string) *tokenizer {
|
||||
return &tokenizer{prefix: prefix, seen: make(map[string]string)}
|
||||
}
|
||||
|
||||
func (t *tokenizer) get(id string) string {
|
||||
if id == "" {
|
||||
return ""
|
||||
}
|
||||
if tok, ok := t.seen[id]; ok {
|
||||
return tok
|
||||
}
|
||||
tok := fmt.Sprintf("%s#%d", t.prefix, len(t.seen))
|
||||
t.seen[id] = tok
|
||||
return tok
|
||||
}
|
||||
|
||||
// orderBy returns indices sorted by key, so placeholder numbering does not
|
||||
// depend on the identifiers being erased.
|
||||
func orderBy[T any](items []T, key func(T) string) []int {
|
||||
idx := make([]int, len(items))
|
||||
for i := range idx {
|
||||
idx[i] = i
|
||||
}
|
||||
sort.SliceStable(idx, func(a, b int) bool { return key(items[idx[a]]) < key(items[idx[b]]) })
|
||||
return idx
|
||||
}
|
||||
|
||||
func routeKeyWithoutID(r *proto.Route) string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("%s|%s|%s|%d|%d|%t|%t|%v",
|
||||
r.Network, r.NetID, r.Peer, r.Metric, r.NetworkType, r.Masquerade, r.KeepRoute, r.Domains)
|
||||
}
|
||||
|
||||
func firewallKeyWithoutPolicy(r *proto.FirewallRule) string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("%s|%d|%d|%d|%s|%s|%v",
|
||||
r.PeerIP, r.Direction, r.Action, r.Protocol, r.Port, portInfoKey(r.PortInfo), r.SourcePrefixes) //nolint:staticcheck
|
||||
}
|
||||
|
||||
func routeFirewallKeyWithoutIDs(r *proto.RouteFirewallRule) string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("%s|%d|%d|%s|%v|%v|%t|%d",
|
||||
r.Destination, r.Protocol, r.Action, portInfoKey(r.PortInfo), r.Domains, r.SourceRanges, r.IsDynamic, r.CustomProtocol)
|
||||
}
|
||||
|
||||
// canonicalize sorts every repeated field of the NetworkMap by a stable key.
|
||||
// The producing paths iterate Go maps while building these slices, so order
|
||||
// can differ between runs even when the content is identical; comparing
|
||||
// without this reports noise.
|
||||
func canonicalize(nm *proto.NetworkMap) {
|
||||
if nm == nil {
|
||||
return
|
||||
}
|
||||
slices.SortFunc(nm.RemotePeers, cmpRemotePeer)
|
||||
slices.SortFunc(nm.OfflinePeers, cmpRemotePeer)
|
||||
slices.SortFunc(nm.Routes, cmpRoute)
|
||||
slices.SortFunc(nm.FirewallRules, cmpFirewallRule)
|
||||
slices.SortFunc(nm.RoutesFirewallRules, cmpRouteFirewallRule)
|
||||
slices.SortFunc(nm.ForwardingRules, cmpForwardingRule)
|
||||
|
||||
for _, r := range nm.FirewallRules {
|
||||
slices.SortFunc(r.SourcePrefixes, bytes.Compare)
|
||||
}
|
||||
for _, r := range nm.RoutesFirewallRules {
|
||||
slices.Sort(r.SourceRanges)
|
||||
}
|
||||
canonicalizeDNSConfig(nm.DNSConfig)
|
||||
canonicalizeSSHAuth(nm.SshAuth)
|
||||
}
|
||||
|
||||
func canonicalizeDNSConfig(d *proto.DNSConfig) {
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
for _, g := range d.NameServerGroups {
|
||||
if g == nil {
|
||||
continue
|
||||
}
|
||||
slices.Sort(g.Domains)
|
||||
slices.SortFunc(g.NameServers, func(a, b *proto.NameServer) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := cmp.Compare(a.IP, b.IP); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Port, b.Port); c != 0 {
|
||||
return c
|
||||
}
|
||||
return cmp.Compare(a.NSType, b.NSType)
|
||||
})
|
||||
}
|
||||
slices.SortFunc(d.NameServerGroups, func(a, b *proto.NameServerGroup) int {
|
||||
return cmp.Compare(nsgKey(a), nsgKey(b))
|
||||
})
|
||||
for _, z := range d.CustomZones {
|
||||
if z == nil {
|
||||
continue
|
||||
}
|
||||
slices.SortFunc(z.Records, cmpSimpleRecord)
|
||||
}
|
||||
slices.SortFunc(d.CustomZones, func(a, b *proto.CustomZone) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
return cmp.Compare(a.Domain, b.Domain)
|
||||
})
|
||||
}
|
||||
|
||||
// canonicalizeSSHAuth sorts AuthorizedUsers and re-keys MachineUsers.Indexes
|
||||
// against the new ordering, preserving which machine user maps to which hashes.
|
||||
func canonicalizeSSHAuth(s *proto.SSHAuth) {
|
||||
if s == nil || len(s.AuthorizedUsers) == 0 {
|
||||
return
|
||||
}
|
||||
type hashed struct {
|
||||
bytes []byte
|
||||
old uint32
|
||||
}
|
||||
entries := make([]hashed, len(s.AuthorizedUsers))
|
||||
for i, b := range s.AuthorizedUsers {
|
||||
entries[i] = hashed{bytes: b, old: uint32(i)}
|
||||
}
|
||||
slices.SortFunc(entries, func(a, b hashed) int { return bytes.Compare(a.bytes, b.bytes) })
|
||||
|
||||
remap := make(map[uint32]uint32, len(entries))
|
||||
sorted := make([][]byte, len(entries))
|
||||
for newIdx, e := range entries {
|
||||
remap[e.old] = uint32(newIdx)
|
||||
sorted[newIdx] = e.bytes
|
||||
}
|
||||
s.AuthorizedUsers = sorted
|
||||
|
||||
for _, mu := range s.MachineUsers {
|
||||
if mu == nil {
|
||||
continue
|
||||
}
|
||||
for i, oldIdx := range mu.Indexes {
|
||||
if newIdx, ok := remap[oldIdx]; ok {
|
||||
mu.Indexes[i] = newIdx
|
||||
}
|
||||
}
|
||||
slices.Sort(mu.Indexes)
|
||||
}
|
||||
}
|
||||
|
||||
func boolCmp(a, b bool) int {
|
||||
if a == b {
|
||||
return 0
|
||||
}
|
||||
if a {
|
||||
return 1
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func nsgKey(g *proto.NameServerGroup) string {
|
||||
if g == nil {
|
||||
return ""
|
||||
}
|
||||
var parts []string
|
||||
for _, ns := range g.NameServers {
|
||||
if ns == nil {
|
||||
continue
|
||||
}
|
||||
parts = append(parts, ns.IP+":"+strconv.FormatInt(ns.Port, 10)+":"+strconv.FormatInt(ns.NSType, 10))
|
||||
}
|
||||
slices.Sort(parts)
|
||||
key := strings.Join(parts, ",")
|
||||
domains := append([]string(nil), g.Domains...)
|
||||
slices.Sort(domains)
|
||||
key += "|" + strings.Join(domains, "|")
|
||||
if g.Primary {
|
||||
key += "|P"
|
||||
}
|
||||
if g.SearchDomainsEnabled {
|
||||
key += "|S"
|
||||
}
|
||||
return key
|
||||
}
|
||||
|
||||
func cmpSimpleRecord(a, b *proto.SimpleRecord) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := cmp.Compare(a.Name, b.Name); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Type, b.Type); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Class, b.Class); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.RData, b.RData); c != 0 {
|
||||
return c
|
||||
}
|
||||
return cmp.Compare(a.TTL, b.TTL)
|
||||
}
|
||||
|
||||
func cmpRemotePeer(a, b *proto.RemotePeerConfig) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
return cmp.Compare(a.WgPubKey, b.WgPubKey)
|
||||
}
|
||||
|
||||
func cmpRoute(a, b *proto.Route) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := cmp.Compare(a.ID, b.ID); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.NetID, b.NetID); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Network, b.Network); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Peer, b.Peer); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Metric, b.Metric); c != 0 {
|
||||
return c
|
||||
}
|
||||
return slices.Compare(a.Domains, b.Domains)
|
||||
}
|
||||
|
||||
func cmpFirewallRule(a, b *proto.FirewallRule) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := bytes.Compare(a.PolicyID, b.PolicyID); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.PeerIP, b.PeerIP); c != 0 { //nolint:staticcheck
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Direction), int32(b.Direction)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Action), int32(b.Action)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Port, b.Port); c != 0 {
|
||||
return c
|
||||
}
|
||||
return cmp.Compare(portInfoKey(a.PortInfo), portInfoKey(b.PortInfo))
|
||||
}
|
||||
|
||||
func cmpRouteFirewallRule(a, b *proto.RouteFirewallRule) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := bytes.Compare(a.PolicyID, b.PolicyID); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.RouteID, b.RouteID); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.Destination, b.Destination); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(portInfoKey(a.PortInfo), portInfoKey(b.PortInfo)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Action), int32(b.Action)); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := slices.Compare(a.Domains, b.Domains); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := slices.Compare(a.SourceRanges, b.SourceRanges); c != 0 {
|
||||
return c
|
||||
}
|
||||
if c := cmp.Compare(a.CustomProtocol, b.CustomProtocol); c != 0 {
|
||||
return c
|
||||
}
|
||||
return boolCmp(a.IsDynamic, b.IsDynamic)
|
||||
}
|
||||
|
||||
func cmpForwardingRule(a, b *proto.ForwardingRule) int {
|
||||
if a == nil || b == nil {
|
||||
return boolCmp(a == nil, b == nil)
|
||||
}
|
||||
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
|
||||
return c
|
||||
}
|
||||
return bytes.Compare(a.TranslatedAddress, b.TranslatedAddress)
|
||||
}
|
||||
|
||||
func portInfoKey(pi *proto.PortInfo) string {
|
||||
if pi == nil {
|
||||
return ""
|
||||
}
|
||||
switch sel := pi.PortSelection.(type) {
|
||||
case *proto.PortInfo_Port:
|
||||
return "P" + strconv.FormatUint(uint64(sel.Port), 10)
|
||||
case *proto.PortInfo_Range_:
|
||||
if sel.Range == nil {
|
||||
return "R"
|
||||
}
|
||||
return "R" + strconv.FormatUint(uint64(sel.Range.Start), 10) + "-" + strconv.FormatUint(uint64(sel.Range.End), 10)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -1,218 +0,0 @@
|
||||
package nmaptest
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net"
|
||||
"os"
|
||||
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
)
|
||||
|
||||
// LoadNetworkMapData reads a fixture holding the NetworkMapData the store
|
||||
// would return for one account. Unknown fields are rejected so fixture typos
|
||||
// fail loudly instead of silently testing a default.
|
||||
func LoadNetworkMapData(path string) (*networkmap.NetworkMapData, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open fixture: %w", err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
dec := json.NewDecoder(f)
|
||||
dec.DisallowUnknownFields()
|
||||
var nmData networkmap.NetworkMapData
|
||||
if err := dec.Decode(&nmData); err != nil {
|
||||
return nil, fmt.Errorf("decode fixture %s: %w", path, err)
|
||||
}
|
||||
return &nmData, nil
|
||||
}
|
||||
|
||||
var defaultNetworkNet = func() net.IPNet {
|
||||
_, ipnet, err := net.ParseCIDR("100.64.0.0/10")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return *ipnet
|
||||
}()
|
||||
|
||||
// applyFixtureDefaults fills the boilerplate a fixture may omit. Map-keyed
|
||||
// objects inherit their key as ID, peers get a deterministic WG-shaped key
|
||||
// and their ID as DNS label, PublicIDs default to the internal ID (the
|
||||
// envelope encoder puts public IDs on the wire and silently degrades on
|
||||
// empty ones), and a nil ValidatedPeers validates every peer — production
|
||||
// fills it through the integrated validator, not the store.
|
||||
func applyFixtureDefaults(nmData *networkmap.NetworkMapData) {
|
||||
if nmData.Network == nil {
|
||||
nmData.Network = &nmdata.Network{}
|
||||
}
|
||||
if nmData.Network.Identifier == "" {
|
||||
nmData.Network.Identifier = "network"
|
||||
}
|
||||
if nmData.Network.Net.IP == nil {
|
||||
nmData.Network.Net = defaultNetworkNet
|
||||
}
|
||||
if nmData.AccountSettings == nil {
|
||||
nmData.AccountSettings = &nmdata.AccountSettingsInfo{}
|
||||
}
|
||||
if nmData.DNSSettings == nil {
|
||||
nmData.DNSSettings = &nmdata.DNSSettings{}
|
||||
}
|
||||
|
||||
for id, p := range nmData.Peers {
|
||||
if p == nil {
|
||||
continue
|
||||
}
|
||||
if p.ID == "" {
|
||||
p.ID = id
|
||||
}
|
||||
if p.Key == "" {
|
||||
p.Key = derivedWgKey(p.ID)
|
||||
}
|
||||
if p.DNSLabel == "" {
|
||||
p.DNSLabel = p.ID
|
||||
}
|
||||
}
|
||||
|
||||
for id, g := range nmData.Groups {
|
||||
if g == nil {
|
||||
continue
|
||||
}
|
||||
if g.ID == "" {
|
||||
g.ID = id
|
||||
}
|
||||
if g.Name == "" {
|
||||
g.Name = g.ID
|
||||
}
|
||||
if g.PublicID == "" {
|
||||
g.PublicID = g.ID
|
||||
}
|
||||
}
|
||||
|
||||
for _, policy := range nmData.Policies {
|
||||
defaultPolicyIDs(policy)
|
||||
}
|
||||
resolveResourcePolicyRefs(nmData)
|
||||
|
||||
for _, r := range nmData.Routes {
|
||||
if r != nil && r.PublicID == "" {
|
||||
r.PublicID = r.ID
|
||||
}
|
||||
}
|
||||
for _, nsg := range nmData.NameServerGroups {
|
||||
if nsg != nil && nsg.PublicID == "" {
|
||||
nsg.PublicID = nsg.ID
|
||||
}
|
||||
}
|
||||
for _, res := range nmData.NetworkResources {
|
||||
if res == nil {
|
||||
continue
|
||||
}
|
||||
if res.PublicID == "" {
|
||||
res.PublicID = res.ID
|
||||
}
|
||||
defaultXIDMapping(&nmData.NetworkXIDToPublicID, res.NetworkID)
|
||||
}
|
||||
for networkID, routers := range nmData.Routers {
|
||||
defaultXIDMapping(&nmData.NetworkXIDToPublicID, networkID)
|
||||
for _, router := range routers {
|
||||
if router != nil && router.PublicID == "" {
|
||||
router.PublicID = networkID
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for id, pc := range nmData.PostureChecks {
|
||||
if pc == nil {
|
||||
continue
|
||||
}
|
||||
if pc.ID == "" {
|
||||
pc.ID = id
|
||||
}
|
||||
defaultXIDMapping(&nmData.PostureCheckXIDToPublicID, pc.ID)
|
||||
}
|
||||
|
||||
if nmData.ValidatedPeers == nil {
|
||||
nmData.ValidatedPeers = make(map[string]struct{}, len(nmData.Peers))
|
||||
for id := range nmData.Peers {
|
||||
nmData.ValidatedPeers[id] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// resolveResourcePolicyRefs lets a fixture name an account policy by ID in
|
||||
// ResourcePolicies — {"ID": "pol-x"} with no rules — instead of repeating it.
|
||||
// The real store puts the same policy pointer in both places, which is what
|
||||
// resolving the reference reproduces.
|
||||
func resolveResourcePolicyRefs(nmData *networkmap.NetworkMapData) {
|
||||
byID := make(map[string]*nmdata.Policy, len(nmData.Policies))
|
||||
for _, policy := range nmData.Policies {
|
||||
if policy != nil && policy.ID != "" {
|
||||
byID[policy.ID] = policy
|
||||
}
|
||||
}
|
||||
|
||||
for _, policies := range nmData.ResourcePolicies {
|
||||
for i, policy := range policies {
|
||||
if policy == nil {
|
||||
continue
|
||||
}
|
||||
if len(policy.Rules) == 0 {
|
||||
if full, ok := byID[policy.ID]; ok {
|
||||
policies[i] = full
|
||||
continue
|
||||
}
|
||||
}
|
||||
defaultPolicyIDs(policy)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func defaultPolicyIDs(policy *nmdata.Policy) {
|
||||
if policy == nil {
|
||||
return
|
||||
}
|
||||
if policy.PublicID == "" {
|
||||
policy.PublicID = policy.ID
|
||||
}
|
||||
for i, rule := range policy.Rules {
|
||||
if rule == nil {
|
||||
continue
|
||||
}
|
||||
if rule.PolicyID == "" {
|
||||
rule.PolicyID = policy.ID
|
||||
}
|
||||
if rule.ID == "" {
|
||||
// Production gives a rule its policy's id (management/server/policy.go:205,
|
||||
// "when policy can contain multiple rules, need refactor"), so a
|
||||
// single-rule policy — the only shape the product can create today —
|
||||
// must be modelled that way or the wire ids come out unrealistic.
|
||||
rule.ID = policy.ID
|
||||
if len(policy.Rules) > 1 {
|
||||
rule.ID = fmt.Sprintf("%s-rule-%d", policy.ID, i)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func defaultXIDMapping(m *map[string]string, id string) {
|
||||
if id == "" {
|
||||
return
|
||||
}
|
||||
if *m == nil {
|
||||
*m = make(map[string]string)
|
||||
}
|
||||
if _, ok := (*m)[id]; !ok {
|
||||
(*m)[id] = id
|
||||
}
|
||||
}
|
||||
|
||||
// derivedWgKey returns a deterministic base64 key of 32 bytes, valid for the
|
||||
// envelope decoder's WG-key identity.
|
||||
func derivedWgKey(peerID string) string {
|
||||
sum := sha256.Sum256([]byte(peerID))
|
||||
return base64.StdEncoding.EncodeToString(sum[:])
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
package nmaptest_test
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map/nmaptest"
|
||||
)
|
||||
|
||||
func TestNetworkMapGolden(t *testing.T) {
|
||||
nmaptest.RunGoldenDir(t, filepath.Join("testdata", "cases"))
|
||||
}
|
||||
@@ -1,311 +0,0 @@
|
||||
// Package nmaptest measures network map generation on the dedicated store
|
||||
// path against committed expectations. A case stands in for the store load
|
||||
// with a NetworkMapData fixture — the value NetworkMapDBStoreImpl returns for
|
||||
// one account — then runs the production per-peer pipeline the controller
|
||||
// uses, PeersCustomZone → GetPeerNetworkMapComponents → proto conversion, in
|
||||
// both wire shapes: the legacy full map (grpc.ToSyncResponse) and the
|
||||
// component envelope expanded client-side (grpc.ToComponentSyncResponse →
|
||||
// networkmap.EnvelopeToNetworkMap).
|
||||
//
|
||||
// The expectation files are the point of the framework. They state what the
|
||||
// output should be, so a failing case means the code disagrees with the
|
||||
// expectation and the answer is normally to fix the code; an expectation
|
||||
// changes only through a deliberate reviewed edit. Nothing in this package
|
||||
// writes to testdata — there is no flag that records current behaviour into an
|
||||
// expectation, because that is how a defect becomes the baseline. Cases whose
|
||||
// expectation encodes correct behaviour the code does not yet deliver stay red
|
||||
// on purpose.
|
||||
//
|
||||
// A case lives in testdata/cases/<name>/ as case.json (manifest: description,
|
||||
// peers, optional accountID, dnsDomain, modes), nmdata.json (the fixture the
|
||||
// mocked store returns, using Go field names; zero values may be omitted and
|
||||
// applyFixtureDefaults fills the boilerplate) and golden/<peerID>.json.
|
||||
//
|
||||
// There is ONE expectation per peer, shared by every mode. The modes are not
|
||||
// different computations: CalculateNetworkMapFromComponents is
|
||||
// components.Calculate, and both sides assemble the proto with the same
|
||||
// encode helpers, so the only variable is what the envelope round-trip did to
|
||||
// the components in transit. Any difference between modes is therefore a
|
||||
// round-trip fidelity defect, and a shared expectation is what exposes it.
|
||||
// Results are canonicalized before comparison, since repeated proto fields
|
||||
// come from map iteration.
|
||||
package nmaptest
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/google/go-cmp/cmp"
|
||||
"github.com/stretchr/testify/require"
|
||||
"golang.org/x/exp/maps"
|
||||
"google.golang.org/protobuf/encoding/protojson"
|
||||
"google.golang.org/protobuf/testing/protocmp"
|
||||
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map"
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller"
|
||||
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller/cache"
|
||||
mgmtgrpc "github.com/netbirdio/netbird/management/internals/shared/grpc"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap"
|
||||
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
|
||||
"github.com/netbirdio/netbird/shared/management/proto"
|
||||
)
|
||||
|
||||
// Mode selects the wire shape a case is verified through. Both end in a
|
||||
// *proto.NetworkMap, the one comparison surface shared by every path.
|
||||
type Mode string
|
||||
|
||||
const (
|
||||
// ModeFull is the legacy wire shape: the server runs Calculate and sends
|
||||
// the expanded map (grpc.ToSyncResponse).
|
||||
ModeFull Mode = "full"
|
||||
// ModeEnvelope is the component wire shape: the server encodes components
|
||||
// into a NetworkMapEnvelope (grpc.ToComponentSyncResponse) and the map is
|
||||
// expanded the way the client engine does (networkmap.EnvelopeToNetworkMap).
|
||||
ModeEnvelope Mode = "envelope"
|
||||
|
||||
defaultAccountID = "account"
|
||||
defaultDNSDomain = "netbird.test"
|
||||
)
|
||||
|
||||
var defaultModes = []Mode{ModeFull, ModeEnvelope}
|
||||
|
||||
// Case is one nmap-generation scenario: store data for a single account, the
|
||||
// peers whose network maps are computed, and the directory holding one expected
|
||||
// *proto.NetworkMap per peer — shared by every mode.
|
||||
type Case struct {
|
||||
Name string
|
||||
AccountID string
|
||||
DNSDomain string
|
||||
Peers []string
|
||||
Modes []Mode
|
||||
Data *networkmap.NetworkMapData
|
||||
GoldenDir string
|
||||
}
|
||||
|
||||
type manifest struct {
|
||||
Description string
|
||||
AccountID string
|
||||
DNSDomain string
|
||||
Peers []string
|
||||
Modes []Mode
|
||||
}
|
||||
|
||||
// RunGoldenDir discovers and runs every fixture case under dir. A case is a
|
||||
// directory containing case.json (manifest), nmdata.json (store fixture) and
|
||||
// golden/<peerID>.json (expected proto.NetworkMap, protojson).
|
||||
func RunGoldenDir(t *testing.T, dir string) {
|
||||
t.Helper()
|
||||
|
||||
entries, err := os.ReadDir(dir)
|
||||
require.NoError(t, err, "read cases dir")
|
||||
|
||||
ran := 0
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() || strings.HasPrefix(entry.Name(), ".") {
|
||||
continue
|
||||
}
|
||||
caseDir := filepath.Join(dir, entry.Name())
|
||||
c, err := loadCase(caseDir)
|
||||
require.NoError(t, err, "load case %s", entry.Name())
|
||||
ran++
|
||||
t.Run(entry.Name(), func(t *testing.T) {
|
||||
RunCase(t, c)
|
||||
})
|
||||
}
|
||||
require.NotZero(t, ran, "no cases found under %s", dir)
|
||||
}
|
||||
|
||||
func loadCase(caseDir string) (Case, error) {
|
||||
raw, err := os.ReadFile(filepath.Join(caseDir, "case.json"))
|
||||
if err != nil {
|
||||
return Case{}, fmt.Errorf("read manifest: %w", err)
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.DisallowUnknownFields()
|
||||
var m manifest
|
||||
if err := dec.Decode(&m); err != nil {
|
||||
return Case{}, fmt.Errorf("decode manifest: %w", err)
|
||||
}
|
||||
|
||||
data, err := LoadNetworkMapData(filepath.Join(caseDir, "nmdata.json"))
|
||||
if err != nil {
|
||||
return Case{}, err
|
||||
}
|
||||
|
||||
return Case{
|
||||
Name: filepath.Base(caseDir),
|
||||
AccountID: m.AccountID,
|
||||
DNSDomain: m.DNSDomain,
|
||||
Peers: m.Peers,
|
||||
Modes: m.Modes,
|
||||
Data: data,
|
||||
GoldenDir: filepath.Join(caseDir, "golden"),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// RunCase computes each target peer's network map through every enabled mode
|
||||
// and compares the canonicalized result against the peer's expectation file.
|
||||
// It mirrors the controller's store path: fill fixture defaults, precompute
|
||||
// posture validation once, then run the per-peer pipeline.
|
||||
func RunCase(t *testing.T, c Case) {
|
||||
t.Helper()
|
||||
|
||||
require.NotNil(t, c.Data, "case %s: Data is required", c.Name)
|
||||
require.NotEmpty(t, c.Peers, "case %s: Peers is required", c.Name)
|
||||
require.NotEmpty(t, c.GoldenDir, "case %s: GoldenDir is required", c.Name)
|
||||
if c.AccountID == "" {
|
||||
c.AccountID = defaultAccountID
|
||||
}
|
||||
if c.DNSDomain == "" {
|
||||
c.DNSDomain = defaultDNSDomain
|
||||
}
|
||||
if len(c.Modes) == 0 {
|
||||
c.Modes = defaultModes
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
nmData := c.Data
|
||||
applyFixtureDefaults(nmData)
|
||||
nmData.PrecomputePostureValidation()
|
||||
|
||||
dnsDomain := c.DNSDomain
|
||||
if nmData.AccountSettings.DNSDomain != "" {
|
||||
dnsDomain = nmData.AccountSettings.DNSDomain
|
||||
}
|
||||
|
||||
zone := networkmap.PeersCustomZone(ctx, c.AccountID, dnsDomain, nmData.Peers, controller.IPv6AllowedPeersFromData(nmData))
|
||||
dnsFwdPort := controller.ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
|
||||
|
||||
for _, mode := range c.Modes {
|
||||
if mode == ModeEnvelope {
|
||||
requireEnvelopeSafeKeys(t, nmData, c.Name)
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
for _, peerID := range c.Peers {
|
||||
peer := nmData.Peers[peerID]
|
||||
require.NotNil(t, peer, "case %s: target peer %q not in fixture", c.Name, peerID)
|
||||
|
||||
for _, mode := range c.Modes {
|
||||
t.Run(peerID+"/"+string(mode), func(t *testing.T) {
|
||||
got := computeMode(t, ctx, mode, nmData, peerID, zone, dnsDomain, dnsFwdPort)
|
||||
canonicalize(got)
|
||||
compareGolden(t, filepath.Join(c.GoldenDir, peerID+".json"), got, mode)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// computeMode produces the peer's proto.NetworkMap the way the controller does
|
||||
// for that wire shape.
|
||||
func computeMode(t *testing.T, ctx context.Context, mode Mode, nmData *networkmap.NetworkMapData,
|
||||
peerID string, zone nmdata.CustomZone, dnsDomain string, dnsFwdPort int64) *proto.NetworkMap {
|
||||
t.Helper()
|
||||
|
||||
peer := nmData.Peers[peerID]
|
||||
require.NotNil(t, peer, "target peer %q not in fixture", peerID)
|
||||
|
||||
switch mode {
|
||||
case ModeFull:
|
||||
nmap := controller.NetworkMapFromData(ctx, nmData, peerID, zone)
|
||||
return mgmtgrpc.ToSyncResponse(ctx, nil, nil, nil, peer, nil, nil, nmap, dnsDomain, nil,
|
||||
&cache.DNSConfigCache{}, nmData.AccountSettings, nil, nil, dnsFwdPort).NetworkMap
|
||||
case ModeEnvelope:
|
||||
components := nmData.GetPeerNetworkMapComponents(peerID, zone)
|
||||
peerGroups := maps.Keys(nmData.GetPeerGroups(peerID))
|
||||
resp := mgmtgrpc.ToComponentSyncResponse(ctx, nil, nil, nil, peer, nil, nil, components, nil,
|
||||
dnsDomain, nil, nmData.AccountSettings, nil, peerGroups, dnsFwdPort)
|
||||
res, err := networkmap.EnvelopeToNetworkMap(ctx, resp.NetworkMapEnvelope, peer.Key, dnsDomain)
|
||||
require.NoError(t, err, "expand envelope")
|
||||
return res.NetworkMap
|
||||
default:
|
||||
t.Fatalf("unknown mode %q", mode)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// requireEnvelopeSafeKeys fails fast on peer keys the envelope decoder would
|
||||
// silently drop: it re-keys peers by base64 of the raw 32-byte WG public key.
|
||||
func requireEnvelopeSafeKeys(t *testing.T, nmData *networkmap.NetworkMapData, caseName string) {
|
||||
t.Helper()
|
||||
for id, p := range nmData.Peers {
|
||||
if p == nil {
|
||||
continue
|
||||
}
|
||||
raw, err := base64.StdEncoding.DecodeString(p.Key)
|
||||
if err != nil || len(raw) != 32 {
|
||||
t.Fatalf("case %s: peer %q Key must be base64 of 32 bytes for mode %q (the envelope decoder drops it otherwise); use a real WireGuard public key or restrict the case to mode %q",
|
||||
caseName, id, ModeEnvelope, ModeFull)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// compareGolden measures got against the committed expectation file. One
|
||||
// expectation serves every mode, because the modes run the same computation and
|
||||
// must therefore agree. The expectation is the authority: a mismatch means the
|
||||
// code does not produce what this case says it should, so it is reported as a
|
||||
// failure and not quietly absorbed.
|
||||
//
|
||||
// The full mode is compared verbatim, identifiers included, so the expectation
|
||||
// pins real ids and stays readable. Other modes have identifiers erased on both
|
||||
// sides first, because the envelope currently rewrites them — a tracked defect
|
||||
// that TestIDSpaceMatches asserts against on its own, so it does not have to
|
||||
// drown out every other case here.
|
||||
// Nothing here writes to testdata. Expectation files are authored by hand and
|
||||
// only ever change through a reviewed edit, so there is no mode in which a run
|
||||
// can create or replace one. When a file is missing the computed map is printed
|
||||
// for the author to read and, if it is genuinely correct, save deliberately.
|
||||
func compareGolden(t *testing.T, path string, got *proto.NetworkMap, mode Mode) {
|
||||
t.Helper()
|
||||
|
||||
if mode != ModeFull {
|
||||
normalizeIDSpace(got)
|
||||
canonicalize(got)
|
||||
}
|
||||
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
rendered, mErr := renderNetworkMap(got)
|
||||
require.NoError(t, mErr)
|
||||
t.Fatalf("no expectation file %s: %v\nThis case has nothing to measure against — write the "+
|
||||
"proto.NetworkMap this peer should receive. Mode %s currently produces:\n%s\nRead it before "+
|
||||
"saving any of it: if the code is wrong, so is this.", path, err, mode, rendered)
|
||||
}
|
||||
want := &proto.NetworkMap{}
|
||||
require.NoError(t, protojson.Unmarshal(raw, want), "parse expectation %s", path)
|
||||
canonicalize(want)
|
||||
if mode != ModeFull {
|
||||
normalizeIDSpace(want)
|
||||
canonicalize(want)
|
||||
}
|
||||
|
||||
if diff := cmp.Diff(want, got, protocmp.Transform()); diff != "" {
|
||||
t.Errorf("mode %s does not produce what %s expects (-want +got):\n%s\n"+
|
||||
"Both modes run the same computation on the same components, so they must produce the same map. "+
|
||||
"The expectation file is the committed statement of correct output — fix the code, or change the "+
|
||||
"expectation deliberately if the intended behaviour really moved.", mode, path, diff)
|
||||
}
|
||||
}
|
||||
|
||||
// renderNetworkMap renders stable protojson: protojson output whitespace is
|
||||
// deliberately unstable, so it is reformatted through json.Indent.
|
||||
func renderNetworkMap(nm *proto.NetworkMap) ([]byte, error) {
|
||||
raw, err := protojson.Marshal(nm)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := json.Indent(&buf, raw, "", " "); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
buf.WriteByte('\n')
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"description": "Two groups joined by one allow-all policy; peer-c has SSH enabled so the legacy-SSH path fills SshAuth from AllowedUserIDs.",
|
||||
"peers": [
|
||||
"peer-a",
|
||||
"peer-c"
|
||||
]
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
{
|
||||
"Serial": "5",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "4deEImv8zGvsyBmmfC2G0eQkbyMzyGuz/YK7pcYETwM=",
|
||||
"allowedIps": [
|
||||
"100.64.0.3/32"
|
||||
],
|
||||
"sshConfig": {
|
||||
"sshPubKey": "c3NoLXBlZXItYw=="
|
||||
},
|
||||
"fqdn": "peer-c.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
},
|
||||
{
|
||||
"Name": "peer-c.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"FirewallRules": [
|
||||
{
|
||||
"PeerIP": "100.64.0.3",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
},
|
||||
{
|
||||
"PeerIP": "100.64.0.3",
|
||||
"Direction": "OUT",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
}
|
||||
],
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,102 +0,0 @@
|
||||
{
|
||||
"Serial": "5",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.3/10",
|
||||
"sshConfig": {
|
||||
"sshEnabled": true
|
||||
},
|
||||
"fqdn": "peer-c.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
|
||||
"allowedIps": [
|
||||
"100.64.0.2/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-b.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
},
|
||||
{
|
||||
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
|
||||
"allowedIps": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
},
|
||||
{
|
||||
"Name": "peer-b.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.2"
|
||||
},
|
||||
{
|
||||
"Name": "peer-c.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"FirewallRules": [
|
||||
{
|
||||
"PeerIP": "100.64.0.1",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
},
|
||||
{
|
||||
"PeerIP": "100.64.0.1",
|
||||
"Direction": "OUT",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
},
|
||||
{
|
||||
"PeerIP": "100.64.0.2",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
},
|
||||
{
|
||||
"PeerIP": "100.64.0.2",
|
||||
"Direction": "OUT",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLWFsbA=="
|
||||
}
|
||||
],
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub",
|
||||
"AuthorizedUsers": [
|
||||
"u9dHvAXZJKiXITuwP9jD/A=="
|
||||
],
|
||||
"machineUsers": {
|
||||
"*": {
|
||||
"indexes": [
|
||||
0
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 5},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-b": {"IP": "100.64.0.2", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-c": {"IP": "100.64.0.3", "SSHEnabled": true, "SSHKey": "ssh-peer-c", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a", "peer-b"]},
|
||||
"grp-ops": {"Peers": ["peer-c"]}
|
||||
},
|
||||
"Policies": [
|
||||
{
|
||||
"ID": "pol-all",
|
||||
"PublicID": "pol-all-pub",
|
||||
"Enabled": true,
|
||||
"Rules": [
|
||||
{
|
||||
"Enabled": true,
|
||||
"Action": "accept",
|
||||
"Protocol": "all",
|
||||
"Bidirectional": true,
|
||||
"Sources": ["grp-dev"],
|
||||
"Destinations": ["grp-ops"]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"AllowedUserIDs": {"user-ops": {}}
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"description": "Nameserver group and applied custom zone distributed to grp-dev; peer-a (with an extra DNS label) receives them, peer-c outside the group receives neither.",
|
||||
"peers": [
|
||||
"peer-a",
|
||||
"peer-c"
|
||||
]
|
||||
}
|
||||
@@ -1,93 +0,0 @@
|
||||
{
|
||||
"Serial": "8",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
|
||||
"allowedIps": [
|
||||
"100.64.0.2/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-b.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"NameServerGroups": [
|
||||
{
|
||||
"NameServers": [
|
||||
{
|
||||
"IP": "8.8.8.8",
|
||||
"Port": "53"
|
||||
}
|
||||
],
|
||||
"Primary": true
|
||||
}
|
||||
],
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "corp.internal.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "db.corp.internal.",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "10.10.0.5"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
},
|
||||
{
|
||||
"Name": "peer-b.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.2"
|
||||
},
|
||||
{
|
||||
"Name": "www.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"FirewallRules": [
|
||||
{
|
||||
"PeerIP": "100.64.0.2",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLW1lc2g="
|
||||
},
|
||||
{
|
||||
"PeerIP": "100.64.0.2",
|
||||
"Direction": "OUT",
|
||||
"Protocol": "ALL",
|
||||
"PolicyID": "cG9sLW1lc2g="
|
||||
}
|
||||
],
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
{
|
||||
"Serial": "8",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.3/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-c.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeersIsEmpty": true,
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-c.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,51 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 8},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "ExtraDNSLabels": ["www"], "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-b": {"IP": "100.64.0.2", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-c": {"IP": "100.64.0.3", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a", "peer-b"]},
|
||||
"grp-ops": {"Peers": ["peer-c"]}
|
||||
},
|
||||
"Policies": [
|
||||
{
|
||||
"ID": "pol-mesh",
|
||||
"PublicID": "pol-mesh-pub",
|
||||
"Enabled": true,
|
||||
"Rules": [
|
||||
{
|
||||
"Enabled": true,
|
||||
"Action": "accept",
|
||||
"Protocol": "all",
|
||||
"Bidirectional": true,
|
||||
"Sources": ["grp-dev"],
|
||||
"Destinations": ["grp-dev"]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"NameServerGroups": [
|
||||
{
|
||||
"ID": "nsg-1",
|
||||
"Name": "dns-primary",
|
||||
"NameServers": [{"IP": "8.8.8.8", "Port": 53}],
|
||||
"Groups": ["grp-dev"],
|
||||
"Primary": true,
|
||||
"Enabled": true
|
||||
}
|
||||
],
|
||||
"AppliedZoneCandidates": [
|
||||
{
|
||||
"DistributionGroups": ["grp-dev"],
|
||||
"Zone": {
|
||||
"Domain": "corp.internal.",
|
||||
"Records": [
|
||||
{"Name": "db.corp.internal.", "Type": 1, "Class": "IN", "TTL": 300, "RData": "10.10.0.5"}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"description": "Domain network resource: the route carries the domain list and the 192.0.2.0/32 placeholder network with NetworkType 3 (dynamic), and peer-r's route firewall rules must be marked dynamic and repeat the domain. Two ports on the policy must produce one rule per port. A domain resource contributes no DNS custom zone of its own — resolution happens through the routing peer's forwarder.",
|
||||
"peers": ["peer-a", "peer-r"]
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
{
|
||||
"Serial": "22",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"allowedIps": [
|
||||
"100.64.0.9/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"Routes": [
|
||||
{
|
||||
"ID": "res-domain:peer-r",
|
||||
"Network": "192.0.2.0/32",
|
||||
"NetworkType": "3",
|
||||
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"Metric": "9999",
|
||||
"Masquerade": true,
|
||||
"NetID": "app-domain",
|
||||
"Domains": [
|
||||
"app.internal"
|
||||
],
|
||||
"keepRoute": true
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,92 +0,0 @@
|
||||
{
|
||||
"Serial": "22",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.9/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
|
||||
"allowedIps": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"Routes": [
|
||||
{
|
||||
"ID": "res-domain:peer-r",
|
||||
"Network": "192.0.2.0/32",
|
||||
"NetworkType": "3",
|
||||
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"Metric": "9999",
|
||||
"Masquerade": true,
|
||||
"NetID": "app-domain",
|
||||
"Domains": [
|
||||
"app.internal"
|
||||
],
|
||||
"keepRoute": true
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-r.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRules": [
|
||||
{
|
||||
"sourceRanges": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"destination": "192.0.2.0/32",
|
||||
"protocol": "TCP",
|
||||
"portInfo": {
|
||||
"port": 443
|
||||
},
|
||||
"isDynamic": true,
|
||||
"domains": [
|
||||
"app.internal"
|
||||
],
|
||||
"PolicyID": "cG9sLWFwcA==",
|
||||
"RouteID": "res-domain:peer-r"
|
||||
},
|
||||
{
|
||||
"sourceRanges": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"destination": "192.0.2.0/32",
|
||||
"protocol": "TCP",
|
||||
"portInfo": {
|
||||
"port": 80
|
||||
},
|
||||
"isDynamic": true,
|
||||
"domains": [
|
||||
"app.internal"
|
||||
],
|
||||
"PolicyID": "cG9sLWFwcA==",
|
||||
"RouteID": "res-domain:peer-r"
|
||||
}
|
||||
],
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,43 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 22},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a"]}
|
||||
},
|
||||
"Policies": [
|
||||
{
|
||||
"ID": "pol-app",
|
||||
"Enabled": true,
|
||||
"Rules": [
|
||||
{
|
||||
"Enabled": true,
|
||||
"Action": "accept",
|
||||
"Protocol": "tcp",
|
||||
"Ports": ["80", "443"],
|
||||
"Sources": ["grp-dev"],
|
||||
"DestinationResource": {"ID": "res-domain", "Type": "domain"}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ResourcePolicies": {"res-domain": [{"ID": "pol-app"}]},
|
||||
"NetworkResources": [
|
||||
{
|
||||
"ID": "res-domain",
|
||||
"NetworkID": "net-1",
|
||||
"Name": "app-domain",
|
||||
"Type": "domain",
|
||||
"Domain": "app.internal",
|
||||
"Enabled": true
|
||||
}
|
||||
],
|
||||
"Routers": {
|
||||
"net-1": {
|
||||
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"description": "Host network resource (single /32) behind one directly-assigned router. peer-a is in the resource policy's source group and must receive one route to 10.10.0.7/32 via peer-r with KeepRoute set and NetID taken from the resource name; peer-r as the router must receive the same route plus a route firewall rule whose SourceRanges are the policy's source peers. A client never gets route firewall rules.",
|
||||
"peers": ["peer-a", "peer-r"]
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
{
|
||||
"Serial": "20",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"allowedIps": [
|
||||
"100.64.0.9/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"Routes": [
|
||||
{
|
||||
"ID": "res-host:peer-r",
|
||||
"Network": "10.10.0.7/32",
|
||||
"NetworkType": "1",
|
||||
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"Metric": "9999",
|
||||
"Masquerade": true,
|
||||
"NetID": "web-host",
|
||||
"keepRoute": true
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,69 +0,0 @@
|
||||
{
|
||||
"Serial": "20",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.9/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeers": [
|
||||
{
|
||||
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
|
||||
"allowedIps": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"agentVersion": "0.60.0"
|
||||
}
|
||||
],
|
||||
"Routes": [
|
||||
{
|
||||
"ID": "res-host:peer-r",
|
||||
"Network": "10.10.0.7/32",
|
||||
"NetworkType": "1",
|
||||
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
|
||||
"Metric": "9999",
|
||||
"Masquerade": true,
|
||||
"NetID": "web-host",
|
||||
"keepRoute": true
|
||||
}
|
||||
],
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-r.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRules": [
|
||||
{
|
||||
"sourceRanges": [
|
||||
"100.64.0.1/32"
|
||||
],
|
||||
"destination": "10.10.0.7/32",
|
||||
"protocol": "TCP",
|
||||
"portInfo": {
|
||||
"port": 443
|
||||
},
|
||||
"PolicyID": "cG9sLXdlYg==",
|
||||
"RouteID": "res-host:peer-r"
|
||||
}
|
||||
],
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,43 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 20},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a"]}
|
||||
},
|
||||
"Policies": [
|
||||
{
|
||||
"ID": "pol-web",
|
||||
"Enabled": true,
|
||||
"Rules": [
|
||||
{
|
||||
"Enabled": true,
|
||||
"Action": "accept",
|
||||
"Protocol": "tcp",
|
||||
"Ports": ["443"],
|
||||
"Sources": ["grp-dev"],
|
||||
"DestinationResource": {"ID": "res-host", "Type": "host"}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ResourcePolicies": {"res-host": [{"ID": "pol-web"}]},
|
||||
"NetworkResources": [
|
||||
{
|
||||
"ID": "res-host",
|
||||
"NetworkID": "net-1",
|
||||
"Name": "web-host",
|
||||
"Type": "host",
|
||||
"Prefix": "10.10.0.7/32",
|
||||
"Enabled": true
|
||||
}
|
||||
],
|
||||
"Routers": {
|
||||
"net-1": {
|
||||
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"description": "A disabled resource with a valid policy and router must leave no trace: no routes and no route firewall rules for either the client or the router. Disabling a resource is the switch that revokes access without deleting the policy.",
|
||||
"peers": ["peer-a", "peer-r"]
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
{
|
||||
"Serial": "25",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeersIsEmpty": true,
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
{
|
||||
"Serial": "25",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.9/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeersIsEmpty": true,
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-r.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,42 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 25},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a"]}
|
||||
},
|
||||
"Policies": [
|
||||
{
|
||||
"ID": "pol-off-resource",
|
||||
"Enabled": true,
|
||||
"Rules": [
|
||||
{
|
||||
"Enabled": true,
|
||||
"Action": "accept",
|
||||
"Protocol": "tcp",
|
||||
"Ports": ["443"],
|
||||
"Sources": ["grp-dev"],
|
||||
"DestinationResource": {"ID": "res-disabled", "Type": "subnet"}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ResourcePolicies": {"res-disabled": [{"ID": "pol-off-resource"}]},
|
||||
"NetworkResources": [
|
||||
{
|
||||
"ID": "res-disabled",
|
||||
"NetworkID": "net-1",
|
||||
"Name": "disabled-subnet",
|
||||
"Type": "subnet",
|
||||
"Prefix": "10.50.0.0/24"
|
||||
}
|
||||
],
|
||||
"Routers": {
|
||||
"net-1": {
|
||||
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"description": "An enabled resource with a healthy router but no policy granting access to it must produce nothing anywhere: no route for the client and none for the router either, since access to a resource is only ever created by a policy. The router also gets no route firewall rules despite being a routing peer for the network.",
|
||||
"peers": ["peer-a", "peer-r"]
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
{
|
||||
"Serial": "24",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.1/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-a.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeersIsEmpty": true,
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-a.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
{
|
||||
"Serial": "24",
|
||||
"peerConfig": {
|
||||
"address": "100.64.0.9/10",
|
||||
"sshConfig": {},
|
||||
"fqdn": "peer-r.netbird.test",
|
||||
"RoutingPeerDnsResolutionEnabled": true,
|
||||
"autoUpdate": {}
|
||||
},
|
||||
"remotePeersIsEmpty": true,
|
||||
"DNSConfig": {
|
||||
"ServiceEnable": true,
|
||||
"CustomZones": [
|
||||
{
|
||||
"Domain": "netbird.test.",
|
||||
"Records": [
|
||||
{
|
||||
"Name": "peer-r.netbird.test",
|
||||
"Type": "1",
|
||||
"Class": "IN",
|
||||
"TTL": "300",
|
||||
"RData": "100.64.0.9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ForwarderPort": "22054"
|
||||
},
|
||||
"firewallRulesIsEmpty": true,
|
||||
"routesFirewallRulesIsEmpty": true,
|
||||
"sshAuth": {
|
||||
"UserIDClaim": "sub"
|
||||
}
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
{
|
||||
"Network": {"Serial": 24},
|
||||
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
|
||||
"Peers": {
|
||||
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
|
||||
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
|
||||
},
|
||||
"Groups": {
|
||||
"grp-dev": {"Peers": ["peer-a"]}
|
||||
},
|
||||
"NetworkResources": [
|
||||
{
|
||||
"ID": "res-orphan",
|
||||
"NetworkID": "net-1",
|
||||
"Name": "orphan-subnet",
|
||||
"Type": "subnet",
|
||||
"Prefix": "10.40.0.0/24",
|
||||
"Enabled": true
|
||||
}
|
||||
],
|
||||
"Routers": {
|
||||
"net-1": {
|
||||
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
|
||||
}
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user