Compare commits

..

145 Commits

Author SHA1 Message Date
Dmitri Dolguikh
4be6603815 added comments re: ordering of fields in intermediate DTOs
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 16:47:39 +02:00
Dmitri Dolguikh
05511cc13d cleanup sqlite store creation
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 16:05:59 +02:00
Dmitri Dolguikh
9427fa87e1 moved GetNetworkMapData implementation to NetworkMapDBStoreImpl
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 15:42:11 +02:00
Dmitri Dolguikh
fe5e5c08eb Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:52:14 +02:00
Dmitri Dolguikh
cb858b335d moved tests out of pgsql dir
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:48:31 +02:00
Dmitri Dolguikh
8cbbea4536 support for GetAllowedUsers in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:46:59 +02:00
Dmitri Dolguikh
cd6f63272b support for GetPrivateServices in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:25:39 +02:00
Dmitri Dolguikh
859af13c12 support for GetRoutes in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:49:36 +02:00
Dmitri Dolguikh
4375fdc7b6 support for GetPostureChecks in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:42:14 +02:00
Dmitri Dolguikh
1caa0ddf27 support for GetPolicies in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:36:15 +02:00
pascal
cd2a91ddd8 fir error handling and return values (linter complaint) 2026-08-11 12:45:25 +02:00
pascal
38c5932375 merge main 2026-08-11 12:01:51 +02:00
Dmitri Dolguikh
fe9ea43198 support for GetPeers in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 11:09:34 +02:00
Dmitri Dolguikh
fad568fdb0 support for GetNetworkXIDToPublicIdMap in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 10:51:34 +02:00
Dmitri Dolguikh
3da27221ac support for GetNetwork in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 10:43:00 +02:00
Dmitri Dolguikh
d954a2dc3e support for GetNetworkRouters in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 20:10:41 +02:00
Brad Ison
27b2d3f351 [management] Add a proxy-connect authorizer seam (#7136)
At proxy connect time, the declared cluster address is validated for
shape and checked for availability (`IsClusterAddressAvailable`), and
from then on the declaration is what routes the cluster's mappings to
the connection. Deployments that embed management through the
integrations seam may need a policy on that claim — deciding which
credential is allowed to declare which address.

This adds an optional `ProxyConnectAuthorizer` hook on
`ProxyServiceServer`, following the pattern of the existing `Set*` seams
(`SetServiceManager`, `SetAgentNetworkSynthesizer`,
`SetAgentNetworkLimitsService`, `SetProxyController`):

- A nil-able interface field plus `SetProxyConnectAuthorizer`, guarded
by the existing mutex.
- One call at the end of `validateProxyConnect`, so both
`GetMappingUpdate` and `SyncMappings` are covered by a single site.
- **Nothing installs it by default** — with the hook unset (always, in
this repo), behavior is byte-for-byte unchanged, which the tests pin.

Design details:

- The authorizer runs **last** — after input validation and the
availability check — and **outside** the account-scoped branch, so
management-wide tokens and token-less connects are also presented to it
rather than bypassing policy.
- The authorizer receives the presented `*types.ProxyAccessToken` (nil
when none), the proxy ID, and the declared address. Everything it needs
is already in the request/context; no proto or schema change.
- A plain error from the authorizer surfaces as `PermissionDenied`,
keeping an authorization rejection distinguishable from the
`AlreadyExists` used for address conflicts in proxy logs. A status error
passes through unchanged so implementations can pick their own code.
2026-08-10 19:50:00 +02:00
Brad Ison
ebfdf7d7b8 [management] Rework Agent Network endpoint identity and settings bootstrap (#7085)
Store the per-account gateway endpoint as {domain, proxy_address} with a
global unique index on the full hostname; dedicated = (domain ==
proxy_address). Bootstrap becomes an explicit POST carrying exactly one
of proxy_address (server allocates an adjective-noun label beneath it)
or endpoint (claimed verbatim, address-first); provider create loses its
bootstrap side effect. PUT is a full replace with every field required —
the immutable identity fields must be echoed unchanged and a mismatch is
rejected with 422. A guarded DELETE releases the endpoint: refused with
412 while providers exist or a proxy is actively serving the endpoint
hostname (matched case-insensitively); re-creating bootstraps fresh. A
self-addressed pin excludes its address from the account's cluster allow
list, and the live mapping update path now addresses the serving proxy
from the synthesized service. Existing rows are migrated on all three
store engines.
2026-08-10 19:06:55 +02:00
Dmitri Dolguikh
433a4f12bf added support for GetNetworkResources in sqlite; moved several more internal types into shared_types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 18:07:31 +02:00
Dmitri Dolguikh
3834e67a51 add support for GetNameServerGroups in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 17:53:34 +02:00
Viktor Liu
e8671a811d [client, relay] Migrate relay QUIC tracer to qlog and bump quic-go to 0.59.1 (#7124) 2026-08-10 17:41:31 +02:00
Dmitri Dolguikh
78947c1611 support for GetGroups in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 17:14:37 +02:00
Zoltan Papp
1ca26d8faa [client] Bump the netbirdio/wails fork to drop the native WebView2 (#7128)
## Describe your changes

The native_webview2loader build tag embedded Microsoft's
WebView2Loader.dll via //go:embed. Those DLLs are gitignored in the fork
and go mod vendor resolves embed patterns regardless of build
constraints, so vendoring the module failed on the missing files when
packaging for openSUSE.

The fork now removes that branch along with the go-winloader dependency,
which drops out of the module graph here. The default GoWebView2Loader
path is unaffected.



## Issue ticket number and link

<!--
Required for anything that changes behavior. Link the issue (or the
validated
discussion it came from) that the NetBird team already agreed on. See

https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#ticket-first-pr-second
-->

## Stack

<!-- branch-stack -->

### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [ ] Created tests that fail without the change (if possible)
- [ ] I ran and tested this change locally — I did not rely on CI to
find out whether it works
- [ ] This PR has a single purpose (not a fix + refactor + feature in
one)
- [ ] This change is a trivial fix, **OR** it links an issue the NetBird
team agreed on beforehand. Changes to the public API, gRPC protocols,
functionality behavior, CLI / service flags, or new features always need
that agreement first. See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#ticket-first-pr-second).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated the application framework revision.
  * Removed an unused dependency.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-10 16:41:50 +02:00
Dmitri Dolguikh
763b8f6933 support for GetDomains in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 15:55:24 +02:00
Dmitri Dolguikh
e530c25812 support for GetAppliedZoneCandidates in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:59:03 +02:00
Dmitri Dolguikh
166b3d7739 moved test for RecordTypeAndRdata into the parent package
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:21:19 +02:00
Dmitri Dolguikh
20f18d1479 extracted struct-handling helpers into their own file; move sql_type_conversion_test to the parent module
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:15:11 +02:00
Dmitri Dolguikh
5594289924 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:08:23 +02:00
Dmitri Dolguikh
a03635680b support for GetDnsSettings in sqlite store
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:07:52 +02:00
pascal
905e1c14ba add validated peers cache for nmdata 2026-08-10 14:02:06 +02:00
Viktor Liu
f9abe2727f [client] Do not misroute WireGuard packets to the STUN handler (#7059) 2026-08-10 13:41:09 +02:00
Dmitri Dolguikh
5e4d6a5e44 run tests against postgres by default (for now)
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 13:39:25 +02:00
Dmitri Dolguikh
65f184141b added support for GetAccountSettings to sqlite store
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 13:31:55 +02:00
Zoltan Papp
664a3d026c [client] Fix credentials for the gtk3 package uploads (#7125) 2026-08-10 12:58:50 +02:00
Zoltan Papp
d2c961f67c [client] Declare the xdg-utils dependency for the netbird-ui packages (#7126)
## Describe your changes

The UI shells out to xdg-open to launch the external browser for the SSO
verification page, which the embedded webview cannot open inline, and to
reveal the debug bundle in the file manager.

client/ui/build/linux/nfpm/nfpm.yaml lists xdg-utils for every package
format, but the released packages are built from the goreleaser configs,
where it was missing: the GTK and WebKitGTK dependencies carried over
and xdg-utils did not. Add it to all four nfpm dependency lists.

## Issue ticket number and link

<!--
Required for anything that changes behavior. Link the issue (or the
validated
discussion it came from) that the NetBird team already agreed on. See

https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#ticket-first-pr-second
-->

## Stack

<!-- branch-stack -->

### Checklist
- [x] Is it a bug fix
- [ ] Is a typo/documentation fix
- [ ] Is a feature enhancement
- [ ] It is a refactor
- [ ] Created tests that fail without the change (if possible)
- [ ] I ran and tested this change locally — I did not rely on CI to
find out whether it works
- [ ] This PR has a single purpose (not a fix + refactor + feature in
one)
- [ ] This change is a trivial fix, **OR** it links an issue the NetBird
team agreed on beforehand. Changes to the public API, gRPC protocols,
functionality behavior, CLI / service flags, or new features always need
that agreement first. See
[CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#ticket-first-pr-second).

> By submitting this pull request, you confirm that you have read and
agree to the terms of the [Contributor License
Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

## Documentation
Select exactly one:

- [ ] I added/updated documentation for this change
- [x] Documentation is **not needed** for this change (explain why)

### Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Added required desktop integration support to Debian and RPM packages.
* Ensured GTK3 packages include the same runtime support for opening
links and files through the system.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-10 11:28:15 +02:00
Viktor Liu
5584f8ef0a [client] Add strict anonymization level and MAC anonymization to debug bundles (#7102) 2026-08-10 11:27:20 +02:00
Dmitri Dolguikh
c54c35beed disable tests when running with sqlite 2026-08-10 10:02:46 +02:00
Dmitri Dolguikh
935d1c5863 move read-only queries to an interface to reuse in tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-07 19:23:52 +02:00
Dmitri Dolguikh
ac3b713dda Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-07 09:56:15 +02:00
Dmitri Dolguikh
4d4de15f48 added 'integration' build tag to nmap db tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 16:14:51 +02:00
pascal
28074405e1 change wording from failer -> failing to pass codespell 2026-08-06 16:02:02 +02:00
pascal
fd23461b03 add network map compute test 2026-08-06 15:57:43 +02:00
Dmitri Dolguikh
b19cf7405d Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 15:18:13 +02:00
Dmitri Dolguikh
90e0c5bd0c added GetPolicies test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 15:17:39 +02:00
pascal
6ccb2bb239 handle nil values in policy, nameserver and resources 2026-08-06 15:12:12 +02:00
Dmitri Dolguikh
b160cb43b1 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 13:03:28 +02:00
Dmitri Dolguikh
1902777509 added a test for GetPeers
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 13:03:00 +02:00
pascal
cff46eee67 introduce peerGroupsIdx to avoid looping through the groups to figure out peer membership 2026-08-06 12:49:36 +02:00
pascal
0513109c35 improve looping on connected peers filtering 2026-08-06 12:31:26 +02:00
Dmitri Dolguikh
6b8393c6b0 add GetAppliedZoneCandidatesViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 18:43:20 +02:00
Dmitri Dolguikh
316e82337f cleanup query execution in tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 18:09:28 +02:00
Dmitri Dolguikh
bacacb8f77 deleted group_test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:45:25 +02:00
Dmitri Dolguikh
994e84a686 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 16:35:08 +02:00
Dmitri Dolguikh
07809fe923 added test for GetAllowedUsersViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:34:44 +02:00
pascal
ec3e37456e use Exec instead of Query for inserts and assert every insert error 2026-08-05 16:33:48 +02:00
pascal
c81103dfa6 fix linter comments 2026-08-05 16:21:01 +02:00
Dmitri Dolguikh
ba574dc739 added tests for GetPrivateServicesViaPgxConnection and GetPrivateServicesViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:06:43 +02:00
Dmitri Dolguikh
067982f772 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 15:26:38 +02:00
Dmitri Dolguikh
ce4f0a7821 added GetAccountSettingsViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 15:26:06 +02:00
pascal
3218e2f744 fix client tests 2026-08-05 15:18:19 +02:00
pascal
7e24c27ef1 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 15:09:41 +02:00
pascal
06817271bf fix embedded client test 2026-08-05 15:09:29 +02:00
Dmitri Dolguikh
1530739b54 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:58:36 +02:00
Dmitri Dolguikh
e745aedc92 added GetRoutesViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:58:09 +02:00
pascal
4cf3903c83 fix management <-> shared dependencies 2026-08-05 14:48:27 +02:00
pascal
1e14b554a6 fix management <-> shared dependencies 2026-08-05 14:48:18 +02:00
Dmitri Dolguikh
11a3627899 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:29:43 +02:00
Dmitri Dolguikh
2442bd6198 tests for GetNameServerGroupsViaPgxConnection and GetPostureChecksViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:29:07 +02:00
pascal
006cee000f Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 13:56:46 +02:00
pascal
c93aa03c0e merge main 2026-08-05 13:56:35 +02:00
Dmitri Dolguikh
efe2eaeb09 added GetDomains test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 12:37:58 +02:00
Dmitri Dolguikh
1926e983fb added GetDnsSettings test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 12:03:36 +02:00
Dmitri Dolguikh
e9b8175915 added GetNetworks test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 11:53:40 +02:00
Dmitri Dolguikh
88f1930450 added account network test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 11:40:23 +02:00
Dmitri Dolguikh
1774a3d9dd add network resource query test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 10:52:39 +02:00
Dmitri Dolguikh
795e06ce83 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 10:16:54 +02:00
Dmitri Dolguikh
748f6b3fbb fixes + tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 10:16:21 +02:00
pascal
d5adfc799f handle policy with no rules to avoid index panic 2026-08-04 20:57:57 +02:00
pascal
60fe56e442 fix assert equal ordering 2026-08-04 20:56:16 +02:00
pascal
327fa4c0d6 fix group clone 2026-08-04 20:52:16 +02:00
pascal
ed015a7972 skip resource from proto failure 2026-08-04 20:46:10 +02:00
pascal
b43708e31b silently skip resource to proto failure 2026-08-04 20:42:43 +02:00
pascal
ce55b8406b handle not found peer on posture vali 2026-08-04 20:34:30 +02:00
pascal
0456b64ea1 silent skip on non existing groups references 2026-08-04 20:28:06 +02:00
pascal
942ee81ec0 add benchmark 2026-08-04 20:19:17 +02:00
Dmitri Dolguikh
33a0e1bc2b adding integration tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 18:58:29 +02:00
Dmitri Dolguikh
dbfdd04c7b adding network-routers integration query test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 16:43:17 +02:00
Dmitri Dolguikh
4d010f60ce fix another import
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 15:40:28 +02:00
Dmitri Dolguikh
3272058e56 fix extra setting manager package
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 13:34:25 +02:00
Dmitri Dolguikh
70c3feb05b Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 13:04:51 +02:00
Dmitri Dolguikh
a0fe80cd99 wire up validated peers
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 12:52:09 +02:00
pascal
6fadf8f24a fix nmdata store expanding router peer groups alongside static peer 2026-08-03 23:58:47 +02:00
pascal
ef0032685b fix nmdata store shipping zones for disabled/public services 2026-08-03 23:54:22 +02:00
pascal
5ed38569f7 fix PrivateServiceCandidates nil assumption 2026-08-03 23:19:55 +02:00
pascal
d4e1c8978e add all group to user group lookup 2026-08-03 23:08:43 +02:00
pascal
d5ac70d806 fix missing ForceRoutingPeerDNSResolution on the nmdata store path 2026-08-03 22:50:06 +02:00
pascal
23579e4dd4 Skip the left-join NULL row instead of failing the account load + drop unsupported record types silently to match buildAppliedZoneCandidates 2026-08-03 22:38:12 +02:00
pascal
a00c5164a8 continue on resource policy loop + exclude disabled policies form index 2026-08-03 22:29:47 +02:00
pascal
02138dffdd fix group id mapping 2026-08-03 22:24:28 +02:00
pascal
02ec1f5dcb fix equivalence test 2026-08-03 22:24:11 +02:00
pascal
07d0440e34 fix slice initialization to avoid nil pointer dereference 2026-08-03 21:51:48 +02:00
pascal
2c85d94c6c hookup validated peers 2026-08-03 21:47:41 +02:00
Dmitri Dolguikh
1f190a50cf Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:22:34 +02:00
Dmitri Dolguikh
52fdfd5bdc add an posture-check-id to public-id index
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:21:07 +02:00
Dmitri Dolguikh
e35a0f3318 added PrivateServiceCandidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:04:11 +02:00
pascal
56c411babd fixed networks query and extended error wrapping 2026-08-03 17:52:33 +02:00
Dmitri Dolguikh
164b2baa4d Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 17:29:42 +02:00
Dmitri Dolguikh
0b29c6ed1a adding buildPrivateServiceCandidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 17:29:09 +02:00
pascal
724b61440b use new path on initial sync 2026-08-03 17:27:58 +02:00
pascal
22590ad66e hookup network map store 2026-08-03 17:27:24 +02:00
Dmitri Dolguikh
b8e004ea89 also build proxy-cluster to peer idx
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:58:53 +02:00
Dmitri Dolguikh
e2797360f4 support for applied zone candidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:32:54 +02:00
Dmitri Dolguikh
2f399f1e6e wire up dnssettings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:12:27 +02:00
Dmitri Dolguikh
82c1e18264 renamed allowed_user_ids.go to user.got 2026-08-03 13:54:22 +02:00
Dmitri Dolguikh
ce9023bd27 wire up accountsettings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 13:51:25 +02:00
Dmitri Dolguikh
7d33356776 support for group to user ids
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 13:47:14 +02:00
Dmitri Dolguikh
993291149c Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 12:20:18 +02:00
Dmitri Dolguikh
98f7ea40a1 added allowed_user_ids call
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 12:19:50 +02:00
pascal
76db9ab94f update integrated validator 2026-08-03 12:11:35 +02:00
Dmitri Dolguikh
9653b15d78 automatically fill empty public ids for view types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 11:34:22 +02:00
Dmitri Dolguikh
206bb1676b build resourcePolicies map
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 10:53:54 +02:00
Dmitri Dolguikh
600b0c752b also return a net resource to groups index from GetGroups call
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-01 11:56:17 +02:00
Dmitri Dolguikh
b02736adc3 cleanup network routers retrieval
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-01 11:54:14 +02:00
Dmitri Dolguikh
5d6117d2c0 adding network_map_data: compute Routers field
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 17:56:52 +02:00
Dmitri Dolguikh
c2f8360b00 adding support for bulding of network_map_data
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 15:58:40 +02:00
Dmitri Dolguikh
ea1b4d56e8 added support for queries via pgx connection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 15:03:15 +02:00
Dmitri Dolguikh
acbe22b831 small cleanups
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 13:41:34 +02:00
Dmitri Dolguikh
3b5c8e2298 started moving pg-specific nmap tests to integration_tests/management/network_map_db/pgsql
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 12:29:35 +02:00
Dmitri Dolguikh
2baeb4bc0d clean up sql types conversion test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 19:35:11 +02:00
Dmitri Dolguikh
4bf75fdc97 support for posture-checks
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 18:00:58 +02:00
Dmitri Dolguikh
799f3a3c62 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 16:03:46 +02:00
Dmitri Dolguikh
129736ad61 support for account settings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 16:03:19 +02:00
pascal
f7be9c4347 legacy network mal and equivalence test 2026-07-30 15:45:15 +02:00
Dmitri Dolguikh
e05cb5264d support for dns custom zones
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 15:34:09 +02:00
Dmitri Dolguikh
5a10561ca1 fix handling of string slices
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 11:31:53 +02:00
Dmitri Dolguikh
42ce83a8f3 added support for account network
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:57:08 +02:00
Dmitri Dolguikh
e620c86cd4 added support for networkrouters
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:32:51 +02:00
Dmitri Dolguikh
9dee2d60b9 added support for networkresources
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:19:34 +02:00
Dmitri Dolguikh
4525014632 support for nameservergroups
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 17:59:44 +02:00
Dmitri Dolguikh
23a5c0de4b added support for routes
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 17:37:00 +02:00
Dmitri Dolguikh
25e882004f added retrieval of policies
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-28 18:06:28 +02:00
Dmitri Dolguikh
15003258d2 networkmap read-only interface for pgsql
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-28 16:46:45 +02:00
Dmitri Dolguikh
2af3a5fba5 do not send resource policies map over the wire
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-27 19:00:30 +02:00
pascal
a6603a2e0a have explicit network map data type and do calculation from there 2026-07-27 15:17:30 +02:00
pascal
7ed3737cda revert component types 2026-07-24 15:16:55 +02:00
249 changed files with 19776 additions and 5356 deletions

View File

@@ -96,6 +96,7 @@ nfpms:
- netbird (>= 0.75.0)
- libgtk-4-1 (>= 4.14)
- libwebkitgtk-6.0-4
- xdg-utils
- maintainer: Netbird <dev@netbird.io>
description: Netbird client UI.
@@ -119,6 +120,7 @@ nfpms:
- netbird >= 0.75.0
- (gtk4 >= 4.14 or libgtk-4-1 >= 4.14)
- (webkitgtk6.0 or libwebkitgtk-6_0-4)
- xdg-utils
rpm:
signature:

View File

@@ -71,6 +71,7 @@ nfpms:
- netbird (>= 0.75.0)
- libgtk-3-0
- libwebkit2gtk-4.1-0
- xdg-utils
- maintainer: Netbird <dev@netbird.io>
description: Netbird client UI.
@@ -95,6 +96,7 @@ nfpms:
- netbird >= 0.75.0
- (gtk3 or libgtk-3-0)
- (webkit2gtk4.1 or libwebkit2gtk-4_1-0)
- xdg-utils
rpm:
signature:
@@ -112,6 +114,13 @@ uploads:
# The gtk3 packages reuse the netbird-ui package name, so they live in
# dedicated repo paths (deb distribution `gtk3`, yum path `yum-gtk3`) that
# legacy distros point their repo config at.
#
# GoReleaser derives the credential env var from the upload name, so these
# would look for UPLOAD_DEBIAN-GTK3_SECRET / UPLOAD_YUM-GTK3_SECRET. The
# release workflow only exports UPLOAD_DEBIAN_SECRET / UPLOAD_YUM_SECRET, and
# a missing secret is a silent skip rather than a failure -- the packages
# reached the GitHub release but never the package repositories. Point
# `password` at the exported vars so both uploads authenticate.
- name: debian-gtk3
skip: "{{ .Env.SKIP_PUBLISH }}"
ids:
@@ -119,6 +128,7 @@ uploads:
mode: archive
target: https://pkgs.wiretrustee.com/debian/pool/{{ .ArtifactName }};deb.distribution=gtk3;deb.component=main;deb.architecture={{ if .Arm }}armhf{{ else }}{{ .Arch }}{{ end }};deb.package=
username: dev@wiretrustee.com
password: "{{ .Env.UPLOAD_DEBIAN_SECRET }}"
method: PUT
- name: yum-gtk3
@@ -128,4 +138,5 @@ uploads:
mode: archive
target: https://pkgs.wiretrustee.com/yum-gtk3/{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}
username: dev@wiretrustee.com
password: "{{ .Env.UPLOAD_YUM_SECRET }}"
method: PUT

View File

@@ -15,6 +15,7 @@ import (
log "github.com/sirupsen/logrus"
nbAnonymize "github.com/netbirdio/netbird/client/anonymize"
"github.com/netbirdio/netbird/client/iface/device"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/debug"
@@ -32,6 +33,13 @@ import (
types "github.com/netbirdio/netbird/upload-server/types"
)
// AnonymizeLevelDefault and AnonymizeLevelStrict are the accepted
// anonymizeLevel values for DebugBundle.
const (
AnonymizeLevelDefault = nbAnonymize.LevelDefaultString
AnonymizeLevelStrict = nbAnonymize.LevelStrictString
)
// ConnectionListener export internal Listener for mobile
type ConnectionListener interface {
peer.Listener
@@ -278,8 +286,10 @@ func (c *Client) GetTunSettings() (*TunSettings, error) {
}
// DebugBundle generates a debug bundle, uploads it, and returns the upload key.
// It works both with and without a running engine.
func (c *Client) DebugBundle(platformFiles PlatformFiles, anonymize bool) (string, error) {
// It works both with and without a running engine. anonymizeLevel is "default"
// or "strict"; strict also anonymizes internal IP ranges, peer names, and
// WireGuard public keys, and implies anonymize.
func (c *Client) DebugBundle(platformFiles PlatformFiles, anonymize bool, anonymizeLevel string) (string, error) {
cfg, cacheDir, cc := c.stateSnapshot()
// If the engine hasn't been started, load config from disk
@@ -298,6 +308,7 @@ func (c *Client) DebugBundle(platformFiles PlatformFiles, anonymize bool) (strin
InternalConfig: cfg,
StatusRecorder: c.recorder,
TempDir: cacheDir,
StatePath: platformFiles.StateFilePath(),
}
if cc != nil {
@@ -321,6 +332,7 @@ func (c *Client) DebugBundle(platformFiles PlatformFiles, anonymize bool) (strin
deps,
debug.BundleConfig{
Anonymize: anonymize,
AnonymizeLevel: nbAnonymize.ParseLevel(anonymizeLevel),
IncludeSystemInfo: true,
},
)

View File

@@ -2,6 +2,7 @@ package anonymize
import (
"crypto/rand"
"encoding/base64"
"fmt"
"math/big"
"net"
@@ -15,13 +16,88 @@ import (
const anonTLD = ".domain"
// Level selects how much the anonymizer redacts. Levels are ordered: a higher
// level redacts strictly more. On the wire (protos, flags) levels travel as
// their string form.
type Level int
const (
// LevelDefault anonymizes public IP addresses, IPv6 ULA, domains, and MAC
// addresses. Internal IPv4 ranges (RFC 1918, CGNAT, link-local) are
// preserved so support can reason about the real topology.
LevelDefault Level = iota
// LevelStrict additionally anonymizes internal IP ranges, peer names, and
// WireGuard public keys.
LevelStrict
)
// LevelDefaultString and LevelStrictString are the wire forms of the levels,
// for boundaries that pass levels as strings (flags, protos, mobile bindings).
const (
LevelDefaultString = "default"
LevelStrictString = "strict"
)
// ParseLevel maps s to a Level. Empty means LevelDefault; anything
// unrecognized maps to LevelStrict so an unknown request never yields less
// anonymization than intended.
func ParseLevel(s string) Level {
switch strings.ToLower(s) {
case "", LevelDefaultString:
return LevelDefault
default:
return LevelStrict
}
}
// String returns the wire form of the level: "default" or "strict".
func (l Level) String() string {
if l >= LevelStrict {
return LevelStrictString
}
return LevelDefaultString
}
// protectedDomains are NetBird-operated suffixes that stay recognizable in an
// anonymized bundle. At LevelStrict the labels in front of them (the peer
// name) are still replaced, except under netbird.io, which only hosts
// NetBird infrastructure (api, signal, flow), never peer names.
var protectedDomains = []string{"netbird.io", "netbird.selfhosted", "netbird.cloud", "netbird.stage"}
const infraDomain = "netbird.io"
var (
macColonRegex = regexp.MustCompile(`\b[0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5}\b`)
macDashRegex = regexp.MustCompile(`\b[0-9a-fA-F]{2}(?:-[0-9a-fA-F]{2}){5}\b`)
wgKeyRegex = regexp.MustCompile(`\b[A-Za-z0-9+/]{43}=`)
)
type Anonymizer struct {
ipAnonymizer map[netip.Addr]netip.Addr
domainAnonymizer map[string]string
currentAnonIPv4 netip.Addr
currentAnonIPv6 netip.Addr
startAnonIPv4 netip.Addr
startAnonIPv6 netip.Addr
// domainOrder caches the keys of domainAnonymizer sorted longest-first
// for AnonymizeString; it is rebuilt when the map gains entries.
domainOrder []string
labelAnonymizer map[string]string
labelAnonymized map[string]struct{}
labelCounter uint32
macAnonymizer map[string]string
macCounter uint32
wgKeyAnonymizer map[string]string
wgKeyAnonymized map[string]struct{}
currentAnonIPv4 netip.Addr
currentAnonIPv6 netip.Addr
startAnonIPv4 netip.Addr
startAnonIPv6 netip.Addr
// LevelStrict also anonymizes internal ranges (RFC 1918, CGNAT,
// link-local), replacing them from the dedicated internal pools below so
// a reader can still tell an internal address from a public one.
level Level
currentAnonInternalIPv4 netip.Addr
currentAnonInternalIPv6 netip.Addr
startAnonInternalIPv4 netip.Addr
startAnonInternalIPv6 netip.Addr
domainKeyRegex *regexp.Regexp
}
@@ -32,25 +108,50 @@ func DefaultAddresses() (netip.Addr, netip.Addr) {
return netip.AddrFrom4([4]byte{198, 51, 100, 0}), netip.MustParseAddr("2001:db8:ffff::")
}
// InternalAddresses returns the pool starts used in strict mode for internal
// ranges. Both are reserved ranges that cannot collide with real addressing:
// 198.18.0.0 (RFC 2544 benchmarking), 2001:db8:1:: (RFC 3849 documentation).
func InternalAddresses() (netip.Addr, netip.Addr) {
return netip.AddrFrom4([4]byte{198, 18, 0, 0}), netip.MustParseAddr("2001:db8:1::")
}
func NewAnonymizer(startIPv4, startIPv6 netip.Addr) *Anonymizer {
internalIPv4, internalIPv6 := InternalAddresses()
return &Anonymizer{
ipAnonymizer: map[netip.Addr]netip.Addr{},
domainAnonymizer: map[string]string{},
labelAnonymizer: map[string]string{},
labelAnonymized: map[string]struct{}{},
macAnonymizer: map[string]string{},
wgKeyAnonymizer: map[string]string{},
wgKeyAnonymized: map[string]struct{}{},
currentAnonIPv4: startIPv4,
currentAnonIPv6: startIPv6,
startAnonIPv4: startIPv4,
startAnonIPv6: startIPv6,
level: LevelDefault,
currentAnonInternalIPv4: internalIPv4,
currentAnonInternalIPv6: internalIPv6,
startAnonInternalIPv4: internalIPv4,
startAnonInternalIPv6: internalIPv6,
domainKeyRegex: regexp.MustCompile(`\bdomain=([^\s,:"]+)`),
}
}
// SetLevel selects the anonymization level. The zero value of a new
// Anonymizer is LevelDefault.
func (a *Anonymizer) SetLevel(level Level) {
a.level = level
}
func (a *Anonymizer) AnonymizeIP(ip netip.Addr) netip.Addr {
// Normalize 4-in-6 addresses so ::ffff:192.168.1.1 classifies and maps
// like 192.168.1.1.
ip = ip.Unmap()
if ip.IsLoopback() ||
ip.IsLinkLocalUnicast() ||
ip.IsLinkLocalMulticast() ||
ip.IsInterfaceLocalMulticast() ||
(ip.Is4() && ip.IsPrivate()) ||
ip.IsUnspecified() ||
ip.IsMulticast() ||
isWellKnown(ip) ||
@@ -59,18 +160,100 @@ func (a *Anonymizer) AnonymizeIP(ip netip.Addr) netip.Addr {
return ip
}
if isInternal(ip) && a.level < LevelStrict {
return ip
}
if _, ok := a.ipAnonymizer[ip]; !ok {
if ip.Is4() {
a.ipAnonymizer[ip] = a.currentAnonIPv4
a.currentAnonIPv4 = a.currentAnonIPv4.Next()
} else {
a.ipAnonymizer[ip] = a.currentAnonIPv6
a.currentAnonIPv6 = a.currentAnonIPv6.Next()
}
a.ipAnonymizer[ip] = a.nextAnonIP(ip)
}
return a.ipAnonymizer[ip]
}
func (a *Anonymizer) nextAnonIP(ip netip.Addr) netip.Addr {
// At the strict level, internal addresses (including IPv6 ULA, matched
// by IsPrivate) come from the internal pools so they remain recognizable
// as internal without disclosing the real values.
if a.level >= LevelStrict && (isInternal(ip) || ip.IsPrivate()) {
if ip.Is4() {
anon := a.currentAnonInternalIPv4
a.currentAnonInternalIPv4 = a.currentAnonInternalIPv4.Next()
return anon
}
anon := a.currentAnonInternalIPv6
a.currentAnonInternalIPv6 = a.currentAnonInternalIPv6.Next()
return anon
}
if ip.Is4() {
anon := a.currentAnonIPv4
a.currentAnonIPv4 = a.currentAnonIPv4.Next()
return anon
}
anon := a.currentAnonIPv6
a.currentAnonIPv6 = a.currentAnonIPv6.Next()
return anon
}
// AnonymizeMAC replaces a MAC address with a consistent placeholder from the
// locally administered range starting at 02:00:00:00:00:01, at every
// anonymization level. Broadcast, multicast, all-zero, and already assigned
// placeholder addresses are preserved. The colon and dash spellings of the
// same address share one placeholder; the output keeps the input's separator.
func (a *Anonymizer) AnonymizeMAC(mac string) string {
hw, err := net.ParseMAC(mac)
if err != nil || len(hw) != 6 {
return mac
}
if isWellKnownMAC(hw) || a.isAnonymizedMAC(hw) {
return mac
}
key := hw.String()
anon, ok := a.macAnonymizer[key]
if !ok {
a.macCounter++
anon = fmt.Sprintf("02:00:00:%02x:%02x:%02x", byte(a.macCounter>>16), byte(a.macCounter>>8), byte(a.macCounter))
a.macAnonymizer[key] = anon
}
if strings.Contains(mac, "-") {
anon = strings.ReplaceAll(anon, ":", "-")
}
return anon
}
// isAnonymizedMAC reports whether hw is a placeholder this anonymizer already
// handed out, so a second pass over anonymized output leaves it unchanged.
func (a *Anonymizer) isAnonymizedMAC(hw net.HardwareAddr) bool {
if hw[0] != 0x02 || hw[1] != 0 || hw[2] != 0 {
return false
}
value := uint32(hw[3])<<16 | uint32(hw[4])<<8 | uint32(hw[5])
return value <= a.macCounter
}
// AnonymizeWGKey replaces a WireGuard public key with a consistent random
// placeholder of the same shape. Keys are only anonymized at LevelStrict;
// placeholders already handed out pass through unchanged.
func (a *Anonymizer) AnonymizeWGKey(key string) string {
if a.level < LevelStrict || !looksLikeWGKey(key) {
return key
}
if _, ok := a.wgKeyAnonymized[key]; ok {
return key
}
anon, ok := a.wgKeyAnonymizer[key]
if !ok {
anon = generateAnonymousKey()
a.wgKeyAnonymizer[key] = anon
a.wgKeyAnonymized[anon] = struct{}{}
}
return anon
}
func (a *Anonymizer) AnonymizeUDPAddr(addr net.UDPAddr) net.UDPAddr {
// Convert IP to netip.Addr
ip, ok := netip.AddrFromSlice(addr.IP)
@@ -89,12 +272,12 @@ func (a *Anonymizer) AnonymizeUDPAddr(addr net.UDPAddr) net.UDPAddr {
// isInAnonymizedRange checks if an IP is within the range of already assigned anonymized IPs
func (a *Anonymizer) isInAnonymizedRange(ip netip.Addr) bool {
if ip.Is4() && ip.Compare(a.startAnonIPv4) >= 0 && ip.Compare(a.currentAnonIPv4) <= 0 {
return true
} else if !ip.Is4() && ip.Compare(a.startAnonIPv6) >= 0 && ip.Compare(a.currentAnonIPv6) <= 0 {
return true
if ip.Is4() {
return inPoolRange(ip, a.startAnonIPv4, a.currentAnonIPv4) ||
inPoolRange(ip, a.startAnonInternalIPv4, a.currentAnonInternalIPv4)
}
return false
return inPoolRange(ip, a.startAnonIPv6, a.currentAnonIPv6) ||
inPoolRange(ip, a.startAnonInternalIPv6, a.currentAnonInternalIPv6)
}
func (a *Anonymizer) AnonymizeIPString(ip string) string {
@@ -118,14 +301,17 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
baseDomain = domain[:len(domain)-1]
}
if strings.HasSuffix(baseDomain, "netbird.io") ||
strings.HasSuffix(baseDomain, "netbird.selfhosted") ||
strings.HasSuffix(baseDomain, "netbird.cloud") ||
strings.HasSuffix(baseDomain, "netbird.stage") ||
strings.HasSuffix(baseDomain, anonTLD) {
if strings.HasSuffix(baseDomain, anonTLD) {
return domain
}
if suffix := protectedSuffix(baseDomain); suffix != "" {
if a.level < LevelStrict || baseDomain == suffix || suffix == infraDomain {
return domain
}
return withTrailingDot(a.anonymizePeerName(baseDomain, suffix), hasDot)
}
parts := strings.Split(baseDomain, ".")
if len(parts) < 2 {
return domain
@@ -141,12 +327,53 @@ func (a *Anonymizer) AnonymizeDomain(domain string) string {
}
result := strings.Replace(baseDomain, baseForLookup, anonymized, 1)
if hasDot {
result += "."
if a.level >= LevelStrict && len(parts) > 2 {
prefix := strings.TrimSuffix(baseDomain, "."+baseForLookup)
result = a.anonymizeLabels(prefix, "host") + "." + anonymized
// The full mapping feeds AnonymizeString so seeded FQDNs are caught
// in log lines as a whole, labels included.
a.domainAnonymizer[baseDomain] = result
}
return withTrailingDot(result, hasDot)
}
// anonymizePeerName replaces the labels in front of a protected suffix with
// numbered peer placeholders, keeping the suffix, and records the full
// mapping for string replacement in logs. The numbering keeps a peer
// recognizable across the whole bundle without disclosing its name.
func (a *Anonymizer) anonymizePeerName(baseDomain, suffix string) string {
prefix := strings.TrimSuffix(baseDomain, "."+suffix)
result := a.anonymizeLabels(prefix, "peer") + "." + suffix
if result != baseDomain {
a.domainAnonymizer[baseDomain] = result
}
return result
}
// anonymizeLabels replaces each dot-separated label with a consistent
// numbered placeholder ("<placeholder>-<n>"). Wildcard labels and
// placeholders already handed out pass through unchanged.
func (a *Anonymizer) anonymizeLabels(prefix, placeholder string) string {
labels := strings.Split(prefix, ".")
for i, label := range labels {
if label == "*" {
continue
}
if _, ok := a.labelAnonymized[label]; ok {
continue
}
anon, ok := a.labelAnonymizer[label]
if !ok {
a.labelCounter++
anon = fmt.Sprintf("%s-%d", placeholder, a.labelCounter)
a.labelAnonymizer[label] = anon
a.labelAnonymized[anon] = struct{}{}
}
labels[i] = anon
}
return strings.Join(labels, ".")
}
func (a *Anonymizer) AnonymizeURI(uri string) string {
u, err := url.Parse(uri)
if err != nil {
@@ -181,16 +408,70 @@ func (a *Anonymizer) AnonymizeString(str string) string {
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
for domain, anonDomain := range a.domainAnonymizer {
str = strings.ReplaceAll(str, domain, anonDomain)
for _, domain := range a.sortedDomains() {
str = strings.ReplaceAll(str, domain, a.domainAnonymizer[domain])
}
str = a.AnonymizeSchemeURI(str)
str = a.AnonymizeDNSLogLine(str)
// MAC handling runs after the IP passes so preserved IPv6 addresses are
// already out of the way; the separator guard skips matches embedded in a
// longer colon- or dash-separated sequence (such as an IPv6 tail).
str = a.anonymizeMACsInString(str, macColonRegex, ':')
str = a.anonymizeMACsInString(str, macDashRegex, '-')
if a.level >= LevelStrict {
str = wgKeyRegex.ReplaceAllStringFunc(str, a.AnonymizeWGKey)
}
return str
}
// sortedDomains returns the domain mappings longest-first, so a full-FQDN
// mapping (strict level) is applied before the base-domain mapping it
// contains. The order is rebuilt only when domainAnonymizer has grown.
func (a *Anonymizer) sortedDomains() []string {
if len(a.domainOrder) == len(a.domainAnonymizer) {
return a.domainOrder
}
a.domainOrder = a.domainOrder[:0]
for domain := range a.domainAnonymizer {
a.domainOrder = append(a.domainOrder, domain)
}
slices.SortFunc(a.domainOrder, func(x, y string) int {
if d := len(y) - len(x); d != 0 {
return d
}
return strings.Compare(x, y)
})
return a.domainOrder
}
// anonymizeMACsInString replaces MAC addresses matched by re, skipping
// matches that directly adjoin another sep so a six-group run inside a longer
// separated sequence is left alone.
func (a *Anonymizer) anonymizeMACsInString(str string, re *regexp.Regexp, sep byte) string {
matches := re.FindAllStringIndex(str, -1)
if len(matches) == 0 {
return str
}
var b strings.Builder
last := 0
for _, m := range matches {
if (m[0] > 0 && str[m[0]-1] == sep) || (m[1] < len(str) && str[m[1]] == sep) {
continue
}
b.WriteString(str[last:m[0]])
b.WriteString(a.AnonymizeMAC(str[m[0]:m[1]]))
last = m[1]
}
b.WriteString(str[last:])
return b.String()
}
// AnonymizeSchemeURI finds and anonymizes URIs with ws, wss, rel, rels, stun, stuns, turn, and turns schemes.
func (a *Anonymizer) AnonymizeSchemeURI(text string) string {
re := regexp.MustCompile(`(?i)\b(wss?://|rels?://|stuns?:|turns?:|https?://)\S+\b`)
@@ -239,10 +520,79 @@ func isWellKnown(addr netip.Addr) bool {
"128.0.0.0", "8000::", // 2nd split subnet for default routes
}
if slices.Contains(wellKnown, addr.String()) {
return slices.Contains(wellKnown, addr.String())
}
// isInternal reports whether ip identifies a host only within the local
// network: IPv4 private (RFC 1918), CGNAT (RFC 6598), and link-local (v4 and
// v6). These are preserved at the default level so support can reason about
// the real topology, and replaced from the internal pools at the strict
// level. IPv6 ULA is deliberately not internal: its random global ID uniquely
// fingerprints the network, so it is anonymized at every level.
func isInternal(ip netip.Addr) bool {
return (ip.Is4() && ip.IsPrivate()) ||
ip.IsLinkLocalUnicast() ||
isCGNAT(ip)
}
func inPoolRange(ip, start, current netip.Addr) bool {
return ip.Compare(start) >= 0 && ip.Compare(current) <= 0
}
// isWellKnownMAC reports whether hw carries no stable host identity: all-zero
// or a group address (broadcast and multicast).
func isWellKnownMAC(hw net.HardwareAddr) bool {
if hw[0]&1 == 1 {
return true
}
for _, b := range hw {
if b != 0 {
return false
}
}
return true
}
// looksLikeWGKey reports whether s has the shape of a WireGuard key:
// 44 base64 characters decoding to 32 bytes.
func looksLikeWGKey(s string) bool {
if len(s) != 44 || s[43] != '=' {
return false
}
decoded, err := base64.StdEncoding.DecodeString(s)
return err == nil && len(decoded) == 32
}
func generateAnonymousKey() string {
buf := make([]byte, 32)
if _, err := rand.Read(buf); err != nil {
return strings.Repeat("A", 43) + "="
}
return base64.StdEncoding.EncodeToString(buf)
}
// protectedSuffix returns the protected NetBird suffix baseDomain ends with,
// or empty. The match is label-anchored so an unrelated domain that merely
// ends in the same characters is not preserved.
func protectedSuffix(baseDomain string) string {
for _, d := range protectedDomains {
if baseDomain == d || strings.HasSuffix(baseDomain, "."+d) {
return d
}
}
return ""
}
func withTrailingDot(domain string, hasDot bool) string {
if hasDot {
return domain + "."
}
return domain
}
// isCGNAT reports whether addr is in 100.64.0.0/10 (RFC 6598), the range
// NetBird assigns overlay peer addresses from.
func isCGNAT(addr netip.Addr) bool {
cgnatRangeStart := netip.AddrFrom4([4]byte{100, 64, 0, 0})
cgnatRange := netip.PrefixFrom(cgnatRangeStart, 10)

View File

@@ -1,8 +1,11 @@
package anonymize_test
import (
"bytes"
"encoding/base64"
"net/netip"
"regexp"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -44,6 +47,301 @@ func TestAnonymizeIP(t *testing.T) {
}
}
func TestParseLevel(t *testing.T) {
tests := []struct {
input string
expect anonymize.Level
}{
{"", anonymize.LevelDefault},
{"default", anonymize.LevelDefault},
{"DEFAULT", anonymize.LevelDefault},
{"strict", anonymize.LevelStrict},
{"STRICT", anonymize.LevelStrict},
// Unknown values must never yield less anonymization than requested.
{"garbage", anonymize.LevelStrict},
}
for _, tc := range tests {
t.Run("input="+tc.input, func(t *testing.T) {
assert.Equal(t, tc.expect, anonymize.ParseLevel(tc.input), "parsed level should match")
})
}
}
func TestAnonymizeIP_DefaultLevelInternalRanges(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
tests := []struct {
name string
ip string
expect string
}{
{"RFC1918 10/8", "10.1.2.3", "10.1.2.3"},
{"RFC1918 172.16/12", "172.16.5.5", "172.16.5.5"},
{"RFC1918 192.168/16", "192.168.1.1", "192.168.1.1"},
{"CGNAT", "100.64.0.5", "100.64.0.5"},
{"IPv4 link-local", "169.254.1.1", "169.254.1.1"},
{"IPv6 link-local", "fe80::1", "fe80::1"},
// ULA is anonymized even at the default level: its random global ID
// uniquely fingerprints the network, unlike shared RFC 1918 space.
{"IPv6 ULA", "fd12:3456:789a::1", "2001:db8:ffff::"},
// 4-in-6 addresses classify like their unmapped IPv4 form.
{"4-in-6 RFC1918", "::ffff:192.168.1.1", "192.168.1.1"},
{"4-in-6 CGNAT", "::ffff:100.64.0.5", "100.64.0.5"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeIP(netip.MustParseAddr(tc.ip))
assert.Equal(t, tc.expect, result.String(), "default level should preserve internal ranges except ULA")
})
}
}
func TestAnonymizeIP_StrictLevel(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
// Order matters: internal pool addresses are assigned sequentially.
tests := []struct {
name string
ip string
expect string
}{
{"RFC1918 192.168/16", "192.168.1.1", "198.18.0.0"},
{"Second RFC1918", "192.168.1.2", "198.18.0.1"},
{"Repeated RFC1918", "192.168.1.1", "198.18.0.0"},
{"RFC1918 10/8", "10.1.2.3", "198.18.0.2"},
{"RFC1918 172.16/12", "172.16.5.5", "198.18.0.3"},
{"CGNAT", "100.64.0.5", "198.18.0.4"},
{"IPv4 link-local", "169.254.1.1", "198.18.0.5"},
{"Public IPv4 uses public pool", "1.2.3.4", "198.51.100.0"},
{"IPv6 link-local", "fe80::1", "2001:db8:1::"},
{"IPv6 ULA", "fd12:3456:789a::1", "2001:db8:1::1"},
{"Public IPv6 uses public pool", "2607:f8b0:4005:805::200e", "2001:db8:ffff::"},
{"Loopback IPv4", "127.0.0.1", "127.0.0.1"},
{"Loopback IPv6", "::1", "::1"},
{"Unspecified", "0.0.0.0", "0.0.0.0"},
{"Multicast", "224.0.0.251", "224.0.0.251"},
{"Well known resolver", "8.8.8.8", "8.8.8.8"},
{"Well known split marker", "128.0.0.0", "128.0.0.0"},
{"In internal pool range", "198.18.0.3", "198.18.0.3"},
{"In public pool range", "198.51.100.0", "198.51.100.0"},
{"4-in-6 repeated RFC1918", "::ffff:192.168.1.1", "198.18.0.0"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeIP(netip.MustParseAddr(tc.ip))
assert.Equal(t, tc.expect, result.String(), "strict level should replace internal ranges from the internal pools")
})
}
}
func TestAnonymizeString_StrictInternalIPs(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
input := "route 10.20.30.0/24 via 192.168.1.1 dev eth0 src 100.64.0.7"
firstPass := anonymizer.AnonymizeString(input)
secondPass := anonymizer.AnonymizeString(firstPass)
assert.NotContains(t, firstPass, "10.20.30.0", "private network address should be anonymized")
assert.NotContains(t, firstPass, "192.168.1.1", "private gateway should be anonymized")
assert.NotContains(t, firstPass, "100.64.0.7", "CGNAT address should be anonymized")
assert.Contains(t, firstPass, "/24", "prefix length should be preserved")
assert.Equal(t, firstPass, secondPass, "second pass should not further anonymize the string")
}
func TestAnonymizeMAC(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
first := anonymizer.AnonymizeMAC("aa:bb:cc:dd:ee:0f")
assert.Equal(t, "02:00:00:00:00:01", first, "first MAC should get the first placeholder")
assert.Equal(t, first, anonymizer.AnonymizeMAC("aa:bb:cc:dd:ee:0f"), "repeated MAC should map to the same placeholder")
assert.Equal(t, first, anonymizer.AnonymizeMAC("AA:BB:CC:DD:EE:0F"), "case should not affect the mapping")
assert.Equal(t, "02-00-00-00-00-01", anonymizer.AnonymizeMAC("AA-BB-CC-DD-EE-0F"), "dash form should keep its separator but share the mapping")
second := anonymizer.AnonymizeMAC("10:22:33:44:55:66")
assert.Equal(t, "02:00:00:00:00:02", second, "second distinct MAC should get the next placeholder")
tests := []struct {
name string
mac string
}{
{"Broadcast", "ff:ff:ff:ff:ff:ff"},
{"IPv4 multicast", "01:00:5e:00:00:fb"},
{"IPv6 multicast", "33:33:00:00:00:01"},
{"All zero", "00:00:00:00:00:00"},
{"Assigned placeholder", "02:00:00:00:00:01"},
{"Invalid", "not-a-mac"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.mac, anonymizer.AnonymizeMAC(tc.mac), "should be preserved")
})
}
}
func TestAnonymizeString_MACAddresses(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
tests := []struct {
name string
input string
expect string
}{
{
name: "nftables ether rule",
input: "ether saddr aa:bb:cc:dd:ee:ff drop",
expect: "ether saddr 02:00:00:00:00:01 drop",
},
{
name: "Windows dash form",
input: "Physical Address : AA-BB-CC-DD-EE-FF",
expect: "Physical Address : 02-00-00-00-00-01",
},
{
name: "IPv6 address tail is not treated as MAC",
input: "addr fe80:0:11:22:33:44:55:66 scope link",
expect: "addr fe80:0:11:22:33:44:55:66 scope link",
},
{
name: "broadcast MAC preserved",
input: "dst ff:ff:ff:ff:ff:ff type ARP",
expect: "dst ff:ff:ff:ff:ff:ff type ARP",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeString(tc.input)
assert.Equal(t, tc.expect, result, "MAC addresses should be anonymized at every level")
assert.Equal(t, result, anonymizer.AnonymizeString(result), "second pass should not change the result")
})
}
}
func TestAnonymizeWGKey(t *testing.T) {
key := base64.StdEncoding.EncodeToString(bytes.Repeat([]byte{0x42}, 32))
t.Run("default level preserves keys", func(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
assert.Equal(t, key, anonymizer.AnonymizeWGKey(key), "default level should not touch WireGuard keys")
})
t.Run("strict level replaces keys", func(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
anon := anonymizer.AnonymizeWGKey(key)
assert.NotEqual(t, key, anon, "strict level should replace the key")
assert.Regexp(t, `^[A-Za-z0-9+/]{43}=$`, anon, "placeholder should keep the WireGuard key shape")
assert.Equal(t, anon, anonymizer.AnonymizeWGKey(key), "repeated key should map to the same placeholder")
assert.Equal(t, anon, anonymizer.AnonymizeWGKey(anon), "an assigned placeholder should pass through unchanged")
assert.Equal(t, "not-a-key", anonymizer.AnonymizeWGKey("not-a-key"), "non-key values should be preserved")
})
}
func TestAnonymizeString_WGKeys(t *testing.T) {
key := base64.StdEncoding.EncodeToString(bytes.Repeat([]byte{0x42}, 32))
input := "peer " + key + " handshake completed"
t.Run("default level preserves keys", func(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
assert.Equal(t, input, anonymizer.AnonymizeString(input), "default level should not touch WireGuard keys in strings")
})
t.Run("strict level replaces keys", func(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
firstPass := anonymizer.AnonymizeString(input)
assert.NotContains(t, firstPass, key, "the key should not survive strict anonymization")
assert.Equal(t, anonymizer.AnonymizeWGKey(key), extractKey(t, firstPass), "string replacement should be consistent with AnonymizeWGKey")
assert.Equal(t, firstPass, anonymizer.AnonymizeString(firstPass), "second pass should not change the result")
})
}
func extractKey(t *testing.T, logLine string) string {
t.Helper()
fields := strings.Fields(logLine)
require.Len(t, fields, 4, "log line should keep its structure")
return fields[1]
}
func TestAnonymizeDomain_StrictLevel(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
t.Run("netbird peer name", func(t *testing.T) {
result := anonymizer.AnonymizeDomain("my-laptop.netbird.cloud")
assert.Regexp(t, `^peer-\d+\.netbird\.cloud$`, result, "peer name should be anonymized, suffix kept")
assert.NotContains(t, result, "my-laptop", "the peer name should not survive")
assert.Equal(t, result, anonymizer.AnonymizeDomain("my-laptop.netbird.cloud"), "repeated domain should map consistently")
assert.Equal(t, result, anonymizer.AnonymizeDomain(result), "an anonymized domain should pass through unchanged")
})
t.Run("bare netbird domain", func(t *testing.T) {
assert.Equal(t, "netbird.cloud", anonymizer.AnonymizeDomain("netbird.cloud"), "the bare protected suffix should be preserved")
})
t.Run("netbird infrastructure preserved", func(t *testing.T) {
assert.Equal(t, "api.netbird.io", anonymizer.AnonymizeDomain("api.netbird.io"),
"netbird.io hosts infrastructure, not peer names, and should stay readable")
})
t.Run("leading labels of other domains", func(t *testing.T) {
result := anonymizer.AnonymizeDomain("host1.corp.example.com")
assert.Regexp(t, `^host-\d+\.host-\d+\.anon-[a-zA-Z0-9]+\.domain$`, result, "every label should be anonymized")
for _, label := range []string{"host1", "corp", "example"} {
assert.NotContains(t, result, label, "no original label should survive")
}
assert.Equal(t, result, anonymizer.AnonymizeDomain("host1.corp.example.com"), "repeated domain should map consistently")
})
t.Run("same label maps consistently across domains", func(t *testing.T) {
first := anonymizer.AnonymizeDomain("shared.one.com")
second := anonymizer.AnonymizeDomain("shared.two.com")
assert.Equal(t, strings.Split(first, ".")[0], strings.Split(second, ".")[0], "the shared host label should get one placeholder")
})
t.Run("wildcard label preserved", func(t *testing.T) {
result := anonymizer.AnonymizeDomain("*.example.com")
assert.Regexp(t, `^\*\.anon-[a-zA-Z0-9]+\.domain$`, result, "the wildcard label should stay a wildcard")
})
}
func TestAnonymizeDomain_DefaultLevelKeepsPeerNames(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
assert.Equal(t, "my-laptop.netbird.cloud", anonymizer.AnonymizeDomain("my-laptop.netbird.cloud"),
"default level should preserve netbird FQDNs including the peer name")
assert.Regexp(t, `^sub\.anon-[a-zA-Z0-9]+\.domain$`, anonymizer.AnonymizeDomain("sub.example.com"),
"default level should keep subdomain labels")
}
func TestAnonymizeString_StrictPeerNames(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(anonymize.LevelStrict)
// Seed like the bundle generator does from the status: base first, then
// the full FQDN, so replacement must prefer the longer mapping.
anonBase := anonymizer.AnonymizeDomain("example.com")
anonPeer := anonymizer.AnonymizeDomain("peer1.netbird.cloud")
anonHost := anonymizer.AnonymizeDomain("host1.example.com")
logLine := "connected to peer1.netbird.cloud via host1.example.com endpoint"
firstPass := anonymizer.AnonymizeString(logLine)
assert.NotContains(t, firstPass, "peer1", "the peer name should not survive in logs")
assert.NotContains(t, firstPass, "host1", "the host label should not survive in logs")
assert.Contains(t, firstPass, anonPeer, "the seeded peer mapping should be applied")
assert.Contains(t, firstPass, anonHost, "the seeded host mapping should be applied, not just the base mapping")
assert.NotContains(t, firstPass, "host1."+anonBase, "the base mapping must not preempt the longer FQDN mapping")
assert.Equal(t, firstPass, anonymizer.AnonymizeString(firstPass), "second pass should not change the result")
}
func TestAnonymizeDNSLogLine(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
tests := []struct {

View File

@@ -27,8 +27,8 @@ import (
const errCloseConnection = "Failed to close connection: %v"
var (
logFileCount uint32
systemInfoFlag bool
logFileCount uint32
systemInfoFlag bool
uploadBundleFlag bool
uploadBundleURLFlag string
uploadBundleInsecureFlag bool
@@ -156,6 +156,11 @@ func debugConfigDump(cmd *cobra.Command, _ []string) error {
// request. Returns an error if the RPC fails or if the daemon reports
// an upload failure reason.
func debugBundle(cmd *cobra.Command, _ []string) error {
anonymizeEnabled, anonymizeLevel, err := effectiveAnonymize()
if err != nil {
return err
}
conn, err := getClient(cmd)
if err != nil {
return err
@@ -168,10 +173,11 @@ func debugBundle(cmd *cobra.Command, _ []string) error {
client := proto.NewDaemonServiceClient(conn)
request := &proto.DebugBundleRequest{
Anonymize: anonymizeFlag,
SystemInfo: systemInfoFlag,
LogFileCount: logFileCount,
CliVersion: version.NetbirdVersion(),
Anonymize: anonymizeEnabled,
AnonymizeLevel: anonymizeLevel.String(),
SystemInfo: systemInfoFlag,
LogFileCount: logFileCount,
CliVersion: version.NetbirdVersion(),
}
if uploadBundleFlag {
request.UploadURL = uploadBundleURLFlag
@@ -229,6 +235,11 @@ func runForDuration(cmd *cobra.Command, args []string) error {
return fmt.Errorf("invalid duration format: %v", err)
}
anonymizeEnabled, anonymizeLevel, err := effectiveAnonymize()
if err != nil {
return err
}
conn, err := getClient(cmd)
if err != nil {
return err
@@ -368,10 +379,11 @@ func runForDuration(cmd *cobra.Command, args []string) error {
cmd.Println("Creating debug bundle...")
request := &proto.DebugBundleRequest{
Anonymize: anonymizeFlag,
SystemInfo: systemInfoFlag,
LogFileCount: logFileCount,
CliVersion: version.NetbirdVersion(),
Anonymize: anonymizeEnabled,
AnonymizeLevel: anonymizeLevel.String(),
SystemInfo: systemInfoFlag,
LogFileCount: logFileCount,
CliVersion: version.NetbirdVersion(),
}
if uploadBundleFlag {
request.UploadURL = uploadBundleURLFlag

View File

@@ -21,6 +21,7 @@ import (
"github.com/spf13/pflag"
"google.golang.org/grpc"
"github.com/netbirdio/netbird/client/anonymize"
daddr "github.com/netbirdio/netbird/client/internal/daemonaddr"
"github.com/netbirdio/netbird/client/internal/profilemanager"
)
@@ -69,6 +70,7 @@ var (
autoConnectDisabled bool
extraIFaceBlackList []string
anonymizeFlag bool
anonymizeLevelFlag string
dnsRouteInterval time.Duration
// lazyConnEnabled is the parse target for the deprecated --enable-lazy-connection
// flag. The flag is inert; the value is no longer read (use NB_LAZY_CONN instead).
@@ -156,7 +158,8 @@ func init() {
rootCmd.MarkFlagsMutuallyExclusive("setup-key", "setup-key-file")
rootCmd.PersistentFlags().StringVar(&preSharedKey, preSharedKeyFlag, "", "Sets WireGuard PreSharedKey property. If set, then only peers that have the same key can communicate.")
rootCmd.PersistentFlags().StringVarP(&hostName, "hostname", "n", "", "Sets a custom hostname for the device")
rootCmd.PersistentFlags().BoolVarP(&anonymizeFlag, "anonymize", "A", false, "anonymize IP addresses and non-netbird.io domains in logs and status output")
rootCmd.PersistentFlags().BoolVarP(&anonymizeFlag, "anonymize", "A", false, "anonymize public IP addresses, MAC addresses, and non-netbird.io domains in logs and status output; private, CGNAT, and link-local IP ranges are kept (see --anonymize-level strict)")
rootCmd.PersistentFlags().StringVar(&anonymizeLevelFlag, "anonymize-level", "", "anonymization level: \"default\" or \"strict\"; strict also anonymizes private, CGNAT, and link-local IP ranges, peer names, and WireGuard public keys. Setting this flag implies --anonymize")
rootCmd.PersistentFlags().StringVarP(&configPath, "config", "c", profilemanager.DefaultConfigPath, "Overrides the default profile file location")
rootCmd.AddCommand(upCmd)
@@ -293,6 +296,19 @@ var CLIBackOffSettings = &backoff.ExponentialBackOff{
Clock: backoff.SystemClock,
}
// effectiveAnonymize resolves the --anonymize and --anonymize-level flags:
// setting a level implies anonymization, and an invalid level is rejected.
func effectiveAnonymize() (bool, anonymize.Level, error) {
if anonymizeLevelFlag == "" {
return anonymizeFlag, anonymize.LevelDefault, nil
}
level := anonymize.ParseLevel(anonymizeLevelFlag)
if !strings.EqualFold(anonymizeLevelFlag, level.String()) {
return false, anonymize.LevelDefault, fmt.Errorf("invalid anonymize level %q: use %q or %q", anonymizeLevelFlag, anonymize.LevelDefault.String(), anonymize.LevelStrict.String())
}
return true, level, nil
}
func getSetupKey() (string, error) {
if setupKeyPath != "" && setupKey == "" {
return getSetupKeyFromFile(setupKeyPath)

View File

@@ -121,8 +121,14 @@ func statusFunc(cmd *cobra.Command, args []string) error {
sessionExpiresAt = ts.AsTime().UTC()
}
anonymizeEnabled, anonymizeLevel, err := effectiveAnonymize()
if err != nil {
return err
}
var outputInformationHolder = nbstatus.ConvertToStatusOutputOverview(resp.GetFullStatus(), nbstatus.ConvertOptions{
Anonymize: anonymizeFlag,
Anonymize: anonymizeEnabled,
AnonymizeLevel: anonymizeLevel,
DaemonVersion: resp.GetDaemonVersion(),
DaemonStatus: nbstatus.ParseDaemonStatus(status),
StatusFilter: statusFilter,

View File

@@ -124,7 +124,7 @@ func startManagement(t *testing.T, config *config.Config, testFile string) (*grp
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := mgmt.NewAccountRequestBuffer(ctx, store)
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config)
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config, nil)
accountManager, err := mgmt.BuildManager(ctx, config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
if err != nil {

View File

@@ -21,8 +21,8 @@ import (
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
nbnet "github.com/netbirdio/netbird/client/net"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/shared/management/domain"

View File

@@ -146,7 +146,7 @@ func startManagement(t *testing.T, signalAddr string) string {
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := mgmt.NewAccountRequestBuffer(context.Background(), testStore)
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg)
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg, nil)
accountManager, err := mgmt.BuildManager(context.Background(), cfg, testStore, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
require.NoError(t, err)

View File

@@ -22,6 +22,16 @@ import (
nbnet "github.com/netbirdio/netbird/client/net"
)
const (
// wgMsgTypeHandshakeInitiation is the lowest WireGuard message type.
wgMsgTypeHandshakeInitiation uint32 = 1
// wgMsgTypeTransport is the highest WireGuard message type.
wgMsgTypeTransport uint32 = 4
// wgMinMsgSize is the smallest WireGuard message: transport data with an empty
// payload, which is what a keepalive is.
wgMinMsgSize = 32
)
type receiverCreator struct {
iceBind *ICEBind
}
@@ -216,8 +226,15 @@ func (s *ICEBind) createReceiverFn(pc wgConn.BatchReader, conn *net.UDPConn, rxO
for i := 0; i < numMsgs; i++ {
msg := &(*msgs)[i]
// todo: handle err
if ok, _ := s.filterOutStunMessages(msg.Buffers, msg.N, msg.Addr); ok {
if ok, err := s.filterOutStunMessages(msg.Buffers, msg.N, msg.Addr); ok {
if err != nil {
log.Debugf("failed to handle STUN packet from %s: %v", msg.Addr, err)
}
// WireGuard reuses sizes and eps across reads and only skips a slot
// whose size is below the minimum message size. Leaving a consumed
// slot untouched makes it process this buffer again under the
// previous packet's length and endpoint.
sizes[i] = 0
continue
}
sizes[i] = msg.N
@@ -271,11 +288,16 @@ func (s *ICEBind) createOrUpdateMux() {
func (s *ICEBind) filterOutStunMessages(buffers [][]byte, n int, addr net.Addr) (bool, error) {
for i := range buffers {
if !stun.IsMessage(buffers[i]) {
if n > len(buffers[i]) {
continue
}
pkt := buffers[i][:n]
if isWireGuardMsg(pkt) || !stun.IsMessage(pkt) {
continue
}
msg, err := s.parseSTUNMessage(buffers[i][:n])
msg, err := s.parseSTUNMessage(pkt)
if err != nil {
buffers[i] = []byte{}
return true, err
@@ -347,18 +369,34 @@ func putMessages(msgs *[]ipv6.Message, msgsPool *sync.Pool) {
msgsPool.Put(msgs)
}
func isTransportPkg(buffers [][]byte, n int) bool {
// The first buffer should contain at least 4 bytes for type
if len(buffers[0]) < 4 {
return true
// isWireGuardMsg reports whether the packet carries a WireGuard message header: a
// little-endian uint32 message type in the range 1..4, which leaves the three bytes
// after the type byte zero, in a packet long enough to hold any WireGuard message.
//
// A well formed STUN message cannot take that shape. Its length field sits in the two
// bytes the type must leave zero, and for a message of at least wgMinMsgSize bytes that
// field holds at least 12, so the two framings do not overlap. The test has to be this
// tight because stun.IsMessage only looks at the magic cookie, which in a WireGuard
// message overlaps the receiver index: a session whose index happens to equal the cookie
// would otherwise have all of its inbound data misrouted to the STUN handler until the
// next rekey.
func isWireGuardMsg(pkt []byte) bool {
if len(pkt) < wgMinMsgSize {
return false
}
// WireGuard packet type is a little-endian uint32 at start
packetType := binary.LittleEndian.Uint32(buffers[0][:4])
// Check if packetType matches known WireGuard message types
if packetType == 4 && n > 32 {
return true
}
return false
msgType := binary.LittleEndian.Uint32(pkt[:4])
return msgType >= wgMsgTypeHandshakeInitiation && msgType <= wgMsgTypeTransport
}
// isTransportPkg reports whether the packet is WireGuard transport data carrying a
// payload, which is what counts as peer activity. A keepalive holds no payload and is
// exactly wgMinMsgSize bytes.
func isTransportPkg(buffers [][]byte, n int) bool {
if n < 4 || n > len(buffers[0]) {
return false
}
msgType := binary.LittleEndian.Uint32(buffers[0][:4])
return msgType == wgMsgTypeTransport && n > wgMinMsgSize
}

View File

@@ -0,0 +1,215 @@
//go:build !js
package bind
import (
"encoding/binary"
"net"
"testing"
"time"
"github.com/pion/stun/v3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/net/ipv4"
wgConn "golang.zx2c4.com/wireguard/conn"
)
// magicCookieBytes is the STUN magic cookie as it appears on the wire. In a
// WireGuard message the same offset holds the receiver (or sender) index, which is
// a random uint32, so a session can draw exactly this value.
var magicCookieBytes = []byte{0x21, 0x12, 0xA4, 0x42}
const testBufSize = 1500
// wgMsg builds a WireGuard message of the given type and size, with the index field
// at bytes 4:8 set to index.
func wgMsg(msgType uint32, size int, index []byte) []byte {
pkt := make([]byte, size)
binary.LittleEndian.PutUint32(pkt[:4], msgType)
copy(pkt[4:8], index)
return pkt
}
// intoBuffer copies pkt into a full-size receive buffer, the way the kernel read
// does, so tests see the same buffer/length split as the hot path.
func intoBuffer(pkt []byte) [][]byte {
buf := make([]byte, testBufSize)
copy(buf, pkt)
return [][]byte{buf}
}
func TestFilterOutStunMessages_PassesWireGuardWithCookieShapedIndex(t *testing.T) {
tests := []struct {
name string
msgType uint32
size int
}{
{"transport data", wgMsgTypeTransport, 128},
{"keepalive", wgMsgTypeTransport, wgMinMsgSize},
{"handshake initiation", wgMsgTypeHandshakeInitiation, 148},
{"handshake response", 2, 92},
{"cookie reply", 3, 64},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
pkt := wgMsg(tc.msgType, tc.size, magicCookieBytes)
require.True(t, stun.IsMessage(pkt), "precondition: pion sees this as STUN")
buffers := intoBuffer(pkt)
bind := &ICEBind{}
filtered, err := bind.filterOutStunMessages(buffers, tc.size, &net.UDPAddr{})
assert.NoError(t, err)
assert.False(t, filtered, "WireGuard message must be handed to WireGuard, not the STUN handler")
assert.Len(t, buffers[0], testBufSize, "buffer must be left intact for WireGuard")
})
}
}
func TestFilterOutStunMessages_FiltersRealSTUNMessage(t *testing.T) {
msg, err := stun.Build(stun.BindingRequest, stun.TransactionID, stun.Fingerprint)
require.NoError(t, err)
buffers := intoBuffer(msg.Raw)
bind := &ICEBind{}
filtered, err := bind.filterOutStunMessages(buffers, len(msg.Raw), &net.UDPAddr{})
assert.NoError(t, err)
assert.True(t, filtered, "STUN message must be consumed by the STUN handler")
assert.Empty(t, buffers[0], "consumed buffer must be emptied so WireGuard does not see it")
}
// TestIsWireGuardMsg_DisjointFromSTUN locks the invariant the filter relies on: a
// well formed STUN message long enough to be a WireGuard message always has a
// non-zero length field, so it cannot be mistaken for a WireGuard header.
func TestIsWireGuardMsg_DisjointFromSTUN(t *testing.T) {
types := []stun.MessageType{
stun.BindingRequest,
stun.BindingSuccess,
stun.BindingError,
{Method: stun.MethodBinding, Class: stun.ClassIndication},
}
for _, msgType := range types {
// Long enough that the length guard is not what makes this pass.
msg, err := stun.Build(msgType, stun.TransactionID,
stun.NewUsername("remoteUfrag:localUfrag"), stun.Fingerprint)
require.NoError(t, err)
require.GreaterOrEqual(t, len(msg.Raw), wgMinMsgSize, "precondition: %s", msgType)
assert.False(t, isWireGuardMsg(msg.Raw),
"%s must not look like a WireGuard message", msgType)
}
}
func TestIsWireGuardMsg(t *testing.T) {
tests := []struct {
name string
pkt []byte
want bool
}{
{"transport data", wgMsg(wgMsgTypeTransport, 128, nil), true},
{"handshake initiation", wgMsg(wgMsgTypeHandshakeInitiation, 148, nil), true},
{"unknown type 5", wgMsg(5, 128, nil), false},
{"type 0", wgMsg(0, 128, nil), false},
{"non-zero reserved byte", []byte{0x04, 0x00, 0x01, 0x00}, false},
{"too short", []byte{0x04, 0x00, 0x00}, false},
{"empty", nil, false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, isWireGuardMsg(tc.pkt), "wrong classification for %s", tc.name)
})
}
}
// TestFilterOutStunMessages_IgnoresBytesBeyondPacket guards against classifying on
// buffer contents left over from an earlier, longer packet.
func TestFilterOutStunMessages_IgnoresBytesBeyondPacket(t *testing.T) {
buf := make([]byte, testBufSize)
copy(buf[4:8], magicCookieBytes)
buffers := [][]byte{buf}
bind := &ICEBind{}
filtered, err := bind.filterOutStunMessages(buffers, 2, &net.UDPAddr{})
assert.NoError(t, err)
assert.False(t, filtered, "a 2 byte packet must not be classified from stale buffer bytes")
}
// TestReceiveFn_ClearsSizeOfConsumedPacket covers the accounting WireGuard relies
// on: sizes is reused across reads, so a slot whose packet was consumed as STUN must
// be reported as empty. Otherwise WireGuard reprocesses the same buffer under the
// previous packet's length, which for a WireGuard-shaped packet means it is handled
// twice.
func TestReceiveFn_ClearsSizeOfConsumedPacket(t *testing.T) {
conn := listenUDP(t, "udp4", "127.0.0.1:0")
defer conn.Close()
recvFn := receiverCreator{setupICEBind(t)}.CreateReceiverFn(
ipv4.NewPacketConn(conn), conn, false, createMsgPool(),
)
msg, err := stun.Build(stun.BindingRequest, stun.TransactionID, stun.Fingerprint)
require.NoError(t, err)
sender := listenUDP(t, "udp4", "127.0.0.1:0")
defer sender.Close()
_, err = sender.WriteTo(msg.Raw, conn.LocalAddr())
require.NoError(t, err)
require.NoError(t, conn.SetReadDeadline(time.Now().Add(3*time.Second)))
bufs := [][]byte{make([]byte, 1500)}
// A leftover size from an earlier read, which is what makes the missing reset
// observable.
sizes := []int{148}
eps := make([]wgConn.Endpoint, 1)
n, err := recvFn(bufs, sizes, eps)
require.NoError(t, err)
require.Equal(t, 1, n)
assert.Zero(t, sizes[0], "consumed STUN packet must not leave a size behind for WireGuard")
}
func TestIsTransportPkg(t *testing.T) {
tests := []struct {
name string
pkt []byte
n int
want bool
}{
{"transport data with payload", wgMsg(wgMsgTypeTransport, 128, nil), 128, true},
{"keepalive", wgMsg(wgMsgTypeTransport, wgMinMsgSize, nil), wgMinMsgSize, false},
{"handshake initiation", wgMsg(wgMsgTypeHandshakeInitiation, 148, nil), 148, false},
{"stale type bytes beyond packet", wgMsg(wgMsgTypeTransport, 128, nil), 2, false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, isTransportPkg(intoBuffer(tc.pkt), tc.n),
"wrong activity classification for %s", tc.name)
})
}
}
// TestFilterOutStunMessages_ConsumesSTUNWithWireGuardShapedType covers the one STUN
// encoding whose leading bytes collide with a WireGuard message type: method 0x080 as a
// request encodes to 0x0200, so the type byte reads as a handshake response and the byte
// after it is zero. Only the length check keeps such a message out of WireGuard's hands.
// pion implements no method in that range, so this is a synthetic worst case rather than
// traffic ICE produces.
func TestFilterOutStunMessages_ConsumesSTUNWithWireGuardShapedType(t *testing.T) {
msg, err := stun.Build(stun.NewType(stun.Method(0x080), stun.ClassRequest), stun.TransactionID)
require.NoError(t, err)
require.Equal(t, []byte{0x02, 0x00, 0x00, 0x00}, msg.Raw[:4],
"precondition: the leading bytes read as a WireGuard message type")
buffers := intoBuffer(msg.Raw)
bind := &ICEBind{}
filtered, err := bind.filterOutStunMessages(buffers, len(msg.Raw), &net.UDPAddr{})
assert.NoError(t, err)
assert.True(t, filtered, "STUN message must be consumed despite its WireGuard-shaped type")
}

View File

@@ -34,9 +34,8 @@ import (
"github.com/netbirdio/netbird/shared/netiputil"
)
const readmeContent = `Netbird debug bundle
This debug bundle contains the following files.
If the --anonymize flag is set, the files are anonymized to protect sensitive information.
const readmeContent = `This debug bundle contains the following files.
If anonymization is enabled (--anonymize / --anonymize-level), the files are anonymized to protect sensitive information.
status.txt: Anonymized status information of the NetBird client.
client.log: Most recent, anonymized client log file of the NetBird client.
@@ -70,21 +69,34 @@ capture.pcap: Packet capture in pcap format. Only present when capture was runni
Anonymization Process
The files in this bundle have been anonymized to protect sensitive information. Here's how the anonymization was applied:
The files in this bundle have been anonymized to protect sensitive information. The level applied to this bundle is recorded at the top of this file. Here's how the anonymization was applied:
IP Addresses
IPv4 addresses are replaced with addresses starting from 198.51.100.0
IPv6 addresses are replaced with addresses starting from 100::
Default level:
- Public IPv4 addresses are replaced with addresses starting from 198.51.100.0
- Public IPv6 addresses are replaced with addresses starting from 2001:db8:ffff::
- IPv6 unique local addresses (fc00::/7) are anonymized as well: their random global ID uniquely identifies the network.
- IP addresses from internal IPv4 ranges and well-known addresses are not anonymized (e.g. 8.8.8.8, 100.64.0.0/10, addresses starting with 192.168., 172.16., 10., 169.254., fe80::).
Strict level (--anonymize-level strict), in addition to the default level:
- Private (RFC 1918), CGNAT (100.64.0.0/10), and link-local (169.254.0.0/16, fe80::/10) addresses are anonymized too.
- Internal IPv4 addresses are replaced with addresses starting from 198.18.0.0 and internal IPv6 addresses with addresses starting from 2001:db8:1::, so internal addresses remain distinguishable from public ones.
- Addresses are mapped in order of first appearance: subnet structure, allocation scheme, and gateway conventions are not preserved. Prefix lengths of networks are preserved.
- Peer names in front of NetBird domains are replaced with numbered placeholders (e.g. peer-1.netbird.cloud), and subdomain labels of other domains with host-N placeholders.
- WireGuard public keys are replaced with consistent placeholder keys.
IP addresses from non public ranges and well known addresses are not anonymized (e.g. 8.8.8.8, 100.64.0.0/10, addresses starting with 192.168., 172.16., 10., etc.).
Reoccuring IP addresses are replaced with the same anonymized address.
Note: The anonymized IP addresses in the status file do not match those in the log and routes files. However, the anonymized IP addresses are consistent within the status file and across the routes and log files.
MAC Addresses
MAC addresses are replaced at every anonymization level with consistent placeholders counting up from 02:00:00:00:00:01. Broadcast, multicast, and all-zero addresses are kept. At the default level a preserved IPv6 link-local address may still embed a MAC address (EUI-64); the strict level anonymizes those addresses.
Domains
All domain names (except for the netbird domains) are replaced with randomly generated strings ending in ".domain". Anonymized domains are consistent across all files in the bundle.
Reoccuring domain names are replaced with the same anonymized domain.
At the strict level, the peer name labels in front of netbird domains are anonymized as well.
Sync Response
The network_map.json file contains the following anonymized information:
@@ -281,6 +293,7 @@ type BundleGenerator struct {
cliVersion string
anonymize bool
anonymizeLevel anonymize.Level
includeSystemInfo bool
logFileCount uint32
@@ -288,7 +301,10 @@ type BundleGenerator struct {
}
type BundleConfig struct {
Anonymize bool
Anonymize bool
// AnonymizeLevel selects how much the anonymizer redacts.
// anonymize.LevelStrict implies Anonymize.
AnonymizeLevel anonymize.Level
IncludeSystemInfo bool
LogFileCount uint32
}
@@ -327,8 +343,11 @@ func NewBundleGenerator(deps GeneratorDependencies, cfg BundleConfig) *BundleGen
uiLogOpener = openLogFile
}
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(cfg.AnonymizeLevel)
return &BundleGenerator{
anonymizer: anonymize.NewAnonymizer(anonymize.DefaultAddresses()),
anonymizer: anonymizer,
internalConfig: deps.InternalConfig,
statusRecorder: deps.StatusRecorder,
@@ -345,7 +364,8 @@ func NewBundleGenerator(deps GeneratorDependencies, cfg BundleConfig) *BundleGen
daemonVersion: deps.DaemonVersion,
cliVersion: deps.CliVersion,
anonymize: cfg.Anonymize,
anonymize: cfg.Anonymize || cfg.AnonymizeLevel >= anonymize.LevelStrict,
anonymizeLevel: cfg.AnonymizeLevel,
includeSystemInfo: cfg.IncludeSystemInfo,
logFileCount: logFileCount,
}
@@ -485,7 +505,13 @@ func (g *BundleGenerator) addSystemInfo() {
}
func (g *BundleGenerator) addReadme() error {
readmeReader := strings.NewReader(readmeContent)
level := "none (anonymization disabled)"
if g.anonymize {
level = g.anonymizeLevel.String()
}
header := fmt.Sprintf("Netbird debug bundle\nAnonymization level applied to this bundle: %s\n", level)
readmeReader := strings.NewReader(header + readmeContent)
if err := g.addFileToZip(readmeReader, "README.txt"); err != nil {
return fmt.Errorf("add README file to zip: %w", err)
}
@@ -507,9 +533,10 @@ func (g *BundleGenerator) addStatus() error {
fullStatus := g.statusRecorder.GetFullStatus()
protoFullStatus := nbstatus.ToProtoFullStatus(fullStatus)
overview := nbstatus.ConvertToStatusOutputOverview(protoFullStatus, nbstatus.ConvertOptions{
Anonymize: g.anonymize,
ProfileName: profName,
DaemonVersion: g.daemonVersion,
Anonymize: g.anonymize,
AnonymizeLevel: g.anonymizeLevel,
ProfileName: profName,
DaemonVersion: g.daemonVersion,
})
overview.CliVersion = g.cliVersion
statusOutput := overview.FullDetailSummary()
@@ -662,7 +689,7 @@ func (g *BundleGenerator) addCommonConfigFields(configContent *strings.Builder)
configContent.WriteString("NetBird Client Configuration:\n\n")
if key, err := wgtypes.ParseKey(g.internalConfig.PrivateKey); err == nil {
configContent.WriteString(fmt.Sprintf("PublicKey: %s\n", key.PublicKey().String()))
configContent.WriteString(fmt.Sprintf("PublicKey: %s\n", g.anonymizer.AnonymizeWGKey(key.PublicKey().String())))
}
configContent.WriteString(fmt.Sprintf("WgIface: %s\n", g.internalConfig.WgIface))
configContent.WriteString(fmt.Sprintf("WgPort: %d\n", g.internalConfig.WgPort))
@@ -952,6 +979,11 @@ func (g *BundleGenerator) addUpdateLogs() error {
}
baseName := filepath.Base(logFile)
data, err = g.anonymizeBytes(data)
if err != nil {
log.Warnf("skipping update log file %s: %v", baseName, err)
continue
}
if err := g.addFileToZip(bytes.NewReader(data), filepath.Join("update-logs", baseName)); err != nil {
return fmt.Errorf("add update log file %s to zip: %w", baseName, err)
}
@@ -979,6 +1011,13 @@ func (g *BundleGenerator) addCorruptedStateFiles() error {
}
fileName := filepath.Base(match)
// Corrupted state files usually fail structured JSON anonymization,
// so run them through the string anonymizer instead.
data, err = g.anonymizeBytes(data)
if err != nil {
log.Warnf("skipping corrupted state file %s: %v", fileName, err)
continue
}
if err := g.addFileToZip(bytes.NewReader(data), "corrupted_states/"+fileName); err != nil {
log.Warnf("Failed to add corrupted state file %s to zip: %v", fileName, err)
continue
@@ -990,6 +1029,27 @@ func (g *BundleGenerator) addCorruptedStateFiles() error {
return nil
}
// anonymizeBytes runs raw file content through the string anonymizer line by
// line when anonymization is enabled. It errors instead of returning partial
// content, so a caller never adds an unanonymized fallback to the bundle.
func (g *BundleGenerator) anonymizeBytes(data []byte) ([]byte, error) {
if !g.anonymize {
return data, nil
}
var buf bytes.Buffer
scanner := bufio.NewScanner(bytes.NewReader(data))
scanner.Buffer(make([]byte, 1024*1024), 1024*1024)
for scanner.Scan() {
buf.WriteString(g.anonymizer.AnonymizeString(scanner.Text()))
buf.WriteByte('\n')
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("anonymize content: %w", err)
}
return buf.Bytes(), nil
}
func (g *BundleGenerator) addMetrics() error {
if g.clientMetrics == nil {
log.Debugf("skipping metrics in debug bundle: no metrics collector")
@@ -1462,6 +1522,7 @@ func anonymizeRemotePeer(peer *mgmProto.RemotePeerConfig, anonymizer *anonymize.
}
peer.Fqdn = anonymizer.AnonymizeDomain(peer.Fqdn)
peer.WgPubKey = anonymizer.AnonymizeWGKey(peer.WgPubKey)
anonymizeSSHConfig(peer.SshConfig)
}

View File

@@ -35,14 +35,14 @@ func (g *BundleGenerator) toWGShowFormat(s *configurer.Stats) string {
var sb strings.Builder
sb.WriteString(fmt.Sprintf("interface: %s\n", s.DeviceName))
sb.WriteString(fmt.Sprintf(" public key: %s\n", s.PublicKey))
sb.WriteString(fmt.Sprintf(" public key: %s\n", g.anonymizer.AnonymizeWGKey(s.PublicKey)))
sb.WriteString(fmt.Sprintf(" listen port: %d\n", s.ListenPort))
if s.FWMark != 0 {
sb.WriteString(fmt.Sprintf(" fwmark: %#x\n", s.FWMark))
}
for _, peer := range s.Peers {
sb.WriteString(fmt.Sprintf("\npeer: %s\n", peer.PublicKey))
sb.WriteString(fmt.Sprintf("\npeer: %s\n", g.anonymizer.AnonymizeWGKey(peer.PublicKey)))
if peer.Endpoint.IP != nil {
if g.anonymize {
anonEndpoint := g.anonymizer.AnonymizeUDPAddr(peer.Endpoint)
@@ -54,7 +54,11 @@ func (g *BundleGenerator) toWGShowFormat(s *configurer.Stats) string {
if len(peer.AllowedIPs) > 0 {
var ipStrings []string
for _, ipnet := range peer.AllowedIPs {
ipStrings = append(ipStrings, ipnet.String())
ipStr := ipnet.String()
if g.anonymize {
ipStr = g.anonymizer.AnonymizeIPString(ipStr)
}
ipStrings = append(ipStrings, ipStr)
}
sb.WriteString(fmt.Sprintf(" allowed ips: %s\n", strings.Join(ipStrings, ", ")))
}

View File

@@ -23,6 +23,7 @@ import (
"golang.zx2c4.com/wireguard/tun/netstack"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"github.com/netbirdio/netbird/client/anonymize"
nberrors "github.com/netbirdio/netbird/client/errors"
"github.com/netbirdio/netbird/client/firewall"
"github.com/netbirdio/netbird/client/firewall/firewalld"
@@ -1385,6 +1386,7 @@ func (e *Engine) handleBundle(params *mgmProto.BundleParameters) (*mgmProto.JobR
bundleJobParams := debug.BundleConfig{
Anonymize: params.Anonymize,
AnonymizeLevel: anonymize.ParseLevel(params.AnonymizeLevel),
IncludeSystemInfo: true,
LogFileCount: uint32(params.LogFileCount),
}

View File

@@ -519,7 +519,7 @@ func startManagement(t *testing.T, dataDir, testFile string) (*grpc.Server, stri
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
if err != nil {
return nil, "", err

View File

@@ -14,6 +14,7 @@ import (
log "github.com/sirupsen/logrus"
nbAnonymize "github.com/netbirdio/netbird/client/anonymize"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
"github.com/netbirdio/netbird/client/internal/debug"
@@ -28,6 +29,13 @@ import (
types "github.com/netbirdio/netbird/upload-server/types"
)
// AnonymizeLevelDefault and AnonymizeLevelStrict are the accepted
// anonymizeLevel values for DebugBundle.
const (
AnonymizeLevelDefault = nbAnonymize.LevelDefaultString
AnonymizeLevelStrict = nbAnonymize.LevelStrictString
)
// ConnectionListener export internal Listener for mobile
type ConnectionListener interface {
peer.Listener
@@ -200,8 +208,10 @@ func (c *Client) Stop() {
// DebugBundle generates a debug bundle, uploads it and returns the upload key.
// It works with or without a running engine: when the engine is up it reuses
// the live config, sync response and client metrics; otherwise it loads the
// config from disk (or the preloaded tvOS config).
func (c *Client) DebugBundle(anonymize bool) (string, error) {
// config from disk (or the preloaded tvOS config). anonymizeLevel is "default"
// or "strict"; strict also anonymizes internal IP ranges, peer names, and
// WireGuard public keys, and implies anonymize.
func (c *Client) DebugBundle(anonymize bool, anonymizeLevel string) (string, error) {
cfg, cc := c.stateSnapshot()
// If the engine hasn't been started, load config so we can reach management.
@@ -251,6 +261,7 @@ func (c *Client) DebugBundle(anonymize bool) (string, error) {
deps,
debug.BundleConfig{
Anonymize: anonymize,
AnonymizeLevel: nbAnonymize.ParseLevel(anonymizeLevel),
IncludeSystemInfo: true,
},
)

View File

@@ -2781,6 +2781,11 @@ type DebugBundleRequest struct {
// untrusted TLS certificate. Restricted to privileged callers; for
// self-hosted upload servers.
UploadInsecure bool `protobuf:"varint,7,opt,name=uploadInsecure,proto3" json:"uploadInsecure,omitempty"`
// anonymizeLevel selects how much the anonymizer redacts: "default"
// (or empty) keeps internal IP ranges, "strict" also anonymizes them.
// Unknown values are treated as "strict". Only meaningful with anonymize;
// "strict" implies it.
AnonymizeLevel string `protobuf:"bytes,8,opt,name=anonymizeLevel,proto3" json:"anonymizeLevel,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -2857,6 +2862,13 @@ func (x *DebugBundleRequest) GetUploadInsecure() bool {
return false
}
func (x *DebugBundleRequest) GetAnonymizeLevel() string {
if x != nil {
return x.AnonymizeLevel
}
return ""
}
type DebugBundleResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Path string `protobuf:"bytes,1,opt,name=path,proto3" json:"path,omitempty"`
@@ -7253,7 +7265,7 @@ const file_daemon_proto_rawDesc = "" +
"\x12translatedHostname\x18\x04 \x01(\tR\x12translatedHostname\x128\n" +
"\x0etranslatedPort\x18\x05 \x01(\v2\x10.daemon.PortInfoR\x0etranslatedPort\"G\n" +
"\x17ForwardingRulesResponse\x12,\n" +
"\x05rules\x18\x01 \x03(\v2\x16.daemon.ForwardingRuleR\x05rules\"\xdc\x01\n" +
"\x05rules\x18\x01 \x03(\v2\x16.daemon.ForwardingRuleR\x05rules\"\x84\x02\n" +
"\x12DebugBundleRequest\x12\x1c\n" +
"\tanonymize\x18\x01 \x01(\bR\tanonymize\x12\x1e\n" +
"\n" +
@@ -7264,7 +7276,8 @@ const file_daemon_proto_rawDesc = "" +
"\n" +
"cliVersion\x18\x06 \x01(\tR\n" +
"cliVersion\x12&\n" +
"\x0euploadInsecure\x18\a \x01(\bR\x0euploadInsecure\"}\n" +
"\x0euploadInsecure\x18\a \x01(\bR\x0euploadInsecure\x12&\n" +
"\x0eanonymizeLevel\x18\b \x01(\tR\x0eanonymizeLevel\"}\n" +
"\x13DebugBundleResponse\x12\x12\n" +
"\x04path\x18\x01 \x01(\tR\x04path\x12 \n" +
"\vuploadedKey\x18\x02 \x01(\tR\vuploadedKey\x120\n" +

View File

@@ -540,6 +540,11 @@ message DebugBundleRequest {
// untrusted TLS certificate. Restricted to privileged callers; for
// self-hosted upload servers.
bool uploadInsecure = 7;
// anonymizeLevel selects how much the anonymizer redacts: "default"
// (or empty) keeps internal IP ranges, "strict" also anonymizes them.
// Unknown values are treated as "strict". Only meaningful with anonymize;
// "strict" implies it.
string anonymizeLevel = 8;
}
message DebugBundleResponse {

View File

@@ -16,6 +16,7 @@ import (
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/anonymize"
"github.com/netbirdio/netbird/client/internal/debug"
"github.com/netbirdio/netbird/client/internal/ipcauth"
"github.com/netbirdio/netbird/client/proto"
@@ -122,6 +123,7 @@ func (s *Server) generateDebugBundle(req *proto.DebugBundleRequest, uiOpener deb
},
debug.BundleConfig{
Anonymize: req.GetAnonymize(),
AnonymizeLevel: anonymize.ParseLevel(req.GetAnonymizeLevel()),
IncludeSystemInfo: req.GetSystemInfo(),
LogFileCount: req.GetLogFileCount(),
},

View File

@@ -232,4 +232,3 @@ func toNetIDs(routes []string) []route.NetID {
}
return netIDs
}

View File

@@ -200,7 +200,7 @@ func startManagement(t *testing.T, signalAddr string, counter *int) (*grpc.Serve
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
peersUpdateManager := update_channel.NewPeersUpdateManager(metrics)
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
if err != nil {
return nil, "", err

View File

@@ -46,7 +46,10 @@ func ParseDaemonStatus(s string) DaemonStatus {
// ConvertOptions holds parameters for ConvertToStatusOutputOverview.
type ConvertOptions struct {
Anonymize bool
Anonymize bool
// AnonymizeLevel selects how much the anonymizer redacts. Only
// meaningful when Anonymize is set.
AnonymizeLevel anonymize.Level
DaemonVersion string
DaemonStatus DaemonStatus
StatusFilter string
@@ -217,6 +220,7 @@ func ConvertToStatusOutputOverview(pbFullStatus *proto.FullStatus, opts ConvertO
if opts.Anonymize {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizer.SetLevel(opts.AnonymizeLevel)
anonymizeOverview(anonymizer, &overview)
}
@@ -976,6 +980,7 @@ func timeAgo(t time.Time) string {
func anonymizePeerDetail(a *anonymize.Anonymizer, peer *PeerStateDetailOutput) {
peer.FQDN = a.AnonymizeDomain(peer.FQDN)
peer.PubKey = a.AnonymizeWGKey(peer.PubKey)
if localIP, port, err := net.SplitHostPort(peer.IceCandidateEndpoint.Local); err == nil {
peer.IceCandidateEndpoint.Local = fmt.Sprintf("%s:%s", a.AnonymizeIPString(localIP), port)
}
@@ -1007,6 +1012,7 @@ func anonymizeOverview(a *anonymize.Anonymizer, overview *OutputOverview) {
overview.SignalState.URL = a.AnonymizeURI(overview.SignalState.URL)
overview.SignalState.Error = a.AnonymizeString(overview.SignalState.Error)
overview.PubKey = a.AnonymizeWGKey(overview.PubKey)
overview.IP = a.AnonymizeIPString(overview.IP)
overview.IPv6 = a.AnonymizeIPString(overview.IPv6)
for i, detail := range overview.Relays.Details {

View File

@@ -71,10 +71,12 @@ type BundleOptions = {
hasWindow: boolean;
totalSec: number;
uploadUrl: string;
anonymize: boolean;
anonymizeLevel: AnonymizeLevel;
systemInfo: boolean;
};
export type AnonymizeLevel = "none" | "default" | "strict";
const startCaptureBestEffort = async (totalSec: number, pcap: CaptureState) => {
try {
// Mirror the CLI's safety margin: window + 30s, server caps at 10m.
@@ -187,7 +189,10 @@ const runBundleFlow = async (
if (opts.uploadUrl) setStage({ kind: "uploading" });
const result = await DebugSvc.Bundle({
anonymize: opts.anonymize,
anonymize: opts.anonymizeLevel !== "none",
// The daemon only knows "default" and "strict"; "none" is expressed
// through the anonymize flag being off.
anonymizeLevel: opts.anonymizeLevel === "strict" ? "strict" : "default",
systemInfo: opts.systemInfo,
uploadUrl: opts.uploadUrl,
logFileCount,
@@ -198,7 +203,7 @@ const runBundleFlow = async (
};
const useDebugBundle = () => {
const [anonymize, setAnonymize] = useState(false);
const [anonymizeLevel, setAnonymizeLevel] = useState<AnonymizeLevel>("none");
const [systemInfo, setSystemInfo] = useState(true);
const [upload, setUpload] = useState(true);
const [trace, setTrace] = useState(true);
@@ -240,7 +245,7 @@ const useDebugBundle = () => {
hasWindow: capture && totalSec > 0,
totalSec,
uploadUrl: upload ? NETBIRD_UPLOAD_URL : "",
anonymize,
anonymizeLevel,
systemInfo,
};
@@ -272,8 +277,8 @@ const useDebugBundle = () => {
};
return {
anonymize,
setAnonymize,
anonymizeLevel,
setAnonymizeLevel,
systemInfo,
setSystemInfo,
upload,

View File

@@ -1,6 +1,6 @@
import { useId, type ReactNode } from "react";
import { Trans, useTranslation } from "react-i18next";
import { CircleCheckBig, FolderOpen, Loader2 } from "lucide-react";
import { ChevronDown, CircleCheckBig, FolderOpen, Info, Loader2 } from "lucide-react";
import { Browser } from "@wailsio/runtime";
import { Debug as DebugSvc } from "@bindings/services";
import type { DebugBundleResult } from "@bindings/services/models.js";
@@ -8,13 +8,22 @@ import { Button } from "@/components/buttons/Button";
import { DialogActions } from "@/components/dialog/DialogActions";
import { DialogDescription } from "@/components/dialog/DialogDescription";
import { DialogHeading } from "@/components/dialog/DialogHeading";
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuRadioGroup,
DropdownMenuRadioItem,
DropdownMenuTrigger,
} from "@/components/DropdownMenu";
import FancyToggleSwitch from "@/components/switches/FancyToggleSwitch";
import HelpText from "@/components/typography/HelpText.tsx";
import { Input } from "@/components/inputs/Input";
import { Label } from "@/components/typography/Label";
import { SquareIcon } from "@/components/SquareIcon";
import { Tooltip } from "@/components/Tooltip";
import { cn } from "@/lib/cn";
import { formatRemaining } from "@/lib/formatters";
import type { DebugStage } from "@/contexts/DebugBundleContext";
import type { AnonymizeLevel, DebugStage } from "@/contexts/DebugBundleContext";
import { useDebugBundleContext } from "@/contexts/DebugBundleContext";
import { SectionGroup, SettingsBottomBar } from "@/modules/settings/SettingsSection.tsx";
@@ -24,8 +33,8 @@ export function SettingsTroubleshooting() {
const { t } = useTranslation();
const durationId = useId();
const {
anonymize,
setAnonymize,
anonymizeLevel,
setAnonymizeLevel,
systemInfo,
setSystemInfo,
upload,
@@ -55,12 +64,71 @@ export function SettingsTroubleshooting() {
return (
<SectionGroup title={t("settings.troubleshooting.section.title")}>
<FancyToggleSwitch
value={anonymize}
onChange={setAnonymize}
label={t("settings.troubleshooting.anonymize.label")}
helpText={t("settings.troubleshooting.anonymize.help")}
/>
<div className={"flex items-center justify-between gap-6"}>
<div className={"max-w-md flex-1"}>
<Label as={"div"}>
<span className={"inline-flex items-center gap-1.5"}>
{t("settings.troubleshooting.anonymize.label")}
<Tooltip
content={
<div className={"max-w-xs whitespace-normal leading-relaxed"}>
{t("settings.troubleshooting.anonymize.info")}
</div>
}
>
<Info
size={14}
aria-label={t("settings.troubleshooting.anonymize.label")}
className={"shrink-0 cursor-default text-nb-gray-400"}
/>
</Tooltip>
</span>
</Label>
<HelpText margin={false}>
{t("settings.troubleshooting.anonymize.help")}
</HelpText>
</div>
<div className={"shrink-0"}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button
type={"button"}
aria-label={t("settings.troubleshooting.anonymize.label")}
className={cn(
"inline-flex h-[40px] min-w-[160px] items-center justify-between gap-2 px-3",
"rounded-md border bg-white dark:bg-nb-gray-900",
"border-neutral-200 dark:border-nb-gray-700",
"cursor-default text-xs font-semibold text-nb-gray-100 outline-none",
"hover:border-nb-gray-600 data-[state=open]:border-nb-gray-600",
)}
>
{t(`settings.troubleshooting.anonymize.${anonymizeLevel}`)}
<ChevronDown
size={16}
aria-hidden={"true"}
className={"shrink-0 text-nb-gray-200"}
/>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align={"end"} className={"min-w-[160px]"}>
<DropdownMenuRadioGroup
value={anonymizeLevel}
onValueChange={(v) => setAnonymizeLevel(v as AnonymizeLevel)}
>
<DropdownMenuRadioItem value={"none"}>
{t("settings.troubleshooting.anonymize.none")}
</DropdownMenuRadioItem>
<DropdownMenuRadioItem value={"default"}>
{t("settings.troubleshooting.anonymize.default")}
</DropdownMenuRadioItem>
<DropdownMenuRadioItem value={"strict"}>
{t("settings.troubleshooting.anonymize.strict")}
</DropdownMenuRadioItem>
</DropdownMenuRadioGroup>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
<FancyToggleSwitch
value={systemInfo}
onChange={setSystemInfo}

View File

@@ -1013,11 +1013,27 @@
},
"settings.troubleshooting.anonymize.label": {
"message": "Anonymize Sensitive Information",
"description": "Toggle label: anonymize sensitive information in the bundle."
"description": "Label for the anonymization level dropdown (None, Default, Strict)."
},
"settings.troubleshooting.anonymize.help": {
"message": "Hides public IP addresses and non-NetBird domains from logs.",
"description": "Helper text for anonymizing logs (hides public IPs and non-NetBird domains)."
"message": "Hides IP addresses, domains, and other sensitive values.",
"description": "Helper text under the anonymization dropdown. The level details live in the info tooltip."
},
"settings.troubleshooting.anonymize.info": {
"message": "Default keeps internal IPv4 addresses and peer names readable for support. Strict additionally anonymizes private (RFC 1918), CGNAT, and link-local IP addresses, peer names, and WireGuard public keys. Recurring values map to the same placeholder, so peers stay distinguishable. Use Strict when sharing the bundle outside your organization.",
"description": "Info tooltip explaining the anonymization levels. 'RFC 1918', 'CGNAT', 'link-local', and 'WireGuard' are technical terms — keep them."
},
"settings.troubleshooting.anonymize.none": {
"message": "None",
"description": "Dropdown option: no anonymization."
},
"settings.troubleshooting.anonymize.default": {
"message": "Default",
"description": "Dropdown option: default anonymization level."
},
"settings.troubleshooting.anonymize.strict": {
"message": "Strict",
"description": "Dropdown option: strict anonymization level."
},
"settings.troubleshooting.systemInfo.label": {
"message": "Include System Information",

View File

@@ -15,10 +15,13 @@ import (
)
type DebugBundleParams struct {
Anonymize bool `json:"anonymize"`
SystemInfo bool `json:"systemInfo"`
UploadURL string `json:"uploadUrl"`
LogFileCount uint32 `json:"logFileCount"`
Anonymize bool `json:"anonymize"`
// AnonymizeLevel is "default" or "strict"; strict also anonymizes
// private IP ranges, peer names, and WireGuard public keys.
AnonymizeLevel string `json:"anonymizeLevel"`
SystemInfo bool `json:"systemInfo"`
UploadURL string `json:"uploadUrl"`
LogFileCount uint32 `json:"logFileCount"`
}
// DebugBundleResult: Path is set for local-only bundles, UploadedKey on upload
@@ -48,11 +51,12 @@ func (s *Debug) Bundle(ctx context.Context, p DebugBundleParams) (DebugBundleRes
return DebugBundleResult{}, err
}
resp, err := cli.DebugBundle(ctx, &proto.DebugBundleRequest{
Anonymize: p.Anonymize,
SystemInfo: p.SystemInfo,
UploadURL: p.UploadURL,
LogFileCount: p.LogFileCount,
CliVersion: version.NetbirdVersion(),
Anonymize: p.Anonymize,
AnonymizeLevel: p.AnonymizeLevel,
SystemInfo: p.SystemInfo,
UploadURL: p.UploadURL,
LogFileCount: p.LogFileCount,
CliVersion: version.NetbirdVersion(),
})
if err != nil {
return DebugBundleResult{}, err

View File

@@ -438,14 +438,10 @@ func TestProvidersMatrix(t *testing.T) {
// Create every provider, all enabled, each with a unique model string so the
// proxy's connect-time snapshot carries them all and model→provider routing
// is unambiguous (provider toggles after connect don't reconcile to the
// proxy, so we enable everything up front). The first create bootstraps the
// cluster.
// proxy, so we enable everything up front).
ids := make([]string, 0, len(matrix))
for i, pc := range matrix {
for _, pc := range matrix {
req := providerRequest(pc)
if i == 0 {
req.BootstrapCluster = ptr(harness.AgentNetworkCluster)
}
prov, perr := srv.CreateProvider(ctx, req)
require.NoError(t, perr, "create provider %s", pc.name)
ids = append(ids, prov.Id)

View File

@@ -82,13 +82,12 @@ func provisionPricedProvider(t *testing.T, ctx context.Context, name string, mod
// need NOT be in the catalog — the operator names it and prices it here.
dummyKey := "sk-price-e2e"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: name,
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
BootstrapCluster: ptr(harness.AgentNetworkCluster),
Models: &models,
Name: name,
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
Models: &models,
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })

View File

@@ -113,15 +113,14 @@ func runPathRoutedGuardrailCase(t *testing.T, tc pathRoutedGuardrailCase) {
// Catch-all provider (no models) so the router forwards any model; a static
// bearer key means the router injects a static auth header instead of minting
// a GCP token. Bootstraps the cluster if it isn't already.
// a GCP token.
staticKey := "static-e2e-token"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: tc.name,
ProviderId: tc.catalogID,
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
BootstrapCluster: ptr(harness.AgentNetworkCluster),
Name: tc.name,
ProviderId: tc.catalogID,
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
})
require.NoError(t, err, "create %s provider", tc.name)
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })

View File

@@ -73,7 +73,6 @@ func TestGuardrailGroupSwitchTakesEffectAfterTTL(t *testing.T) {
{Id: modelA, InputPer1k: 0.001, OutputPer1k: 0.001},
{Id: modelB, InputPer1k: 0.001, OutputPer1k: 0.001},
},
BootstrapCluster: ptr(harness.AgentNetworkCluster),
})
require.NoError(t, err, "create provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })

View File

@@ -61,15 +61,14 @@ func TestGuardrailMultiPolicyModelAllowlist(t *testing.T) {
}
// pRestricted declares the two guardrailed models so routing is deterministic
// (model -> provider). Created first, so it carries the bootstrap cluster.
// (model -> provider).
pRestricted, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "restricted",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
Models: models(modelSelected, modelOther),
BootstrapCluster: ptr(harness.AgentNetworkCluster),
Name: "restricted",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
Models: models(modelSelected, modelOther),
})
require.NoError(t, err, "create restricted provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), pRestricted.Id) })

View File

@@ -115,7 +115,7 @@ func TestGuardrailPerGroupAllowlist_AllProviders(t *testing.T) {
staticKey := "static-e2e-token"
enabled := true
for i, c := range cases {
for _, c := range cases {
req := api.AgentNetworkProviderRequest{
Name: "e2e-pergroup-" + c.name,
ProviderId: c.catalogID,
@@ -124,9 +124,6 @@ func TestGuardrailPerGroupAllowlist_AllProviders(t *testing.T) {
Enabled: ptr(true),
Models: c.models,
}
if i == 0 {
req.BootstrapCluster = ptr(harness.AgentNetworkCluster)
}
prov, perr := srv.CreateProvider(ctx, req)
require.NoError(t, perr, "create provider %s", c.name)
c.providerID = prov.Id
@@ -283,13 +280,12 @@ func TestGuardrailMultiGroupUser(t *testing.T) {
// P1 — union scenario: two restricting policies, one per group.
p1, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "e2e-mg-union",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
Models: priced(unionA, unionB, unionC),
BootstrapCluster: ptr(harness.AgentNetworkCluster),
Name: "e2e-mg-union",
ProviderId: "openai_api",
UpstreamUrl: vllm.URL,
ApiKey: &staticKey,
Enabled: ptr(true),
Models: priced(unionA, unionB, unionC),
})
require.NoError(t, err, "create union provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), p1.Id) })

View File

@@ -115,14 +115,11 @@ func TestModelAllowlistEnforced(t *testing.T) {
})
require.NoError(t, err, "mint setup key")
// Providers with their configured (allowed) models; the first bootstraps the cluster.
// Providers with their configured (allowed) models
ids := make([]string, 0, len(providers))
allowed := make([]string, 0, len(providers))
for i, pc := range providers {
for _, pc := range providers {
req := providerRequest(pc)
if i == 0 {
req.BootstrapCluster = ptr(harness.AgentNetworkCluster)
}
prov, perr := srv.CreateProvider(ctx, req)
require.NoError(t, perr, "create provider %s", pc.name)
id := prov.Id

View File

@@ -14,6 +14,7 @@ import (
"time"
"github.com/netbirdio/netbird/e2e/harness"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// srv is the shared combined server for the package, ready (PAT-authenticated)
@@ -42,5 +43,14 @@ func run(m *testing.M) int {
return 1
}
// Bootstrap the account's agent-network endpoint once for the package:
// providers no longer have settings side effects, and every data-plane
// test expects the shared account pinned to the combined proxy cluster.
cluster := harness.AgentNetworkCluster
if _, err := srv.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{ProxyAddress: &cluster}); err != nil {
fmt.Fprintf(os.Stderr, "e2e: bootstrap agent-network endpoint: %v\n", err)
return 1
}
return m.Run()
}

View File

@@ -21,11 +21,10 @@ func ptr[T any](v T) *T { return &v }
func newProvider(t *testing.T, ctx context.Context, name string) api.AgentNetworkProvider {
t.Helper()
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: name,
ProviderId: "openai_api",
UpstreamUrl: "https://api.openai.com",
ApiKey: ptr("sk-dummy-e2e-key"),
BootstrapCluster: ptr("eu.proxy.netbird.test"),
Name: name,
ProviderId: "openai_api",
UpstreamUrl: "https://api.openai.com",
ApiKey: ptr("sk-dummy-e2e-key"),
})
require.NoError(t, err, "create provider %q", name)
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
@@ -57,17 +56,11 @@ func TestProviderLifecycle(t *testing.T) {
}}
}
for i, pc := range cases {
i, pc := i, pc
for _, pc := range cases {
pc := pc
t.Run(pc.name, func(t *testing.T) {
req := providerRequest(pc)
req.Name = "lc-" + pc.name
// Bootstrap the cluster on the first create in case the matrix has
// not run (e.g. no provider keys → settings not yet bootstrapped).
if i == 0 {
req.BootstrapCluster = ptr(harness.AgentNetworkCluster)
}
prov, err := srv.CreateProvider(ctx, req)
require.NoError(t, err, "create %s provider", pc.name)
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), prov.Id) })
@@ -137,45 +130,65 @@ func TestProviderValidation(t *testing.T) {
requireClientError(t, err)
}
// TestSettingsRoundTrip flips the collection toggles and confirms cluster /
// subdomain stay immutable, then restores the original state.
// TestSettingsRoundTrip flips the collection toggles and confirms the
// endpoint and proxy address stay immutable, then restores the original
// state. A second bootstrap attempt must be rejected as a conflict.
func TestSettingsRoundTrip(t *testing.T) {
ctx := context.Background()
// Settings are bootstrapped on first provider create.
newProvider(t, ctx, "Settings Bootstrap")
// The package's TestMain bootstrapped the shared account's endpoint.
before, err := srv.GetSettings(ctx)
require.NoError(t, err, "get settings")
require.NotEmpty(t, before.Cluster, "settings must carry an assigned cluster")
require.NotEmpty(t, before.Endpoint, "settings must carry the bootstrapped endpoint")
require.NotEmpty(t, before.ProxyAddress, "settings must carry the bootstrapped proxy address")
require.NotNil(t, before.AccessLogRetentionDays, "bootstrapped settings must carry a retention")
beforeRetention := *before.AccessLogRetentionDays
flipped, err := srv.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Endpoint: before.Endpoint,
ProxyAddress: before.ProxyAddress,
EnableLogCollection: !before.EnableLogCollection,
EnablePromptCollection: !before.EnablePromptCollection,
RedactPii: !before.RedactPii,
AccessLogRetentionDays: beforeRetention,
})
require.NoError(t, err, "update settings")
assert.Equal(t, !before.EnableLogCollection, flipped.EnableLogCollection, "log collection toggle must flip")
assert.Equal(t, !before.EnablePromptCollection, flipped.EnablePromptCollection, "prompt collection toggle must flip")
assert.Equal(t, before.Cluster, flipped.Cluster, "cluster must be immutable across updates")
assert.Equal(t, before.Subdomain, flipped.Subdomain, "subdomain must be immutable across updates")
require.NotNil(t, flipped.AccessLogRetentionDays)
assert.Equal(t, beforeRetention, *flipped.AccessLogRetentionDays,
"retention sent unchanged must round-trip, not reset to the zero value")
assert.Equal(t, before.Endpoint, flipped.Endpoint, "endpoint must be immutable across updates")
assert.Equal(t, before.ProxyAddress, flipped.ProxyAddress, "proxy address must be immutable across updates")
// A cluster different from the pinned one must be rejected; echoing the
// pinned one back is valid.
// The account is already bootstrapped: a second bootstrap is a conflict,
// whatever shape it asks for.
_, err = srv.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{
Endpoint: ptr("attacker.cluster.invalid"),
})
requireClientError(t, err)
// The identity fields ride along on the PUT as a required echo: a request
// carrying a different endpoint is rejected without applying anything.
_, err = srv.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Cluster: ptr("attacker.cluster.invalid"),
Endpoint: "other.cluster.invalid",
ProxyAddress: before.ProxyAddress,
EnableLogCollection: before.EnableLogCollection,
EnablePromptCollection: before.EnablePromptCollection,
RedactPii: before.RedactPii,
AccessLogRetentionDays: beforeRetention,
})
requireClientError(t, err)
// Restore the original toggles.
_, err = srv.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Cluster: ptr(before.Cluster),
Endpoint: before.Endpoint,
ProxyAddress: before.ProxyAddress,
EnableLogCollection: before.EnableLogCollection,
EnablePromptCollection: before.EnablePromptCollection,
RedactPii: before.RedactPii,
AccessLogRetentionDays: beforeRetention,
})
require.NoError(t, err, "restore settings")
}

View File

@@ -4,6 +4,7 @@ package agentnetwork
import (
"context"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -14,7 +15,8 @@ import (
)
// harnessStartFresh boots a dedicated combined server with its own fresh
// account and registers its teardown on t.
// account and registers its teardown on t. Unlike the shared srv, the fresh
// account has NOT had its agent-network endpoint bootstrapped.
func harnessStartFresh(ctx context.Context, t *testing.T) (*harness.Combined, error) {
t.Helper()
fresh, err := harness.StartCombined(ctx)
@@ -28,16 +30,16 @@ func harnessStartFresh(ctx context.Context, t *testing.T) (*harness.Combined, er
return fresh, nil
}
// TestSettingsBootstrapViaPut covers the settings-first bootstrap path on an
// TestSettingsBootstrapViaPost covers the explicit bootstrap contract on an
// account that has never been bootstrapped: the GET reads as the defaults
// with an empty cluster/subdomain/endpoint, a PUT without a cluster has
// nothing to pin and fails, and a PUT carrying a cluster creates the row and
// pins it immutably. The shared srv cannot provide that starting state (any
// provider-creating test bootstraps it, and test order is deliberately not
// relied on), so this boots a dedicated combined server — the image is
// already built and cached by TestMain's StartCombined, so the extra cost is
// one container start.
func TestSettingsBootstrapViaPut(t *testing.T) {
// with an empty endpoint/proxy_address, a PUT has no row to update and fails,
// and a POST creates the row and assigns the immutable endpoint — labeled
// beneath a proxy address here, with the toggle overrides from the same
// request applied. The shared srv cannot provide that starting state
// (TestMain bootstraps it), so this boots a dedicated combined server — the
// image is already built and cached by TestMain's StartCombined, so the extra
// cost is one container start.
func TestSettingsBootstrapViaPost(t *testing.T) {
ctx := context.Background()
fresh, err := harnessStartFresh(ctx, t)
@@ -47,32 +49,35 @@ func TestSettingsBootstrapViaPut(t *testing.T) {
// as an error and not as a null body.
before, err := fresh.GetSettings(ctx)
require.NoError(t, err, "get settings on a fresh account must succeed")
assert.Empty(t, before.Cluster, "cluster must be empty before bootstrap")
assert.Empty(t, before.Subdomain, "subdomain must be empty before bootstrap")
assert.Empty(t, before.Endpoint, "endpoint must be empty before bootstrap, not a bare dot")
assert.Empty(t, before.Endpoint, "endpoint must be empty before bootstrap")
assert.Empty(t, before.ProxyAddress, "proxy address must be empty before bootstrap")
assert.False(t, before.Dedicated, "an unbootstrapped account has no serving shape")
assert.True(t, before.EnableLogCollection, "defaults must show log collection on, matching bootstrap")
assert.False(t, before.EnablePromptCollection, "defaults must show prompt collection off")
// A PUT without a cluster has nothing to pin the account to.
// A PUT has no row to update yet — bootstrap is the explicit POST.
_, err = fresh.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
EnableLogCollection: true,
EnableLogCollection: true,
AccessLogRetentionDays: 30,
})
requireClientError(t, err)
// A PUT carrying a cluster bootstraps the account and applies the
// mutable fields from the same request. Every toggle is set away from
// its bootstrap default so each assertion can actually fail.
// A POST with a proxy address bootstraps a labeled endpoint and applies
// the toggles from the same request. Every toggle is set away from its
// bootstrap default so each assertion can actually fail.
const cluster = "e2e.bootstrap.netbird.selfhosted"
bootstrapped, err := fresh.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Cluster: ptr(cluster),
EnableLogCollection: false,
EnablePromptCollection: true,
RedactPii: true,
bootstrapped, err := fresh.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{
ProxyAddress: ptr(cluster),
EnableLogCollection: ptr(false),
EnablePromptCollection: ptr(true),
RedactPii: ptr(true),
})
require.NoError(t, err, "bootstrap settings via PUT must succeed")
assert.Equal(t, cluster, bootstrapped.Cluster, "cluster must be pinned from the request")
require.NotEmpty(t, bootstrapped.Subdomain, "subdomain must be assigned at bootstrap")
assert.Equal(t, bootstrapped.Subdomain+"."+cluster, bootstrapped.Endpoint, "endpoint must combine subdomain and cluster")
require.NoError(t, err, "bootstrap settings via POST must succeed")
assert.Equal(t, cluster, bootstrapped.ProxyAddress, "proxy address must be pinned from the request")
require.NotEmpty(t, bootstrapped.Endpoint, "endpoint must be assigned at bootstrap")
assert.True(t, strings.HasSuffix(bootstrapped.Endpoint, "."+cluster),
"labeled endpoint must hang one label beneath the proxy address: %s", bootstrapped.Endpoint)
assert.False(t, bootstrapped.Dedicated, "a labeled pin is not dedicated")
assert.False(t, bootstrapped.EnableLogCollection, "log collection from the bootstrap request must override the default")
assert.True(t, bootstrapped.EnablePromptCollection, "prompt collection from the bootstrap request must apply")
assert.True(t, bootstrapped.RedactPii, "redact toggle from the bootstrap request must apply")
@@ -85,30 +90,90 @@ func TestSettingsBootstrapViaPut(t *testing.T) {
assert.Equal(t, bootstrapped.EnablePromptCollection, after.EnablePromptCollection, "prompt collection must persist")
assert.Equal(t, bootstrapped.RedactPii, after.RedactPii, "redact toggle must persist")
// Once bootstrapped, later updates may omit the cluster entirely.
// Once bootstrapped, PUT updates the toggles. The identity fields ride
// along as a required echo of the assigned values; a matching echo is
// accepted and never written.
persisted, err := fresh.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Endpoint: bootstrapped.Endpoint,
ProxyAddress: bootstrapped.ProxyAddress,
EnableLogCollection: true,
EnablePromptCollection: false,
RedactPii: true,
AccessLogRetentionDays: 21,
})
require.NoError(t, err, "post-bootstrap update without cluster must succeed")
assert.Equal(t, cluster, persisted.Cluster, "omitted cluster must keep the pinned value")
require.NoError(t, err, "post-bootstrap update must succeed")
require.NotNil(t, persisted.AccessLogRetentionDays)
assert.Equal(t, 21, *persisted.AccessLogRetentionDays, "retention from the update must apply")
assert.Equal(t, bootstrapped.Endpoint, persisted.Endpoint, "endpoint must survive updates untouched")
assert.Equal(t, cluster, persisted.ProxyAddress, "proxy address must survive updates untouched")
assert.True(t, persisted.EnableLogCollection, "post-bootstrap toggle must apply")
assert.False(t, persisted.EnablePromptCollection, "post-bootstrap toggle must apply")
// The cluster is immutable: a different value is rejected rather than
// silently ignored, and the rejected update must not disturb anything.
// The endpoint is immutable: a PUT carrying a different endpoint is
// rejected, and a second bootstrap is rejected as a conflict. Neither
// rejected write may disturb anything.
_, err = fresh.UpdateSettings(ctx, api.AgentNetworkSettingsRequest{
Cluster: ptr("other.cluster.invalid"),
EnableLogCollection: false,
Endpoint: "other.cluster.invalid",
ProxyAddress: persisted.ProxyAddress,
EnableLogCollection: persisted.EnableLogCollection,
EnablePromptCollection: persisted.EnablePromptCollection,
RedactPii: persisted.RedactPii,
AccessLogRetentionDays: 21,
})
requireClientError(t, err)
_, err = fresh.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{
Endpoint: ptr("other.cluster.invalid"),
})
requireClientError(t, err)
final, err := fresh.GetSettings(ctx)
require.NoError(t, err, "get settings after the rejected cluster change must succeed")
assert.Equal(t, persisted.Cluster, final.Cluster, "rejected update must not change the cluster")
assert.Equal(t, persisted.Endpoint, final.Endpoint, "rejected update must not change the endpoint")
assert.Equal(t, persisted.EnableLogCollection, final.EnableLogCollection, "rejected update must not apply its toggles")
assert.Equal(t, persisted.EnablePromptCollection, final.EnablePromptCollection, "rejected update must not apply its toggles")
assert.Equal(t, persisted.RedactPii, final.RedactPii, "rejected update must not apply its toggles")
require.NoError(t, err, "get settings after the rejected bootstrap must succeed")
assert.Equal(t, persisted.Endpoint, final.Endpoint, "rejected bootstrap must not change the endpoint")
assert.Equal(t, persisted.ProxyAddress, final.ProxyAddress, "rejected bootstrap must not change the proxy address")
assert.Equal(t, persisted.EnableLogCollection, final.EnableLogCollection, "rejected bootstrap must not apply its toggles")
assert.Equal(t, persisted.EnablePromptCollection, final.EnablePromptCollection, "rejected bootstrap must not apply its toggles")
assert.Equal(t, persisted.RedactPii, final.RedactPii, "rejected bootstrap must not apply its toggles")
}
// TestSettingsBootstrapSelfAddressed covers the dedicated shape end to end:
// a POST carrying an endpoint claims the hostname verbatim, the proxy address
// equals it, and the pin reads as dedicated — the address-first flow a
// self-hosted operator uses before deploying the proxy that will declare it.
// The tail covers the recovery path the guarded DELETE exists for: with no
// providers and no proxy at the address, the claim can be released and a
// fresh bootstrap succeeds — the fix for a typo'd immutable endpoint.
func TestSettingsBootstrapSelfAddressed(t *testing.T) {
ctx := context.Background()
fresh, err := harnessStartFresh(ctx, t)
require.NoError(t, err, "start dedicated combined server")
created, err := fresh.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{
Endpoint: ptr("gw.e2e.netbird.selfhosted"),
})
require.NoError(t, err, "self-addressed bootstrap must succeed")
assert.Equal(t, "gw.e2e.netbird.selfhosted", created.Endpoint, "endpoint must be claimed verbatim")
assert.Equal(t, created.Endpoint, created.ProxyAddress, "self-addressed: proxy address is the endpoint")
assert.True(t, created.Dedicated, "a self-addressed pin is dedicated")
// No providers exist and no proxy declares the address, so both delete
// guards are clear: the delete releases the claim and the account reads
// as unbootstrapped defaults again.
require.NoError(t, fresh.DeleteSettings(ctx), "guarded delete with both guards clear must succeed")
after, err := fresh.GetSettings(ctx)
require.NoError(t, err, "get settings after delete must succeed")
assert.Empty(t, after.Endpoint, "a deleted account must read as unbootstrapped")
// A second delete has nothing to remove.
requireClientError(t, fresh.DeleteSettings(ctx))
// Re-creating is a fresh bootstrap — the released hostname is free to be
// claimed again, or a different one chosen.
recreated, err := fresh.CreateSettings(ctx, api.AgentNetworkSettingsCreateRequest{
Endpoint: ptr("gw2.e2e.netbird.selfhosted"),
})
require.NoError(t, err, "bootstrap after delete must succeed")
assert.Equal(t, "gw2.e2e.netbird.selfhosted", recreated.Endpoint, "the fresh bootstrap claims the new hostname")
}

View File

@@ -66,9 +66,7 @@ func TestProviderSkipTLSVerification(t *testing.T) {
}
}
// First create bootstraps the account cluster.
insecureReq := newReq("skip-tls", insecureModel, true)
insecureReq.BootstrapCluster = ptr(harness.AgentNetworkCluster)
insecureProv, err := srv.CreateProvider(ctx, insecureReq)
require.NoError(t, err, "create skip-tls provider")
t.Cleanup(func() { _ = srv.DeleteProvider(context.Background(), insecureProv.Id) })

View File

@@ -57,12 +57,11 @@ func TestVLLMProvider(t *testing.T) {
// is enumerated so the router dispatches this model string to this provider.
dummyKey := "sk-vllm-e2e"
prov, err := srv.CreateProvider(ctx, api.AgentNetworkProviderRequest{
Name: "vllm",
ProviderId: "vllm",
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
BootstrapCluster: ptr(harness.AgentNetworkCluster),
Name: "vllm",
ProviderId: "vllm",
UpstreamUrl: vllm.URL,
ApiKey: &dummyKey,
Enabled: ptr(true),
Models: &[]api.AgentNetworkProviderModel{
{Id: harness.VLLMModel, InputPer1k: 0.001, OutputPer1k: 0.002},
},

View File

@@ -126,17 +126,33 @@ func (c *Combined) DeleteGuardrail(ctx context.Context, id string) error {
return anDelete(ctx, c, "/api/agent-network/guardrails/"+id)
}
// GetSettings returns the account's agent-network settings row. It exists only
// after the first provider create bootstraps it.
// CreateSettings bootstraps the account's agent-network settings row,
// assigning the immutable endpoint. Exactly one of req.ProxyAddress (labeled
// endpoint beneath that cluster) and req.Endpoint (self-addressed dedicated
// endpoint) must be set; a second bootstrap returns a conflict.
func (c *Combined) CreateSettings(ctx context.Context, req api.AgentNetworkSettingsCreateRequest) (api.AgentNetworkSettings, error) {
return anRequest[api.AgentNetworkSettings](ctx, c, http.MethodPost, "/api/agent-network/settings", req)
}
// GetSettings returns the account's agent-network settings row. Before the
// CreateSettings bootstrap it reads as the defaults with an empty endpoint.
func (c *Combined) GetSettings(ctx context.Context) (api.AgentNetworkSettings, error) {
return anRequest[api.AgentNetworkSettings](ctx, c, http.MethodGet, "/api/agent-network/settings", nil)
}
// UpdateSettings applies the mutable collection toggles.
// UpdateSettings applies the mutable collection toggles. The request must
// echo the assigned endpoint and proxy address unchanged — the server rejects
// a PUT that tries to change them.
func (c *Combined) UpdateSettings(ctx context.Context, req api.AgentNetworkSettingsRequest) (api.AgentNetworkSettings, error) {
return anRequest[api.AgentNetworkSettings](ctx, c, http.MethodPut, "/api/agent-network/settings", req)
}
// DeleteSettings removes the account's settings row, releasing the endpoint.
// Refused while providers exist or a proxy is actively serving the endpoint.
func (c *Combined) DeleteSettings(ctx context.Context) error {
return anDelete(ctx, c, "/api/agent-network/settings")
}
// ListConsumption returns the account's consumption rows (possibly empty).
func (c *Combined) ListConsumption(ctx context.Context) ([]api.AgentNetworkConsumption, error) {
return anRequest[[]api.AgentNetworkConsumption](ctx, c, http.MethodGet, "/api/agent-network/consumption", nil)

7
go.mod
View File

@@ -81,7 +81,7 @@ require (
github.com/miekg/dns v1.1.72
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/moby/moby/api v1.54.1
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45
github.com/oapi-codegen/runtime v1.1.2
github.com/okta/okta-sdk-golang/v2 v2.18.0
@@ -99,7 +99,7 @@ require (
github.com/pires/go-proxyproto v0.11.0
github.com/pkg/sftp v1.13.9
github.com/prometheus/client_golang v1.23.2
github.com/quic-go/quic-go v0.55.0
github.com/quic-go/quic-go v0.59.1
github.com/redis/go-redis/v9 v9.7.3
github.com/rs/xid v1.3.0
github.com/shirou/gopsutil/v4 v4.25.8
@@ -239,7 +239,6 @@ require (
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/puddle/v2 v2.2.1 // indirect
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
@@ -340,4 +339,4 @@ replace github.com/dexidp/dex/api/v2 => github.com/netbirdio/dex/api/v2 v2.0.0-2
replace github.com/mailru/easyjson => github.com/netbirdio/easyjson v0.9.0
replace github.com/wailsapp/wails/v3 => github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260807055527-fc03f984d701
replace github.com/wailsapp/wails/v3 => github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260810103952-24e716aea4db

15
go.sum
View File

@@ -349,8 +349,6 @@ github.com/jackc/puddle/v2 v2.2.1 h1:RhxXJtFG022u4ibrCSMSiu5aOq1i77R3OHKNJj77OAk
github.com/jackc/puddle/v2 v2.2.1/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=
github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 h1:njuLRcjAuMKr7kI3D85AXWkw6/+v9PwtV6M6o11sWHQ=
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1/go.mod h1:alcuEEnZsY1WQsagKhZDsoPCRoOijYqhZvPwLG0kzVs=
github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
github.com/jcmturner/aescts/v2 v2.0.0/go.mod h1:AiaICIRyfYg35RUkr8yESTqvSy7csK90qZ5xfvvsoNs=
github.com/jcmturner/dnsutils/v2 v2.0.0 h1:lltnkeZGL0wILNvrNiVCR6Ro5PGU/SeBvVO/8c/iPbo=
@@ -484,14 +482,14 @@ github.com/netbirdio/easyjson v0.9.0 h1:6Nw2lghSVuy8RSkAYDhDv1thBVEmfVbKZnV7T7Z6
github.com/netbirdio/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51 h1:Ov4qdafATOgGMB1wbSuh+0aAHcwz9hdvB6VZjh1mVMI=
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51/go.mod h1:ZSIbPdBn5hePO8CpF1PekH2SfpTxg1PDhEwtbqZS7R8=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42 h1:F3zS5fT9xzD1OFLfcdAE+3FfyiwjGukF1hvj0jErgs8=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42/go.mod h1:n47r67ZSPgwSmT/Z1o48JjZQW9YJ6m/6Bd/uAXkL3Pg=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87 h1:iJeUvSMC0BTpkw7u4JyWcY4/3dl7fEL9DR/TpKf2+1w=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87/go.mod h1:pmsCPx1S0nuZRxCextGpc9AV4hLgGSuTsc4NMuwGeCo=
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502 h1:3tHlFmhTdX9axERMVN63dqyFqnvuD+EMJHzM7mNGON8=
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502/go.mod h1:CIMRFEJVL+0DS1a3Nx06NaMn4Dz63Ng6O7dl0qH0zVM=
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45 h1:ujgviVYmx243Ksy7NdSwrdGPSRNE3pb8kEDSpH0QuAQ=
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45/go.mod h1:5/sjFmLb8O96B5737VCqhHyGRzNFIaN/Bu7ZodXc3qQ=
github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260807055527-fc03f984d701 h1:QL9nupfRom0L9jcY7N9l/Bc6QK2PtC6pHzC+ftpTqpw=
github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260807055527-fc03f984d701/go.mod h1:BzATbK71VFikMMMCo434wAi0QcaI03P+xeaWgDvQvjw=
github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260810103952-24e716aea4db h1:gBOE2r4AW1soSmpYJC5/n9/1L8UQ8+HLjed8CY/TzZY=
github.com/netbirdio/wails/v3 v3.0.0-beta.3.0.20260810103952-24e716aea4db/go.mod h1:bsdahLwBQxXjlmdPPeQyrTcDJfcqAr/ymFj0RXhwtWI=
github.com/netbirdio/wireguard-go v0.0.0-20260628102922-2834bebf6c1a h1:3CWK+yTvRKOcC0Q8VCTGy4l60TEb27CQVS7LkMxwjmw=
github.com/netbirdio/wireguard-go v0.0.0-20260628102922-2834bebf6c1a/go.mod h1:rpwXGsirqLqN2L0JDJQlwOboGHmptD5ZD6T2VmcqhTw=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
@@ -582,8 +580,8 @@ github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEo
github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM=
github.com/prometheus/procfs v0.19.2 h1:zUMhqEW66Ex7OXIiDkll3tl9a1ZdilUOd/F6ZXw4Vws=
github.com/prometheus/procfs v0.19.2/go.mod h1:M0aotyiemPhBCM0z5w87kL22CxfcH05ZpYlu+b4J7mw=
github.com/quic-go/quic-go v0.55.0 h1:zccPQIqYCXDt5NmcEabyYvOnomjs8Tlwl7tISjJh9Mk=
github.com/quic-go/quic-go v0.55.0/go.mod h1:DR51ilwU1uE164KuWXhinFcKWGlEjzys2l8zUl5Ss1U=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/redis/go-redis/v9 v9.7.3 h1:YpPyAayJV+XErNsatSElgRZZVCwXX9QzkKYNvO7x0wM=
github.com/redis/go-redis/v9 v9.7.3/go.mod h1:bGUrSggJ9X9GUmZpZNEOQKaANxSGgOEBRltRTZHSvrA=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
@@ -793,7 +791,6 @@ golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200810151505-1b9f1253b3ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

View File

@@ -0,0 +1,58 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"time"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetAccountSettings(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into accounts (id, settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
values('account-3',null,null,null,null,null,null,null,null,null,null,null)`)
accountSettings, err := conn(t, ctx).GetAccountSettings(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
PeerLoginExpirationEnabled: true,
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
PeerInactivityExpirationEnabled: false,
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
DNSDomain: "",
IPv6EnabledGroups: []string{"group-one-resource-id"},
RoutingPeerDNSResolutionEnabled: false,
LazyConnectionEnabled: false,
AutoUpdateVersion: "disabled",
AutoUpdateAlways: false,
MetricsPushEnabled: false,
})
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
PeerLoginExpirationEnabled: true,
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
PeerInactivityExpirationEnabled: false,
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
DNSDomain: "",
IPv6EnabledGroups: []string{"group-two-resources-id"},
RoutingPeerDNSResolutionEnabled: false,
LazyConnectionEnabled: false,
AutoUpdateVersion: "disabled",
AutoUpdateAlways: false,
MetricsPushEnabled: false,
})
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-3")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{})
}

View File

@@ -0,0 +1,22 @@
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
VALUES('account-1','network-1','{"IP":"100.103.0.0","Mask":"//8AAA=="}','{"IP":"fdde:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',1,'["disabled-group-1","disabled-group-2"]',
true, 86400000000000, false,
86400000000000, null, '["group-one-resource-id"]', false,
false, 'disabled', false, false);
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
VALUES('account-2','network-2','{"IP":"110.0.0.0","Mask":"//8AAA=="}','{"IP":"fddf:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',2,null,
true, 86400000000000, false,
86400000000000, null, '["group-two-resources-id"]', false,
false, 'disabled', false, false);
insert into groups (id, account_id, name, resources, public_id) VALUES('group-one-resource-id','account-1','group-1-name', '[{"ID":"host-id-1","Type":"host"}]','group-one-resource-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('group-two-resources-id','account-1','group-2-name', '[{"ID":"subnet-id-1","Type":"subnet"}, {"ID":"host-id-2","Type":"host"}]','group-two-resources-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('group-no-resources-id','account-1','group-3-name', null,'group-no-resources-id-public');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-1','group-one-resource-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-2','group-two-resources-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-3','group-two-resources-id');

View File

@@ -0,0 +1,25 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetDnsSettings(t *testing.T) {
ctx := context.TODO()
settings, err := conn(t, ctx).GetDnsSettings(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, settings, nmdata.DNSSettings{
DisabledManagementGroups: []string{"disabled-group-1", "disabled-group-2"},
})
settings, err = conn(t, ctx).GetDnsSettings(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, settings, nmdata.DNSSettings{})
}

View File

@@ -0,0 +1,62 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/miekg/dns"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetAppliedZoneCandidatesViaPgxConnection(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
VALUES('zone-1','account-1','test-1.com',true,'["group-one-resource-id"]')`)
execQuery(t, ctx,
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
VALUES('zone-2','account-1','test-2.com',false,'["group-two-resources-id"]')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-1','account-1','zone-1','test.test-1.com','A',1800,'1.1.1.1')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-2','account-1','zone-1','test2.test-1.com','A',1800,'1.1.1.2')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-3','account-1','zone-1','test3.test-1.com','CNAME',1800,'test4.test-1.com')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-4','account-1','zone-2','test2.test-2.com','CNAME',1800,'test3.test-2.com')`)
zoneCandidates, err := conn(t, ctx).GetAppliedZoneCandidates(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
DistributionGroups: []string{"group-one-resource-id"},
Zone: nmdata.CustomZone{
Domain: "test-1.com",
SearchDomainDisabled: false,
Records: []nmdata.SimpleRecord{
{Name: "test.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.1"},
{Name: "test2.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.2"},
{Name: "test3.test-1.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test4.test-1.com."},
},
},
})
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
DistributionGroups: []string{"group-two-resources-id"},
Zone: nmdata.CustomZone{
Domain: "test-2.com",
SearchDomainDisabled: true,
Records: []nmdata.SimpleRecord{
{Name: "test2.test-2.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test3.test-2.com."},
},
},
})
}

View File

@@ -0,0 +1,39 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"database/sql"
"testing"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/stretchr/testify/assert"
)
func TestGetDomains(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-1','account-1','test-1.com','target-1.cluster.local')`)
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-2','account-1','test-2.com','target-2.cluster.local')`)
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-3','account-1',null,null)`)
domains, err := conn(t, ctx).GetDomains(ctx, "account-1")
assert.NoError(t, err)
assert.Len(t, domains, 2)
assert.Contains(t, domains, networkmapdb.Domain{
Domain: sql.NullString{String: "test-1.com", Valid: true},
TargetCluster: sql.NullString{String: "target-1.cluster.local", Valid: true},
})
assert.Contains(t, domains, networkmapdb.Domain{
Domain: sql.NullString{String: "test-2.com", Valid: true},
TargetCluster: sql.NullString{String: "target-2.cluster.local", Valid: true},
})
}

View File

@@ -0,0 +1,58 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGetGroups(t *testing.T) {
ctx := context.TODO()
groups, resourceToGroupIdx, err := conn(t, ctx).GetGroups(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t,
groups,
nmdata.Group{ID: "group-one-resource-id", Name: "group-1-name", PublicID: "group-one-resource-id-public", Resources: []nmdata.Resource{{ID: "host-id-1", Type: "host"}}, Peers: []string{"peer-id-1"}},
)
assert.NotNil(t, resourceToGroupIdx["host-id-1"]["group-one-resource-id"])
assert.Contains(t,
groups,
nmdata.Group{ID: "group-two-resources-id", Name: "group-2-name", PublicID: "group-two-resources-id-public",
Resources: []nmdata.Resource{{ID: "subnet-id-1", Type: "subnet"}, {ID: "host-id-2", Type: "host"}},
Peers: []string{"peer-id-2", "peer-id-3"}},
)
assert.NotNil(t, resourceToGroupIdx["host-id-2"]["group-two-resources-id"])
assert.NotNil(t, resourceToGroupIdx["subnet-id-1"]["group-two-resources-id"])
assert.Contains(t,
groups,
nmdata.Group{ID: "group-no-resources-id", Name: "group-3-name", PublicID: "group-no-resources-id-public"})
}
// Verify handling of empty fields in groups table
// Verify that group's PublicID gets populated on retrieval
// TODO (dmitri) PublicID should not be populated with delta updates,
// which require stable PublicIDs
func TestGetGroupsWithoutExpectedFields(t *testing.T) {
if engine == string(types.SqliteStoreEngine) {
t.Skip()
}
ctx := context.TODO()
execQuery(t, ctx,
"insert into accounts (id) VALUES('random-id')")
execQuery(t, ctx,
"insert into groups (id, account_id) VALUES('g2-test-group-id-1','random-id')")
groups, _, err := conn(t, ctx).GetGroups(ctx, "random-id")
assert.NoError(t, err)
require.Len(t, groups, 1)
assert.NotEmpty(t, groups[0].PublicID)
}

View File

@@ -0,0 +1,99 @@
//go:build integration
package networkmap_pgsql
import (
"context"
_ "embed"
"os"
"testing"
"time"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
"github.com/netbirdio/netbird/management/server/types"
)
//go:embed base_data.sql
var baseData string
//go:embed pg_data.sql
var pgData string
//go:embed sqlite_data.sql
var sqliteData string
var (
pgstore *networkmap_pgsql.PgStore
sqlitestore *networkmap_sqlite.SqliteStore
engine string
)
func TestMain(m *testing.M) {
var cleanup func()
kind, _ := os.LookupEnv("NETBIRD_STORE_ENGINE")
switch kind {
case string(types.PostgresStoreEngine):
engine = string(types.PostgresStoreEngine)
pgstore, cleanup = createPGTestStore(baseData, pgData)
case "", string(types.SqliteStoreEngine):
engine = string(types.SqliteStoreEngine)
sqlitestore, cleanup = createSqliteTestStore(baseData, sqliteData)
default:
log.Fatalf("unsupported db '%s' in NETBIRD_STORE_ENGINE env var", kind)
}
code := m.Run()
cleanup()
os.Exit(code)
}
func conn(t *testing.T, ctx context.Context) networkmapdb.NetworkMapDBStoreConn {
t.Helper()
switch engine {
case string(types.PostgresStoreEngine):
c, err := pgstore.Pool.Acquire(ctx)
assert.NoError(t, err)
return pgstore.UsingConnection(c.Conn())
case string(types.SqliteStoreEngine):
return sqlitestore.UsingConn()
}
log.Fatalf("unknown db engine kind %s", engine)
return nil
}
func execQuery(t *testing.T, ctx context.Context, q string) {
t.Helper()
switch engine {
case string(types.PostgresStoreEngine):
_, err := pgstore.Pool.Exec(ctx, q)
assert.NoError(t, err)
case string(types.SqliteStoreEngine):
_, err := sqlitestore.Db.ExecContext(ctx, q)
assert.NoError(t, err)
}
}
// use to parse time in time.RFC3339Nano format
// returns the time in the local time zone, as that's what being returned from sql queries
// pgx returns time in the "local" timezone
// sql with sqlite driver returns time in UTC timezone
func mustParseTime(t string) *time.Time {
tt, err := time.Parse(time.RFC3339Nano, t)
if err != nil {
panic(err)
}
if engine == string(types.SqliteStoreEngine) {
utc := tt.UTC()
return &utc
}
local := tt.Local()
return &local
}

View File

@@ -0,0 +1,61 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNameServerGroups(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled, "primary", account_id)
VALUES('nsgroup-1','nsgroup-1-public','nsgroup-1','nsgroup-1','[{"IP":"192.168.31.2","NSType":1,"Port":53}]','["group-one-resource-id"]','["test-1.com"]',TRUE,FALSE,TRUE,'account-1')`)
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
VALUES('nsgroup-2','nsgroup-2-public','nsgroup-2','nsgroup-2','[{"IP":"192.168.32.3","NSType":1,"Port":53}]','["group-one-resource-id","group-no-resources-id"]','["test-1.com","test-2.com"]',TRUE,FALSE,TRUE,'account-1')`)
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
VALUES('nsgroup-3','nsgroup-3-public',null,null,null,null,null,TRUE,FALSE,FALSE,'account-1')`)
nsgroups, err := conn(t, ctx).GetNameServerGroups(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-1",
PublicID: "nsgroup-1-public",
Name: "nsgroup-1",
Description: "nsgroup-1",
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.31.2"), NSType: 1, Port: 53}},
Groups: []string{"group-one-resource-id"},
Domains: []string{"test-1.com"},
Primary: true,
SearchDomainsEnabled: false,
Enabled: true,
})
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-2",
PublicID: "nsgroup-2-public",
Name: "nsgroup-2",
Description: "nsgroup-2",
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.32.3"), NSType: 1, Port: 53}},
Groups: []string{"group-one-resource-id", "group-no-resources-id"},
Domains: []string{"test-1.com", "test-2.com"},
Primary: true,
SearchDomainsEnabled: false,
Enabled: true,
})
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-3",
PublicID: "nsgroup-3-public",
Primary: false,
SearchDomainsEnabled: false,
Enabled: true,
})
}

View File

@@ -0,0 +1,65 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetworkResources(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-1','account-1','network-1','net-resource-public-1','network-resource-1','network-resource-1','subnet','','"10.0.0.0/16"',TRUE)`)
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-2','account-1','network-2','net-resource-public-2','network-resource-2','network-resource-2','domain','test.com','',TRUE)`)
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-3','account-1','network-3','net-resource-public-3','network-resource-3','network-resource-3','host','','"10.0.0.1/32"',TRUE)`)
resources, err := conn(t, ctx).GetNetworkResources(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-1",
AccountID: "account-1",
NetworkID: "network-1",
PublicID: "net-resource-public-1",
Name: "network-resource-1",
Description: "network-resource-1",
Type: "subnet",
Domain: "",
Prefix: netip.MustParsePrefix("10.0.0.0/16"),
Enabled: true,
})
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-2",
AccountID: "account-1",
NetworkID: "network-2",
PublicID: "net-resource-public-2",
Name: "network-resource-2",
Description: "network-resource-2",
Type: "domain",
Domain: "test.com",
Enabled: true,
})
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-3",
AccountID: "account-1",
NetworkID: "network-3",
PublicID: "net-resource-public-3",
Name: "network-resource-3",
Description: "network-resource-3",
Type: "host",
Domain: "",
Prefix: netip.MustParsePrefix("10.0.0.1/32"),
Enabled: true,
})
}

View File

@@ -0,0 +1,33 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetworkRouters(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-1','account-1','public-id-1','peer-id-1','network-id-1',TRUE,999,TRUE,'["group-one-resource-id"]')`)
execQuery(t, ctx,
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-2','account-1','public-id-2','','network-id-2',TRUE,333,TRUE,'["group-two-resources-id","group-no-resources-id"]')`)
routers, err := conn(t, ctx).GetNetworkRouters(ctx, "account-1")
assert.NoError(t, err)
assert.NotEmpty(t, routers)
assert.Equal(t, routers["network-id-1"],
map[string]*nmdata.NetworkRouter{"peer-id-1": {PublicID: "public-id-1", Masquerade: true, Metric: 999, Enabled: true, PeerGroups: []string{"group-one-resource-id"}}})
assert.Equal(t, routers["network-id-2"],
map[string]*nmdata.NetworkRouter{
"peer-id-2": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}},
"peer-id-3": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}}})
}

View File

@@ -0,0 +1,56 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"encoding/json"
"net"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetwork(t *testing.T) {
ctx := context.TODO()
network, err := conn(t, ctx).GetNetwork(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, network, nmdata.Network{
Identifier: "network-1",
Net: mustParseCIDR("100.103.0.0/16"),
NetV6: mustParseCIDR("fdde:e995:fd38:a465::/64"),
Serial: 1,
})
network, err = conn(t, ctx).GetNetwork(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, network, nmdata.Network{
Identifier: "network-2",
Net: mustParseCIDR("110.0.0.0/16"),
NetV6: mustParseCIDR("fddf:e995:fd38:a465::/64"),
Serial: 2,
})
}
func mustParseCIDR(s string) net.IPNet {
var toret net.IPNet
_, net, err := net.ParseCIDR(s)
if err != nil {
panic(err)
}
jn, err := json.Marshal(net)
if err != nil {
panic(err)
}
err = json.Unmarshal(jn, &toret)
if err != nil {
panic(err)
}
return toret
}

View File

@@ -0,0 +1,26 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetNetworks(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into networks (id, account_id, public_id) VALUES('network-1','account-1','network-1-public')`)
execQuery(t, ctx,
`insert into networks (id, account_id, public_id) VALUES('network-2','account-1','network-2-public')`)
networksIdx, err := conn(t, ctx).GetNetworkXIDToPublicIdMap(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, networksIdx, map[string]string{
"network-1": "network-1-public",
"network-2": "network-2-public",
})
}

View File

@@ -0,0 +1,163 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPeers(t *testing.T) {
ctx := context.TODO()
peers, clusterToPeersIdx, err := conn(t, ctx).GetPeers(ctx, "account-1")
assert.NoError(t, err)
// shouldn't be returned in the index, as it's not connected
execQuery(t, ctx,
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
values('peer-4','account-1','key-4','ssh-key-4',true,false)`)
// shouldn't be returned in the index as it doesn't have cluster set
execQuery(t, ctx,
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
values('peer-5','account-1','key-5','ssh-key-5',false,true)`)
peer1 := nmdata.Peer{
ID: "peer-id-1",
Key: "key-1",
SSHKey: "ssh-key-1",
DNSLabel: "peer-1",
ExtraDNSLabels: []string{"extra-peer-1"},
UserID: "user-id-1",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T13:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.10.1"),
IPv6: netip.MustParseAddr("fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.0",
GoOS: "linux",
OSVersion: "26.4.1",
KernelVersion: "6.8.0-134-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe80::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.16.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 1,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.148.187"),
},
}
peer2 := nmdata.Peer{
ID: "peer-id-2",
Key: "key-2",
SSHKey: "ssh-key-2",
DNSLabel: "peer-2",
ExtraDNSLabels: []string{"extra-peer-2"},
UserID: "user-id-2",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T14:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.100.1"),
IPv6: netip.MustParseAddr("fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.1",
GoOS: "linux",
OSVersion: "26.4.2",
KernelVersion: "6.8.0-135-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe81::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.17.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 0,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.149.187"),
},
}
peer3 := nmdata.Peer{
ID: "peer-id-3",
Key: "key-3",
SSHKey: "ssh-key-3",
DNSLabel: "peer-3",
ExtraDNSLabels: []string{"extra-peer-3"},
UserID: "user-id-3",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T12:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.200.1"),
IPv6: netip.MustParseAddr("fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.2",
GoOS: "linux",
OSVersion: "26.4.3",
KernelVersion: "6.8.0-136-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe82::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.18.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 1,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.150.187"),
},
}
assert.Contains(t, peers, peer1)
assert.Contains(t, peers, peer2)
assert.Contains(t, peers, peer3)
assert.Equal(t, clusterToPeersIdx, map[string][]*nmdata.Peer{
"cluster-1.netbird.services": {&peer1},
"cluster-2.netbird.services": {&peer2},
"cluster-3.netbird.services": {&peer3},
})
}

View File

@@ -0,0 +1,30 @@
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-1','account-1','key-1','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999+00','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-1.netbird.services',
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.148.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-2','account-1','key-2','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999+00','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-2.netbird.services',
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
'DE','Berlin','"46.201.149.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-3','account-1','key-3','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999+00','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-3.netbird.services',
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.150.187"');

View File

@@ -0,0 +1,122 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"fmt"
"regexp"
"slices"
"strings"
"time"
log "github.com/sirupsen/logrus"
"github.com/google/uuid"
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
gormstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/testutil"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func createPGTestStore(baseData, pgData string) (*networkmap_pgsql.PgStore, func()) {
_, tmpdsn, err := testutil.CreatePostgresTestContainer()
if err != nil {
log.Fatalf("error starting postres container %v", err)
}
var db *gorm.DB
for i := range 5 {
db, err = gorm.Open(postgres.Open(tmpdsn), &gorm.Config{})
if err == nil {
break
}
if i < 5 {
waitTime := time.Duration(100*(i+1)) * time.Millisecond
time.Sleep(waitTime)
continue
}
log.Fatalf("error connecting to postres db %v", err)
}
var cleanup func()
dsn, cleanup, err := createRandomDB(tmpdsn, db)
sqlDB, _ := db.DB()
if sqlDB != nil {
sqlDB.Close()
}
if err != nil {
log.Fatalf("error creating postres db %v", err)
}
_, err = gormstore.NewPostgresqlStoreForTests(context.TODO(), dsn, nil, false)
if err != nil {
log.Fatalf("error running migrations %v", err)
}
ctx := context.TODO()
pgstore, err := networkmap_pgsql.NewPostgresqlStore(ctx, dsn)
if err != nil {
log.Fatal("error creating postgres store %w", err)
}
for _, query := range slices.Concat(strings.Split(baseData, ";"), strings.Split(pgData, ";")) {
if _, err := pgstore.Pool.Exec(ctx, query); err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
}
return pgstore, cleanup
}
func createRandomDB(dsn string, db *gorm.DB) (string, func(), error) {
dbName := fmt.Sprintf("test_db_%s", strings.ReplaceAll(uuid.New().String(), "-", "_"))
if err := db.Exec(fmt.Sprintf("CREATE DATABASE %s", dbName)).Error; err != nil {
return "", nil, fmt.Errorf("failed to create database: %v", err)
}
originalDSN := dsn
cleanup := func() {
var dropDB *gorm.DB
var err error
dropDB, err = gorm.Open(postgres.Open(originalDSN), &gorm.Config{
SkipDefaultTransaction: true,
PrepareStmt: false,
})
if err != nil {
log.Errorf("failed to connect for dropping database %s: %v", dbName, err)
return
}
defer func() {
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
sqlDB.Close()
}
}()
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
sqlDB.SetMaxOpenConns(1)
sqlDB.SetMaxIdleConns(0)
sqlDB.SetConnMaxLifetime(time.Second)
}
err = dropDB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s WITH (FORCE)", dbName)).Error
if err != nil {
log.Errorf("failed to drop database %s: %v", dbName, err)
}
}
return replaceDBName(dsn, dbName), cleanup, nil
}
func replaceDBName(dsn, newDBName string) string {
re := regexp.MustCompile(`(?P<pre>[:/@])(?P<dbname>[^/?]+)(?P<post>\?|$)`)
return re.ReplaceAllString(dsn, `${pre}`+newDBName+`${post}`)
}

View File

@@ -0,0 +1,146 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPolicies(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-1','policy-1-public','account-1',true,'["posture-checks-1","posture-checks-2"]')`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-1-rule-1','policy-1',true,'accept','tcp',true,'["group-one-resource-id","group-two-resources-id"]','["group-one-resource-id","group-two-resources-id"]',
'{"ID":"host-id-1","Type":"host"}','{"ID":"domain-1","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
'{"group-one-resource-id":["user-1", "user-2"]}','user-3')`)
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-2','policy-2-public','account-1',true,'["posture-checks-3","posture-checks-4"]')`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-2-rule-1','policy-2',true,'accept','tcp',true,'["group-one-resource-id"]','["group-two-resources-id"]',
'{"ID":"host-id-3","Type":"host"}','{"ID":"domain-3","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
'{"group-one-resource-id":["user-6", "user-7"]}','user-8')`)
// policy with a rule with null fields
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-3','policy-3-public','account-1',true,null)`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-3-rule-1','policy-3',true,null,null,null,null,null,null,null,null,null,null,null)`)
// policy with a disabled rule, destination resource and groups should not be in indexes
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-4','policy-4-public','account-1',true,null)`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-4-rule-1','policy-4',false,null,null,null,null,'["group-two-resources-id"]',
null,'{"ID":"domain-3","Type":"domain"}',null,null,null,null)`)
policies, policyToDestinationResourceIdx, policyToDestinationGroupIdx, err := conn(t, ctx).GetPolicies(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-1",
PublicID: "policy-1-public",
Enabled: true,
SourcePostureChecks: []string{"posture-checks-1", "posture-checks-2"},
Rules: []*nmdata.PolicyRule{
{
ID: "policy-1",
PolicyID: "policy-1",
Enabled: true,
Action: "accept",
Protocol: "tcp",
Bidirectional: true,
Sources: []string{"group-one-resource-id", "group-two-resources-id"},
Destinations: []string{"group-one-resource-id", "group-two-resources-id"},
SourceResource: nmdata.Resource{ID: "host-id-1", Type: "host"},
DestinationResource: nmdata.Resource{ID: "domain-1", Type: "domain"},
Ports: []string{"8080", "8443"},
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-1", "user-2"}},
AuthorizedUser: "user-3",
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-2",
PublicID: "policy-2-public",
Enabled: true,
SourcePostureChecks: []string{"posture-checks-3", "posture-checks-4"},
Rules: []*nmdata.PolicyRule{
{
ID: "policy-2",
PolicyID: "policy-2",
Enabled: true,
Action: "accept",
Protocol: "tcp",
Bidirectional: true,
Sources: []string{"group-one-resource-id"},
Destinations: []string{"group-two-resources-id"},
SourceResource: nmdata.Resource{ID: "host-id-3", Type: "host"},
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
Ports: []string{"8080", "8443"},
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-6", "user-7"}},
AuthorizedUser: "user-8",
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-3",
PublicID: "policy-3-public",
Enabled: true,
SourcePostureChecks: nil,
Rules: []*nmdata.PolicyRule{
{
ID: "policy-3",
PolicyID: "policy-3",
Enabled: true,
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-4",
PublicID: "policy-4-public",
Enabled: true,
SourcePostureChecks: nil,
Rules: []*nmdata.PolicyRule{
{
ID: "policy-4",
PolicyID: "policy-4",
Enabled: false,
Destinations: []string{"group-two-resources-id"},
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
},
},
})
assert.Equal(t, policyToDestinationGroupIdx, map[string]map[string]any{
"policy-1": {"group-one-resource-id": struct{}{}, "group-two-resources-id": struct{}{}},
"policy-2": {"group-two-resources-id": struct{}{}},
})
assert.Equal(t, policyToDestinationResourceIdx, map[string]map[string]any{
"policy-1": {"domain-1": struct{}{}},
"policy-2": {"domain-3": struct{}{}},
})
}

View File

@@ -0,0 +1,61 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPostureChecks(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-1','account-1','posturecheck-1-public',
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
"OSVersionCheck":{"Darwin":{"MinVersion":"12.0"}},
"GeoLocationCheck":{"Locations":[{"CountryCode":"FI","CityName":""}],"Action":"allow"},
"PeerNetworkRangeCheck":{"Action":"deny","Ranges":["192.168.0.1/24"]}}')`)
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-2','account-1','posturecheck-2-public',
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
"OSVersionCheck":{"Android":{"MinVersion":"0"}},
"GeoLocationCheck":{"Locations":[{"CountryCode":"US","CityName":"Harker Heights"}],"Action":"allow"},
"PeerNetworkRangeCheck":{"Action":"allow","Ranges":["0.0.0.0/0"]}}')`)
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-3','account-1','posturecheck-3-public', null)`)
postureChecks, idToPublicIDIdx, err := conn(t, ctx).GetPostureChecks(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, idToPublicIDIdx, map[string]string{
"posturecheck-1": "posturecheck-1-public",
"posturecheck-2": "posturecheck-2-public",
"posturecheck-3": "posturecheck-3-public",
})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-1",
Checks: nmdata.ChecksDefinition{
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
OSVersionCheck: &nmdata.OSVersionCheck{Darwin: &nmdata.MinVersionCheck{MinVersion: "12.0"}},
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "FI"}}, Action: "allow"},
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "deny", Ranges: []netip.Prefix{netip.MustParsePrefix("192.168.0.1/24")}},
}})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-2",
Checks: nmdata.ChecksDefinition{
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
OSVersionCheck: &nmdata.OSVersionCheck{Android: &nmdata.MinVersionCheck{MinVersion: "0"}},
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "US", CityName: "Harker Heights"}}, Action: "allow"},
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "allow", Ranges: []netip.Prefix{netip.MustParsePrefix("0.0.0.0/0")}},
}})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-3"})
}

View File

@@ -0,0 +1,87 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/domain"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetRoutes(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-1','account-1','route-1-public','"172.0.0.0/16"','["test-1.com"]',true,'route-1-net-id','route-1',
'peer-id-1','["group-one-resource-id"]',1,true,9999,true,
'["group-one-resource-id"]','["group-one-resource-id"]',false)`)
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-2','account-1','route-2-public','"172.10.0.0/16"','["test-1.com","test-2.com"]',true,'route-2-net-id','route-2',
'peer-id-2','["group-two-resources-id"]',1,true,9999,true,
'["group-two-resources-id"]','["group-two-resources-id"]',false)`)
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-3','account-1','route-3-public',null,null,null,null,'route-3',
null,null,null,null,null,null,null,null,null)`)
routes, err := conn(t, ctx).GetRoutes(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, routes, nmdata.Route{
ID: "route-1",
AccountID: "account-1",
PublicID: "route-1-public",
Network: netip.MustParsePrefix("172.0.0.0/16"),
Domains: domain.List{"test-1.com"},
KeepRoute: true,
NetID: "route-1-net-id",
Description: "route-1",
Peer: "peer-id-1",
PeerID: "peer-id-1",
PeerGroups: []string{"group-one-resource-id"},
NetworkType: 1,
Masquerade: true,
Metric: 9999,
Enabled: true,
Groups: []string{"group-one-resource-id"},
AccessControlGroups: []string{"group-one-resource-id"},
SkipAutoApply: false,
})
assert.Contains(t, routes, nmdata.Route{
ID: "route-2",
AccountID: "account-1",
PublicID: "route-2-public",
Network: netip.MustParsePrefix("172.10.0.0/16"),
Domains: domain.List{"test-1.com", "test-2.com"},
KeepRoute: true,
NetID: "route-2-net-id",
Description: "route-2",
Peer: "peer-id-2",
PeerID: "peer-id-2",
PeerGroups: []string{"group-two-resources-id"},
NetworkType: 1,
Masquerade: true,
Metric: 9999,
Enabled: true,
Groups: []string{"group-two-resources-id"},
AccessControlGroups: []string{"group-two-resources-id"},
SkipAutoApply: false,
})
assert.Contains(t, routes, nmdata.Route{
ID: "route-3",
AccountID: "account-1",
PublicID: "route-3-public",
Description: "route-3",
})
}

View File

@@ -0,0 +1,109 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"database/sql"
"testing"
"github.com/stretchr/testify/assert"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
)
func TestGetPrivateServices(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-1','account-1',true,true,'["group-one-resource-id"]','test-1.com','test-2.com')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-2','account-1',true,true,'["group-one-resource-id","group-two-resources-id"]','test-3.com','test-4.com')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-3','account-1',null,null,null,null,null)`)
services, err := conn(t, ctx).GetPrivateServices(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: true, Valid: true},
Private: sql.NullBool{Bool: true, Valid: true},
AccessGroups: []string{"group-one-resource-id"},
ProxyCluster: sql.NullString{String: "test-1.com", Valid: true},
Domain: sql.NullString{String: "test-2.com", Valid: true},
})
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: true, Valid: true},
Private: sql.NullBool{Bool: true, Valid: true},
AccessGroups: []string{"group-one-resource-id", "group-two-resources-id"},
ProxyCluster: sql.NullString{String: "test-3.com", Valid: true},
Domain: sql.NullString{String: "test-4.com", Valid: true},
})
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: false, Valid: false},
Private: sql.NullBool{Bool: false, Valid: false},
AccessGroups: []string{},
ProxyCluster: sql.NullString{String: "", Valid: false},
Domain: sql.NullString{String: "", Valid: false},
})
}
func TestGetProxyTargetedDomainResourceIDs(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-4','account-1',true,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-1','account-1','service-4',true,'domain')`)
// id shouldn't be returned as the taget_type is not "domain"
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-2','account-1','service-4',true,'cluster')`)
// id shouldn't be included as the target is disabled
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-3','account-1','service-4',false,'domain')`)
// id shouldn't be included as the service is disabled
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-5','account-1',false,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-4','account-1','service-5',false,'domain')`)
// id shouldn't be included as the service is terminated (explicitly)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-6','account-1',true,true)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-5','account-1','service-6',true,'domain')`)
// id shouldn't be included as the service is terminated (implicitly)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-7','account-1',true,null)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-6','account-1','service-7',true,'domain')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-8','account-1',true,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-7','account-1','service-8',true,'domain')`)
// id shouldn't be returned as the taget_id is null
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values(null,'account-1','service-4',true,'cluster')`)
servtargetedDomains, err := conn(t, ctx).GetProxyTargetedDomainResourceIDs(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, servtargetedDomains, map[string]struct{}{
"target-1": {},
"target-6": {},
"target-7": {},
})
}

View File

@@ -0,0 +1,30 @@
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-1','account-1','key-1','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-1.netbird.services',
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.148.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-2','account-1','key-2','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-2.netbird.services',
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
'DE','Berlin','"46.201.149.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-3','account-1','key-3','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-3.netbird.services',
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.150.187"');

View File

@@ -0,0 +1,49 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"fmt"
"runtime"
"slices"
"strings"
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
gormstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
log "github.com/sirupsen/logrus"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
func createSqliteTestStore(baseData, sqliteData string) (*networkmap_sqlite.SqliteStore, func()) {
storeSqliteFileName := ":memory:"
storeStr := fmt.Sprintf("%s?cache=shared", storeSqliteFileName)
if runtime.GOOS == "windows" {
// Vo avoid `The process cannot access the file because it is being used by another process` on Windows
storeStr = storeSqliteFileName
}
db, err := gorm.Open(sqlite.Open(storeStr), &gorm.Config{})
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
_, err = gormstore.NewSqlStore(context.TODO(), db, types.SqliteStoreEngine, nil, false)
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
sqldb, err := db.DB()
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
for _, query := range slices.Concat(strings.Split(baseData, ";"), strings.Split(sqliteData, ";")) {
if _, err := sqldb.Exec(query); err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
}
return &networkmap_sqlite.SqliteStore{Db: sqldb}, func() {}
}

View File

@@ -0,0 +1,57 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetAllowedUsers(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-1','user-1','account-1','["group-one-resource-id"]',false,false)`)
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-2','user-2','account-1','["group-one-resource-id","group-two-resources-id"]',false,false)`)
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-3','user-3','account-1','["group-two-resources-id"]',false,false)`)
// shouldn't be included as it's blocked
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-4','user-4','account-1','["group-two-resources-id"]',true,false)`)
// shouldn't be included as it's a service_user
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-5','user-5','account-1','["group-two-resources-id"]',false,true)`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-1','All','account-1')`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-2','All','account-1')`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-3','All','account-1')`)
userIdx, groupIdToUserIds, err := conn(t, ctx).GetAllowedUsers(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, userIdx, map[string]struct{}{
"user-1": {},
"user-2": {},
"user-3": {},
})
assert.Equal(t, groupIdToUserIds, map[string][]string{
"group-one-resource-id": {"user-1", "user-2"},
"group-two-resources-id": {"user-2", "user-3"},
"all-group-1": {"user-1", "user-2", "user-3"},
"all-group-2": {"user-1", "user-2", "user-3"},
"all-group-3": {"user-1", "user-2", "user-3"},
})
}

View File

@@ -18,6 +18,7 @@ import (
"github.com/netbirdio/netbird/management/internals/controllers/network_map"
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller/cache"
"github.com/netbirdio/netbird/management/internals/modules/peers/ephemeral"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/internals/shared/grpc"
"github.com/netbirdio/netbird/management/server/account"
@@ -30,6 +31,8 @@ import (
"github.com/netbirdio/netbird/management/server/telemetry"
"github.com/netbirdio/netbird/management/server/types"
sharedgrpc "github.com/netbirdio/netbird/shared/management/grpc"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/netbirdio/netbird/shared/management/proto"
"github.com/netbirdio/netbird/shared/management/status"
"github.com/netbirdio/netbird/util"
@@ -61,6 +64,8 @@ type Controller struct {
serverSupportedSyncMessageVersion sharedgrpc.SyncMessageVersion
perAccountServerSupportedSyncMessageVersions map[string]sharedgrpc.SyncMessageVersion
nmdataStore *networkmapdb.NetworkMapDBStoreImpl
}
type bufferUpdate struct {
@@ -78,7 +83,7 @@ type bufferAffectedUpdate struct {
var _ network_map.Controller = (*Controller)(nil)
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config) *Controller {
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config, nmdataStore *networkmapdb.NetworkMapDBStoreImpl) *Controller {
nMetrics, err := newMetrics(metrics.UpdateChannelMetrics())
if err != nil {
log.Fatal(fmt.Errorf("error creating metrics: %w", err))
@@ -99,6 +104,7 @@ func NewController(ctx context.Context, store store.Store, metrics telemetry.App
EphemeralPeersManager: ephemeralPeersManager,
serverSupportedSyncMessageVersion: sharedgrpc.SyncMessageVersionFromConfig(config.HighestSupportedSyncMessageVersion),
perAccountServerSupportedSyncMessageVersions: sharedgrpc.SyncMessageVersionsFromMap(config.PerAccountHighestSupportedSyncMessageVersion),
nmdataStore: nmdataStore,
}
}
@@ -147,6 +153,11 @@ func (c *Controller) CountStreams() int {
func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID string, reason types.UpdateReason) error {
log.WithContext(ctx).Tracef("updating peers for account %s from %s", accountID, util.GetCallerName())
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.sendUpdateAccountPeersFromData(ctx, accountID, reason, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get account: %v", err)
@@ -167,7 +178,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
return nil
}
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validate peers: %v", err)
}
@@ -255,7 +266,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -276,7 +287,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
}
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -294,6 +305,261 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
return nil
}
// sendUpdateAccountPeersFromData is the account-free variant of
// sendUpdateAccountPeers: everything is computed from the network-map DB
// store's twin data; only extra settings and validated peers are resolved at
// runtime. Proxy network maps and policy injection, private-service zones,
// group-to-user SSH mappings and forced routing-peer DNS resolution have no
// DB-backed source yet and are omitted.
func (c *Controller) sendUpdateAccountPeersFromData(ctx context.Context, accountID string, reason types.UpdateReason, nmData *networkmap.NetworkMapData) error {
peersToUpdate := c.connectedPeersFromData(nmData, nil)
if len(peersToUpdate) == 0 {
return nil
}
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, &reason)
}
// sendUpdateForAffectedPeersFromData is the account-free variant of
// sendUpdateForAffectedPeers.
func (c *Controller) sendUpdateForAffectedPeersFromData(ctx context.Context, accountID string, peerIDs []string, nmData *networkmap.NetworkMapData) error {
if len(peerIDs) == 0 {
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no affected peers")
return nil
}
peersToUpdate := c.connectedPeersFromData(nmData, peerIDs)
if len(peersToUpdate) == 0 {
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no peers to update (affected peers not found in data or no channels)")
return nil
}
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: sending network map to %d connected peers", len(peersToUpdate))
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, nil)
}
// connectedPeersFromData returns the peers with an open update channel. An
// empty affected list means all peers; a non-empty list restricts the result
// to those peer IDs.
func (c *Controller) connectedPeersFromData(nmData *networkmap.NetworkMapData, affected []string) []*nmdata.Peer {
if len(affected) == 0 {
result := make([]*nmdata.Peer, 0, len(nmData.Peers))
for _, peer := range nmData.Peers {
if c.peersUpdateManager.HasChannel(peer.ID) {
result = append(result, peer)
}
}
return result
}
result := make([]*nmdata.Peer, 0, len(affected))
for _, peerID := range affected {
peer := nmData.Peers[peerID]
if peer == nil {
continue
}
if c.peersUpdateManager.HasChannel(peerID) {
result = append(result, peer)
}
}
return result
}
func (c *Controller) sendUpdatesFromData(ctx context.Context, accountID string, nmData *networkmap.NetworkMapData, peersToUpdate []*nmdata.Peer, reason *types.UpdateReason) error {
globalStart := time.Now()
extraSettings, err := c.settingsManager.GetExtraSettings(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get flow enabled status: %v", err)
}
nmData.PrecomputePostureValidation()
dnsCache := &cache.DNSConfigCache{}
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, ipv6AllowedPeersFromData(nmData))
dnsFwdPort := computeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
var wg sync.WaitGroup
semaphore := make(chan struct{}, 10)
for _, peer := range peersToUpdate {
if reason != nil && c.accountManagerMetrics != nil {
c.accountManagerMetrics.CountNmapTriggered(string(reason.Resource), string(reason.Operation))
}
wg.Add(1)
semaphore <- struct{}{}
go func(p *nmdata.Peer) {
defer wg.Done()
defer func() { <-semaphore }()
start := time.Now()
postureChecks := peerPostureChecksFromData(nmData, p.ID)
c.metrics.CountCalcPostureChecksDuration(time.Since(start))
start = time.Now()
peerGroups := maps.Keys(nmData.GetPeerGroups(p.ID))
var update *proto.SyncResponse
commonSyncMessageVersion := sharedgrpc.HighestCommonSyncMessageVersion(
c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion))
log.WithContext(ctx).
WithFields(log.Fields{
"sync_message_version": commonSyncMessageVersion,
"server_sync_message_version": c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
"peer_sync_message_version": sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion),
}).Debug("common highest sync message version")
if commonSyncMessageVersion == sharedgrpc.ComponentNetworkMap {
components := nmData.GetPeerNetworkMapComponents(p.ID, peersCustomZone)
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
start = time.Now()
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, nil, dnsDomain, postureChecks, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
Update: update,
MessageType: network_map.MessageTypeNetworkMap,
})
return
}
nmap := networkMapFromData(ctx, nmData, p.ID, peersCustomZone)
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
Update: update,
MessageType: network_map.MessageTypeNetworkMap,
})
}(peer)
}
wg.Wait()
if c.accountManagerMetrics != nil {
c.accountManagerMetrics.CountUpdateAccountPeersDuration(time.Since(globalStart))
}
return nil
}
func (c *Controller) getNetworkMapData(ctx context.Context, accountID string) *networkmap.NetworkMapData {
if c.nmdataStore == nil {
return nil
}
nmData, err := c.nmdataStore.GetNetworkMapData(ctx, accountID)
if err != nil {
log.WithContext(ctx).Errorf("failed to get network map data for account %s, falling back to account-based computation: %v", accountID, err)
return nil
}
return nmData
}
func (c *Controller) getDNSDomainFromData(settings *nmdata.AccountSettingsInfo) string {
if settings == nil || settings.DNSDomain == "" {
return c.dnsDomain
}
return settings.DNSDomain
}
func ipv6AllowedPeersFromData(nmData *networkmap.NetworkMapData) map[string]struct{} {
result := make(map[string]struct{})
if nmData.AccountSettings != nil {
for _, groupID := range nmData.AccountSettings.IPv6EnabledGroups {
group := nmData.Groups[groupID]
if group == nil {
continue
}
for _, peerID := range group.Peers {
result[peerID] = struct{}{}
}
}
}
for id, p := range nmData.Peers {
if p != nil && p.ProxyMeta.Embedded {
result[id] = struct{}{}
}
}
return result
}
func networkMapFromData(ctx context.Context, nmData *networkmap.NetworkMapData, peerID string, peersCustomZone nmdata.CustomZone) *types.NetworkMap {
components := nmData.GetPeerNetworkMapComponents(peerID, peersCustomZone)
if components.IsEmpty() {
return &types.NetworkMap{Network: components.Network}
}
return types.CalculateNetworkMapFromComponents(ctx, components)
}
// peerPostureChecksFromData mirrors getPeerPostureChecks on the twin store. The
// sync response only encodes process-check file paths, so only ProcessCheck is
// converted back to the server posture type.
func peerPostureChecksFromData(nmData *networkmap.NetworkMapData, peerID string) []*posture.Checks {
if len(nmData.PostureChecks) == 0 {
return nil
}
peerPostureChecks := make(map[string]*posture.Checks)
for _, policy := range nmData.Policies {
if policy == nil || !policy.Enabled || len(policy.SourcePostureChecks) == 0 {
continue
}
if !isPeerInPolicySourceGroupsFromData(nmData, peerID, policy) {
continue
}
for _, checkID := range policy.SourcePostureChecks {
twin := nmData.PostureChecks[checkID]
if twin == nil {
continue
}
peerPostureChecks[checkID] = postureChecksFromTwin(twin)
}
}
return maps.Values(peerPostureChecks)
}
func isPeerInPolicySourceGroupsFromData(nmData *networkmap.NetworkMapData, peerID string, policy *nmdata.Policy) bool {
for _, rule := range policy.Rules {
if rule == nil || !rule.Enabled {
continue
}
for _, groupID := range rule.Sources {
if group := nmData.Groups[groupID]; group != nil && slices.Contains(group.Peers, peerID) {
return true
}
}
}
return false
}
func postureChecksFromTwin(twin *nmdata.PostureChecks) *posture.Checks {
checks := &posture.Checks{ID: twin.ID}
if twin.Checks.ProcessCheck != nil {
processes := make([]posture.Process, 0, len(twin.Checks.ProcessCheck.Processes))
for _, p := range twin.Checks.ProcessCheck.Processes {
processes = append(processes, posture.Process{LinuxPath: p.LinuxPath, MacPath: p.MacPath, WindowsPath: p.WindowsPath})
}
checks.Checks.ProcessCheck = &posture.ProcessCheck{Processes: processes}
}
return checks
}
func (c *Controller) perAccountOrGlobalSupportedSyncMessageVersions(accountId string) sharedgrpc.SyncMessageVersion {
if perAccount, ok := c.perAccountServerSupportedSyncMessageVersions[accountId]; ok {
return perAccount
@@ -326,6 +592,10 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
return nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.sendUpdateForAffectedPeersFromData(ctx, accountID, peerIDs, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get account: %v", err)
@@ -341,7 +611,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeers: sending network map to %d connected peers", len(peersToUpdate))
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validate peers: %v", err)
}
@@ -428,7 +698,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -449,7 +719,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
}
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -506,7 +776,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
return fmt.Errorf("peer %s doesn't exists in account %s", peerId, accountId)
}
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validated peers: %v", err)
}
@@ -566,7 +836,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
Update: update,
@@ -583,7 +853,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
nmap.Merge(proxyNetworkMap)
}
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
Update: update,
@@ -643,7 +913,11 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
if err != nil {
return nil, nil, nil, nil, 0, err
}
return peer, &types.NetworkMapComponents{Network: network.Copy()}, nil, nil, 0, nil
return peer, &types.NetworkMapComponents{Network: types.TwinNetwork(network)}, nil, nil, 0, nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.getValidatedPeerWithComponentsFromData(ctx, accountID, peer, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
@@ -653,7 +927,7 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
c.injectAllProxyPolicies(ctx, account)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, nil, nil, nil, 0, err
}
@@ -690,6 +964,21 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
return peer, components, proxyNetworkMaps[peer.ID], postureChecks, dnsFwdPort, nil
}
// getValidatedPeerWithComponentsFromData is the account-free variant of
// GetValidatedPeerWithComponents. The proxy network map fragment is omitted
// like on the other nmdata paths.
func (c *Controller) getValidatedPeerWithComponentsFromData(ctx context.Context, accountID string, peer *nbpeer.Peer, nmData *networkmap.NetworkMapData) (*nbpeer.Peer, *types.NetworkMapComponents, *types.NetworkMap, []*posture.Checks, int64, error) {
postureChecks := peerPostureChecksFromData(nmData, peer.ID)
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, ipv6AllowedPeersFromData(nmData))
components := nmData.GetPeerNetworkMapComponents(peer.ID, peersCustomZone)
dnsFwdPort := computeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
return peer, components, nil, postureChecks, dnsFwdPort, nil
}
// BufferUpdateAffectedPeers accumulates peer IDs and flushes them after the buffer interval.
func (c *Controller) BufferUpdateAffectedPeers(ctx context.Context, accountID string, peerIDs []string, reason types.UpdateReason) error {
if len(peerIDs) == 0 {
@@ -796,11 +1085,15 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
}
emptyMap := &types.NetworkMap{
Network: network.Copy(),
Network: types.TwinNetwork(network),
}
return emptyMap, nil, 0, nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.getValidatedPeerWithMapFromData(ctx, accountID, peerID, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return nil, nil, 0, err
@@ -808,7 +1101,7 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
c.injectAllProxyPolicies(ctx, account)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, nil, 0, err
}
@@ -848,6 +1141,21 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
return networkMap, postureChecks, dnsFwdPort, nil
}
// getValidatedPeerWithMapFromData is the account-free variant of
// GetValidatedPeerWithMap. The proxy network map fragment is omitted like on
// the other nmdata paths.
func (c *Controller) getValidatedPeerWithMapFromData(ctx context.Context, accountID string, peerID string, nmData *networkmap.NetworkMapData) (*types.NetworkMap, []*posture.Checks, int64, error) {
postureChecks := peerPostureChecksFromData(nmData, peerID)
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, ipv6AllowedPeersFromData(nmData))
networkMap := networkMapFromData(ctx, nmData, peerID, peersCustomZone)
dnsFwdPort := computeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
return networkMap, postureChecks, dnsFwdPort, nil
}
// GetDNSDomain returns the configured dnsDomain
func (c *Controller) GetDNSDomain(settings *types.Settings) string {
if settings == nil {
@@ -910,20 +1218,36 @@ func (c *Controller) StartWarmup(ctx context.Context) {
// computeForwarderPort checks if all peers in the account have updated to a specific version or newer.
// If all peers have the required version, it returns the new well-known port (22054), otherwise returns 0.
func computeForwarderPort(peers []*nbpeer.Peer, requiredVersion string) int64 {
if len(peers) == 0 {
versions := make([]string, 0, len(peers))
for _, peer := range peers {
versions = append(versions, peer.Meta.WtVersion)
}
return computeForwarderPortFromVersions(versions, requiredVersion)
}
func computeForwarderPortFromData(peers map[string]*nmdata.Peer, requiredVersion string) int64 {
versions := make([]string, 0, len(peers))
for _, peer := range peers {
versions = append(versions, peer.Meta.WtVersion)
}
return computeForwarderPortFromVersions(versions, requiredVersion)
}
func computeForwarderPortFromVersions(wtVersions []string, requiredVersion string) int64 {
if len(wtVersions) == 0 {
return int64(network_map.OldForwarderPort)
}
reqVer := semver.Canonical(requiredVersion)
// Check if all peers have the required version or newer
for _, peer := range peers {
for _, wtVersion := range wtVersions {
// Development version is always supported
if version.IsDevelopmentVersion(peer.Meta.WtVersion) {
if version.IsDevelopmentVersion(wtVersion) {
continue
}
peerVersion := semver.Canonical("v" + peer.Meta.WtVersion)
peerVersion := semver.Canonical("v" + wtVersion)
if peerVersion == "" {
// If any peer doesn't have version info, return 0
return int64(network_map.OldForwarderPort)
@@ -1057,7 +1381,7 @@ func (c *Controller) GetNetworkMap(ctx context.Context, peerID string) (*types.N
groups[groupID] = group.Peers
}
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, err
}

View File

@@ -102,8 +102,8 @@ func TestSettingsHandler_GetExposesCollectionToggles(t *testing.T) {
require.NoError(t, f.store.SaveAgentNetworkSettings(context.Background(), &agentNetworkTypes.Settings{
AccountID: testAccountID,
Cluster: "eu.proxy.netbird.io",
Subdomain: "violet",
Domain: "violet.eu.proxy.netbird.io",
ProxyAddress: "eu.proxy.netbird.io",
EnableLogCollection: true,
EnablePromptCollection: true,
RedactPii: false,

View File

@@ -155,12 +155,7 @@ func (h *handler) createProvider(w http.ResponseWriter, r *http.Request) {
provider := types.NewProvider(userAuth.AccountId)
provider.FromAPIRequest(&req)
bootstrapCluster := ""
if req.BootstrapCluster != nil {
bootstrapCluster = *req.BootstrapCluster
}
created, err := h.manager.CreateProvider(r.Context(), userAuth.UserId, provider, bootstrapCluster)
created, err := h.manager.CreateProvider(r.Context(), userAuth.UserId, provider)
if err != nil {
util.WriteError(r.Context(), err, w)
return

View File

@@ -12,13 +12,55 @@ import (
"github.com/netbirdio/netbird/shared/management/http/util"
)
// addSettingsEndpoints registers the Agent Network settings routes. The
// settings row is bootstrapped server-side on first provider create or on the
// first PUT carrying a cluster; GET reads it and PUT applies a partial update
// of the mutable collection toggles (cluster/subdomain stay immutable).
// addSettingsEndpoints registers the Agent Network settings routes. POST
// bootstraps the settings row, assigning the account's immutable endpoint;
// GET reads it (defaults with an empty endpoint before bootstrap); PUT
// carries every field, replacing the mutable collection toggles and rejecting
// any change to the identity fields; DELETE removes the row — guarded so it
// stays a bootstrap-repair operation — releasing the endpoint for a fresh
// bootstrap.
func (h *handler) addSettingsEndpoints(router *mux.Router) {
router.HandleFunc("/agent-network/settings", h.getSettings).Methods("GET", "OPTIONS")
router.HandleFunc("/agent-network/settings", h.createSettings).Methods("POST", "OPTIONS")
router.HandleFunc("/agent-network/settings", h.updateSettings).Methods("PUT", "OPTIONS")
router.HandleFunc("/agent-network/settings", h.deleteSettings).Methods("DELETE", "OPTIONS")
}
// createSettings bootstraps the account's settings row. Exactly one of
// proxy_address (labeled endpoint; the server allocates the label) and
// endpoint (self-addressed, claimed verbatim) must be provided; optional
// collection toggles ride along with defaults for omitted fields.
func (h *handler) createSettings(w http.ResponseWriter, r *http.Request) {
userAuth, err := nbcontext.GetUserAuthFromContext(r.Context())
if err != nil {
util.WriteError(r.Context(), err, w)
return
}
var req api.AgentNetworkSettingsCreateRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
util.WriteErrorResponse("couldn't parse JSON request", http.StatusBadRequest, w)
return
}
settings := types.DefaultSettings(userAuth.AccountId)
settings.FromAPICreateRequest(&req)
proxyAddress := ""
if req.ProxyAddress != nil {
proxyAddress = *req.ProxyAddress
}
endpoint := ""
if req.Endpoint != nil {
endpoint = *req.Endpoint
}
created, err := h.manager.CreateSettings(r.Context(), userAuth.UserId, settings, proxyAddress, endpoint)
if err != nil {
util.WriteError(r.Context(), err, w)
return
}
util.WriteJSONObject(r.Context(), w, created.ToAPIResponse())
}
// updateSettings replaces the mutable settings fields on the account's row.
@@ -48,6 +90,24 @@ func (h *handler) updateSettings(w http.ResponseWriter, r *http.Request) {
util.WriteJSONObject(r.Context(), w, updated.ToAPIResponse())
}
// deleteSettings removes the account's settings row, releasing the endpoint.
// The manager refuses (412) while providers exist or a proxy is actively
// serving the endpoint; a later POST bootstraps fresh, allocating a new
// endpoint.
func (h *handler) deleteSettings(w http.ResponseWriter, r *http.Request) {
userAuth, err := nbcontext.GetUserAuthFromContext(r.Context())
if err != nil {
util.WriteError(r.Context(), err, w)
return
}
if err := h.manager.DeleteSettings(r.Context(), userAuth.AccountId, userAuth.UserId); err != nil {
util.WriteError(r.Context(), err, w)
return
}
util.WriteJSONObject(r.Context(), w, util.EmptyObject{})
}
// getSettings returns the account's agent-network settings. Accounts that
// haven't been bootstrapped yet read as the defaults with an empty cluster,
// subdomain and endpoint; the manager synthesises that view.

View File

@@ -1,20 +1,25 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
rpproxy "github.com/netbirdio/netbird/management/internals/modules/reverseproxy/proxy"
"github.com/netbirdio/netbird/shared/management/http/api"
)
// TestSettingsHandler_GetUnbootstrappedReturnsDefaults pins the settings-read
// convention shared with the account and DNS settings endpoints: settings
// always read as a JSON object. Before bootstrap that object carries the
// defaults with an empty cluster/subdomain/endpoint (the "not bootstrapped"
// defaults with an empty endpoint/proxy_address (the "not bootstrapped"
// signal) and no timestamps — never a 404 and never the legacy null body.
func TestSettingsHandler_GetUnbootstrappedReturnsDefaults(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
@@ -27,9 +32,9 @@ func TestSettingsHandler_GetUnbootstrappedReturnsDefaults(t *testing.T) {
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Empty(t, got.Cluster, "cluster must be empty until bootstrapped")
assert.Empty(t, got.Subdomain, "subdomain must be empty until bootstrapped")
assert.Empty(t, got.Endpoint, "endpoint must be empty until bootstrapped, not a bare dot")
assert.Empty(t, got.Endpoint, "endpoint must be empty until bootstrapped")
assert.Empty(t, got.ProxyAddress, "proxy address must be empty until bootstrapped")
assert.False(t, got.Dedicated, "an unbootstrapped account has no serving shape")
assert.True(t, got.EnableLogCollection, "defaults must show log collection on, matching bootstrap")
assert.False(t, got.EnablePromptCollection, "defaults must show prompt collection off")
assert.False(t, got.RedactPii, "defaults must show redaction off")
@@ -39,62 +44,149 @@ func TestSettingsHandler_GetUnbootstrappedReturnsDefaults(t *testing.T) {
assert.Nil(t, got.UpdatedAt, "no timestamps before a row exists")
}
// TestSettingsHandler_PutBootstrapsWithCluster covers the settings-first
// bootstrap path: a PUT carrying a cluster on an unbootstrapped account
// creates the row (cluster pinned, subdomain assigned) and applies the
// mutable fields from the same request.
func TestSettingsHandler_PutBootstrapsWithCluster(t *testing.T) {
// TestSettingsHandler_PostBootstrapsLabeled covers the labeled bootstrap
// shape: a POST carrying a proxy_address allocates a label beneath it, so the
// endpoint hangs one label under the shared cluster's address and the pin is
// not dedicated. Toggles riding along apply; omitted ones keep defaults.
func TestSettingsHandler_PostBootstrapsLabeled(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPut, "/agent-network/settings",
`{"cluster": "eu.proxy.netbird.io", "enable_log_collection": true, "enable_prompt_collection": true, "redact_pii": false, "access_log_retention_days": 30}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap PUT must succeed: %s", rec.Body.String())
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "enable_prompt_collection": true, "access_log_retention_days": 14}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, "eu.proxy.netbird.io", got.Cluster, "cluster must be pinned from the request")
assert.NotEmpty(t, got.Subdomain, "subdomain must be assigned at bootstrap")
assert.Equal(t, got.Subdomain+".eu.proxy.netbird.io", got.Endpoint, "endpoint must combine subdomain and cluster")
assert.True(t, got.EnableLogCollection, "toggle from the bootstrap request must apply")
assert.Equal(t, "eu.proxy.netbird.io", got.ProxyAddress, "proxy address must be pinned from the request")
require.NotEmpty(t, got.Endpoint, "endpoint must be allocated at bootstrap")
assert.True(t, strings.HasSuffix(got.Endpoint, ".eu.proxy.netbird.io"),
"labeled endpoint must hang off the proxy address: %s", got.Endpoint)
label := strings.TrimSuffix(got.Endpoint, ".eu.proxy.netbird.io")
assert.NotContains(t, label, ".", "the allocated label must be a single DNS label: %s", label)
assert.False(t, got.Dedicated, "a labeled pin is not dedicated")
assert.True(t, got.EnableLogCollection, "omitted toggle must keep its default")
assert.True(t, got.EnablePromptCollection, "toggle from the bootstrap request must apply")
require.NotNil(t, got.AccessLogRetentionDays)
assert.Equal(t, 30, *got.AccessLogRetentionDays, "retention from the bootstrap request must apply")
assert.Equal(t, 14, *got.AccessLogRetentionDays, "retention from the bootstrap request must apply")
assert.NotNil(t, got.CreatedAt, "a persisted row carries timestamps")
// The row is now readable via GET.
rec = f.do(t, http.MethodGet, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code, "GET after bootstrap must succeed")
var read api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &read))
assert.Equal(t, got.Endpoint, read.Endpoint, "GET must return the bootstrapped endpoint")
}
// TestSettingsHandler_PutWithoutClusterOnUnbootstrapped pins that a PUT
// without a cluster cannot conjure a settings row out of nothing — there is
// no cluster to pin — and surfaces as 404 like the GET.
func TestSettingsHandler_PutWithoutClusterOnUnbootstrapped(t *testing.T) {
// TestSettingsHandler_PostBootstrapsSelfAddressed covers the dedicated shape:
// a POST carrying an endpoint claims the hostname verbatim, the proxy address
// equals it, and the pin reads as dedicated. The claim is legitimate before
// any proxy declares the address (address-first).
func TestSettingsHandler_PostBootstrapsSelfAddressed(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"endpoint": "Brave-Otter.Gateway.Example.com"}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, "brave-otter.gateway.example.com", got.Endpoint,
"endpoint must be claimed verbatim, lowercased")
assert.Equal(t, got.Endpoint, got.ProxyAddress, "self-addressed: the proxy address is the endpoint")
assert.True(t, got.Dedicated, "a self-addressed pin is dedicated")
assert.True(t, got.EnableLogCollection, "omitted toggles must keep their defaults")
}
// TestSettingsHandler_PostRequiresExactlyOneIdentityField pins the request
// contract: proxy_address and endpoint are mutually exclusive and one is
// required — both or neither is a validation error, not a guess.
func TestSettingsHandler_PostRequiresExactlyOneIdentityField(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings", `{}`)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code,
"empty POST must be rejected: got %d body=%s", rec.Code, rec.Body.String())
rec = f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "endpoint": "brave-otter.gateway.example.com"}`)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code,
"POST with both identity fields must be rejected: got %d body=%s", rec.Code, rec.Body.String())
}
// TestSettingsHandler_PostRejectsMalformedHostnames pins per-write input
// validation: shapes canonicalization cannot repair — trailing dots, embedded
// whitespace, empty labels — are rejected with a validation error instead of
// landing in an immutable column.
func TestSettingsHandler_PostRejectsMalformedHostnames(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
for name, body := range map[string]string{
"trailing dot": `{"endpoint": "gateway.example.com."}`,
"leading dot": `{"endpoint": ".gateway.example.com"}`,
"inner whitespace": `{"endpoint": "gate way.example.com"}`,
"empty label": `{"proxy_address": "eu..proxy.netbird.io"}`,
} {
rec := f.do(t, http.MethodPost, "/agent-network/settings", body)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code,
"%s must be rejected: got %d body=%s", name, rec.Code, rec.Body.String())
}
}
// TestSettingsHandler_PostConflictsOnSecondBootstrap pins that bootstrap is a
// one-time create: a second POST returns 409 and leaves the row untouched.
func TestSettingsHandler_PostConflictsOnSecondBootstrap(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings", `{"proxy_address": "eu.proxy.netbird.io"}`)
require.Equal(t, http.StatusOK, rec.Code, "first bootstrap must succeed: %s", rec.Body.String())
var first api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &first))
rec = f.do(t, http.MethodPost, "/agent-network/settings", `{"proxy_address": "us.proxy.netbird.io"}`)
assert.Equal(t, http.StatusConflict, rec.Code,
"second bootstrap must 409: got %d body=%s", rec.Code, rec.Body.String())
rec = f.do(t, http.MethodGet, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code)
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, first.Endpoint, got.Endpoint, "the original endpoint must survive the rejected bootstrap")
assert.Equal(t, first.ProxyAddress, got.ProxyAddress, "the original proxy address must survive")
}
// TestSettingsHandler_PutBeforeBootstrapIs404 pins that a PUT cannot conjure a
// settings row out of nothing — bootstrap is the explicit POST — and the
// error points the caller there.
func TestSettingsHandler_PutBeforeBootstrapIs404(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPut, "/agent-network/settings",
`{"enable_log_collection": false, "enable_prompt_collection": false, "redact_pii": false}`)
assert.Equal(t, http.StatusNotFound, rec.Code,
"cluster-less PUT on an unbootstrapped account must 404: got %d body=%s", rec.Code, rec.Body.String())
assert.Contains(t, rec.Body.String(), "cluster",
"the error must point the caller at the bootstrap paths: %s", rec.Body.String())
"PUT on an unbootstrapped account must 404: got %d body=%s", rec.Code, rec.Body.String())
assert.Contains(t, rec.Body.String(), "/api/agent-network/settings",
"the error must point the caller at the bootstrap POST: %s", rec.Body.String())
}
// TestSettingsHandler_PutReplacesMutableFields pins the update contract shared
// with the other PUT endpoints: the request replaces every mutable field, so a
// toggle absent from the JSON lands as its zero value rather than being
// preserved. Cluster and subdomain survive untouched.
// with the other PUT endpoints: the request carries every field, replacing the
// mutable ones. The identity fields ride along as a required echo of the
// assigned values — compared, never written — so the endpoint and proxy
// address survive every accepted update.
func TestSettingsHandler_PutReplacesMutableFields(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPut, "/agent-network/settings",
`{"cluster": "eu.proxy.netbird.io", "enable_log_collection": true, "enable_prompt_collection": true, "redact_pii": true, "access_log_retention_days": 14}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap PUT must succeed: %s", rec.Body.String())
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "enable_prompt_collection": true, "redact_pii": true, "access_log_retention_days": 14}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var before api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &before))
rec = f.do(t, http.MethodPut, "/agent-network/settings",
`{"enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": false}`)
rec = f.do(t, http.MethodPut, "/agent-network/settings", fmt.Sprintf(
`{"endpoint": %q, "proxy_address": %q, "enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": false, "access_log_retention_days": 7}`,
before.Endpoint, before.ProxyAddress))
require.Equal(t, http.StatusOK, rec.Code, "update PUT must succeed: %s", rec.Body.String())
var got api.AgentNetworkSettings
@@ -103,35 +195,201 @@ func TestSettingsHandler_PutReplacesMutableFields(t *testing.T) {
assert.False(t, got.EnablePromptCollection, "sent toggle must apply")
assert.False(t, got.RedactPii, "sent toggle must apply")
require.NotNil(t, got.AccessLogRetentionDays)
assert.Equal(t, 0, *got.AccessLogRetentionDays,
"retention absent from the request must land as the zero value — PUT replaces all mutable fields")
assert.Equal(t, before.Cluster, got.Cluster, "cluster must survive updates untouched")
assert.Equal(t, before.Subdomain, got.Subdomain, "subdomain must survive updates untouched")
assert.Equal(t, 7, *got.AccessLogRetentionDays, "sent retention must apply")
assert.Equal(t, before.Endpoint, got.Endpoint, "endpoint must survive updates untouched")
assert.Equal(t, before.ProxyAddress, got.ProxyAddress, "proxy address must survive updates untouched")
}
// TestSettingsHandler_PutRejectsClusterChange pins cluster immutability: once
// assigned, a differing cluster is rejected as a validation error instead of
// being silently ignored, so callers never observe a value other than the one
// they sent. Echoing the assigned cluster back stays valid, which lets
// declarative clients send their full desired state idempotently.
func TestSettingsHandler_PutRejectsClusterChange(t *testing.T) {
// TestSettingsHandler_PutRejectsChangedIdentity pins the immutability contract:
// the PUT carries the identity fields like every other field, but they are an
// echo — a request carrying a different endpoint or proxy address is rejected
// as a validation error and the row is left untouched. The comparison is
// lenient about casing (the stored values are normalized lowercase), so a
// client replaying a GET response with different casing is not rejected.
func TestSettingsHandler_PutRejectsChangedIdentity(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPut, "/agent-network/settings",
`{"cluster": "eu.proxy.netbird.io", "enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": false}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap PUT must succeed: %s", rec.Body.String())
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "enable_prompt_collection": true}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var before api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &before))
rec = f.do(t, http.MethodPut, "/agent-network/settings",
`{"cluster": "us.proxy.netbird.io", "enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": false}`)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code,
"cluster change must be rejected as a validation error: got %d body=%s", rec.Code, rec.Body.String())
for name, body := range map[string]string{
"changed endpoint": fmt.Sprintf(
`{"endpoint": "other.gateway.example.com", "proxy_address": %q, "enable_log_collection": false, "enable_prompt_collection": false, "redact_pii": false, "access_log_retention_days": 30}`,
before.ProxyAddress),
"changed proxy_address": fmt.Sprintf(
`{"endpoint": %q, "proxy_address": "us.proxy.netbird.io", "enable_log_collection": false, "enable_prompt_collection": false, "redact_pii": false, "access_log_retention_days": 30}`,
before.Endpoint),
"omitted identity": `{"enable_log_collection": false, "enable_prompt_collection": false, "redact_pii": false, "access_log_retention_days": 30}`,
} {
rec = f.do(t, http.MethodPut, "/agent-network/settings", body)
assert.Equal(t, http.StatusUnprocessableEntity, rec.Code,
"%s must be rejected: got %d body=%s", name, rec.Code, rec.Body.String())
}
rec = f.do(t, http.MethodPut, "/agent-network/settings",
`{"cluster": "eu.proxy.netbird.io", "enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": true}`)
require.Equal(t, http.StatusOK, rec.Code, "echoing the assigned cluster must stay valid: %s", rec.Body.String())
// The rejected updates must not have applied anything — toggles included.
rec = f.do(t, http.MethodGet, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code)
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, before.Endpoint, got.Endpoint, "rejected PUT must not change the endpoint")
assert.Equal(t, before.ProxyAddress, got.ProxyAddress, "rejected PUT must not change the proxy address")
assert.True(t, got.EnablePromptCollection, "rejected PUT must not apply its toggles")
// An uppercased echo of the assigned values still names the same host and
// must be accepted.
rec = f.do(t, http.MethodPut, "/agent-network/settings", fmt.Sprintf(
`{"endpoint": %q, "proxy_address": %q, "enable_log_collection": true, "enable_prompt_collection": true, "redact_pii": false, "access_log_retention_days": 30}`,
strings.ToUpper(before.Endpoint), strings.ToUpper(before.ProxyAddress)))
assert.Equal(t, http.StatusOK, rec.Code,
"an uppercased identity echo must be accepted: got %d body=%s", rec.Code, rec.Body.String())
}
// TestSettingsHandler_PutOmittedRetentionLandsAsZero documents a residual the
// required-ness of access_log_retention_days does not remove. Marking the field
// required changes the generated client type from *int to int, so a generated
// client cannot omit it — but nothing validates OpenAPI required-ness at
// runtime, so a hand-rolled body without the field still decodes as 0, which
// the API documents as "keep indefinitely".
//
// That is the same latitude the three booleans already have, so it is left
// consistent rather than special-cased. This test exists to make the gap
// explicit: if request validation is ever added, this expectation is what
// changes.
func TestSettingsHandler_PutOmittedRetentionLandsAsZero(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "access_log_retention_days": 14}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var before api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &before))
rec = f.do(t, http.MethodPut, "/agent-network/settings", fmt.Sprintf(
`{"endpoint": %q, "proxy_address": %q, "enable_log_collection": true, "enable_prompt_collection": false, "redact_pii": false}`,
before.Endpoint, before.ProxyAddress))
require.Equal(t, http.StatusOK, rec.Code, "update PUT must succeed: %s", rec.Body.String())
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, "eu.proxy.netbird.io", got.Cluster, "cluster must be unchanged")
assert.True(t, got.RedactPii, "toggle sent alongside the echoed cluster must apply")
require.NotNil(t, got.AccessLogRetentionDays)
assert.Equal(t, 0, *got.AccessLogRetentionDays,
"a non-conforming body that omits retention still replaces it with the zero value")
}
// TestSettingsHandler_DeleteBeforeBootstrapIs404 pins that DELETE on an
// account with no settings row is a 404, mirroring the PUT.
func TestSettingsHandler_DeleteBeforeBootstrapIs404(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodDelete, "/agent-network/settings", "")
assert.Equal(t, http.StatusNotFound, rec.Code,
"DELETE on an unbootstrapped account must 404: got %d body=%s", rec.Code, rec.Body.String())
}
// TestSettingsHandler_DeleteBlockedByProviders pins the first delete guard:
// while any provider exists for the account, the delete is refused with 412
// and the row survives. Providers route through the endpoint — the guard
// keeps DELETE a bootstrap-repair operation rather than a way to abandon a
// configured gateway.
func TestSettingsHandler_DeleteBlockedByProviders(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings", `{"proxy_address": "eu.proxy.netbird.io"}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
var before api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &before))
f.seedProvider(t, "prov-guard")
rec = f.do(t, http.MethodDelete, "/agent-network/settings", "")
assert.Equal(t, http.StatusPreconditionFailed, rec.Code,
"delete with a provider present must be refused: got %d body=%s", rec.Code, rec.Body.String())
rec = f.do(t, http.MethodGet, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code)
var got api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got))
assert.Equal(t, before.Endpoint, got.Endpoint, "the refused delete must leave the row intact")
}
// TestSettingsHandler_DeleteBlockedByActiveProxy pins the second delete
// guard: while a proxy is actively serving the endpoint — an active proxy
// row declaring the endpoint hostname as its cluster address, the dedicated
// shape — the delete is refused with 412. A proxy that has disconnected no
// longer blocks: the guard is about a live serving path, not history.
//
// The proxy declares its address with mixed casing on purpose: Connect
// stores the declared address verbatim while the settings row is normalized
// lowercase, and hostnames are case-insensitive, so the guard must match
// across the casing difference rather than be sidestepped by it.
func TestSettingsHandler_DeleteBlockedByActiveProxy(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
const endpoint = "gw.dedicated.example.com"
rec := f.do(t, http.MethodPost, "/agent-network/settings", fmt.Sprintf(`{"endpoint": %q}`, endpoint))
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
now := time.Now()
accountID := testAccountID
proxyRow := &rpproxy.Proxy{
ID: "proxy-guard",
SessionID: "sess-1",
ClusterAddress: "GW.Dedicated.Example.Com",
AccountID: &accountID,
LastSeen: now,
ConnectedAt: &now,
Status: rpproxy.StatusConnected,
}
require.NoError(t, f.store.SaveProxy(context.Background(), proxyRow))
rec = f.do(t, http.MethodDelete, "/agent-network/settings", "")
assert.Equal(t, http.StatusPreconditionFailed, rec.Code,
"delete with an active proxy at the endpoint must be refused: got %d body=%s", rec.Code, rec.Body.String())
// Once the proxy disconnects it no longer serves the endpoint, so the
// delete goes through.
require.NoError(t, f.store.DisconnectProxy(context.Background(), proxyRow.ID, proxyRow.SessionID))
rec = f.do(t, http.MethodDelete, "/agent-network/settings", "")
assert.Equal(t, http.StatusOK, rec.Code,
"delete after the proxy disconnected must succeed: got %d body=%s", rec.Code, rec.Body.String())
}
// TestSettingsHandler_DeleteReleasesEndpointForFreshBootstrap pins the
// full-reset semantic that gives replace-on-change clients (e.g. Terraform's
// RequiresReplace) a real path: with both guards clear the delete succeeds,
// the account reads as the defaults again, and a fresh bootstrap draws a
// fresh label. The released hostname is not reserved — a fresh draw may even
// legitimately re-pick it — so the assertions check the new row's shape, not
// that the label differs.
func TestSettingsHandler_DeleteReleasesEndpointForFreshBootstrap(t *testing.T) {
f := newAgentNetworkHandlerFixture(t)
rec := f.do(t, http.MethodPost, "/agent-network/settings",
`{"proxy_address": "eu.proxy.netbird.io", "enable_prompt_collection": true}`)
require.Equal(t, http.StatusOK, rec.Code, "bootstrap POST must succeed: %s", rec.Body.String())
rec = f.do(t, http.MethodDelete, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code,
"delete with both guards clear must succeed: got %d body=%s", rec.Code, rec.Body.String())
rec = f.do(t, http.MethodGet, "/agent-network/settings", "")
require.Equal(t, http.StatusOK, rec.Code)
var after api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &after))
assert.Empty(t, after.Endpoint, "a deleted account must read as unbootstrapped defaults")
assert.False(t, after.EnablePromptCollection, "the deleted row's toggles must not linger")
rec = f.do(t, http.MethodPost, "/agent-network/settings", `{"proxy_address": "eu.proxy.netbird.io"}`)
require.Equal(t, http.StatusOK, rec.Code, "re-bootstrap after delete must succeed: %s", rec.Body.String())
var second api.AgentNetworkSettings
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &second))
require.NotEmpty(t, second.Endpoint, "the fresh bootstrap must allocate an endpoint")
assert.True(t, strings.HasSuffix(second.Endpoint, ".eu.proxy.netbird.io"),
"the fresh endpoint must hang beneath the requested proxy address: %s", second.Endpoint)
assert.False(t, second.EnablePromptCollection,
"the fresh row must carry bootstrap defaults, not the deleted row's toggles")
assert.NotNil(t, second.CreatedAt, "the fresh row is persisted and carries timestamps")
}

View File

@@ -0,0 +1,37 @@
package labelgen
// adjectives is the descriptor half of a generated label. It pairs with the
// noun pool in words.go to form `<adjective>-<noun>` labels, and is kept
// separate because words.go is almost entirely nouns — drawing both halves
// from it produced unreadable pairs like "millet-hammock". Entries are
// lowercase ASCII, 4-12 chars, free of hyphens and digits, screened for
// offensive/brand/region-specific terms, and disjoint from the noun pool
// (enforced by TestAdjectives_AreDisjointFromNouns).
var adjectives = []string{
"able", "active", "adept", "agile", "airy", "alert", "amiable", "ample",
"ancient", "ardent", "artful", "astute", "balmy", "blithe", "bold", "bonny",
"brave", "breezy", "brisk", "bubbly", "buoyant", "bushy", "candid", "canny",
"cheery", "chilly", "chipper", "chunky", "civil", "classic", "clever", "comely",
"compact", "cordial", "cosmic", "courtly", "crafty", "creamy", "crisp", "cuddly",
"curious", "dainty", "dapper", "daring", "dashing", "deft", "dewy", "diligent",
"downy", "dreamy", "dulcet", "durable", "dusky", "eager", "earnest", "earthy",
"easy", "elated", "elegant", "epic", "fabled", "faithful", "fancy", "fearless",
"feisty", "fervent", "fleet", "fluffy", "fond", "frisky", "frosty", "gallant",
"genial", "genteel", "gentle", "giddy", "gilded", "glad", "glassy", "gleaming",
"glossy", "graceful", "grand", "grainy", "hale", "hardy", "hearty", "hefty",
"honest", "hopeful", "humble", "hushed", "immense", "jaunty", "jolly", "jovial",
"joyful", "jubilant", "keen", "kindly", "kindred", "lanky", "leafy", "limber",
"lively", "lofty", "loyal", "lucent", "lucid", "luminous", "lush", "maroon",
"mellow", "merry", "mighty", "mindful", "mirthful", "misty", "modest", "muted",
"nifty", "nimble", "noble", "patient", "peaceful", "pearly", "peppy", "perky",
"petite", "placid", "playful", "pleasant", "plucky", "plush", "polite", "posh",
"prancing", "pristine", "prompt", "proud", "prudent", "quaint", "quick", "quirky",
"radiant", "ready", "regal", "restful", "robust", "rosy", "ruddy", "rugged",
"sandy", "satin", "saucy", "savvy", "sedate", "serene", "shady", "shiny",
"silken", "silky", "sincere", "sleek", "slender", "smart", "smooth", "snappy",
"snug", "soaring", "sparkly", "spiffy", "spirited", "sprightly", "spry", "stalwart",
"stately", "steady", "sterling", "stoic", "stormy", "stout", "sturdy", "sunlit",
"supple", "svelte", "tawny", "tender", "tidy", "timeless", "trusty", "upbeat",
"urbane", "valiant", "vast", "vernal", "vibrant", "vintage", "whimsy", "willing",
"windy", "winsome", "wintry", "witty", "worthy", "zesty", "zippy",
}

View File

@@ -64,3 +64,20 @@ func PickUnique(rng *rand.Rand, taken map[string]struct{}, fallbackSuffix string
w := pool[rng.Intn(len(pool))]
return fmt.Sprintf("%s-%s", w, fallbackSuffix)
}
// PickTuple returns an adjective-noun label such as "brave-otter". It is still
// a single DNS label.
//
// Unlike PickUnique it takes no `taken` set and has no fallback suffix. The
// noun pool holds 857 entries, which is ample per cluster but a hard ceiling
// once labels must be unique across one shared zone; pairing an adjective with
// a noun spans len(adjectives) * 857 instead. Uniqueness is enforced by a
// database constraint and retried by the caller, rather than guessed from a
// pre-read set that a concurrent allocation can invalidate.
func PickTuple(rng *rand.Rand) string {
nouns := uniqueWords()
if len(nouns) == 0 || len(adjectives) == 0 {
return ""
}
return adjectives[rng.Intn(len(adjectives))] + "-" + nouns[rng.Intn(len(nouns))]
}

View File

@@ -99,3 +99,82 @@ func TestUniqueWords_DropsDuplicates(t *testing.T) {
}
assert.GreaterOrEqual(t, len(pool), 500, "Pool must contain at least 500 unique words")
}
// TestPickTuple_ShapeAndPoolMembership locks the wire-visible shape: an
// adjective and a noun, each from its own pool, joined by a single hyphen so
// the result stays one DNS label.
func TestPickTuple_ShapeAndPoolMembership(t *testing.T) {
nouns := uniqueWords()
inNouns := make(map[string]struct{}, len(nouns))
for _, w := range nouns {
inNouns[w] = struct{}{}
}
inAdjectives := make(map[string]struct{}, len(adjectives))
for _, a := range adjectives {
inAdjectives[a] = struct{}{}
}
rng := rand.New(rand.NewSource(7))
for i := 0; i < 200; i++ {
got := PickTuple(rng)
parts := strings.Split(got, "-")
require.Len(t, parts, 2, "PickTuple must produce exactly two hyphen-joined words; got %q", got)
_, adjOK := inAdjectives[parts[0]]
assert.True(t, adjOK, "First half must be an adjective; %q not in adjectives (from %q)", parts[0], got)
_, nounOK := inNouns[parts[1]]
assert.True(t, nounOK, "Second half must be a noun; %q not in words (from %q)", parts[1], got)
assert.LessOrEqual(t, len(got), 63, "Label must fit a DNS label; got %q (%d chars)", got, len(got))
}
}
// TestAdjectives_AreDisjointFromNouns keeps the namespace a clean product and
// prevents nonsense like "azure-azure": a handful of the noun pool's entries
// are adjectival, and any overlap would let the same word land on both sides.
func TestAdjectives_AreDisjointFromNouns(t *testing.T) {
nouns := make(map[string]struct{}, len(uniqueWords()))
for _, w := range uniqueWords() {
nouns[w] = struct{}{}
}
for _, a := range adjectives {
_, clash := nouns[a]
assert.False(t, clash, "Adjective %q also appears in the noun pool; remove it from one list", a)
}
}
// TestAdjectives_AreDNSSafeAndDeduplicated mirrors the curation contract stated
// in words.go: lowercase ASCII, 4-12 chars, no digits or hyphens, no repeats.
func TestAdjectives_AreDNSSafeAndDeduplicated(t *testing.T) {
seen := make(map[string]struct{}, len(adjectives))
for _, a := range adjectives {
_, dup := seen[a]
assert.False(t, dup, "Duplicate adjective %q", a)
seen[a] = struct{}{}
assert.Regexp(t, `^[a-z]{4,12}$`, a, "Adjective %q must be 4-12 lowercase ASCII letters", a)
}
assert.Greater(t, len(adjectives), 150, "Adjective pool too small to give a useful namespace")
}
// TestPickTuple_DeterministicWithSeededRng documents that generation is a pure
// function of the rng, which is what makes allocation retries reproducible in tests.
func TestPickTuple_DeterministicWithSeededRng(t *testing.T) {
a := PickTuple(rand.New(rand.NewSource(42)))
b := PickTuple(rand.New(rand.NewSource(42)))
assert.Equal(t, a, b, "Same seed must yield the same tuple")
}
// TestPickTuple_SpansALargeNamespace guards the reason we moved to tuples: a
// single-word pool caps the GLOBAL namespace at 857. Drawing many tuples must
// yield overwhelmingly distinct values.
func TestPickTuple_SpansALargeNamespace(t *testing.T) {
rng := rand.New(rand.NewSource(11))
seen := make(map[string]struct{}, 2000)
for i := 0; i < 2000; i++ {
seen[PickTuple(rng)] = struct{}{}
}
assert.Greater(t, len(seen), 1900,
"2000 draws should be nearly all distinct across a ~200k namespace; got %d unique", len(seen))
}

View File

@@ -22,7 +22,6 @@ import (
"github.com/netbirdio/netbird/management/server/permissions/modules"
"github.com/netbirdio/netbird/management/server/permissions/operations"
"github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/shared/management/proto"
"github.com/netbirdio/netbird/shared/management/status"
)
@@ -48,7 +47,7 @@ func ensureSessionKeys(p *types.Provider) error {
type Manager interface {
GetAllProviders(ctx context.Context, accountID, userID string) ([]*types.Provider, error)
GetProvider(ctx context.Context, accountID, userID, providerID string) (*types.Provider, error)
CreateProvider(ctx context.Context, userID string, provider *types.Provider, bootstrapCluster string) (*types.Provider, error)
CreateProvider(ctx context.Context, userID string, provider *types.Provider) (*types.Provider, error)
UpdateProvider(ctx context.Context, userID string, provider *types.Provider) (*types.Provider, error)
DeleteProvider(ctx context.Context, accountID, userID, providerID string) error
@@ -71,7 +70,9 @@ type Manager interface {
DeleteBudgetRule(ctx context.Context, accountID, userID, ruleID string) error
GetSettings(ctx context.Context, accountID, userID string) (*types.Settings, error)
CreateSettings(ctx context.Context, userID string, settings *types.Settings, proxyAddress, endpoint string) (*types.Settings, error)
UpdateSettings(ctx context.Context, userID string, settings *types.Settings) (*types.Settings, error)
DeleteSettings(ctx context.Context, accountID, userID string) error
ListConsumption(ctx context.Context, accountID, userID string) ([]*types.Consumption, error)
ListAccessLogs(ctx context.Context, accountID, userID string, filter types.AgentNetworkAccessLogFilter) ([]*types.AgentNetworkAccessLog, int64, error)
@@ -123,11 +124,10 @@ type managerImpl struct {
proxyController proxy.Controller
// reconcileCache holds the last set of synthesised proxy mappings
// per account so reconcile can emit precise Create/Update/Delete
// updates instead of a full re-push on every mutation. Keyed by
// accountID, then by synthesised service ID.
// per account, each paired with the proxy that served it, so a change
// of serving proxy can be diffed without re-deriving it.
reconcileMu sync.Mutex
reconcileCache map[string]map[string]*proto.ProxyMapping
reconcileCache map[string]map[string]syntheticMapping
// labelRngMu guards labelRng. PickUnique consumes math/rand.Source
// state; concurrent provider creates would otherwise race.
@@ -151,7 +151,7 @@ func NewManager(
accountManager: accountManager,
permissionsManager: permissionsManager,
proxyController: proxyController,
reconcileCache: make(map[string]map[string]*proto.ProxyMapping),
reconcileCache: make(map[string]map[string]syntheticMapping),
labelRng: rand.New(rand.NewSource(time.Now().UnixNano())),
}
}
@@ -170,19 +170,14 @@ func (m *managerImpl) GetProvider(ctx context.Context, accountID, userID, provid
return m.store.GetAgentNetworkProviderByID(ctx, store.LockingStrengthNone, accountID, providerID)
}
// CreateProvider persists a new provider for the account. bootstrapCluster
// is used only when the per-account agent-network Settings row hasn't
// been created yet; otherwise it is ignored (the cluster is pinned on
// Settings and every provider in the account routes through it).
func (m *managerImpl) CreateProvider(ctx context.Context, userID string, provider *types.Provider, bootstrapCluster string) (*types.Provider, error) {
// CreateProvider persists a new provider for the account. Providers have no
// settings side effects: the account's endpoint is bootstrapped separately and
// explicitly via CreateSettings, and every provider in the account routes
// through it.
func (m *managerImpl) CreateProvider(ctx context.Context, userID string, provider *types.Provider) (*types.Provider, error) {
if err := m.requirePermission(ctx, provider.AccountID, userID, modules.AgentNetworkProviders, operations.Create); err != nil {
return nil, err
}
if strings.TrimSpace(bootstrapCluster) != "" {
if err := m.requireSettingsBootstrapPermission(ctx, provider.AccountID, userID); err != nil {
return nil, err
}
}
// An empty api_key would silently produce a synthesised service
// that 401s on every upstream request. Surface the misconfiguration
@@ -206,16 +201,6 @@ func (m *managerImpl) CreateProvider(ctx context.Context, userID string, provide
return nil, fmt.Errorf("save agent network provider: %w", err)
}
if strings.TrimSpace(bootstrapCluster) != "" {
if _, err := m.bootstrapSettingsIfNeeded(ctx, m.store, provider.AccountID, bootstrapCluster); err != nil {
// The provider create has already succeeded; logging the
// bootstrap miss matches the plan's PoC behaviour. The synth
// path treats a missing settings row as a no-op, and the next
// provider create retries the bootstrap.
log.WithContext(ctx).Debugf("agent-network bootstrap settings for account %s on cluster %s: %v", provider.AccountID, bootstrapCluster, err)
}
}
m.accountManager.StoreEvent(ctx, userID, provider.ID, provider.AccountID, activity.AgentNetworkProviderCreated, provider.EventMeta())
m.reconcile(ctx, provider.AccountID)
@@ -560,52 +545,44 @@ func (m *managerImpl) DeleteBudgetRule(ctx context.Context, accountID, userID, r
}
// UpdateSettings replaces the mutable account-level settings — the collection
// toggles and retention — on the account's row. When the account has no
// settings row yet, a non-empty settings.Cluster bootstraps one (same path as
// first provider create); without it the update fails with NotFound. On an
// existing row the cluster and subdomain are immutable: a differing
// settings.Cluster is rejected rather than silently ignored so callers never
// observe a value other than what they sent. Because the collection toggles
// change the synthesised service config (prompt-capture gating, access-log
// emission), a reconcile is triggered so the proxy and peer network maps
// converge on the new state.
// toggles and retention — on the account's row. The identity fields (Domain,
// ProxyAddress) are assigned at bootstrap (CreateSettings) and immutable: the
// request carries them, matching the PUT convention of every other endpoint,
// but they are only compared against the stored row — a request carrying
// different values is rejected, and the stored values are never overwritten.
// When the account has no settings row yet the update fails with NotFound.
// Because the collection toggles change the synthesised service config
// (prompt-capture gating, access-log emission), a reconcile is triggered so
// the proxy and peer network maps converge on the new state.
func (m *managerImpl) UpdateSettings(ctx context.Context, userID string, settings *types.Settings) (*types.Settings, error) {
if err := m.requirePermission(ctx, settings.AccountID, userID, modules.AgentNetworkSettings, operations.Update); err != nil {
return nil, err
}
requestedCluster := strings.TrimSpace(settings.Cluster)
// The row lock from LockingStrengthUpdate only holds for the duration of
// the surrounding transaction, so the read, the cluster-immutability
// check, and the save must share one — otherwise concurrent PUTs could
// interleave between them.
// the surrounding transaction, so the read and the save must share one —
// otherwise concurrent PUTs could interleave between them.
var updated *types.Settings
err := m.store.ExecuteInTransaction(ctx, func(tx store.Store) error {
existing, err := tx.GetAgentNetworkSettings(ctx, store.LockingStrengthUpdate, settings.AccountID)
switch {
case err == nil:
if requestedCluster != "" && requestedCluster != existing.Cluster {
return status.Errorf(status.InvalidArgument, "cluster is immutable once assigned (current: %s)", existing.Cluster)
}
case isNotFound(err):
if requestedCluster == "" {
return status.Errorf(status.NotFound, "agent network settings have not been bootstrapped yet; pass cluster to bootstrap them, or create a provider with bootstrap_cluster set")
}
// Bootstrapping pins the cluster and subdomain — a settings
// create on top of the update the caller already passed, matching
// the gate on the provider-create bootstrap path.
if err := m.requirePermission(ctx, settings.AccountID, userID, modules.AgentNetworkSettings, operations.Create); err != nil {
return err
}
existing, err = m.bootstrapSettingsIfNeeded(ctx, tx, settings.AccountID, requestedCluster)
if err != nil {
return err
}
return status.Errorf(status.NotFound, "agent network settings have not been bootstrapped yet; POST /api/agent-network/settings to bootstrap them")
default:
return fmt.Errorf("get agent network settings: %w", err)
}
// The identity echo is compared leniently (trimmed, case-insensitive):
// the stored values are normalized lowercase, and a client replaying a
// GET response must never be rejected over casing it didn't choose.
if !hostnamesEquivalent(settings.Domain, existing.Domain) {
return status.Errorf(status.InvalidArgument, "endpoint is immutable: it must match the assigned endpoint %q; delete the settings to release it and bootstrap again", existing.Domain)
}
if !hostnamesEquivalent(settings.ProxyAddress, existing.ProxyAddress) {
return status.Errorf(status.InvalidArgument, "proxy_address is immutable: it must match the assigned proxy address %q; delete the settings to release it and bootstrap again", existing.ProxyAddress)
}
existing.EnableLogCollection = settings.EnableLogCollection
existing.EnablePromptCollection = settings.EnablePromptCollection
existing.RedactPii = settings.RedactPii
@@ -632,6 +609,83 @@ func (m *managerImpl) UpdateSettings(ctx context.Context, userID string, setting
return updated, nil
}
// hostnamesEquivalent reports whether a caller-supplied hostname names the
// same host as a stored (normalized, lowercase) one: equal after trimming and
// case folding. No structural validation — an arbitrary mismatch and a
// malformed value are both simply "not the assigned value".
func hostnamesEquivalent(supplied, stored string) bool {
return strings.EqualFold(strings.TrimSpace(supplied), stored)
}
// DeleteSettings removes the account's settings row, releasing the endpoint.
// Two guards make this a bootstrap-repair operation rather than a way to tear
// down a serving gateway, both re-checked under the row lock:
//
// - No Agent Network providers may exist for the account. Providers route
// through the endpoint; delete them first.
// - No proxy may be actively serving the endpoint — that is, no active proxy
// declares the endpoint hostname as its cluster address. This is the
// dedicated (self-addressed) shape's guard: the proxy at the address IS
// this account's gateway. A labeled endpoint hangs beneath a shared
// cluster's address, and with the account's providers already gone the
// shared proxy serves nothing of the account's, so the parent cluster
// being up does not block the delete.
//
// Bootstrapping again after a delete allocates fresh — the released hostname
// is not reserved. That full-reset semantic is what gives clients that model
// immutability as replace-on-change (e.g. Terraform's RequiresReplace) a real
// path: tear down providers, delete, re-create.
func (m *managerImpl) DeleteSettings(ctx context.Context, accountID, userID string) error {
if err := m.requirePermission(ctx, accountID, userID, modules.AgentNetworkSettings, operations.Delete); err != nil {
return err
}
var deleted *types.Settings
err := m.store.ExecuteInTransaction(ctx, func(tx store.Store) error {
existing, err := tx.GetAgentNetworkSettings(ctx, store.LockingStrengthUpdate, accountID)
switch {
case err == nil:
case isNotFound(err):
return status.Errorf(status.NotFound, "agent network settings have not been bootstrapped yet; there is nothing to delete")
default:
return fmt.Errorf("get agent network settings: %w", err)
}
providers, err := tx.GetAccountAgentNetworkProviders(ctx, store.LockingStrengthNone, accountID)
if err != nil {
return fmt.Errorf("get agent network providers: %w", err)
}
if len(providers) > 0 {
return status.Errorf(status.PreconditionFailed, "agent network settings cannot be deleted while %d provider(s) exist; delete the providers first", len(providers))
}
serving, err := tx.HasActiveProxyAtClusterAddress(ctx, existing.Domain)
if err != nil {
return fmt.Errorf("check for a proxy serving the endpoint: %w", err)
}
if serving {
return status.Errorf(status.PreconditionFailed, "agent network settings cannot be deleted while a proxy is actively serving the endpoint %q", existing.Domain)
}
if err := tx.DeleteAgentNetworkSettings(ctx, accountID); err != nil {
return fmt.Errorf("delete agent network settings: %w", err)
}
deleted = existing
return nil
})
if err != nil {
return err
}
m.accountManager.StoreEvent(ctx, userID, accountID, accountID, activity.AgentNetworkSettingsDeleted, map[string]any{
"endpoint": deleted.Domain,
"proxy_address": deleted.ProxyAddress,
})
m.reconcile(ctx, accountID)
return nil
}
// isNotFound reports whether err is a status.NotFound error.
func isNotFound(err error) bool {
var sErr *status.Error
@@ -678,74 +732,162 @@ func (m *managerImpl) GetSettings(ctx context.Context, accountID, userID string)
}
}
// requireSettingsBootstrapPermission gates the one-time settings bootstrap a
// first provider create performs. Pinning the account's cluster and subdomain
// is a settings write, so it needs the settings permission on top of the
// provider one. No-op once the settings row exists.
func (m *managerImpl) requireSettingsBootstrapPermission(ctx context.Context, accountID, userID string) error {
_, err := m.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, accountID)
if err == nil {
return nil
}
if !isNotFound(err) {
return fmt.Errorf("get agent network settings: %w", err)
}
return m.requirePermission(ctx, accountID, userID, modules.AgentNetworkSettings, operations.Create)
}
// maxDomainAllocationAttempts bounds the label search when bootstrapping a
// labeled endpoint. Package-level (rather than function-local) so tests can
// assert on the exhaustion path without duplicating the literal.
const maxDomainAllocationAttempts = 10
// bootstrapSettingsIfNeeded creates the per-account agent-network
// settings row when missing. The cluster comes from the create-time
// hint the dashboard sends (auto-picked from the active cluster list);
// the subdomain is picked from the curated wordlist avoiding
// collisions on the same cluster. Idempotent: if a row already exists
// it is returned untouched and the hint is ignored. st is the store to
// operate on — pass the transaction store when calling from within one.
func (m *managerImpl) bootstrapSettingsIfNeeded(ctx context.Context, st store.Store, accountID, providerCluster string) (*types.Settings, error) {
if accountID == "" {
return nil, fmt.Errorf("bootstrap settings: account id is required")
// CreateSettings bootstraps the per-account settings row, assigning the
// account's immutable endpoint. Exactly one of proxyAddress and endpoint must
// be non-empty: proxyAddress allocates a labeled endpoint one label beneath
// the given cluster address; endpoint claims the given hostname verbatim as a
// self-addressed (dedicated) endpoint — a legitimate claim before any proxy
// declares the address (address-first). settings carries the account ID and
// the initial collection toggles; its identity fields are assigned here.
func (m *managerImpl) CreateSettings(ctx context.Context, userID string, settings *types.Settings, proxyAddress, endpoint string) (*types.Settings, error) {
if settings == nil || settings.AccountID == "" {
return nil, status.Errorf(status.InvalidArgument, "account id is required")
}
if strings.TrimSpace(providerCluster) == "" {
return nil, fmt.Errorf("bootstrap settings: provider cluster is required")
if err := m.requirePermission(ctx, settings.AccountID, userID, modules.AgentNetworkSettings, operations.Create); err != nil {
return nil, err
}
existing, err := st.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, accountID)
if err == nil {
return existing, nil
hasProxyAddress := strings.TrimSpace(proxyAddress) != ""
hasEndpoint := strings.TrimSpace(endpoint) != ""
if hasProxyAddress == hasEndpoint {
return nil, status.Errorf(status.InvalidArgument, "exactly one of proxy_address and endpoint is required")
}
if !isNotFound(err) {
// Fail fast on an existing row for a clean 409; the insert below stays
// the authority against concurrent bootstraps (the primary key wins).
if _, err := m.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, settings.AccountID); err == nil {
return nil, status.Errorf(status.AlreadyExists, "agent network settings already bootstrapped for account %s", settings.AccountID)
} else if !isNotFound(err) {
return nil, fmt.Errorf("get agent network settings: %w", err)
}
siblings, err := st.GetAgentNetworkSettingsByCluster(ctx, store.LockingStrengthNone, providerCluster)
if err != nil {
return nil, fmt.Errorf("list agent network settings on cluster: %w", err)
}
taken := make(map[string]struct{}, len(siblings))
for _, s := range siblings {
taken[s.Subdomain] = struct{}{}
}
suffix := accountID
if len(suffix) > 4 {
suffix = suffix[:4]
}
m.labelRngMu.Lock()
subdomain := labelgen.PickUnique(m.labelRng, taken, suffix)
m.labelRngMu.Unlock()
now := time.Now().UTC()
settings := types.DefaultSettings(accountID)
settings.Cluster = providerCluster
settings.Subdomain = subdomain
settings.CreatedAt = now
settings.UpdatedAt = now
if err := st.SaveAgentNetworkSettings(ctx, settings); err != nil {
return nil, fmt.Errorf("save agent network settings: %w", err)
var err error
if hasEndpoint {
err = m.bootstrapSelfAddressed(ctx, settings, endpoint)
} else {
err = m.bootstrapLabeled(ctx, settings, proxyAddress)
}
if err != nil {
return nil, err
}
m.accountManager.StoreEvent(ctx, userID, settings.AccountID, settings.AccountID, activity.AgentNetworkSettingsUpdated, map[string]any{
"bootstrapped": true,
"endpoint": settings.Domain,
"dedicated": settings.Dedicated(),
})
m.reconcile(ctx, settings.AccountID)
return settings, nil
}
// bootstrapSelfAddressed claims the given hostname as the account's endpoint,
// served only by a proxy declaring exactly that address (Domain ==
// ProxyAddress). The domain unique index is the arbiter of availability.
func (m *managerImpl) bootstrapSelfAddressed(ctx context.Context, settings *types.Settings, endpoint string) error {
hostname, err := types.NormalizeHostname(endpoint)
if err != nil {
return status.Errorf(status.InvalidArgument, "invalid endpoint: %s", err)
}
settings.Domain = hostname
settings.ProxyAddress = hostname
if err := m.store.CreateAgentNetworkSettings(ctx, settings); err != nil {
if isUniqueConstraintError(err) {
// The violation is either the account primary key (a concurrent
// bootstrap for the same account won) or the domain index
// (another account holds the hostname). Distinguish by re-read.
if _, getErr := m.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, settings.AccountID); getErr == nil {
return status.Errorf(status.AlreadyExists, "agent network settings already bootstrapped for account %s", settings.AccountID)
}
return status.Errorf(status.AlreadyExists, "endpoint %s is already taken", hostname)
}
return fmt.Errorf("create agent network settings: %w", err)
}
return nil
}
// bootstrapLabeled allocates a labeled endpoint one label beneath the given
// cluster address: Domain = <label>.<proxyAddress>, served by whichever proxy
// declares the parent. Labels are adjective-noun tuples; a candidate is
// checked by read and the domain unique index stays the authority, so a
// concurrent allocation of the same tuple surfaces as a unique violation and
// another tuple is drawn.
func (m *managerImpl) bootstrapLabeled(ctx context.Context, settings *types.Settings, proxyAddress string) error {
parent, err := types.NormalizeHostname(proxyAddress)
if err != nil {
return status.Errorf(status.InvalidArgument, "invalid proxy_address: %s", err)
}
for attempt := 1; attempt <= maxDomainAllocationAttempts; attempt++ {
m.labelRngMu.Lock()
label := labelgen.PickTuple(m.labelRng)
m.labelRngMu.Unlock()
if label == "" {
// Only reachable if either word pool were emptied. An empty label
// would produce a broken endpoint like ".example.com", so fail
// loudly rather than looping or inserting.
return fmt.Errorf("allocate agent network endpoint for account %s: label generator returned an empty label", settings.AccountID)
}
candidate, err := types.NormalizeHostname(label + "." + parent)
if err != nil {
return status.Errorf(status.InvalidArgument, "proxy_address leaves no room for a label: %s", err)
}
_, err = m.store.GetAgentNetworkSettingsByDomain(ctx, store.LockingStrengthNone, candidate)
if err == nil {
log.WithContext(ctx).Tracef("agent-network endpoint %q taken, retrying (attempt %d/%d)", candidate, attempt, maxDomainAllocationAttempts)
continue
}
if !isNotFound(err) {
return fmt.Errorf("check agent network endpoint availability: %w", err)
}
settings.Domain = candidate
settings.ProxyAddress = parent
if err := m.store.CreateAgentNetworkSettings(ctx, settings); err != nil {
if isUniqueConstraintError(err) {
// A concurrent bootstrap for the same account may have won on
// the primary key — return the conflict. A lost race on the
// domain index just means the tuple was taken between the
// read and the insert: draw another.
if _, getErr := m.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, settings.AccountID); getErr == nil {
return status.Errorf(status.AlreadyExists, "agent network settings already bootstrapped for account %s", settings.AccountID)
}
log.WithContext(ctx).Tracef("agent-network endpoint %q lost an allocation race, retrying (attempt %d/%d)", candidate, attempt, maxDomainAllocationAttempts)
continue
}
return fmt.Errorf("create agent network settings: %w", err)
}
return nil
}
return fmt.Errorf("allocate agent network endpoint for account %s: %d attempts exhausted", settings.AccountID, maxDomainAllocationAttempts)
}
// isUniqueConstraintError reports whether err is a database unique-constraint
// violation, matched on the driver message because CreateAgentNetworkSettings
// deliberately returns the driver error unwrapped.
func isUniqueConstraintError(err error) bool {
if err == nil {
return false
}
msg := err.Error()
return strings.Contains(msg, "(SQLSTATE 23505)") || // postgres
strings.Contains(msg, "Error 1062 (23000)") || // mysql
strings.Contains(msg, "UNIQUE constraint failed") // sqlite
}
// ListConsumption returns every consumption row recorded for the
// account, ordered window-newest-first. Backs the dashboard's basic
// counter view; permission gate is the same Read role that gates
@@ -879,7 +1021,7 @@ func (*mockManager) GetProvider(_ context.Context, _, _, _ string) (*types.Provi
return &types.Provider{}, nil
}
func (*mockManager) CreateProvider(_ context.Context, _ string, p *types.Provider, _ string) (*types.Provider, error) {
func (*mockManager) CreateProvider(_ context.Context, _ string, p *types.Provider) (*types.Provider, error) {
return p, nil
}
@@ -947,10 +1089,23 @@ func (*mockManager) GetSettings(_ context.Context, accountID, _ string) (*types.
return types.DefaultSettings(accountID), nil
}
func (*mockManager) CreateSettings(_ context.Context, _ string, s *types.Settings, proxyAddress, endpoint string) (*types.Settings, error) {
if endpoint != "" {
s.Domain = endpoint
s.ProxyAddress = endpoint
} else {
s.Domain = "mock." + proxyAddress
s.ProxyAddress = proxyAddress
}
return s, nil
}
func (*mockManager) UpdateSettings(_ context.Context, _ string, s *types.Settings) (*types.Settings, error) {
return s, nil
}
func (*mockManager) DeleteSettings(_ context.Context, _, _ string) error { return nil }
func (*mockManager) ListConsumption(_ context.Context, _, _ string) ([]*types.Consumption, error) {
return nil, nil
}

View File

@@ -1,134 +0,0 @@
package agentnetwork
import (
"context"
"runtime"
"testing"
"github.com/golang/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
"github.com/netbirdio/netbird/management/server/account"
"github.com/netbirdio/netbird/management/server/permissions"
"github.com/netbirdio/netbird/management/server/permissions/modules"
"github.com/netbirdio/netbird/management/server/permissions/operations"
"github.com/netbirdio/netbird/management/server/store"
nbtypes "github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/status"
)
// bootstrapFixture wires a real sqlite store to a gomock permissions manager
// so tests can grant the provider permission while denying (or never
// expecting) the settings one.
type bootstrapFixture struct {
manager Manager
store store.Store
perms *permissions.MockManager
}
func newBootstrapFixture(t *testing.T) *bootstrapFixture {
t.Helper()
if runtime.GOOS == "windows" {
t.Skip("sqlite store not properly supported on Windows yet")
}
t.Setenv("NETBIRD_STORE_ENGINE", string(nbtypes.SqliteStoreEngine))
st, cleanUp, err := store.NewTestStoreFromSQL(context.Background(), "", t.TempDir())
require.NoError(t, err, "test store setup must succeed")
t.Cleanup(cleanUp)
ctrl := gomock.NewController(t)
perms := permissions.NewMockManager(ctrl)
accounts := account.NewMockManager(ctrl)
accounts.EXPECT().StoreEvent(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
accounts.EXPECT().UpdateAccountPeers(gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
accounts.EXPECT().BufferUpdateAccountPeers(gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
return &bootstrapFixture{
manager: NewManager(st, perms, accounts, nil),
store: st,
perms: perms,
}
}
func (f *bootstrapFixture) expectPermission(accountID, userID string, module modules.Module, op operations.Operation, allowed bool) {
f.perms.EXPECT().
ValidateUserPermissions(gomock.Any(), accountID, userID, module, op).
Return(allowed, context.Background(), nil)
}
func newBootstrapProvider(accountID string) *types.Provider {
p := types.NewProvider(accountID)
p.Name = "openai"
p.UpstreamURL = "https://api.openai.com"
p.APIKey = "sk-test"
p.Enabled = true
return p
}
// TestCreateProviderBootstrapRequiresSettingsPermission pins the gate on the
// one-time settings bootstrap: creating the first provider with a
// bootstrap_cluster pins the account's cluster and subdomain, which is a
// settings write and must not ride on the providers permission alone.
func TestCreateProviderBootstrapRequiresSettingsPermission(t *testing.T) {
ctx := context.Background()
t.Run("denied without settings permission", func(t *testing.T) {
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkProviders, operations.Create, true)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, false)
_, err := f.manager.CreateProvider(ctx, "user1", newBootstrapProvider("account1"), "cluster1.example.com")
require.Error(t, err, "bootstrap without settings permission must fail")
var sErr *status.Error
require.ErrorAs(t, err, &sErr)
assert.Equal(t, status.PermissionDenied, sErr.Type(), "denial should surface as permission denied")
providers, err := f.store.GetAccountAgentNetworkProviders(ctx, store.LockingStrengthNone, "account1")
require.NoError(t, err)
assert.Empty(t, providers, "provider must not be persisted when bootstrap is denied")
_, err = f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
assert.Error(t, err, "settings row must not be created when bootstrap is denied")
})
t.Run("allowed with settings permission", func(t *testing.T) {
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkProviders, operations.Create, true)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
created, err := f.manager.CreateProvider(ctx, "user1", newBootstrapProvider("account1"), "cluster1.example.com")
require.NoError(t, err, "bootstrap with both permissions must succeed")
require.NotNil(t, created)
settings, err := f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
require.NoError(t, err, "bootstrap must create the settings row")
assert.Equal(t, "cluster1.example.com", settings.Cluster, "settings should pin the bootstrap cluster")
})
t.Run("existing settings need no settings permission", func(t *testing.T) {
f := newBootstrapFixture(t)
require.NoError(t, f.store.SaveAgentNetworkSettings(ctx, &types.Settings{
AccountID: "account1",
Cluster: "cluster1.example.com",
Subdomain: "existing",
}), "pre-existing settings row setup must succeed")
// Only the providers permission may be consulted: gomock fails the
// test on any unexpected settings-permission call.
f.expectPermission("account1", "user1", modules.AgentNetworkProviders, operations.Create, true)
_, err := f.manager.CreateProvider(ctx, "user1", newBootstrapProvider("account1"), "cluster1.example.com")
require.NoError(t, err, "create with existing settings must not require the settings permission")
})
t.Run("no bootstrap cluster needs no settings permission", func(t *testing.T) {
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkProviders, operations.Create, true)
_, err := f.manager.CreateProvider(ctx, "user1", newBootstrapProvider("account1"), "")
require.NoError(t, err, "create without bootstrap must not require the settings permission")
})
}

View File

@@ -10,6 +10,17 @@ import (
"github.com/netbirdio/netbird/shared/management/proto"
)
// syntheticMapping pairs a synthesised proxy mapping with the address of the
// proxy that serves it. The cluster is recorded rather than derived from the
// mapping's domain: ProxyMapping does not carry it, and the previous derivation
// -- everything after the first DNS label -- is wrong whenever the service's
// domain is not one label under its proxy's address, which silently addressed
// updates to a cluster no proxy declares.
type syntheticMapping struct {
mapping *proto.ProxyMapping
cluster string
}
// reconcile recomputes the synthesised reverse-proxy services for an
// account, diffs them against the previously-synthesised set in the
// in-memory cache, and emits Create / Update / Delete proxy mappings
@@ -45,18 +56,21 @@ func (m *managerImpl) reconcile(ctx context.Context, accountID string) {
}
oidcCfg := m.proxyController.GetOIDCValidationConfig()
current := make(map[string]*proto.ProxyMapping, len(services))
current := make(map[string]syntheticMapping, len(services))
for _, svc := range services {
if svc == nil || svc.ID == "" {
continue
}
current[svc.ID] = svc.ToProtoMapping(rpservice.Update, "", oidcCfg)
current[svc.ID] = syntheticMapping{
mapping: svc.ToProtoMapping(rpservice.Update, "", oidcCfg),
cluster: svc.ProxyCluster,
}
}
m.reconcileMu.Lock()
previous := m.reconcileCache[accountID]
if previous == nil {
previous = make(map[string]*proto.ProxyMapping)
previous = make(map[string]syntheticMapping)
}
creates, updates, deletes := diffMappings(previous, current)
@@ -67,34 +81,36 @@ func (m *managerImpl) reconcile(ctx context.Context, accountID string) {
}
m.reconcileMu.Unlock()
for _, mapping := range creates {
mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_CREATED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, mapping, clusterFromMapping(mapping))
for _, entry := range creates {
entry.mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_CREATED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, entry.mapping, entry.cluster)
}
for _, mapping := range updates {
mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_MODIFIED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, mapping, clusterFromMapping(mapping))
for _, entry := range updates {
entry.mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_MODIFIED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, entry.mapping, entry.cluster)
}
for _, mapping := range deletes {
mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_REMOVED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, mapping, clusterFromMapping(mapping))
for _, entry := range deletes {
entry.mapping.Type = proto.ProxyMappingUpdateType_UPDATE_TYPE_REMOVED
m.proxyController.SendServiceUpdateToCluster(ctx, accountID, entry.mapping, entry.cluster)
}
}
// diffMappings classifies the previous→current transition for a
// single account into Create / Update / Delete sets.
// diffMappings classifies the previous→current transition for a single
// account into Create / Update / Delete sets.
//
// Cluster moves (current.cluster != previous.cluster) are surfaced as
// a Delete on the old cluster + Create on the new — handled by
// emitting both a delete (on previous mapping) and a create (on the
// current mapping) for that service ID.
func diffMappings(previous, current map[string]*proto.ProxyMapping) (creates, updates, deletes []*proto.ProxyMapping) {
// A change of serving proxy for the same service ID is surfaced as a Delete
// addressed to the old proxy plus a Create addressed to the new one, so the
// mapping actually moves. Comparing the recorded cluster is what makes that
// detectable: with a placement-free endpoint the mapping's domain is identical
// before and after the move, so nothing about the mapping itself reveals it.
func diffMappings(previous, current map[string]syntheticMapping) (creates, updates, deletes []syntheticMapping) {
for id, cur := range current {
prev, existed := previous[id]
switch {
case !existed:
creates = append(creates, cur)
case prev.GetDomain() == "" || cur.GetAccountId() == prev.GetAccountId() && currentClusterChanged(prev, cur):
case prev.mapping.GetDomain() == "" ||
cur.mapping.GetAccountId() == prev.mapping.GetAccountId() && prev.cluster != cur.cluster:
deletes = append(deletes, prev)
creates = append(creates, cur)
default:
@@ -108,24 +124,3 @@ func diffMappings(previous, current map[string]*proto.ProxyMapping) (creates, up
}
return creates, updates, deletes
}
func currentClusterChanged(prev, cur *proto.ProxyMapping) bool {
return clusterFromMapping(prev) != clusterFromMapping(cur)
}
// clusterFromMapping returns the cluster the mapping should be sent
// to. ProxyMapping doesn't carry the cluster directly, so we rely on
// the synthesised service's domain (`<slug>.<cluster>`) and split on
// the first '.'.
func clusterFromMapping(m *proto.ProxyMapping) string {
if m == nil {
return ""
}
domain := m.GetDomain()
for i := 0; i < len(domain); i++ {
if domain[i] == '.' {
return domain[i+1:]
}
}
return ""
}

View File

@@ -21,7 +21,7 @@ func newReconcileMgr(t *testing.T, ctrl *gomock.Controller) (*managerImpl, *stor
return &managerImpl{
store: mockStore,
proxyController: mockProxy,
reconcileCache: make(map[string]map[string]*proto.ProxyMapping),
reconcileCache: make(map[string]map[string]syntheticMapping),
}, mockStore, mockProxy
}
@@ -52,9 +52,9 @@ func newReconcileTestPolicy(providerID, sourceGroupID string) *types.Policy {
func newReconcileTestSettings() *types.Settings {
return &types.Settings{
AccountID: "acct-1",
Cluster: "eu.proxy.netbird.io",
Subdomain: "violet",
AccountID: "acct-1",
Domain: "violet.eu.proxy.netbird.io",
ProxyAddress: "eu.proxy.netbird.io",
}
}
@@ -196,7 +196,7 @@ func TestReconcile_PolicyRemoved_EmitsDelete(t *testing.T) {
func TestReconcile_NilProxyController_NoOp(t *testing.T) {
ctx := context.Background()
mgr := &managerImpl{
reconcileCache: make(map[string]map[string]*proto.ProxyMapping),
reconcileCache: make(map[string]map[string]syntheticMapping),
}
// Must not panic; must not query the store.
mgr.reconcile(ctx, "acct-1")
@@ -212,21 +212,78 @@ func TestReconcile_EmptyAccountID_NoOp(t *testing.T) {
mgr.reconcile(ctx, "")
}
func TestClusterFromMapping(t *testing.T) {
tests := []struct {
name string
domain string
want string
}{
{"simple", "openai.eu.proxy.netbird.io", "eu.proxy.netbird.io"},
{"deeply nested", "a.b.c.d", "b.c.d"},
{"no dot", "openai", ""},
{"empty", "", ""},
// TestDiffMappings_ServingProxyChange — when the proxy serving an account
// changes, the same service ID must be deleted on the old proxy and created on
// the new one. The cluster cannot be recovered from the mapping's domain: with a
// placement-free endpoint the domain does not change at all when the serving
// proxy does, so a domain-derived cluster sees no change and emits a plain
// update, addressed to a proxy that does not exist.
func TestDiffMappings_ServingProxyChange(t *testing.T) {
previous := map[string]syntheticMapping{
"svc-1": {
mapping: &proto.ProxyMapping{Id: "svc-1", AccountId: "acct-1", Domain: "brave-otter.gateway.example.com"},
cluster: "proxy.example.com",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := clusterFromMapping(&proto.ProxyMapping{Domain: tt.domain})
assert.Equal(t, tt.want, got)
})
current := map[string]syntheticMapping{
"svc-1": {
mapping: &proto.ProxyMapping{Id: "svc-1", AccountId: "acct-1", Domain: "brave-otter.gateway.example.com"},
cluster: "brave-otter.gateway.example.com",
},
}
creates, updates, deletes := diffMappings(previous, current)
if assert.Len(t, deletes, 1, "the old proxy must be told to drop the mapping") {
assert.Equal(t, "proxy.example.com", deletes[0].cluster)
}
if assert.Len(t, creates, 1, "the new proxy must be told to add it") {
assert.Equal(t, "brave-otter.gateway.example.com", creates[0].cluster)
}
assert.Empty(t, updates, "a serving-proxy move is a delete plus a create, not an update")
}
// TestDiffMappings_UnchangedClusterIsAnUpdate keeps the ordinary path: same
// service, same proxy, changed contents.
func TestDiffMappings_UnchangedClusterIsAnUpdate(t *testing.T) {
previous := map[string]syntheticMapping{
"svc-1": {
mapping: &proto.ProxyMapping{Id: "svc-1", AccountId: "acct-1", Domain: "otter.proxy.example.com"},
cluster: "proxy.example.com",
},
}
current := map[string]syntheticMapping{
"svc-1": {
mapping: &proto.ProxyMapping{Id: "svc-1", AccountId: "acct-1", Domain: "otter.proxy.example.com"},
cluster: "proxy.example.com",
},
}
creates, updates, deletes := diffMappings(previous, current)
assert.Empty(t, creates)
assert.Empty(t, deletes)
if assert.Len(t, updates, 1) {
assert.Equal(t, "proxy.example.com", updates[0].cluster)
}
}
// TestDiffMappings_RemovedServiceIsDeletedOnItsOwnCluster — a service that has
// gone away is deleted on the cluster it was last served by, which is recorded
// rather than re-derived.
func TestDiffMappings_RemovedServiceIsDeletedOnItsOwnCluster(t *testing.T) {
previous := map[string]syntheticMapping{
"svc-1": {
mapping: &proto.ProxyMapping{Id: "svc-1", AccountId: "acct-1", Domain: "brave-otter.gateway.example.com"},
cluster: "brave-otter.gateway.example.com",
},
}
creates, updates, deletes := diffMappings(previous, map[string]syntheticMapping{})
assert.Empty(t, creates)
assert.Empty(t, updates)
if assert.Len(t, deletes, 1) {
assert.Equal(t, "brave-otter.gateway.example.com", deletes[0].cluster)
}
}

View File

@@ -0,0 +1,225 @@
package agentnetwork
import (
"context"
"runtime"
"strings"
"testing"
"github.com/golang/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
"github.com/netbirdio/netbird/management/server/account"
"github.com/netbirdio/netbird/management/server/permissions"
"github.com/netbirdio/netbird/management/server/permissions/modules"
"github.com/netbirdio/netbird/management/server/permissions/operations"
"github.com/netbirdio/netbird/management/server/store"
nbtypes "github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/status"
)
// bootstrapFixture wires a real sqlite store to a gomock permissions manager
// so tests can grant or deny the settings permission per case.
type bootstrapFixture struct {
manager Manager
store store.Store
perms *permissions.MockManager
}
func newBootstrapFixture(t *testing.T) *bootstrapFixture {
t.Helper()
if runtime.GOOS == "windows" {
t.Skip("sqlite store not properly supported on Windows yet")
}
t.Setenv("NETBIRD_STORE_ENGINE", string(nbtypes.SqliteStoreEngine))
st, cleanUp, err := store.NewTestStoreFromSQL(context.Background(), "", t.TempDir())
require.NoError(t, err, "test store setup must succeed")
t.Cleanup(cleanUp)
ctrl := gomock.NewController(t)
perms := permissions.NewMockManager(ctrl)
accounts := account.NewMockManager(ctrl)
accounts.EXPECT().StoreEvent(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
accounts.EXPECT().UpdateAccountPeers(gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
accounts.EXPECT().BufferUpdateAccountPeers(gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
return &bootstrapFixture{
manager: NewManager(st, perms, accounts, nil),
store: st,
perms: perms,
}
}
func (f *bootstrapFixture) expectPermission(accountID, userID string, module modules.Module, op operations.Operation, allowed bool) {
f.perms.EXPECT().
ValidateUserPermissions(gomock.Any(), accountID, userID, module, op).
Return(allowed, context.Background(), nil)
}
func (f *bootstrapFixture) createSettings(ctx context.Context, accountID, userID, proxyAddress, endpoint string) (*types.Settings, error) {
return f.manager.CreateSettings(ctx, userID, types.DefaultSettings(accountID), proxyAddress, endpoint)
}
// TestCreateSettingsRequiresPermission pins the gate: bootstrap assigns the
// account's immutable endpoint, a settings write requiring the settings
// Create permission — and a denial leaves no row behind.
func TestCreateSettingsRequiresPermission(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, false)
_, err := f.createSettings(ctx, "account1", "user1", "cluster1.example.com", "")
require.Error(t, err, "bootstrap without the settings permission must fail")
var sErr *status.Error
require.ErrorAs(t, err, &sErr)
assert.Equal(t, status.PermissionDenied, sErr.Type(), "denial should surface as permission denied")
_, err = f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
assert.Error(t, err, "settings row must not be created when bootstrap is denied")
}
// TestCreateSettingsLabeled pins the labeled shape: the server allocates an
// adjective-noun label beneath the proxy address, the pin is not dedicated,
// and the domain records the full endpoint hostname.
func TestCreateSettingsLabeled(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
created, err := f.createSettings(ctx, "account1", "user1", "Cluster1.Example.com", "")
require.NoError(t, err, "labeled bootstrap must succeed")
assert.Equal(t, "cluster1.example.com", created.ProxyAddress, "proxy address must be pinned lowercased")
require.True(t, strings.HasSuffix(created.Domain, ".cluster1.example.com"),
"domain must hang one label beneath the proxy address: %s", created.Domain)
label := strings.TrimSuffix(created.Domain, ".cluster1.example.com")
assert.NotContains(t, label, ".", "the allocated label must be a single DNS label: %s", label)
assert.False(t, created.Dedicated(), "a labeled pin is not dedicated")
assert.Equal(t, created.Domain, created.Endpoint(), "the endpoint is the domain column")
stored, err := f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
require.NoError(t, err, "bootstrap must persist the row")
assert.Equal(t, created.Domain, stored.Domain)
assert.Equal(t, created.ProxyAddress, stored.ProxyAddress)
}
// TestCreateSettingsSelfAddressed pins the dedicated shape: the endpoint is
// claimed verbatim (normalized), Domain == ProxyAddress, and the claim
// succeeds with no proxy declaring the address yet (address-first).
func TestCreateSettingsSelfAddressed(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
created, err := f.createSettings(ctx, "account1", "user1", "", "Brave-Otter.GW.Example.com")
require.NoError(t, err, "self-addressed bootstrap must succeed")
assert.Equal(t, "brave-otter.gw.example.com", created.Domain, "endpoint must be claimed lowercased")
assert.Equal(t, created.Domain, created.ProxyAddress, "self-addressed: proxy address is the endpoint")
assert.True(t, created.Dedicated(), "a self-addressed pin is dedicated")
}
// TestCreateSettingsIdentityFieldValidation pins the request contract: exactly
// one of proxyAddress and endpoint, and both must be well-formed hostnames.
func TestCreateSettingsIdentityFieldValidation(t *testing.T) {
ctx := context.Background()
cases := map[string]struct {
proxyAddress string
endpoint string
}{
"neither": {"", ""},
"both": {"cluster1.example.com", "gw.example.com"},
"trailing dot endpoint": {"", "gw.example.com."},
"leading dot endpoint": {"", ".gw.example.com"},
"whitespace inside": {"", "g w.example.com"},
"empty label in parent": {"eu..example.com", ""},
"hyphen-edged label": {"", "-gw.example.com"},
}
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
_, err := f.createSettings(ctx, "account1", "user1", tc.proxyAddress, tc.endpoint)
require.Error(t, err, "invalid identity input must be rejected")
var sErr *status.Error
require.ErrorAs(t, err, &sErr)
assert.Equal(t, status.InvalidArgument, sErr.Type(), "rejection must be a validation error")
_, err = f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
assert.Error(t, err, "no row may be left behind by a rejected bootstrap")
})
}
}
// TestCreateSettingsConflictsOnSecondBootstrap pins that bootstrap is a
// one-time create per account: a second call is a conflict, whatever shape it
// asks for, and the original row survives untouched.
func TestCreateSettingsConflictsOnSecondBootstrap(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
first, err := f.createSettings(ctx, "account1", "user1", "cluster1.example.com", "")
require.NoError(t, err)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
_, err = f.createSettings(ctx, "account1", "user1", "", "other.example.com")
require.Error(t, err, "second bootstrap must fail")
var sErr *status.Error
require.ErrorAs(t, err, &sErr)
assert.Equal(t, status.AlreadyExists, sErr.Type(), "second bootstrap must surface as a conflict")
stored, err := f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
require.NoError(t, err)
assert.Equal(t, first.Domain, stored.Domain, "the original endpoint must survive the rejected bootstrap")
}
// TestCreateSettingsEndpointTaken pins global hostname uniqueness: a hostname
// held by one account cannot be claimed by another, in either direction —
// self-addressed onto self-addressed, or self-addressed onto an allocated
// labeled endpoint.
func TestCreateSettingsEndpointTaken(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkSettings, operations.Create, true)
first, err := f.createSettings(ctx, "account1", "user1", "", "gw.example.com")
require.NoError(t, err)
f.expectPermission("account2", "user2", modules.AgentNetworkSettings, operations.Create, true)
_, err = f.createSettings(ctx, "account2", "user2", "", "gw.example.com")
require.Error(t, err, "a taken hostname must be refused")
var sErr *status.Error
require.ErrorAs(t, err, &sErr)
assert.Equal(t, status.AlreadyExists, sErr.Type(), "the refusal must surface as a conflict")
f.expectPermission("account3", "user3", modules.AgentNetworkSettings, operations.Create, true)
_, err = f.createSettings(ctx, "account3", "user3", "", first.Domain)
require.Error(t, err, "claiming another account's endpoint must be refused")
}
// TestCreateProviderHasNoSettingsSideEffects pins the decoupling: provider
// create needs only the providers permission (gomock fails the test on any
// settings-permission call) and never creates a settings row.
func TestCreateProviderHasNoSettingsSideEffects(t *testing.T) {
ctx := context.Background()
f := newBootstrapFixture(t)
f.expectPermission("account1", "user1", modules.AgentNetworkProviders, operations.Create, true)
provider := types.NewProvider("account1")
provider.Name = "openai"
provider.UpstreamURL = "https://api.openai.com"
provider.APIKey = "sk-test"
provider.Enabled = true
created, err := f.manager.CreateProvider(ctx, "user1", provider)
require.NoError(t, err, "provider create must succeed on the providers permission alone")
require.NotNil(t, created)
_, err = f.store.GetAgentNetworkSettings(ctx, store.LockingStrengthNone, "account1")
assert.Error(t, err, "provider create must not conjure a settings row")
}

View File

@@ -89,7 +89,7 @@ func SynthesizeServicesForCluster(ctx context.Context, s store.Store, clusterAdd
return nil, nil
}
settingsRows, err := s.GetAgentNetworkSettingsByCluster(ctx, store.LockingStrengthNone, clusterAddr)
settingsRows, err := s.GetAgentNetworkSettingsByProxyAddress(ctx, store.LockingStrengthNone, clusterAddr)
if err != nil {
return nil, fmt.Errorf("list agent network settings on cluster: %w", err)
}
@@ -116,53 +116,41 @@ func SynthesizeServicesForCluster(ctx context.Context, s store.Store, clusterAdd
}
// SynthesizeServiceForDomain resolves a single agent-network service by its
// public endpoint domain. It lists the (few) settings rows on the domain's
// cluster, matches the one whose endpoint equals the domain, and synthesises
// only that account — avoiding full per-account synthesis for every tenant on
// the cluster, which is what auth/session paths previously paid. Returns nil
// (no error) when no account owns the domain.
// public endpoint domain — a point query on the settings domain unique index,
// then synthesis of just that account. Returns nil (no error) when no account
// owns the domain.
func SynthesizeServiceForDomain(ctx context.Context, s store.Store, domain string) (*rpservice.Service, error) {
domain = strings.TrimSpace(domain)
cluster := clusterFromDomain(domain)
if domain != "" && cluster != "" {
settingsRows, err := s.GetAgentNetworkSettingsByCluster(ctx, store.LockingStrengthNone, cluster)
if err != nil {
return nil, fmt.Errorf("list agent network settings on cluster: %w", err)
domain = strings.ToLower(strings.TrimSpace(domain))
if domain == "" {
return nil, nil //nolint:nilnil // optional lookup: no account owns the domain
}
settings, err := s.GetAgentNetworkSettingsByDomain(ctx, store.LockingStrengthNone, domain)
if err != nil {
if isNotFound(err) {
return nil, nil //nolint:nilnil // optional lookup: no account owns the domain
}
for _, settings := range settingsRows {
if settings == nil || settings.Endpoint() != domain {
continue
}
services, serr := SynthesizeServices(ctx, s, settings.AccountID)
if serr != nil {
return nil, serr
}
for _, svc := range services {
if svc != nil && svc.Domain == domain {
return svc, nil
}
}
break
return nil, fmt.Errorf("get agent network settings by domain: %w", err)
}
services, err := SynthesizeServices(ctx, s, settings.AccountID)
if err != nil {
return nil, err
}
for _, svc := range services {
if svc != nil && svc.Domain == domain {
return svc, nil
}
}
return nil, nil //nolint:nilnil // optional lookup: no account owns the domain
}
// clusterFromDomain returns the cluster portion of an endpoint domain (every
// label after the first).
func clusterFromDomain(domain string) string {
if i := strings.IndexByte(domain, '.'); i >= 0 {
return domain[i+1:]
}
return ""
}
// SynthesizeServices builds the in-memory reverse-proxy service that
// fronts the account's agent-network gateway. Returns nil when the
// account has no settings row, no enabled providers, or no enabled
// policies — in any of those cases there's nothing useful to expose.
//
// One service per (account, settings.Cluster) is emitted. The router
// One service per (account, settings.ProxyAddress) is emitted. The router
// middleware encodes a denormalised model→provider routing table
// (auth headers + decrypted API keys baked in); the policy_check
// middleware encodes per-provider authorised group IDs derived from
@@ -175,7 +163,7 @@ func SynthesizeServices(ctx context.Context, s store.Store, accountID string) ([
if err != nil {
return nil, err
}
if !ok || strings.TrimSpace(settings.Cluster) == "" {
if !ok || strings.TrimSpace(settings.ProxyAddress) == "" {
return nil, nil
}
@@ -934,7 +922,7 @@ func buildAccountService(
middlewares []rpservice.MiddlewareConfig,
sessionPriv, sessionPub string,
) *rpservice.Service {
cluster := settings.Cluster
cluster := settings.ProxyAddress
domain := settings.Endpoint()
serviceID := SynthesizedServiceIDPrefix + accountID

View File

@@ -147,7 +147,7 @@ func TestReconcile_RealStore_PushesPrivateAfterStatusToggle(t *testing.T) {
store: s,
accountManager: noopAccountManager{},
proxyController: ctrl,
reconcileCache: make(map[string]map[string]*proto.ProxyMapping),
reconcileCache: make(map[string]map[string]syntheticMapping),
}
m.reconcile(ctx, testAccountID) // initial, provider enabled

View File

@@ -19,15 +19,14 @@ import (
const (
testAccountID = "acct-1"
testCluster = "eu.proxy.netbird.io"
testSubdomain = "violet"
testEndpoint = "violet.eu.proxy.netbird.io"
)
func newSynthTestSettings() *types.Settings {
return &types.Settings{
AccountID: testAccountID,
Cluster: testCluster,
Subdomain: testSubdomain,
AccountID: testAccountID,
Domain: testEndpoint,
ProxyAddress: testCluster,
}
}

View File

@@ -1,6 +1,7 @@
package types
import (
"fmt"
"strings"
"time"
@@ -12,13 +13,23 @@ import (
// the long-term aggregate and are retained independently.
const DefaultAccessLogRetentionDays = 30
// Settings is the per-account agent-network configuration row. One
// row per account. Cluster + Subdomain are immutable once written and
// produce the public endpoint agents call (`<subdomain>.<cluster>`).
// Settings is the per-account agent-network configuration row. One row per
// account. Domain and ProxyAddress are assigned at bootstrap and immutable
// thereafter; a persisted row is always fully allocated — there is no "row
// exists, endpoint pending" state.
type Settings struct {
AccountID string `gorm:"primaryKey"`
Cluster string
Subdomain string `gorm:"index:idx_agent_network_settings_cluster_subdomain"`
// Domain is the gateway endpoint hostname agents call. Globally unique
// across accounts. Sized explicitly because MySQL cannot index an
// unbounded TEXT column; 255 covers the RFC 1035 253-octet bound.
Domain string `gorm:"type:varchar(255);uniqueIndex:idx_agent_network_settings_domain"`
// ProxyAddress is the declared cluster address of the proxy serving this
// account's gateway. Either equal to Domain — a proxy dedicated to this
// account, declaring the tenant's own hostname — or Domain's immediate
// parent, with the endpoint one label beneath it on a shared cluster.
ProxyAddress string `gorm:"type:varchar(255);index:idx_agent_network_settings_proxy_address"`
// Account-level collection controls sourced by the synthesizer.
// EnableLogCollection gates the per-request access-log trail and defaults
@@ -45,9 +56,9 @@ func (Settings) TableName() string { return "agent_network_settings" }
// DefaultSettings returns the settings an account observes before its row is
// bootstrapped: log collection on with the default retention, everything else
// off, and no cluster/subdomain assigned yet. Bootstrap persists exactly these
// values plus the assigned cluster and subdomain, so the pre-bootstrap read
// and the freshly bootstrapped row agree.
// off, and no domain or proxy address assigned yet. Bootstrap persists exactly
// these values plus the assigned domain and proxy address, so the
// pre-bootstrap read and the freshly bootstrapped row agree.
func DefaultSettings(accountID string) *Settings {
return &Settings{
AccountID: accountID,
@@ -56,14 +67,15 @@ func DefaultSettings(accountID string) *Settings {
}
}
// Endpoint returns the bare hostname agents reach this account at:
// `<subdomain>.<cluster>`. Empty until both halves are assigned at bootstrap.
func (s *Settings) Endpoint() string {
if s.Cluster == "" || s.Subdomain == "" {
return ""
}
return s.Subdomain + "." + s.Cluster
}
// Endpoint returns the bare hostname agents reach this account at — the
// Domain column. Empty until the row is bootstrapped.
func (s *Settings) Endpoint() string { return s.Domain }
// Dedicated reports whether the account's gateway is served by a proxy
// dedicated to it — the self-addressed shape, where the serving proxy declares
// the endpoint hostname itself. The alternative (labeled) shape has the
// endpoint one label beneath a shared cluster's address.
func (s *Settings) Dedicated() bool { return s.Domain != "" && s.Domain == s.ProxyAddress }
// ToAPIResponse renders the settings as the API representation. The
// timestamps are omitted while zero — a default (not yet bootstrapped) view
@@ -71,9 +83,9 @@ func (s *Settings) Endpoint() string {
func (s *Settings) ToAPIResponse() *api.AgentNetworkSettings {
retention := s.AccessLogRetentionDays
resp := &api.AgentNetworkSettings{
Cluster: s.Cluster,
Subdomain: s.Subdomain,
Endpoint: s.Endpoint(),
ProxyAddress: s.ProxyAddress,
Dedicated: s.Dedicated(),
EnableLogCollection: s.EnableLogCollection,
EnablePromptCollection: s.EnablePromptCollection,
RedactPii: s.RedactPii,
@@ -90,19 +102,91 @@ func (s *Settings) ToAPIResponse() *api.AgentNetworkSettings {
return resp
}
// FromAPIRequest applies the request onto the receiver. The mutable
// collection fields are always replaced with the request values. Cluster
// participates only in bootstrap and the immutability check (see
// Manager.UpdateSettings); Subdomain is server-assigned and never taken
// from a request.
// FromAPIRequest applies the update request onto the receiver: every mutable
// field is replaced with the request value, and the identity fields (Domain,
// ProxyAddress) carry the request's echo of the assigned values. The identity
// fields are never written to the stored row — UpdateSettings compares them
// against it and rejects the request when they differ, so PUT keeps the
// house convention of requiring every field while the endpoint and proxy
// address stay immutable.
//
// Every field is required by the schema, so none is presence-sensitive.
// AccessLogRetentionDays in particular must stay required: the caller receives
// a zero-valued Settings, and UpdateSettings copies each field onto the stored
// row unconditionally, so an omitted value would be written as 0 — which the
// API documents as "keep indefinitely". Making retention optional would
// therefore let a client silently maximise log retention by leaving it out.
func (s *Settings) FromAPIRequest(req *api.AgentNetworkSettingsRequest) {
if req.Cluster != nil {
s.Cluster = strings.TrimSpace(*req.Cluster)
}
s.Domain = req.Endpoint
s.ProxyAddress = req.ProxyAddress
s.EnableLogCollection = req.EnableLogCollection
s.EnablePromptCollection = req.EnablePromptCollection
s.RedactPii = req.RedactPii
s.AccessLogRetentionDays = req.AccessLogRetentionDays
}
// FromAPICreateRequest applies the optional collection toggles of a bootstrap
// request onto the receiver (typically DefaultSettings), leaving defaults in
// place for omitted fields. The identity fields are resolved by the manager
// from the request's proxy_address / endpoint, not copied here.
func (s *Settings) FromAPICreateRequest(req *api.AgentNetworkSettingsCreateRequest) {
if req.EnableLogCollection != nil {
s.EnableLogCollection = *req.EnableLogCollection
}
if req.EnablePromptCollection != nil {
s.EnablePromptCollection = *req.EnablePromptCollection
}
if req.RedactPii != nil {
s.RedactPii = *req.RedactPii
}
if req.AccessLogRetentionDays != nil {
s.AccessLogRetentionDays = *req.AccessLogRetentionDays
}
}
// maxHostnameLength is the RFC 1035 bound on a full domain name.
const maxHostnameLength = 253
// NormalizeHostname lowercases and trims a caller-supplied hostname and
// validates its shape: non-empty DNS labels of letters, digits and inner
// hyphens, joined by single dots, within length bounds. Shapes that
// canonicalization cannot repair — leading/trailing dots, empty labels,
// whitespace inside the name — are rejected rather than guessed at, because
// the value lands in an immutable column.
func NormalizeHostname(raw string) (string, error) {
hostname := strings.ToLower(strings.TrimSpace(raw))
if hostname == "" {
return "", fmt.Errorf("hostname is empty")
}
if len(hostname) > maxHostnameLength {
return "", fmt.Errorf("hostname exceeds %d characters", maxHostnameLength)
}
for _, label := range strings.Split(hostname, ".") {
if err := validateHostnameLabel(label); err != nil {
return "", fmt.Errorf("invalid hostname %q: %w", hostname, err)
}
}
return hostname, nil
}
func validateHostnameLabel(label string) error {
if label == "" {
return fmt.Errorf("empty label (leading, trailing or doubled dot)")
}
if len(label) > 63 {
return fmt.Errorf("label %q exceeds 63 characters", label)
}
if label[0] == '-' || label[len(label)-1] == '-' {
return fmt.Errorf("label %q must not start or end with a hyphen", label)
}
for _, r := range label {
switch {
case r >= 'a' && r <= 'z':
case r >= '0' && r <= '9':
case r == '-':
default:
return fmt.Errorf("label %q contains invalid character %q", label, r)
}
}
return nil
}

View File

@@ -2,24 +2,28 @@ package manager
import (
"context"
"errors"
"fmt"
"net"
"strings"
log "github.com/sirupsen/logrus"
agentnetworkTypes "github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
"github.com/netbirdio/netbird/management/internals/modules/reverseproxy/domain"
"github.com/netbirdio/netbird/management/server/account"
"github.com/netbirdio/netbird/management/server/activity"
"github.com/netbirdio/netbird/management/server/permissions"
"github.com/netbirdio/netbird/management/server/permissions/modules"
"github.com/netbirdio/netbird/management/server/permissions/operations"
nbstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/status"
)
type store interface {
GetAccount(ctx context.Context, accountID string) (*types.Account, error)
GetAgentNetworkSettings(ctx context.Context, lockStrength nbstore.LockingStrength, accountID string) (*agentnetworkTypes.Settings, error)
GetCustomDomain(ctx context.Context, accountID string, domainID string) (*domain.Domain, error)
ListFreeDomains(ctx context.Context, accountID string) ([]string, error)
@@ -311,17 +315,21 @@ func (m Manager) getClusterAllowList(ctx context.Context, accountID string) ([]s
if err != nil {
return nil, fmt.Errorf("get public cluster addresses: %w", err)
}
reserved, err := m.reservedGatewayAddress(ctx, accountID)
if err != nil {
return nil, err
}
seen := make(map[string]struct{}, len(byopAddresses)+len(publicAddresses))
merged := make([]string, 0, len(byopAddresses)+len(publicAddresses))
for _, addr := range byopAddresses {
if _, ok := seen[addr]; ok {
if _, ok := seen[addr]; ok || addr == reserved {
continue
}
seen[addr] = struct{}{}
merged = append(merged, addr)
}
for _, addr := range publicAddresses {
if _, ok := seen[addr]; ok {
if _, ok := seen[addr]; ok || addr == reserved {
continue
}
seen[addr] = struct{}{}
@@ -330,6 +338,31 @@ func (m Manager) getClusterAllowList(ctx context.Context, accountID string) ([]s
return merged, nil
}
// reservedGatewayAddress returns the account's agent-network gateway address
// when its settings pin is self-addressed — a proxy dedicated to serving
// exactly the gateway. Dropping that address from the cluster allow list keeps
// it from being offered as a cluster for ordinary services, and because the
// free-domain suffix match is depth-independent, dropping the address rejects
// every name beneath it as well as the bare one. Only the account's own
// gateway address can ever appear in its allow list (another tenant's gateway
// proxy is account-scoped to them), so this single-address exclusion is
// sufficient. Returns "" when the account has no settings row or a labeled
// (shared-cluster) pin.
func (m Manager) reservedGatewayAddress(ctx context.Context, accountID string) (string, error) {
settings, err := m.store.GetAgentNetworkSettings(ctx, nbstore.LockingStrengthNone, accountID)
if err != nil {
var sErr *status.Error
if errors.As(err, &sErr) && sErr.Type() == status.NotFound {
return "", nil
}
return "", fmt.Errorf("get agent network settings: %w", err)
}
if settings == nil || !settings.Dedicated() {
return "", nil
}
return settings.ProxyAddress, nil
}
func extractClusterFromCustomDomains(serviceDomain string, customDomains []*domain.Domain) (string, bool) {
bestCluster := ""
bestLen := -1

View File

@@ -7,6 +7,12 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
agentnetworkTypes "github.com/netbirdio/netbird/management/internals/modules/agentnetwork/types"
"github.com/netbirdio/netbird/management/internals/modules/reverseproxy/domain"
nbstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/status"
)
type mockProxyManager struct {
@@ -55,7 +61,7 @@ func TestGetClusterAllowList_BYOPMergedWithPublic(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"byop.example.com", "eu.proxy.netbird.io"}, result)
@@ -71,7 +77,7 @@ func TestGetClusterAllowList_DeduplicatesBYOPAndPublic(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"shared.example.com", "byop.example.com", "eu.proxy.netbird.io"}, result)
@@ -87,7 +93,7 @@ func TestGetClusterAllowList_NoBYOP_FallbackToShared(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"eu.proxy.netbird.io", "us.proxy.netbird.io"}, result)
@@ -100,7 +106,7 @@ func TestGetClusterAllowList_BYOPError_ReturnsError(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.Error(t, err)
assert.Nil(t, result)
@@ -117,7 +123,7 @@ func TestGetClusterAllowList_PublicError_ReturnsError(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.Error(t, err)
assert.Nil(t, result)
@@ -134,7 +140,7 @@ func TestGetClusterAllowList_BYOPEmptySlice_FallbackToShared(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"eu.proxy.netbird.io"}, result)
@@ -150,8 +156,138 @@ func TestGetClusterAllowList_PublicEmpty_BYOPOnly(t *testing.T) {
},
}
mgr := Manager{proxyManager: pm}
mgr := Manager{store: &stubStore{}, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"byop.example.com"}, result)
}
// stubStore satisfies the manager's narrow store interface for allow-list
// tests. Only the agent-network settings lookup participates; the default (a
// nil func) reads as "no settings row", the state most accounts are in.
type stubStore struct {
getAgentNetworkSettingsFunc func(ctx context.Context, accountID string) (*agentnetworkTypes.Settings, error)
}
func (s *stubStore) GetAccount(context.Context, string) (*types.Account, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) GetAgentNetworkSettings(ctx context.Context, _ nbstore.LockingStrength, accountID string) (*agentnetworkTypes.Settings, error) {
if s.getAgentNetworkSettingsFunc != nil {
return s.getAgentNetworkSettingsFunc(ctx, accountID)
}
return nil, status.Errorf(status.NotFound, "agent network settings for account %s not found", accountID)
}
func (s *stubStore) GetCustomDomain(context.Context, string, string) (*domain.Domain, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) ListFreeDomains(context.Context, string) ([]string, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) ListCustomDomains(context.Context, string) ([]*domain.Domain, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) CreateCustomDomain(context.Context, string, string, string, bool) (*domain.Domain, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) UpdateCustomDomain(context.Context, string, *domain.Domain) (*domain.Domain, error) {
panic("not used in allow-list tests")
}
func (s *stubStore) DeleteCustomDomain(context.Context, string, string) error {
panic("not used in allow-list tests")
}
// TestGetClusterAllowList_DedicatedGatewayAddressExcluded pins invariant (B)'s
// chokepoint: a self-addressed settings pin reserves the account's gateway
// address, so it is dropped from the allow list — which, because the
// free-domain suffix match is depth-independent, rejects every name beneath
// it as well as the bare one. Other addresses are unaffected.
func TestGetClusterAllowList_DedicatedGatewayAddressExcluded(t *testing.T) {
pm := &mockProxyManager{
getActiveClusterAddressesForAccountFunc: func(_ context.Context, _ string) ([]string, error) {
return []string{"brave-otter.gateway.example.com", "byop.example.com"}, nil
},
getActiveClusterAddressesFunc: func(_ context.Context) ([]string, error) {
return []string{"eu.proxy.netbird.io"}, nil
},
}
st := &stubStore{
getAgentNetworkSettingsFunc: func(_ context.Context, accountID string) (*agentnetworkTypes.Settings, error) {
assert.Equal(t, "acc-123", accountID,
"the exclusion must look up the requesting account's own settings")
return &agentnetworkTypes.Settings{
AccountID: accountID,
Domain: "brave-otter.gateway.example.com",
ProxyAddress: "brave-otter.gateway.example.com",
}, nil
},
}
mgr := Manager{store: st, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"byop.example.com", "eu.proxy.netbird.io"}, result,
"the dedicated gateway address must be reserved from cluster selection")
}
// TestGetClusterAllowList_LabeledPinDoesNotExclude pins the counterpart: a
// labeled pin means the gateway rides on a shared cluster serving ordinary
// services too, so nothing is reserved.
func TestGetClusterAllowList_LabeledPinDoesNotExclude(t *testing.T) {
pm := &mockProxyManager{
getActiveClusterAddressesForAccountFunc: func(_ context.Context, _ string) ([]string, error) {
return []string{"byop.example.com"}, nil
},
getActiveClusterAddressesFunc: func(_ context.Context) ([]string, error) {
return []string{"eu.proxy.netbird.io"}, nil
},
}
st := &stubStore{
getAgentNetworkSettingsFunc: func(_ context.Context, accountID string) (*agentnetworkTypes.Settings, error) {
return &agentnetworkTypes.Settings{
AccountID: accountID,
Domain: "violet.eu.proxy.netbird.io",
ProxyAddress: "eu.proxy.netbird.io",
}, nil
},
}
mgr := Manager{store: st, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.NoError(t, err)
assert.Equal(t, []string{"byop.example.com", "eu.proxy.netbird.io"}, result,
"a labeled pin reserves nothing")
}
// TestGetClusterAllowList_SettingsLookupError_ReturnsError pins that a store
// outage is surfaced rather than silently treated as "nothing reserved" —
// failing open here would offer a reserved gateway address for ordinary
// services.
func TestGetClusterAllowList_SettingsLookupError_ReturnsError(t *testing.T) {
pm := &mockProxyManager{
getActiveClusterAddressesForAccountFunc: func(_ context.Context, _ string) ([]string, error) {
return []string{"byop.example.com"}, nil
},
getActiveClusterAddressesFunc: func(_ context.Context) ([]string, error) {
return []string{"eu.proxy.netbird.io"}, nil
},
}
st := &stubStore{
getAgentNetworkSettingsFunc: func(_ context.Context, _ string) (*agentnetworkTypes.Settings, error) {
return nil, status.Errorf(status.Internal, "store outage")
},
}
mgr := Manager{store: st, proxyManager: pm}
result, err := mgr.getClusterAllowList(context.Background(), "acc-123")
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "agent network settings")
}

View File

@@ -0,0 +1,51 @@
package networkmapdb
import (
"context"
"github.com/netbirdio/netbird/management/server/integrations/integrated_validator"
"github.com/netbirdio/netbird/management/server/settings"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
type NetworkMapDBStore interface { //nolint:revive // established name across the codebase
BeginTx(ctx context.Context) (NetworkMapDBStoreConn, error)
}
type NetworkMapDBStoreConn interface { //nolint:revive // established name across the codebase
GetGroups(ctx context.Context, accountId string) ([]nmdata.Group, map[string]map[string]any, error)
GetDomains(ctx context.Context, accountId string) ([]Domain, error)
GetPeers(ctx context.Context, accountId string) ([]nmdata.Peer, map[string][]*nmdata.Peer, error)
GetPolicies(ctx context.Context, accountId string) ([]nmdata.Policy, map[string]map[string]any, map[string]map[string]any, error)
GetRoutes(ctx context.Context, accountId string) ([]nmdata.Route, error)
GetNameServerGroups(ctx context.Context, accountId string) ([]nmdata.NameServerGroup, error)
GetNetworkResources(ctx context.Context, accountId string) ([]nmdata.NetworkResource, error)
GetNetworkRouters(ctx context.Context, accountId string) (map[string]map[string]*nmdata.NetworkRouter, error)
GetNetwork(ctx context.Context, accountId string) (nmdata.Network, error)
GetAppliedZoneCandidates(ctx context.Context, accountId string) ([]networkmap.AppliedZoneCandidate, error)
GetAccountSettings(ctx context.Context, accountId string) (nmdata.AccountSettingsInfo, error)
GetPostureChecks(ctx context.Context, accountId string) ([]nmdata.PostureChecks, map[string]string, error)
GetAllowedUsers(ctx context.Context, accountId string) (map[string]struct{}, map[string][]string, error)
GetDnsSettings(ctx context.Context, accountId string) (nmdata.DNSSettings, error)
GetNetworkXIDToPublicIdMap(ctx context.Context, accountId string) (map[string]string, error)
GetPrivateServices(ctx context.Context, accountId string) ([]Service, error)
GetProxyTargetedDomainResourceIDs(ctx context.Context, accountId string) (map[string]struct{}, error)
CommitTx(ctx context.Context) error
RollbackTx(ctx context.Context) error
}
type NetworkMapDBStoreImpl struct { //nolint:revive // established name across the codebase
store NetworkMapDBStore
integratedPeerValidator integrated_validator.IntegratedValidator
extraSettingsManager settings.Manager
}
func NewNetworkMapDBStoreImpl(store NetworkMapDBStore, integratedPeerValidator integrated_validator.IntegratedValidator, extraSettingsManager settings.Manager) *NetworkMapDBStoreImpl {
return &NetworkMapDBStoreImpl{
store: store,
integratedPeerValidator: integratedPeerValidator,
extraSettingsManager: extraSettingsManager,
}
}

View File

@@ -0,0 +1,252 @@
package networkmapdb
import (
"context"
"fmt"
"strings"
"github.com/miekg/dns"
log "github.com/sirupsen/logrus"
"golang.org/x/exp/maps"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
func (s *NetworkMapDBStoreImpl) GetNetworkMapData(ctx context.Context, accountId string) (*networkmap.NetworkMapData, error) {
tx, err := s.store.BeginTx(ctx)
if err != nil {
return nil, err
}
acctSettings, err := tx.GetAccountSettings(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get account settings: %w", err))
}
dnsZones, err := tx.GetAppliedZoneCandidates(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get applied zone candidates: %w", err))
}
groups, resourceToGroupIdx, err := tx.GetGroups(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get groups: %w", err))
}
nsGroups, err := tx.GetNameServerGroups(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get nameserver groups: %w", err))
}
networkResources, err := tx.GetNetworkResources(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get network resources: %w", err))
}
routers, err := tx.GetNetworkRouters(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get network routers: %w", err))
}
network, err := tx.GetNetwork(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get network: %w", err))
}
peers, proxyPeers, err := tx.GetPeers(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get peers: %w", err))
}
policies, policyToDestinationResourceIdx, policyToDestinationGroupIdx, err := tx.GetPolicies(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get policies: %w", err))
}
postureChecks, postureCheckXIDToPublicID, err := tx.GetPostureChecks(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get posture checks: %w", err))
}
routes, err := tx.GetRoutes(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get routes: %w", err))
}
networkXIDToPublicID, err := tx.GetNetworkXIDToPublicIdMap(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get network xid to public id map: %w", err))
}
allowedUserIds, groupsToUserIds, err := tx.GetAllowedUsers(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get allowed users: %w", err))
}
dnsSettings, err := tx.GetDnsSettings(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get dns settings: %w", err))
}
domains, err := tx.GetDomains(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, err)
}
services, err := tx.GetPrivateServices(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, err)
}
proxyTargetedDomainResourceIDs, err := tx.GetProxyTargetedDomainResourceIDs(ctx, accountId)
if err != nil {
return rollbackAndReturnError(ctx, tx, fmt.Errorf("failed to get proxy targeted domain resources: %w", err))
}
resourcePolicies := make(map[string][]*nmdata.Policy)
for _, resource := range networkResources {
if !resource.Enabled {
continue
}
networkResourceGroups := resourceToGroupIdx[resource.ID]
for _, policy := range policies {
if !policy.Enabled {
continue
}
if _, ok := policyToDestinationResourceIdx[policy.ID][resource.ID]; ok {
resourcePolicies[resource.ID] = append(resourcePolicies[resource.ID], &policy) // TODO (dmitri) maybe use public id?
continue
}
if groupIds, ok := policyToDestinationGroupIdx[policy.ID]; ok {
for networkResourceGroup := range networkResourceGroups {
if _, ok := groupIds[networkResourceGroup]; ok {
resourcePolicies[resource.ID] = append(resourcePolicies[resource.ID], &policy)
break
}
}
}
}
}
if err = tx.CommitTx(ctx); err != nil {
log.WithContext(ctx).Warnf("failed to commit network map read transaction: %v", err)
}
toret := networkmap.NetworkMapData{
AccountSettings: &acctSettings,
DNSSettings: &dnsSettings,
Network: &network,
Peers: toMap(peers, func(p nmdata.Peer) string { return p.ID }),
Groups: toMap(groups, func(g nmdata.Group) string { return g.ID }),
Policies: toSliceOfPtrs(policies),
ResourcePolicies: resourcePolicies,
Routes: toSliceOfPtrs(routes),
Routers: routers,
NameServerGroups: toSliceOfPtrs(nsGroups),
NetworkResources: toSliceOfPtrs(networkResources),
PostureChecks: toMap(postureChecks, func(pc nmdata.PostureChecks) string { return pc.ID }),
AllowedUserIDs: allowedUserIds,
GroupIDToUserIDs: groupsToUserIds,
NetworkXIDToPublicID: networkXIDToPublicID, // TODO (dmitri) maybe we can switch to public ids everywhere?
AppliedZoneCandidates: dnsZones,
PrivateServiceCandidates: buildPrivateServiceCandidates(services, domains, proxyPeers),
PostureCheckXIDToPublicID: postureCheckXIDToPublicID,
ProxyTargetedDomainResourceIDs: proxyTargetedDomainResourceIDs,
}
extraSettings, err := s.extraSettingsManager.GetExtraSettings(ctx, accountId)
if err != nil {
return nil, err
}
toret.ValidatedPeers, err = s.integratedPeerValidator.GetValidatedPeers(ctx, accountId, maps.Values(toret.Groups), maps.Values(toret.Peers), extraSettings)
if err != nil {
return nil, err
}
return &toret, nil
}
func rollbackAndReturnError(ctx context.Context, tx NetworkMapDBStoreConn, err error) (*networkmap.NetworkMapData, error) {
if errr := tx.RollbackTx(ctx); errr != nil {
log.WithContext(ctx).Warnf("failed to rollback network map read transaction: %v", errr)
}
return nil, err
}
func toMap[T any](all []T, id func(t T) string) map[string]*T {
toret := make(map[string]*T, len(all))
for _, t := range all {
toret[id(t)] = &t
}
return toret
}
func toSliceOfPtrs[T any](all []T) []*T {
toret := make([]*T, 0, len(all))
for _, t := range all {
toret = append(toret, &t)
}
return toret
}
func serviceDomainZone(svc Service, ds []Domain) string {
if domainFromSuffix(svc.Domain.String, svc.ProxyCluster.String) {
return svc.ProxyCluster.String
}
var zoneName string
for _, domain := range ds {
if domain.TargetCluster.String != svc.ProxyCluster.String {
continue
}
if domainFromSuffix(svc.Domain.String, domain.Domain.String) && len(domain.Domain.String) > len(zoneName) {
zoneName = domain.Domain.String
}
}
return zoneName
}
func domainFromSuffix(domain, suffix string) bool {
if suffix == "" {
return false
}
return domain == suffix || strings.HasSuffix(domain, "."+suffix)
}
func buildPrivateServiceCandidates(svcs []Service, domains []Domain, proxyPeersByCluster map[string][]*nmdata.Peer) []networkmap.PrivateServiceCandidate {
var out []networkmap.PrivateServiceCandidate
if len(proxyPeersByCluster) == 0 {
return out
}
for _, svc := range svcs {
if !svc.Enabled.Bool || !svc.Private.Bool {
continue
}
if len(svc.AccessGroups) == 0 {
continue
}
domainZone := serviceDomainZone(svc, domains)
if domainZone == "" {
continue
}
var records []nmdata.SimpleRecord
for _, proxyPeer := range proxyPeersByCluster[svc.ProxyCluster.String] {
if !proxyPeer.IP.IsValid() {
continue
}
records = append(records, nmdata.SimpleRecord{
Name: dns.Fqdn(svc.Domain.String),
Type: int(dns.TypeA),
Class: "IN",
TTL: 5,
RData: proxyPeer.IP.String(),
})
}
if len(records) == 0 {
continue
}
out = append(out, networkmap.PrivateServiceCandidate{
AccessGroups: svc.AccessGroups,
Zone: nmdata.CustomZone{
Domain: dns.Fqdn(domainZone),
Records: records,
NonAuthoritative: true,
SearchDomainDisabled: true,
},
})
}
return out
}

View File

@@ -0,0 +1,61 @@
package networkmap_pgsql
import (
"context"
"encoding/json"
"time"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetAccountSettingsQuery = `
select settings_peer_login_expiration_enabled as peer_login_expiration_enabled,
settings_peer_login_expiration as peer_login_expiration,
settings_peer_inactivity_expiration_enabled as peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration as peer_inactivity_expiration,
settings_dns_domain as dns_domain,
settings_ipv6_enabled_groups as ipv6_enabled_groups,
settings_routing_peer_dns_resolution_enabled as routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled as lazy_connection_enabled,
settings_auto_update_version as auto_update_version,
settings_auto_update_always as auto_update_always,
settings_metrics_push_enabled as metrics_push_enabled
from accounts
where id=$1
`
)
func (pgc *PgStoreConn) GetAccountSettings(ctx context.Context, accountId string) (nmdata.AccountSettingsInfo, error) {
rows, err := pgc.Conn.Query(ctx, GetAccountSettingsQuery, accountId)
if err != nil {
return nmdata.AccountSettingsInfo{}, err
}
settings, err := pgx.CollectOneRow(rows, pgx.RowToStructByName[networkmapdb.Account])
if err != nil {
return nmdata.AccountSettingsInfo{}, err
}
settingsInfo := nmdata.AccountSettingsInfo{
PeerLoginExpirationEnabled: settings.PeerLoginExpirationEnabled.Bool,
PeerLoginExpiration: time.Duration(settings.PeerLoginExpiration.Int64),
PeerInactivityExpirationEnabled: settings.PeerInactivityExpirationEnabled.Bool,
PeerInactivityExpiration: time.Duration(settings.PeerInactivityExpiration.Int64),
DNSDomain: settings.DNSDomain.String,
RoutingPeerDNSResolutionEnabled: settings.RoutingPeerDNSResolutionEnabled.Bool,
LazyConnectionEnabled: settings.LazyConnectionEnabled.Bool,
AutoUpdateVersion: settings.AutoUpdateVersion.String,
AutoUpdateAlways: settings.AutoUpdateAlways.Bool,
MetricsPushEnabled: settings.MetricsPushEnabled.Bool,
}
if settings.IPv6EnabledGroups != nil {
if err := json.Unmarshal(settings.IPv6EnabledGroups, &settingsInfo.IPv6EnabledGroups); err != nil {
return nmdata.AccountSettingsInfo{}, err
}
}
return settingsInfo, nil
}

View File

@@ -0,0 +1,33 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap"
)
const (
GetAccountZonesQuery = `
select zones.id as id, domain, not enable_search_domain as search_domain_disabled, distribution_groups,
r.name as record_name, r.type as record_type, 'IN' record_class, r.ttl as record_ttl, r.content as record_rdata
from zones
left join records as r on r.zone_id = zones.id
where zones.account_id=$1
`
)
func (pgc *PgStoreConn) GetAppliedZoneCandidates(ctx context.Context, accountId string) ([]networkmap.AppliedZoneCandidate, error) {
rows, err := pgc.Conn.Query(ctx, GetAccountZonesQuery, accountId)
if err != nil {
return nil, err
}
zones, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Zone])
if err != nil {
return nil, err
}
return networkmapdb.ZonesToAppliedZoneCandidates(zones)
}

View File

@@ -0,0 +1,45 @@
package networkmap_pgsql
import (
"context"
"encoding/json"
"github.com/jackc/pgx/v5"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetDnsSettingsQuery = `
select dns_settings_disabled_management_groups
from accounts
where id=$1
`
)
func (pgc *PgStoreConn) GetDnsSettings(ctx context.Context, accountId string) (nmdata.DNSSettings, error) {
rows, err := pgc.Conn.Query(ctx, GetDnsSettingsQuery, accountId)
if err != nil {
return nmdata.DNSSettings{}, err
}
return pgx.CollectOneRow(rows, rowToDnsSettings)
}
func rowToDnsSettings(row pgx.CollectableRow) (nmdata.DNSSettings, error) {
var value nmdata.DNSSettings
var settings json.RawMessage
if err := row.Scan(&settings); err != nil {
return value, err
}
if settings == nil {
return nmdata.DNSSettings{}, nil
}
if err := json.Unmarshal(settings, &value.DisabledManagementGroups); err != nil {
return value, err
}
return value, nil
}

View File

@@ -0,0 +1,25 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
)
const (
GetDomainsQuery = `
select domain, target_cluster
from domains
where account_id=$1 and domain<>'' and target_cluster<>''
`
)
func (pgc *PgStoreConn) GetDomains(ctx context.Context, accountId string) ([]networkmapdb.Domain, error) {
rows, err := pgc.Conn.Query(ctx, GetDomainsQuery, accountId)
if err != nil {
return nil, err
}
return pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Domain])
}

View File

@@ -0,0 +1,64 @@
package networkmap_pgsql
import (
"context"
"database/sql"
"encoding/json"
"reflect"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetGroupsQuery = `
select id, name, public_id, resources,
(
select array_agg(group_peers.peer_id)
from group_peers
where group_peers.group_id = groups.id and group_peers.account_id=$1
) as peers
from groups where account_id=$1
`
)
// we also return a resource-to-group index.
// an alternative is to add json indexes, query this directly. Not sure how expensive
// json indexes are. TODO (dmitri) verify and maybe change the implementation here.
func (pgc *PgStoreConn) GetGroups(ctx context.Context, accountId string) ([]nmdata.Group, map[string]map[string]any, error) {
rows, err := pgc.Conn.Query(ctx, GetGroupsQuery, accountId)
if err != nil {
return nil, nil, err
}
groups, err := pgx.CollectRows(rows, pgx.RowToStructByName[group])
toret := make([]nmdata.Group, 0, len(groups))
resourceToGroupIdx := make(map[string]map[string]any)
for _, g := range groups {
dg := nmdata.Group{}
err := networkmapdb.FromSqlTypesToSharedTypes(
reflect.ValueOf(&g), reflect.ValueOf(&dg))
if err != nil {
return nil, nil, err
}
toret = append(toret, dg)
for _, resource := range dg.Resources {
if _, ok := resourceToGroupIdx[resource.ID]; !ok {
resourceToGroupIdx[resource.ID] = make(map[string]any)
}
resourceToGroupIdx[resource.ID][g.ID] = struct{}{}
}
}
return toret, resourceToGroupIdx, err
}
type group struct {
ID string
Name sql.NullString
PublicID sql.NullString
Resources json.RawMessage
Peers []string
}

View File

@@ -0,0 +1,31 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetNameserversQuery = `
select id, public_id, name, description, name_servers, groups, "primary", domains, enabled, search_domains_enabled
from name_server_groups
where account_id=$1
`
)
func (pgc *PgStoreConn) GetNameServerGroups(ctx context.Context, accountId string) ([]nmdata.NameServerGroup, error) {
rows, err := pgc.Conn.Query(ctx, GetNameserversQuery, accountId)
if err != nil {
return nil, err
}
nsgroups, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.NameserverGroup])
if err != nil {
return nil, err
}
return networkmapdb.ConvertAllToSharedTypes[networkmapdb.NameserverGroup, nmdata.NameServerGroup](nsgroups)
}

View File

@@ -0,0 +1,39 @@
package networkmap_pgsql
import (
"context"
"reflect"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetNetworkQuery = `
select network_identifier as identifier, network_net as net, network_net_v6 as net_v6, network_dns as dns, network_serial as serial
from accounts
where id=$1
`
)
func (pgc *PgStoreConn) GetNetwork(ctx context.Context, accountId string) (nmdata.Network, error) {
rows, err := pgc.Conn.Query(ctx, GetNetworkQuery, accountId)
if err != nil {
return nmdata.Network{}, err
}
n, err := pgx.CollectOneRow(rows, pgx.RowToStructByName[networkmapdb.AccountNetwork])
if err != nil {
return nmdata.Network{}, err
}
toret := nmdata.Network{}
err = networkmapdb.FromSqlTypesToSharedTypes(
reflect.ValueOf(&n), reflect.ValueOf(&toret))
if err != nil {
return nmdata.Network{}, err
}
return toret, nil
}

View File

@@ -0,0 +1,31 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetNetworkResourcesQuery = `
select id, network_id, account_id, public_id, name, description, type, domain, prefix, enabled
from network_resources
where account_id=$1
`
)
func (pgc *PgStoreConn) GetNetworkResources(ctx context.Context, accountId string) ([]nmdata.NetworkResource, error) {
rows, err := pgc.Conn.Query(ctx, GetNetworkResourcesQuery, accountId)
if err != nil {
return nil, err
}
netresorces, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Networkresource])
if err != nil {
return nil, err
}
return networkmapdb.ConvertAllToSharedTypes[networkmapdb.Networkresource, nmdata.NetworkResource](netresorces)
}

View File

@@ -0,0 +1,80 @@
package networkmap_pgsql
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"reflect"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetNetworkRouterQuery = `
select public_id, peer, network_id, masquerade, metric, enabled, peer_groups,
(
select array_agg(group_peers.peer_id)
from group_peers
where group_peers.account_id=$1 and group_peers.group_id in (select json_array_elements_text(peer_groups::json))
) as peers_via_groups
from network_routers
where account_id=$1
`
)
func (pgc *PgStoreConn) GetNetworkRouters(ctx context.Context, accountId string) (map[string]map[string]*nmdata.NetworkRouter, error) {
rows, err := pgc.Conn.Query(ctx, GetNetworkRouterQuery, accountId)
if err != nil {
return nil, err
}
routers, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkrouter])
if err != nil {
return nil, err
}
toret := make(map[string]map[string]*nmdata.NetworkRouter)
for _, router := range routers {
if !router.Enabled.Bool {
continue
}
networkId := router.NetworkID.String
if networkId == "" {
return nil, fmt.Errorf("router with public_id %s doesn't have network_id set", router.PublicID.String)
}
nmdatarouter := nmdata.NetworkRouter{}
err := networkmapdb.FromSqlTypesToSharedTypes(reflect.ValueOf(&router), reflect.ValueOf(&nmdatarouter))
if err != nil {
return nil, err
}
if toret[networkId] == nil {
toret[networkId] = make(map[string]*nmdata.NetworkRouter)
}
if router.Peer.String != "" {
toret[networkId][router.Peer.String] = &nmdatarouter
continue
}
for _, peerId := range router.PeersViaGroups {
toret[networkId][peerId] = &nmdatarouter
}
}
return toret, nil
}
type networkrouter struct {
PublicID sql.NullString
NetworkID sql.NullString `nmap:"skip"`
Peer sql.NullString `nmap:"skip"`
PeerGroups json.RawMessage
PeersViaGroups []string `nmap:"skip"`
Masquerade sql.NullBool
Metric sql.NullInt64
Enabled sql.NullBool
}

View File

@@ -0,0 +1,36 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
)
const (
GetNetworksQuery = `
select id, public_id
from networks where account_id=$1
`
)
func (pgc *PgStoreConn) GetNetworkXIDToPublicIdMap(ctx context.Context, accountId string) (map[string]string, error) {
rows, err := pgc.Conn.Query(ctx, GetNetworksQuery, accountId)
if err != nil {
return nil, err
}
networks, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Network])
if err != nil {
return nil, err
}
toret := make(map[string]string)
for _, n := range networks {
if n.PublicID.Valid {
toret[n.ID] = n.PublicID.String
}
}
return toret, nil
}

View File

@@ -0,0 +1,34 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetPeersQuery = `
select id, key, ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files, meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip
from peers
where account_id = $1
`
)
func (pgc *PgStoreConn) GetPeers(ctx context.Context, accountId string) ([]nmdata.Peer, map[string][]*nmdata.Peer, error) {
rows, err := pgc.Conn.Query(ctx, GetPeersQuery, accountId)
if err != nil {
return nil, nil, err
}
peers, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Peer])
if err != nil {
return nil, nil, err
}
return networkmapdb.ConvertToNmdataPeers(peers)
}

View File

@@ -0,0 +1,98 @@
package networkmap_pgsql
import (
"context"
"fmt"
"reflect"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
)
const (
pgMaxConnections = 30
pgMinConnections = 1
pgMaxConnLifetime = 60 * time.Minute
pgHealthCheckPeriod = 1 * time.Minute
)
var _ networkmapdb.NetworkMapDBStore = &PgStore{}
type PgStore struct {
Pool *pgxpool.Pool
}
type PgStoreConn struct {
Conn pgInterface
}
type pgInterface interface {
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
Exec(ctx context.Context, sql string, arguments ...any) (pgconn.CommandTag, error)
}
var _ networkmapdb.NetworkMapDBStoreConn = &PgStoreConn{}
func NewPostgresqlStore(ctx context.Context, dsn string) (*PgStore, error) {
pool, err := connectToPgDb(ctx, dsn)
if err != nil {
return nil, err
}
return &PgStore{Pool: pool}, nil
}
func (p *PgStore) UsingConnection(c *pgx.Conn) networkmapdb.NetworkMapDBStoreConn {
return &PgStoreConn{Conn: c}
}
func (p *PgStore) BeginTx(ctx context.Context) (networkmapdb.NetworkMapDBStoreConn, error) {
tx, err := p.Pool.BeginTx(ctx, pgx.TxOptions{IsoLevel: pgx.RepeatableRead, AccessMode: pgx.ReadOnly})
if err != nil {
return nil, err
}
return &PgStoreConn{Conn: tx}, nil
}
func (c *PgStoreConn) RollbackTx(ctx context.Context) error {
tx, ok := c.Conn.(pgx.Tx)
if !ok {
return fmt.Errorf("expected an pgx.Tx got %s", reflect.TypeOf(c.Conn).Kind())
}
return tx.Rollback(ctx)
}
func (c *PgStoreConn) CommitTx(ctx context.Context) error {
tx, ok := c.Conn.(pgx.Tx)
if !ok {
return fmt.Errorf("expected an sql.Tx got %s", reflect.TypeOf(c.Conn).Kind())
}
return tx.Commit(ctx)
}
func connectToPgDb(ctx context.Context, dsn string) (*pgxpool.Pool, error) {
config, err := pgxpool.ParseConfig(dsn)
if err != nil {
return nil, fmt.Errorf("unable to parse database config: %w", err)
}
config.MaxConns = pgMaxConnections
config.MinConns = pgMinConnections
config.MaxConnLifetime = pgMaxConnLifetime
config.HealthCheckPeriod = pgHealthCheckPeriod
pool, err := pgxpool.NewWithConfig(ctx, config)
if err != nil {
return nil, fmt.Errorf("unable to create connection pool: %w", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
return nil, fmt.Errorf("unable to ping database: %w", err)
}
return pool, nil
}

View File

@@ -0,0 +1,34 @@
package networkmap_pgsql
import (
"context"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetPoliciesQuery = `
select p.id, p.public_id, p.enabled, p.source_posture_checks, pr.enabled as rule_enabled, pr.action, pr.protocol, pr.bidirectional,
pr.sources, pr.destinations, pr.source_resource, pr.destination_resource, pr.ports, pr.port_ranges,
pr.authorized_groups, pr.authorized_user
from policies as p
left join policy_rules as pr on p.id = pr.policy_id
where account_id=$1
`
)
func (pgc *PgStoreConn) GetPolicies(ctx context.Context, accountId string) ([]nmdata.Policy, map[string]map[string]any, map[string]map[string]any, error) {
rows, err := pgc.Conn.Query(ctx, GetPoliciesQuery, accountId)
if err != nil {
return nil, nil, nil, err
}
policies, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.Policy])
if err != nil {
return nil, nil, nil, err
}
return networkmapdb.ConvertToNmdataPolicy(policies)
}

View File

@@ -0,0 +1,44 @@
package networkmap_pgsql
import (
"context"
"reflect"
"github.com/jackc/pgx/v5"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
const (
GetPostureChecksQuery = `
select id, public_id, checks
from posture_checks
where account_id=$1
`
)
func (pgc *PgStoreConn) GetPostureChecks(ctx context.Context, accountId string) ([]nmdata.PostureChecks, map[string]string, error) {
rows, err := pgc.Conn.Query(ctx, GetPostureChecksQuery, accountId)
if err != nil {
return nil, nil, err
}
checks, err := pgx.CollectRows(rows, pgx.RowToStructByName[networkmapdb.PostureChecks])
if err != nil {
return nil, nil, err
}
toret := make([]nmdata.PostureChecks, 0, len(checks))
idToPublicIDIdx := make(map[string]string)
for _, c := range checks {
checks := nmdata.PostureChecks{}
err := networkmapdb.FromSqlTypesToSharedTypes(reflect.ValueOf(&c), reflect.ValueOf(&checks))
if err != nil {
return nil, nil, err
}
toret = append(toret, checks)
idToPublicIDIdx[checks.ID] = c.PublicID.String
}
return toret, idToPublicIDIdx, nil
}

Some files were not shown because too many files have changed in this diff Show More