Compare commits

...

161 Commits

Author SHA1 Message Date
Dmitri Dolguikh
a803ad398d Merge remote-tracking branch 'origin/main' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-17 11:10:51 +02:00
Dmitri Dolguikh
6ee0987cba updated GetNetworkMapData test to verify all calls -- policies, routes, etc
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-17 11:10:31 +02:00
Dmitri Dolguikh
6fdde1385f switch to mocks in network_map_data test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-17 10:17:50 +02:00
Misha Bragin
4e5b632490 [infrastructure] Don't override the dashboard image on enterprise migration (#7206) 2026-08-16 16:40:20 +02:00
Misha Bragin
93e97f4bf1 [doc] Agent network docs update (#7020)
* [docs] Update agent-network docs for management-owned pricing

  The docs still described the retired proxy-side pricing: pricing.Loader,
  pricing_path, MiddlewareDataDir, embedded defaults_pricing.yaml, and the
  symlink-safe Unix loader. Rewrite them for the current design — management
  synthesizes the whole table and ships it in cost_meter's ConfigJSON, so the
  proxy carries no price list and has nothing to reload.
2026-08-15 19:31:49 +02:00
Zoltan Papp
16544dbc58 [client] Pass stored email as login hint from UI and keep it on logout (#7199)
* [client] Pass stored email as login hint from UI and keep it on logout

Follow the CLI pattern: the Wails UI now reads the account email from the
user-owned profile state file and passes it as the OIDC login_hint on login
and session extend, since the daemon-side fallback runs as root and cannot
see the user's state file. Logout no longer deletes the stored email, so a
later login preselects the account at the IdP; profile removal remains the
operation that deletes it.

* [client] Log ignored profile lookup errors in extend-session hint fallback
2026-08-15 11:21:57 +02:00
Zoltan Papp
f458c1f265 [client] Skip IPv6 route tests when the default nexthop is unusable (#7212)
* [client] Skip IPv6 route tests when the default nexthop is unusable

ensureIPv6DefaultRoute treated a successful netlink RouteAdd as proof that
a usable IPv6 nexthop exists. Installing ::/0 via loopback can succeed while
the kernel still rejects that nexthop for a concrete prefix, which surfaced
on ubuntu22/20260810.260 runners as:

    add route to table: netlink add route: invalid argument

Probe the resolved nexthop by installing and removing a discard-prefix route
through the same code path the tests use, and skip when it fails. EEXIST
means the nexthop already carries a route, so it counts as usable.

* [client] Probe the IPv6 nexthop through raw netlink

addRoute swallows EAFNOSUPPORT and EOPNOTSUPP via isOpErr, so a nil return
did not prove the probe route was installed. Call netlink directly so an
unsupported operation skips the test instead of passing as usable.
2026-08-15 10:13:06 +02:00
Viktor Liu
ec6f1b8c27 [client] Rank Windows route candidates by combined route and interface metric (#7210) 2026-08-15 09:06:22 +02:00
Dmitri Dolguikh
aafc233310 go mod tidy
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-14 17:30:11 +02:00
Dmitri Dolguikh
7bc38d9b85 fix linter issue
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-14 16:54:15 +02:00
Dmitri Dolguikh
84e4b08056 added tests to cover GetNetworkMapData() call
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-14 16:45:33 +02:00
Dmitri Dolguikh
14f976bade bump pgx to v5.10.0
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-14 16:44:26 +02:00
Dmitri Dolguikh
fdb956a974 Merge remote-tracking branch 'origin/main' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 16:30:08 +02:00
Dmitri Dolguikh
c6db3ad575 removed unused func
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 16:28:21 +02:00
Dmitri Dolguikh
cb4460d1d9 fix error name
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 16:11:31 +02:00
Dmitri Dolguikh
c004d09d77 fix a linter issue
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 15:45:30 +02:00
Dmitri Dolguikh
9fed4ce414 ping mage-action to sha of v4.0.0 tag
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 13:24:57 +02:00
Dmitri Dolguikh
4aa123b6ad wired up sqlite store for use in networkmap controller
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-13 13:16:23 +02:00
pascal
931598e593 fix authorized user groups for ssh + fix ignoring disabled policies + fix ignore invalid router 2026-08-12 17:43:43 +02:00
pascal
9bb1db28c3 Merge branch 'revert/component-types-test-suite' into revert/component-types 2026-08-12 16:55:06 +02:00
Dmitri Dolguikh
4952f2e8cf remove 'mage' tag from mage files
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-12 11:57:36 +02:00
Dmitri Dolguikh
32c7918c80 go mod tidy
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-12 11:50:31 +02:00
Dmitri Dolguikh
56d1607f72 added mage + targets for integration tests + updated github workflow
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-12 11:32:15 +02:00
pascal
460778abb9 add test harness and first tests 2026-08-11 17:34:43 +02:00
Dmitri Dolguikh
4be6603815 added comments re: ordering of fields in intermediate DTOs
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 16:47:39 +02:00
Dmitri Dolguikh
05511cc13d cleanup sqlite store creation
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 16:05:59 +02:00
Dmitri Dolguikh
9427fa87e1 moved GetNetworkMapData implementation to NetworkMapDBStoreImpl
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 15:42:11 +02:00
Dmitri Dolguikh
fe5e5c08eb Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:52:14 +02:00
Dmitri Dolguikh
cb858b335d moved tests out of pgsql dir
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:48:31 +02:00
Dmitri Dolguikh
8cbbea4536 support for GetAllowedUsers in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:46:59 +02:00
Dmitri Dolguikh
cd6f63272b support for GetPrivateServices in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 14:25:39 +02:00
Dmitri Dolguikh
859af13c12 support for GetRoutes in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:49:36 +02:00
Dmitri Dolguikh
4375fdc7b6 support for GetPostureChecks in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:42:14 +02:00
Dmitri Dolguikh
1caa0ddf27 support for GetPolicies in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 13:36:15 +02:00
pascal
cd2a91ddd8 fir error handling and return values (linter complaint) 2026-08-11 12:45:25 +02:00
pascal
38c5932375 merge main 2026-08-11 12:01:51 +02:00
Dmitri Dolguikh
fe9ea43198 support for GetPeers in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 11:09:34 +02:00
Dmitri Dolguikh
fad568fdb0 support for GetNetworkXIDToPublicIdMap in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 10:51:34 +02:00
Dmitri Dolguikh
3da27221ac support for GetNetwork in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-11 10:43:00 +02:00
Dmitri Dolguikh
d954a2dc3e support for GetNetworkRouters in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 20:10:41 +02:00
Dmitri Dolguikh
433a4f12bf added support for GetNetworkResources in sqlite; moved several more internal types into shared_types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 18:07:31 +02:00
Dmitri Dolguikh
3834e67a51 add support for GetNameServerGroups in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 17:53:34 +02:00
Dmitri Dolguikh
78947c1611 support for GetGroups in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 17:14:37 +02:00
Dmitri Dolguikh
763b8f6933 support for GetDomains in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 15:55:24 +02:00
Dmitri Dolguikh
e530c25812 support for GetAppliedZoneCandidates in sqlite
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:59:03 +02:00
Dmitri Dolguikh
166b3d7739 moved test for RecordTypeAndRdata into the parent package
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:21:19 +02:00
Dmitri Dolguikh
20f18d1479 extracted struct-handling helpers into their own file; move sql_type_conversion_test to the parent module
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:15:11 +02:00
Dmitri Dolguikh
5594289924 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:08:23 +02:00
Dmitri Dolguikh
a03635680b support for GetDnsSettings in sqlite store
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 14:07:52 +02:00
pascal
905e1c14ba add validated peers cache for nmdata 2026-08-10 14:02:06 +02:00
Dmitri Dolguikh
5e4d6a5e44 run tests against postgres by default (for now)
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 13:39:25 +02:00
Dmitri Dolguikh
65f184141b added support for GetAccountSettings to sqlite store
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-10 13:31:55 +02:00
Dmitri Dolguikh
c54c35beed disable tests when running with sqlite 2026-08-10 10:02:46 +02:00
Dmitri Dolguikh
935d1c5863 move read-only queries to an interface to reuse in tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-07 19:23:52 +02:00
Dmitri Dolguikh
ac3b713dda Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-07 09:56:15 +02:00
Dmitri Dolguikh
4d4de15f48 added 'integration' build tag to nmap db tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 16:14:51 +02:00
pascal
28074405e1 change wording from failer -> failing to pass codespell 2026-08-06 16:02:02 +02:00
pascal
fd23461b03 add network map compute test 2026-08-06 15:57:43 +02:00
Dmitri Dolguikh
b19cf7405d Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 15:18:13 +02:00
Dmitri Dolguikh
90e0c5bd0c added GetPolicies test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 15:17:39 +02:00
pascal
6ccb2bb239 handle nil values in policy, nameserver and resources 2026-08-06 15:12:12 +02:00
Dmitri Dolguikh
b160cb43b1 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 13:03:28 +02:00
Dmitri Dolguikh
1902777509 added a test for GetPeers
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-06 13:03:00 +02:00
pascal
cff46eee67 introduce peerGroupsIdx to avoid looping through the groups to figure out peer membership 2026-08-06 12:49:36 +02:00
pascal
0513109c35 improve looping on connected peers filtering 2026-08-06 12:31:26 +02:00
Dmitri Dolguikh
6b8393c6b0 add GetAppliedZoneCandidatesViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 18:43:20 +02:00
Dmitri Dolguikh
316e82337f cleanup query execution in tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 18:09:28 +02:00
Dmitri Dolguikh
bacacb8f77 deleted group_test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:45:25 +02:00
Dmitri Dolguikh
994e84a686 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 16:35:08 +02:00
Dmitri Dolguikh
07809fe923 added test for GetAllowedUsersViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:34:44 +02:00
pascal
ec3e37456e use Exec instead of Query for inserts and assert every insert error 2026-08-05 16:33:48 +02:00
pascal
c81103dfa6 fix linter comments 2026-08-05 16:21:01 +02:00
Dmitri Dolguikh
ba574dc739 added tests for GetPrivateServicesViaPgxConnection and GetPrivateServicesViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 16:06:43 +02:00
Dmitri Dolguikh
067982f772 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 15:26:38 +02:00
Dmitri Dolguikh
ce4f0a7821 added GetAccountSettingsViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 15:26:06 +02:00
pascal
3218e2f744 fix client tests 2026-08-05 15:18:19 +02:00
pascal
7e24c27ef1 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 15:09:41 +02:00
pascal
06817271bf fix embedded client test 2026-08-05 15:09:29 +02:00
Dmitri Dolguikh
1530739b54 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:58:36 +02:00
Dmitri Dolguikh
e745aedc92 added GetRoutesViaPgxConnection test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:58:09 +02:00
pascal
4cf3903c83 fix management <-> shared dependencies 2026-08-05 14:48:27 +02:00
pascal
1e14b554a6 fix management <-> shared dependencies 2026-08-05 14:48:18 +02:00
Dmitri Dolguikh
11a3627899 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:29:43 +02:00
Dmitri Dolguikh
2442bd6198 tests for GetNameServerGroupsViaPgxConnection and GetPostureChecksViaPgxConnection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 14:29:07 +02:00
pascal
006cee000f Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 13:56:46 +02:00
pascal
c93aa03c0e merge main 2026-08-05 13:56:35 +02:00
Dmitri Dolguikh
efe2eaeb09 added GetDomains test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 12:37:58 +02:00
Dmitri Dolguikh
1926e983fb added GetDnsSettings test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 12:03:36 +02:00
Dmitri Dolguikh
e9b8175915 added GetNetworks test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 11:53:40 +02:00
Dmitri Dolguikh
88f1930450 added account network test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 11:40:23 +02:00
Dmitri Dolguikh
1774a3d9dd add network resource query test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 10:52:39 +02:00
Dmitri Dolguikh
795e06ce83 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types 2026-08-05 10:16:54 +02:00
Dmitri Dolguikh
748f6b3fbb fixes + tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-05 10:16:21 +02:00
pascal
d5adfc799f handle policy with no rules to avoid index panic 2026-08-04 20:57:57 +02:00
pascal
60fe56e442 fix assert equal ordering 2026-08-04 20:56:16 +02:00
pascal
327fa4c0d6 fix group clone 2026-08-04 20:52:16 +02:00
pascal
ed015a7972 skip resource from proto failure 2026-08-04 20:46:10 +02:00
pascal
b43708e31b silently skip resource to proto failure 2026-08-04 20:42:43 +02:00
pascal
ce55b8406b handle not found peer on posture vali 2026-08-04 20:34:30 +02:00
pascal
0456b64ea1 silent skip on non existing groups references 2026-08-04 20:28:06 +02:00
pascal
942ee81ec0 add benchmark 2026-08-04 20:19:17 +02:00
Dmitri Dolguikh
33a0e1bc2b adding integration tests
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 18:58:29 +02:00
Dmitri Dolguikh
dbfdd04c7b adding network-routers integration query test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 16:43:17 +02:00
Dmitri Dolguikh
4d010f60ce fix another import
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 15:40:28 +02:00
Dmitri Dolguikh
3272058e56 fix extra setting manager package
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 13:34:25 +02:00
Dmitri Dolguikh
70c3feb05b Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 13:04:51 +02:00
Dmitri Dolguikh
a0fe80cd99 wire up validated peers
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-04 12:52:09 +02:00
pascal
6fadf8f24a fix nmdata store expanding router peer groups alongside static peer 2026-08-03 23:58:47 +02:00
pascal
ef0032685b fix nmdata store shipping zones for disabled/public services 2026-08-03 23:54:22 +02:00
pascal
5ed38569f7 fix PrivateServiceCandidates nil assumption 2026-08-03 23:19:55 +02:00
pascal
d4e1c8978e add all group to user group lookup 2026-08-03 23:08:43 +02:00
pascal
d5ac70d806 fix missing ForceRoutingPeerDNSResolution on the nmdata store path 2026-08-03 22:50:06 +02:00
pascal
23579e4dd4 Skip the left-join NULL row instead of failing the account load + drop unsupported record types silently to match buildAppliedZoneCandidates 2026-08-03 22:38:12 +02:00
pascal
a00c5164a8 continue on resource policy loop + exclude disabled policies form index 2026-08-03 22:29:47 +02:00
pascal
02138dffdd fix group id mapping 2026-08-03 22:24:28 +02:00
pascal
02ec1f5dcb fix equivalence test 2026-08-03 22:24:11 +02:00
pascal
07d0440e34 fix slice initialization to avoid nil pointer dereference 2026-08-03 21:51:48 +02:00
pascal
2c85d94c6c hookup validated peers 2026-08-03 21:47:41 +02:00
Dmitri Dolguikh
1f190a50cf Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:22:34 +02:00
Dmitri Dolguikh
52fdfd5bdc add an posture-check-id to public-id index
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:21:07 +02:00
Dmitri Dolguikh
e35a0f3318 added PrivateServiceCandidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 18:04:11 +02:00
pascal
56c411babd fixed networks query and extended error wrapping 2026-08-03 17:52:33 +02:00
Dmitri Dolguikh
164b2baa4d Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 17:29:42 +02:00
Dmitri Dolguikh
0b29c6ed1a adding buildPrivateServiceCandidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 17:29:09 +02:00
pascal
724b61440b use new path on initial sync 2026-08-03 17:27:58 +02:00
pascal
22590ad66e hookup network map store 2026-08-03 17:27:24 +02:00
Dmitri Dolguikh
b8e004ea89 also build proxy-cluster to peer idx
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:58:53 +02:00
Dmitri Dolguikh
e2797360f4 support for applied zone candidates
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:32:54 +02:00
Dmitri Dolguikh
2f399f1e6e wire up dnssettings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 14:12:27 +02:00
Dmitri Dolguikh
82c1e18264 renamed allowed_user_ids.go to user.got 2026-08-03 13:54:22 +02:00
Dmitri Dolguikh
ce9023bd27 wire up accountsettings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 13:51:25 +02:00
Dmitri Dolguikh
7d33356776 support for group to user ids
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 13:47:14 +02:00
Dmitri Dolguikh
993291149c Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 12:20:18 +02:00
Dmitri Dolguikh
98f7ea40a1 added allowed_user_ids call
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 12:19:50 +02:00
pascal
76db9ab94f update integrated validator 2026-08-03 12:11:35 +02:00
Dmitri Dolguikh
9653b15d78 automatically fill empty public ids for view types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 11:34:22 +02:00
Dmitri Dolguikh
206bb1676b build resourcePolicies map
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-03 10:53:54 +02:00
Dmitri Dolguikh
600b0c752b also return a net resource to groups index from GetGroups call
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-01 11:56:17 +02:00
Dmitri Dolguikh
b02736adc3 cleanup network routers retrieval
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-08-01 11:54:14 +02:00
Dmitri Dolguikh
5d6117d2c0 adding network_map_data: compute Routers field
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 17:56:52 +02:00
Dmitri Dolguikh
c2f8360b00 adding support for bulding of network_map_data
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 15:58:40 +02:00
Dmitri Dolguikh
ea1b4d56e8 added support for queries via pgx connection
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 15:03:15 +02:00
Dmitri Dolguikh
acbe22b831 small cleanups
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 13:41:34 +02:00
Dmitri Dolguikh
3b5c8e2298 started moving pg-specific nmap tests to integration_tests/management/network_map_db/pgsql
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-31 12:29:35 +02:00
Dmitri Dolguikh
2baeb4bc0d clean up sql types conversion test
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 19:35:11 +02:00
Dmitri Dolguikh
4bf75fdc97 support for posture-checks
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 18:00:58 +02:00
Dmitri Dolguikh
799f3a3c62 Merge remote-tracking branch 'origin/revert/component-types' into revert/component-types
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 16:03:46 +02:00
Dmitri Dolguikh
129736ad61 support for account settings
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 16:03:19 +02:00
pascal
f7be9c4347 legacy network mal and equivalence test 2026-07-30 15:45:15 +02:00
Dmitri Dolguikh
e05cb5264d support for dns custom zones
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 15:34:09 +02:00
Dmitri Dolguikh
5a10561ca1 fix handling of string slices
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-30 11:31:53 +02:00
Dmitri Dolguikh
42ce83a8f3 added support for account network
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:57:08 +02:00
Dmitri Dolguikh
e620c86cd4 added support for networkrouters
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:32:51 +02:00
Dmitri Dolguikh
9dee2d60b9 added support for networkresources
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 18:19:34 +02:00
Dmitri Dolguikh
4525014632 support for nameservergroups
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 17:59:44 +02:00
Dmitri Dolguikh
23a5c0de4b added support for routes
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-29 17:37:00 +02:00
Dmitri Dolguikh
25e882004f added retrieval of policies
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-28 18:06:28 +02:00
Dmitri Dolguikh
15003258d2 networkmap read-only interface for pgsql
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-28 16:46:45 +02:00
Dmitri Dolguikh
2af3a5fba5 do not send resource policies map over the wire
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
2026-07-27 19:00:30 +02:00
pascal
a6603a2e0a have explicit network map data type and do calculation from there 2026-07-27 15:17:30 +02:00
pascal
7ed3737cda revert component types 2026-07-24 15:16:55 +02:00
335 changed files with 23441 additions and 3852 deletions

View File

@@ -730,6 +730,11 @@ jobs:
- name: Install modules
run: go mod tidy
- name: Run Mage
uses: magefile/mage-action@a662bd8c29d8106879588cfff83b2faf6e6f59db # v4.0.0
with:
install-only: true
- name: check git status
run: git --no-pager diff --exit-code
@@ -738,9 +743,7 @@ jobs:
CGO_ENABLED=1 GOARCH=${{ matrix.arch }} \
NETBIRD_STORE_ENGINE=${{ matrix.store }} \
CI=true \
go test -tags=integration -coverprofile=coverage.txt \
-exec 'sudo --preserve-env=CI,NETBIRD_STORE_ENGINE' \
-timeout 20m ./management/server/http/...
mage integrationtest:all -gotestflags="-coverprofile=coverage.txt"
- name: Upload coverage reports to Codecov
if: matrix.arch == 'amd64'

View File

@@ -1,6 +1,6 @@
# NetBird Agent Guidelines
**NetBird** is an open-source connectivity platform: a WireGuard®-based overlay
**NetBird** is an open source connectivity platform: a WireGuard®-based overlay
network with a control plane. The **agent** (`client/`) runs on user machines as
a privileged daemon and manages the WireGuard interface, routing, firewall, and
DNS. **Management** (`management/`) is the control plane and REST/gRPC API,

View File

@@ -479,7 +479,7 @@ go test -race ./client/internal/dns/...
## Checklist before submitting a PR
As a critical network service and open-source project, we must enforce a few
As a critical network service and open source project, we must enforce a few
things before submitting a pull request. The
[pull request template](/.github/pull_request_template.md) mirrors this list —
fill it in rather than deleting it.

View File

@@ -130,7 +130,7 @@ In November 2022, NetBird joined the [StartUpSecure program](https://www.forschu
![CISPA_Logo_BLACK_EN_RZ_RGB (1)](https://user-images.githubusercontent.com/700848/203091324-c6d311a0-22b5-4b05-a288-91cbc6cdcc46.png)
### Acknowledgements
We build on open-source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
We build on open source technologies like [WireGuard®](https://www.wireguard.com/), [Pion ICE](https://github.com/pion/ice), and [Rosenpass](https://rosenpass.eu). We greatly appreciate the work these projects are doing, and we'd love it if you could support them too (e.g., by starring or contributing).
### Legal
This repository is licensed under the BSD-3-Clause license, which applies to all parts of the repository except for the directories management/, signal/ and relay/.

View File

@@ -14,7 +14,7 @@ Report security issues one of these two ways:
on this repository. This is the preferred route: it keeps the discussion, the draft advisory, and the credit in one place.
- **Email** — `security@netbird.io`.
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open-source code, email us rather than
If the finding affects NetBird Cloud or our hosted infrastructure rather than the open source code, email us rather than
filing a repository report.
### What to include

View File

@@ -124,7 +124,7 @@ func startManagement(t *testing.T, config *config.Config, testFile string) (*grp
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := mgmt.NewAccountRequestBuffer(ctx, store)
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config)
networkMapController := controller.NewController(ctx, store, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.cloud", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersmanager), config, nil)
accountManager, err := mgmt.BuildManager(ctx, config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
if err != nil {

View File

@@ -146,7 +146,7 @@ func startManagement(t *testing.T, signalAddr string) string {
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := mgmt.NewAccountRequestBuffer(context.Background(), testStore)
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg)
networkMapController := controller.NewController(context.Background(), testStore, metrics, updateManager, requestBuffer, mgmt.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(testStore, peersManager), cfg, nil)
accountManager, err := mgmt.BuildManager(context.Background(), cfg, testStore, networkMapController, jobManager, nil, "", eventStore, nil, false, iv, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
require.NoError(t, err)

View File

@@ -519,7 +519,7 @@ func startManagement(t *testing.T, dataDir, testFile string) (*grpc.Server, stri
updateManager := update_channel.NewPeersUpdateManager(metrics)
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
networkMapController := controller.NewController(context.Background(), store, metrics, updateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManager, false, cacheStore)
if err != nil {
return nil, "", err

View File

@@ -87,9 +87,10 @@ func (pm *ProfileManager) SetActiveProfileState(state *ProfileState) error {
// RemoveProfileState deletes the per-profile state file (which holds the
// account email used for the SSO login hint and the UI display). Called after
// a successful logout so a logged-out profile no longer shows a stale account
// email. The state file only stores the email, so deleting it is equivalent to
// clearing it; the next SSO login recreates it. A missing file is not an error.
// profile removal; logout keeps the file so the next login can pass the email
// as the login_hint. The state file only stores the email, so deleting it is
// equivalent to clearing it; the next SSO login recreates it. A missing file
// is not an error.
func (pm *ProfileManager) RemoveProfileState(profileName string) error {
configDir, err := getConfigDir()
if err != nil {

View File

@@ -0,0 +1,82 @@
//go:build windows
package systemops
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortRouteCandidates(t *testing.T) {
tests := []struct {
name string
candidates []candidateRoute
wantOrder []uint32
}{
{
name: "longest prefix wins over metrics",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 0, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 24, routeMetric: 100, interfaceMetric: 50},
},
wantOrder: []uint32{2, 1},
},
{
// Windows ranks equal-length prefixes by route metric + interface metric,
// so a higher route metric on a low metric interface can still win.
name: "combined metric beats route metric alone",
candidates: []candidateRoute{
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
{interfaceIndex: 5, prefixLength: 0, routeMetric: 10, interfaceMetric: 5},
},
wantOrder: []uint32{5, 8},
},
{
name: "lower combined metric wins",
candidates: []candidateRoute{
{interfaceIndex: 5, prefixLength: 0, routeMetric: 300, interfaceMetric: 5},
{interfaceIndex: 8, prefixLength: 0, routeMetric: 0, interfaceMetric: 100},
},
wantOrder: []uint32{8, 5},
},
{
name: "equal combined metric falls back to route metric",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 20, interfaceMetric: 10},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 25},
},
wantOrder: []uint32{2, 1},
},
{
// The metrics are uint32 on the Windows side, so the sum must not wrap.
name: "combined metric beyond the uint32 range",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: math.MaxUint32, interfaceMetric: 5},
{interfaceIndex: 2, prefixLength: 0, routeMetric: math.MaxUint32 - 10, interfaceMetric: 5},
},
wantOrder: []uint32{2, 1},
},
{
name: "unknown interface metric ranks on route metric only",
candidates: []candidateRoute{
{interfaceIndex: 1, prefixLength: 0, routeMetric: 30, interfaceMetric: -1},
{interfaceIndex: 2, prefixLength: 0, routeMetric: 5, interfaceMetric: 10},
},
wantOrder: []uint32{2, 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sortRouteCandidates(tt.candidates)
got := make([]uint32, 0, len(tt.candidates))
for _, c := range tt.candidates {
got = append(got, c.interfaceIndex)
}
assert.Equal(t, tt.wantOrder, got)
})
}
}

View File

@@ -882,26 +882,40 @@ func getInterfaceMetric(interfaceIndex uint32, family int16) int {
return int(ipInterfaceRow.Metric)
}
// sortRouteCandidates sorts route candidates by priority: prefix length -> route metric -> interface metric
// sortRouteCandidates sorts route candidates by priority: prefix length -> combined metric -> route metric.
// Windows prefers the longest matching prefix and, among prefixes of the same length, the lowest metric, see
// https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-tcpip-interfaces-interface-routes-route-metric
func sortRouteCandidates(candidates []candidateRoute) {
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].prefixLength != candidates[j].prefixLength {
return candidates[i].prefixLength > candidates[j].prefixLength
}
if candidates[i].routeMetric != candidates[j].routeMetric {
return candidates[i].routeMetric < candidates[j].routeMetric
mi, mj := combinedMetric(candidates[i]), combinedMetric(candidates[j])
if mi != mj {
return mi < mj
}
return candidates[i].interfaceMetric < candidates[j].interfaceMetric
return candidates[i].routeMetric < candidates[j].routeMetric
})
}
// combinedMetric returns the effective metric Windows uses to rank routes with an equal prefix length:
// the sum of the route metric and the metric of the interface the route is on, see
// https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric
// An unknown interface metric contributes nothing.
func combinedMetric(candidate candidateRoute) uint64 {
if candidate.interfaceMetric < 0 {
return uint64(candidate.routeMetric)
}
return uint64(candidate.routeMetric) + uint64(candidate.interfaceMetric)
}
// GetBestInterface finds the best interface for reaching a destination,
// excluding the VPN interface to avoid routing loops.
//
// Route selection priority:
// 1. Longest prefix match (most specific route)
// 2. Lowest route metric
// 3. Lowest interface metric
// 2. Lowest combined metric (route metric + interface metric)
// 3. Lowest route metric.
func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
var skipInterfaceIndex int
if vpnIntf != "" {
@@ -925,7 +939,6 @@ func GetBestInterface(dest netip.Addr, vpnIntf string) (*net.Interface, error) {
return nil, fmt.Errorf("no route to %s", dest)
}
// Sort routes: prefix length -> route metric -> interface metric
sortRouteCandidates(candidates)
for _, candidate := range candidates {

View File

@@ -5,6 +5,7 @@ package systemops
import (
"errors"
"net"
"net/netip"
"syscall"
"testing"
@@ -29,6 +30,7 @@ func ensureIPv6DefaultRoute(t *testing.T) {
}
if err := netlink.RouteAdd(route); err != nil {
if errors.Is(err, syscall.EEXIST) {
requireUsableIPv6Nexthop(t)
return
}
t.Skipf("install IPv6 fallback default route: %v", err)
@@ -38,4 +40,36 @@ func ensureIPv6DefaultRoute(t *testing.T) {
t.Logf("delete IPv6 fallback default route: %v", err)
}
})
requireUsableIPv6Nexthop(t)
}
// requireUsableIPv6Nexthop skips the test unless the resolved IPv6 default
// nexthop can actually carry a route. Installing the default route succeeding
// does not imply the kernel accepts it as a nexthop for a concrete prefix.
func requireUsableIPv6Nexthop(t *testing.T) {
t.Helper()
nexthop, err := GetNextHop(netip.IPv6Unspecified())
if err != nil {
t.Skipf("resolve IPv6 default nexthop: %v", err)
}
probe := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: syscall.RT_TABLE_MAIN,
Family: netlink.FAMILY_V6,
Dst: &net.IPNet{IP: net.ParseIP("100::64"), Mask: net.CIDRMask(128, 128)},
}
require.NoError(t, addNextHop(nexthop, probe), "build IPv6 probe route")
switch err := netlink.RouteAdd(probe); {
case err == nil:
if err := netlink.RouteDel(probe); err != nil && !errors.Is(err, syscall.ESRCH) {
t.Logf("delete IPv6 probe route: %v", err)
}
case errors.Is(err, syscall.EEXIST):
default:
t.Skipf("IPv6 nexthop %s unusable for route installation: %v", nexthop, err)
}
}

View File

@@ -232,4 +232,3 @@ func toNetIDs(routes []string) []route.NetID {
}
return netIDs
}

View File

@@ -200,7 +200,7 @@ func startManagement(t *testing.T, signalAddr string, counter *int) (*grpc.Serve
requestBuffer := server.NewAccountRequestBuffer(context.Background(), store)
peersUpdateManager := update_channel.NewPeersUpdateManager(metrics)
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config)
networkMapController := controller.NewController(context.Background(), store, metrics, peersUpdateManager, requestBuffer, server.MockIntegratedValidator{}, settingsMockManager, "netbird.selfhosted", port_forwarding.NewControllerMock(), manager.NewEphemeralManager(store, peersManager), config, nil)
accountManager, err := server.BuildManager(context.Background(), config, store, networkMapController, jobManager, nil, "", eventStore, nil, false, ia, metrics, port_forwarding.NewControllerMock(), settingsMockManager, permissionsManagerMock, false, cacheStore)
if err != nil {
return nil, "", err

View File

@@ -6,9 +6,11 @@ import (
"context"
"time"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
gstatus "google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal/profilemanager"
"github.com/netbirdio/netbird/client/proto"
)
@@ -60,9 +62,19 @@ func (s *Session) RequestExtend(ctx context.Context, p ExtendStartParams) (Exten
// a request from the UI implies a graphical session, which the daemon cannot detect itself
req := &proto.RequestExtendAuthSessionRequest{HasGraphicalSession: true}
if p.Hint != "" {
h := p.Hint
req.Hint = &h
hint := p.Hint
if hint == "" {
pm := profilemanager.NewProfileManager()
if active, perr := pm.GetActiveProfile(); perr != nil {
log.Debugf("failed to get active profile for login hint: %v", perr)
} else if state, serr := pm.GetProfileState(active.ID); serr != nil {
log.Debugf("failed to get profile state for login hint: %v", serr)
} else {
hint = state.Email
}
}
if hint != "" {
req.Hint = &hint
}
resp, err := cli.RequestExtendAuthSession(ctx, req)

View File

@@ -123,8 +123,16 @@ func (s *Connection) Login(ctx context.Context, p LoginParams) (LoginResult, err
if p.PreSharedKey != "" {
req.OptionalPreSharedKey = ptrStr(p.PreSharedKey)
}
if p.Hint != "" {
req.Hint = ptrStr(p.Hint)
hint := p.Hint
if hint == "" && profileID != "" {
if state, serr := profilemanager.NewProfileManager().GetProfileState(profilemanager.ID(profileID)); serr == nil {
hint = state.Email
} else {
log.Debugf("failed to get profile state for login hint: %v", serr)
}
}
if hint != "" {
req.Hint = ptrStr(hint)
}
resp, err := cli.Login(ctx, req)
@@ -228,16 +236,6 @@ func (s *Connection) Logout(ctx context.Context, p LogoutParams) error {
return s.classifyDaemonError(err)
}
// The daemon runs as root and can't reach the user-owned per-profile state
// file holding the account email (see Profiles.List), so clear the stale
// email here; the next SSO login recreates it.
if p.ProfileName != "" {
if err := profilemanager.NewProfileManager().RemoveProfileState(p.ProfileName); err != nil {
// Non-fatal: the logout itself succeeded.
log.Warnf("failed to remove profile state for %s: %v", p.ProfileName, err)
}
}
return nil
}
@@ -261,7 +259,7 @@ func (s *Connection) waitSSOLogin(ctx context.Context, p WaitSSOParams) (string,
// Persist the account email the same way the CLI does after its own
// WaitSSOLogin: the daemon returns it but cannot store it, since it runs as
// root and the per-profile state file is user-owned (see Logout below).
// root and the per-profile state file is user-owned (see Profiles.List).
// Without this the profile has no email, so Profiles.List shows no account
// and later logins and session extends go out without a login_hint —
// leaving the IdP to guess which account was meant.

View File

@@ -162,8 +162,9 @@ func (s *Profiles) Remove(ctx context.Context, p ProfileRef) error {
}
// The daemon deletes what it owns but runs as root, so it leaves the
// user-owned state file holding the account email behind (same split as
// Connection.Logout). Legacy profiles are keyed by name rather than by a
// user-owned state file holding the account email behind. Logout keeps the
// email on purpose so later logins can pass it as the login_hint; profile
// removal is what deletes it. Legacy profiles are keyed by name rather than by a
// generated ID, so a recreated profile of the same name would inherit the
// deleted one's email and offer it as the login_hint.
//

View File

@@ -115,7 +115,7 @@ sequenceDiagram
Resp->>Resp: parse usage tokens, completion
Note over Resp: capture_completion gates raw<br/>completion capture
Resp->>Cost: tokens
Cost->>Cost: lookup pricing.yaml + compute cost
Cost->>Cost: lookup rates from config-delivered<br/>pricing table + compute cost
Cost->>Rec: tokens + cost
Rec->>MgmtGrpc: RecordLLMUsage(provider, model, prompt_t, completion_t, cost, groups, user)
Rec-->>Log: emit access-log entry<br/>(if EnableLogCollection)

View File

@@ -15,6 +15,10 @@ Inside the package: `manager.go` is the CRUD + permissions-gated facade; `synthe
| ---- | ---- |
| `agentnetwork/manager.go` | Manager interface + CRUD + permission gates + bootstrap-settings + reconcile trigger |
| `agentnetwork/synthesizer.go` | Settings/policy → wire-format synthesis; sole writer of the proxy middleware chain |
| `agentnetwork/synthesizer_pricing.go` | `buildCostMeterConfigJSON` — default table + per-provider prices → `cost_meter` config |
| `agentnetwork/pricing/defaults.go` | Default pricing table derived from the catalog + supplementals; `DefaultTable`, `LookupDefault`, wire `Entry` |
| `agentnetwork/pricing/override.go` | `LoadFile`/`StartReloader` for `AgentNetwork.PricingDefaultsFile` (mtime poll, merge over compiled-in base) |
| `agentnetwork/pricing/{exampleyaml,gen}.go` | Generates `defaults_llm_pricing.example.yaml` from the compiled-in table (golden-tested) |
| `agentnetwork/policyselect.go` | Per-request policy attribution + account-budget ceiling (min-wins) |
| `agentnetwork/reconcile.go` | Per-account synth diff vs in-memory cache → Create/Update/Delete |
| `agentnetwork/catalog/catalog.go` | Static provider catalogue (auth headers, identity-injection shapes) |
@@ -48,6 +52,8 @@ flowchart TD
I --> J[indexProviderGroups: providerID -> sorted source groups]
J --> K[buildRouterConfigJSON drops orphan providers]
J --> L[buildIdentityInjectConfigJSON per catalog entry]
J --> K2[buildCostMeterConfigJSON: default table + per-provider prices]
K2 --> P
H --> M[mergeGuardrails: union allowlist, OR redact]
M --> N[applyAccountCollectionControls account toggle = SOLE capture control]
N --> O[marshalGuardrailConfig]
@@ -60,6 +66,84 @@ flowchart TD
R --> T[accountManager.UpdateAccountPeers — fans synth ACLs into network map]
```
### LLM pricing (management is the sole authority)
**The proxy carries no price list.** Management synthesizes the entire pricing
table and ships it inside `cost_meter`'s `ConfigJSON`, so a price change reaches
the proxies as an ordinary mapping push — the chain rebuild installs a fresh
table and there is nothing to reload on the proxy side.
```mermaid
flowchart TD
A[catalog.All — PricingSurfaces x Models] --> B[buildDefaultTable + supplementalDefaults]
B --> C{AgentNetwork.PricingDefaultsFile}
C -- absent --> D[compiled-in table serves]
C -- loaded --> E[LoadFile: merge file entries WHOLE over compiled base]
E --> F[mergedTable atomic.Pointer]
D --> G[DefaultTable]
F --> G
G --> H[buildCostMeterConfigJSON — pricing.defaults]
I[types.Provider.Models operator prices] --> J[normalizePricingModelID<br/>bedrock ARN/region/version, vertex @version]
J --> K[materializeEntry: default entry as base,<br/>operator input/output verbatim,<br/>cache pointers only when non-nil]
K --> L[pricing.providers keyed by provider record ID]
H --> M[cost_meter ConfigJSON]
L --> M
G --> N[GET /catalog — applyDefaultPricing prefills dashboard rows]
O[StartReloader: mtime poll every ReloadInterval 1m] --> E
```
**Two tiers, resolved per request on the proxy** (`synthesizer_pricing.go:22-35`):
- `pricing.defaults` — surface (`openai`/`anthropic`/`bedrock`) → normalized model
id → rates. The **full** default table ships to every account: it is small
(~10 KB) and it is what keeps gateway-style providers (which enumerate no
models, so they claim every model) priced.
- `pricing.providers` — provider **record** id → normalized model id → rates,
matched against the `llm.resolved_provider_id` the router stamps. Entries are
**fully materialized here**, at synth time: `materializeEntry` starts from the
default entry for that model so cache rates the operator didn't state are
inherited, overlays operator `input`/`output` verbatim (**including an explicit
0**, which prices a self-hosted or internal endpoint as free rather than
silently reverting to list price), and overlays cache-rate **pointers only when
non-nil** — `nil` means "inherit the default", an explicit `0` means "no
discount, bill this bucket at the input rate". The proxy therefore does two map
lookups and no merging.
Same orphan rule as the router: a provider no enabled policy authorises is
unreachable, so its prices aren't shipped. Model ids are normalized with the
**same** functions the request parser uses (`NormalizeBedrockModel` /
`NormalizeVertexModel`), which is what makes the per-record lookup key compare
equal to the `llm.model` the proxy meters. Post-normalization duplicates resolve
first-occurrence-wins, matching the routing dedup order.
**`AgentNetwork.PricingDefaultsFile`** (`config.go:190-207`) lets an operator
replace default rates without a rebuild. Schema is `surface → model → rates`
(`input_per_1k`, `output_per_1k`, and optional `cached_input_per_1k` /
`cache_read_per_1k` / `cache_creation_per_1k`). Semantics:
- A **relative** path resolves against `<Datadir>`, so a bare filename lands
alongside the store. Empty config probes `<Datadir>/defaults_llm_pricing.yaml`.
- An **explicitly configured** path is *required to load*: a typo or malformed
file fails startup, because the operator believes those rates are live. The
conventional probe is optional — an absent file just serves compiled-in
defaults, and the path stays watched in case it appears later.
- File entries **replace** the compiled-in entry for the same (surface, model)
**whole** — they are not field-merged, so an entry must repeat the cache rates
it wants to keep. Everything the file doesn't mention keeps built-in rates.
- Unknown YAML fields are rejected (`KnownFields(true)`) and every rate must be
finite and non-negative — the same constraints the HTTP API enforces on
operator per-provider prices.
- Reload is an mtime poll (`ReloadInterval`, 1 min) and is **lenient at runtime**:
a parse error keeps the previous table, a deleted file reverts to compiled-in
defaults. A mid-edit save can never take pricing down.
The live table feeds **both** consumers, which is what keeps them consistent: the
synthesizer (what proxies actually bill with) and `GET /api/agent-network/catalog`
via `applyDefaultPricing` (what the dashboard's model-row prices prefill with).
`defaults_llm_pricing.example.yaml` is generated from the compiled-in table
(`go generate ./management/internals/modules/agentnetwork/pricing`) and
golden-tested, so operators start from a file matching the built-in rates exactly.
### Budget rule resolution (min-wins, group+user bound)
```mermaid
@@ -124,7 +208,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| on_request | 3 | `llm_identity_inject` | `{"providers":[{provider_id, header_pair?, json_metadata?, extra_headers?}]}` | **true** |
| on_request | 4 | `llm_guardrail` | `{"provider_allowlists"?: {providerID: []model}, "prompt_capture":{enabled,redact_pii}}` | |
| on_response | 5 | `llm_limit_record` | `{}` (runs LAST at runtime) | |
| on_response | 6 | `cost_meter` | `{}` | |
| on_response | 6 | `cost_meter` | `{"pricing":{"defaults":{surface:{model:rates}},"providers"?:{providerRecordID:{model:rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}` | |
| on_response | 7 | `llm_response_parser` | `{"capture_completion": <bool>, "redact_pii"?: true}` | |
- **Synthesized service shape** (`synthesizer.go:739`): `Mode=HTTP`, `Private=true`, `Domain=<subdomain>.<cluster>`, `AccessGroups=unionSourceGroups(enabledPolicies)`, one `TargetTypeCluster` target with `Host=noop.invalid:443` (router rewrites per request), `Options.{DirectUpstream,AgentNetwork}=true`, `DisableAccessLog=!settings.EnableLogCollection`, `CaptureMax{Req,Resp}Bytes=1<<20`, `CaptureContentTypes=["application/json","text/event-stream"]`.
@@ -139,6 +223,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Orphan providers (no enabled policy authorises them) NEVER reach the router** (`synthesizer.go:351-357`); skipped from `identity_inject` for symmetry.
- **Provider creation refuses empty `api_key`** (`manager.go:175`); **deletion refuses while any policy still references it** (`manager.go:265-273`).
- **Session keypair stability across provider edits** (`manager.go:226-228`) — server-managed, copied through every `UpdateProvider`, never API-surfaced.
- **Management is the sole pricing authority.** The proxy has no embedded price list, so an account whose `cost_meter` config carries no `pricing` block bills **nothing** (`cost.skipped=unknown_model`, $0) rather than falling back to stale built-ins. The top-level `pricing` wrapper is also the feature-detection signal in both directions: an old proxy ignores it as an unknown field, and a new proxy reads its absence as "old management".
- **Per-provider prices are materialized at synth time, not merged on the proxy** (`synthesizer_pricing.go:114-131`). A per-record entry is always complete, so the proxy's lookup is per-record-then-defaults with no field-level fallback between tiers.
- **An explicit operator price of `0` prices the model as free** — it must not be treated as "unset" and reverted to list price (`synthesizer_pricing.go:49-54`). Only *cache*-rate fields distinguish unset from zero, via `*float64`.
- **Pricing model ids are normalized with the same functions the request parser uses** (`normalizePricingModelID`). If the two ever diverge, per-record prices silently stop matching and every request falls through to surface defaults.
- **The default table's coverage is structural, not curated.** It is derived from the catalog via each provider's `PricingSurfaces`; `TestDefaultTable_CoversEveryCatalogModel` fails on an unpriced catalog model and `TestDefaultTable_NoConflictingContributions` fails if two providers contribute the same (surface, model) at different rates.
- **A pricing-defaults file failure is fatal only at startup, and only for an explicitly configured path.** Runtime reload failures keep the previous table; a deleted file reverts to compiled-in defaults (`pricing/override.go:62-81, 113-148`).
## Things to scrutinize
@@ -176,10 +266,12 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Capture-pointer semantics (restated):** non-agent-network callers see no field → legacy nil-default emit, identical to pre-PR. Agent-network targets always carry an explicit `capture_*` value.
- **`TestSynthesizeServices_HappyPath` was updated:** request-parser config moved from `{}` to `{"capture_prompt":false}` (`synthesizer_test.go:174`). External snapshot tests against synth output need updating.
- **`MergedGuardrails` retains zeroed `TokenLimits`/`Budget`/`Retention`** even though `Policy.Limits` carries the real values now; `llm_limit_check` is the authoritative enforcement. Comment at `synthesizer.go:940-948` calls this out.
- **`cost_meter`'s `pricing` block is version-skew-safe in both directions.** A proxy predating config-delivered pricing ignores the field as unknown JSON (it previously priced from its own embedded table, so it keeps billing — at its own rates, which is the skew to be aware of during a rolling upgrade). A current proxy paired with old management sees no `pricing` block, logs one warning at chain-build time, and records `cost.skipped=unknown_model` — token counting and cap enforcement are unaffected, only the USD annotation goes to $0.
### Performance
- **`SynthesizeServices` runs on every controller tick / mutation reconcile.** Cost: 4 store reads + optional per-provider keypair backfill. Sort + index + merge are O(N log N) / O(P × G); dominant cost is JSON marshalling. No nested loops escape these dimensions.
- **The full default pricing table is marshalled into every account's `cost_meter` config on every synth** (~10 KB serialized). This is a deliberate trade: it keeps gateway-style providers priced for every catalog model, and it is the largest single contributor to the synth JSON. `DefaultTable()` itself is a pointer load (or a `sync.Once`-built map) — the cost is the marshal, not the build.
- **`reconcile.diffMappings` is O(N + M)** with N=M=1 per account today — effectively constant.
- **`SynthesizeServicesForCluster`** (`synthesizer.go:71`) walks every account on a cluster; per-account failures are **swallowed** (`synthesizer.go:91-93`) so a single misconfigured account doesn't drop the cluster. Runs per proxy reconnect.
@@ -188,6 +280,7 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
- **Activity codes:** `AgentNetwork{Provider,Policy,Guardrail,BudgetRule}{Created,Updated,Deleted}`; `AgentNetworkSettingsUpdated` with `log_collection/prompt_collection/redact_pii` payload (`manager.go:567-571`). **No activity code for `SelectPolicyForRequest` denies** — surfaced via proxy access log only (likely intentional given volume).
- **Deny codes** namespaced: `llm_policy.{token,budget}_cap_exceeded`, `llm_account.{token,budget}_cap_exceeded` (`policyselect.go:18-26`).
- **Reconcile failures are logged at warn and swallowed** (`reconcile.go:42-44`). Persistent synth failures (e.g. unknown catalog id) silently keep the proxy out of sync — consider a manager-level synth-health surface if this becomes a support burden.
- **Pricing-file lifecycle logs at info** (load, reload, revert-to-built-ins) and **at warn** for a runtime reload failure; the mtime check itself is `Debugf`. There is no metric on reload failures, so an operator who breaks the file mid-flight keeps billing at the previous table with only a log line to show it (`pricing/override.go:113-148`).
## Test coverage
@@ -198,6 +291,9 @@ At request time the path is independent: the proxy calls `SelectPolicyForRequest
| `synthesizer_guardrail_realstore_test.go` | `PromptCaptureAccountIsSoleControl`; `PromptCaptureFlowsWhenAccountOptsIn`; `AccountRedactWithoutGuardrailRedact`; `NoGuardrail_CaptureOff`. |
| `synthesizer_log_collection_realstore_test.go` | `LogCollection{Off_SuppressesAccessLog,On_PermitsAccessLog}` — verifies `DisableAccessLog` propagation through `ToProtoMapping`. |
| `synthesizer_parser_redact_realstore_test.go` | **Capture-pointer regression suite:** `ParserConfigsCarryRedactPii`; `ParserConfigsSuppressCaptureWhenLogCollectionOnly` (log=on/prompt=off ⇒ both capture flags false); `ParserConfigsOmitRedactPiiWhenOff`. |
| `synthesizer_pricing_test.go` | `BuildCostMeterConfig_{BedrockModelNormalization,CacheRateNilVsZero,OrphanAndGatewayProviders}` — the per-record tier's three load-bearing rules: keys normalized like the parser's, `nil` cache pointer inherits vs explicit `0` bills at input rate, and orphan / gateway (empty `Models`) providers ship no per-record entry. |
| `pricing/defaults_test.go` | `DefaultTable_{CoversEveryCatalogModel,NoConflictingContributions,AllRatesFiniteNonNegative,PinnedRates}`; `LookupDefault_SurfaceOrder`. Catalog-derived coverage + rate sanity are structural, not curated. |
| `pricing/override_test.go` | `LoadFile_{MergesOverCompiledDefaults,MissingPath,RejectsInvalid}`; `Reload_LifeCycle` (mtime detect, parse error keeps previous, delete reverts to built-ins); `ExampleYAML_InSyncWithBuiltins` golden. |
| `policyselect_test.go` | Mock-store: `NoApplicablePolicies`; `AllowWithLowestGroupAttribution`; `LargerPoolWinsAcrossUsageLevels`; `StaysOnLargerPoolAfterPartialDrain`; `FallsThroughToSmallerPoolWhenLargerExhausted`; `TiebreakBy{LargerGroupPool,CreatedAt}`; `DeniesWhenAllExhausted`; `UncappedPolicyAlwaysWinsAgainstCapped`; `DisabledPolicyIgnored`; `StoreErrorPropagates`; `RejectsEmptyAccount`; `SharesGroupCounterAcrossPolicies`; `AntiFallThroughOnLowestGroup`; `BudgetOnlyExhaustionDenies`; `BudgetTighterThanTokenWins`. |
| `policyselect_realstore_test.go` | Real-sqlite regression guard: `NoApplicablePolicies`; `AllowAndLowestGroupAttribution`; `LargerPoolWins_FallsThroughWhenExhausted`; `BudgetCapDenies`; `GroupCounterSharedAcrossPolicies`; `DisabledPolicyIgnored`. |
| `policyselect_account_realstore_test.go` | Account budget rules: `AccountCeilingBindsEvenWithUncappedPolicy` (min-wins); `AccountGroupCeiling`; `AccountTargetUsersBindsOnlyThatUser`; `AccountRuleRecordsToOwnWindow`. |

View File

@@ -5,7 +5,7 @@ LLM request. The two highest-blast-radius areas are the **capture-pointer
semantics** and the **limit_check ⇒ limit_record** record-once invariant.
Sibling module: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — the SDK
adapters + pricing catalog this chain delegates to.
adapters + pricing table and cost formula this chain delegates to.
---
@@ -34,7 +34,7 @@ rewrites.
| `llm_identity_inject` | OnRequest | `llm.{resolved_provider_id,authorising_groups}`, `Input.{UserEmail,UserID,UserGroups,UserGroupNames}` | none | header strip/inject + optional body rewrite |
| `llm_guardrail` | OnRequest | `llm.{model,request_prompt_raw}` | `llm_policy.{decision,reason}`, `llm.request_prompt` | none (model allowlist deny) |
| `llm_response_parser` | OnResponse | `llm.provider`, `Input.{RespHeaders,RespBody,Status}` | `llm.{input,output,total,cached_input,cache_creation}_tokens`, `llm.response_completion` | none |
| `cost_meter` | OnResponse | `llm.{provider,model}`, token buckets | `cost.usd_total` or `cost.skipped` | pricing lookup |
| `cost_meter` | OnResponse | `llm.{provider,model,resolved_provider_id}`, token buckets | `cost.usd_{input,cached_input,cache_creation,output,total,cache}` or `cost.skipped` | none (in-memory pricing lookup) |
| `llm_limit_record` | OnResponse | `llm.{attribution_group_id,attribution_window_seconds,input_tokens,output_tokens}`, `cost.usd_total` | none | gRPC `RecordLLMUsage` |
[all_test.go:2640](../../../proxy/internal/middleware/builtin/all_test.go)
@@ -44,7 +44,7 @@ locks the ID set; adding or removing one is a conscious extension.
| File | LOC | Notes |
|---|---:|---|
| `builtin.go` | 86 | Registry + `FactoryContext` (ctx, data dir, meter, logger, mgmt client) |
| `builtin.go` | 90 | Registry + `FactoryContext` (ctx, meter, logger, mgmt client) |
| `all_test.go` | 41 | Locks the 8-ID registry surface |
| `agentnetwork_chain_integration_test.go` | 319 | Live sqlite + real gRPC bufconn; gate→recorder wire path |
| `llm_request_parser/*` | 162 / 66 / 356 | Provider detection, body parse, prompt extraction with capture-pointer gating |
@@ -53,7 +53,7 @@ locks the ID set; adding or removing one is a conscious extension.
| `llm_identity_inject/*` | 440 / 108 / 666 | HeaderPair (LiteLLM) + JSONMetadata (Portkey) + ExtraHeaders |
| `llm_guardrail/*` | 176 / 82 / 75 / 219 / 217 | Model allowlist + optional prompt capture with PII redaction |
| `llm_response_parser/*` | 258 / 222 / 43 / 433 / 169 / 111 | Buffered + SSE accumulation; AWS event-stream accumulator (`streaming_bedrock.go`) for Bedrock; capture-pointer gates completion emit |
| `cost_meter/*` | 181 / 84 / 439 | Token → USD via `proxy/internal/llm/pricing` |
| `cost_meter/*` | 236 / 98 / 586 | Token → USD via `proxy/internal/llm/pricing`; both pricing tiers arrive in the middleware config |
| `llm_limit_record/*` | 144 / 35 / 191 | Post-flight `RecordLLMUsage` (5s, debug-on-error) |
## Per-middleware
@@ -168,12 +168,46 @@ token schema.
### cost_meter
Reads `llm.provider` + `llm.model` + token buckets, looks up per-1k rate via
`pricing.Loader`, emits `cost.usd_total` or a closed-set `cost.skipped`
reason (`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`). Loader's hot-reload goroutine is bound to proxy-lifetime
context via `startReloader`. **Key invariant:** provider-shape switch lives
in `pricing.Table.Cost` (sibling doc) — `cost_meter` stays provider-agnostic.
Reads `llm.provider` + `llm.model` + token buckets, looks up the per-1k rates,
and emits the full `cost.usd_*` breakdown (four per-bucket values plus the
`_total` and `_cache` aggregates) or a closed-set `cost.skipped` reason
(`missing_provider/model/tokens`, `unparseable_tokens`, `zero_tokens`,
`unknown_model`).
**Management owns pricing.** The proxy carries no embedded price list: the whole
table arrives in this middleware's `ConfigJSON` as
`{pricing: {defaults, providers}}`, synthesized by management from the catalog
plus the operator's stored per-provider prices
([factory.go:1334](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
Both tiers are validated by `pricing.NewTable` / `pricing.NewEntries` at
construction, so a non-finite or negative rate fails the chain build. A price
change is an ordinary mapping push — the chain rebuild yields a fresh instance
over a fresh immutable table, so there is no data dir, no pricing file, no
reload goroutine, and nothing to invalidate.
**Two-tier lookup**
([middleware.go:165183](../../../proxy/internal/middleware/builtin/cost_meter/middleware.go)):
1. **Per-provider-record** — the operator's stored price for the route that
actually served the request, keyed by the `llm.resolved_provider_id` that
`llm_router` stamped on the allow path, then by normalized model id. Entries
arrive fully materialized (management folds default cache rates in at synth
time), so there is no merging here. Absent metadata — no router in the chain
— skips this tier.
2. **Surface defaults** — the catalog-derived table keyed by `llm.provider`
(`openai`/`anthropic`/`bedrock`). This is also what prices gateway-style
providers, which enumerate no models and therefore get no per-record entry.
**Backward compatibility:** a config with no `pricing` block means management
predates config-delivered pricing. The factory logs one warning at build time
and the instance records `cost.skipped=unknown_model` ($0) for every request
rather than falling back to a stale built-in price list
([factory.go:5560](../../../proxy/internal/middleware/builtin/cost_meter/factory.go)).
**Key invariant:** the provider-shape switch lives in `pricing.EntryCosts`
(sibling doc) and is selected by the **surface**, not by which tier the entry
came from — `cost_meter` stays provider-agnostic, and a per-record override on
an Anthropic route still bills its cache buckets additively.
### llm_limit_record
@@ -246,12 +280,14 @@ no mocks. Tests: `TestChain_AllowPath_StampsAttributionAndRecordsCounter`
| `llm_identity_inject` | `{providers: [{provider_id, header_pair?|json_metadata?, extra_headers?}]}` |
| `llm_guardrail` | `{provider_allowlists: {providerID: []string}, prompt_capture: {enabled, redact_pii}}` — allowlist keyed by resolved provider id; a provider absent from the map is unrestricted (fail-closed backstop; authoritative per-policy/group check is management's `CheckLLMPolicyLimits`) |
| `llm_response_parser` | `{redact_pii?, capture_completion?: *bool}` |
| `cost_meter` | `{pricing_path?}` (basename inside data-dir; defaults `pricing.yaml`) |
| `cost_meter` | `{pricing: {defaults: {surface: {model: rates}}, providers: {providerRecordID: {model: rates}}}}` — rates are `{input_per_1k, output_per_1k, cached_input_per_1k?, cache_read_per_1k?, cache_creation_per_1k?}`. A missing `pricing` key means "management predates config-delivered pricing": every request records `cost.skipped=unknown_model` |
| `llm_limit_record` | `{}` — same pattern as `llm_limit_check` |
All factories accept empty / null / `{}` / whitespace as zero-value config;
only structurally invalid JSON is rejected so misconfig surfaces at chain
build time.
build time. `cost_meter` adds a semantic check on top of that: a `pricing`
block carrying a negative or non-finite rate fails the build too, rather than
mispricing live traffic.
## Invariants
@@ -320,10 +356,11 @@ non-object `metadata` field
— header path still attributes, but body-level tag-budget enforcement
doesn't run for that request.
**Concurrency.** `cost_meter` shares a `pricing.Loader` via
`atomic.Pointer[Table]`; readers always see a consistent table. Every
middleware is a stateless value receiver. Integration test uses real bufconn
gRPC — race detector is the meaningful bar.
**Concurrency.** `cost_meter`'s two pricing tables are built once from the
middleware config and never mutated, so the lookup path needs no lock or atomic
swap — a price change replaces the whole instance. Every middleware is
otherwise a stateless value receiver. Integration test uses real bufconn gRPC —
race detector is the meaningful bar.
**Perf.** Hot path is `lookupKV` linear scan over <10 KVs; `cost_meter.Cost`
is O(1); SSE accumulation is single-pass. No map allocation per call.
@@ -349,13 +386,13 @@ counter accuracy.
| `llm_guardrail/redact_test.go` | 15 | Email, SSN, phone (E.164 + NA), bearer, IPv4; fixture-driven |
| `llm_response_parser/middleware_test.go` | 18 | Buffered OAI+Anthro, capture-pointer, redact, truncation |
| `llm_response_parser/streaming_test.go` | 7 | OAI usage frame, Anthro message_delta, truncated body best-effort |
| `cost_meter/middleware_test.go` | 17 | Each skip reason, provider-shape, pricing loader integration |
| `cost_meter/middleware_test.go` | 22 | Each skip reason, provider-shape formulas, config-delivered defaults, per-record-beats-defaults + miss-falls-back, per-record uses surface formula, nil-pricing skips everything, invalid-rate rejection |
| `llm_limit_record/middleware_test.go` | 7 | Skip-on-no-signal, skip-on-missing-attribution, RPC failure swallowed |
## Cross-references
- Sibling: [32-proxy-llm-parsers.md](./32-proxy-llm-parsers.md) — SDK adapters
+ SSE framer + pricing loader.
+ SSE framer + pricing table and cost formula.
- Path-routed providers (Vertex AI + Bedrock), `keyfile::` credential, GCP
token minting, `/bedrock` prefix:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -9,7 +9,7 @@ pricing table's per-provider cost formula is the highest-leverage place a
small bug would silently mis-bill operators.
Sibling module: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the 8 middlewares that consume this package's parsers + pricing loader.
— the 8 middlewares that consume this package's parsers + pricing table.
---
@@ -24,8 +24,9 @@ proxy-framework dependencies:
- `openai.go` / `anthropic.go` / `bedrock.go` — per-provider `Parser` impls.
- `sse.go` — SSE scanner (`Scanner`, `Event`, `NewScanner`).
- `errors.go` — sentinels callers branch on with `errors.Is`.
- `pricing/`embedded-default + hot-reload override table with
symlink-safe Unix loader (build-tagged stub elsewhere).
- `pricing/`immutable pricing table + the per-surface cost formula. The
rates themselves come from management inside `cost_meter`'s middleware
config; this package holds no price list and reads no files.
- `fixtures/` — captured request/response/stream bodies the tests replay.
The package carries zero proxy-framework dependencies so the same parsers can
@@ -47,12 +48,9 @@ be reused later by a WASM adapter
| `sse_test.go` | 175 | 12 tests; fixture replay + multiline + size limits |
| `parser_test.go` | 53 | `Parsers()`, `DetectParser`, provider enum values |
| `errors.go` | 31 | 6 sentinels: `Err{Unknown,Unsupported}Provider/Model`, `Err{NotLLM,Malformed}Response`, `ErrStreamingUnsupported`, `ErrMalformedRequest` |
| `pricing/pricing.go` | 421 | `Loader`, `Table`, `Entry`; embedded defaults + atomic swap + mtime reload |
| `pricing/pricing_unix.go` | 69 | `O_NOFOLLOW` + fstat-from-FD + 1 MiB cap |
| `pricing/pricing_other.go` | 21 | Stub returning "not supported on this platform" |
| `pricing/pricing_test.go` | 432 | 21 tests — symlink rejection, reload race, path traversal, oversize |
| `pricing/defaults_pricing.yaml` | 85 | go:embed source of truth |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream + pricing starter |
| `pricing/pricing.go` | 234 | `Table`, `Entry`, `EntryJSON`, `Costs`; `NewTable`/`NewEntries` validation + `EntryCosts` formula. No I/O, no reload, no embedded rates |
| `pricing/pricing_test.go` | 177 | 10 tests — provider-shape formulas, cached clamp, rate fallback, nil-safety, rate validation |
| `fixtures/*` | 2159 | OAI chat/responses/stream + Anthro messages/stream |
## Request body → parser dispatch
@@ -188,9 +186,11 @@ response leg, covering both Bedrock body shapes:
`totalTokens`). `firstNonZero` folds the two naming conventions into one
`Usage`; when Converse omits `totalTokens` the parser sums the buckets.
`ProviderName()` returns `"bedrock"` — its own `defaults_pricing.yaml` block,
keyed by the **normalised** model id (region prefix + version suffix stripped by
the request parser). `ParseResponse` returns `ErrStreamingUnsupported` for an
`ProviderName()` returns `"bedrock"` — its own pricing surface in the table
management ships, keyed by the **normalised** model id (region prefix + version
suffix stripped by the request parser; management normalises its keys the same
way at synth time so the two compare equal). `ParseResponse` returns
`ErrStreamingUnsupported` for an
AWS binary event-stream content-type (`application/vnd.amazon.eventstream`,
`isAWSEventStream`) so the caller routes to the streaming accumulator instead.
@@ -205,11 +205,34 @@ response body. Streaming accumulators live in the middleware package
([llm_response_parser/streaming.go](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go))
but use `llm.NewScanner` so the framing contract stays here.
### Pricing catalog
### Pricing table
`Table.Cost`
([pricing.go:129174](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module:
**Management is the sole pricing authority.** The proxy carries no embedded
price list and reads no pricing file: the whole table arrives inside
`cost_meter`'s `ConfigJSON` on the ordinary mapping push, and a price change
is just another push — the chain rebuild constructs a fresh `Table`, so there
is nothing to reload
([pricing.go:17](../../../proxy/internal/llm/pricing/pricing.go)). The
management side of the contract (catalog defaults, the operator's stored
per-provider prices, and `AgentNetwork.PricingDefaultsFile`) is covered in the
management-side module guide; `cost_meter`'s wire shape is in
[31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md).
`EntryJSON`
([pricing.go:3645](../../../proxy/internal/llm/pricing/pricing.go)) is the
management→proxy contract — five USD-per-1k rates under `input_per_1k`,
`output_per_1k`, `cached_input_per_1k`, `cache_read_per_1k`,
`cache_creation_per_1k`. Management's `pricing.Entry` marshals the identical
names, and `EntryJSON`/`Entry` are field-identical so `NewEntries` converts by
direct struct conversion rather than field-by-field copying (a new rate can't
be silently dropped in transit).
`EntryCosts`
([pricing.go:183234](../../../proxy/internal/llm/pricing/pricing.go))
is the cost formula — most security-relevant math in this module. The
**surface** (the `llm.provider` value the request parser stamped) selects the
formula, never the tier the entry came from: a per-provider-record override on
an Anthropic route still bills its cache buckets additively.
| Provider | Formula |
|---|---|
@@ -218,7 +241,7 @@ is the cost formula — most security-relevant math in this module:
| default | `inTokens × InputPer1K + outTokens × OutputPer1K` |
`bedrock` shares the Anthropic additive-cache formula
([pricing.go:172-174](../../../proxy/internal/llm/pricing/pricing.go)):
([pricing.go:214229](../../../proxy/internal/llm/pricing/pricing.go)):
Anthropic-on-Bedrock reports the same additive cache buckets, while non-Anthropic
Bedrock models (Nova, Llama) simply report zero in those buckets so cost reduces
to `input + output`.
@@ -226,15 +249,12 @@ to `input + output`.
Each per-bucket rate falls back to `InputPer1K` when zero — operators opt in
to discounts by setting the field.
`Loader`
([pricing.go:212268](../../../proxy/internal/llm/pricing/pricing.go))
overlays an optional `pricing.yaml` from data-dir on top of the go:embed
defaults. Atomic pointer swap means readers never observe a partial update.
The mtime-poll reloader (30s default cadence) keeps the previous table on
parse failure so cost annotation never goes blank during a botched edit.
`defaults_pricing.yaml` is the source of truth for built-in pricing.
Operator overrides only carry the entries they want to change.
`Costs`
([pricing.go:143163](../../../proxy/internal/llm/pricing/pricing.go)) is the
per-request split. The four per-bucket fields are the base; `TotalUSD` and
`CacheUSD` are **derived** in `newCosts` so the aggregates can never drift from
the breakdown. `InputUSD` is always the non-cached input bucket on both
provider shapes, so input and cached-input never double-count.
## Public contracts
@@ -264,29 +284,38 @@ Order matters: `DetectFromURL` ties resolve by registration order.
`ProviderBedrock = 3`. Numeric values are persisted in nothing today but treat
them as wire-stable — new providers must take fresh numbers.
**`Pricing` lookup**
([pricing.go:129](../../../proxy/internal/llm/pricing/pricing.go)):
**`Pricing` construction + lookup**
([pricing.go:60130](../../../proxy/internal/llm/pricing/pricing.go)):
```go
func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
func NewTable(raw map[string]map[string]EntryJSON) (*Table, error)
func (t *Table) Lookup(provider, model string) (Entry, bool)
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
```
Nil-safe: `t.Cost` on a nil receiver returns `(0, false)`
([pricing.go:130132](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means provider or model is absent from the loaded table; the caller
emits `cost.skipped=unknown_model`.
`NewTable` is the surface-keyed defaults table; `NewEntries` returns the raw
two-level map `cost_meter` uses for the per-provider-record tier (it looks up an
`Entry` directly and calls `EntryCosts`, so it needs no `Table` wrapper). Both
reject any non-finite or negative rate, so a corrupt config fails the chain
build rather than mispricing silently. Nil input yields an empty,
never-matching table.
Nil-safe: `t.Cost`/`t.Lookup` on a nil receiver returns `ok=false`
([pricing.go:9699](../../../proxy/internal/llm/pricing/pricing.go)).
`ok=false` means the surface or model is absent from the table management sent;
the caller emits `cost.skipped=unknown_model`.
## Invariants
1. **Cross-platform pricing build.** `pricing_unix.go` carries the only
functional `loadPricing` (uses `syscall.O_NOFOLLOW` and `f.Stat()` on an
open descriptor — both Unix-only). `pricing_other.go` is a build-tag
fallback that returns `"not supported on this platform"`
([pricing_other.go:1416](../../../proxy/internal/llm/pricing/pricing_other.go)).
The proxy is Linux-only in production today; a Windows port needs an
equivalent path-as-handle implementation. Reviewers building on Windows
should expect this surface to return an error at startup if an override
file is configured.
1. **The pricing package is pure and platform-independent.** No file I/O, no
`//go:embed`, no goroutines, no build tags — the rates arrive as config, so
there is nothing platform-specific left to port. Anything reintroducing a
read-from-disk path here re-splits pricing authority between management and
the proxy, which is exactly what this design removed.
2. **SSE scanner handles partial chunks.** A buffered prefix that doesn't end
in `\n\n` still yields its accumulated event before `io.EOF`
@@ -298,38 +327,45 @@ emits `cost.skipped=unknown_model`.
usage rather than aborting
([streaming.go:6873, 144150](../../../proxy/internal/middleware/builtin/llm_response_parser/streaming.go)).
3. **`defaults_pricing.yaml` is the source of truth.** Compiled into the
binary via `//go:embed`
([pricing.go:2930](../../../proxy/internal/llm/pricing/pricing.go)).
`DefaultTable()` parses once and panics on parse failure
([pricing.go:4249](../../../proxy/internal/llm/pricing/pricing.go))
— by design: a broken embedded YAML must not ship to production.
3. **Management is the only source of rates.** `Table` has no constructor that
invents prices: the only way in is `NewTable`/`NewEntries` over the wire map
management sent. A missing or empty `pricing` block therefore means *no
prices at all* (`cost_meter` records `cost.skipped=unknown_model`, $0) —
never a stale built-in fallback that would silently bill list price.
4. **Loader path validation.** `resolveMiddlewareDataPath`
([pricing.go:370394](../../../proxy/internal/llm/pricing/pricing.go))
rejects absolute paths, traversal segments, and basenames that fail
`basenameRegex = ^[a-zA-Z0-9._-]+$`. The resolved path must remain
inside `baseDir` even after `filepath.Clean`. Tests:
`TestNewLoader_PathValidation`, `TestNewLoader_PathValidation_Extended`,
`TestNewLoader_SymlinkOutsideBaseDirRejected`, `TestNewLoader_SymlinkRejected`.
4. **Tables are immutable once built.** `Table.entries` is written only in
`NewEntries` and never mutated afterwards, and `cost_meter`'s `perRecord`
map is likewise build-time-only
([pricing.go:4752](../../../proxy/internal/llm/pricing/pricing.go)). This
is what makes the no-reload design safe: a price change arrives as a mapping
push that builds a new middleware instance over a new table, so concurrent
readers can't observe a half-updated price list and no atomic swap or lock
is needed on the hot path.
5. **Unix loader symlink safety.** `O_NOFOLLOW` on open, `f.Stat()` on the
open descriptor (never re-stat by path), `info.Mode().IsRegular()` check,
`io.LimitReader(f, maxPricingBytes+1)` with a final size assertion
([pricing_unix.go:2557](../../../proxy/internal/llm/pricing/pricing_unix.go)).
A mid-read symlink swap is detected because the fstat is on the original
fd. Test: `TestNewLoader_RejectsOversizedFile_FixesM4`.
5. **Rate validation happens at chain-build time, not per request.**
`NewEntries` rejects negative, NaN, and ±Inf rates field by field
([pricing.go:6083](../../../proxy/internal/llm/pricing/pricing.go)), naming
the offending surface/model/field in the error. Management enforces the same
constraints at its API boundary and in its YAML parser, so this is
defense-in-depth — but it means a corrupt push fails loudly at build instead
of producing negative costs on live traffic. Test:
`TestNewTable_ValidatesRates`.
6. **`yaml.NewDecoder(...).KnownFields(true)`**
([pricing.go:397398](../../../proxy/internal/llm/pricing/pricing.go))
rejects YAML files that carry fields not in the schema. A typo in an
operator override file fails loud instead of silently zeroing rates.
6. **New rates must be added to `Entry`, `EntryJSON`, *and* management's
`pricing.Entry` together.** `NewEntries` converts by direct struct
conversion `Entry(e)`
([pricing.go:7678](../../../proxy/internal/llm/pricing/pricing.go)), which
only compiles while the two structs stay field-identical — so the proxy half
is compiler-enforced. The management half is not: a rate added there but not
here unmarshals into nothing and prices that bucket at `InputPer1K`.
## Things to scrutinise
**Correctness.** Verify OpenAI cached-prompt clamp at
[pricing.go:147149](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. `Anthropic.TotalTokens` sums all four
**Correctness.** Verify the OpenAI cached-prompt clamp at
[pricing.go:203206](../../../proxy/internal/llm/pricing/pricing.go)
short-circuits before subtraction. Negative token counts are clamped to zero up
front ([pricing.go:186197](../../../proxy/internal/llm/pricing/pricing.go)) so
no formula can yield a negative cost. `Anthropic.TotalTokens` sums all four
buckets (in + out + cache_read + cache_creation) — downstream dashboards
need to know this differs from `input + output`.
`OpenAIParser.ExtractPrompt` falls through `messages → input → prompt`; a
@@ -338,22 +374,27 @@ noting).
**Security.** `Scanner.maxLine = 1 MiB`; a 2 MiB single-line `data:` event
errors from `Scanner.Next` and both accumulators stop with partial usage.
Pricing file 1 MiB cap is orders of magnitude larger than realistic. Confirm
new schema additions are mirrored in both `pricingFile` and `Entry`;
`KnownFields(true)` will reject silently-typo'd operator overrides
otherwise.
Pricing is no longer file-backed, so the loader's path-traversal / symlink /
oversize surface is gone entirely — the config channel (an authenticated
mapping push from management) is now the only way rates enter the proxy, and
`NewEntries` is the validation boundary on it. A new rate added to management's
`pricing.Entry` but not to `EntryJSON` here is the remaining silent-mispricing
path (see invariant 6).
**Concurrency.** `Loader.table` is `atomic.Pointer[Table]`; readers never
block or see a torn table. `Loader.Reload` is one goroutine, cancelled via
context (`TestLoader_ReloadBackgroundLoopCancellation`). `DefaultTable()`
uses `sync.Once`. Per-call `Scanner` instances mean no shared state across
concurrent response-parser calls.
**Concurrency.** Nothing in this package is shared mutable state: tables are
built once and never written again, so `cost_meter`'s hot path is lock-free by
construction rather than by atomic swap. Per-call `Scanner` instances mean no
shared state across concurrent response-parser calls.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1).
`Scanner.Next` is one `ReadString('\n')` per line. Pricing reload poll 30s.
**Perf.** `Table.Cost` is two map lookups + multiplications, O(1); the
per-provider-record tier adds at most one more lookup. `Scanner.Next` is one
`ReadString('\n')` per line. No background goroutines and no per-request
allocation of pricing state.
**Observability.** Reload failures count via `metric.Int64Counter` keyed
`plugin`; warning log rate-limited at 5 min so a broken file doesn't flood.
**Observability.** A config carrying no `pricing` block logs one warning at
chain-build time (`cost_meter` factory) and then records
`cost.skipped=unknown_model` per request, so an old-management deployment is
visible in both logs and the access log rather than quietly reporting $0.
Parser errors return sentinels — middleware uses `errors.Is` to map to the
right `cost.skipped` reason.
@@ -365,7 +406,7 @@ right `cost.skipped` reason.
| `openai_test.go` | 11 | Chat Completions + Responses API + legacy `prompt`; cached-tokens subset for both naming conventions; fixture replays |
| `anthropic_test.go` | 7 | Messages + legacy `/v1/complete`; streaming REJECTED on `ParseResponse` (must use scanner); fixture replays |
| `sse_test.go` | 12 | Fixture replay both providers; multiline `data:`; CRLF; comment skip; trailing-event-without-blank-line; oversize rejection |
| `pricing/pricing_test.go` | 21 | Provider-shape switch; cached-rate fallback; cached-clamp; symlink rejection (target outside basedir + symlink to file); path validation matrix; oversize rejection; reload-keeps-previous-on-parse-error; mtime change detection; goroutine cancellation |
| `pricing/pricing_test.go` | 10 | Provider-shape switch (surface selects the formula); cached-rate + cache-read/creation fallback to `InputPer1K`; cached-clamp; negative-token clamp; nil-receiver safety; rate validation (negative / NaN / Inf rejected); nil + empty table |
**Fixtures** ([proxy/internal/llm/fixtures/](../../../proxy/internal/llm/fixtures/)):
`openai_chat_completion.json` (chat.completions with usage),
@@ -373,14 +414,15 @@ right `cost.skipped` reason.
`openai_stream.txt` (3 deltas + usage + `[DONE]`),
`anthropic_messages.json` (Messages API non-streaming),
`anthropic_stream.txt` (full 7-event sequence: message_start →
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop),
`pricing.yaml` (realistic-pricing starter for operator overrides).
content_block_{start,delta×2,stop} → message_delta (usage) → message_stop).
No pricing fixture: the table is config-delivered, so pricing tests construct
it in-process from a wire-shape map.
## Cross-references
- Sibling: [31-proxy-middleware-builtin.md](./31-proxy-middleware-builtin.md)
— the chain that calls `llm.Parsers()`, `llm.ParserByName`,
`llm.NewScanner`, `pricing.NewLoader`.
`llm.NewScanner`, `pricing.NewTable` / `pricing.NewEntries`.
- Path-routed providers (Vertex AI + Bedrock), credential syntax, and the
Bedrock AWS event-stream accumulator:
[50-path-routed-providers.md](./50-path-routed-providers.md).

View File

@@ -1,7 +1,7 @@
# proxy/runtime — translate + serve + log
> **Risk level:** High — every config push from management is translated here, and the chain runs on every HTTP request to a synth target.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareDataDir`, `MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path.
> **Backward-compat impact:** Additive at the wire (`PathTargetOptions.middlewares`, `agent_network`, `disable_access_log`, capture caps) and on the proxy `Server` struct (`MiddlewareCaptureBudgetBytes`). Non-agent-network targets stay on the no-middleware fast path. Middleware config is entirely wire-delivered — no proxy-side data dir is involved, including for LLM pricing, which management ships inside `cost_meter`'s config.
## Module boundary
@@ -114,8 +114,7 @@ At **request time** the access-log middleware stamps `CapturedData`; the auth ch
## Public contracts touched
- `proxy.Server.MiddlewareDataDir` (string) — base dir for file-backed middleware config (server.go:238-241).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:248-250).
- `proxy.Server.MiddlewareCaptureBudgetBytes` (int64) — process-wide capture cap; defaults to 256 MiB (server.go:249-253). There is no `MiddlewareDataDir`: no built-in middleware reads config from disk, so `builtin.FactoryContext` carries only the proxy-lifetime context, meter, logger, and management client.
- `proxy/internal/proxy.WithMiddlewareManager(*middleware.Manager) Option` — new option on `NewReverseProxy`; nil keeps the fast path (reverseproxy.go:48-56).
- `proxy/internal/proxy.PathTarget` adds `Middlewares`, `CaptureConfig`, `AgentNetwork`, `DisableAccessLog` (servicemapping.go:27-51), all zero-default.
- `proxy/internal/proxy.CapturedData` adds `agentNetwork`, `suppressAccessLog`, `userGroupNames` behind `sync.RWMutex`; slices deep-copied (context.go:47-66, 183-258).

View File

@@ -87,9 +87,9 @@ strips the `@version` suffix from the model, and maps the publisher to a parser
surface via `vertexPublisherVendor`:
- `anthropic``llm.provider="anthropic"` → metered through the Anthropic
parser, priced under the **`anthropic`** block in `defaults_pricing.yaml`
(the parser emits the standard Anthropic provider label, so Vertex Claude
reuses first-party Anthropic prices).
parser, priced under the **`anthropic`** surface of the pricing table
management ships (the parser emits the standard Anthropic provider label, so
Vertex Claude reuses first-party Anthropic prices).
- `openai``llm.provider="openai"` (reserved; not in the catalog lineup
today).
- anything else (notably `google` / Gemini) → empty vendor → **no parser**.
@@ -104,8 +104,9 @@ is omitted from the catalog.
> Caveat: cross-region inference profiles in `eu` / `apac` carry a ~10% price
> premium that the base per-token rates do **not** model — cost annotations for
> those regions read low. Operators who need exact regional billing override
> the affected entries in `pricing.yaml`.
> those regions read low. Operators who need exact regional billing set the
> affected models' prices on the provider record, or replace the default entries
> via management's `AgentNetwork.PricingDefaultsFile`.
## AWS Bedrock (`bedrock_api`)
@@ -211,15 +212,19 @@ so a model-listing call can't be rewritten onto an upstream that would 404 it.
## Catalog ↔ pricing cross-check
Catalog prices and context windows are cross-checked against LiteLLM's
`model_prices_and_context_window.json`. The proxy's embedded
`defaults_pricing.yaml` covers **every metered first-party model** the catalog
enumerates — guarded by
`TestDefaultTable_FirstPartyModelCoverage`
([pricing/defaults_coverage_test.go](../../../proxy/internal/llm/pricing/defaults_coverage_test.go)),
which fails if a catalog model has no embedded price. Bedrock entries are keyed
by the **normalised** id the request parser emits (region prefix + version
suffix stripped). Vertex Claude carries no Bedrock-style prefix, so it prices
straight off the `anthropic` block.
`model_prices_and_context_window.json`. The **catalog is the source of default
prices**: management's `pricing.DefaultTable` folds every catalog provider's
models into the surfaces that provider declares (`PricingSurfaces`), so coverage
is structural rather than maintained in a parallel file
([pricing/defaults.go](../../../management/internals/modules/agentnetwork/pricing/defaults.go)).
`TestDefaultTable_CoversEveryCatalogModel` fails if a catalog model ends up
unpriced, and `TestDefaultTable_NoConflictingContributions` fails if two
providers contribute the same (surface, model) at different rates. Bedrock
entries are keyed by the **normalised** id the request parser emits (region
prefix + version suffix stripped) — management applies the same normalisation to
per-provider prices at synth time, so the two keys compare equal. Vertex Claude
carries no Bedrock-style prefix, so it prices straight off the `anthropic`
surface.
## Things to scrutinise
@@ -232,16 +237,17 @@ operator-misconfigured Vertex provider and unmetered Gemini traffic; verify
publishers).
**Correctness.** `normalizeBedrockModel` is the join between the wire id and the
pricing key — a model that normalises to something not in `defaults_pricing.yaml`
meters at `cost.skipped=unknown_model` rather than failing the request. The
pricing key — a model that normalises to something absent from the shipped
pricing table meters at `cost.skipped=unknown_model` rather than failing the
request. The
`/bedrock` prefix strip must run on both the parser side (so the model is
extracted) and the router side (so the upstream path is native); a regression in
either silently breaks the other.
**Metering caveats.** eu/apac cross-region Bedrock + Vertex profiles carry a
~10% premium not modelled by base pricing — flagged in both the catalog comment
and `defaults_pricing.yaml`. Operators needing exact regional billing override
the relevant entries.
~10% premium not modelled by base pricing — flagged in the catalog comment.
Operators needing exact regional billing set per-provider prices on the model
rows (or replace the default entries via `AgentNetwork.PricingDefaultsFile`).
## Cross-references

View File

@@ -6,7 +6,7 @@
"name": "NetBird GmbH",
"email": "hello@netbird.io",
"phone": "",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open-source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open-source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"description": "NetBird GmbH is a Berlin-based software company specializing in the development of open source network security solutions. Network security is utterly complex and expensive, accessible only to companies with multi-million dollar IT budgets. In contrast, there are millions of companies left behind. Our mission is to create an advanced network and cybersecurity platform that is both easy-to-use and affordable for teams of all sizes and budgets. By leveraging the open source strategy and technological advancements, NetBird aims to set the industry standard for connecting and securing IT infrastructure.",
"webpageUrl": {
"url": "https://github.com/netbirdio"
}
@@ -15,7 +15,7 @@
{
"guid": "netbird",
"name": "NetBird",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open-source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"description": "NetBird is a configuration-free peer-to-peer private network and a centralized access control system combined in a single open source platform. It makes it easy to create secure WireGuard-based private networks for your organization or home.",
"webpageUrl": {
"url": "https://github.com/netbirdio/netbird"
},
@@ -59,7 +59,7 @@
"guid": "support-yearly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Yearly",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "yearly",
@@ -72,7 +72,7 @@
"guid": "support-one-time-year",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Year",
"description": "This will help us partially cover the yearly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the yearly cost of maintaining the open source NetBird project.",
"amount": 100000,
"currency": "USD",
"frequency": "one-time",
@@ -85,7 +85,7 @@
"guid": "support-one-time-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - Monthly",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",
@@ -98,7 +98,7 @@
"guid": "support-monthly",
"status": "active",
"name": "Support Open Source Development and Maintenance - One Month",
"description": "This will help us partially cover the monthly cost of maintaining the open-source NetBird project.",
"description": "This will help us partially cover the monthly cost of maintaining the open source NetBird project.",
"amount": 10000,
"currency": "USD",
"frequency": "monthly",

9
go.mod
View File

@@ -72,17 +72,18 @@ require (
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/go-secure-stdlib/base62 v0.1.2
github.com/hashicorp/go-version v1.7.0
github.com/jackc/pgx/v5 v5.5.5
github.com/jackc/pgx/v5 v5.10.0
github.com/libdns/route53 v1.5.0
github.com/libp2p/go-nat v0.2.0
github.com/libp2p/go-netroute v0.4.0
github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81
github.com/magefile/mage v1.17.2
github.com/mdlayher/socket v0.5.1
github.com/mdp/qrterminal/v3 v3.2.1
github.com/miekg/dns v1.1.72
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/moby/moby/api v1.54.1
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45
github.com/oapi-codegen/runtime v1.1.2
github.com/okta/okta-sdk-golang/v2 v2.18.0
@@ -236,8 +237,8 @@ require (
github.com/huin/goupnp v1.2.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/puddle/v2 v2.2.1 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect

18
go.sum
View File

@@ -341,12 +341,12 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.5.5 h1:amBjrZVmksIdNjxGW/IiIMzxMKZFelXbUoPNb+8sjQw=
github.com/jackc/pgx/v5 v5.5.5/go.mod h1:ez9gk+OAat140fv9ErkZDYFWmXLfV+++K0uAOiwgm1A=
github.com/jackc/puddle/v2 v2.2.1 h1:RhxXJtFG022u4ibrCSMSiu5aOq1i77R3OHKNJj77OAk=
github.com/jackc/puddle/v2 v2.2.1/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=
github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=
github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
@@ -415,6 +415,8 @@ github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81 h1:J56rFEfUTFT9j9
github.com/lrh3321/ipset-go v0.0.0-20250619021614-54a0a98ace81/go.mod h1:RD8ML/YdXctQ7qbcizZkw5mZ6l8Ogrl1dodBzVJduwI=
github.com/lufia/plan9stats v0.0.0-20240513124658-fba389f38bae h1:dIZY4ULFcto4tAFlj1FYZl8ztUZ13bdq+PLY+NOfbyI=
github.com/lufia/plan9stats v0.0.0-20240513124658-fba389f38bae/go.mod h1:ilwx/Dta8jXAgpFYFvSWEMwxmbWXyiUHkd5FwyKhb5k=
github.com/magefile/mage v1.17.2 h1:fyXVu1eadI8Ap1HCCNgEhJ5McIWiYhLR8uol64ZZc40=
github.com/magefile/mage v1.17.2/go.mod h1:Yj51kqllmsgFpvvSzgrZPK9WtluG3kUhFaBUVLo4feA=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/matryer/is v1.4.1 h1:55ehd8zaGABKLXQUe2awZ99BD/PTc2ls+KV/dXphgEQ=
@@ -482,8 +484,8 @@ github.com/netbirdio/easyjson v0.9.0 h1:6Nw2lghSVuy8RSkAYDhDv1thBVEmfVbKZnV7T7Z6
github.com/netbirdio/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51 h1:Ov4qdafATOgGMB1wbSuh+0aAHcwz9hdvB6VZjh1mVMI=
github.com/netbirdio/ice/v4 v4.0.0-20250908184934-6202be846b51/go.mod h1:ZSIbPdBn5hePO8CpF1PekH2SfpTxg1PDhEwtbqZS7R8=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42 h1:F3zS5fT9xzD1OFLfcdAE+3FfyiwjGukF1hvj0jErgs8=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260416123949-2355d972be42/go.mod h1:n47r67ZSPgwSmT/Z1o48JjZQW9YJ6m/6Bd/uAXkL3Pg=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87 h1:iJeUvSMC0BTpkw7u4JyWcY4/3dl7fEL9DR/TpKf2+1w=
github.com/netbirdio/management-integrations/integrations v0.0.0-20260803100840-78e79ba20f87/go.mod h1:pmsCPx1S0nuZRxCextGpc9AV4hLgGSuTsc4NMuwGeCo=
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502 h1:3tHlFmhTdX9axERMVN63dqyFqnvuD+EMJHzM7mNGON8=
github.com/netbirdio/service v0.0.0-20240911161631-f62744f42502/go.mod h1:CIMRFEJVL+0DS1a3Nx06NaMn4Dz63Ng6O7dl0qH0zVM=
github.com/netbirdio/signal-dispatcher/dispatcher v0.0.0-20250805121659-6b4ac470ca45 h1:ujgviVYmx243Ksy7NdSwrdGPSRNE3pb8kEDSpH0QuAQ=

View File

@@ -221,9 +221,6 @@ render_override() {
# Remove this file (and config.yaml.enterprise if present) to revert.
services:
${DASHBOARD_SERVICE}:
image: \${NETBIRD_DASHBOARD_IMAGE:-ghcr.io/netbirdio/dashboard-cloud:latest}
${COMBINED_SERVICE}:
image: \${NETBIRD_SERVER_IMAGE:-ghcr.io/netbirdio/netbird-server-cloud:latest}
environment:

View File

@@ -0,0 +1,58 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"time"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetAccountSettings(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into accounts (id, settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
values('account-3',null,null,null,null,null,null,null,null,null,null,null)`)
accountSettings, err := conn(t, ctx).GetAccountSettings(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
PeerLoginExpirationEnabled: true,
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
PeerInactivityExpirationEnabled: false,
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
DNSDomain: "",
IPv6EnabledGroups: []string{"group-one-resource-id"},
RoutingPeerDNSResolutionEnabled: false,
LazyConnectionEnabled: false,
AutoUpdateVersion: "disabled",
AutoUpdateAlways: false,
MetricsPushEnabled: false,
})
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{
PeerLoginExpirationEnabled: true,
PeerLoginExpiration: 86400000000000 * time.Nanosecond,
PeerInactivityExpirationEnabled: false,
PeerInactivityExpiration: 86400000000000 * time.Nanosecond,
DNSDomain: "",
IPv6EnabledGroups: []string{"group-two-resources-id"},
RoutingPeerDNSResolutionEnabled: false,
LazyConnectionEnabled: false,
AutoUpdateVersion: "disabled",
AutoUpdateAlways: false,
MetricsPushEnabled: false,
})
accountSettings, err = conn(t, ctx).GetAccountSettings(ctx, "account-3")
assert.NoError(t, err)
assert.Equal(t, accountSettings, nmdata.AccountSettingsInfo{})
}

View File

@@ -0,0 +1,53 @@
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
VALUES('account-1','network-1','{"IP":"100.103.0.0","Mask":"//8AAA=="}','{"IP":"fdde:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',1,'["disabled-group-1","disabled-group-2"]',
true, 86400000000000, false,
86400000000000, null, '["group-one-resource-id"]', false,
false, 'disabled', false, false);
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
VALUES('account-2','network-2','{"IP":"110.0.0.0","Mask":"//8AAA=="}','{"IP":"fddf:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',2,null,
true, 86400000000000, false,
86400000000000, null, '["group-two-resources-id"]', false,
false, 'disabled', false, false);
insert into groups (id, account_id, name, resources, public_id) VALUES('group-one-resource-id','account-1','group-1-name', '[{"ID":"host-id-1","Type":"host"}]','group-one-resource-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('group-two-resources-id','account-1','group-2-name', '[{"ID":"subnet-id-1","Type":"subnet"}, {"ID":"host-id-2","Type":"host"}]','group-two-resources-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('group-no-resources-id','account-1','group-3-name', null,'group-no-resources-id-public');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-1','group-one-resource-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-2','group-two-resources-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-1','peer-id-3','group-two-resources-id');
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-1','account-1','key-1','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-1.netbird.services',
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.148.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-2','account-1','key-2','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-2.netbird.services',
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
'DE','Berlin','"46.201.149.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-3','account-1','key-3','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-3.netbird.services',
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.150.187"');

View File

@@ -0,0 +1,25 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetDnsSettings(t *testing.T) {
ctx := context.TODO()
settings, err := conn(t, ctx).GetDnsSettings(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, settings, nmdata.DNSSettings{
DisabledManagementGroups: []string{"disabled-group-1", "disabled-group-2"},
})
settings, err = conn(t, ctx).GetDnsSettings(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, settings, nmdata.DNSSettings{})
}

View File

@@ -0,0 +1,62 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/miekg/dns"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetAppliedZoneCandidatesViaPgxConnection(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
VALUES('zone-1','account-1','test-1.com',true,'["group-one-resource-id"]')`)
execQuery(t, ctx,
`insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
VALUES('zone-2','account-1','test-2.com',false,'["group-two-resources-id"]')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-1','account-1','zone-1','test.test-1.com','A',1800,'1.1.1.1')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-2','account-1','zone-1','test2.test-1.com','A',1800,'1.1.1.2')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-3','account-1','zone-1','test3.test-1.com','CNAME',1800,'test4.test-1.com')`)
execQuery(t, ctx,
`insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-4','account-1','zone-2','test2.test-2.com','CNAME',1800,'test3.test-2.com')`)
zoneCandidates, err := conn(t, ctx).GetAppliedZoneCandidates(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
DistributionGroups: []string{"group-one-resource-id"},
Zone: nmdata.CustomZone{
Domain: "test-1.com",
SearchDomainDisabled: false,
Records: []nmdata.SimpleRecord{
{Name: "test.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.1"},
{Name: "test2.test-1.com", Type: int(dns.TypeA), Class: "IN", TTL: 1800, RData: "1.1.1.2"},
{Name: "test3.test-1.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test4.test-1.com."},
},
},
})
assert.Contains(t, zoneCandidates, networkmap.AppliedZoneCandidate{
DistributionGroups: []string{"group-two-resources-id"},
Zone: nmdata.CustomZone{
Domain: "test-2.com",
SearchDomainDisabled: true,
Records: []nmdata.SimpleRecord{
{Name: "test2.test-2.com", Type: int(dns.TypeCNAME), Class: "IN", TTL: 1800, RData: "test3.test-2.com."},
},
},
})
}

View File

@@ -0,0 +1,39 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"database/sql"
"testing"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/stretchr/testify/assert"
)
func TestGetDomains(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-1','account-1','test-1.com','target-1.cluster.local')`)
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-2','account-1','test-2.com','target-2.cluster.local')`)
execQuery(t, ctx,
`insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-3','account-1',null,null)`)
domains, err := conn(t, ctx).GetDomains(ctx, "account-1")
assert.NoError(t, err)
assert.Len(t, domains, 2)
assert.Contains(t, domains, networkmapdb.Domain{
Domain: sql.NullString{String: "test-1.com", Valid: true},
TargetCluster: sql.NullString{String: "target-1.cluster.local", Valid: true},
})
assert.Contains(t, domains, networkmapdb.Domain{
Domain: sql.NullString{String: "test-2.com", Valid: true},
TargetCluster: sql.NullString{String: "target-2.cluster.local", Valid: true},
})
}

View File

@@ -0,0 +1,58 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/management/server/types"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGetGroups(t *testing.T) {
ctx := context.TODO()
groups, resourceToGroupIdx, err := conn(t, ctx).GetGroups(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t,
groups,
nmdata.Group{ID: "group-one-resource-id", Name: "group-1-name", PublicID: "group-one-resource-id-public", Resources: []nmdata.Resource{{ID: "host-id-1", Type: "host"}}, Peers: []string{"peer-id-1"}},
)
assert.NotNil(t, resourceToGroupIdx["host-id-1"]["group-one-resource-id"])
assert.Contains(t,
groups,
nmdata.Group{ID: "group-two-resources-id", Name: "group-2-name", PublicID: "group-two-resources-id-public",
Resources: []nmdata.Resource{{ID: "subnet-id-1", Type: "subnet"}, {ID: "host-id-2", Type: "host"}},
Peers: []string{"peer-id-2", "peer-id-3"}},
)
assert.NotNil(t, resourceToGroupIdx["host-id-2"]["group-two-resources-id"])
assert.NotNil(t, resourceToGroupIdx["subnet-id-1"]["group-two-resources-id"])
assert.Contains(t,
groups,
nmdata.Group{ID: "group-no-resources-id", Name: "group-3-name", PublicID: "group-no-resources-id-public"})
}
// Verify handling of empty fields in groups table
// Verify that group's PublicID gets populated on retrieval
// TODO (dmitri) PublicID should not be populated with delta updates,
// which require stable PublicIDs
func TestGetGroupsWithoutExpectedFields(t *testing.T) {
if engine == string(types.SqliteStoreEngine) {
t.Skip()
}
ctx := context.TODO()
execQuery(t, ctx,
"insert into accounts (id) VALUES('random-id')")
execQuery(t, ctx,
"insert into groups (id, account_id) VALUES('g2-test-group-id-1','random-id')")
groups, _, err := conn(t, ctx).GetGroups(ctx, "random-id")
assert.NoError(t, err)
require.Len(t, groups, 1)
assert.NotEmpty(t, groups[0].PublicID)
}

View File

@@ -0,0 +1,99 @@
//go:build integration
package networkmap_pgsql
import (
"context"
_ "embed"
"os"
"testing"
"time"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
"github.com/netbirdio/netbird/management/server/types"
)
//go:embed base_data.sql
var baseData string
var (
pgstore *networkmap_pgsql.PgStore
sqlitestore *networkmap_sqlite.SqliteStore
engine string
)
func TestMain(m *testing.M) {
var cleanup func()
kind, _ := os.LookupEnv("NETBIRD_STORE_ENGINE")
switch kind {
case string(types.PostgresStoreEngine):
engine = string(types.PostgresStoreEngine)
pgstore, cleanup = createPGTestStore(baseData)
pgstore.UsingTimeZone(time.UTC)
case "", string(types.SqliteStoreEngine):
engine = string(types.SqliteStoreEngine)
sqlitestore, cleanup = createSqliteTestStore(baseData)
default:
log.Fatalf("unsupported db '%s' in NETBIRD_STORE_ENGINE env var", kind)
}
code := m.Run()
cleanup()
os.Exit(code)
}
func conn(t *testing.T, ctx context.Context) networkmapdb.NetworkMapDBStoreConn {
t.Helper()
switch engine {
case string(types.PostgresStoreEngine):
c, err := pgstore.Pool.Acquire(ctx)
assert.NoError(t, err)
return pgstore.UsingConnection(c.Conn())
case string(types.SqliteStoreEngine):
return sqlitestore.UsingConn()
}
log.Fatalf("unknown db engine kind %s", engine)
return nil
}
func store(t *testing.T) networkmapdb.NetworkMapDBStore {
t.Helper()
switch engine {
case string(types.PostgresStoreEngine):
return pgstore
case string(types.SqliteStoreEngine):
return sqlitestore
}
log.Fatalf("unknown db engine kind %s", engine)
return nil
}
func execQuery(t *testing.T, ctx context.Context, q string) {
t.Helper()
switch engine {
case string(types.PostgresStoreEngine):
_, err := pgstore.Pool.Exec(ctx, q)
assert.NoError(t, err)
case string(types.SqliteStoreEngine):
_, err := sqlitestore.Db.ExecContext(ctx, q)
assert.NoError(t, err)
}
}
// use to parse time in time.RFC3339Nano format
// returns the time in the UTC time zone
func mustParseTime(t string) *time.Time {
tt, err := time.Parse(time.RFC3339Nano, t)
if err != nil {
panic(err)
}
utc := tt.UTC()
return &utc
}

View File

@@ -0,0 +1,61 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNameServerGroups(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled, "primary", account_id)
VALUES('nsgroup-1','nsgroup-1-public','nsgroup-1','nsgroup-1','[{"IP":"192.168.31.2","NSType":1,"Port":53}]','["group-one-resource-id"]','["test-1.com"]',TRUE,FALSE,TRUE,'account-1')`)
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
VALUES('nsgroup-2','nsgroup-2-public','nsgroup-2','nsgroup-2','[{"IP":"192.168.32.3","NSType":1,"Port":53}]','["group-one-resource-id","group-no-resources-id"]','["test-1.com","test-2.com"]',TRUE,FALSE,TRUE,'account-1')`)
execQuery(t, ctx,
`insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
VALUES('nsgroup-3','nsgroup-3-public',null,null,null,null,null,TRUE,FALSE,FALSE,'account-1')`)
nsgroups, err := conn(t, ctx).GetNameServerGroups(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-1",
PublicID: "nsgroup-1-public",
Name: "nsgroup-1",
Description: "nsgroup-1",
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.31.2"), NSType: 1, Port: 53}},
Groups: []string{"group-one-resource-id"},
Domains: []string{"test-1.com"},
Primary: true,
SearchDomainsEnabled: false,
Enabled: true,
})
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-2",
PublicID: "nsgroup-2-public",
Name: "nsgroup-2",
Description: "nsgroup-2",
NameServers: []nmdata.NameServer{{IP: netip.MustParseAddr("192.168.32.3"), NSType: 1, Port: 53}},
Groups: []string{"group-one-resource-id", "group-no-resources-id"},
Domains: []string{"test-1.com", "test-2.com"},
Primary: true,
SearchDomainsEnabled: false,
Enabled: true,
})
assert.Contains(t, nsgroups, nmdata.NameServerGroup{
ID: "nsgroup-3",
PublicID: "nsgroup-3-public",
Primary: false,
SearchDomainsEnabled: false,
Enabled: true,
})
}

View File

@@ -0,0 +1,98 @@
insert into accounts (id, network_identifier, network_net, network_net_v6, network_dns, network_serial,dns_settings_disabled_management_groups,
settings_peer_login_expiration_enabled, settings_peer_login_expiration, settings_peer_inactivity_expiration_enabled,
settings_peer_inactivity_expiration, settings_dns_domain, settings_ipv6_enabled_groups, settings_routing_peer_dns_resolution_enabled,
settings_lazy_connection_enabled, settings_auto_update_version, settings_auto_update_always, settings_metrics_push_enabled)
VALUES('account-33','network-331','{"IP":"100.103.0.0","Mask":"//8AAA=="}','{"IP":"fdde:e995:fd38:a465::","Mask":"//////////8AAAAAAAAAAA=="}','',1,'["disabled-group-1","disabled-group-2"]',
true, 86400000000000, false,
86400000000000, null, '["33-group-one-resource-id"]', false,
false, 'disabled', false, false);
insert into groups (id, account_id, name, resources, public_id) VALUES('33-group-one-resource-id','account-33','group-1-name', '[{"ID":"host-id-1","Type":"host"}]','group-one-resource-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('33-group-two-resources-id','account-33','group-2-name', '[{"ID":"subnet-id-1","Type":"subnet"}, {"ID":"host-id-2","Type":"host"}]','33-group-two-resources-id-public');
insert into groups (id, account_id, name, resources, public_id) VALUES('33-group-no-resources-id','account-33','group-3-name', null,'33-group-no-resources-id-public');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-33','peer-id-331','33-group-one-resource-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-33','peer-id-332','33-group-two-resources-id');
insert into group_peers (account_id, peer_id, group_id) VALUES('account-33','peer-id-333','33-group-two-resources-id');
insert into peers (id, account_id, "key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-331','account-33','key-331','ssh-key-1','peer-1','["extra-peer-1"]','user-id-1',true,true,'2026-08-06 13:25:59.12999','"10.10.10.1"','"fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-1.netbird.services',
'0.76.0','linux','26.4.1','6.8.0-134-generic','[{"NetIP":"fe80::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ac"},{"NetIP":"192.168.16.1/20","Mac":"00:15:5d:24:0c:ac"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.148.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-332','account-33','key-332','ssh-key-2','peer-2','["extra-peer-2"]','user-id-2',true,true,'2026-08-06 14:25:59.12999','"10.10.100.1"','"fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-2.netbird.services',
'0.76.1','linux','26.4.2','6.8.0-135-generic','[{"NetIP":"fe81::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ad"},{"NetIP":"192.168.17.1/20","Mac":"00:15:5d:24:0c:ad"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',0,
'DE','Berlin','"46.201.149.187"');
insert into peers (id,account_id,"key", ssh_key, dns_label, extra_dns_labels, user_id, ssh_enabled, login_expiration_enabled, last_login, ip, ipv6,
peer_status_requires_approval, peer_status_connected, proxy_meta_embedded, proxy_meta_cluster,
meta_wt_version, meta_go_os, meta_os_version, meta_kernel_version, meta_network_addresses, meta_files,
meta_capabilities, meta_flags, meta_sync_message_version,
location_country_code, location_city_name, location_connection_ip)
values('peer-id-333','account-33','key-333','ssh-key-3','peer-3','["extra-peer-3"]','user-id-3',true,true,'2026-08-06 12:25:59.12999','"10.10.200.1"','"fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"',
false,true,true,'cluster-3.netbird.services',
'0.76.2','linux','26.4.3','6.8.0-136-generic','[{"NetIP":"fe82::8b4c:973f:a76b:3771/64","Mac":"00:15:5d:24:0c:ae"},{"NetIP":"192.168.18.1/20","Mac":"00:15:5d:24:0c:ae"}]','[{"Path":"/usr/bin/netbird","Exist":false,"ProcessIsRunning":false}]',
'[1,2]','{"RosenpassEnabled":false,"RosenpassPermissive":false,"ServerSSHAllowed":true,"DisableClientRoutes":false,"DisableServerRoutes":false,"DisableDNS":false,"DisableFirewall":false,"BlockLANAccess":false,"BlockInbound":false,"DisableIPv6":false,"LazyConnectionEnabled":false}',1,
'DE','Berlin','"46.201.150.187"');
insert into zones (id, account_id, domain, enable_search_domain, distribution_groups)
VALUES('zone-331','account-33','test-331.com',true,'["33-group-one-resource-id"]');
insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-331','account-33','zone-331','test.test-331.com','A',1800,'1.1.1.1');
insert into records (id, account_id, zone_id, name, type, ttl, content)
VALUES('record-332','account-33','zone-331','test2.test-331.com','A',1800,'1.1.1.2');
insert into domains (id, account_id, domain, target_cluster)
VALUES('domain-331','account-33','test-331.com','target-1.cluster.local');
insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled, "primary", account_id)
VALUES('nsgroup-331','nsgroup-1-public','nsgroup-1','nsgroup-1','[{"IP":"192.168.31.2","NSType":1,"Port":53}]','["33-group-one-resource-id"]','["test-1.com"]',TRUE,FALSE,TRUE,'account-33');
insert into name_server_groups (id, public_id, name, description, name_servers, groups, domains, enabled, search_domains_enabled,"primary",account_id)
VALUES('nsgroup-332','nsgroup-2-public','nsgroup-2','nsgroup-2','[{"IP":"192.168.32.3","NSType":1,"Port":53}]','["33-group-one-resource-id","33-group-no-resources-id"]','["test-1.com","test-2.com"]',TRUE,FALSE,TRUE,'account-33');
insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-331','account-33','network-331','net-resource-public-1','network-resource-1','network-resource-1','subnet','','"10.0.0.0/16"',TRUE);
insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-332','account-33','network-332','net-resource-public-2','network-resource-2','network-resource-2','domain','test.com','',TRUE);
insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-331','account-33','public-id-1','peer-id-331','network-id-1',TRUE,999,TRUE,'["33-group-one-resource-id"]');
insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-332','account-33','public-id-2','','network-id-2',TRUE,333,TRUE,'["33-group-two-resources-id","33-group-no-resources-id"]');
insert into networks (id, account_id, public_id) VALUES('network-331','account-33','network-1-public');
insert into networks (id, account_id, public_id) VALUES('network-332','account-33','network-2-public');
insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-331','policy-1-public','account-33',true,'["posture-checks-1","posture-checks-2"]');
insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-331-rule-1','policy-331',true,'accept','tcp',true,'["33-group-one-resource-id","33-group-two-resources-id"]','["33-group-one-resource-id","33-group-two-resources-id"]',
'{"ID":"host-id-1","Type":"host"}','{"ID":"domain-331","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
'{"33-group-one-resource-id":["user-1", "user-2"]}','user-3');
insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-331','account-33','posturecheck-1-public',
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
"OSVersionCheck":{"Darwin":{"MinVersion":"12.0"}},
"GeoLocationCheck":{"Locations":[{"CountryCode":"FI","CityName":""}],"Action":"allow"},
"PeerNetworkRangeCheck":{"Action":"deny","Ranges":["192.168.0.1/24"]}}');
insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-331','account-33','route-1-public','"172.0.0.0/16"','["test-1.com"]',true,'route-331-net-id','route-1',
'peer-id-331','["33-group-one-resource-id"]',1,true,9999,true,
'["33-group-one-resource-id"]','["33-group-one-resource-id"]',false);
insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-331','account-33',true,true,'["33-group-one-resource-id"]','test-1.com','test-332.com');

View File

@@ -0,0 +1,516 @@
{
"Peers": {
"peer-id-331": {
"ID": "peer-id-331",
"Key": "key-331",
"SSHKey": "ssh-key-1",
"DNSLabel": "peer-1",
"UserID": "user-id-1",
"SSHEnabled": true,
"LoginExpirationEnabled": true,
"LastLogin": "2026-08-06T13:25:59.12999Z",
"IP": "10.10.10.1",
"IPv6": "fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940",
"RequiresApproval": false,
"ExtraDNSLabels": [
"extra-peer-1"
],
"Meta": {
"WtVersion": "0.76.0",
"GoOS": "linux",
"OSVersion": "26.4.1",
"KernelVersion": "6.8.0-134-generic",
"NetworkAddresses": [
{
"NetIP": "fe80::8b4c:973f:a76b:3771/64"
},
{
"NetIP": "192.168.16.1/20"
}
],
"Files": [
{
"Path": "/usr/bin/netbird",
"ProcessIsRunning": false
}
],
"Capabilities": [
1,
2
],
"Flags": {
"ServerSSHAllowed": true,
"DisableIPv6": false
},
"SyncMessageVersion": 1
},
"ProxyMeta": {
"Embedded": true
},
"Location": {
"CountryCode": "DE",
"CityName": "Berlin",
"ConnectionIP": "46.201.148.187"
}
},
"peer-id-332": {
"ID": "peer-id-332",
"Key": "key-332",
"SSHKey": "ssh-key-2",
"DNSLabel": "peer-2",
"UserID": "user-id-2",
"SSHEnabled": true,
"LoginExpirationEnabled": true,
"LastLogin": "2026-08-06T14:25:59.12999Z",
"IP": "10.10.100.1",
"IPv6": "fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940",
"RequiresApproval": false,
"ExtraDNSLabels": [
"extra-peer-2"
],
"Meta": {
"WtVersion": "0.76.1",
"GoOS": "linux",
"OSVersion": "26.4.2",
"KernelVersion": "6.8.0-135-generic",
"NetworkAddresses": [
{
"NetIP": "fe81::8b4c:973f:a76b:3771/64"
},
{
"NetIP": "192.168.17.1/20"
}
],
"Files": [
{
"Path": "/usr/bin/netbird",
"ProcessIsRunning": false
}
],
"Capabilities": [
1,
2
],
"Flags": {
"ServerSSHAllowed": true,
"DisableIPv6": false
},
"SyncMessageVersion": 0
},
"ProxyMeta": {
"Embedded": true
},
"Location": {
"CountryCode": "DE",
"CityName": "Berlin",
"ConnectionIP": "46.201.149.187"
}
},
"peer-id-333": {
"ID": "peer-id-333",
"Key": "key-333",
"SSHKey": "ssh-key-3",
"DNSLabel": "peer-3",
"UserID": "user-id-3",
"SSHEnabled": true,
"LoginExpirationEnabled": true,
"LastLogin": "2026-08-06T12:25:59.12999Z",
"IP": "10.10.200.1",
"IPv6": "fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940",
"RequiresApproval": false,
"ExtraDNSLabels": [
"extra-peer-3"
],
"Meta": {
"WtVersion": "0.76.2",
"GoOS": "linux",
"OSVersion": "26.4.3",
"KernelVersion": "6.8.0-136-generic",
"NetworkAddresses": [
{
"NetIP": "fe82::8b4c:973f:a76b:3771/64"
},
{
"NetIP": "192.168.18.1/20"
}
],
"Files": [
{
"Path": "/usr/bin/netbird",
"ProcessIsRunning": false
}
],
"Capabilities": [
1,
2
],
"Flags": {
"ServerSSHAllowed": true,
"DisableIPv6": false
},
"SyncMessageVersion": 1
},
"ProxyMeta": {
"Embedded": true
},
"Location": {
"CountryCode": "DE",
"CityName": "Berlin",
"ConnectionIP": "46.201.150.187"
}
}
},
"Groups": {
"33-group-no-resources-id": {
"ID": "33-group-no-resources-id",
"Name": "group-3-name",
"PublicID": "33-group-no-resources-id-public",
"Peers": null,
"Resources": null
},
"33-group-one-resource-id": {
"ID": "33-group-one-resource-id",
"Name": "group-1-name",
"PublicID": "group-one-resource-id-public",
"Peers": [
"peer-id-331"
],
"Resources": [
{
"ID": "host-id-1",
"Type": "host"
}
]
},
"33-group-two-resources-id": {
"ID": "33-group-two-resources-id",
"Name": "group-2-name",
"PublicID": "33-group-two-resources-id-public",
"Peers": [
"peer-id-332",
"peer-id-333"
],
"Resources": [
{
"ID": "subnet-id-1",
"Type": "subnet"
},
{
"ID": "host-id-2",
"Type": "host"
}
]
}
},
"Policies": [
{
"ID": "policy-331",
"PublicID": "policy-1-public",
"Enabled": true,
"SourcePostureChecks": [
"posture-checks-1",
"posture-checks-2"
],
"Rules": [
{
"ID": "policy-331",
"PolicyID": "policy-331",
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Bidirectional": true,
"Sources": [
"33-group-one-resource-id",
"33-group-two-resources-id"
],
"Destinations": [
"33-group-one-resource-id",
"33-group-two-resources-id"
],
"SourceResource": {
"ID": "host-id-1",
"Type": "host"
},
"DestinationResource": {
"ID": "domain-331",
"Type": "domain"
},
"Ports": [
"8080",
"8443"
],
"PortRanges": [
{
"Start": 8080,
"End": 8090
}
],
"AuthorizedGroups": {
"33-group-one-resource-id": [
"user-1",
"user-2"
]
},
"AuthorizedUser": "user-3"
}
]
}
],
"Routes": [
{
"ID": "route-331",
"AccountID": "account-33",
"PublicID": "route-1-public",
"Network": "172.0.0.0/16",
"Domains": [
"test-1.com"
],
"KeepRoute": true,
"NetID": "route-331-net-id",
"Description": "route-1",
"Peer": "peer-id-331",
"PeerID": "peer-id-331",
"PeerGroups": [
"33-group-one-resource-id"
],
"NetworkType": 1,
"Masquerade": true,
"Metric": 9999,
"Enabled": true,
"Groups": [
"33-group-one-resource-id"
],
"AccessControlGroups": [
"33-group-one-resource-id"
],
"SkipAutoApply": false
}
],
"NameServerGroups": [
{
"ID": "nsgroup-331",
"PublicID": "nsgroup-1-public",
"Name": "nsgroup-1",
"Description": "nsgroup-1",
"NameServers": [
{
"IP": "192.168.31.2",
"NSType": 1,
"Port": 53
}
],
"Groups": [
"33-group-one-resource-id"
],
"Primary": true,
"Domains": [
"test-1.com"
],
"Enabled": true,
"SearchDomainsEnabled": false
},
{
"ID": "nsgroup-332",
"PublicID": "nsgroup-2-public",
"Name": "nsgroup-2",
"Description": "nsgroup-2",
"NameServers": [
{
"IP": "192.168.32.3",
"NSType": 1,
"Port": 53
}
],
"Groups": [
"33-group-one-resource-id",
"33-group-no-resources-id"
],
"Primary": true,
"Domains": [
"test-1.com",
"test-2.com"
],
"Enabled": true,
"SearchDomainsEnabled": false
}
],
"NetworkResources": [
{
"ID": "net-resource-331",
"NetworkID": "network-331",
"AccountID": "account-33",
"PublicID": "net-resource-public-1",
"Name": "network-resource-1",
"Description": "network-resource-1",
"Type": "subnet",
"Address": "",
"Domain": "",
"Prefix": "10.0.0.0/16",
"Enabled": true
},
{
"ID": "net-resource-332",
"NetworkID": "network-332",
"AccountID": "account-33",
"PublicID": "net-resource-public-2",
"Name": "network-resource-2",
"Description": "network-resource-2",
"Type": "domain",
"Address": "",
"Domain": "test.com",
"Prefix": "",
"Enabled": true
}
],
"Network": {
"Identifier": "network-331",
"Net": {
"IP": "100.103.0.0",
"Mask": "//8AAA=="
},
"NetV6": {
"IP": "fdde:e995:fd38:a465::",
"Mask": "//////////8AAAAAAAAAAA=="
},
"Dns": "",
"Serial": 1
},
"DNSSettings": {
"DisabledManagementGroups": [
"disabled-group-1",
"disabled-group-2"
]
},
"AccountSettings": {
"PeerLoginExpirationEnabled": true,
"PeerLoginExpiration": 86400000000000,
"PeerInactivityExpirationEnabled": false,
"PeerInactivityExpiration": 86400000000000,
"DNSDomain": "",
"IPv6EnabledGroups": [
"33-group-one-resource-id"
],
"RoutingPeerDNSResolutionEnabled": false,
"LazyConnectionEnabled": false,
"AutoUpdateVersion": "disabled",
"AutoUpdateAlways": false,
"MetricsPushEnabled": false
},
"PostureChecks": {
"posturecheck-331": {
"ID": "posturecheck-331",
"Checks": {
"NBVersionCheck": {
"MinVersion": "0.25.0"
},
"OSVersionCheck": {
"Android": null,
"Darwin": {
"MinVersion": "12.0"
},
"Ios": null,
"Linux": null,
"Windows": null
},
"GeoLocationCheck": {
"Locations": [
{
"CountryCode": "FI",
"CityName": ""
}
],
"Action": "allow"
},
"PeerNetworkRangeCheck": {
"Action": "deny",
"Ranges": [
"192.168.0.1/24"
]
},
"ProcessCheck": null
}
}
},
"PostureValidation": null,
"AllowedUserIDs": {},
"NetworkXIDToPublicID": {
"network-331": "network-1-public",
"network-332": "network-2-public"
},
"PostureCheckXIDToPublicID": {
"posturecheck-331": "posturecheck-1-public"
},
"ValidatedPeers": {
"peer-id-1": {},
"peer-id-2": {},
"peer-id-3": {}
},
"ResourcePolicies": {},
"Routers": {
"network-id-1": {
"peer-id-331": {
"PublicID": "public-id-1",
"PeerGroups": [
"33-group-one-resource-id"
],
"Masquerade": true,
"Metric": 999,
"Enabled": true
}
},
"network-id-2": {
"peer-id-332": {
"PublicID": "public-id-2",
"PeerGroups": [
"33-group-two-resources-id",
"33-group-no-resources-id"
],
"Masquerade": true,
"Metric": 333,
"Enabled": true
},
"peer-id-333": {
"PublicID": "public-id-2",
"PeerGroups": [
"33-group-two-resources-id",
"33-group-no-resources-id"
],
"Masquerade": true,
"Metric": 333,
"Enabled": true
}
}
},
"GroupIDToUserIDs": {},
"DNSDomain": "",
"ProxyTargetedDomainResourceIDs": {},
"AppliedZoneCandidates": [
{
"DistributionGroups": [
"33-group-one-resource-id"
],
"Zone": {
"Domain": "test-331.com",
"Records": [
{
"Name": "test.test-331.com",
"Type": 1,
"Class": "IN",
"TTL": 1800,
"RData": "1.1.1.1"
},
{
"Name": "test2.test-331.com",
"Type": 1,
"Class": "IN",
"TTL": 1800,
"RData": "1.1.1.2"
}
],
"SearchDomainDisabled": false,
"NonAuthoritative": false
}
}
],
"PrivateServiceCandidates": null
}

View File

@@ -0,0 +1,73 @@
//go:build integration
package networkmap_pgsql
import (
"context"
_ "embed"
"encoding/json"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"github.com/golang/mock/gomock"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/management/server/integrations/integrated_validator"
"github.com/netbirdio/netbird/management/server/settings"
"github.com/netbirdio/netbird/management/server/types"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
)
//go:embed network_map_data.sql
var nmapData string
//go:embed network_map_data_golden.json
var goldenNMap string
const EnvUpdateGoldenData = "NMAP_UPDATE_GOLDEN_DATA"
func TestGetNetworkMapData(t *testing.T) {
ctx := context.TODO()
ctrl := gomock.NewController(t)
extraSettingsManager := settings.NewMockManager(ctrl)
extraSettingsManager.EXPECT().GetExtraSettings(gomock.Any(), gomock.Any()).Return(&types.ExtraSettings{}, nil)
peerValidators := integrated_validator.NewMockIntegratedValidator(ctrl)
peerValidators.EXPECT().GetValidatedPeers(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(
map[string]struct{}{
"peer-id-1": {},
"peer-id-2": {},
"peer-id-3": {},
}, nil)
storeImpl := networkmapdb.NetworkMapDBStoreImpl{
Store: store(t),
ExtraSettingsManager: extraSettingsManager,
IntegratedPeerValidator: peerValidators,
}
for _, query := range strings.Split(nmapData, ";") {
if err := store(t).Exec(ctx, query); err != nil {
log.Fatalf("error initializing nmap test: %s", err.Error())
}
}
nmap, err := storeImpl.GetNetworkMapData(ctx, "account-33")
assert.NoError(t, err)
serializedNMap, err := json.MarshalIndent(nmap, "", " ")
assert.NoError(t, err)
if _, ok := os.LookupEnv(EnvUpdateGoldenData); ok {
_, filename, _, _ := runtime.Caller(0)
tosavepath := filepath.Join(filepath.Dir(filename), "network_map_data_golden.json")
err = os.WriteFile(tosavepath, serializedNMap, 0644)
assert.NoError(t, err)
goldenNMap = string(serializedNMap)
}
assert.Equal(t, goldenNMap, string(serializedNMap))
}

View File

@@ -0,0 +1,65 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetworkResources(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-1','account-1','network-1','net-resource-public-1','network-resource-1','network-resource-1','subnet','','"10.0.0.0/16"',TRUE)`)
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-2','account-1','network-2','net-resource-public-2','network-resource-2','network-resource-2','domain','test.com','',TRUE)`)
execQuery(t, ctx,
`insert into network_resources (id, account_id, network_id, public_id, name, description, type, domain, prefix, enabled)
VALUES('net-resource-3','account-1','network-3','net-resource-public-3','network-resource-3','network-resource-3','host','','"10.0.0.1/32"',TRUE)`)
resources, err := conn(t, ctx).GetNetworkResources(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-1",
AccountID: "account-1",
NetworkID: "network-1",
PublicID: "net-resource-public-1",
Name: "network-resource-1",
Description: "network-resource-1",
Type: "subnet",
Domain: "",
Prefix: netip.MustParsePrefix("10.0.0.0/16"),
Enabled: true,
})
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-2",
AccountID: "account-1",
NetworkID: "network-2",
PublicID: "net-resource-public-2",
Name: "network-resource-2",
Description: "network-resource-2",
Type: "domain",
Domain: "test.com",
Enabled: true,
})
assert.Contains(t, resources, nmdata.NetworkResource{
ID: "net-resource-3",
AccountID: "account-1",
NetworkID: "network-3",
PublicID: "net-resource-public-3",
Name: "network-resource-3",
Description: "network-resource-3",
Type: "host",
Domain: "",
Prefix: netip.MustParsePrefix("10.0.0.1/32"),
Enabled: true,
})
}

View File

@@ -0,0 +1,33 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetworkRouters(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-1','account-1','public-id-1','peer-id-1','network-id-1',TRUE,999,TRUE,'["group-one-resource-id"]')`)
execQuery(t, ctx,
`insert into network_routers (id, account_id, public_id, peer, network_id, masquerade, metric, enabled, peer_groups)
VALUES('test-nr-id-2','account-1','public-id-2','','network-id-2',TRUE,333,TRUE,'["group-two-resources-id","group-no-resources-id"]')`)
routers, err := conn(t, ctx).GetNetworkRouters(ctx, "account-1")
assert.NoError(t, err)
assert.NotEmpty(t, routers)
assert.Equal(t, routers["network-id-1"],
map[string]*nmdata.NetworkRouter{"peer-id-1": {PublicID: "public-id-1", Masquerade: true, Metric: 999, Enabled: true, PeerGroups: []string{"group-one-resource-id"}}})
assert.Equal(t, routers["network-id-2"],
map[string]*nmdata.NetworkRouter{
"peer-id-2": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}},
"peer-id-3": {PublicID: "public-id-2", Masquerade: true, Metric: 333, Enabled: true, PeerGroups: []string{"group-two-resources-id", "group-no-resources-id"}}})
}

View File

@@ -0,0 +1,56 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"encoding/json"
"net"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetNetwork(t *testing.T) {
ctx := context.TODO()
network, err := conn(t, ctx).GetNetwork(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, network, nmdata.Network{
Identifier: "network-1",
Net: mustParseCIDR("100.103.0.0/16"),
NetV6: mustParseCIDR("fdde:e995:fd38:a465::/64"),
Serial: 1,
})
network, err = conn(t, ctx).GetNetwork(ctx, "account-2")
assert.NoError(t, err)
assert.Equal(t, network, nmdata.Network{
Identifier: "network-2",
Net: mustParseCIDR("110.0.0.0/16"),
NetV6: mustParseCIDR("fddf:e995:fd38:a465::/64"),
Serial: 2,
})
}
func mustParseCIDR(s string) net.IPNet {
var toret net.IPNet
_, net, err := net.ParseCIDR(s)
if err != nil {
panic(err)
}
jn, err := json.Marshal(net)
if err != nil {
panic(err)
}
err = json.Unmarshal(jn, &toret)
if err != nil {
panic(err)
}
return toret
}

View File

@@ -0,0 +1,26 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetNetworks(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into networks (id, account_id, public_id) VALUES('network-1','account-1','network-1-public')`)
execQuery(t, ctx,
`insert into networks (id, account_id, public_id) VALUES('network-2','account-1','network-2-public')`)
networksIdx, err := conn(t, ctx).GetNetworkXIDToPublicIdMap(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, networksIdx, map[string]string{
"network-1": "network-1-public",
"network-2": "network-2-public",
})
}

View File

@@ -0,0 +1,163 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPeers(t *testing.T) {
ctx := context.TODO()
peers, clusterToPeersIdx, err := conn(t, ctx).GetPeers(ctx, "account-1")
assert.NoError(t, err)
// shouldn't be returned in the index, as it's not connected
execQuery(t, ctx,
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
values('peer-4','account-1','key-4','ssh-key-4',true,false)`)
// shouldn't be returned in the index as it doesn't have cluster set
execQuery(t, ctx,
`insert into peers (id,account_id,"key",ssh_key,proxy_meta_embedded,peer_status_connected)
values('peer-5','account-1','key-5','ssh-key-5',false,true)`)
peer1 := nmdata.Peer{
ID: "peer-id-1",
Key: "key-1",
SSHKey: "ssh-key-1",
DNSLabel: "peer-1",
ExtraDNSLabels: []string{"extra-peer-1"},
UserID: "user-id-1",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T13:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.10.1"),
IPv6: netip.MustParseAddr("fdf4:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.0",
GoOS: "linux",
OSVersion: "26.4.1",
KernelVersion: "6.8.0-134-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe80::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.16.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 1,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.148.187"),
},
}
peer2 := nmdata.Peer{
ID: "peer-id-2",
Key: "key-2",
SSHKey: "ssh-key-2",
DNSLabel: "peer-2",
ExtraDNSLabels: []string{"extra-peer-2"},
UserID: "user-id-2",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T14:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.100.1"),
IPv6: netip.MustParseAddr("fdf5:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.1",
GoOS: "linux",
OSVersion: "26.4.2",
KernelVersion: "6.8.0-135-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe81::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.17.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 0,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.149.187"),
},
}
peer3 := nmdata.Peer{
ID: "peer-id-3",
Key: "key-3",
SSHKey: "ssh-key-3",
DNSLabel: "peer-3",
ExtraDNSLabels: []string{"extra-peer-3"},
UserID: "user-id-3",
SSHEnabled: true,
LoginExpirationEnabled: true,
LastLogin: mustParseTime("2026-08-06T12:25:59.12999+00:00"),
IP: netip.MustParseAddr("10.10.200.1"),
IPv6: netip.MustParseAddr("fdf6:ba80:6aa5:89f1:44d7:8701:8699:4940"),
RequiresApproval: false,
Meta: nmdata.PeerSystemMeta{
WtVersion: "0.76.2",
GoOS: "linux",
OSVersion: "26.4.3",
KernelVersion: "6.8.0-136-generic",
NetworkAddresses: []nmdata.NetworkAddress{
{NetIP: netip.MustParsePrefix("fe82::8b4c:973f:a76b:3771/64")},
{NetIP: netip.MustParsePrefix("192.168.18.1/20")},
},
Files: []nmdata.File{
{Path: "/usr/bin/netbird", ProcessIsRunning: false},
},
Capabilities: []int32{1, 2},
Flags: nmdata.Flags{
ServerSSHAllowed: true,
DisableIPv6: false,
},
SyncMessageVersion: 1,
},
ProxyMeta: nmdata.ProxyMeta{
Embedded: true,
},
Location: nmdata.PeerLocation{
CountryCode: "DE",
CityName: "Berlin",
ConnectionIP: net.ParseIP("46.201.150.187"),
},
}
assert.Contains(t, peers, peer1)
assert.Contains(t, peers, peer2)
assert.Contains(t, peers, peer3)
assert.Equal(t, clusterToPeersIdx, map[string][]*nmdata.Peer{
"cluster-1.netbird.services": {&peer1},
"cluster-2.netbird.services": {&peer2},
"cluster-3.netbird.services": {&peer3},
})
}

View File

@@ -0,0 +1,121 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"fmt"
"regexp"
"strings"
"time"
log "github.com/sirupsen/logrus"
"github.com/google/uuid"
networkmap_pgsql "github.com/netbirdio/netbird/management/internals/network_map_db/pgsql"
gormstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/testutil"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func createPGTestStore(baseData string) (*networkmap_pgsql.PgStore, func()) {
_, tmpdsn, err := testutil.CreatePostgresTestContainer()
if err != nil {
log.Fatalf("error starting postres container %v", err)
}
var db *gorm.DB
for i := range 5 {
db, err = gorm.Open(postgres.Open(tmpdsn), &gorm.Config{})
if err == nil {
break
}
if i < 5 {
waitTime := time.Duration(100*(i+1)) * time.Millisecond
time.Sleep(waitTime)
continue
}
log.Fatalf("error connecting to postres db %v", err)
}
var cleanup func()
dsn, cleanup, err := createRandomDB(tmpdsn, db)
sqlDB, _ := db.DB()
if sqlDB != nil {
sqlDB.Close()
}
if err != nil {
log.Fatalf("error creating postres db %v", err)
}
_, err = gormstore.NewPostgresqlStoreForTests(context.TODO(), dsn, nil, false)
if err != nil {
log.Fatalf("error running migrations %v", err)
}
ctx := context.TODO()
pgstore, err := networkmap_pgsql.NewPostgresqlStore(ctx, dsn)
if err != nil {
log.Fatal("error creating postgres store %w", err)
}
for _, query := range strings.Split(baseData, ";") {
if _, err := pgstore.Pool.Exec(ctx, query); err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
}
return pgstore, cleanup
}
func createRandomDB(dsn string, db *gorm.DB) (string, func(), error) {
dbName := fmt.Sprintf("test_db_%s", strings.ReplaceAll(uuid.New().String(), "-", "_"))
if err := db.Exec(fmt.Sprintf("CREATE DATABASE %s", dbName)).Error; err != nil {
return "", nil, fmt.Errorf("failed to create database: %v", err)
}
originalDSN := dsn
cleanup := func() {
var dropDB *gorm.DB
var err error
dropDB, err = gorm.Open(postgres.Open(originalDSN), &gorm.Config{
SkipDefaultTransaction: true,
PrepareStmt: false,
})
if err != nil {
log.Errorf("failed to connect for dropping database %s: %v", dbName, err)
return
}
defer func() {
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
sqlDB.Close()
}
}()
if sqlDB, _ := dropDB.DB(); sqlDB != nil {
sqlDB.SetMaxOpenConns(1)
sqlDB.SetMaxIdleConns(0)
sqlDB.SetConnMaxLifetime(time.Second)
}
err = dropDB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s WITH (FORCE)", dbName)).Error
if err != nil {
log.Errorf("failed to drop database %s: %v", dbName, err)
}
}
return replaceDBName(dsn, dbName), cleanup, nil
}
func replaceDBName(dsn, newDBName string) string {
re := regexp.MustCompile(`(?P<pre>[:/@])(?P<dbname>[^/?]+)(?P<post>\?|$)`)
return re.ReplaceAllString(dsn, `${pre}`+newDBName+`${post}`)
}

View File

@@ -0,0 +1,146 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPolicies(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-1','policy-1-public','account-1',true,'["posture-checks-1","posture-checks-2"]')`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-1-rule-1','policy-1',true,'accept','tcp',true,'["group-one-resource-id","group-two-resources-id"]','["group-one-resource-id","group-two-resources-id"]',
'{"ID":"host-id-1","Type":"host"}','{"ID":"domain-1","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
'{"group-one-resource-id":["user-1", "user-2"]}','user-3')`)
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-2','policy-2-public','account-1',true,'["posture-checks-3","posture-checks-4"]')`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-2-rule-1','policy-2',true,'accept','tcp',true,'["group-one-resource-id"]','["group-two-resources-id"]',
'{"ID":"host-id-3","Type":"host"}','{"ID":"domain-3","Type":"domain"}','["8080","8443"]', '[{"Start":8080,"End":8090}]',
'{"group-one-resource-id":["user-6", "user-7"]}','user-8')`)
// policy with a rule with null fields
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-3','policy-3-public','account-1',true,null)`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-3-rule-1','policy-3',true,null,null,null,null,null,null,null,null,null,null,null)`)
// policy with a disabled rule, destination resource and groups should not be in indexes
execQuery(t, ctx,
`insert into policies (id, public_id, account_id, enabled, source_posture_checks)
values('policy-4','policy-4-public','account-1',true,null)`)
execQuery(t, ctx,
`insert into policy_rules (id, policy_id, enabled, action, protocol, bidirectional, sources, destinations,
source_resource, destination_resource, ports, port_ranges,
authorized_groups, authorized_user)
values('policy-4-rule-1','policy-4',false,null,null,null,null,'["group-two-resources-id"]',
null,'{"ID":"domain-3","Type":"domain"}',null,null,null,null)`)
policies, policyToDestinationResourceIdx, policyToDestinationGroupIdx, err := conn(t, ctx).GetPolicies(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-1",
PublicID: "policy-1-public",
Enabled: true,
SourcePostureChecks: []string{"posture-checks-1", "posture-checks-2"},
Rules: []*nmdata.PolicyRule{
{
ID: "policy-1",
PolicyID: "policy-1",
Enabled: true,
Action: "accept",
Protocol: "tcp",
Bidirectional: true,
Sources: []string{"group-one-resource-id", "group-two-resources-id"},
Destinations: []string{"group-one-resource-id", "group-two-resources-id"},
SourceResource: nmdata.Resource{ID: "host-id-1", Type: "host"},
DestinationResource: nmdata.Resource{ID: "domain-1", Type: "domain"},
Ports: []string{"8080", "8443"},
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-1", "user-2"}},
AuthorizedUser: "user-3",
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-2",
PublicID: "policy-2-public",
Enabled: true,
SourcePostureChecks: []string{"posture-checks-3", "posture-checks-4"},
Rules: []*nmdata.PolicyRule{
{
ID: "policy-2",
PolicyID: "policy-2",
Enabled: true,
Action: "accept",
Protocol: "tcp",
Bidirectional: true,
Sources: []string{"group-one-resource-id"},
Destinations: []string{"group-two-resources-id"},
SourceResource: nmdata.Resource{ID: "host-id-3", Type: "host"},
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
Ports: []string{"8080", "8443"},
PortRanges: []nmdata.RulePortRange{{Start: 8080, End: 8090}},
AuthorizedGroups: map[string][]string{"group-one-resource-id": {"user-6", "user-7"}},
AuthorizedUser: "user-8",
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-3",
PublicID: "policy-3-public",
Enabled: true,
SourcePostureChecks: nil,
Rules: []*nmdata.PolicyRule{
{
ID: "policy-3",
PolicyID: "policy-3",
Enabled: true,
},
},
})
assert.Contains(t, policies, nmdata.Policy{
ID: "policy-4",
PublicID: "policy-4-public",
Enabled: true,
SourcePostureChecks: nil,
Rules: []*nmdata.PolicyRule{
{
ID: "policy-4",
PolicyID: "policy-4",
Enabled: false,
Destinations: []string{"group-two-resources-id"},
DestinationResource: nmdata.Resource{ID: "domain-3", Type: "domain"},
},
},
})
assert.Equal(t, policyToDestinationGroupIdx, map[string]map[string]any{
"policy-1": {"group-one-resource-id": struct{}{}, "group-two-resources-id": struct{}{}},
"policy-2": {"group-two-resources-id": struct{}{}},
})
assert.Equal(t, policyToDestinationResourceIdx, map[string]map[string]any{
"policy-1": {"domain-1": struct{}{}},
"policy-2": {"domain-3": struct{}{}},
})
}

View File

@@ -0,0 +1,61 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetPostureChecks(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-1','account-1','posturecheck-1-public',
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
"OSVersionCheck":{"Darwin":{"MinVersion":"12.0"}},
"GeoLocationCheck":{"Locations":[{"CountryCode":"FI","CityName":""}],"Action":"allow"},
"PeerNetworkRangeCheck":{"Action":"deny","Ranges":["192.168.0.1/24"]}}')`)
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-2','account-1','posturecheck-2-public',
'{"NBVersionCheck":{"MinVersion":"0.25.0"},
"OSVersionCheck":{"Android":{"MinVersion":"0"}},
"GeoLocationCheck":{"Locations":[{"CountryCode":"US","CityName":"Harker Heights"}],"Action":"allow"},
"PeerNetworkRangeCheck":{"Action":"allow","Ranges":["0.0.0.0/0"]}}')`)
execQuery(t, ctx,
`insert into posture_checks (id, account_id, public_id, checks)
VALUES('posturecheck-3','account-1','posturecheck-3-public', null)`)
postureChecks, idToPublicIDIdx, err := conn(t, ctx).GetPostureChecks(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, idToPublicIDIdx, map[string]string{
"posturecheck-1": "posturecheck-1-public",
"posturecheck-2": "posturecheck-2-public",
"posturecheck-3": "posturecheck-3-public",
})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-1",
Checks: nmdata.ChecksDefinition{
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
OSVersionCheck: &nmdata.OSVersionCheck{Darwin: &nmdata.MinVersionCheck{MinVersion: "12.0"}},
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "FI"}}, Action: "allow"},
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "deny", Ranges: []netip.Prefix{netip.MustParsePrefix("192.168.0.1/24")}},
}})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-2",
Checks: nmdata.ChecksDefinition{
NBVersionCheck: &nmdata.NBVersionCheck{MinVersion: "0.25.0"},
OSVersionCheck: &nmdata.OSVersionCheck{Android: &nmdata.MinVersionCheck{MinVersion: "0"}},
GeoLocationCheck: &nmdata.GeoLocationCheck{Locations: []nmdata.GeoLocation{{CountryCode: "US", CityName: "Harker Heights"}}, Action: "allow"},
PeerNetworkRangeCheck: &nmdata.PeerNetworkRangeCheck{Action: "allow", Ranges: []netip.Prefix{netip.MustParsePrefix("0.0.0.0/0")}},
}})
assert.Contains(t, postureChecks, nmdata.PostureChecks{
ID: "posturecheck-3"})
}

View File

@@ -0,0 +1,87 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"net/netip"
"testing"
"github.com/netbirdio/netbird/shared/management/domain"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/stretchr/testify/assert"
)
func TestGetRoutes(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-1','account-1','route-1-public','"172.0.0.0/16"','["test-1.com"]',true,'route-1-net-id','route-1',
'peer-id-1','["group-one-resource-id"]',1,true,9999,true,
'["group-one-resource-id"]','["group-one-resource-id"]',false)`)
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-2','account-1','route-2-public','"172.10.0.0/16"','["test-1.com","test-2.com"]',true,'route-2-net-id','route-2',
'peer-id-2','["group-two-resources-id"]',1,true,9999,true,
'["group-two-resources-id"]','["group-two-resources-id"]',false)`)
execQuery(t, ctx,
`insert into routes (id, account_id, public_id, network, domains, keep_route, net_id, description,
peer, peer_groups, network_type, masquerade, metric, enabled,
groups, access_control_groups, skip_auto_apply)
VALUES('route-3','account-1','route-3-public',null,null,null,null,'route-3',
null,null,null,null,null,null,null,null,null)`)
routes, err := conn(t, ctx).GetRoutes(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, routes, nmdata.Route{
ID: "route-1",
AccountID: "account-1",
PublicID: "route-1-public",
Network: netip.MustParsePrefix("172.0.0.0/16"),
Domains: domain.List{"test-1.com"},
KeepRoute: true,
NetID: "route-1-net-id",
Description: "route-1",
Peer: "peer-id-1",
PeerID: "peer-id-1",
PeerGroups: []string{"group-one-resource-id"},
NetworkType: 1,
Masquerade: true,
Metric: 9999,
Enabled: true,
Groups: []string{"group-one-resource-id"},
AccessControlGroups: []string{"group-one-resource-id"},
SkipAutoApply: false,
})
assert.Contains(t, routes, nmdata.Route{
ID: "route-2",
AccountID: "account-1",
PublicID: "route-2-public",
Network: netip.MustParsePrefix("172.10.0.0/16"),
Domains: domain.List{"test-1.com", "test-2.com"},
KeepRoute: true,
NetID: "route-2-net-id",
Description: "route-2",
Peer: "peer-id-2",
PeerID: "peer-id-2",
PeerGroups: []string{"group-two-resources-id"},
NetworkType: 1,
Masquerade: true,
Metric: 9999,
Enabled: true,
Groups: []string{"group-two-resources-id"},
AccessControlGroups: []string{"group-two-resources-id"},
SkipAutoApply: false,
})
assert.Contains(t, routes, nmdata.Route{
ID: "route-3",
AccountID: "account-1",
PublicID: "route-3-public",
Description: "route-3",
})
}

View File

@@ -0,0 +1,109 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"database/sql"
"testing"
"github.com/stretchr/testify/assert"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
)
func TestGetPrivateServices(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-1','account-1',true,true,'["group-one-resource-id"]','test-1.com','test-2.com')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-2','account-1',true,true,'["group-one-resource-id","group-two-resources-id"]','test-3.com','test-4.com')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, private, access_groups, proxy_cluster, domain)
values('service-3','account-1',null,null,null,null,null)`)
services, err := conn(t, ctx).GetPrivateServices(ctx, "account-1")
assert.NoError(t, err)
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: true, Valid: true},
Private: sql.NullBool{Bool: true, Valid: true},
AccessGroups: []string{"group-one-resource-id"},
ProxyCluster: sql.NullString{String: "test-1.com", Valid: true},
Domain: sql.NullString{String: "test-2.com", Valid: true},
})
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: true, Valid: true},
Private: sql.NullBool{Bool: true, Valid: true},
AccessGroups: []string{"group-one-resource-id", "group-two-resources-id"},
ProxyCluster: sql.NullString{String: "test-3.com", Valid: true},
Domain: sql.NullString{String: "test-4.com", Valid: true},
})
assert.Contains(t, services, networkmapdb.Service{
Enabled: sql.NullBool{Bool: false, Valid: false},
Private: sql.NullBool{Bool: false, Valid: false},
AccessGroups: []string{},
ProxyCluster: sql.NullString{String: "", Valid: false},
Domain: sql.NullString{String: "", Valid: false},
})
}
func TestGetProxyTargetedDomainResourceIDs(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-4','account-1',true,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-1','account-1','service-4',true,'domain')`)
// id shouldn't be returned as the taget_type is not "domain"
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-2','account-1','service-4',true,'cluster')`)
// id shouldn't be included as the target is disabled
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-3','account-1','service-4',false,'domain')`)
// id shouldn't be included as the service is disabled
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-5','account-1',false,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-4','account-1','service-5',false,'domain')`)
// id shouldn't be included as the service is terminated (explicitly)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-6','account-1',true,true)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-5','account-1','service-6',true,'domain')`)
// id shouldn't be included as the service is terminated (implicitly)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-7','account-1',true,null)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-6','account-1','service-7',true,'domain')`)
execQuery(t, ctx,
`insert into services (id, account_id, enabled, terminated)
values('service-8','account-1',true,false)`)
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values('target-7','account-1','service-8',true,'domain')`)
// id shouldn't be returned as the taget_id is null
execQuery(t, ctx,
`insert into targets (target_id, account_id, service_id, enabled, target_type)
values(null,'account-1','service-4',true,'cluster')`)
servtargetedDomains, err := conn(t, ctx).GetProxyTargetedDomainResourceIDs(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, servtargetedDomains, map[string]struct{}{
"target-1": {},
"target-6": {},
"target-7": {},
})
}

View File

@@ -0,0 +1,48 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"fmt"
"runtime"
"strings"
networkmap_sqlite "github.com/netbirdio/netbird/management/internals/network_map_db/sqlite"
gormstore "github.com/netbirdio/netbird/management/server/store"
"github.com/netbirdio/netbird/management/server/types"
log "github.com/sirupsen/logrus"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
func createSqliteTestStore(baseData string) (*networkmap_sqlite.SqliteStore, func()) {
storeSqliteFileName := ":memory:"
storeStr := fmt.Sprintf("%s?cache=shared", storeSqliteFileName)
if runtime.GOOS == "windows" {
// Vo avoid `The process cannot access the file because it is being used by another process` on Windows
storeStr = storeSqliteFileName
}
db, err := gorm.Open(sqlite.Open(storeStr), &gorm.Config{})
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
_, err = gormstore.NewSqlStore(context.TODO(), db, types.SqliteStoreEngine, nil, false)
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
sqldb, err := db.DB()
if err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
for _, query := range strings.Split(baseData, ";") {
if _, err := sqldb.Exec(query); err != nil {
log.Fatalf("error initializing db: %s", err.Error())
}
}
return &networkmap_sqlite.SqliteStore{Db: sqldb}, func() {}
}

View File

@@ -0,0 +1,57 @@
//go:build integration
package networkmap_pgsql
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetAllowedUsers(t *testing.T) {
ctx := context.TODO()
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-1','user-1','account-1','["group-one-resource-id"]',false,false)`)
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-2','user-2','account-1','["group-one-resource-id","group-two-resources-id"]',false,false)`)
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-3','user-3','account-1','["group-two-resources-id"]',false,false)`)
// shouldn't be included as it's blocked
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-4','user-4','account-1','["group-two-resources-id"]',true,false)`)
// shouldn't be included as it's a service_user
execQuery(t, ctx,
`insert into users (id, name, account_id, auto_groups, blocked, is_service_user)
VALUES('user-5','user-5','account-1','["group-two-resources-id"]',false,true)`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-1','All','account-1')`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-2','All','account-1')`)
execQuery(t, ctx,
`insert into groups (id, name, account_id)
VALUES('all-group-3','All','account-1')`)
userIdx, groupIdToUserIds, err := conn(t, ctx).GetAllowedUsers(ctx, "account-1")
assert.NoError(t, err)
assert.Equal(t, userIdx, map[string]struct{}{
"user-1": {},
"user-2": {},
"user-3": {},
})
assert.Equal(t, groupIdToUserIds, map[string][]string{
"group-one-resource-id": {"user-1", "user-2"},
"group-two-resources-id": {"user-2", "user-3"},
"all-group-1": {"user-1", "user-2", "user-3"},
"all-group-2": {"user-1", "user-2", "user-3"},
"all-group-3": {"user-1", "user-2", "user-3"},
})
}

10
magefiles/magefile.go Normal file
View File

@@ -0,0 +1,10 @@
//mage:multiline
// Set the general description you want to have displayed with mage -l here.
package main
// mg contains helpful utility functions, like Deps
// Default target to run when none is specified
// If not set, running mage will list available targets
//var Default = Integrationtest.All

74
magefiles/test.go Normal file
View File

@@ -0,0 +1,74 @@
package main
import (
"errors"
"strings"
"github.com/magefile/mage/mg"
"github.com/magefile/mage/sh"
)
var defaultcli = []string{"test", "-tags=integration", "-timeout=20m"}
type Integrationtest mg.Namespace
func (i Integrationtest) All(gotestflags *string) error {
var errs []error
if err := i.Api(gotestflags); err != nil {
errs = append(errs, err)
}
if err := i.NmapDb(gotestflags); err != nil {
errs = append(errs, err)
}
if len(errs) > 0 {
return errors.Join(errs...)
}
return nil
}
func (Integrationtest) NmapDb(gotestflags *string) error {
cli := defaultcli
if gotestflags != nil {
cli = append(cli, strings.Split(*gotestflags, " ")...)
}
cli = append(cli, "./integration_tests/management/network_map_db/...")
return sh.RunV("go", cli...)
}
func (Integrationtest) NmapDbPostgres(gotestflags *string) error {
cli := defaultcli
if gotestflags != nil {
cli = append(cli, strings.Split(*gotestflags, " ")...)
}
cli = append(cli, "./integration_tests/management/network_map_db/...")
return sh.RunWithV(map[string]string{"NETBIRD_STORE_ENGINE": "postgres"}, "go", cli...)
}
func (Integrationtest) NmapDbSqlite(gotestflags *string) error {
cli := defaultcli
if gotestflags != nil {
cli = append(cli, strings.Split(*gotestflags, " ")...)
}
cli = append(cli, "./integration_tests/management/network_map_db/...")
return sh.RunWithV(map[string]string{"NETBIRD_STORE_ENGINE": "sqlite"}, "go", cli...)
}
func (Integrationtest) RegenerateNmapGoldenData(gotestflags *string) error {
cli := defaultcli
if gotestflags != nil {
cli = append(cli, strings.Split(*gotestflags, " ")...)
}
cli = append(cli, "./integration_tests/management/network_map_db/...")
return sh.RunWithV(map[string]string{"NMAP_UPDATE_GOLDEN_DATA": "true", "NETBIRD_STORE_ENGINE": "sqlite"}, "go", cli...)
}
func (Integrationtest) Api(gotestflags *string) error {
cli := defaultcli
if gotestflags != nil {
cli = append(cli, strings.Split(*gotestflags, " ")...)
}
cli = append(cli, "./management/server/http/...")
return sh.RunV("go", cli...)
}

View File

@@ -18,6 +18,7 @@ import (
"github.com/netbirdio/netbird/management/internals/controllers/network_map"
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller/cache"
"github.com/netbirdio/netbird/management/internals/modules/peers/ephemeral"
networkmapdb "github.com/netbirdio/netbird/management/internals/network_map_db"
"github.com/netbirdio/netbird/management/internals/server/config"
"github.com/netbirdio/netbird/management/internals/shared/grpc"
"github.com/netbirdio/netbird/management/server/account"
@@ -30,6 +31,8 @@ import (
"github.com/netbirdio/netbird/management/server/telemetry"
"github.com/netbirdio/netbird/management/server/types"
sharedgrpc "github.com/netbirdio/netbird/shared/management/grpc"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/netbirdio/netbird/shared/management/proto"
"github.com/netbirdio/netbird/shared/management/status"
"github.com/netbirdio/netbird/util"
@@ -61,6 +64,8 @@ type Controller struct {
serverSupportedSyncMessageVersion sharedgrpc.SyncMessageVersion
perAccountServerSupportedSyncMessageVersions map[string]sharedgrpc.SyncMessageVersion
nmdataStore *networkmapdb.NetworkMapDBStoreImpl
}
type bufferUpdate struct {
@@ -78,7 +83,7 @@ type bufferAffectedUpdate struct {
var _ network_map.Controller = (*Controller)(nil)
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config) *Controller {
func NewController(ctx context.Context, store store.Store, metrics telemetry.AppMetrics, peersUpdateManager network_map.PeersUpdateManager, requestBuffer account.RequestBuffer, integratedPeerValidator integrated_validator.IntegratedValidator, settingsManager settings.Manager, dnsDomain string, proxyController port_forwarding.Controller, ephemeralPeersManager ephemeral.Manager, config *config.Config, nmdataStore *networkmapdb.NetworkMapDBStoreImpl) *Controller {
nMetrics, err := newMetrics(metrics.UpdateChannelMetrics())
if err != nil {
log.Fatal(fmt.Errorf("error creating metrics: %w", err))
@@ -99,6 +104,7 @@ func NewController(ctx context.Context, store store.Store, metrics telemetry.App
EphemeralPeersManager: ephemeralPeersManager,
serverSupportedSyncMessageVersion: sharedgrpc.SyncMessageVersionFromConfig(config.HighestSupportedSyncMessageVersion),
perAccountServerSupportedSyncMessageVersions: sharedgrpc.SyncMessageVersionsFromMap(config.PerAccountHighestSupportedSyncMessageVersion),
nmdataStore: nmdataStore,
}
}
@@ -147,6 +153,11 @@ func (c *Controller) CountStreams() int {
func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID string, reason types.UpdateReason) error {
log.WithContext(ctx).Tracef("updating peers for account %s from %s", accountID, util.GetCallerName())
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.sendUpdateAccountPeersFromData(ctx, accountID, reason, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get account: %v", err)
@@ -167,7 +178,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
return nil
}
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validate peers: %v", err)
}
@@ -255,7 +266,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -276,7 +287,7 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
}
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -294,6 +305,261 @@ func (c *Controller) sendUpdateAccountPeers(ctx context.Context, accountID strin
return nil
}
// sendUpdateAccountPeersFromData is the account-free variant of
// sendUpdateAccountPeers: everything is computed from the network-map DB
// store's twin data; only extra settings and validated peers are resolved at
// runtime. Proxy network maps and policy injection, private-service zones,
// group-to-user SSH mappings and forced routing-peer DNS resolution have no
// DB-backed source yet and are omitted.
func (c *Controller) sendUpdateAccountPeersFromData(ctx context.Context, accountID string, reason types.UpdateReason, nmData *networkmap.NetworkMapData) error {
peersToUpdate := c.connectedPeersFromData(nmData, nil)
if len(peersToUpdate) == 0 {
return nil
}
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, &reason)
}
// sendUpdateForAffectedPeersFromData is the account-free variant of
// sendUpdateForAffectedPeers.
func (c *Controller) sendUpdateForAffectedPeersFromData(ctx context.Context, accountID string, peerIDs []string, nmData *networkmap.NetworkMapData) error {
if len(peerIDs) == 0 {
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no affected peers")
return nil
}
peersToUpdate := c.connectedPeersFromData(nmData, peerIDs)
if len(peersToUpdate) == 0 {
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: no peers to update (affected peers not found in data or no channels)")
return nil
}
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeersFromData: sending network map to %d connected peers", len(peersToUpdate))
return c.sendUpdatesFromData(ctx, accountID, nmData, peersToUpdate, nil)
}
// connectedPeersFromData returns the peers with an open update channel. An
// empty affected list means all peers; a non-empty list restricts the result
// to those peer IDs.
func (c *Controller) connectedPeersFromData(nmData *networkmap.NetworkMapData, affected []string) []*nmdata.Peer {
if len(affected) == 0 {
result := make([]*nmdata.Peer, 0, len(nmData.Peers))
for _, peer := range nmData.Peers {
if c.peersUpdateManager.HasChannel(peer.ID) {
result = append(result, peer)
}
}
return result
}
result := make([]*nmdata.Peer, 0, len(affected))
for _, peerID := range affected {
peer := nmData.Peers[peerID]
if peer == nil {
continue
}
if c.peersUpdateManager.HasChannel(peerID) {
result = append(result, peer)
}
}
return result
}
func (c *Controller) sendUpdatesFromData(ctx context.Context, accountID string, nmData *networkmap.NetworkMapData, peersToUpdate []*nmdata.Peer, reason *types.UpdateReason) error {
globalStart := time.Now()
extraSettings, err := c.settingsManager.GetExtraSettings(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get flow enabled status: %v", err)
}
nmData.PrecomputePostureValidation()
dnsCache := &cache.DNSConfigCache{}
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
var wg sync.WaitGroup
semaphore := make(chan struct{}, 10)
for _, peer := range peersToUpdate {
if reason != nil && c.accountManagerMetrics != nil {
c.accountManagerMetrics.CountNmapTriggered(string(reason.Resource), string(reason.Operation))
}
wg.Add(1)
semaphore <- struct{}{}
go func(p *nmdata.Peer) {
defer wg.Done()
defer func() { <-semaphore }()
start := time.Now()
postureChecks := peerPostureChecksFromData(nmData, p.ID)
c.metrics.CountCalcPostureChecksDuration(time.Since(start))
start = time.Now()
peerGroups := maps.Keys(nmData.GetPeerGroups(p.ID))
var update *proto.SyncResponse
commonSyncMessageVersion := sharedgrpc.HighestCommonSyncMessageVersion(
c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion))
log.WithContext(ctx).
WithFields(log.Fields{
"sync_message_version": commonSyncMessageVersion,
"server_sync_message_version": c.perAccountOrGlobalSupportedSyncMessageVersions(accountID),
"peer_sync_message_version": sharedgrpc.SyncMessageVersionFromConfig(&p.Meta.SyncMessageVersion),
}).Debug("common highest sync message version")
if commonSyncMessageVersion == sharedgrpc.ComponentNetworkMap {
components := nmData.GetPeerNetworkMapComponents(p.ID, peersCustomZone)
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
start = time.Now()
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, nil, dnsDomain, postureChecks, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
Update: update,
MessageType: network_map.MessageTypeNetworkMap,
})
return
}
nmap := NetworkMapFromData(ctx, nmData, p.ID, peersCustomZone)
c.metrics.CountCalcPeerNetworkMapDuration(time.Since(start))
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, nmData.AccountSettings, extraSettings, peerGroups, dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
Update: update,
MessageType: network_map.MessageTypeNetworkMap,
})
}(peer)
}
wg.Wait()
if c.accountManagerMetrics != nil {
c.accountManagerMetrics.CountUpdateAccountPeersDuration(time.Since(globalStart))
}
return nil
}
func (c *Controller) getNetworkMapData(ctx context.Context, accountID string) *networkmap.NetworkMapData {
if c.nmdataStore == nil {
return nil
}
nmData, err := c.nmdataStore.GetNetworkMapData(ctx, accountID)
if err != nil {
log.WithContext(ctx).Errorf("failed to get network map data for account %s, falling back to account-based computation: %v", accountID, err)
return nil
}
return nmData
}
func (c *Controller) getDNSDomainFromData(settings *nmdata.AccountSettingsInfo) string {
if settings == nil || settings.DNSDomain == "" {
return c.dnsDomain
}
return settings.DNSDomain
}
func IPv6AllowedPeersFromData(nmData *networkmap.NetworkMapData) map[string]struct{} {
result := make(map[string]struct{})
if nmData.AccountSettings != nil {
for _, groupID := range nmData.AccountSettings.IPv6EnabledGroups {
group := nmData.Groups[groupID]
if group == nil {
continue
}
for _, peerID := range group.Peers {
result[peerID] = struct{}{}
}
}
}
for id, p := range nmData.Peers {
if p != nil && p.ProxyMeta.Embedded {
result[id] = struct{}{}
}
}
return result
}
func NetworkMapFromData(ctx context.Context, nmData *networkmap.NetworkMapData, peerID string, peersCustomZone nmdata.CustomZone) *types.NetworkMap {
components := nmData.GetPeerNetworkMapComponents(peerID, peersCustomZone)
if components.IsEmpty() {
return &types.NetworkMap{Network: components.Network}
}
return types.CalculateNetworkMapFromComponents(ctx, components)
}
// peerPostureChecksFromData mirrors getPeerPostureChecks on the twin store. The
// sync response only encodes process-check file paths, so only ProcessCheck is
// converted back to the server posture type.
func peerPostureChecksFromData(nmData *networkmap.NetworkMapData, peerID string) []*posture.Checks {
if len(nmData.PostureChecks) == 0 {
return nil
}
peerPostureChecks := make(map[string]*posture.Checks)
for _, policy := range nmData.Policies {
if policy == nil || !policy.Enabled || len(policy.SourcePostureChecks) == 0 {
continue
}
if !isPeerInPolicySourceGroupsFromData(nmData, peerID, policy) {
continue
}
for _, checkID := range policy.SourcePostureChecks {
twin := nmData.PostureChecks[checkID]
if twin == nil {
continue
}
peerPostureChecks[checkID] = postureChecksFromTwin(twin)
}
}
return maps.Values(peerPostureChecks)
}
func isPeerInPolicySourceGroupsFromData(nmData *networkmap.NetworkMapData, peerID string, policy *nmdata.Policy) bool {
for _, rule := range policy.Rules {
if rule == nil || !rule.Enabled {
continue
}
for _, groupID := range rule.Sources {
if group := nmData.Groups[groupID]; group != nil && slices.Contains(group.Peers, peerID) {
return true
}
}
}
return false
}
func postureChecksFromTwin(twin *nmdata.PostureChecks) *posture.Checks {
checks := &posture.Checks{ID: twin.ID}
if twin.Checks.ProcessCheck != nil {
processes := make([]posture.Process, 0, len(twin.Checks.ProcessCheck.Processes))
for _, p := range twin.Checks.ProcessCheck.Processes {
processes = append(processes, posture.Process{LinuxPath: p.LinuxPath, MacPath: p.MacPath, WindowsPath: p.WindowsPath})
}
checks.Checks.ProcessCheck = &posture.ProcessCheck{Processes: processes}
}
return checks
}
func (c *Controller) perAccountOrGlobalSupportedSyncMessageVersions(accountId string) sharedgrpc.SyncMessageVersion {
if perAccount, ok := c.perAccountServerSupportedSyncMessageVersions[accountId]; ok {
return perAccount
@@ -326,6 +592,10 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
return nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.sendUpdateForAffectedPeersFromData(ctx, accountID, peerIDs, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return fmt.Errorf("failed to get account: %v", err)
@@ -341,7 +611,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
log.WithContext(ctx).Tracef("sendUpdateForAffectedPeers: sending network map to %d connected peers", len(peersToUpdate))
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validate peers: %v", err)
}
@@ -428,7 +698,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToComponentSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -449,7 +719,7 @@ func (c *Controller) sendUpdateForAffectedPeers(ctx context.Context, accountID s
}
start = time.Now()
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, p, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSetting, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(p), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSetting, maps.Keys(peerGroups), dnsFwdPort)
c.metrics.CountToSyncResponseDuration(time.Since(start))
c.peersUpdateManager.SendUpdate(ctx, p.ID, &network_map.UpdateMessage{
@@ -506,7 +776,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
return fmt.Errorf("peer %s doesn't exists in account %s", peerId, accountId)
}
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return fmt.Errorf("failed to get validated peers: %v", err)
}
@@ -566,7 +836,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
// proxyNetworkMap rides the envelope as a ProxyPatch sidecar;
// the client merges it into Calculate()'s output the same
// way the legacy server did via NetworkMap.Merge.
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToComponentSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, components, proxyNetworkMap, dnsDomain, postureChecks, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
Update: update,
@@ -583,7 +853,7 @@ func (c *Controller) UpdateAccountPeer(ctx context.Context, accountId string, pe
nmap.Merge(proxyNetworkMap)
}
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, peer, nil, nil, nmap, dnsDomain, postureChecks, dnsCache, account.Settings, extraSettings, maps.Keys(peerGroups), dnsFwdPort)
update = grpc.ToSyncResponse(ctx, nil, c.config.HttpConfig, c.config.DeviceAuthorizationFlow, types.TwinPeer(peer), nil, nil, nmap, dnsDomain, postureChecks, dnsCache, types.TwinAccountSettings(account.Settings), extraSettings, maps.Keys(peerGroups), dnsFwdPort)
c.peersUpdateManager.SendUpdate(ctx, peer.ID, &network_map.UpdateMessage{
Update: update,
@@ -643,7 +913,11 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
if err != nil {
return nil, nil, nil, nil, 0, err
}
return peer, &types.NetworkMapComponents{Network: network.Copy()}, nil, nil, 0, nil
return peer, &types.NetworkMapComponents{Network: types.TwinNetwork(network)}, nil, nil, 0, nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.getValidatedPeerWithComponentsFromData(ctx, accountID, peer, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
@@ -653,7 +927,7 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
c.injectAllProxyPolicies(ctx, account)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, nil, nil, nil, 0, err
}
@@ -690,6 +964,21 @@ func (c *Controller) GetValidatedPeerWithComponents(ctx context.Context, isRequi
return peer, components, proxyNetworkMaps[peer.ID], postureChecks, dnsFwdPort, nil
}
// getValidatedPeerWithComponentsFromData is the account-free variant of
// GetValidatedPeerWithComponents. The proxy network map fragment is omitted
// like on the other nmdata paths.
func (c *Controller) getValidatedPeerWithComponentsFromData(ctx context.Context, accountID string, peer *nbpeer.Peer, nmData *networkmap.NetworkMapData) (*nbpeer.Peer, *types.NetworkMapComponents, *types.NetworkMap, []*posture.Checks, int64, error) {
postureChecks := peerPostureChecksFromData(nmData, peer.ID)
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
components := nmData.GetPeerNetworkMapComponents(peer.ID, peersCustomZone)
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
return peer, components, nil, postureChecks, dnsFwdPort, nil
}
// BufferUpdateAffectedPeers accumulates peer IDs and flushes them after the buffer interval.
func (c *Controller) BufferUpdateAffectedPeers(ctx context.Context, accountID string, peerIDs []string, reason types.UpdateReason) error {
if len(peerIDs) == 0 {
@@ -796,11 +1085,15 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
}
emptyMap := &types.NetworkMap{
Network: network.Copy(),
Network: types.TwinNetwork(network),
}
return emptyMap, nil, 0, nil
}
if nmData := c.getNetworkMapData(ctx, accountID); nmData != nil {
return c.getValidatedPeerWithMapFromData(ctx, accountID, peerID, nmData)
}
account, err := c.requestBuffer.GetAccountWithBackpressure(ctx, accountID)
if err != nil {
return nil, nil, 0, err
@@ -808,7 +1101,7 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
c.injectAllProxyPolicies(ctx, account)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
approvedPeersMap, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, nil, 0, err
}
@@ -848,6 +1141,21 @@ func (c *Controller) GetValidatedPeerWithMap(ctx context.Context, isRequiresAppr
return networkMap, postureChecks, dnsFwdPort, nil
}
// getValidatedPeerWithMapFromData is the account-free variant of
// GetValidatedPeerWithMap. The proxy network map fragment is omitted like on
// the other nmdata paths.
func (c *Controller) getValidatedPeerWithMapFromData(ctx context.Context, accountID string, peerID string, nmData *networkmap.NetworkMapData) (*types.NetworkMap, []*posture.Checks, int64, error) {
postureChecks := peerPostureChecksFromData(nmData, peerID)
dnsDomain := c.getDNSDomainFromData(nmData.AccountSettings)
peersCustomZone := networkmap.PeersCustomZone(ctx, accountID, dnsDomain, nmData.Peers, IPv6AllowedPeersFromData(nmData))
networkMap := NetworkMapFromData(ctx, nmData, peerID, peersCustomZone)
dnsFwdPort := ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
return networkMap, postureChecks, dnsFwdPort, nil
}
// GetDNSDomain returns the configured dnsDomain
func (c *Controller) GetDNSDomain(settings *types.Settings) string {
if settings == nil {
@@ -910,20 +1218,36 @@ func (c *Controller) StartWarmup(ctx context.Context) {
// computeForwarderPort checks if all peers in the account have updated to a specific version or newer.
// If all peers have the required version, it returns the new well-known port (22054), otherwise returns 0.
func computeForwarderPort(peers []*nbpeer.Peer, requiredVersion string) int64 {
if len(peers) == 0 {
versions := make([]string, 0, len(peers))
for _, peer := range peers {
versions = append(versions, peer.Meta.WtVersion)
}
return computeForwarderPortFromVersions(versions, requiredVersion)
}
func ComputeForwarderPortFromData(peers map[string]*nmdata.Peer, requiredVersion string) int64 {
versions := make([]string, 0, len(peers))
for _, peer := range peers {
versions = append(versions, peer.Meta.WtVersion)
}
return computeForwarderPortFromVersions(versions, requiredVersion)
}
func computeForwarderPortFromVersions(wtVersions []string, requiredVersion string) int64 {
if len(wtVersions) == 0 {
return int64(network_map.OldForwarderPort)
}
reqVer := semver.Canonical(requiredVersion)
// Check if all peers have the required version or newer
for _, peer := range peers {
for _, wtVersion := range wtVersions {
// Development version is always supported
if version.IsDevelopmentVersion(peer.Meta.WtVersion) {
if version.IsDevelopmentVersion(wtVersion) {
continue
}
peerVersion := semver.Canonical("v" + peer.Meta.WtVersion)
peerVersion := semver.Canonical("v" + wtVersion)
if peerVersion == "" {
// If any peer doesn't have version info, return 0
return int64(network_map.OldForwarderPort)
@@ -1057,7 +1381,7 @@ func (c *Controller) GetNetworkMap(ctx context.Context, peerID string) (*types.N
groups[groupID] = group.Peers
}
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, maps.Values(account.Groups), maps.Values(account.Peers), account.Settings.Extra)
validatedPeers, err := c.integratedPeerValidator.GetValidatedPeers(ctx, account.Id, types.TwinGroups(maps.Values(account.Groups)), types.TwinPeers(maps.Values(account.Peers)), account.Settings.Extra)
if err != nil {
return nil, err
}

View File

@@ -0,0 +1,380 @@
package nmaptest
import (
"bytes"
"cmp"
"fmt"
"slices"
"sort"
"strconv"
"strings"
"github.com/netbirdio/netbird/shared/management/proto"
)
// normalizeIDSpace replaces policy and route identifiers with positional
// placeholders so a comparison can reach everything else.
//
// This exists only because the envelope round-trip currently substitutes each
// internal xid with the object's public id, which is a tracked defect and not a
// licence to differ: those identifiers reach the server again inside flow
// events, which resolve them by internal id, so the substitution silently
// breaks flow attribution for component-format peers. TestIDSpaceMatches
// asserts the equality that must eventually hold; this erasure keeps the other
// 40-odd cases reporting on semantics meanwhile. When the id space is unified,
// delete this and the calls to it — every case should still pass.
//
// Cardinality and cross-references survive the erasure: two rules under one
// policy still share a token and a route firewall rule still points at its
// route, so a path that drops a policy, merges two policies, or misattributes a
// rule to the wrong route still fails.
func normalizeIDSpace(nm *proto.NetworkMap) {
if nm == nil {
return
}
policies := newTokenizer("policy")
routes := newTokenizer("route")
for _, i := range orderBy(nm.Routes, routeKeyWithoutID) {
nm.Routes[i].ID = routes.get(nm.Routes[i].ID)
}
for _, i := range orderBy(nm.FirewallRules, firewallKeyWithoutPolicy) {
r := nm.FirewallRules[i]
if len(r.PolicyID) > 0 {
r.PolicyID = []byte(policies.get(string(r.PolicyID)))
}
}
for _, i := range orderBy(nm.RoutesFirewallRules, routeFirewallKeyWithoutIDs) {
r := nm.RoutesFirewallRules[i]
if len(r.PolicyID) > 0 {
r.PolicyID = []byte(policies.get(string(r.PolicyID)))
}
r.RouteID = routes.get(r.RouteID)
}
}
// tokenizer maps identifiers to positional placeholders in order of first use.
type tokenizer struct {
prefix string
seen map[string]string
}
func newTokenizer(prefix string) *tokenizer {
return &tokenizer{prefix: prefix, seen: make(map[string]string)}
}
func (t *tokenizer) get(id string) string {
if id == "" {
return ""
}
if tok, ok := t.seen[id]; ok {
return tok
}
tok := fmt.Sprintf("%s#%d", t.prefix, len(t.seen))
t.seen[id] = tok
return tok
}
// orderBy returns indices sorted by key, so placeholder numbering does not
// depend on the identifiers being erased.
func orderBy[T any](items []T, key func(T) string) []int {
idx := make([]int, len(items))
for i := range idx {
idx[i] = i
}
sort.SliceStable(idx, func(a, b int) bool { return key(items[idx[a]]) < key(items[idx[b]]) })
return idx
}
func routeKeyWithoutID(r *proto.Route) string {
if r == nil {
return ""
}
return fmt.Sprintf("%s|%s|%s|%d|%d|%t|%t|%v",
r.Network, r.NetID, r.Peer, r.Metric, r.NetworkType, r.Masquerade, r.KeepRoute, r.Domains)
}
func firewallKeyWithoutPolicy(r *proto.FirewallRule) string {
if r == nil {
return ""
}
return fmt.Sprintf("%s|%d|%d|%d|%s|%s|%v",
r.PeerIP, r.Direction, r.Action, r.Protocol, r.Port, portInfoKey(r.PortInfo), r.SourcePrefixes) //nolint:staticcheck
}
func routeFirewallKeyWithoutIDs(r *proto.RouteFirewallRule) string {
if r == nil {
return ""
}
return fmt.Sprintf("%s|%d|%d|%s|%v|%v|%t|%d",
r.Destination, r.Protocol, r.Action, portInfoKey(r.PortInfo), r.Domains, r.SourceRanges, r.IsDynamic, r.CustomProtocol)
}
// canonicalize sorts every repeated field of the NetworkMap by a stable key.
// The producing paths iterate Go maps while building these slices, so order
// can differ between runs even when the content is identical; comparing
// without this reports noise.
func canonicalize(nm *proto.NetworkMap) {
if nm == nil {
return
}
slices.SortFunc(nm.RemotePeers, cmpRemotePeer)
slices.SortFunc(nm.OfflinePeers, cmpRemotePeer)
slices.SortFunc(nm.Routes, cmpRoute)
slices.SortFunc(nm.FirewallRules, cmpFirewallRule)
slices.SortFunc(nm.RoutesFirewallRules, cmpRouteFirewallRule)
slices.SortFunc(nm.ForwardingRules, cmpForwardingRule)
for _, r := range nm.FirewallRules {
slices.SortFunc(r.SourcePrefixes, bytes.Compare)
}
for _, r := range nm.RoutesFirewallRules {
slices.Sort(r.SourceRanges)
}
canonicalizeDNSConfig(nm.DNSConfig)
canonicalizeSSHAuth(nm.SshAuth)
}
func canonicalizeDNSConfig(d *proto.DNSConfig) {
if d == nil {
return
}
for _, g := range d.NameServerGroups {
if g == nil {
continue
}
slices.Sort(g.Domains)
slices.SortFunc(g.NameServers, func(a, b *proto.NameServer) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := cmp.Compare(a.IP, b.IP); c != 0 {
return c
}
if c := cmp.Compare(a.Port, b.Port); c != 0 {
return c
}
return cmp.Compare(a.NSType, b.NSType)
})
}
slices.SortFunc(d.NameServerGroups, func(a, b *proto.NameServerGroup) int {
return cmp.Compare(nsgKey(a), nsgKey(b))
})
for _, z := range d.CustomZones {
if z == nil {
continue
}
slices.SortFunc(z.Records, cmpSimpleRecord)
}
slices.SortFunc(d.CustomZones, func(a, b *proto.CustomZone) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
return cmp.Compare(a.Domain, b.Domain)
})
}
// canonicalizeSSHAuth sorts AuthorizedUsers and re-keys MachineUsers.Indexes
// against the new ordering, preserving which machine user maps to which hashes.
func canonicalizeSSHAuth(s *proto.SSHAuth) {
if s == nil || len(s.AuthorizedUsers) == 0 {
return
}
type hashed struct {
bytes []byte
old uint32
}
entries := make([]hashed, len(s.AuthorizedUsers))
for i, b := range s.AuthorizedUsers {
entries[i] = hashed{bytes: b, old: uint32(i)}
}
slices.SortFunc(entries, func(a, b hashed) int { return bytes.Compare(a.bytes, b.bytes) })
remap := make(map[uint32]uint32, len(entries))
sorted := make([][]byte, len(entries))
for newIdx, e := range entries {
remap[e.old] = uint32(newIdx)
sorted[newIdx] = e.bytes
}
s.AuthorizedUsers = sorted
for _, mu := range s.MachineUsers {
if mu == nil {
continue
}
for i, oldIdx := range mu.Indexes {
if newIdx, ok := remap[oldIdx]; ok {
mu.Indexes[i] = newIdx
}
}
slices.Sort(mu.Indexes)
}
}
func boolCmp(a, b bool) int {
if a == b {
return 0
}
if a {
return 1
}
return -1
}
func nsgKey(g *proto.NameServerGroup) string {
if g == nil {
return ""
}
var parts []string
for _, ns := range g.NameServers {
if ns == nil {
continue
}
parts = append(parts, ns.IP+":"+strconv.FormatInt(ns.Port, 10)+":"+strconv.FormatInt(ns.NSType, 10))
}
slices.Sort(parts)
key := strings.Join(parts, ",")
domains := append([]string(nil), g.Domains...)
slices.Sort(domains)
key += "|" + strings.Join(domains, "|")
if g.Primary {
key += "|P"
}
if g.SearchDomainsEnabled {
key += "|S"
}
return key
}
func cmpSimpleRecord(a, b *proto.SimpleRecord) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := cmp.Compare(a.Name, b.Name); c != 0 {
return c
}
if c := cmp.Compare(a.Type, b.Type); c != 0 {
return c
}
if c := cmp.Compare(a.Class, b.Class); c != 0 {
return c
}
if c := cmp.Compare(a.RData, b.RData); c != 0 {
return c
}
return cmp.Compare(a.TTL, b.TTL)
}
func cmpRemotePeer(a, b *proto.RemotePeerConfig) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
return cmp.Compare(a.WgPubKey, b.WgPubKey)
}
func cmpRoute(a, b *proto.Route) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := cmp.Compare(a.ID, b.ID); c != 0 {
return c
}
if c := cmp.Compare(a.NetID, b.NetID); c != 0 {
return c
}
if c := cmp.Compare(a.Network, b.Network); c != 0 {
return c
}
if c := cmp.Compare(a.Peer, b.Peer); c != 0 {
return c
}
if c := cmp.Compare(a.Metric, b.Metric); c != 0 {
return c
}
return slices.Compare(a.Domains, b.Domains)
}
func cmpFirewallRule(a, b *proto.FirewallRule) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := bytes.Compare(a.PolicyID, b.PolicyID); c != 0 {
return c
}
if c := cmp.Compare(a.PeerIP, b.PeerIP); c != 0 { //nolint:staticcheck
return c
}
if c := cmp.Compare(int32(a.Direction), int32(b.Direction)); c != 0 {
return c
}
if c := cmp.Compare(int32(a.Action), int32(b.Action)); c != 0 {
return c
}
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
return c
}
if c := cmp.Compare(a.Port, b.Port); c != 0 {
return c
}
return cmp.Compare(portInfoKey(a.PortInfo), portInfoKey(b.PortInfo))
}
func cmpRouteFirewallRule(a, b *proto.RouteFirewallRule) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := bytes.Compare(a.PolicyID, b.PolicyID); c != 0 {
return c
}
if c := cmp.Compare(a.RouteID, b.RouteID); c != 0 {
return c
}
if c := cmp.Compare(a.Destination, b.Destination); c != 0 {
return c
}
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
return c
}
if c := cmp.Compare(portInfoKey(a.PortInfo), portInfoKey(b.PortInfo)); c != 0 {
return c
}
if c := cmp.Compare(int32(a.Action), int32(b.Action)); c != 0 {
return c
}
if c := slices.Compare(a.Domains, b.Domains); c != 0 {
return c
}
if c := slices.Compare(a.SourceRanges, b.SourceRanges); c != 0 {
return c
}
if c := cmp.Compare(a.CustomProtocol, b.CustomProtocol); c != 0 {
return c
}
return boolCmp(a.IsDynamic, b.IsDynamic)
}
func cmpForwardingRule(a, b *proto.ForwardingRule) int {
if a == nil || b == nil {
return boolCmp(a == nil, b == nil)
}
if c := cmp.Compare(int32(a.Protocol), int32(b.Protocol)); c != 0 {
return c
}
return bytes.Compare(a.TranslatedAddress, b.TranslatedAddress)
}
func portInfoKey(pi *proto.PortInfo) string {
if pi == nil {
return ""
}
switch sel := pi.PortSelection.(type) {
case *proto.PortInfo_Port:
return "P" + strconv.FormatUint(uint64(sel.Port), 10)
case *proto.PortInfo_Range_:
if sel.Range == nil {
return "R"
}
return "R" + strconv.FormatUint(uint64(sel.Range.Start), 10) + "-" + strconv.FormatUint(uint64(sel.Range.End), 10)
}
return ""
}

View File

@@ -0,0 +1,218 @@
package nmaptest
import (
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net"
"os"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
)
// LoadNetworkMapData reads a fixture holding the NetworkMapData the store
// would return for one account. Unknown fields are rejected so fixture typos
// fail loudly instead of silently testing a default.
func LoadNetworkMapData(path string) (*networkmap.NetworkMapData, error) {
f, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open fixture: %w", err)
}
defer f.Close()
dec := json.NewDecoder(f)
dec.DisallowUnknownFields()
var nmData networkmap.NetworkMapData
if err := dec.Decode(&nmData); err != nil {
return nil, fmt.Errorf("decode fixture %s: %w", path, err)
}
return &nmData, nil
}
var defaultNetworkNet = func() net.IPNet {
_, ipnet, err := net.ParseCIDR("100.64.0.0/10")
if err != nil {
panic(err)
}
return *ipnet
}()
// applyFixtureDefaults fills the boilerplate a fixture may omit. Map-keyed
// objects inherit their key as ID, peers get a deterministic WG-shaped key
// and their ID as DNS label, PublicIDs default to the internal ID (the
// envelope encoder puts public IDs on the wire and silently degrades on
// empty ones), and a nil ValidatedPeers validates every peer — production
// fills it through the integrated validator, not the store.
func applyFixtureDefaults(nmData *networkmap.NetworkMapData) {
if nmData.Network == nil {
nmData.Network = &nmdata.Network{}
}
if nmData.Network.Identifier == "" {
nmData.Network.Identifier = "network"
}
if nmData.Network.Net.IP == nil {
nmData.Network.Net = defaultNetworkNet
}
if nmData.AccountSettings == nil {
nmData.AccountSettings = &nmdata.AccountSettingsInfo{}
}
if nmData.DNSSettings == nil {
nmData.DNSSettings = &nmdata.DNSSettings{}
}
for id, p := range nmData.Peers {
if p == nil {
continue
}
if p.ID == "" {
p.ID = id
}
if p.Key == "" {
p.Key = derivedWgKey(p.ID)
}
if p.DNSLabel == "" {
p.DNSLabel = p.ID
}
}
for id, g := range nmData.Groups {
if g == nil {
continue
}
if g.ID == "" {
g.ID = id
}
if g.Name == "" {
g.Name = g.ID
}
if g.PublicID == "" {
g.PublicID = g.ID
}
}
for _, policy := range nmData.Policies {
defaultPolicyIDs(policy)
}
resolveResourcePolicyRefs(nmData)
for _, r := range nmData.Routes {
if r != nil && r.PublicID == "" {
r.PublicID = r.ID
}
}
for _, nsg := range nmData.NameServerGroups {
if nsg != nil && nsg.PublicID == "" {
nsg.PublicID = nsg.ID
}
}
for _, res := range nmData.NetworkResources {
if res == nil {
continue
}
if res.PublicID == "" {
res.PublicID = res.ID
}
defaultXIDMapping(&nmData.NetworkXIDToPublicID, res.NetworkID)
}
for networkID, routers := range nmData.Routers {
defaultXIDMapping(&nmData.NetworkXIDToPublicID, networkID)
for _, router := range routers {
if router != nil && router.PublicID == "" {
router.PublicID = networkID
}
}
}
for id, pc := range nmData.PostureChecks {
if pc == nil {
continue
}
if pc.ID == "" {
pc.ID = id
}
defaultXIDMapping(&nmData.PostureCheckXIDToPublicID, pc.ID)
}
if nmData.ValidatedPeers == nil {
nmData.ValidatedPeers = make(map[string]struct{}, len(nmData.Peers))
for id := range nmData.Peers {
nmData.ValidatedPeers[id] = struct{}{}
}
}
}
// resolveResourcePolicyRefs lets a fixture name an account policy by ID in
// ResourcePolicies — {"ID": "pol-x"} with no rules — instead of repeating it.
// The real store puts the same policy pointer in both places, which is what
// resolving the reference reproduces.
func resolveResourcePolicyRefs(nmData *networkmap.NetworkMapData) {
byID := make(map[string]*nmdata.Policy, len(nmData.Policies))
for _, policy := range nmData.Policies {
if policy != nil && policy.ID != "" {
byID[policy.ID] = policy
}
}
for _, policies := range nmData.ResourcePolicies {
for i, policy := range policies {
if policy == nil {
continue
}
if len(policy.Rules) == 0 {
if full, ok := byID[policy.ID]; ok {
policies[i] = full
continue
}
}
defaultPolicyIDs(policy)
}
}
}
func defaultPolicyIDs(policy *nmdata.Policy) {
if policy == nil {
return
}
if policy.PublicID == "" {
policy.PublicID = policy.ID
}
for i, rule := range policy.Rules {
if rule == nil {
continue
}
if rule.PolicyID == "" {
rule.PolicyID = policy.ID
}
if rule.ID == "" {
// Production gives a rule its policy's id (management/server/policy.go:205,
// "when policy can contain multiple rules, need refactor"), so a
// single-rule policy — the only shape the product can create today —
// must be modelled that way or the wire ids come out unrealistic.
rule.ID = policy.ID
if len(policy.Rules) > 1 {
rule.ID = fmt.Sprintf("%s-rule-%d", policy.ID, i)
}
}
}
}
func defaultXIDMapping(m *map[string]string, id string) {
if id == "" {
return
}
if *m == nil {
*m = make(map[string]string)
}
if _, ok := (*m)[id]; !ok {
(*m)[id] = id
}
}
// derivedWgKey returns a deterministic base64 key of 32 bytes, valid for the
// envelope decoder's WG-key identity.
func derivedWgKey(peerID string) string {
sum := sha256.Sum256([]byte(peerID))
return base64.StdEncoding.EncodeToString(sum[:])
}

View File

@@ -0,0 +1,12 @@
package nmaptest_test
import (
"path/filepath"
"testing"
"github.com/netbirdio/netbird/management/internals/controllers/network_map/nmaptest"
)
func TestNetworkMapGolden(t *testing.T) {
nmaptest.RunGoldenDir(t, filepath.Join("testdata", "cases"))
}

View File

@@ -0,0 +1,311 @@
// Package nmaptest measures network map generation on the dedicated store
// path against committed expectations. A case stands in for the store load
// with a NetworkMapData fixture — the value NetworkMapDBStoreImpl returns for
// one account — then runs the production per-peer pipeline the controller
// uses, PeersCustomZone → GetPeerNetworkMapComponents → proto conversion, in
// both wire shapes: the legacy full map (grpc.ToSyncResponse) and the
// component envelope expanded client-side (grpc.ToComponentSyncResponse →
// networkmap.EnvelopeToNetworkMap).
//
// The expectation files are the point of the framework. They state what the
// output should be, so a failing case means the code disagrees with the
// expectation and the answer is normally to fix the code; an expectation
// changes only through a deliberate reviewed edit. Nothing in this package
// writes to testdata — there is no flag that records current behaviour into an
// expectation, because that is how a defect becomes the baseline. Cases whose
// expectation encodes correct behaviour the code does not yet deliver stay red
// on purpose.
//
// A case lives in testdata/cases/<name>/ as case.json (manifest: description,
// peers, optional accountID, dnsDomain, modes), nmdata.json (the fixture the
// mocked store returns, using Go field names; zero values may be omitted and
// applyFixtureDefaults fills the boilerplate) and golden/<peerID>.json.
//
// There is ONE expectation per peer, shared by every mode. The modes are not
// different computations: CalculateNetworkMapFromComponents is
// components.Calculate, and both sides assemble the proto with the same
// encode helpers, so the only variable is what the envelope round-trip did to
// the components in transit. Any difference between modes is therefore a
// round-trip fidelity defect, and a shared expectation is what exposes it.
// Results are canonicalized before comparison, since repeated proto fields
// come from map iteration.
package nmaptest
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/require"
"golang.org/x/exp/maps"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/testing/protocmp"
"github.com/netbirdio/netbird/management/internals/controllers/network_map"
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller"
"github.com/netbirdio/netbird/management/internals/controllers/network_map/controller/cache"
mgmtgrpc "github.com/netbirdio/netbird/management/internals/shared/grpc"
"github.com/netbirdio/netbird/shared/management/networkmap"
"github.com/netbirdio/netbird/shared/management/networkmap/nmdata"
"github.com/netbirdio/netbird/shared/management/proto"
)
// Mode selects the wire shape a case is verified through. Both end in a
// *proto.NetworkMap, the one comparison surface shared by every path.
type Mode string
const (
// ModeFull is the legacy wire shape: the server runs Calculate and sends
// the expanded map (grpc.ToSyncResponse).
ModeFull Mode = "full"
// ModeEnvelope is the component wire shape: the server encodes components
// into a NetworkMapEnvelope (grpc.ToComponentSyncResponse) and the map is
// expanded the way the client engine does (networkmap.EnvelopeToNetworkMap).
ModeEnvelope Mode = "envelope"
defaultAccountID = "account"
defaultDNSDomain = "netbird.test"
)
var defaultModes = []Mode{ModeFull, ModeEnvelope}
// Case is one nmap-generation scenario: store data for a single account, the
// peers whose network maps are computed, and the directory holding one expected
// *proto.NetworkMap per peer — shared by every mode.
type Case struct {
Name string
AccountID string
DNSDomain string
Peers []string
Modes []Mode
Data *networkmap.NetworkMapData
GoldenDir string
}
type manifest struct {
Description string
AccountID string
DNSDomain string
Peers []string
Modes []Mode
}
// RunGoldenDir discovers and runs every fixture case under dir. A case is a
// directory containing case.json (manifest), nmdata.json (store fixture) and
// golden/<peerID>.json (expected proto.NetworkMap, protojson).
func RunGoldenDir(t *testing.T, dir string) {
t.Helper()
entries, err := os.ReadDir(dir)
require.NoError(t, err, "read cases dir")
ran := 0
for _, entry := range entries {
if !entry.IsDir() || strings.HasPrefix(entry.Name(), ".") {
continue
}
caseDir := filepath.Join(dir, entry.Name())
c, err := loadCase(caseDir)
require.NoError(t, err, "load case %s", entry.Name())
ran++
t.Run(entry.Name(), func(t *testing.T) {
RunCase(t, c)
})
}
require.NotZero(t, ran, "no cases found under %s", dir)
}
func loadCase(caseDir string) (Case, error) {
raw, err := os.ReadFile(filepath.Join(caseDir, "case.json"))
if err != nil {
return Case{}, fmt.Errorf("read manifest: %w", err)
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
var m manifest
if err := dec.Decode(&m); err != nil {
return Case{}, fmt.Errorf("decode manifest: %w", err)
}
data, err := LoadNetworkMapData(filepath.Join(caseDir, "nmdata.json"))
if err != nil {
return Case{}, err
}
return Case{
Name: filepath.Base(caseDir),
AccountID: m.AccountID,
DNSDomain: m.DNSDomain,
Peers: m.Peers,
Modes: m.Modes,
Data: data,
GoldenDir: filepath.Join(caseDir, "golden"),
}, nil
}
// RunCase computes each target peer's network map through every enabled mode
// and compares the canonicalized result against the peer's expectation file.
// It mirrors the controller's store path: fill fixture defaults, precompute
// posture validation once, then run the per-peer pipeline.
func RunCase(t *testing.T, c Case) {
t.Helper()
require.NotNil(t, c.Data, "case %s: Data is required", c.Name)
require.NotEmpty(t, c.Peers, "case %s: Peers is required", c.Name)
require.NotEmpty(t, c.GoldenDir, "case %s: GoldenDir is required", c.Name)
if c.AccountID == "" {
c.AccountID = defaultAccountID
}
if c.DNSDomain == "" {
c.DNSDomain = defaultDNSDomain
}
if len(c.Modes) == 0 {
c.Modes = defaultModes
}
ctx := context.Background()
nmData := c.Data
applyFixtureDefaults(nmData)
nmData.PrecomputePostureValidation()
dnsDomain := c.DNSDomain
if nmData.AccountSettings.DNSDomain != "" {
dnsDomain = nmData.AccountSettings.DNSDomain
}
zone := networkmap.PeersCustomZone(ctx, c.AccountID, dnsDomain, nmData.Peers, controller.IPv6AllowedPeersFromData(nmData))
dnsFwdPort := controller.ComputeForwarderPortFromData(nmData.Peers, network_map.DnsForwarderPortMinVersion)
for _, mode := range c.Modes {
if mode == ModeEnvelope {
requireEnvelopeSafeKeys(t, nmData, c.Name)
break
}
}
for _, peerID := range c.Peers {
peer := nmData.Peers[peerID]
require.NotNil(t, peer, "case %s: target peer %q not in fixture", c.Name, peerID)
for _, mode := range c.Modes {
t.Run(peerID+"/"+string(mode), func(t *testing.T) {
got := computeMode(t, ctx, mode, nmData, peerID, zone, dnsDomain, dnsFwdPort)
canonicalize(got)
compareGolden(t, filepath.Join(c.GoldenDir, peerID+".json"), got, mode)
})
}
}
}
// computeMode produces the peer's proto.NetworkMap the way the controller does
// for that wire shape.
func computeMode(t *testing.T, ctx context.Context, mode Mode, nmData *networkmap.NetworkMapData,
peerID string, zone nmdata.CustomZone, dnsDomain string, dnsFwdPort int64) *proto.NetworkMap {
t.Helper()
peer := nmData.Peers[peerID]
require.NotNil(t, peer, "target peer %q not in fixture", peerID)
switch mode {
case ModeFull:
nmap := controller.NetworkMapFromData(ctx, nmData, peerID, zone)
return mgmtgrpc.ToSyncResponse(ctx, nil, nil, nil, peer, nil, nil, nmap, dnsDomain, nil,
&cache.DNSConfigCache{}, nmData.AccountSettings, nil, nil, dnsFwdPort).NetworkMap
case ModeEnvelope:
components := nmData.GetPeerNetworkMapComponents(peerID, zone)
peerGroups := maps.Keys(nmData.GetPeerGroups(peerID))
resp := mgmtgrpc.ToComponentSyncResponse(ctx, nil, nil, nil, peer, nil, nil, components, nil,
dnsDomain, nil, nmData.AccountSettings, nil, peerGroups, dnsFwdPort)
res, err := networkmap.EnvelopeToNetworkMap(ctx, resp.NetworkMapEnvelope, peer.Key, dnsDomain)
require.NoError(t, err, "expand envelope")
return res.NetworkMap
default:
t.Fatalf("unknown mode %q", mode)
return nil
}
}
// requireEnvelopeSafeKeys fails fast on peer keys the envelope decoder would
// silently drop: it re-keys peers by base64 of the raw 32-byte WG public key.
func requireEnvelopeSafeKeys(t *testing.T, nmData *networkmap.NetworkMapData, caseName string) {
t.Helper()
for id, p := range nmData.Peers {
if p == nil {
continue
}
raw, err := base64.StdEncoding.DecodeString(p.Key)
if err != nil || len(raw) != 32 {
t.Fatalf("case %s: peer %q Key must be base64 of 32 bytes for mode %q (the envelope decoder drops it otherwise); use a real WireGuard public key or restrict the case to mode %q",
caseName, id, ModeEnvelope, ModeFull)
}
}
}
// compareGolden measures got against the committed expectation file. One
// expectation serves every mode, because the modes run the same computation and
// must therefore agree. The expectation is the authority: a mismatch means the
// code does not produce what this case says it should, so it is reported as a
// failure and not quietly absorbed.
//
// The full mode is compared verbatim, identifiers included, so the expectation
// pins real ids and stays readable. Other modes have identifiers erased on both
// sides first, because the envelope currently rewrites them — a tracked defect
// that TestIDSpaceMatches asserts against on its own, so it does not have to
// drown out every other case here.
// Nothing here writes to testdata. Expectation files are authored by hand and
// only ever change through a reviewed edit, so there is no mode in which a run
// can create or replace one. When a file is missing the computed map is printed
// for the author to read and, if it is genuinely correct, save deliberately.
func compareGolden(t *testing.T, path string, got *proto.NetworkMap, mode Mode) {
t.Helper()
if mode != ModeFull {
normalizeIDSpace(got)
canonicalize(got)
}
raw, err := os.ReadFile(path)
if err != nil {
rendered, mErr := renderNetworkMap(got)
require.NoError(t, mErr)
t.Fatalf("no expectation file %s: %v\nThis case has nothing to measure against — write the "+
"proto.NetworkMap this peer should receive. Mode %s currently produces:\n%s\nRead it before "+
"saving any of it: if the code is wrong, so is this.", path, err, mode, rendered)
}
want := &proto.NetworkMap{}
require.NoError(t, protojson.Unmarshal(raw, want), "parse expectation %s", path)
canonicalize(want)
if mode != ModeFull {
normalizeIDSpace(want)
canonicalize(want)
}
if diff := cmp.Diff(want, got, protocmp.Transform()); diff != "" {
t.Errorf("mode %s does not produce what %s expects (-want +got):\n%s\n"+
"Both modes run the same computation on the same components, so they must produce the same map. "+
"The expectation file is the committed statement of correct output — fix the code, or change the "+
"expectation deliberately if the intended behaviour really moved.", mode, path, diff)
}
}
// renderNetworkMap renders stable protojson: protojson output whitespace is
// deliberately unstable, so it is reformatted through json.Indent.
func renderNetworkMap(nm *proto.NetworkMap) ([]byte, error) {
raw, err := protojson.Marshal(nm)
if err != nil {
return nil, err
}
var buf bytes.Buffer
if err := json.Indent(&buf, raw, "", " "); err != nil {
return nil, err
}
buf.WriteByte('\n')
return buf.Bytes(), nil
}

View File

@@ -0,0 +1,7 @@
{
"description": "Two groups joined by one allow-all policy; peer-c has SSH enabled so the legacy-SSH path fills SshAuth from AllowedUserIDs.",
"peers": [
"peer-a",
"peer-c"
]
}

View File

@@ -0,0 +1,65 @@
{
"Serial": "5",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "4deEImv8zGvsyBmmfC2G0eQkbyMzyGuz/YK7pcYETwM=",
"allowedIps": [
"100.64.0.3/32"
],
"sshConfig": {
"sshPubKey": "c3NoLXBlZXItYw=="
},
"fqdn": "peer-c.netbird.test",
"agentVersion": "0.60.0"
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
},
{
"Name": "peer-c.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.3"
}
]
}
],
"ForwarderPort": "22054"
},
"FirewallRules": [
{
"PeerIP": "100.64.0.3",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
},
{
"PeerIP": "100.64.0.3",
"Direction": "OUT",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
}
],
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,102 @@
{
"Serial": "5",
"peerConfig": {
"address": "100.64.0.3/10",
"sshConfig": {
"sshEnabled": true
},
"fqdn": "peer-c.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
"allowedIps": [
"100.64.0.2/32"
],
"sshConfig": {},
"fqdn": "peer-b.netbird.test",
"agentVersion": "0.60.0"
},
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
},
{
"Name": "peer-b.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.2"
},
{
"Name": "peer-c.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.3"
}
]
}
],
"ForwarderPort": "22054"
},
"FirewallRules": [
{
"PeerIP": "100.64.0.1",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
},
{
"PeerIP": "100.64.0.1",
"Direction": "OUT",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
},
{
"PeerIP": "100.64.0.2",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
},
{
"PeerIP": "100.64.0.2",
"Direction": "OUT",
"Protocol": "ALL",
"PolicyID": "cG9sLWFsbA=="
}
],
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub",
"AuthorizedUsers": [
"u9dHvAXZJKiXITuwP9jD/A=="
],
"machineUsers": {
"*": {
"indexes": [
0
]
}
}
}
}

View File

@@ -0,0 +1,31 @@
{
"Network": {"Serial": 5},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-b": {"IP": "100.64.0.2", "Meta": {"WtVersion": "0.60.0"}},
"peer-c": {"IP": "100.64.0.3", "SSHEnabled": true, "SSHKey": "ssh-peer-c", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a", "peer-b"]},
"grp-ops": {"Peers": ["peer-c"]}
},
"Policies": [
{
"ID": "pol-all",
"PublicID": "pol-all-pub",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "all",
"Bidirectional": true,
"Sources": ["grp-dev"],
"Destinations": ["grp-ops"]
}
]
}
],
"AllowedUserIDs": {"user-ops": {}}
}

View File

@@ -0,0 +1,7 @@
{
"description": "Nameserver group and applied custom zone distributed to grp-dev; peer-a (with an extra DNS label) receives them, peer-c outside the group receives neither.",
"peers": [
"peer-a",
"peer-c"
]
}

View File

@@ -0,0 +1,93 @@
{
"Serial": "8",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
"allowedIps": [
"100.64.0.2/32"
],
"sshConfig": {},
"fqdn": "peer-b.netbird.test",
"agentVersion": "0.60.0"
}
],
"DNSConfig": {
"ServiceEnable": true,
"NameServerGroups": [
{
"NameServers": [
{
"IP": "8.8.8.8",
"Port": "53"
}
],
"Primary": true
}
],
"CustomZones": [
{
"Domain": "corp.internal.",
"Records": [
{
"Name": "db.corp.internal.",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "10.10.0.5"
}
]
},
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
},
{
"Name": "peer-b.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.2"
},
{
"Name": "www.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"FirewallRules": [
{
"PeerIP": "100.64.0.2",
"Protocol": "ALL",
"PolicyID": "cG9sLW1lc2g="
},
{
"PeerIP": "100.64.0.2",
"Direction": "OUT",
"Protocol": "ALL",
"PolicyID": "cG9sLW1lc2g="
}
],
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "8",
"peerConfig": {
"address": "100.64.0.3/10",
"sshConfig": {},
"fqdn": "peer-c.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-c.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.3"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,51 @@
{
"Network": {"Serial": 8},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "ExtraDNSLabels": ["www"], "Meta": {"WtVersion": "0.60.0"}},
"peer-b": {"IP": "100.64.0.2", "Meta": {"WtVersion": "0.60.0"}},
"peer-c": {"IP": "100.64.0.3", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a", "peer-b"]},
"grp-ops": {"Peers": ["peer-c"]}
},
"Policies": [
{
"ID": "pol-mesh",
"PublicID": "pol-mesh-pub",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "all",
"Bidirectional": true,
"Sources": ["grp-dev"],
"Destinations": ["grp-dev"]
}
]
}
],
"NameServerGroups": [
{
"ID": "nsg-1",
"Name": "dns-primary",
"NameServers": [{"IP": "8.8.8.8", "Port": 53}],
"Groups": ["grp-dev"],
"Primary": true,
"Enabled": true
}
],
"AppliedZoneCandidates": [
{
"DistributionGroups": ["grp-dev"],
"Zone": {
"Domain": "corp.internal.",
"Records": [
{"Name": "db.corp.internal.", "Type": 1, "Class": "IN", "TTL": 300, "RData": "10.10.0.5"}
]
}
}
]
}

View File

@@ -0,0 +1,4 @@
{
"description": "Domain network resource: the route carries the domain list and the 192.0.2.0/32 placeholder network with NetworkType 3 (dynamic), and peer-r's route firewall rules must be marked dynamic and repeat the domain. Two ports on the policy must produce one rule per port. A domain resource contributes no DNS custom zone of its own — resolution happens through the routing peer's forwarder.",
"peers": ["peer-a", "peer-r"]
}

View File

@@ -0,0 +1,59 @@
{
"Serial": "22",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"allowedIps": [
"100.64.0.9/32"
],
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-domain:peer-r",
"Network": "192.0.2.0/32",
"NetworkType": "3",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "9999",
"Masquerade": true,
"NetID": "app-domain",
"Domains": [
"app.internal"
],
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,92 @@
{
"Serial": "22",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-domain:peer-r",
"Network": "192.0.2.0/32",
"NetworkType": "3",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "9999",
"Masquerade": true,
"NetID": "app-domain",
"Domains": [
"app.internal"
],
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRules": [
{
"sourceRanges": [
"100.64.0.1/32"
],
"destination": "192.0.2.0/32",
"protocol": "TCP",
"portInfo": {
"port": 443
},
"isDynamic": true,
"domains": [
"app.internal"
],
"PolicyID": "cG9sLWFwcA==",
"RouteID": "res-domain:peer-r"
},
{
"sourceRanges": [
"100.64.0.1/32"
],
"destination": "192.0.2.0/32",
"protocol": "TCP",
"portInfo": {
"port": 80
},
"isDynamic": true,
"domains": [
"app.internal"
],
"PolicyID": "cG9sLWFwcA==",
"RouteID": "res-domain:peer-r"
}
],
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,43 @@
{
"Network": {"Serial": 22},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"Policies": [
{
"ID": "pol-app",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["80", "443"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-domain", "Type": "domain"}
}
]
}
],
"ResourcePolicies": {"res-domain": [{"ID": "pol-app"}]},
"NetworkResources": [
{
"ID": "res-domain",
"NetworkID": "net-1",
"Name": "app-domain",
"Type": "domain",
"Domain": "app.internal",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "Host network resource (single /32) behind one directly-assigned router. peer-a is in the resource policy's source group and must receive one route to 10.10.0.7/32 via peer-r with KeepRoute set and NetID taken from the resource name; peer-r as the router must receive the same route plus a route firewall rule whose SourceRanges are the policy's source peers. A client never gets route firewall rules.",
"peers": ["peer-a", "peer-r"]
}

View File

@@ -0,0 +1,56 @@
{
"Serial": "20",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"allowedIps": [
"100.64.0.9/32"
],
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-host:peer-r",
"Network": "10.10.0.7/32",
"NetworkType": "1",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "9999",
"Masquerade": true,
"NetID": "web-host",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,69 @@
{
"Serial": "20",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-host:peer-r",
"Network": "10.10.0.7/32",
"NetworkType": "1",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "9999",
"Masquerade": true,
"NetID": "web-host",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRules": [
{
"sourceRanges": [
"100.64.0.1/32"
],
"destination": "10.10.0.7/32",
"protocol": "TCP",
"portInfo": {
"port": 443
},
"PolicyID": "cG9sLXdlYg==",
"RouteID": "res-host:peer-r"
}
],
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,43 @@
{
"Network": {"Serial": 20},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"Policies": [
{
"ID": "pol-web",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["443"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-host", "Type": "host"}
}
]
}
],
"ResourcePolicies": {"res-host": [{"ID": "pol-web"}]},
"NetworkResources": [
{
"ID": "res-host",
"NetworkID": "net-1",
"Name": "web-host",
"Type": "host",
"Prefix": "10.10.0.7/32",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "A disabled resource with a valid policy and router must leave no trace: no routes and no route firewall rules for either the client or the router. Disabling a resource is the switch that revokes access without deleting the policy.",
"peers": ["peer-a", "peer-r"]
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "25",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "25",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,42 @@
{
"Network": {"Serial": 25},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"Policies": [
{
"ID": "pol-off-resource",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["443"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-disabled", "Type": "subnet"}
}
]
}
],
"ResourcePolicies": {"res-disabled": [{"ID": "pol-off-resource"}]},
"NetworkResources": [
{
"ID": "res-disabled",
"NetworkID": "net-1",
"Name": "disabled-subnet",
"Type": "subnet",
"Prefix": "10.50.0.0/24"
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "An enabled resource with a healthy router but no policy granting access to it must produce nothing anywhere: no route for the client and none for the router either, since access to a resource is only ever created by a policy. The router also gets no route firewall rules despite being a routing peer for the network.",
"peers": ["peer-a", "peer-r"]
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "24",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "24",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,26 @@
{
"Network": {"Serial": 24},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"NetworkResources": [
{
"ID": "res-orphan",
"NetworkID": "net-1",
"Name": "orphan-subnet",
"Type": "subnet",
"Prefix": "10.40.0.0/24",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "A DISABLED policy granting access to a network resource must grant nothing: no route to 10.90.0.0/24 for peer-a and none for the router either, exactly as if the policy were absent. THE FULL EXPECTATION CURRENTLY FAILS, and should: resource-policy selection never checks policy.Enabled (networkmapcompute.go and networkmap_components.go both test only nil/len(Rules)/Rules[0]), so the legacy path still hands out the route — access survives disabling the policy. The envelope path happens to be correct because the encoder drops disabled policies from the wire. Fix the compute path, do not weaken this expectation.",
"peers": ["peer-a", "peer-r"]
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "39",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "39",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,42 @@
{
"Network": {"Serial": 39},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"Policies": [
{
"ID": "pol-revoked",
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["5432"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-db", "Type": "subnet"}
}
]
}
],
"ResourcePolicies": {"res-db": [{"ID": "pol-revoked"}]},
"NetworkResources": [
{
"ID": "res-db",
"NetworkID": "net-1",
"Name": "db-subnet",
"Type": "subnet",
"Prefix": "10.90.0.0/24",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "The routing peer for the resource is not in ValidatedPeers — an unapproved peer, which the integrated validator withholds. peer-a must therefore receive no route through it and must not see it as a peer at all: traffic may not be routed through a peer the account has not approved. THE ENVELOPE EXPECTATION CURRENTLY FAILS, and should: component selection puts every routing peer into RouterPeers without checking validation, the encoder indexes them into the envelope's peer table, and the client decoder puts every peer it finds back into its peer map, so the unapproved router reappears client-side with a working route. The full path drops it correctly. Fix the component/encoder path, do not weaken this expectation.",
"peers": ["peer-a"]
}

View File

@@ -0,0 +1,34 @@
{
"Serial": "40",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeersIsEmpty": true,
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,44 @@
{
"Network": {"Serial": 40},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"ValidatedPeers": {"peer-a": {}},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]}
},
"Policies": [
{
"ID": "pol-db",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["5432"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-db", "Type": "subnet"}
}
]
}
],
"ResourcePolicies": {"res-db": [{"ID": "pol-db"}]},
"NetworkResources": [
{
"ID": "res-db",
"NetworkID": "net-1",
"Name": "db-subnet",
"Type": "subnet",
"Prefix": "10.100.0.0/24",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "Routing peer group: one router record assigned to a peer group, which the store expands into one entry per member peer sharing the router's settings. peer-a must receive one route per routing peer — same NetID and destination, different route ID and peer — which is what gives the client an HA pair to choose between. Each router must receive only its own route, never its sibling's, plus its own route firewall rule.",
"peers": ["peer-a", "peer-r1", "peer-r2"]
}

View File

@@ -0,0 +1,75 @@
{
"Serial": "23",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "Iu6Lj1HqDXfgDh5RppHCRI6RO3lMlZBCndRNrdS7QUI=",
"allowedIps": [
"100.64.0.11/32"
],
"sshConfig": {},
"fqdn": "peer-r1.netbird.test",
"agentVersion": "0.60.0"
},
{
"wgPubKey": "YC6sbWtvpB2d6W/XqsNIV9crsrXBoVDdwmhCoszjCps=",
"allowedIps": [
"100.64.0.12/32"
],
"sshConfig": {},
"fqdn": "peer-r2.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-ha:peer-r1",
"Network": "10.30.0.0/24",
"NetworkType": "1",
"Peer": "Iu6Lj1HqDXfgDh5RppHCRI6RO3lMlZBCndRNrdS7QUI=",
"Metric": "9999",
"Masquerade": true,
"NetID": "ha-subnet",
"keepRoute": true
},
{
"ID": "res-ha:peer-r2",
"Network": "10.30.0.0/24",
"NetworkType": "1",
"Peer": "YC6sbWtvpB2d6W/XqsNIV9crsrXBoVDdwmhCoszjCps=",
"Metric": "9999",
"Masquerade": true,
"NetID": "ha-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,69 @@
{
"Serial": "23",
"peerConfig": {
"address": "100.64.0.11/10",
"sshConfig": {},
"fqdn": "peer-r1.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-ha:peer-r1",
"Network": "10.30.0.0/24",
"NetworkType": "1",
"Peer": "Iu6Lj1HqDXfgDh5RppHCRI6RO3lMlZBCndRNrdS7QUI=",
"Metric": "9999",
"Masquerade": true,
"NetID": "ha-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r1.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.11"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRules": [
{
"sourceRanges": [
"100.64.0.1/32"
],
"destination": "10.30.0.0/24",
"protocol": "TCP",
"portInfo": {
"port": 5432
},
"PolicyID": "cG9sLWhh",
"RouteID": "res-ha:peer-r1"
}
],
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,69 @@
{
"Serial": "23",
"peerConfig": {
"address": "100.64.0.12/10",
"sshConfig": {},
"fqdn": "peer-r2.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-ha:peer-r2",
"Network": "10.30.0.0/24",
"NetworkType": "1",
"Peer": "YC6sbWtvpB2d6W/XqsNIV9crsrXBoVDdwmhCoszjCps=",
"Metric": "9999",
"Masquerade": true,
"NetID": "ha-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r2.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.12"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRules": [
{
"sourceRanges": [
"100.64.0.1/32"
],
"destination": "10.30.0.0/24",
"protocol": "TCP",
"portInfo": {
"port": 5432
},
"PolicyID": "cG9sLWhh",
"RouteID": "res-ha:peer-r2"
}
],
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,46 @@
{
"Network": {"Serial": 23},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-r1": {"IP": "100.64.0.11", "Meta": {"WtVersion": "0.60.0"}},
"peer-r2": {"IP": "100.64.0.12", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a"]},
"grp-routers": {"Peers": ["peer-r1", "peer-r2"]}
},
"Policies": [
{
"ID": "pol-ha",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "tcp",
"Ports": ["5432"],
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-ha", "Type": "subnet"}
}
]
}
],
"ResourcePolicies": {"res-ha": [{"ID": "pol-ha"}]},
"NetworkResources": [
{
"ID": "res-ha",
"NetworkID": "net-ha",
"Name": "ha-subnet",
"Type": "subnet",
"Prefix": "10.30.0.0/24",
"Enabled": true
}
],
"Routers": {
"net-ha": {
"peer-r1": {"PublicID": "router-ha", "PeerGroups": ["grp-routers"], "Masquerade": true, "Metric": 9999, "Enabled": true},
"peer-r2": {"PublicID": "router-ha", "PeerGroups": ["grp-routers"], "Masquerade": true, "Metric": 9999, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "Subnet network resource behind one directly-assigned router, with masquerade off and a non-default metric so both reach the wire verbatim, and an all-protocol policy from a two-peer source group. peer-r's route firewall rule must list both source peers; peer-b confirms a second client in the same group gets its own identical route.",
"peers": ["peer-a", "peer-b", "peer-r"]
}

View File

@@ -0,0 +1,55 @@
{
"Serial": "21",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"allowedIps": [
"100.64.0.9/32"
],
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-subnet:peer-r",
"Network": "10.20.0.0/24",
"NetworkType": "1",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "500",
"NetID": "office-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,55 @@
{
"Serial": "21",
"peerConfig": {
"address": "100.64.0.2/10",
"sshConfig": {},
"fqdn": "peer-b.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"allowedIps": [
"100.64.0.9/32"
],
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-subnet:peer-r",
"Network": "10.20.0.0/24",
"NetworkType": "1",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "500",
"NetID": "office-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-b.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.2"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,76 @@
{
"Serial": "21",
"peerConfig": {
"address": "100.64.0.9/10",
"sshConfig": {},
"fqdn": "peer-r.netbird.test",
"RoutingPeerDnsResolutionEnabled": true,
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
"allowedIps": [
"100.64.0.2/32"
],
"sshConfig": {},
"fqdn": "peer-b.netbird.test",
"agentVersion": "0.60.0"
},
{
"wgPubKey": "vblMc9U8RAI6cVopcKEMTVT6lVC3D9nTTMSwot5d3L4=",
"allowedIps": [
"100.64.0.1/32"
],
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"agentVersion": "0.60.0"
}
],
"Routes": [
{
"ID": "res-subnet:peer-r",
"Network": "10.20.0.0/24",
"NetworkType": "1",
"Peer": "ImPDKs2PJxHA24/N7umWi8lfEf2B0B5W/7dYZUzNS3s=",
"Metric": "500",
"NetID": "office-subnet",
"keepRoute": true
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-r.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.9"
}
]
}
],
"ForwarderPort": "22054"
},
"firewallRulesIsEmpty": true,
"routesFirewallRules": [
{
"sourceRanges": [
"100.64.0.1/32",
"100.64.0.2/32"
],
"destination": "10.20.0.0/24",
"protocol": "ALL",
"portInfo": {},
"PolicyID": "cG9sLXN1Ym5ldA==",
"RouteID": "res-subnet:peer-r"
}
],
"sshAuth": {
"UserIDClaim": "sub"
}
}

View File

@@ -0,0 +1,43 @@
{
"Network": {"Serial": 21},
"AccountSettings": {"RoutingPeerDNSResolutionEnabled": true},
"Peers": {
"peer-a": {"IP": "100.64.0.1", "Meta": {"WtVersion": "0.60.0"}},
"peer-b": {"IP": "100.64.0.2", "Meta": {"WtVersion": "0.60.0"}},
"peer-r": {"IP": "100.64.0.9", "Meta": {"WtVersion": "0.60.0"}}
},
"Groups": {
"grp-dev": {"Peers": ["peer-a", "peer-b"]}
},
"Policies": [
{
"ID": "pol-subnet",
"Enabled": true,
"Rules": [
{
"Enabled": true,
"Action": "accept",
"Protocol": "all",
"Sources": ["grp-dev"],
"DestinationResource": {"ID": "res-subnet", "Type": "subnet"}
}
]
}
],
"ResourcePolicies": {"res-subnet": [{"ID": "pol-subnet"}]},
"NetworkResources": [
{
"ID": "res-subnet",
"NetworkID": "net-1",
"Name": "office-subnet",
"Type": "subnet",
"Prefix": "10.20.0.0/24",
"Enabled": true
}
],
"Routers": {
"net-1": {
"peer-r": {"PublicID": "router-direct", "Metric": 500, "Enabled": true}
}
}
}

View File

@@ -0,0 +1,4 @@
{
"description": "Direct peer-to-peer policy via Source/DestinationResource of type peer, no groups involved; peer-a and peer-b see each other, bystander peer-c sees nobody.",
"peers": ["peer-a", "peer-b", "peer-c"]
}

View File

@@ -0,0 +1,64 @@
{
"Serial": "15",
"peerConfig": {
"address": "100.64.0.1/10",
"sshConfig": {},
"fqdn": "peer-a.netbird.test",
"autoUpdate": {}
},
"remotePeers": [
{
"wgPubKey": "AvldyrZ12Pf90jzf3AXmhPwg3UcI+jtJHfbpBlupvko=",
"allowedIps": [
"100.64.0.2/32"
],
"sshConfig": {},
"fqdn": "peer-b.netbird.test",
"agentVersion": "0.60.0"
}
],
"DNSConfig": {
"ServiceEnable": true,
"CustomZones": [
{
"Domain": "netbird.test.",
"Records": [
{
"Name": "peer-a.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.1"
},
{
"Name": "peer-b.netbird.test",
"Type": "1",
"Class": "IN",
"TTL": "300",
"RData": "100.64.0.2"
}
]
}
],
"ForwarderPort": "22054"
},
"FirewallRules": [
{
"PeerIP": "100.64.0.2",
"Protocol": "TCP",
"Port": "443",
"PolicyID": "cG9sLWRpcmVjdA=="
},
{
"PeerIP": "100.64.0.2",
"Direction": "OUT",
"Protocol": "TCP",
"Port": "443",
"PolicyID": "cG9sLWRpcmVjdA=="
}
],
"routesFirewallRulesIsEmpty": true,
"sshAuth": {
"UserIDClaim": "sub"
}
}

Some files were not shown because too many files have changed in this diff Show More