sendnrw/netbird
2
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-10-24 06:19:08 +02:00
Code Issues Packages Projects Releases Wiki Activity
3,530 Commits 616 Branches 388 Tags
netmap_progressive_alignment
Commit Graph
49 Commits
Author SHA1 Message Date
dmitri-netbird 8fef487e42 [management] Add support for pagination in GetAllPeers call (#8190) 
* use libopenapi OpenAPI generator

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* added a handler for v1alpha1/peers

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* wire up request validator

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* wired up spec-based validation

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* use sync validation; set base url to v1alpha1

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* moved stuff around, extracted runtime libopenapu deps into runtime_tooling

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* testing /peers path params

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* fixed tests

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* added user schemas and paths

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* use api validation in tests

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* post-merge fixes

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* deleted tmp command used to test validator integration

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* require request body in post/put requests in order to force validation

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* making linter happy

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* switch to api/v1alpha1

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* go mod tidy

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* keep the original order of middleware: metrics, cors, then auth

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* return 422 on validation errors

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* cleanups

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* more cleanups

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* v1alpha1 spec cleanups

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* no need for a double-pointer

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* more linter fixes

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* removed more double pointers

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* use di to inject api_v0 and api_v1 http routers

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* do not re-add middleware on repeated call to ApiHandler

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* support for pagination and filtering in /peers call

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* calling ApiRouter() now also calls ApiV1Router()

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* adding peer filters

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* introduced paginated GetAllPeers call

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* wire up pagination

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* convert pagination state to query scope

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* peer filter wip

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* implemented peer filter

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* test for filtering by os

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* support for sorting of peers

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* support for ordering + tests

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* more tests

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* wire up total number of records in paginated GetAllPeers call

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* cleaned up GetAllPeers response

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* pacify linter

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* cleanups and fixes

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* moar cleanups

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* pacify linter

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* remove duplicated test

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* make linter happy

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* moving GetAllPeersPaginated to peer manager, p. 1

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* moving GetAllPeersPaginated to peer manager, p. 2

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* added peer manager tests for GetPeersPaginated

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* order peers in GetAccountPeersPaginated by their ids by default

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* always order by peer_id, filter out proxies

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* fixed broken tests

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* support for filtering of peers by peer_id in GetAllPeers()

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* filtering by peer_ids

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* use lowercase in 'like' queries, drop ordering by ip

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* drop sort_by ip key

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* drop forced_approve filter

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* replace mac with darwin in peer os filter

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* fix peers query parameter: use peer_ids

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* fixed broken peer_test

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

* responded to feedback

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>

---------

Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
 2026-10-19 22:19:39 +02:00
Pascal Fischer 454ff66518 [management] scope network router update call (#6222) 2026-05-20 18:24:00 +02:00
Pascal Fischer fa0d58d093 [management] exclude peers for expiration job that have already been marked expired (#5970) 2026-04-23 16:01:54 +02:00
Pascal FischerAlisdair MacLeodmlsmayconEduard GertViktor LiuDiego NoguêsDiego NoguêsBethuel MmbagaZoltan PappAshley Mensah
 		f53155562f [management, reverse proxy] Add reverse proxy feature (#5291) 
* implement reverse proxy


---------

Co-authored-by: Alisdair MacLeod <git@alisdairmacleod.co.uk>
Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
Co-authored-by: Eduard Gert <kontakt@eduardgert.de>
Co-authored-by: Viktor Liu <viktor@netbird.io>
Co-authored-by: Diego Noguês <diego.sure@gmail.com>
Co-authored-by: Diego Noguês <49420+diegocn@users.noreply.github.com>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
Co-authored-by: Ashley Mensah <ashleyamo982@gmail.com>
 2026-02-13 19:37:43 +01:00
Vlad 684fc0d2a2 [management] fix the issue with duplicated peers with the same key (#5053) 2026-01-09 11:49:26 +01:00
Maycon Santos 2c81cf2c1e [management] Add account onboarding (#4084) 
This PR introduces a new onboarding feature to handle such flows in the dashboard by defining an AccountOnboarding model, persisting it in the store, exposing CRUD operations in the manager and HTTP handlers, and updating API schemas and tests accordingly.

Add AccountOnboarding struct and embed it in Account
Extend Store and DefaultAccountManager with onboarding methods and SQL migrations
Update HTTP handlers, API types, OpenAPI spec, and add end-to-end tests
 2025-07-03 09:01:32 +02:00
Pascal Fischer 22678bce7f [management] add uniqueness constraint for peer ip and label and optimize generation (#4042) 2025-07-02 18:13:10 +02:00
Bethuel Mmbaga 7c6b85b4cb [management] Refactor routes to use store methods (#2928) 2025-06-18 16:40:29 +03:00
Vlad adf494e1ac [management] fix a bug with missed extra dns labels for a new peer (#3798) 2025-05-14 17:50:21 +02:00
Misha Bragin c69df13515 [management] Add account meta (#3724) 2025-04-23 18:44:22 +02:00
Pascal Fischer 5ea2806663 [management] use permission modules (#3622) 2025-04-10 11:06:52 +02:00
Maycon Santos a4f04f5570 [management] fix extend call and move config to types (#3575) 
This PR fixes configuration inconsistencies and updates the store engine type usage throughout the management code. Key changes include:
- Replacing outdated server.Config references with types.Config and updating related flag variables (e.g. types.MgmtConfigPath).
- Converting engine constants (SqliteStoreEngine, PostgresStoreEngine, MysqlStoreEngine) to use types.Engine for consistent type–safety.
- Adjusting various test and migration code paths to correctly reference the new configuration and engine types.
 2025-03-27 13:04:50 +01:00
Pedro Maia Costa b64bee35fa [management] faster server bootstrap (#3365) 
Faster server bootstrap by counting accounts rather than fetching all from storage in the account manager instantiation.

This change moved the deprecated need to ensure accounts have an All group to tests instead.
 2025-02-22 11:31:39 +01:00
Bethuel MmbagaPascal FischerPascal FischerPedro Costa
 		4cdb2e533a [management] Refactor users to use store methods (#2917) 
* Refactor setup key handling to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add lock to get account groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add check for regular user

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* get only required groups for auto-group validation

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add account lock and return auto groups map on validation

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor account peers update

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor groups to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor GetGroupByID and add NewGroupNotFoundError

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add AddPeer and RemovePeer methods to Group struct

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Preserve store engine in SqlStore transactions

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Run groups ops in transaction

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix missing group removed from setup key activity

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor posture checks to remove get and save account

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix refactor

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix sonar

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Change setup key log level to debug for missing group

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Retrieve modified peers once for group events

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor policy get and save account to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Retrieve policy groups and posture checks once for validation

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor anyGroupHasPeers to retrieve all groups once

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor dns settings to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add account locking and merge group deletion methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor name server groups to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add peer store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor ephemeral peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add lock for peer store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor peer handlers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor peer to use store methods

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add locks and remove log

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* run peer ops in transaction

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* remove duplicate store method

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix peer fields updated after save

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Use update strength and simplify check

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* prevent changing ruleID when not empty

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* prevent duplicate rules during updates

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix lint

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor auth middleware

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor account methods and mock

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor user and PAT handling

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Remove db query context and fix get user by id

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix database transaction locking issue

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Use UTC time in test

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add account locks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix prevent users from creating PATs for other users

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add store locks and prevent fetching setup keys peers when retrieving user peers with empty userID

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add missing tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor test names and remove duplicate TestPostgresql_SavePeerStatus

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add account locks and remove redundant ephemeral check

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Retrieve all groups for peers and restrict groups for regular users

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix store tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* use account object to get validated peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Improve peer performance

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Get account direct from store without buffer

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Add get peer groups tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Adjust benchmarks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Adjust benchmarks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Update benchmark workflow (#3181)

* update local benchmark expectations

* update cloud expectations

* Add status error for generic result error

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Use integrated validator direct

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

* update expectations

* update expectations

* Refactor peer scheduler to retry every 3 seconds on errors

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

* fix validator

* fix validator

* fix validator

* update timeouts

* Refactor ToGroupsInfo to process slices of groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

* update expectations

* update expectations

* Bump integrations version

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor GetValidatedPeers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* go mod tidy

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Use peers and groups map for peers validation

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* remove mysql from api benchmark tests

* Fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix blocked db calls on user auto groups update

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Skip user check for system initiated peer deletion

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Remove context in db calls

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Improve group peer/resource counting (#3192)

* Fix sonar

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Adjust bench expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Rename GetAccountInfoFromPAT to GetTokenInfo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Remove global account lock for ListUsers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* build userinfo after updating users in db

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Optimize user bulk deletion  (#3315)

* refactor building user infos

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* remove unused code

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor GetUsersFromAccount to return a map of UserInfo instead of a slice

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Export BuildUserInfosForAccount to account manager

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fetch account user info once for bulk users save

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update user deletion expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Set max open conns for activity store

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update bench expectations

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com>
Co-authored-by: Pascal Fischer <pascal@netbird.io>
Co-authored-by: Pedro Costa <550684+pnmcosta@users.noreply.github.com>
 2025-02-17 21:43:12 +03:00
Viktor Liu 97d498c59c [misc, client, management] Replace Wiretrustee with Netbird (#3267) 2025-02-05 16:49:41 +01:00
M. Essam 7d385b8dc3 [management] REST client package (#3278) 2025-02-04 10:10:10 +00:00
Bethuel Mmbaga 1ad2cb5582 [management] Refactor peers to use store methods (#2893) 2025-01-20 18:41:46 +01:00
Bethuel Mmbaga 02a3feddb8 [management] Add MySQL Support (#3108) 
* Add mysql store support
* Add support to disable activity events recording
 2025-01-06 13:38:30 +01:00
Viktor LiuPascal FischerbcmmbagaMaycon SantosZoltan Papp
 		ddc365f7a0 [client, management] Add new network concept (#3047) 
---------

Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com>
Co-authored-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
 2024-12-20 11:30:28 +01:00
Bethuel Mmbaga 9683da54b0 [management] Refactor nameserver groups to use store methods (#2888) 2024-11-26 17:39:04 +01:00
Bethuel Mmbaga f118d81d32 [management] Refactor policy to use store methods (#2878) 2024-11-26 10:46:05 +01:00
Bethuel Mmbaga ca12bc6953 [management] Refactor posture check to use store methods (#2874) 2024-11-25 16:26:24 +01:00
Bethuel MmbagaandMaycon Santos 7bda385e1b [management] Optimize network map updates (#2718) 
* Skip peer update on unchanged network map (#2236)

* Enhance network updates by skipping unchanged messages

Optimizes the network update process
by skipping updates where no changes in the peer update message received.

* Add unit tests

* add locks

* Improve concurrency and update peer message handling

* Refactor account manager network update tests

* fix test

* Fix inverted network map update condition

* Add default group and policy to test data

* Run peer updates in a separate goroutine

* Refactor

* Refactor lock

* Fix peers update by including NetworkMap and posture Checks

* go mod tidy

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Skip account peers update if no changes affect peers (#2310)

* Remove incrementing network serial and updating peers after group deletion

* Update account peer if posture check is linked to policy

* Remove account peers update on saving setup key

* Refactor group link checking into re-usable functions

* Add HasPeers function to group

* Refactor group management

* Optimize group change effects on account peers

* Update account peers if ns group has peers

* Refactor group changes

* Optimize account peers update in DNS settings

* Optimize update of account peers on jwt groups sync

* Refactor peer account updates for efficiency

* Optimize peer update on user deletion and changes

* Remove condition check for network serial update

* Optimize account peers updates on route changes

* Remove UpdatePeerSSHKey method

* Remove unused isPolicyRuleGroupsEmpty

* Add tests for peer update behavior on posture check changes

* Add tests for peer update behavior on policy changes

* Add tests for peer update behavior on group changes

* Add tests for peer update behavior on dns settings changes

* Refactor

* Add tests for peer update behavior on name server changes

* Add tests for peer update behavior on user changes

* Add tests for peer update behavior on route changes

* fix tests

* Add tests for peer update behavior on setup key changes

* Add tests for peer update behavior on peers changes

* fix merge

* Fix tests

* go mod tidy

* Add NameServer and Route comparators

* Update network map diff logic with custom comparators

* Add tests

* Refactor duplicate diff handling logic

* fix linter

* fix tests

* Refactor policy group handling and update logic.

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update route check by checking if group has peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor posture check policy linking logic

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Simplify peer update condition in DNS management

Refactor the condition for updating account peers to remove redundant checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add posture checks tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix user and setup key tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix account and route tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix routes tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* upgrade diff package

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* use generic differ for netip.Addr and netip.Prefix

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* go mod tidy

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add peer tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix management suite tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix postgres tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* enable diff nil structs comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip the update only last sent the serial is larger

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor peer and user

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for groupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor group, ns group, policy and posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for GroupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update account policy check before verifying policy status

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* add tests missing tests for dns setting groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add tests for posture checks changes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add ns group and policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add route and group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* increase Linux test timeout to 10 minutes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Run diff for client posture checks only

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add panic recovery and detailed logging in peer update comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
 2024-10-23 13:05:02 +03:00
pascal-fischer 8284ae959c [management] Move testdata to sql files (#2693) 2024-10-10 12:35:03 +02:00
pascal-fischer 158936fb15 [management] Remove file store (#2689) 2024-10-03 15:50:35 +02:00
pascal-fischer 6c50b0c84b [management] Add transaction to addPeer (#2469) 
This PR removes the GetAccount and SaveAccount operations from the AddPeer and instead makes use of gorm.Transaction to add the new peer.
 2024-09-16 15:47:03 +02:00
benniekiss 12c36312b5 [management] Auto update geolite (#2297) 
introduces helper functions to fetch and verify database versions, downloads new files if outdated, and deletes old ones. It also refactors filename handling to improve clarity and consistency, adding options to disable auto-updating via a flag. The changes aim to simplify GeoLite database management for admins.
 2024-09-09 18:27:42 +02:00
Viktor Liu 199bf73103 Remove usage stats (#1665) 2024-03-05 09:45:32 +01:00
Viktor LiuandYury Gargay b7a6cbfaa5 Add account usage logic (#1567) 
---------

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
 2024-02-22 12:27:08 +01:00
Yury Gargay 9bc7b9e897 Add initial support of device posture checks (#1540) 
This PR implements the following posture checks:

* Agent minimum version allowed
* OS minimum version allowed
* Geo-location based on connection IP

For the geo-based location, we rely on GeoLite2 databases which are free IP geolocation databases. MaxMind was tested and we provide a script that easily allows to download of all necessary files, see infrastructure_files/download-geolite2.sh.

The OpenAPI spec should extensively cover the life cycle of current version posture checks.
 2024-02-20 09:59:56 +01:00
Yury Gargay 32880c56a4 Implement SQLite Store using gorm and relational approach (#1065) 
Restructure data handling for improved performance and flexibility. 
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management. 
Eliminate complexity in lookup tables for enhanced query and write speed. 
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
 2023-10-12 15:42:36 +02:00
Pascal Fischer e30def175b switch PATs to map and add deletion 2023-03-20 16:14:55 +01:00
Pascal Fischer b8cab2882b storing and retrieving PATs 2023-03-16 15:57:44 +01:00
Givi Khojanashvili 3bfa26b13b Feat rego default policy (#700) 
Converts rules to Rego policies and allow users to write raw policies to set up connectivity and firewall on the clients.
 2023-03-13 18:14:18 +04:00
Misha Bragin 9ce8056b17 Use global login expiration setting when sending network map (#731) 
Peers were considered expired and not sent to remote peers
when global expiration was disabled.
 2023-03-09 11:24:42 +01:00
Misha Bragin ed4f90b6aa Report offline peers to agents (#728) 
The peer login expiration ACL check introduced in #714
filters out peers that are expired and agents receive a network map 
without that expired peers.
However, the agents should see those peers in status "Disconnected".

This PR extends the Agent <-> Management protocol 
by introducing a new field OfflinePeers
that contain expired peers. Agents keep track of those and display 
then just in the Status response.
 2023-03-07 10:17:25 +01:00
Misha Bragin 87631cbc8b Replace IP allocation logic (#342) 
The peer IP allocation logic was allocating sequential peer IP from the 100.64.0.0/10 
address block.
Each account is created with a random subnet from 100.64.0.0/10.
The total amount of potential subnets is 64.
The new logic allocates random peer IP
from the account subnet.
This gives us flexibility to add support for
multi subnet accounts without overlapping IPs.
 2022-05-29 22:43:39 +02:00
Maycon Santos 605ca03519 Fix IDP Manager config structs with correct tags (#253) 
* Fix IDP Manager config structs with correct tags

When loading the configuration from file
we will use the Auth0ClientConfig and when
sending the post to retrieve a token
 we use the auth0JWTRequest with proper tags

 Also, removed the idle timeout as it was closing
 all idle connections
 2022-03-09 17:22:47 +01:00
Maycon Santosandbraginini 0b8387bd2c Group users of same private domain (#243) 
* Added Domain Category field and fix store tests

* Add GetAccountByDomain method

* Add Domain Category to authorization claims

* Initial GetAccountWithAuthorizationClaims test cases

* Renamed Private Domain map and index it on saving account

* New Go build tags

* Added NewRegularUser function

* Updated restore to account for primary domain account

Also, added another test case

* Added grouping user of private domains

Also added auxiliary methods for update metadata and domain attributes

* Update http handles get account method and tests

* Fix lint and document another case

* Removed unnecessary log

* Move use cases to method and add flow comments

* Split the new user and existing logic from GetAccountWithAuthorizationClaims

* Review: minor corrections

Co-authored-by: braginini <bangvalo@gmail.com>
 2022-03-01 15:22:18 +01:00
Maycon Santos fd7282d3cf Link account id with the external user store (#184) 
* get account id from access token claim

* use GetOrCreateAccountByUser and add test

* correct account id claim

* remove unused account

* Idp manager interface

* auth0 idp manager

* use if instead of switch case

* remove unnecessary lock

* NewAuth0Manager

* move idpmanager to its own package

* update metadata when accountId is not supplied

* update tests with idpmanager field

* format

* new idp manager and config support

* validate if we fetch the interface before converting to string

* split getJWTToken

* improve tests

* proper json fields and handle defer body close

* fix ci lint notes

* documentation and proper defer position

* UpdateUserAppMetadata tests

* update documentation

* ManagerCredentials interface

* Marshal and Unmarshal functions

* fix tests

* ManagerHelper and ManagerHTTPClient

* further tests with mocking

* rename package and custom http client

* sync local packages

* remove idp suffix
 2022-01-24 11:21:30 +01:00
Mikhail Bragin 6ae27c9a9b Refactor: support multiple users under the same account (#170) 
* feature: add User entity to Account

* test: new file store creation test

* test: add FileStore persist-restore tests

* test: add GetOrCreateAccountByUser Accountmanager test

* refactor: rename account manager users file

* refactor: use userId instead of accountId when handling Management HTTP API

* fix: new account creation for every request

* fix: golint

* chore: add account creator to Account Entity to identify who created the account.

* chore: use xid ID generator for account IDs

* fix: test failures

* test: check that CreatedBy is stored when account is stored

* chore: add account copy method

* test: remove test for non existent GetOrCreateAccount func

* chore: add accounts conversion function

* fix: golint

* refactor: simplify admin user creation

* refactor: move migration script to a separate package
 2021-12-27 13:17:15 +01:00
braginini 081162864d fix: management config 2021-09-07 10:35:33 +02:00
Mikhail Bragin a4db0b4e94 client update of TURNs and STUNs (#106) 
* feature: update STUNs and TURNs in engine

* fix: setup TURN credentials request only when refresh enabled

* feature: update TURNs and STUNs in teh client app on Management update

* chore: disable peer reflexive candidates in ICE

* chore: relocate management.json

* chore: make TURN secret and pwd plain text in config
 2021-09-03 17:47:40 +02:00
Mikhail Braginandmlsmaycon b17424d630 Turn credentials generation (#102) 
* abstract peer channel

* remove wip code

* refactor NewServer with Peer updates channel

* feature: add TURN credentials manager

* hmac logic

* example test function

* test: add TimeBasedAuthSecretsManager_GenerateCredentials  test

* test: make tests for now with hardcoded secret

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_CancelRefresh test

* feature: extract TURNConfig to the management config

* feature: return hash based TURN credentials only on initial sync

* feature: make TURN time based secret credentials optional

Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
 2021-09-02 14:41:54 +02:00
braginini 708835afa8 test: change test data to support new setup key fields 2021-08-20 13:40:07 +02:00
braginini 08d44b1d5f refactor: move LetsEncryptDomain to HttpServer config 2021-08-07 13:35:52 +02:00
andpar83 11982d6dde Add client's interaction with management service (#71) 
* Add client's interaction with management service

* Getting updates

* Fixed key and nil ptr

* Added setupKey param

* Added managment address parameter

* Fixed test

* feature: use RemotePeers from the management server instead of deprecated Peers

* merge: merge changes from main
 2021-08-01 19:06:01 +02:00
Mikhail Bragin 2c2c1e19df Peer configuration management (#69) 
* feature: add config properties to the SyncResponse of the management gRpc service

* fix: lint errors

* chore: modify management protocol according to the review notes

* fix: management proto fields sequence

* feature: add proper peer configuration to be synced

* chore: minor changes

* feature: finalize peer config management

* fix: lint errors

* feature: add management server config file

* refactor: extract hosts-config to a separate file

* refactor: review notes applied to correct file_store usage

* refactor: extract management service configuration to a file

* refactor: simplify management config
 2021-07-30 17:46:38 +02:00
andpar83 9e4aa4f1f1 Move management server to a separate directory (#67) 
* Move management server to a separate directory
 2021-07-24 16:14:29 +02:00