9 Commits

Author	SHA1	Message	Date
Vlad	684fc0d2a2	[management] fix the issue with duplicated peers with the same key (#5053)	2026-01-09 11:49:26 +01:00
Maycon Santos	2c81cf2c1e	[management] Add account onboarding (#4084) ... This PR introduces a new onboarding feature to handle such flows in the dashboard by defining an AccountOnboarding model, persisting it in the store, exposing CRUD operations in the manager and HTTP handlers, and updating API schemas and tests accordingly. Add AccountOnboarding struct and embed it in Account Extend Store and DefaultAccountManager with onboarding methods and SQL migrations Update HTTP handlers, API types, OpenAPI spec, and add end-to-end tests	2025-07-03 09:01:32 +02:00
Pascal Fischer	22678bce7f	[management] add uniqueness constraint for peer ip and label and optimize generation (#4042)	2025-07-02 18:13:10 +02:00
Bethuel Mmbaga	4cdb2e533a	[management] Refactor users to use store methods (#2917) ... * Refactor setup key handling to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add lock to get account groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add check for regular user Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * get only required groups for auto-group validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add account lock and return auto groups map on validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor account peers update Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor groups to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor GetGroupByID and add NewGroupNotFoundError Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add AddPeer and RemovePeer methods to Group struct Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Preserve store engine in SqlStore transactions Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Run groups ops in transaction Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix missing group removed from setup key activity Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor posture checks to remove get and save account Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix refactor Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix sonar Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Change setup key log level to debug for missing group Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve modified peers once for group events Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor policy get and save account to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve policy groups and posture checks once for validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor anyGroupHasPeers to retrieve all groups once Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor dns settings to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locking and merge group deletion methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor name server groups to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add peer store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor ephemeral peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add lock for peer store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor peer handlers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor peer to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add locks and remove log Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * run peer ops in transaction Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove duplicate store method Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix peer fields updated after save Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use update strength and simplify check Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * prevent changing ruleID when not empty Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * prevent duplicate rules during updates Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix lint Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor auth middleware Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor account methods and mock Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor user and PAT handling Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove db query context and fix get user by id Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix database transaction locking issue Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use UTC time in test Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix prevent users from creating PATs for other users Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add store locks and prevent fetching setup keys peers when retrieving user peers with empty userID Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add missing tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor test names and remove duplicate TestPostgresql_SavePeerStatus Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locks and remove redundant ephemeral check Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve all groups for peers and restrict groups for regular users Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix store tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * use account object to get validated peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Improve peer performance Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Get account direct from store without buffer Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add get peer groups tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust benchmarks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust benchmarks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Update benchmark workflow (#3181) * update local benchmark expectations * update cloud expectations * Add status error for generic result error Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use integrated validator direct Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * update expectations * update expectations * Refactor peer scheduler to retry every 3 seconds on errors Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * fix validator * fix validator * fix validator * update timeouts * Refactor ToGroupsInfo to process slices of groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * update expectations * update expectations * Bump integrations version Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor GetValidatedPeers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * go mod tidy Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use peers and groups map for peers validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove mysql from api benchmark tests * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix blocked db calls on user auto groups update Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Skip user check for system initiated peer deletion Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove context in db calls Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Improve group peer/resource counting (#3192) * Fix sonar Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust bench expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Rename GetAccountInfoFromPAT to GetTokenInfo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove global account lock for ListUsers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * build userinfo after updating users in db Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Optimize user bulk deletion (#3315) * refactor building user infos Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove unused code Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor GetUsersFromAccount to return a map of UserInfo instead of a slice Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Export BuildUserInfosForAccount to account manager Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fetch account user info once for bulk users save Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update user deletion expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Set max open conns for activity store Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update bench expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com> Co-authored-by: Pascal Fischer <pascal@netbird.io> Co-authored-by: Pedro Costa <550684+pnmcosta@users.noreply.github.com>	2025-02-17 21:43:12 +03:00
M. Essam	7d385b8dc3	[management] REST client package (#3278)	2025-02-04 10:10:10 +00:00
Bethuel Mmbaga	02a3feddb8	[management] Add MySQL Support (#3108) ... * Add mysql store support * Add support to disable activity events recording	2025-01-06 13:38:30 +01:00
Viktor Liu	ddc365f7a0	[client, management] Add new network concept (#3047) ... --------- Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com> Co-authored-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>	2024-12-20 11:30:28 +01:00
Bethuel Mmbaga	7bda385e1b	[management] Optimize network map updates (#2718) ... * Skip peer update on unchanged network map (#2236) * Enhance network updates by skipping unchanged messages Optimizes the network update process by skipping updates where no changes in the peer update message received. * Add unit tests * add locks * Improve concurrency and update peer message handling * Refactor account manager network update tests * fix test * Fix inverted network map update condition * Add default group and policy to test data * Run peer updates in a separate goroutine * Refactor * Refactor lock * Fix peers update by including NetworkMap and posture Checks * go mod tidy * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Skip account peers update if no changes affect peers (#2310) * Remove incrementing network serial and updating peers after group deletion * Update account peer if posture check is linked to policy * Remove account peers update on saving setup key * Refactor group link checking into re-usable functions * Add HasPeers function to group * Refactor group management * Optimize group change effects on account peers * Update account peers if ns group has peers * Refactor group changes * Optimize account peers update in DNS settings * Optimize update of account peers on jwt groups sync * Refactor peer account updates for efficiency * Optimize peer update on user deletion and changes * Remove condition check for network serial update * Optimize account peers updates on route changes * Remove UpdatePeerSSHKey method * Remove unused isPolicyRuleGroupsEmpty * Add tests for peer update behavior on posture check changes * Add tests for peer update behavior on policy changes * Add tests for peer update behavior on group changes * Add tests for peer update behavior on dns settings changes * Refactor * Add tests for peer update behavior on name server changes * Add tests for peer update behavior on user changes * Add tests for peer update behavior on route changes * fix tests * Add tests for peer update behavior on setup key changes * Add tests for peer update behavior on peers changes * fix merge * Fix tests * go mod tidy * Add NameServer and Route comparators * Update network map diff logic with custom comparators * Add tests * Refactor duplicate diff handling logic * fix linter * fix tests * Refactor policy group handling and update logic. Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update route check by checking if group has peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor posture check policy linking logic Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Simplify peer update condition in DNS management Refactor the condition for updating account peers to remove redundant checks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add posture checks tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix user and setup key tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix account and route tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix nameserver tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix routes tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix group tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * upgrade diff package Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix nameserver tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * use generic differ for netip.Addr and netip.Prefix Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * go mod tidy Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add peer tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix management suite tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix postgres tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * enable diff nil structs comparison Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip the update only last sent the serial is larger Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor peer and user Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip spell check for groupD Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor group, ns group, policy and posture checks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip spell check for GroupD Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update account policy check before verifying policy status Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * add tests missing tests for dns setting groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add tests for posture checks changes Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add ns group and policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add route and group tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * increase Linux test timeout to 10 minutes Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Run diff for client posture checks only Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add panic recovery and detailed logging in peer update comparison Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>	2024-10-23 13:05:02 +03:00
pascal-fischer	8284ae959c	[management] Move testdata to sql files (#2693)	2024-10-10 12:35:03 +02:00