mirror of
https://github.com/netbirdio/netbird.git
synced 2026-04-16 15:26:40 +00:00
7b64953eedfb271f64fe290a112f3e41cb832fe6
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bethuel Mmbaga
|
4cdb2e533a |
[management] Refactor users to use store methods (#2917)
* Refactor setup key handling to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add lock to get account groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add check for regular user Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * get only required groups for auto-group validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add account lock and return auto groups map on validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor account peers update Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor groups to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor GetGroupByID and add NewGroupNotFoundError Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add AddPeer and RemovePeer methods to Group struct Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Preserve store engine in SqlStore transactions Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Run groups ops in transaction Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix missing group removed from setup key activity Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor posture checks to remove get and save account Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix refactor Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix sonar Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Change setup key log level to debug for missing group Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve modified peers once for group events Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor policy get and save account to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve policy groups and posture checks once for validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor anyGroupHasPeers to retrieve all groups once Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor dns settings to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locking and merge group deletion methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor name server groups to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add peer store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor ephemeral peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add lock for peer store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor peer handlers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor peer to use store methods Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add locks and remove log Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * run peer ops in transaction Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove duplicate store method Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix peer fields updated after save Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use update strength and simplify check Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * prevent changing ruleID when not empty Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * prevent duplicate rules during updates Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix lint Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor auth middleware Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor account methods and mock Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor user and PAT handling Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove db query context and fix get user by id Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix database transaction locking issue Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use UTC time in test Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix prevent users from creating PATs for other users Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add store locks and prevent fetching setup keys peers when retrieving user peers with empty userID Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add missing tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor test names and remove duplicate TestPostgresql_SavePeerStatus Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add account locks and remove redundant ephemeral check Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Retrieve all groups for peers and restrict groups for regular users Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix store tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * use account object to get validated peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Improve peer performance Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Get account direct from store without buffer Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Add get peer groups tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust benchmarks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust benchmarks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Update benchmark workflow (#3181) * update local benchmark expectations * update cloud expectations * Add status error for generic result error Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use integrated validator direct Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * update expectations * update expectations * Refactor peer scheduler to retry every 3 seconds on errors Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * fix validator * fix validator * fix validator * update timeouts * Refactor ToGroupsInfo to process slices of groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations * update expectations * update expectations * Bump integrations version Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor GetValidatedPeers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * go mod tidy Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Use peers and groups map for peers validation Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove mysql from api benchmark tests * Fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix blocked db calls on user auto groups update Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Skip user check for system initiated peer deletion Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove context in db calls Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Improve group peer/resource counting (#3192) * Fix sonar Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Adjust bench expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Rename GetAccountInfoFromPAT to GetTokenInfo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Remove global account lock for ListUsers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * build userinfo after updating users in db Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Optimize user bulk deletion (#3315) * refactor building user infos Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * remove unused code Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor GetUsersFromAccount to return a map of UserInfo instead of a slice Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Export BuildUserInfosForAccount to account manager Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fetch account user info once for bulk users save Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update user deletion expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Set max open conns for activity store Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update bench expectations Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com> Co-authored-by: Pascal Fischer <pascal@netbird.io> Co-authored-by: Pedro Costa <550684+pnmcosta@users.noreply.github.com> |
||
|
M. Essam
|
7d385b8dc3 | [management] REST client package (#3278) | ||
|
Bethuel Mmbaga
|
02a3feddb8 |
[management] Add MySQL Support (#3108)
* Add mysql store support * Add support to disable activity events recording |
||
|
Viktor Liu
|
ddc365f7a0 |
[client, management] Add new network concept (#3047)
--------- Co-authored-by: Pascal Fischer <32096965+pascal-fischer@users.noreply.github.com> Co-authored-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com> |
||
|
Bethuel Mmbaga
|
7bda385e1b |
[management] Optimize network map updates (#2718)
* Skip peer update on unchanged network map (#2236) * Enhance network updates by skipping unchanged messages Optimizes the network update process by skipping updates where no changes in the peer update message received. * Add unit tests * add locks * Improve concurrency and update peer message handling * Refactor account manager network update tests * fix test * Fix inverted network map update condition * Add default group and policy to test data * Run peer updates in a separate goroutine * Refactor * Refactor lock * Fix peers update by including NetworkMap and posture Checks * go mod tidy * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * [management] Skip account peers update if no changes affect peers (#2310) * Remove incrementing network serial and updating peers after group deletion * Update account peer if posture check is linked to policy * Remove account peers update on saving setup key * Refactor group link checking into re-usable functions * Add HasPeers function to group * Refactor group management * Optimize group change effects on account peers * Update account peers if ns group has peers * Refactor group changes * Optimize account peers update in DNS settings * Optimize update of account peers on jwt groups sync * Refactor peer account updates for efficiency * Optimize peer update on user deletion and changes * Remove condition check for network serial update * Optimize account peers updates on route changes * Remove UpdatePeerSSHKey method * Remove unused isPolicyRuleGroupsEmpty * Add tests for peer update behavior on posture check changes * Add tests for peer update behavior on policy changes * Add tests for peer update behavior on group changes * Add tests for peer update behavior on dns settings changes * Refactor * Add tests for peer update behavior on name server changes * Add tests for peer update behavior on user changes * Add tests for peer update behavior on route changes * fix tests * Add tests for peer update behavior on setup key changes * Add tests for peer update behavior on peers changes * fix merge * Fix tests * go mod tidy * Add NameServer and Route comparators * Update network map diff logic with custom comparators * Add tests * Refactor duplicate diff handling logic * fix linter * fix tests * Refactor policy group handling and update logic. Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update route check by checking if group has peers Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor posture check policy linking logic Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Simplify peer update condition in DNS management Refactor the condition for updating account peers to remove redundant checks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add posture checks tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix user and setup key tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix account and route tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix typo Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix nameserver tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix routes tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix group tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * upgrade diff package Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix nameserver tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * use generic differ for netip.Addr and netip.Prefix Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * go mod tidy Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add peer tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix merge Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix management suite tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * fix postgres tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * enable diff nil structs comparison Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip the update only last sent the serial is larger Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * refactor peer and user Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip spell check for groupD Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Refactor group, ns group, policy and posture checks Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * skip spell check for GroupD Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * update account policy check before verifying policy status Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * Update management/server/route_test.go Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> * add tests missing tests for dns setting groups Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add tests for posture checks changes Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add ns group and policy tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add route and group tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * increase Linux test timeout to 10 minutes Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Run diff for client posture checks only Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * add panic recovery and detailed logging in peer update comparison Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> * Fix tests Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> --------- Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Maycon Santos <mlsmaycon@gmail.com> |
||
|
pascal-fischer
|
8284ae959c | [management] Move testdata to sql files (#2693) |