142 Commits

Include renames

Author	SHA1	Message	Date
4 people	f0a40e4395	[client, management] Harden the certificate posture client and keep challenge nonces fresh (#8052) ... * implement certificate posture check * log signal address * add keychain and cert store support * read the console user's keychain through a user session helper A root daemon cannot reach a login keychain: securityd is per session and a key ACL needs a session to prompt in, so dropping uid is not enough. The daemon now answers certificate challenges from the System keychain itself, where MDM installs device identities, and launches "netbird posture cert-proof" into the console user's desktop session with launchctl asuser for the login keychain. Only the signature and the chain cross back, never the private key. The console user comes from SCDynamicStoreCopyConsoleUser, bound with purego like the keychain calls. The login window reports no user, root, or "loginwindow", and all three are treated as no keychain to read, so a Mac at the lock screen sends device proofs alone. Adds info logging across the path: the keychain search list, per class query status and item counts, the chain built per candidate, and the verification error for every rejected candidate. A run that sends nothing now says why. README.md documents the trust model, the console user limitation and how to read the logs. * read the signed-in user's certificate store on Windows A service reads LocalMachine\MY, where AD and Intune enrol device certificates. CurrentUser\MY lives in the signed-in user's registry hive with keys protected against their profile, and a service that opens it does not fail: "current user" resolves to HKU\S-1-5-18, so it silently reads the service account's own empty store. The service therefore reads the machine store itself and launches "netbird posture cert-proof" with the session token for the rest, mirroring the macOS console user helper. Windows lets a privileged service assume a user identity, so the token goes straight into the child process and no external tooling is involved. CREATE_NO_WINDOW keeps a console window from flashing on the desktop every sync. In-process impersonation would also work but is per OS thread while goroutines migrate, so the child process avoids that class of bug. Session selection prefers the physical console and falls back to any active session, so remote desktop and VDI hosts are covered. WTSQueryUserToken needs SE_TCB_NAME, so a user-run client skips the helper and reads the machine store alone. SystemStore takes a store location, gaining NewUserStore alongside NewSystemStore and the per candidate logging macOS already had. The request building and proof merging move to helper_spawn.go, shared by both platforms, and helperStore picks what the helper reads per platform. * start TPM support * split goreleaser to support pkcs11 and exclude on docker * update goreleaser * go mod tidy * add tpm pin to netbird config * split cert and key location and allow key lookup on tpm * add unsupported flag for mobile devices * Isolate the cert proof helper from the service environment and cap its output * Read the PKCS#11 token PIN from NB_TPM_PIN instead of the profile config * Bound certificate proof collection so a stuck token or keychain cannot hold the sync loop * Stop retrying a PKCS#11 PIN the token rejected * Log certificate posture details at debug level * Sign only nonces and peer keys of the size management issues * Skip certificate files whose key belongs to another certificate * Bound PKCS#11 driver sizes, pin template values, and log out only a login the session owns * Never pass NULL to CFRelease and skip unreadable keychain identities * Keep the macOS keychain code out of iOS and the PKCS#11 driver out of Android * Find a chain to each challenge's CAs through every intermediate the store holds * Require a token label whenever a PKCS#11 PIN is set * Read user certificates only from the session of the active profile's owner * Collect certificate proofs again when the owner's session changes and report lost proofs * Test the PKCS#11 build against SoftHSM in CI and warn once where the build has no driver * Document where an inline PKCS#11 PIN is stored and how it is protected * Refuse PKCS#11 URIs that this client cannot honour instead of widening the match * Trust certificate and key files only when no other user can write or redirect them * Explain a Windows certificate whose key only a legacy CryptoAPI provider holds * Use platform absolute module paths in tests and add a real owner session test for Windows * Match the Windows profile owner by name instead of resolving it through the domain controller * Keep the certificate stores and TPM library out of the WebAssembly build * [client] Read TSS2 key files on go-tpm, checked against the library it replaces The TSS2 parser was the only reason this repository depended on a crypto suite whose own build tooling it inherits. The replacement sits on go-tpm, which was already a direct dependency and is in fact what that suite calls underneath, so this removes a wrapper rather than porting onto a different library: the load, the derived storage root key and the signing commands are the same calls. Swapping a parser on the one path a customer actually runs is not something to assert, so the two are held side by side for this commit. One test feeds the replacement bytes the old library wrote and requires the same key type, empty auth flag, parent handle, blobs and decoded public key; the other feeds both the fixtures the tests are built on, so those are the shape the format calls for and not merely the shape the new parser reads. The scaffolding goes away with the dependency in the commit that follows. The encoder behind the fixtures is written out separately from the parser under test, so an encoder bug and a decoder bug cannot cancel each other out. * [client] Drop go.step.sm/crypto and the repo-wide upgrades it imposed The TSS2 parser was the only thing in the repository that used this module, and it brought 302 modules into the graph to do it — 35 of them linters, along with Google Cloud KMS and IAM, the AWS SDK and a terminal styling library. Those are the module's own development dependencies, which minimal version selection turns into floors in ours, and they are the whole reason gRPC, protobuf, the AWS SDK, OpenTelemetry, logrus and five x/ packages had moved. Management, signal, relay and proxy inherited every one of them for a feature none of them runs. Removing the import is not enough, because tidy never downgrades: the raised floors stay written in go.mod. Each one is pinned back to the version main had, then tidy is left to raise again whatever something still genuinely needs. It raised nothing: all 43 are back where they were, and go-tpm was already in the graph at the same version, so the certificate feature now costs no new module at all. The differential tests go with it. They existed to check the swap against the library while both were present, and there is nothing left to compare against. * [client] Clear the lint findings only the macOS and Windows runners see golangci-lint analyses one build at a time, so running it on Linux says nothing about the two platforms CI also lints. Against those builds the feature's packages reported eight findings, and the structural one is Config.dir: it is dead on macOS and Windows because neither reads a directory at all, their collectors take the configuration and discard it. Moving the method beside its only callers makes that visible in the layout instead of in a linter, and leaves the gap itself — no file or token store on those platforms — where it belongs, as something to decide rather than something to silence. An absent key file beside a certificate was reported as a nil signer with a nil error, which the caller then had to recognise by its nilness. It is a sentinel now, so the meaning is in the error rather than in the absence of one. The rest follow the standard library: the elliptic coordinates and the private scalar come from the encoding helpers rather than the deprecated big.Int fields, and an error string loses its trailing colon. Lint is clean on linux, darwin and windows; the hardware TPM path was exercised separately against a real device and passes. * Accept the TSS2 emptyAuth boolean OpenSSL writes and persistent parents on 32-bit builds * Count the certificates field in the peer meta store test * Check the store directory before listing it, refuse group-writable files, and reject a URI with two PIN sources * Share a PKCS#11 login between sessions and send each PIN at most once at a time * Collect certificate proofs again when the meta sync carrying them failed * Use no Windows user store when a domainless owner matches accounts of several domains * Use no user certificate store when the active profile's owner cannot be read * Document the PIN sources on CertPKCS11URI and keep the README PIN example off the command line * Test that the PKCS#11 URI stays out of the debug bundle and run the wrong-PIN test only on a disposable token * Refuse a TPM PSS signature request for the maximum salt length * Add the certificate fields to the network map golden data * Retry posture checks whose meta sync timed out instead of dropping them * Start no system info gathering while a timed-out one is still running * Guard the applied posture checks across goroutines and keep refreshing proofs while a pending update times out * Log what a successful certificate proof helper wrote to stderr * Send recollected certificate proofs to management only when the proven chains changed * Explain a macOS keychain key whose access list does not allow netbird * Kill the whole macOS certificate helper process group when it times out * End sudo option parsing before the macOS certificate helper binary * Hold off system info gathering only while a timed-out one is still running * Collect certificate proofs on the posture watcher instead of under the sync lock * Read the certificate store directory and PKCS#11 URI from the daemon environment, not the profile config * Install the RPM sysconfig file readable by root only and show the certificate posture variables * Move the certificate posture README into the package doc and the docs site * Name NB_CERT_PKCS11_URI in the PIN-without-token error * Keep the file check results of the latest-started system info refresh * Give the full import command for a keychain key netbird may not use, and correct the package doc * Restrict the service environment file to root on every package install * Search only the System keychain in the macOS daemon and only the login keychain in the user helper * Let the certificate proof helper read the PKCS#11 token from the environment on Linux * Ask a macOS user's keychain again only after an hour when it proved nothing * Clear the lint findings in certificate posture * Hold off the keychain helper only after a completed or timed-out run, independent of CA order * Keep free functions out of the method lists of PKCS11Store, URI and Challenger * Name the post-install permission helper in snake case and shorten the sysconfig certificate block * Drop the certificate store directory from certproof.Config, which only NB_CERT_STORE_DIR sets * [management] Renew certificate challenge nonces on quiet accounts A certificate challenge nonce is accepted for its own window and the one before it, and it only reaches a peer attached to a network map. An account where nothing changes sends no map, so after a day the peer re-sends the nonce it still holds, verification rejects its whole proof set, and the certificates stored for it are dropped. It fails the certificate check and loses every policy gated on it until some unrelated change happens to push a map. The outage repairs itself in seconds, which is what makes it expensive: it is intermittent, it only hits stable networks, and it is not reproducible on demand. Push the account's peers an update often enough that the nonce they hold is never close to expiring. Only accounts whose posture checks actually ask for a certificate are tracked, so a deployment without the feature does no extra work. The refresh runs from one goroutine over a map of accounts rather than a timer per account: the period is hours, so one pass every few minutes costs nothing next to it, and there is no timer to re-arm when an account that falls due sooner appears. Each account's first run is offset by a hash of its ID, because the challenge window is global and an instance restart would otherwise arm every account in the same moment. The push carries no administrative change, so it is counted as a refresh rather than an update and stays out of the figures that track what was edited. (cherry picked from commit 7ad4a0df37) * [management] Make the certificate challenge window one knob to turn Renewal was timed against the window in two different ways: the period derived from it, the sweep interval did not. Shortening the window to watch a renewal in an end-to-end run would have left the refresher still looking for due accounts every quarter of an hour, so nothing would have been renewed in time and the test would have reported the feature broken. Derive the sweep from the period, within bounds that keep a very short window from spinning and a normal one from checking less often than is useful, and allow the window itself to be set through the environment so a run can take seconds instead of half a day. A value that cannot be parsed or falls outside the bounds keeps the default, because a window nobody intended is a security property nobody chose, and an override is logged at warning level since it sets how long a device keeps passing the check after its key is gone. Every instance has to be given the same value: the window is part of the nonce, so instances that disagree reject each other's. (cherry picked from commit 0e38fcf409) * [management] Pin the property that makes per-peer nonce state unnecessary A nonce carries the window it was minted in, not the instant, and is accepted for that window and the one before it. So a peer re-stamped at least once per window can never be left holding one outside the accepted pair, whenever it was last served and however much life its own nonce had left. That is the whole reason management tracks nothing per peer, and it was resting on an argument rather than a test. The phases are part of the property, not decoration: accounts are deliberately given a refresh phase of their own, so the guarantee has to hold off the window boundary too. The negative case shows why that matters — a cadence of exactly two windows lands inside the grace window when it is aligned to the boundary and leaves a gap when it is not. (cherry picked from commit dee68facfd) * [management] Renew challenges only for the peers that answer one The refresh pushed an update to every connected peer of the account, while only the peers a certificate check applies to carry a nonce. On an account where a handful of peers sit behind the check and the rest do not, everyone was woken several times a day to be handed a map that changed nothing for them. Push to the sources of the enabled policies whose posture checks include a certificate check, which is exactly the set that is sent a challenge. Resolving the set the other way round than the gRPC layer does is the risk here: a peer the refresh forgets stops being renewed and falls out of its policies silently, which is the failure this whole mechanism exists to prevent. So the selection is held against processPeerPostureChecks, the per-peer rule that decides who receives a challenge in the first place, by a test that asks both the same question and requires the same answer. (cherry picked from commit dc4d0e0274) * [management] Derive certificate challenge nonces from the stored encryption key The nonce secret came from the server's WireGuard key, which is generated afresh in every process and never persisted. A nonce carries no state, so the only thing that lets one instance verify what another issued is deriving the same secret — and that premise, written in the comment above the challenger, was not met: every instance had its own key. A peer reconnecting after a restart therefore presented a nonce minted under the previous secret, verification failed with a mismatch, its whole proof set was rejected and the certificates stored for it were dropped until it signed again. Reproduced three times on the lab, each one logging "nonce was not issued to this peer", which only a changed secret produces. On a single instance it costs seconds of lost policy access per restart; across instances it is not transient at all, because every reconnect that lands elsewhere is rejected the same way. Derive from the data store encryption key instead: it is generated once, written back to the configuration and read by every instance, so it survives restarts and is shared. Where none is configured the secret falls back to the WireGuard key with a warning — degraded but still unpredictable, which is the property that matters most: a peer able to guess it could mint the nonces of future windows, sign them while its key is present and keep passing after it is gone. The challenger is now built once and passed to the two places that need it, rather than re-derived per message. (cherry picked from commit 278f2f3807) * [management] Register an account for renewal where its nonce is issued Renewal was armed when a peer connected or when a posture check was saved, both of which ask the store whether the account has a certificate check. That misses the case it most needs to catch: the check is created through one instance while the peers are connected to another, so the instance serving them never learns it has anything to renew and their nonce expires. It also charged a query to every peer connect in every account, including the ones that will never use the feature, which a fleet reconnecting after a restart pays all at once. Register where the nonce is actually stamped instead. A nonce is verified from a shared secret and so travels between instances, but the renewal that keeps it fresh cannot: only the instance holding a peer's stream can push to it. Issuing and renewing now line up by construction — an instance renews exactly the accounts it has issued nonces for — and an instance that never issues one has nothing to renew, so there is no case left to miss. The registration is a map insert with no store access, which is what lets it sit on a path taken by every login and every initial sync. Reported by Viktor Liu, who also proposed registering at the point of issue. (cherry picked from commit 2d16dd7d7cf54762f2e64c5630ea092f32ef63ab) * [management] Register for renewal on pushed updates, not only on connect Registering where the nonce is stamped only covered the login and the initial sync, which both happen when a peer opens a stream. That left out the path the mechanism exists for. On the cloud the network map controller is wrapped so that an update publishes to an event bus instead of pushing locally: an instance handling a REST change broadcasts, and every instance holding a peer of that account pushes to its own. Those pushes stamp a nonce through the update handler, and nothing there registered, so an instance learned about an account only when one of its peers happened to reconnect. For a quiet fleet that is the original bug: the check is created, the peers are told about it, and nobody renews what they were told. Registering on the pushed update closes it, and is the difference between stamping and marking a peer connected — one happens on every push, the other only when a stream opens. Reported by Viktor Liu; the broadcast that makes it work was pointed out by Pascal Fischer. (cherry picked from commit 59efe8d93e53bacdf57cb546f4ab2c19dc4eddab) * [management] Let the challenge refresh loop stop with the manager that owns it The loop was started on a context explicitly detached from the caller's, so nothing could ever stop it. Production is unaffected either way, since BuildManager is called with context.Background(), but a test that builds a manager leaked a sweeping goroutine for the rest of the run, and a shutdown path added later would have had no way to reach it. Take the manager's context as the request buffer built on the line above already does. The test pins the contract the loop offers, so a detached context cannot come back inside Start either. * [management] Bound one account's challenge refresh so it cannot starve the rest Resolving which peers answer a challenge reads the store three times, and the refresher sweeps accounts one after another on a single goroutine. A read that never returns held the sweep for the life of the process, so every other account on the instance stopped being renewed and its peers fell out of the policies gated on the check: one account's bad luck became an outage for all of them. Give each refresh the sweep interval it is allowed to occupy, capped at 30s so a 12-hour window does not grant minutes to a query that should take milliseconds. A refresh that runs out of time keeps its account tracked, since a deadline says nothing about whether that account still has a certificate check. * [management] Send challenge refreshes down the path the rest of management uses The refresh dispatched through UpdateAffectedPeers, the one variant that takes no reason, so it was missing from the update counters and coalesced with nothing. An administrator editing a policy while the sweep ran made the account's network map twice over, and UpdateOperationRefresh, added for exactly this caller, was never referenced. Buffer it with a posture_check/refresh reason instead. The periodic push is now visible in the metrics as what it is, distinct from an edit, and the send detaches from the sweep deadline on its own, so that deadline bounds the store reads it was meant for. * [management] Keep the certificate challenge comments to what the history does not say Four of these ran to three and four times the comment budget, the longest at 992 characters. Most of the excess argued against designs that were never written or explained a bug that no longer exists in the code, which is what the commit that fixed it is for. What is left is the part a reader cannot recover from the code: that the nonce secret has to be persisted and unpredictable, that stamping and renewing are decided together because only the serving instance can push, and that the target rule is the inverse of processPeerPostureChecks. * Keep the newest posture checks pending whatever made their meta sync fail * Report no lost certificate when the engine stops during a proof collection * Share the proof collection single-flight across engine restarts * Close a PKCS#11 module that loads but cannot be used * Fix the pending checks comments * Renew certificate challenges only for the peers streamed to this instance * Ignore a challenge stamp from an older sync stream of the same peer * Kill the Windows certificate proof helper with its whole process tree * Expect the challenge untrack in the session ownership test * Drop an invalid certificate proof without discarding the valid ones * Start a system info gathering beside one that has been stuck for ten timeouts --------- Co-authored-by: pascal <pascal@netbird.io> Co-authored-by: mlsmaycon <mlsmaycon@gmail.com> Co-authored-by: riccardom <riccardomanfrin@gmail.com>	2026-10-09 15:42:18 +02:00
Pascal Fischerandmlsmaycon	53a14551c8	[client, management] implement certificate posture check (#7535) ... Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>	2026-10-09 14:57:00 +02:00
dmitri-netbird	ae7ad5e0c8	[management] Return permission-denied error when setup key is expired or invalid (#8086) ... * return permission-denied error when setup key is expired or invalid Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * fix tests, don't return permission-denied on internal errors Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * return not found error when key fetch failed Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * normalise errors returned from each call-site of GetSetupKeyBySecret Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * return PermissionDenied after isValid() check failure Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> --------- Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>	2026-10-07 10:21:24 +02:00
Maycon Santos	3c4358dd36	[management] Require a private proxy cluster for cluster and direct upstream targets (#7984) ... * [management] Require a private proxy cluster for cluster and direct upstream targets Cluster targets and direct upstream targets make the proxy dial the upstream from its own host network instead of through the embedded NetBird client. Only clusters running in private mode are meant to do that, but the service API accepted these targets on any cluster. Service create and update now reject such targets unless the service's proxy cluster reports the private capability. An unreported capability is treated as unsupported. * [management] Require every proxy in the cluster to be private The private capability is aggregated as any-true, so a cluster where only one proxy runs in private mode passed the check. The mapping is delivered to every proxy in the cluster, so the non-private ones would serve cluster and direct upstream targets from their host network too. Validate these targets against a unanimous aggregation instead. The existing any-true lookup stays as is for the dashboard flags and the agent network gateway.	2026-10-02 19:03:00 +02:00
Allan ELKAIM	e2678d4e05	[management] expose peer MAC addresses and make peers searchable by MAC (#6553)	2026-10-02 15:00:26 +02:00
Brad Ison	fd1a0203c7	[management] Clean up after account deletion (#7812) ... Deleting an account left state behind that DeleteAccount's store associations don't reach. The Agent Network tables outlived the account, keeping its gateway domain claimed and its provider API keys stored. The proxies kept serving its gateway until they next resynced. Cloud-side state, such as managed proxy deployments, had no way to be cleaned up at all. Account deletion now runs registered hooks after the permission check and before any users or data are removed. A failing hook aborts the deletion. Agent Network registers one that tells the proxies to drop the account's gateway mappings. The account's settings, providers, policies, guardrails and budget rules are deleted in the account's transaction. Consumption counters, and the access logs of deleted accounts, are left to the background cleanup; usage records are kept.	2026-09-30 18:25:38 +02:00
Pascal Fischer	39de33ceca	[management] handle db conn close on errpr (#7740)	2026-09-30 11:35:18 +02:00
Bethuel Mmbaga	30dd076b36	[management,proxy] Use single-use codes for OIDC session handoff (#7635) ... * Generalize PKCE verifier store into SingleUseStore * Generalize PKCE verifier store into SingleUseStore * Extend single-use store to generate one-time retrieval codes * Hand off proxy OIDC session via one-time code instead of URL token * Use the single-use store in integration tests * Read active proxy versions by cluster * Detect proxy clusters that support session codes * Bind OIDC session handoff mode to signed state * Deprecate legacy OIDC session token handoff * Remove unrelated session code test stub * fix tests * fix merge * Fix session code compatibility detection * Isolate proxy session codes in shared cache * bump min session version	2026-09-29 18:29:55 +03:00
Pascal Fischer	782c943410	[management] extract shared db conn + data repository (#7649) ... * extract shared db conn + data repository * extract repository interface * protect against nested transactions * fix mysql and db conn creation * fix context management * remove query warpper * remove withContext and withLock wrapper * remove context from function call * remove in memory mode * fix nested transaction handling * use db directly * remove leftover test * remove pool close on error during conn creation	2026-09-29 00:45:55 +02:00
Maycon Santos	979571a99f	[management] Prevent deleting custom domains used by services (#7515) ... Deleting a custom domain released its name while services still pointed at it, leaving them on a namespace the account no longer held. Deletion now refuses with 412 when a service in the same account uses the domain or a subdomain, including disabled ones. Service writes revalidate authorization inside their transaction and hold a shared lock on the matching registrations, so a delete racing a create cannot strand either. The dependency lookup is account-scoped: registrations are unique by name, so another account can hold team.example.com under example.com and its services are authorized by its own registration.	2026-09-28 14:14:32 +02:00
Pascal Fischer	15a13bc99b	[management] split store by table (#7646)	2026-09-24 16:58:37 +02:00
Pascal Fischer	9a5395d314	[management] add store support to filter by public id (#7208)	2026-09-22 15:16:57 +02:00
Maycon Santos	ea294e1d46	[management] Refuse to pin an agent network gateway onto another account's host (#7519) ... An agent network bootstrap stores its cluster as proxy_address, which selects the proxy that serves the endpoint. An account-scoped proxy only receives its own account's mappings, so a pin onto a host another account's proxy declares can never be served, and the endpoint is immutable — a dead gateway until the account deletes its settings. Nothing refused that pin; the domain unique index only arbitrates between endpoints. Both bootstrap paths now refuse, before the insert, a host that another account's proxy declares, a host another account has labeled pins beneath (self-addressed path), or a hostname that is another account's endpoint (labeled path). Shared clusters are unaffected: shared proxies are never foreign, and labeled pins under one cluster are never asked about, so any number of accounts still pin beneath eu.proxy.netbird.io. Registration is deliberately unchanged — refusing a proxy for another account's pin would let a pin lock a tenant out after the reaper drops its rows.	2026-09-14 22:20:25 +02:00
Maycon Santos	ea216f8e73	[management] Speed up test store setup and summarize the unit test run (#7518) ... Management / Unit (amd64, mysql) hit the 20 minute go test budget on #7516. The package was not hung: each of the 133 test store creations in management/server paid about 1.6s on MySQL for CREATE DATABASE, the pre-migrations, a 40-table AutoMigrate and the post-migrations, which puts the package at 10 minutes on a healthy runner and over the budget on a slow one. The migration now runs once per test binary into a template database and each test database is cloned from it, with CREATE DATABASE ... TEMPLATE on Postgres and a replay of SHOW CREATE TABLE on MySQL. The MySQL test container also drops the binary log, doublewrite buffer and per-commit redo fsync. Two goroutine leaks in the test helpers are fixed. tools/gotestsummary turns the go test -json stream into a readable log, and the Management unit and integration jobs now pipe through it, so a timeout names the tests still running. On MySQL, management/server went from 10m16s to 6m36s.	2026-09-14 19:42:02 +02:00
Maycon Santos	b789ffbb9f	[management] Expire unvalidated custom domain registrations (#7497) ... Prevent unvalidated registrations from reserving domain names indefinitely. Give pending registrations a 48-hour validation window and clean up expired entries at startup and every 60 minutes. Emit CustomDomainValidationExpired for each deletion and preserve registrations referenced by services. Reject validation after expiry and prevent concurrent validation from recreating deleted registrations. Normalize domain names with the shared parser before registration. Migrate existing pending registrations to receive a fresh 48-hour validation window.	2026-09-11 17:57:58 +02:00
Pascal Fischer	add8a75981	[management] validate peer existence when adding to group (#7486)	2026-09-11 13:49:20 +02:00
Maycon Santos	21b4a83cea	[management] Refuse services on unvalidated custom domains (#7341) ... Require validated custom domains when creating or updating reverse proxy services. Propagate validation errors during updates and return HTTP 409 for duplicate domain claims. Add regression tests for domain validation, ownership, and service creation and updates.	2026-09-10 11:57:14 +02:00
Maycon Santos	ebc259e30b	[management,client] Gate remote jobs behind an admin opt-in with MDM support (#7153) ... This introduces a disabled-by-default allow-remote-jobs setting that controls whether the management server may run jobs (such as debug bundles) on a peer. The flag propagates end to end: through client configuration, the daemon SetConfig and Login requests, authentication, and system info, up to management, where it is stored on the peer and exposed on the peers API as remote_jobs_allowed. The client refuses any management-requested job unless the peer has opted in. Because enabling remote jobs crosses the user-to-root boundary, turning it on requires privilege, mirroring the SSH-server gate. Administrators can enforce the setting through MDM policy on both macOS and Windows, and MDM can also override the debug-bundle upload URL. The change ships policy documentation and generated profile templates, and adds configuration, conflict, and enforcement tests covering the opt-in, privilege, and MDM paths.	2026-09-01 17:53:41 +02:00
Pascal FischerandDmitri Dolguikh	e06c17cf59	[management] network map from nmap data type (#6919) ... Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> Co-authored-by: Dmitri Dolguikh <dmitri.external@netbird.io>	2026-08-27 11:28:05 +02:00
dmitri-netbird	a144e8c144	[client, management] switch to go.uber.org/mock (#7253) ... * switch to go.uber.org/mock/gomock Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * updated go:generate commands + regenerated mocks Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * update go:generate mockgen commands Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * removed duplicate import Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> * fix go:generate Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> --------- Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>	2026-08-20 11:53:19 +02:00
Maycon Santos	f805c149d9	[management] Record reverse proxy usage for activity accounting (#7116) ... People who only ever reach private services through the reverse proxy were invisible to activity accounting. Active users are counted from user.LastLogin or from the LastSeen of a peer they own, and neither column was written on the proxy paths — so a person signing in via SSO to a proxied service, or a peer serving one over the mesh, never showed up in the 24 hour numbers. Both writes now happen where the proxy already authenticates: - GenerateSessionToken stamps LastLogin after the session token is signed, the same column and the same way the dashboard and device login paths do. - ValidateTunnelPeer stamps the calling peer's LastSeen, the column its owner activates through. The policy lives in a new reverseproxy/activity manager rather than in the gRPC service, matching the module layout the other reverse proxy domains use. It skips what can never count — service users, embedded proxy peers and WASM clients — and throttles peer writes to once an hour, well inside the window accounting asks about and far above the proxy's five minute tunnel cache. The peer write is a single indexed UPDATE that touches only peer_status_last_seen. Connected and SessionStartedAt are left alone so the session-ownership fencing MarkPeerConnectedIfNewerSession relies on is never disturbed, and the timestamp comes from the database clock rather than the caller, for the same reason the other status writers take it from there. The caller's cutoff travels into the statement's WHERE, so concurrent requests for one peer collapse into a single write instead of each acting on its own stale read, and a peer that was never seen — NULL last seen, since Status is an embedded pointer — still records its first activity. Nothing outside the reverse proxy changes behaviour: the only addition elsewhere is the RefreshPeerLastSeen store method the manager calls.	2026-08-11 15:54:39 +02:00
Brad Ison	ebfdf7d7b8	[management] Rework Agent Network endpoint identity and settings bootstrap (#7085) ... Store the per-account gateway endpoint as {domain, proxy_address} with a global unique index on the full hostname; dedicated = (domain == proxy_address). Bootstrap becomes an explicit POST carrying exactly one of proxy_address (server allocates an adjective-noun label beneath it) or endpoint (claimed verbatim, address-first); provider create loses its bootstrap side effect. PUT is a full replace with every field required — the immutable identity fields must be echoed unchanged and a mismatch is rejected with 422. A guarded DELETE releases the endpoint: refused with 412 while providers exist or a proxy is actively serving the endpoint hostname (matched case-insensitively); re-creating bootstraps fresh. A self-addressed pin excludes its address from the account's cluster allow list, and the live mapping update path now addresses the serving proxy from the synthesized service. Existing rows are migrated on all three store engines.	2026-08-10 19:06:55 +02:00
Viktor Liu	9269b56386	[management] Read reverse-proxy service and target columns in Postgres path (#6886)	2026-07-28 14:46:19 +02:00
Maycon Santosandbraginini	4f6247b5c3	[management, proxy] Add prompt-cache token and cost accounting to agent network usage (#6900) ... Co-authored-by: braginini <bangvalo@gmail.com>	2026-07-26 21:42:41 +02:00
Nicolas Frati	b0c1ed31b8	[infrastructure] Add unified admin CLI for self-hosted helpers (#6507) ... ## Summary Adds a unified `admin` CLI for self-hosted instance administrators in both the management and combined binaries. ## User Management ### `admin user change-password` - Changes a local embedded IdP user's password. - Selects the user with `--email` or `--user-id`. - Reads the new password from `--password` or `--password-file`. - Clears the user's local authentication session so the new password is required on the next login. - **Alias:** `admin user set-password`. ### `admin user reset-mfa` - Resets a local embedded IdP user's MFA enrollment. - Selects the user with `--email` or `--user-id`. - Clears TOTP/WebAuthn enrollment data and removes the local authentication session. - The user will re-enroll MFA on the next login. ## MFA Management ### `admin mfa status` - Shows whether local MFA is enabled in the account settings. - Checks the embedded IdP client configuration and reports whether MFA is enabled there. ### `admin mfa enable` - Enables local MFA for embedded IdP users. - Updates embedded IdP clients and saves the account MFA setting. - Records an audit event on a best-effort basis. ### `admin mfa disable` - Disables local MFA for embedded IdP users. - Updates embedded IdP clients and saves the account MFA setting. - Records an audit event on a best-effort basis. ## Reverse Proxy Tokens ### `admin token create --name <name> [--expires-in <duration>]` - Creates a reverse proxy access token. - Prints the plaintext token once, along with the token ID. - `--expires-in` supports values such as `24h`, `30d`, or `365d`. If omitted, the token never expires. ### `admin token list` - Lists reverse proxy access tokens. - Shows the token ID, name, creation date, expiration, last-used time, and revocation status. - **Alias:** `admin token ls`. ### `admin token revoke <token-id>` - Revokes a reverse proxy access token. - Revoked tokens can no longer authenticate reverse proxy instances. ## Reverse Proxy Management ### `admin proxy disconnect-all` - Lists registered reverse proxy instances and force-marks all connected instances as disconnected. - Useful for repairing stale proxy state after an unclean management server shutdown. - Prompts for confirmation by default. - `--dry-run` previews the changes without applying them. - `--force` skips the confirmation prompt. - Live proxies may appear again after their next heartbeat, reconnect, or re-registration. ## Compatibility Commands ### `token ...` - Deprecated top-level compatibility path. - Behaves the same as `admin token ...`. - Retained so existing scripts using `token create`, `token list`, or `token revoke` continue to work. ## Changes - Adds reusable `management/cmd/admin` command package. - Wires `admin` into `netbird-mgmt` and `combined`. - Adds local user password reset with existing password strength validation. - Adds local MFA enrollment reset by clearing Dex TOTP/WebAuthn credentials and local auth sessions. - Adds local MFA enable/disable/status helpers for embedded IdP deployments. - Moves proxy access token commands under `admin token` for a single admin-focused CLI entry point. - Exports `server.ValidatePassword` for reuse by CLI helpers. ## Tests ```bash go test ./management/cmd/... go test ./management/cmd/admin ./management/cmd ./combined/cmd go test ./management/server -run TestValidatePassword ``` Pre-push lint also passed. ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [x] Is a feature enhancement - [ ] It is a refactor - [ ] Created tests that fail without the change (if possible) - [ ] This change does **not** modify the public API, gRPC protocols, functionality behavior, CLI / service flags, or introduce a new feature — **OR** I have discussed it with the NetBird team beforehand (link the issue / Slack thread in the description). See [CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first). > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [x] I added/updated documentation for this change - [ ] Documentation is **not needed** for this change (explain why) ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/832 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added self-hosted admin CLI commands for changing passwords, resetting MFA (including WebAuthn), and managing embedded IdP client MFA (enable/disable/status). * Introduced a unified admin command entry point and improved data-directory handling for embedded IdP storage. * **Refactor** * Centralized password strength validation into a shared exported validator. * **Tests** * Added a comprehensive admin command test suite covering password input, selectors, MFA reset, and client MFA state handling. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-07-22 18:47:19 +02:00
3 people	8435682ac8	[client, management] offload client config generation to the client (#6711) ... Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io> Co-authored-by: crn4 <vladimir@netbird.io> Co-authored-by: pascal <pascal@netbird.io>	2026-07-22 18:20:27 +02:00
dmitri-netbird	877e889250	[management] fix fetching of missing settings in GetAccount call (#6800) ... ## Describe your changes ## Issue ticket number and link ## Stack <!-- branch-stack --> ### Checklist - [x] Is it a bug fix - [ ] Is a typo/documentation fix - [ ] Is a feature enhancement - [ ] It is a refactor - [ ] Created tests that fail without the change (if possible) - [x] This change does **not** modify the public API, gRPC protocols, functionality behavior, CLI / service flags, or introduce a new feature — **OR** I have discussed it with the NetBird team beforehand (link the issue / Slack thread in the description). See [CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first). > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [ ] I added/updated documentation for this change - [x] Documentation is **not needed** for this change (explain why) ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/__ <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/netbirdio/codesmith/netbird/pr/6800"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with Codesmith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1786797566&installation_id=146802194&pr_number=6800&repository=netbirdio%2Fnetbird&return_to=https%3A%2F%2Fgithub.com%2Fnetbirdio%2Fnetbird%2Fpull%2F6800&signature=36a7053e1029e5de1d496fbd9c428a1745d0ec22b3e6479d64aafb1b8350b0ef"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with Codesmith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>/codesmith</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Ensure account settings are fully preserved through save/load, including automatic update and peer exposure preferences. * **Tests** * Added coverage to verify account settings remain unchanged after database persistence and retrieval (skipped on Windows due to SQLite limitations). * Introduced deterministic test-data population helpers to reliably set struct fields for deeper settings verification. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>	2026-07-17 10:38:43 +02:00
Maycon Santos	ecd398d895	[management] Add dashboard_features account setting (#6742) ... Introduce a nullable dashboard_features object on account settings, serialized to a single JSON column so new dashboard sections can be added without schema changes. Starts with agent_network (show the Agent Network menu for an account without the deployment flag). Wires the API handler mapping, the pgx GetAccount loader, and adds store round-trip and handler tests.	2026-07-12 21:16:37 +02:00
Maycon Santos	aa92ad3fb1	[management] Add agent_network_only account setting (#6736) ... * [management] Add agent_network_only account setting * [management] Load agent_network_only in pgx account loader and cover persistence	2026-07-12 14:46:08 +02:00
Zoltan Papp	980598ed4a	[management, client] Add management-controlled client metrics push (#5886) ... * [management, client] Add management-controlled client metrics push Allow enabling/disabling client metrics push from the dashboard via account settings instead of requiring env vars on every client. - Add MetricsConfig proto message to NetbirdConfig - Add MetricsPushEnabled to account Settings (DB-persisted) - Expose metrics_push_enabled in OpenAPI and dashboard API handler - Populate MetricsConfig in sync and login responses - Client dynamically starts/stops push based on management config - NB_METRICS_PUSH_ENABLED env var overrides management when explicitly set - Add activity events for metrics push enable/disable * Remove log line * [management] Fix peer update test for MetricsConfig in NetbirdConfig Update TestUpdateAccountPeers assertions: NetbirdConfig is no longer nil in peer update responses since it now carries MetricsConfig even when STUN/TURN config is absent. * Regenerate proto files with protoc v7.34.1 * [management] Read metrics push setting in Postgres account query getAccountPgx omitted settings_metrics_push_enabled from its hand-written SELECT and Scan, so the toggle was always read back as false on Postgres and never reached clients. * [client] Fix metrics push getting stuck off after engine restart Engine restarts (backoff retries within the same login session) cancel e.ctx, which the push goroutine's lifetime was tied to. The goroutine died silently but ClientMetrics.push stayed non-nil since only an explicit stop clears it, so the next UpdatePushFromMgm call saw a "push already running" state and never restarted it. Give the Engine its own metricsCtx sourced from ConnectClient.ctx, which outlives engine restarts, so handleMetricsUpdate stops tying the push to the wrong-scoped context. Additionally make ClientMetrics.push an atomic.Pointer that the push goroutine clears via CompareAndSwap on exit, so the tracked state can never drift from the goroutine's actual lifetime regardless of which context a future caller passes in. * [management] Regenerate OpenAPI types with oapi-codegen v2.7.1 types.gen.go was regenerated with a stale local v2.6.0 binary, causing the CI git-diff check against generate.sh's pinned v2.7.1 to fail.	2026-07-01 14:50:25 +02:00
3 people	92a66cdd20	[management,proxy,client] 0.74.0 version (#6563) ... * [management,proxy] Agent network: per-account LLM gateway (policy, metering, multi-provider) (#6555) * [agent-network] Shared proto, OpenAPI schema, and generated types * [agent-network] Management: store, manager, synthesizer, policy engine, provider catalog, HTTP/gRPC API Adds the account-scoped agent-network module: provider/policy/budget CRUD and store, the reverse-proxy service synthesizer, policy selection + limit enforcement, the provider catalog (incl. Vertex AI and AWS Bedrock entries), and the management HTTP + proxy gRPC surfaces. * [management] Fix agent-network proxy-peer fan-out on affected-peer recompute The affected-peers resolver loaded only persisted reverse-proxy services, but agent-network services are synthesized on demand and never persisted. As a result the embedded proxy peer was never folded into the affected set when a client's group changed, so the proxy received no network-map update for a newly authorised client and rejected its handshake until a full resync (restart). loadProxyServices now merges the synthesized agent-network services (injected via a registration hook to avoid an import cycle), so proxy peers learn newly authorised clients immediately. * [proxy] Reverse-proxy middleware framework, chain, and request plumbing The per-target middleware chain (slots, dispatcher, mutation gate, metadata merger), body capture, access-log terminal sink, and the proxy wiring that builds + runs chains for synthesized agent-network services. * [proxy] LLM parsers, pricing, and builtin middlewares (OpenAI, Anthropic, Vertex AI, AWS Bedrock) Request/response parsers and SSE/event-stream metering, the embedded pricing table, and the builtin middleware set: request parser, router, policy limit-check/record, cost meter, guardrail, identity inject, response parser. Includes the path-routed providers — Google Vertex AI (keyfile:: service-account OAuth minting) and AWS Bedrock (bearer auth, invoke/converse/streaming, optional /bedrock prefix) — plus the Models allowlist and unmeterable-publisher deny. * [proxy] IPv6 in-place apply and TCP accept-loop hardening on netstack listeners * [agent-network] End-to-end test suite, module docs, and deployment preset * [agent-network] Fix codespell typos and exclude false positives - labelgen word pool: vermillion -> vermilion, racoon -> raccoon. - codespell ignore list: add flate (Go compress/flate package), recordin (a test-local identifier), and unparseable (a valid alternative spelling used consistently across identifiers + a metadata-value constant). * [management] Set LastSeen on injected proxy peer in realstack test (MySQL strict-mode) The injected embedded proxy peer had a PeerStatus with a zero LastSeen, which serializes to '0000-00-00' and is rejected by MySQL in strict mode (SQLite tolerates it). Set LastSeen to a valid time so SaveAccount succeeds on both engines. * [agent-network] Remove e2e shell-script suite from this branch The end-to-end shell scripts under scripts/e2e/ are maintained in a separate testing suite and are not part of this change set. * [agent-network] Polish module docs: remove internal review scaffolding, fix links, verify diagrams Strip PR-review framing, commit references, absolute paths, and stale internal references from the agent-network module docs; fix broken relative links; verify all diagrams against the current architecture. Remove the internal AI-reviewer prompt file. * [management] Refine session expiration handling to support 3-state encoding for SSO deadlines * [agent-network] Relocate agentnetwork package to internals/modules Move management/server/agentnetwork (and its catalog/, labelgen/, types/ subpackages) to management/internals/modules/agentnetwork, alongside the reverse-proxy module, and rewrite all importers. Pure relocation: package names, the synthesizer + affectedpeers registration hook, and store access (shared store.Store) are unchanged, so no import cycle is introduced (affectedpeers still depends only on the agentnetwork/types leaf). * [agent-network] Co-locate HTTP handlers in the module (RegisterEndpoints) Move the agent-network HTTP handlers from server/http/handlers/agentnetwork into the module at internals/modules/agentnetwork/handlers (package handlers) and rename the entrypoint AddEndpoints -> RegisterEndpoints, matching the reverse-proxy module convention. Wiring in http/handler.go updated accordingly. * Update getting started to point to rc when agent network enabled * Add a reference to a commercial license * Fix docs localhost link * Fix docs localhost link * Add private services domain note * [management] Add agent-network telemetry metrics (#6561) Surface agent-network adoption and usage in the self-hosted metrics worker: distinct accounts, providers, policies, budget rules, accounts with log collection enabled, and aggregated input/output tokens plus cost. Tokens and cost are summed from agent_network_request_usage (the always-written per-request ledger) so the figures are accurate regardless of the log-collection toggle and carry no double-counting. All values come from a handful of indexed aggregate queries run only on the worker's periodic tick. Adds store.AgentNetworkMetrics with GetAgentNetworkMetrics on the Store interface, the SqlStore implementation, and a zero-valued FileStore stub. * Update NetBird server and proxy image versions to 0.74.0-rc.2 * [management,proxy] Reduce agent-network cognitive complexity (#6566) Address the SonarCloud quality-gate findings in new agent-network code by extracting focused helpers. No behavior change. - synthesizer.go: split buildIdentityInjectConfigJSON into per-shape rule builders; extract mergeGuardrail from mergeGuardrails to cut nesting depth. - llm_identity_inject: extract injectionEmitsAnything validation predicate from New. - llm_response_parser/streaming.go: extract applyOpenAIStreamUsage and applyAnthropicStreamUsage (via a named anthropicStreamUsage type) and simplify the OpenAI scanner loop. - reverseproxy.go: decompose ServeHTTP into serveRouteError, buildTargetContext, serveDirect, serveWithChain, captureRequestForChain, serveDeny, newResponseWriter, observeResponse, and forwardUpstream, preserving the defer ordering so response observation still reads the captured writer before it is released. * [management] Move agent-network access-log ingest into the agentnetwork module (#6568) The agent-network access-log ingest path (metaKey wire contract, flatten, usage derivation, and the dual-write of the usage ledger + settings-gated full row) lived in the reverseproxy accesslogs manager, even though the agentnetwork module already owns the rest of that domain — types, read (ListAccessLogs / GetUsageOverview), the budget-counter writes, and retention cleanup. Move it next to the rest: a stateless agentnetwork.IngestAccessLog(ctx, store, entry) that the reverseproxy SaveAccessLog delegates to when the entry is agent-network. Removes the agentNetworkTypes import from the reverseproxy manager. No behavior change; the write/read table separation is unchanged. Adds real-store coverage for the disable->enable log-collection toggle (usage ledger always written, full row gated) plus the metadata parse and group-dedup helpers, which previously had no dedicated tests. * Add session view support in the access log * [management,proxy] Container-based agent-network e2e harness (#6577) * [e2e] Add container-based agent-network e2e harness (Pillar 1) Introduce a self-contained, OIDC-free e2e harness that stands up NetBird in containers, so suites no longer depend on the hand-maintained Tilt stack or a real IdP. - harness brings up the combined server (management + signal + relay + STUN + embedded IdP) in a single container built from combined/Dockerfile.multistage, and mints an admin PAT through the unauthenticated /api/setup bootstrap (NB_SETUP_PAT_ENABLED). API access goes through the existing shared/management/client/rest typed client. - the image is built via the docker CLI (BuildKit) so the Dockerfile's cache mounts are honored; testcontainers then runs the tagged image. - everything is behind the `e2e` build tag so normal builds and unit tests never pull in testcontainers. Adds BuildKit cache mounts to combined/Dockerfile.multistage so source changes recompile incrementally rather than from scratch. Pillar 1 proven by TestCombinedBootstrap: server builds, boots, mints a PAT, and the PAT authenticates a real management API call. * [e2e] Add management-side agent-network scenarios (Pillar 2) Port the API-driven agent-network scenarios from the bash suites to Go, sharing one combined server per package run (TestMain) with each test owning its resource cleanup. Drives the /api/agent-network/* endpoints through the shared REST client's NewRequest primitive with the generated api types. Scenarios: - provider lifecycle (create/get/list/delete + 404 after delete) - provider validation (missing api_key, unknown catalog id → 4xx) - settings collection-toggle round-trip with cluster/subdomain immutability - policy window floor (reject <60s enabled limit, accept at 60s) - consumption read endpoint returns an array All deterministic and dependency-free (dummy provider keys; no upstream calls), so they run headless in CI. * [e2e] Add live chat-through-proxy scenario (Pillar 3) Stand up the full agent-network data path in containers and drive a real chat-completion through the gateway: - harness: a shared docker network (combined server reachable by alias), a proxy container built from the published reverse-proxy image (NB_PROXY_PRIVATE, NB_PROXY_ALLOW_INSECURE, NB_RELAY_TRANSPORT=ws to match the combined server's WS-multiplexed relay) with a generated self-signed wildcard cert, and a netbird client container that joins via a setup key. - the combined image, proxy image, and client image default to the published rc.2 releases (overridable via NB_E2E_*_IMAGE; a bare local tag is built from source instead). Geolocation download is disabled so the server starts without external fetches. - one shared domain is used for the management exposed address, the proxy domain, and the agent-network cluster; the proxy token is minted via the server CLI (global) to match the manual install. TestChatCompletionThroughProxy provisions provider+policy+group+setup key, runs proxy+client, drives an OpenAI chat-completion through the tunnel, and asserts a 200 plus the ingested access-log row. Requires OPENAI_TOKEN (skips otherwise). The provider must be created with enabled=true explicitly — the create default is false despite the API doc. * [e2e] Run the live chat scenario across a provider matrix Replace the single-provider chat test with a data-driven matrix that runs the same scenario through every provider whose credentials are present in the environment (keys/URLs sourced from ~/.llm-keys locally, Actions secrets in CI): - OpenAI (chat), Anthropic (messages), Vercel, OpenRouter, Cloudflare (OpenAI-compatible gateways), and Bedrock (path-routed, bearer, via the messages shape) — covering both wire shapes and the gateway routing. - all providers are created enabled with a unique model string so the proxy's connect-time snapshot carries them all and model->provider routing is unambiguous (provider toggles after connect don't reconcile to a connected proxy). - the client supports both wire shapes (/v1/chat/completions and /v1/messages); Cloudflare gets the openai provider segment appended to its gateway URL. Each provider must return 200 through the tunnel and produce an ingested access-log row. Vertex is intentionally excluded from the uniform matrix: it needs a bespoke rawPredict request shape rather than the shared chat/messages path, so it warrants a dedicated scenario. * [ci] Add manual workflow for the agent-network e2e suite The e2e suite (build tag `e2e`) stands up the combined server + proxy + client in Docker and drives live chat-completions, so it is slow and needs provider credentials. Gate it out of normal CI (it already is, via the build tag) and run it on demand via workflow_dispatch. Provider scenarios skip when their secret is unset, so it degrades gracefully. * [e2e] Add Vertex to the provider matrix; run e2e on ubuntu-latest Vertex (Anthropic-on-Vertex) doesn't share the chat/messages wire shapes: the model travels in a rawPredict path and the proxy mints the service account's OAuth token. Add a Vertex client method that posts /v1/projects/<project>/locations/<region>/publishers/anthropic/models/<model>:rawPredict with the Vertex anthropic_version body, and wire it into the matrix as a path-routed provider (created without a models array). It is keyed off GOOGLE_VERTEX_SA_BASE64 + GOOGLE_VERTEX_PROJECT (region defaults to "global", model to a pinned claude snapshot, both overridable). Also bump the e2e workflow runner to ubuntu-latest and add the Vertex secrets. * Add docker/docker and docker/go-connections as direct dependencies in go.mod * [ci] Trigger agent-network e2e workflow on push to main and pull requests * [e2e] Fix proxy cert permission denied on Linux CI runners The proxy bind-mounts a temp dir of self-signed certs. MkdirTemp creates it 0700 and the key was 0600, which Docker Desktop on macOS ignores but a non-root proxy container on Linux runners cannot traverse/read, so the cert watcher failed with "open /certs/tls.crt: permission denied" and the container exited. Widen the cert dir to 0755 and write the throwaway key 0644 so the proxy uid can read the bind-mounted material. * [e2e] Build images from source by default instead of pulling rc.2 The agent-network code under test lives in this branch, so the e2e should exercise it rather than a frozen published release. Flip the harness default: combined/proxy/client are now built from their in-repo Dockerfiles (combined/Dockerfile.multistage, proxy/Dockerfile.multistage, e2e/harness/Dockerfile.client) under local tags. Pulling a published image stays available by setting NB_E2E_*_IMAGE to a registry reference. Builds now go through buildx --load so the Dockerfile cache mounts are honored and the result is loaded for testcontainers. The CI workflow adds a container-driver builder and a local layer cache (NB_E2E_BUILDX_CACHE) persisted via actions/cache, which caches the base/apt/dep-download layers across runs. The Go compile still re-runs each time, as BuildKit mount caches cannot be exported to the GitHub cache. * [e2e] Cover real providers in lifecycle + assert real consumption metering - TestProviderLifecycle now runs per available real provider (create → get → list → delete → 404) instead of a single dummy provider, exercising each catalog's create and field round-trip. Create is offline, so it stays fast and burns no provider quota; falls back to a synthetic OpenAI provider when no keys are set. - TestProvidersMatrix attaches a token limit (high caps, 60s window) to its policy, which switches on usage metering, and asserts consumption rows are recorded with positive token counts after the live traffic. Consumption is account-scoped (keyed by source group / user and window, not per provider), so the assertion is aggregate. - TestProviderValidation gains invalid-upstream and blank-name cases. Create validation is uniform across catalogs (no per-provider required-field rules), so per-provider rejection cases would be redundant. * [e2e] Assert session id propagates per provider Each matrix request now sends a unique session id as the universal x-session-id header and asserts it round-trips into that provider's access-log row. This guards the session-grouping contract end to end for every provider (header extraction runs in llm_request_parser ahead of the parser-specific body extraction, so it is provider-agnostic). * [e2e] Drop accidentally committed sync-phases dashboard netbird-sync-phases.json was swept into the Pillar 1 commit by a broad git add; it belongs to the unrelated sync-phases metrics work, not this e2e harness. Remove it from the branch so the PR diff is scoped to the e2e changes. * [e2e] Revert accidentally committed sync-phase ingest spec The netbird_sync_phase measurement spec in metrics ingest was swept into the Pillar 1 commit; it belongs to the unrelated sync-phases metrics work, not this e2e harness. Its emission side never landed here, so the spec was orphaned anyway. Restore ingest/main.go to its origin/main state. * Fix golint issues * Fix sonar * Add access log session test * Fix access log tests --------- Co-authored-by: braginini <bangvalo@gmail.com> Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>	2026-07-01 12:45:14 +02:00
Pascal Fischer	211a26019a	[management] validate meta change against posture checks (#6510)	2026-06-22 19:42:04 +02:00
Riccardo Manfrin	5095e17cc5	[management] fix flaky Test_SaveAccount_Large from random IP collision (#6452)	2026-06-17 14:00:50 +02:00
Pascal Fischer	967e2d6864	[management] network map for affected peers (#6105)	2026-06-15 17:43:22 +02:00
Maycon Santos	1e7b16db0a	[management] resolve private services on custom domains in synthesized DNS zones (#6348) ... private services on a custom domain didn't resolve on clients — the synthesized DNS zone was anchored to the cluster, and the account's custom domains weren't even loaded. - account.go — SynthesizePrivateServiceZones now keys zones by a resolved apex (privateServiceDomainZone): cluster suffix → registered account.Domains (filtered by matching TargetCluster, longest wins) → skip if none. One zone per apex; custom-domain services group under their registered domain. - sql_store.go — GetAccount now loads account.Domains on both loaders (gorm Preload("Domains") + pgx goroutine via ListCustomDomains; errChan buffer bumped 12→16). This was the reason the deploy didn't work — the relation was empty in prod. - Tests — custom-domain zone synthesis cases (apex resolution, free+custom separation, sibling collapse, cluster mismatch, mixed cluster/custom/public) + GetAccount domain-preload tests on sqlite and Postgres.	2026-06-06 12:56:01 +02:00
Maycon Santos	fa1e241aea	[management, client, proxy] Follow-up fixes for private reverse-proxy services (#6268) ... * fix(proxy): gate tunnel-peer fast-path on inbound listener marker forwardWithTunnelPeer previously accepted any RFC1918 / ULA / CGNAT source IP, so a public client whose address happened to fall in those ranges could bypass the configured operator auth scheme by colliding with a known tunnel IP. The fast-path is now gated on TunnelLookupFromContext(r.Context()) being present — that context value is attached only by the per-account inbound (overlay) listener, so the host-facing listener never enters this branch. Tests updated to reflect the new requirement: requests that don't carry the inbound marker now fall through to the regular auth flow. * fix(proxy): harden inbound listener resource + startup-ctx handling Three correctness fixes on the per-account inbound path, with tests: - Close the logrus ErrorLog PipeWriter on tearDown. WriterLevel hands back an *io.PipeWriter backed by a pipe + scanner goroutine that the caller owns; the two writers per account (https + plain) were never closed, leaking the pipe and goroutine on every teardown. - Run the post-Start hooks on context.Background(). runClientStartup is launched in a goroutine from AddPeer and was inheriting the caller's request-scoped ctx, so a cancelled request could abort the inbound bring-up or fail the management status notification. The tail is split into notifyClientReady so the contract is testable. Tests cover the PipeWriter close behaviour and assert the readyHandler + NotifyStatus calls receive a non-cancelled background context. * feat(proxy): short-circuit peer-own-target loops with 421 When a peer that hosts the target of a private service dials its own service URL the request was being looped through the proxy and back over WireGuard to the same peer — twice the WG round-trip for no benefit, with no signal to the caller that something was wrong. Add isSelfTargetLoop to ReverseProxy.ServeHTTP: when the request arrived on the per-account overlay listener (IsOverlayOrigin) and the source tunnel IP matches the target host, refuse the request with 421 Misdirected Request and a body pointing the operator at the backend directly. The gate is scoped to overlay origin so requests on the public listener that happen to share a source IP with the target host are forwarded normally. * fix(management): private-service validation + tunnel-IP lookup semantics - Require an explicit port for L4 cluster targets. validateL4Target exempted TargetTypeCluster from the port check, but buildPathMappings serializes every L4 target via net.JoinHostPort(host, port) — port=0 shipped a ":0" upstream. Cluster targets use the same Host/Port fields, so the same requirement applies. - GetPeerByIP returns NotFound on a tunnel-IP miss instead of mapping every error to Internal. The proxy's ValidateTunnelPeer probes IPs that legitimately aren't in the roster; the miss is expected and now distinguishable from a real store failure. - Thread ctx into getClusterCapability's gorm query so a cancelled request doesn't keep the store busy. Tests updated for the L4-cluster port requirement and the GetPeerByIP NotFound path. * fix(client): include offlinePeers in PeerStateByIP lookup ReplaceOfflinePeers moves peers into d.offlinePeers but PeerStateByIP only scanned d.peers. Callers (the local DNS filter via localPeerConnectivity, embed.Client.IdentityForIP used by the proxy's tunnel-peer validator) were treating known-but-offline peers as unknown, which: - causes the DNS filter to keep returning records pointing at peers that have no live tunnel, AND - makes the proxy's local-roster check deny a request from such a peer rather than letting the cached management RPC carry the authorisation decision. Search both slices in PeerStateByIP. Adds a unit test for the IPv4 and IPv6 offline-match paths. * fix(rest): reject empty Delete path params in reverse-proxy clients ReverseProxyClustersAPI.Delete and ReverseProxyTokensAPI.Delete passed the path parameter into url.PathEscape without an empty check. PathEscape("") returns "" which collapses the request onto the collection endpoint ("/api/reverse-proxies/clusters/" / "/api/reverse-proxies/proxy-tokens/"), so a caller bug delete with no id reached a routable URL with surprising semantics (typically 405). Short-circuit with a typed error before the request is built. Tests mount a handler on the collection path that fails the test if hit, so the regression is impossible to reintroduce silently. * chore(api,ci,docs,test): private-service schema, proto-check, fixups Non-functional cleanups and contract/CI hardening around the private-service work: API schema (openapi.yml): - Require a non-empty access_groups and mode=http when private=true, on both Service and ServiceRequest, mirroring validatePrivateRequirements. mode stays optional-but-constrained (empty defaults to http server-side), matching runtime. CI (proto-version-check.yml): - Cover renamed .pb.go files (read base via previous_filename). - Match protoc-gen-go-grpc version headers (optional "- " prefix and -gen-go-grpc suffix) so grpc-generated files are in scope. Docs / comments: - Reword Config field docs to say defaults are applied at Server.Start (initDefaults), not New. - Rename the obsolete --private-inbound flag to --private across comments and the proto doc. Pre-existing test fixups surfaced by review: - Repair the integration-tagged validate_session_test.go (SignToken signature growth + new Manager interface methods). - Fix the CI-skip boolean precedence so Windows isn't skipped unconditionally. - Guard the router.HTTPListener type assertion with comma-ok. * fix(proxy): background ctx for already-started AddPeer notification The earlier ctx fix covered the async runClientStartup path but missed the synchronous branch: when a service is added to an already-started client, AddPeer called NotifyStatus with the caller's request-scoped ctx. A cancelled request/stream could drop the connected notification to management. Use context.Background() here too, matching notifyClientReady. Extends TestNetBird_AddPeer_ExistingStartedClient_NotifiesStatus to pass a pre-cancelled caller ctx and assert the notification still ran on a non-cancelled context. * use the cmd context for roundtripper	2026-06-02 13:40:09 +02:00
Maycon Santos	7aebdd69dd	[management, client, proxy] add expose NetBird-only services over tunnel peers (#6226) ... Adds a new "private" service mode for the reverse proxy: services reachable exclusively over the embedded WireGuard tunnel, gated by per-peer group membership instead of operator auth schemes. Wire contract - ProxyMapping.private (field 13): the proxy MUST call ValidateTunnelPeer and fail closed; operator schemes are bypassed. - ProxyCapabilities.private (4) + supports_private_service (5): capability gate. Management never streams private mappings to proxies that don't claim the capability; the broadcast path applies the same filter via filterMappingsForProxy. - ValidateTunnelPeer RPC: resolves an inbound tunnel IP to a peer, checks the peer's groups against service.AccessGroups, and mints a session JWT on success. checkPeerGroupAccess fails closed when a private service has empty AccessGroups. - ValidateSession/ValidateTunnelPeer responses now carry peer_group_ids + peer_group_names so the proxy can authorise policy-aware middlewares without an extra management round-trip. - ProxyInboundListener + SendStatusUpdate.inbound_listener: per-account inbound listener state surfaced to dashboards. - PathTargetOptions.direct_upstream (11): bypass the embedded NetBird client and dial the target via the proxy host's network stack for upstreams reachable without WireGuard. Data model - Service.Private (bool) + Service.AccessGroups ([]string, JSON- serialised). Validate() rejects bearer auth on private services. Copy() deep-copies AccessGroups. pgx getServices loads the columns. - DomainConfig.Private threaded into the proxy auth middleware. Request handler routes private services through forwardWithTunnelPeer and returns 403 on validation failure. - Account-level SynthesizePrivateServiceZones (synthetic DNS) and injectPrivateServicePolicies (synthetic ACL) gate on len(svc.AccessGroups) > 0. Proxy - /netbird proxy --private (embedded mode) flag; Config.Private in proxy/lifecycle.go. - Per-account inbound listener (proxy/inbound.go) binding HTTP/HTTPS on the embedded NetBird client's WireGuard tunnel netstack. - proxy/internal/auth/tunnel_cache: ValidateTunnelPeer response cache with single-flight de-duplication and per-account eviction. - Local peerstore short-circuit: when the inbound IP isn't in the account roster, deny fast without an RPC. - proxy/server.go reports SupportsPrivateService=true and redacts the full ProxyMapping JSON from info logs (auth_token + header-auth hashed values now only at debug level). Identity forwarding - ValidateSessionJWT returns user_id, email, method, groups, group_names. sessionkey.Claims carries Email + Groups + GroupNames so the proxy can stamp identity onto upstream requests without an extra management round-trip on every cookie-bearing request. - CapturedData carries userEmail / userGroups / userGroupNames; the proxy stamps X-NetBird-User and X-NetBird-Groups on r.Out from the authenticated identity (strips client-supplied values first to prevent spoofing). - AccessLog.UserGroups: access-log enrichment captures the user's group memberships at write time so the dashboard can render group context without reverse-resolving stale memberships. OpenAPI/dashboard surface - ReverseProxyService gains private + access_groups; ReverseProxyCluster gains private + supports_private. ReverseProxyTarget target_type enum gains "cluster". ServiceTargetOptions gains direct_upstream. ProxyAccessLog gains user_groups.	2026-05-25 17:41:50 +02:00
Pascal Fischer	454ff66518	[management] scope network router update call (#6222)	2026-05-20 18:24:00 +02:00
Maycon Santos	d250f92c43	feat(reverse-proxy): clusters API surfaces type, online status, and capability flags (#6148) ... The cluster listing now answers three questions in one round-trip instead of forcing the dashboard to cross-reference the domains API: which clusters can this account see, are they currently up, and what do they support. The ProxyCluster wire type drops the boolean self_hosted in favour of a `type` enum (`account` / `shared`) plus explicit `online`, `supports_custom_ports`, `require_subdomain`, and `supports_crowdsec` fields. Store query reworked so offline clusters still appear (no last_seen WHERE), with online and connected_proxies both derived from the existing 2-min active window via portable CASE expressions; the 1-hour heartbeat reaper still removes long-stale rows. Service manager enriches each cluster with the capability flags via the existing per-cluster lookups (CapabilityProvider now also exposes ClusterSupportsCrowdSec). GetActiveClusterAddresses* keep their tight 2-min filter so service routing and domain enumeration aren't pulled into the wider window. The hard cut removes self_hosted from the response — the dashboard is the only consumer and is updated in the matching PR; no transitional field is shipped. Adds a cross-engine regression test asserting offline clusters surface, connected_proxies counts only fresh proxies, and account-scoped BYOP clusters never leak across accounts.	2026-05-20 10:08:34 +02:00
Maycon Santos	80966ab1b0	[management] Ensure SessionStartedAt has a default value (#6211) ... * [management] Ensure SessionStartedAt has a default value Avoid null values for the new column * [management] Add PeerStatus with LastSeen in peer_test * [management] Add migration for PeerStatusSessionStartedAt default value * [management] Add PeerStatus with LastSeen in migration tests	2026-05-20 08:25:30 +02:00
Maycon Santos	13d32d274f	[management] Fence peer status updates with a session token (#6193) ... * [management] Fence peer status updates with a session token The connect/disconnect path used a best-effort LastSeen-after-streamStart comparison to decide whether a status update should land. Under contention — a re-sync arriving while the previous stream's disconnect was still in flight, or two management replicas seeing the same peer at once — the check was a read-then-decide-then-write window: any UPDATE in between caused the wrong row to be written. The Go-side time.Now() that fed the comparison also drifted under lock contention, since it was captured seconds before the write actually committed. Replace it with an integer-nanosecond fencing token stored alongside the status. Every gRPC sync stream uses its open time (UnixNano) as its token. Connects only land when the incoming token is strictly greater than the stored one; disconnects only land when the incoming token equals the stored one (i.e. we're the stream that owns the current session). Both are single optimistic-locked UPDATEs — no read-then-write, no transaction wrapper. LastSeen is now written by the database itself (CURRENT_TIMESTAMP). The caller never supplies it, so the value always reflects the real moment of the UPDATE rather than the moment the caller queued the work — which was already off by minutes under heavy lock contention. Side effects (geo lookup, peer-login-expiration scheduling, network-map fan-out) are explicitly documented as running after the fence UPDATE commits, never inside it. Geo also skips the update when realIP equals the stored ConnectionIP, dropping a redundant SavePeerLocation call on same-IP reconnects. Tests cover the three semantic cases (matched disconnect lands, stale disconnect dropped, stale connect dropped) plus a 16-goroutine race test that asserts the highest token always wins. * [management] Add SessionStartedAt to peer status updates Stored `SessionStartedAt` for fencing token propagation across goroutines and updated database queries/functions to handle the new field. Removed outdated geolocation handling logic and adjusted tests for concurrency safety. * Rename `peer_status_required_approval` to `peer_status_requires_approval` in SQL store fields	2026-05-18 20:25:12 +02:00
Vlad	77b479286e	[management] fix offline statuses for public proxy clusters (#6133)	2026-05-14 13:27:50 +02:00
Vlad	07cbfdbede	[proxy] feature: bring your own proxy (#5627)	2026-05-11 14:31:38 +02:00
Nicolas Fratiandbraginini	e89aad09f5	[management] Enable MFA for local users (#5804) ... * wip: totp for local users * fix providers not getting populated * polished UI and fix post_login_redirect_uri * fix: make sure logout is only prompted from oidc flow Signed-off-by: jnfrati <nicofrati@gmail.com> * update templates Signed-off-by: jnfrati <nicofrati@gmail.com> * deps: update dex dependency Signed-off-by: jnfrati <nicofrati@gmail.com> * fix qube issues Signed-off-by: jnfrati <nicofrati@gmail.com> * replace window with globalThis on home html Signed-off-by: jnfrati <nicofrati@gmail.com> * fixed coderabbit comments Signed-off-by: jnfrati <nicofrati@gmail.com> * debug * remove unused config and rename totp issuer * deps: update dex reference to latest * add dashboard post logout redirect uri to embedded config * implemented api for mfa configuration * update docs and config parsing * catch error on idp manager init mfa * fix tests * Add remember me for MFA * Add cookie encryption and session share between tabs * fixed logout showing non actionable error and session cookie encription key * fixed missing mfa settings on sql query for account * fix code index for mfa activity --------- Signed-off-by: jnfrati <nicofrati@gmail.com> Co-authored-by: braginini <bangvalo@gmail.com>	2026-05-08 16:31:20 +02:00
Viktor Liu	205ebcfda2	[management, client] Add IPv6 overlay support (#5631)	2026-05-07 11:33:37 +02:00
Pascal Fischer	97db824929	[management] fix proxy reconnect (#6063)	2026-05-04 20:43:25 +02:00
Vlad	154b81645a	[management] removed legacy network map code (#5565)	2026-04-27 16:02:54 +02:00
Pascal Fischer	fa0d58d093	[management] exclude peers for expiration job that have already been marked expired (#5970)	2026-04-23 16:01:54 +02:00
Pascal Fischer	c5623307cc	[management] add context cancel monitoring (#5879)	2026-04-14 12:49:18 +02:00
Viktor Liu	0a30b9b275	[management, proxy] Add CrowdSec IP reputation integration for reverse proxy (#5722)	2026-04-14 12:14:58 +02:00