The nonce secret came from the server's WireGuard key, which is generated afresh
in every process and never persisted. A nonce carries no state, so the only
thing that lets one instance verify what another issued is deriving the same
secret — and that premise, written in the comment above the challenger, was not
met: every instance had its own key.
A peer reconnecting after a restart therefore presented a nonce minted under the
previous secret, verification failed with a mismatch, its whole proof set was
rejected and the certificates stored for it were dropped until it signed again.
Reproduced three times on the lab, each one logging "nonce was not issued to
this peer", which only a changed secret produces. On a single instance it costs
seconds of lost policy access per restart; across instances it is not transient
at all, because every reconnect that lands elsewhere is rejected the same way.
Derive from the data store encryption key instead: it is generated once, written
back to the configuration and read by every instance, so it survives restarts
and is shared. Where none is configured the secret falls back to the WireGuard
key with a warning — degraded but still unpredictable, which is the property
that matters most: a peer able to guess it could mint the nonces of future
windows, sign them while its key is present and keep passing after it is gone.
The challenger is now built once and passed to the two places that need it,
rather than re-derived per message.
The refresh pushed an update to every connected peer of the account, while only
the peers a certificate check applies to carry a nonce. On an account where a
handful of peers sit behind the check and the rest do not, everyone was woken
several times a day to be handed a map that changed nothing for them.
Push to the sources of the enabled policies whose posture checks include a
certificate check, which is exactly the set that is sent a challenge.
Resolving the set the other way round than the gRPC layer does is the risk here:
a peer the refresh forgets stops being renewed and falls out of its policies
silently, which is the failure this whole mechanism exists to prevent. So the
selection is held against processPeerPostureChecks, the per-peer rule that
decides who receives a challenge in the first place, by a test that asks both
the same question and requires the same answer.
Renewal was timed against the window in two different ways: the period derived
from it, the sweep interval did not. Shortening the window to watch a renewal in
an end-to-end run would have left the refresher still looking for due accounts
every quarter of an hour, so nothing would have been renewed in time and the
test would have reported the feature broken.
Derive the sweep from the period, within bounds that keep a very short window
from spinning and a normal one from checking less often than is useful, and
allow the window itself to be set through the environment so a run can take
seconds instead of half a day. A value that cannot be parsed or falls outside
the bounds keeps the default, because a window nobody intended is a security
property nobody chose, and an override is logged at warning level since it sets
how long a device keeps passing the check after its key is gone.
Every instance has to be given the same value: the window is part of the nonce,
so instances that disagree reject each other's.
A certificate challenge nonce is accepted for its own window and the one before
it, and it only reaches a peer attached to a network map. An account where
nothing changes sends no map, so after a day the peer re-sends the nonce it
still holds, verification rejects its whole proof set, and the certificates
stored for it are dropped. It fails the certificate check and loses every policy
gated on it until some unrelated change happens to push a map. The outage
repairs itself in seconds, which is what makes it expensive: it is intermittent,
it only hits stable networks, and it is not reproducible on demand.
Push the account's peers an update often enough that the nonce they hold is
never close to expiring. Only accounts whose posture checks actually ask for a
certificate are tracked, so a deployment without the feature does no extra work.
The refresh runs from one goroutine over a map of accounts rather than a timer
per account: the period is hours, so one pass every few minutes costs nothing
next to it, and there is no timer to re-arm when an account that falls due
sooner appears. Each account's first run is offset by a hash of its ID, because
the challenge window is global and an instance restart would otherwise arm every
account in the same moment.
The push carries no administrative change, so it is counted as a refresh rather
than an update and stays out of the figures that track what was edited.
* Name the account owner in the pending approval error
A user refused because their account is pending approval had no way to
learn who could approve them. The refusal now carries the account
owner's address, masked, so a caller can name someone to contact without
being handed the address itself.
Resolving the owner is best effort: a lookup failure, or an account
predating the stored email, falls back to the refusal as it was.
* Name only the caller's own owner in the pending approval error
The refusal is raised before ValidateAccountAccess has established that
the caller belongs to the account the request asked about, and the user
is loaded by ID alone. Resolving the owner of the requested account
therefore disclosed that owner's address to a pending user with no claim
to it, reachable through any handler that takes an account ID from the
caller — DELETE /api/accounts/{accountId} passes one straight through.
The owner who can approve a pending user is the owner of their own
account, so resolve that one. The requested account is never read.
* Mask short local parts whole in MaskedEmail
Keeping the first two characters and the last hides nothing until the
local part is four long: at three or fewer they are the whole of it, so
"abc@example.com" masked to "ab****c@example.com" and a pending user
could recover the owner's address in full from what is meant to conceal
it. Short local parts are now replaced entirely.
* Name the owner from GetCurrentUserInfo instead of the permission gate
The gate could only read the stored user row, which carries no address
when an external IdP owns the identities — the usual case — so it named
no one in practice. It also had no way to reach the IdP without being
handed the account manager, which meant restoring bootstrap wiring that
a refactor had dropped.
GetCurrentUserInfo already holds that account manager, so it answers for
a pending user itself and reuses GetOwnerInfo, the same lookup /msp uses
to resolve an owner's address. The gate returns to exactly what it was,
and with it goes the risk of naming the owner of an account the caller
only asked about.
MaskedEmail becomes MaskEmail: with a UserInfo in hand there is no stored
row to hang it off.
* [management] Cover the pending approval refusal in GetCurrentUserInfo
The branch that names the owner had no coverage at the manager level, so
neither the named refusal nor the fallback for an owner without a resolvable
address was pinned down.
* [management] Cover the failed owner lookup in the pending approval refusal
The generic fallback has two ways in: no address on the resolved owner, and no
owner to resolve at all. Only the first was pinned down.
* [management] Pin the owner lookup to the caller's own account
A mismatched account claim must not steer which owner the refusal names, and
a blocked user is still answered before the claim is validated. Both are load
bearing and neither was covered.
The agent network gateway service is private: agents reach it over the WireGuard tunnel, authorised by peer identity, with the cluster as its only target. Only a reverse proxy cluster with private capabilities can serve that, reported per cluster as the `private` capability — the same `supports_private` flag the dashboard gates NetBird-only services on.
A bootstrap could pin an account to a cluster without private capabilities, leaving an immutable dead gateway. Both bootstrap paths now validate the picked cluster: one the account can see must have a connected proxy reporting the capability. Shared and account-owned clusters qualify alike. Known-ness comes from proxy rows, not heartbeat freshness, so a cluster without the capability stays refused while merely offline. A hostname no proxy has declared stays pinnable (address-first). Identity is compared case-insensitively over the account's cluster list.
An agent network bootstrap stores its cluster as proxy_address, which selects the proxy that serves the endpoint. An account-scoped proxy only receives its own account's mappings, so a pin onto a host another account's proxy declares can never be served, and the endpoint is immutable — a dead gateway until the account deletes its settings. Nothing refused that pin; the domain unique index only arbitrates between endpoints.
Both bootstrap paths now refuse, before the insert, a host that another account's proxy declares, a host another account has labeled pins beneath (self-addressed path), or a hostname that is another account's endpoint (labeled path).
Shared clusters are unaffected: shared proxies are never foreign, and labeled pins under one cluster are never asked about, so any number of accounts still pin beneath eu.proxy.netbird.io. Registration is deliberately unchanged — refusing a proxy for another account's pin would let a pin lock a tenant out after the reaper drops its rows.
Management / Unit (amd64, mysql) hit the 20 minute go test budget on #7516. The package was not hung: each of the 133 test store creations in management/server paid about 1.6s on MySQL for CREATE DATABASE, the pre-migrations, a 40-table AutoMigrate and the post-migrations, which puts the package at 10 minutes on a healthy runner and over the budget on a slow one.
The migration now runs once per test binary into a template database and each test database is cloned from it, with CREATE DATABASE ... TEMPLATE on Postgres and a replay of SHOW CREATE TABLE on MySQL. The MySQL test container also drops the binary log, doublewrite buffer and per-commit redo fsync. Two goroutine leaks in the test helpers are fixed.
tools/gotestsummary turns the go test -json stream into a readable log, and the Management unit and integration jobs now pipe through it, so a timeout names the tests still running. On MySQL, management/server went from 10m16s to 6m36s.
`validateDeleteGroup` already refuses to delete a group that is still used by routes, policies, nameservers, setup keys, users, network routers, reverse proxy services, and agent network policies. Account-level agent network budget rules also store group IDs in `TargetGroups`, but that check was missing.
Deleting such a group left a dangling ID on the budget rule. `budgetRuleApplies` then never matched callers by group, so the spend cap silently stopped applying.
This adds `isGroupLinkedToAgentNetworkBudgetRule` and uses it in `validateDeleteGroup`, matching the existing helpers.
Prevent unvalidated registrations from reserving domain names indefinitely.
Give pending registrations a 48-hour validation window and clean up expired entries at startup and every 60 minutes. Emit CustomDomainValidationExpired for each deletion and preserve registrations referenced by services.
Reject validation after expiry and prevent concurrent validation from recreating deleted registrations. Normalize domain names with the shared parser before registration.
Migrate existing pending registrations to receive a fresh 48-hour validation window.
Require validated custom domains when creating or updating reverse proxy services.
Propagate validation errors during updates and return HTTP 409 for duplicate domain claims.
Add regression tests for domain validation, ownership, and service creation and updates.
The management binary is embedded by downstream builds that override
server construction via SetNewServer, but the cobra command tree itself
was closed: rootCmd is unexported and fully assembled in init, with no
way to attach additional subcommands. Customize hands the built root
command to a caller-supplied function before Execute, so an embedding
binary can add its own commands next to — or under — the built-in ones,
such as extra administrative helpers beneath the existing admin group.
* Gather fresh system info on every management sync stream connect
The engine collected the peer meta once at start and reused the same
Info for every Sync stream reconnect, so a mobile network switch that
redials management kept reporting the old local network addresses.
The peer network range posture check was then evaluated against stale
data until the client restarted.
Sync now takes a gatherer that runs at each stream connect. The
gatherer is cheap: GetInfo plus the cached posture check file results,
kept in the new system.InfoSource, which the engine refreshes whenever
the checks list changes. No process enumeration runs on the reconnect
path.
Also fix the management mock server calling itself instead of SyncFunc.
* Evaluate the login response posture checks before the first sync connect
The engine starts with the checks the login response carried, and the
first sync stream request used to send their evaluated file results.
After moving the gather into InfoSource, the stream opened with an empty
cache and the first sync response did not refill it, because its checks
equal the ones the engine already holds. Desktop peers therefore never
reported process or file posture results.
Seed the cache once before the first connect, where the old gather ran,
so a timed out evaluation still falls through to the address-only info.
* Harden the sync info source against nil callbacks and shared slices
A nil getInfo opens the stream without metadata, as a nil sysInfo did
before. The cached posture results are a copy, so the Info returned by
Refresh cannot alias the snapshot later Current calls report. The
exclusion test asserts the remaining address count so it cannot pass
vacuously on a single-address host.
* Retry a posture check refresh that timed out or failed to sync
The checks list was recorded before the gather ran, so once the gather
timed out or SyncMeta failed, the next sync response carrying the same
list matched the recorded one and nothing retried. The peer kept
reporting the previous posture results until the list changed again.
Record the checks only after the meta reached management, so a failed
cycle is repeated on the next sync response.
* Log the skipped posture refresh, let the mock Sync return errors and deflake the reconnect test
* Drop the nil guard around the sync info callback
* Send the refreshed info on the first sync connect instead of gathering it twice
The SQLite network-map query expanded a router's groups with from network_routers, json_each(peer_groups). That comma is an inner join, so a router row survives only when json_each returns at least one row. A router targeting an individual peer carries no groups — the write path stores NULL for a nil slice and '[]' for an empty one — and json_each yields nothing for either, so the join erased the router before it could be keyed by its peer. Postgres reads the same rows through a correlated subquery and was never affected.
The fix expands the groups with a left join, so the router survives with a NULL group_peers.peer_id and the existing scan loop keys it by router.Peer. Group routers still fan out one row per member.
A bad upstream or key saved cleanly and surfaced minutes later as a failed
request or an empty model picker, with nothing pointing back at the record.
CreateProvider now spends the credential once against the vendor's model
listing. UpdateProvider does the same when the upstream, the key, the catalog
provider or the skip-TLS flag changed — only then, so renames and price edits
neither wait on a vendor nor fail because one is down. Both run before the store
write, so a rejected rotation leaves the working key where it was.
What cannot be checked still saves: no listing endpoint, no derivable Bedrock
control-plane host, a private upstream, a record skipping TLS verification.
Everything else blocks, outages included — 5xx, 429 and timeouts leave the
record unverified just as a refusal does. Refusals return 422 and carry no
status code or echoed URL.
Discovery now reads as a partial edit, so a retyped URL can be listed against
without also rotating the credential. Entries with their own listing host
(Bedrock) get their configured upstream resolved separately, since a successful
listing said nothing about it.
After #7221, guardrail allowlists built from a path-style provider's declared
model ids (Bedrock, Vertex) stopped working: the raw region/version form was
compared against the parser's canonical id, so the agent config advertised an
empty model list and requests for the allowlisted model were refused.
Make every allowlist compare provider-aware, keyed on the destination
provider's catalog id: the agent config, the policy gate, and the synthesized
proxy allowlists match an entry on both its verbatim and canonical form —
Bedrock's strip only under bedrock_api, Vertex's only under vertex_ai_api,
verbatim everywhere else, so a plain provider's suffixed entries never widen.
The router's claim compare learns the Vertex @version strip.
New e2e, realstore, and unit tests reproduce both regressions and pin the fix.
This introduces a disabled-by-default allow-remote-jobs setting that
controls whether the management server may run jobs (such as debug
bundles) on a peer. The flag propagates end to end: through client
configuration, the daemon SetConfig and Login requests, authentication,
and system info, up to management, where it is stored on the peer and
exposed on the peers API as remote_jobs_allowed. The client refuses any
management-requested job unless the peer has opted in. Because enabling
remote jobs crosses the user-to-root boundary, turning it on requires
privilege, mirroring the SSH-server gate. Administrators can enforce the
setting through MDM policy on both macOS and Windows, and MDM can also
override the debug-bundle upload URL. The change ships policy
documentation and generated profile templates, and adds configuration,
conflict, and enforcement tests covering the opt-in, privilege, and MDM
paths.
Delegating Agent Network today means handing out full account admin, and
regular users cannot see their own usage or how to connect a local tool.
Add two roles on top of the existing agent_network permission
submodules. agent_network_admin owns the whole area (providers,
policies, guardrails, budgets, usage, logs, settings) with read-only
users, groups, peers, and account info needed to build policies, and
nothing else in the account. usage_viewer is the regular User baseline
plus read on the aggregated usage and cost overview: no provider
configuration, no policies, no request-level logs, which can contain
captured prompts. billing_admin gets a proper permission-map entry with
the User baseline so role resolution stops failing with role-not-found;
its plan and invoice permissions stay enforced cloud-side.
Add the self-service endpoints behind the "My Agent Network" view,
available to every authenticated user because both answers are scoped
strictly to the caller. GET /api/agent-network/me/setup returns the
account endpoint plus the providers and models the caller's own groups
authorize, computed with the same rules the proxy enforces: policy
filtering as in policy selection, model allowlist union intersected
with declared models, orphan and disabled providers omitted. Not set up
and no access are deliberately indistinguishable, and the response
carries display metadata only. GET /api/agent-network/me/consumption
returns the caller's own user-dimension counters.
This extends the management-requested remote debug-bundle job with two
new, optional parameters. anonymize_level selects how aggressively the
bundle is scrubbed: "default" keeps internal (private) IP ranges
readable, while "strict" also anonymizes private, CGNAT and link-local
addresses; the value is trimmed and lowercased, and an unknown level is
rejected at creation. upload_url lets an operator point the peer at a
specific upload service instead of the default one; it must be a
well-formed https URL with a host, and an empty value falls back to the
default upload server. Both fields flow through the job workload API and
are surfaced in the create-debug-job modal on the dashboard. Validation
is shared so the client executor and the management boundary agree on
what a valid upload URL is, preventing drift between the two checks.
* Bump go version to 1.26 and go-quic to v0.62.0
* Replace deprecated ecdsa public key assembly and add tests for jwt
* Update goversioninfo
* Pin go toolchain to 1.26.7
riccardom