The normaliser's predicate never matched under MySQL's default
case-insensitive collation, and the comment called that the right answer
because nothing on MySQL was invisible. Only the SQL lookups are tolerant
there: the domain lookup is followed by an exact Go compare in
SynthesizeServiceForDomain, and the proxy's host map is keyed by the domain
verbatim, so a mixed-case row still missed. Spell the predicate byte-wise on
MySQL so the fold applies.
Two rows whose identities differ only by case would fold onto one endpoint,
which the unique index refuses with a driver message that names no row.
Both the reshape and the normaliser now stop first and name the hostname
that needs a human, in the same voice as the reshape's existing loud
failure.
The comments cited SNI folding as the reason the columns must be lowercase;
the SNI router folds both sides and would have matched. The readers that
miss are the exact ones, and the comments now name those.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
SaveProxy upserts on the proxy ID, so on a reconnect the row Connect just
wrote is the claim the account has held since its first connect, and the
session guard on DeleteProxy matches because the upsert wrote the new
session. Withdrawing that row whenever the post-write re-read errored
surrendered an established claim on a transient store error — a window in
which any other account could take the address — where the pre-existing
code left the row untouched.
An inconclusive re-read still refuses the connect, but marks the session
disconnected instead of deleting the row; only a conclusive answer that the
address is claimed withdraws it. The write-then-re-read argument moves to
the doc of the exported ErrClusterAddressUnavailable, where the API needs
it, and both helpers point there instead of carrying it twice.
Store-backed tests drive the re-read through the real queries — a
reconnect keeps its row, the account's own pin is not a competing claim,
another account's is — since the whole path now depends on the store
excluding the account's own claims.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
A cluster address is claimed two ways: an account-scoped proxy row, and an
agent network gateway pin on the address. Each side checked the other
before writing — IsClusterAddressAvailable before SaveProxy,
HasForeignAccountProxyAtHost before the settings insert — but check and
write are separate autocommit statements, so two concurrent claimants could
each pass their check and both commit, leaving a pin no proxy will ever
serve next to the proxy row that displaces it.
Both sides now re-read after they write. Manager.Connect re-asks
availability once the proxy row is committed and, if the address is no
longer free or the answer is inconclusive, deletes its own row and returns
ErrClusterAddressUnavailable, which the connect path reports as
AlreadyExists exactly as the pre-write check would have. bootstrapLabeled
re-asks ownership once the settings row is committed and withdraws the pin
on the same terms. Because both write before they re-read, of two
concurrent claimants at least one re-reads after the other has committed
and backs off — on sqlite, postgres and mysql alike, since each statement
sees every commit before it. Both may back off, which costs a retry;
neither keeps a claim the other holds.
No lock spans the proxies and settings tables portably, and a claims table
would be more machinery than the property needs, so the re-read is the
whole mechanism. DeleteProxy is session-guarded like DisconnectProxy, so a
stale session withdrawing itself cannot take out a newer session's row.
Reported by CodeRabbit on #7402 (CWE-362).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
The legacy bootstrap stored the cluster as the caller spelled it — trimmed,
never folded — and the reshape that shipped in 0.78 copied it into
proxy_address, and subdomain.cluster into domain, verbatim. Everything that
reads those columns compares against canonical lowercase: proxies
canonicalise their address at connect, the gateway-pin check a proxy
registration runs matches proxy_address exactly, cluster-scoped synthesis
finds an account's row by proxy_address, and the proxy folds the SNI host
before matching a mapping's domain. A row that kept capitals is invisible
to all of them — its pin protects nothing and its endpoint never routes.
The reshape now writes LOWER() for both columns, and an idempotent
normaliser lowercases rows a released reshape already copied, registered
right after it. The predicate selects only rows that would change, so a
normalised table costs one pass over one row per account; on MySQL the
default collation compares case-insensitively already and it is a no-op.
Reported by cubic on #7402: the exact proxy_address match a proxy
registration relies on misses a migrated pin with capitals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
Belongs with 1b6a9d8 — the file was new and untracked, so the commit that
added the query it exercises went out without it.
Drives HasGatewayPinnedByOtherAccount through sqlite: another account is
refused the host, the pinning account may still claim it (pin first,
deploy the proxy after), an unpinned host stays free, and a labeled pin
claims the cluster address rather than the labeled endpoint beneath it.
The bootstrap check refused pinning onto a cluster another account runs,
but nothing held that decision afterwards: a proxy registration consulted
the proxies table alone, so the same host could be claimed by another
account a moment later, or a week later, and the pin it stranded was
immutable. Validating only at bootstrap meant the check was true when it
ran and not after.
Make the claim symmetric. IsClusterAddressAvailable now treats a gateway
pin as what it is — a claim on the host, served by whichever proxy
declares that address — and refuses a proxy from a different account,
since an account-scoped proxy never receives another account's mappings
and so cannot serve the pin it would displace. Both claims are checked in
one place so a caller cannot consult one and forget the other. An account
claiming the address its own gateway is pinned to is the documented
order, not a conflict: pin first, deploy the proxy after.
Shared (NetBird-operated) proxies register without an account and are
unaffected; an account-scoped proxy reaching for a shared cluster address
that accounts are pinned to was already refused by the proxy-row conflict
and now stays refused even if those rows are momentarily gone.
Whichever of the two lands first now wins and the second is refused,
which is what the bootstrap check could not do on its own. A genuinely
concurrent pair can still pass both checks — closing that needs an
invariant spanning the proxies and settings tables, not a wider read.
The ownership check only ran when the account's own view of the cluster
came back empty, so an account holding any row for the host — its own
canonical proxy — skipped it entirely. A row another account left behind
under a non-canonical spelling was then never seen, and the pin the check
exists to refuse went through. That is the case worth catching, not the
one to skip: two accounts claiming one hostname is the ambiguity the
connect-time conflict check prevents going forward and cannot see for a
row written before addresses were canonicalized, and the endpoint pinned
here cannot be moved afterwards.
Ask ownership first, and narrow what counts as foreign while doing it.
The query treated a shared proxy as outside the account, which was
harmless while it ran only for a host the account had no view of — a
shared row would have given it one — but refuses the cluster most
accounts pin to once it runs first. Only a row owned by a different
account is foreign now, which is also what the name says.
Prevent unvalidated registrations from reserving domain names indefinitely.
Give pending registrations a 48-hour validation window and clean up expired entries at startup and every 60 minutes. Emit CustomDomainValidationExpired for each deletion and preserve registrations referenced by services.
Reject validation after expiry and prevent concurrent validation from recreating deleted registrations. Normalize domain names with the shared parser before registration.
Migrate existing pending registrations to receive a fresh 48-hour validation window.
Require validated custom domains when creating or updating reverse proxy services.
Propagate validation errors during updates and return HTTP 409 for duplicate domain claims.
Add regression tests for domain validation, ownership, and service creation and updates.
* Gather fresh system info on every management sync stream connect
The engine collected the peer meta once at start and reused the same
Info for every Sync stream reconnect, so a mobile network switch that
redials management kept reporting the old local network addresses.
The peer network range posture check was then evaluated against stale
data until the client restarted.
Sync now takes a gatherer that runs at each stream connect. The
gatherer is cheap: GetInfo plus the cached posture check file results,
kept in the new system.InfoSource, which the engine refreshes whenever
the checks list changes. No process enumeration runs on the reconnect
path.
Also fix the management mock server calling itself instead of SyncFunc.
* Evaluate the login response posture checks before the first sync connect
The engine starts with the checks the login response carried, and the
first sync stream request used to send their evaluated file results.
After moving the gather into InfoSource, the stream opened with an empty
cache and the first sync response did not refill it, because its checks
equal the ones the engine already holds. Desktop peers therefore never
reported process or file posture results.
Seed the cache once before the first connect, where the old gather ran,
so a timed out evaluation still falls through to the address-only info.
* Harden the sync info source against nil callbacks and shared slices
A nil getInfo opens the stream without metadata, as a nil sysInfo did
before. The cached posture results are a copy, so the Info returned by
Refresh cannot alias the snapshot later Current calls report. The
exclusion test asserts the remaining address count so it cannot pass
vacuously on a single-address host.
* Retry a posture check refresh that timed out or failed to sync
The checks list was recorded before the gather ran, so once the gather
timed out or SyncMeta failed, the next sync response carrying the same
list matched the recorded one and nothing retried. The peer kept
reporting the previous posture results until the list changed again.
Record the checks only after the meta reached management, so a failed
cycle is repeated on the next sync response.
* Log the skipped posture refresh, let the mock Sync return errors and deflake the reconnect test
* Drop the nil guard around the sync info callback
* Send the refreshed info on the first sync connect instead of gathering it twice
Two follow-ups to canonicalizing the proxy address.
netip accepts a zoned literal where net.ParseIP did not, so "fe80::1%eth0"
started passing validation and would have been stored as a cluster key. An
address scoped to one host's interface cannot name a cluster others reach,
so zones are rejected, as before.
Making the ownership query exact also left the bootstrap check without a
fallback for rows written before canonicalization: a foreign cluster
stored as "BYOP.Account2.Example.com" no longer matches the normalized
address, reads as never declared, and the pin it should refuse goes
through. Split the two callers instead of choosing between them.
IsClusterAddressConflicting stays exact for the per-connect path that
needs the index; HasProxyOutsideAccountAtHost folds case for the
bootstrap, which runs once per account and is the only thing standing
between it and pinning its immutable endpoint to somebody else's cluster.
The settings delete guard already made that trade for the same reason.
Restores the foreign-casing case that went with the exact query, and adds
the zone cases to the ingress test.
The proxy-connect path already computes the canonical form of the address
a proxy declares — ValidateDomains returns lowercase punycode — and then
throws it away, storing the string as declared. cluster_address is the key
every capability, ownership and routing lookup matches on, so one host
could sit in that column under two spellings, and the previous commit
compensated with LOWER() in the ownership query, which gives up the
cluster_address index on a query that runs for every account-scoped proxy
connect.
Keep the canonical form instead. Connect is the only writer of the column
(heartbeats touch last_seen and status), and proxy rows are session state
rebuilt on every connect rather than durable config, so the column
converges without a migration and the lookups can stay exact and indexed.
Folding happens before punycode conversion, not after: idna lowercases the
ASCII it produces but does not case-fold the unicode it consumes, so
PRÖXY.example.com and pröxy.example.com would otherwise encode to two
different labels for one host.
The agent network check keeps comparing normalised forms in memory, which
costs nothing there — it is a pass over the account's cluster list, not a
query — and covers rows written before this landed.
Proxies store their cluster address as they declared it, while
proxy_address is normalised lowercase before validation. The capability
and ownership lookups match cluster_address exactly, so feeding them the
normalised form asked about a spelling the store may never have seen: a
private cluster declared with capitals came back unproven and was
refused, and — worse — another account's cluster declared with capitals
came back as "never declared" and let the pin through.
Compare identity on the normalised form but keep the stored spellings,
and read the capability under each of them, any-true, the same way it
aggregates over a cluster's proxies. Ownership gets the same treatment at
the source: hostnames are case-insensitive, so two spellings of one host
are one cluster and must conflict rather than being claimable side by
side, which also closes the same gap in the proxy-registration
availability check that shares the query.
The e2e's wait for a stopped cluster to leave the active list now allows
for the active window rather than 90s: a proxy that dies without closing
its stream is only dropped once its last heartbeat ages past
proxyActiveThreshold, so the old budget could fail the test on the slow
path alone.
A bad upstream or key saved cleanly and surfaced minutes later as a failed
request or an empty model picker, with nothing pointing back at the record.
CreateProvider now spends the credential once against the vendor's model
listing. UpdateProvider does the same when the upstream, the key, the catalog
provider or the skip-TLS flag changed — only then, so renames and price edits
neither wait on a vendor nor fail because one is down. Both run before the store
write, so a rejected rotation leaves the working key where it was.
What cannot be checked still saves: no listing endpoint, no derivable Bedrock
control-plane host, a private upstream, a record skipping TLS verification.
Everything else blocks, outages included — 5xx, 429 and timeouts leave the
record unverified just as a refusal does. Refusals return 422 and carry no
status code or echoed URL.
Discovery now reads as a partial edit, so a retyped URL can be listed against
without also rotating the credential. Entries with their own listing host
(Bedrock) get their configured upstream resolved separately, since a successful
listing said nothing about it.
The synthesised agent network gateway service is unconditionally private:
agents reach it over the WireGuard tunnel and are authorised by
ValidateTunnelPeer against the enabled policies' source groups, and its
only target is the cluster itself with DirectUpstream. Only a proxy
running embedded in a netbird client can serve that, which management
already reports per cluster as the `private` capability.
CreateSettings accepted any hostname as proxy_address, so a labeled
bootstrap could pin an account to a cluster that cannot serve its
gateway — another account's BYOP cluster, or one whose proxies are all
centralised. The endpoint assigned at bootstrap is immutable, so the
account is then stuck with a dead gateway until someone deletes and
re-bootstraps the settings row.
Validate the cluster before allocating an endpoint beneath it. Whether
management knows a cluster is decided on its proxy rows, never on how
fresh their heartbeats are: the rows outlive their proxies' liveness, so
a known cluster stays judged as one and has to prove with a live embedded
proxy that it can serve the gateway. Deciding on liveness instead would
let the same centralised cluster pass or fail depending on whether its
proxies had heartbeated in the last couple of minutes, turning "wait for
the proxy to go quiet" into a way to pin the endpoint to a cluster that
can never serve it. Ownership comes from the same time-independent
source, so a foreign cluster stays refused while it is offline.
Only a cluster no proxy has ever declared is still pinnable — that is the
address-first order the dedicated (self-addressed) path documents, and
the one self-hosted setups follow when they configure before deploying.
This introduces a disabled-by-default allow-remote-jobs setting that
controls whether the management server may run jobs (such as debug
bundles) on a peer. The flag propagates end to end: through client
configuration, the daemon SetConfig and Login requests, authentication,
and system info, up to management, where it is stored on the peer and
exposed on the peers API as remote_jobs_allowed. The client refuses any
management-requested job unless the peer has opted in. Because enabling
remote jobs crosses the user-to-root boundary, turning it on requires
privilege, mirroring the SSH-server gate. Administrators can enforce the
setting through MDM policy on both macOS and Windows, and MDM can also
override the debug-bundle upload URL. The change ships policy
documentation and generated profile templates, and adds configuration,
conflict, and enforcement tests covering the opt-in, privilege, and MDM
paths.
Delegating Agent Network today means handing out full account admin, and
regular users cannot see their own usage or how to connect a local tool.
Add two roles on top of the existing agent_network permission
submodules. agent_network_admin owns the whole area (providers,
policies, guardrails, budgets, usage, logs, settings) with read-only
users, groups, peers, and account info needed to build policies, and
nothing else in the account. usage_viewer is the regular User baseline
plus read on the aggregated usage and cost overview: no provider
configuration, no policies, no request-level logs, which can contain
captured prompts. billing_admin gets a proper permission-map entry with
the User baseline so role resolution stops failing with role-not-found;
its plan and invoice permissions stay enforced cloud-side.
Add the self-service endpoints behind the "My Agent Network" view,
available to every authenticated user because both answers are scoped
strictly to the caller. GET /api/agent-network/me/setup returns the
account endpoint plus the providers and models the caller's own groups
authorize, computed with the same rules the proxy enforces: policy
filtering as in policy selection, model allowlist union intersected
with declared models, orphan and disabled providers omitted. Not set up
and no access are deliberately indistinguishable, and the response
carries display metadata only. GET /api/agent-network/me/consumption
returns the caller's own user-dimension counters.
This extends the management-requested remote debug-bundle job with two
new, optional parameters. anonymize_level selects how aggressively the
bundle is scrubbed: "default" keeps internal (private) IP ranges
readable, while "strict" also anonymizes private, CGNAT and link-local
addresses; the value is trimmed and lowercased, and an unknown level is
rejected at creation. upload_url lets an operator point the peer at a
specific upload service instead of the default one; it must be a
well-formed https URL with a host, and an empty value falls back to the
default upload server. Both fields flow through the job workload API and
are surfaced in the create-debug-job modal on the dashboard. Validation
is shared so the client executor and the management boundary agree on
what a valid upload URL is, preventing drift between the two checks.
* Support per-peer lazy connection state and default proxy peers to lazy
* Classify forward targets from incoming config in lazy exclusion
* Set IsUserspaceBind mock so lazy manager starts in engine test
* Skip lazy exclude reconciliation when the set is unchanged
* Keep cached lazy flag when a sync carries no peer config
* fix a nil-ptr error occuring in sendInitialSync when the peer being synced is deleted
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
* handle a nil ptr in GetPeerNetworkMapComponents
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
---------
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
People who only ever reach private services through the reverse proxy were
invisible to activity accounting. Active users are counted from user.LastLogin
or from the LastSeen of a peer they own, and neither column was written on the
proxy paths — so a person signing in via SSO to a proxied service, or a peer
serving one over the mesh, never showed up in the 24 hour numbers.
Both writes now happen where the proxy already authenticates:
- GenerateSessionToken stamps LastLogin after the session token is signed,
the same column and the same way the dashboard and device login paths do.
- ValidateTunnelPeer stamps the calling peer's LastSeen, the column its owner
activates through.
The policy lives in a new reverseproxy/activity manager rather than in the gRPC
service, matching the module layout the other reverse proxy domains use. It
skips what can never count — service users, embedded proxy peers and WASM
clients — and throttles peer writes to once an hour, well inside the window
accounting asks about and far above the proxy's five minute tunnel cache.
The peer write is a single indexed UPDATE that touches only
peer_status_last_seen. Connected and SessionStartedAt are left alone so the
session-ownership fencing MarkPeerConnectedIfNewerSession relies on is never
disturbed, and the timestamp comes from the database clock rather than the
caller, for the same reason the other status writers take it from there. The
caller's cutoff travels into the statement's WHERE, so concurrent requests for
one peer collapse into a single write instead of each acting on its own stale
read, and a peer that was never seen — NULL last seen, since Status is an
embedded pointer — still records its first activity.
Nothing outside the reverse proxy changes behaviour: the only addition
elsewhere is the RefreshPeerLastSeen store method the manager calls.
Store the per-account gateway endpoint as {domain, proxy_address} with a
global unique index on the full hostname; dedicated = (domain ==
proxy_address). Bootstrap becomes an explicit POST carrying exactly one
of proxy_address (server allocates an adjective-noun label beneath it)
or endpoint (claimed verbatim, address-first); provider create loses its
bootstrap side effect. PUT is a full replace with every field required —
the immutable identity fields must be echoed unchanged and a mismatch is
rejected with 422. A guarded DELETE releases the endpoint: refused with
412 while providers exist or a proxy is actively serving the endpoint
hostname (matched case-insensitively); re-creating bootstraps fresh. A
self-addressed pin excludes its address from the account's cluster allow
list, and the live mapping update path now addresses the serving proxy
from the synthesized service. Existing rows are migrated on all three
store engines.
A user in the Pending Approval state could complete SSO and reach any
SSO-protected reverse proxy service distributed to a group they belong
to, including the All Users group. The reverse proxy authorization path
checked the session token signature, that the user exists, that the
user's account matches the service's account, and group membership —
never the user's account status. The REST API (`permissions/manager.go`)
and peer registration both gate on that state, but the proxy gRPC
service does not go through the permissions manager, so neither gate
applied. A pending user is persisted as blocked and pending approval, so
blocked users reached those services the same way.
`ValidateSession` now denies on account status, reporting
`pending_approval` or `user_blocked` so the proxy access log and the
denied page carry the cause rather than a generic refusal.
`GenerateSessionToken` refuses to mint a token for such a user at all,
so the browser never receives a session cookie and the OIDC callback can
tell the user why instead of showing "Service configuration error".
`ValidateUserGroupAccess` and `ValidateTunnelPeer` close the same gap;
for the tunnel path this covers a user blocked after their peer was
registered, since peer group membership alone kept mesh-origin access
open.
A single helper produces both the denied reason for the RPC responses
and the sentinel error for the error-returning callers, so the four
entry points cannot drift apart. A user the store cannot resolve is
denied rather than passed through.
One thing deliberately left out: session cookies are validated locally
by the proxy against the service public key with no management
round-trip, so a cookie issued before a user is blocked stays valid
until it expires (24h by default). That is a revocation-propagation
problem rather than this authorization gap, and every option for it
(per-request validation with a cache, short-lived tokens with refresh,
push-based revocation) changes the proxy hot path or the
proxy/management protocol. Worth its own ticket.
## Describe your changes
A group could be deleted while a reverse proxy service still referenced
it, silently breaking the service's access control: private services
list groups in `access_groups` as the peer allowlist, and SSO bearer
auth distributes tokens to `distribution_groups`.
Group deletion now runs through the same linkage validation as routes,
policies, and agent network policies: deleting a group that backs a
private service allowlist or an enabled bearer-auth distribution list
fails with a `GroupLinkError` naming the service domain. Disabled bearer
configs and stale `access_groups` on non-private services are inert and
do not block deletion.
Tests cover both linked cases in single and bulk deletion, and pin the
non-blocking cases. The test account seeds decoy services ahead of the
linked ones so the check is proven to scan the full service list.
Block deleting a group referenced as a source group by an agent network
policy, and drop unresolvable groups from synthesised private-service
ACLs. A deleted group survived in agent_network_policies.source_groups
and was carried into the injected in-memory policy, where network-map
assembly resolved it to a nil group and panicked on every proxy peer
sync.
## Describe your changes
Work on the Terraform provider (terraform-provider-netbird #177–#183)
surfaced places where the agent-network API broke its own contracts or
deviated from the conventions the rest of the management API follows,
forcing client-side workarounds.
Settings reads now follow the settings-endpoint convention: GET always
answers with a JSON object. Before bootstrap it returns the defaults
with an empty cluster/subdomain/endpoint (previously 200 with a JSON
`null` body, while the spec said 404). The settings PUT can bootstrap
the account by carrying a `cluster` — previously the row could only come
into existence through the first provider create, and a settings-first
setup was impossible; a differing cluster on a bootstrapped account is
rejected instead of silently ignored. PUT remains full-state.
The provider PUT schema promised omit-preserves semantics for several
operator-editable fields that the handler never delivered (it builds the
row from the request, like every other update handler). The schema
wording now matches the shipped full-state behavior; only the api_key
(secret) and session keys stay preserved by the manager. Identity
headers are always present in provider responses so an explicitly
cleared value round-trips as an empty string.
The Go REST client gains the full agent-network surface (catalog,
providers, policies, guardrails, budget rules, settings), including a
shim translating the legacy 200+`null` settings body from older servers
into an `IsNotFound` error.
Note for reviewers: the dashboard special-cased the `null` settings
body; it needs a small follow-up for the new defaults response (in
progress).
## Describe your changes
Agent Network gates providers, policies, guardrails, budgets, usage,
access logs, and settings behind the single `agent_network` permission
module, so access is all-or-nothing: a future delegated role cannot be
scoped to a subset of the area (for example usage-only visibility).
This introduces dotted submodules (`agent_network.providers`,
`.policies`, `.guardrails`, `.budgets`, `.usage`, `.logs`, `.settings`)
and resolves grants with a cascade: exact module first, then its parent,
then the role's `AutoAllowNew` default. The agent network manager now
validates each operation against its matching submodule. `usage`
(aggregated counters, overview) is deliberately separate from `logs`
(request-level entries, which can contain captured prompts).
No role definitions change. No built-in role carries an explicit
`agent_network` entry, so every role resolves the submodules exactly as
it resolved the parent module before — pinned by a test that compares
each built-in role's answer on every submodule against its answer on
`agent_network`. Role additions that use these submodules come
separately.
