The normaliser's predicate never matched under MySQL's default
case-insensitive collation, and the comment called that the right answer
because nothing on MySQL was invisible. Only the SQL lookups are tolerant
there: the domain lookup is followed by an exact Go compare in
SynthesizeServiceForDomain, and the proxy's host map is keyed by the domain
verbatim, so a mixed-case row still missed. Spell the predicate byte-wise on
MySQL so the fold applies.
Two rows whose identities differ only by case would fold onto one endpoint,
which the unique index refuses with a driver message that names no row.
Both the reshape and the normaliser now stop first and name the hostname
that needs a human, in the same voice as the reshape's existing loud
failure.
The comments cited SNI folding as the reason the columns must be lowercase;
the SNI router folds both sides and would have matched. The readers that
miss are the exact ones, and the comments now name those.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
SaveProxy upserts on the proxy ID, so on a reconnect the row Connect just
wrote is the claim the account has held since its first connect, and the
session guard on DeleteProxy matches because the upsert wrote the new
session. Withdrawing that row whenever the post-write re-read errored
surrendered an established claim on a transient store error — a window in
which any other account could take the address — where the pre-existing
code left the row untouched.
An inconclusive re-read still refuses the connect, but marks the session
disconnected instead of deleting the row; only a conclusive answer that the
address is claimed withdraws it. The write-then-re-read argument moves to
the doc of the exported ErrClusterAddressUnavailable, where the API needs
it, and both helpers point there instead of carrying it twice.
Store-backed tests drive the re-read through the real queries — a
reconnect keeps its row, the account's own pin is not a competing claim,
another account's is — since the whole path now depends on the store
excluding the account's own claims.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
A self-addressed bootstrap stores the hostname as proxy_address, which is
exactly what a proxy registration is refused on when another account holds
it there. It asked nobody whether it could: the domain unique index
arbitrated between pins, and a foreign proxy already declaring the host was
never consulted, before or after the insert. Any account could therefore pin
an endpoint onto a host another account's proxy serves — owning nothing —
and lock that proxy out on its next reconnect, and a proxy racing such a pin
could end with both claims standing, since only the labeled path re-read
ownership after its write.
The self-addressed path now asks HasForeignAccountProxyAtHost before the
insert and confirmGatewayClusterOwnership after it, the same as the labeled
one. Address-first stays intact: only a row owned by a different account
refuses, so pinning ahead of any proxy, or onto the account's own, is
unchanged.
Also pins the bootstrap side's failure paths — an ownership re-read that
cannot answer leaves no pin behind and surfaces the store's error, and a
withdrawal that fails still reports the claim as lost — and shortens the
helper's comment to point at the shared argument on
proxy.ErrClusterAddressUnavailable rather than restate it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
A cluster address is claimed two ways: an account-scoped proxy row, and an
agent network gateway pin on the address. Each side checked the other
before writing — IsClusterAddressAvailable before SaveProxy,
HasForeignAccountProxyAtHost before the settings insert — but check and
write are separate autocommit statements, so two concurrent claimants could
each pass their check and both commit, leaving a pin no proxy will ever
serve next to the proxy row that displaces it.
Both sides now re-read after they write. Manager.Connect re-asks
availability once the proxy row is committed and, if the address is no
longer free or the answer is inconclusive, deletes its own row and returns
ErrClusterAddressUnavailable, which the connect path reports as
AlreadyExists exactly as the pre-write check would have. bootstrapLabeled
re-asks ownership once the settings row is committed and withdraws the pin
on the same terms. Because both write before they re-read, of two
concurrent claimants at least one re-reads after the other has committed
and backs off — on sqlite, postgres and mysql alike, since each statement
sees every commit before it. Both may back off, which costs a retry;
neither keeps a claim the other holds.
No lock spans the proxies and settings tables portably, and a claims table
would be more machinery than the property needs, so the re-read is the
whole mechanism. DeleteProxy is session-guarded like DisconnectProxy, so a
stale session withdrawing itself cannot take out a newer session's row.
Reported by CodeRabbit on #7402 (CWE-362).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
The legacy bootstrap stored the cluster as the caller spelled it — trimmed,
never folded — and the reshape that shipped in 0.78 copied it into
proxy_address, and subdomain.cluster into domain, verbatim. Everything that
reads those columns compares against canonical lowercase: proxies
canonicalise their address at connect, the gateway-pin check a proxy
registration runs matches proxy_address exactly, cluster-scoped synthesis
finds an account's row by proxy_address, and the proxy folds the SNI host
before matching a mapping's domain. A row that kept capitals is invisible
to all of them — its pin protects nothing and its endpoint never routes.
The reshape now writes LOWER() for both columns, and an idempotent
normaliser lowercases rows a released reshape already copied, registered
right after it. The predicate selects only rows that would change, so a
normalised table costs one pass over one row per account; on MySQL the
default collation compares case-insensitively already and it is a no-op.
Reported by cubic on #7402: the exact proxy_address match a proxy
registration relies on misses a migrated pin with capitals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
Belongs with 1b6a9d8 — the file was new and untracked, so the commit that
added the query it exercises went out without it.
Drives HasGatewayPinnedByOtherAccount through sqlite: another account is
refused the host, the pinning account may still claim it (pin first,
deploy the proxy after), an unpinned host stays free, and a labeled pin
claims the cluster address rather than the labeled endpoint beneath it.
The bootstrap check refused pinning onto a cluster another account runs,
but nothing held that decision afterwards: a proxy registration consulted
the proxies table alone, so the same host could be claimed by another
account a moment later, or a week later, and the pin it stranded was
immutable. Validating only at bootstrap meant the check was true when it
ran and not after.
Make the claim symmetric. IsClusterAddressAvailable now treats a gateway
pin as what it is — a claim on the host, served by whichever proxy
declares that address — and refuses a proxy from a different account,
since an account-scoped proxy never receives another account's mappings
and so cannot serve the pin it would displace. Both claims are checked in
one place so a caller cannot consult one and forget the other. An account
claiming the address its own gateway is pinned to is the documented
order, not a conflict: pin first, deploy the proxy after.
Shared (NetBird-operated) proxies register without an account and are
unaffected; an account-scoped proxy reaching for a shared cluster address
that accounts are pinned to was already refused by the proxy-row conflict
and now stays refused even if those rows are momentarily gone.
Whichever of the two lands first now wins and the second is refused,
which is what the bootstrap check could not do on its own. A genuinely
concurrent pair can still pass both checks — closing that needs an
invariant spanning the proxies and settings tables, not a wider read.
The ownership check only ran when the account's own view of the cluster
came back empty, so an account holding any row for the host — its own
canonical proxy — skipped it entirely. A row another account left behind
under a non-canonical spelling was then never seen, and the pin the check
exists to refuse went through. That is the case worth catching, not the
one to skip: two accounts claiming one hostname is the ambiguity the
connect-time conflict check prevents going forward and cannot see for a
row written before addresses were canonicalized, and the endpoint pinned
here cannot be moved afterwards.
Ask ownership first, and narrow what counts as foreign while doing it.
The query treated a shared proxy as outside the account, which was
harmless while it ran only for a host the account had no view of — a
shared row would have given it one — but refuses the cluster most
accounts pin to once it runs first. Only a row owned by a different
account is foreign now, which is also what the name says.
Prevent unvalidated registrations from reserving domain names indefinitely.
Give pending registrations a 48-hour validation window and clean up expired entries at startup and every 60 minutes. Emit CustomDomainValidationExpired for each deletion and preserve registrations referenced by services.
Reject validation after expiry and prevent concurrent validation from recreating deleted registrations. Normalize domain names with the shared parser before registration.
Migrate existing pending registrations to receive a fresh 48-hour validation window.
Require validated custom domains when creating or updating reverse proxy services.
Propagate validation errors during updates and return HTTP 409 for duplicate domain claims.
Add regression tests for domain validation, ownership, and service creation and updates.
The management binary is embedded by downstream builds that override
server construction via SetNewServer, but the cobra command tree itself
was closed: rootCmd is unexported and fully assembled in init, with no
way to attach additional subcommands. Customize hands the built root
command to a caller-supplied function before Execute, so an embedding
binary can add its own commands next to — or under — the built-in ones,
such as extra administrative helpers beneath the existing admin group.
* Gather fresh system info on every management sync stream connect
The engine collected the peer meta once at start and reused the same
Info for every Sync stream reconnect, so a mobile network switch that
redials management kept reporting the old local network addresses.
The peer network range posture check was then evaluated against stale
data until the client restarted.
Sync now takes a gatherer that runs at each stream connect. The
gatherer is cheap: GetInfo plus the cached posture check file results,
kept in the new system.InfoSource, which the engine refreshes whenever
the checks list changes. No process enumeration runs on the reconnect
path.
Also fix the management mock server calling itself instead of SyncFunc.
* Evaluate the login response posture checks before the first sync connect
The engine starts with the checks the login response carried, and the
first sync stream request used to send their evaluated file results.
After moving the gather into InfoSource, the stream opened with an empty
cache and the first sync response did not refill it, because its checks
equal the ones the engine already holds. Desktop peers therefore never
reported process or file posture results.
Seed the cache once before the first connect, where the old gather ran,
so a timed out evaluation still falls through to the address-only info.
* Harden the sync info source against nil callbacks and shared slices
A nil getInfo opens the stream without metadata, as a nil sysInfo did
before. The cached posture results are a copy, so the Info returned by
Refresh cannot alias the snapshot later Current calls report. The
exclusion test asserts the remaining address count so it cannot pass
vacuously on a single-address host.
* Retry a posture check refresh that timed out or failed to sync
The checks list was recorded before the gather ran, so once the gather
timed out or SyncMeta failed, the next sync response carrying the same
list matched the recorded one and nothing retried. The peer kept
reporting the previous posture results until the list changed again.
Record the checks only after the meta reached management, so a failed
cycle is repeated on the next sync response.
* Log the skipped posture refresh, let the mock Sync return errors and deflake the reconnect test
* Drop the nil guard around the sync info callback
* Send the refreshed info on the first sync connect instead of gathering it twice
The SQLite network-map query expanded a router's groups with from network_routers, json_each(peer_groups). That comma is an inner join, so a router row survives only when json_each returns at least one row. A router targeting an individual peer carries no groups — the write path stores NULL for a nil slice and '[]' for an empty one — and json_each yields nothing for either, so the join erased the router before it could be keyed by its peer. Postgres reads the same rows through a correlated subquery and was never affected.
The fix expands the groups with a left join, so the router survives with a NULL group_peers.peer_id and the existing scan loop keys it by router.Peer. Group routers still fan out one row per member.
Two follow-ups to canonicalizing the proxy address.
netip accepts a zoned literal where net.ParseIP did not, so "fe80::1%eth0"
started passing validation and would have been stored as a cluster key. An
address scoped to one host's interface cannot name a cluster others reach,
so zones are rejected, as before.
Making the ownership query exact also left the bootstrap check without a
fallback for rows written before canonicalization: a foreign cluster
stored as "BYOP.Account2.Example.com" no longer matches the normalized
address, reads as never declared, and the pin it should refuse goes
through. Split the two callers instead of choosing between them.
IsClusterAddressConflicting stays exact for the per-connect path that
needs the index; HasProxyOutsideAccountAtHost folds case for the
bootstrap, which runs once per account and is the only thing standing
between it and pinning its immutable endpoint to somebody else's cluster.
The settings delete guard already made that trade for the same reason.
Restores the foreign-casing case that went with the exact query, and adds
the zone cases to the ingress test.
The proxy-connect path already computes the canonical form of the address
a proxy declares — ValidateDomains returns lowercase punycode — and then
throws it away, storing the string as declared. cluster_address is the key
every capability, ownership and routing lookup matches on, so one host
could sit in that column under two spellings, and the previous commit
compensated with LOWER() in the ownership query, which gives up the
cluster_address index on a query that runs for every account-scoped proxy
connect.
Keep the canonical form instead. Connect is the only writer of the column
(heartbeats touch last_seen and status), and proxy rows are session state
rebuilt on every connect rather than durable config, so the column
converges without a migration and the lookups can stay exact and indexed.
Folding happens before punycode conversion, not after: idna lowercases the
ASCII it produces but does not case-fold the unicode it consumes, so
PRÖXY.example.com and pröxy.example.com would otherwise encode to two
different labels for one host.
The agent network check keeps comparing normalised forms in memory, which
costs nothing there — it is a pass over the account's cluster list, not a
query — and covers rows written before this landed.
Proxies store their cluster address as they declared it, while
proxy_address is normalised lowercase before validation. The capability
and ownership lookups match cluster_address exactly, so feeding them the
normalised form asked about a spelling the store may never have seen: a
private cluster declared with capitals came back unproven and was
refused, and — worse — another account's cluster declared with capitals
came back as "never declared" and let the pin through.
Compare identity on the normalised form but keep the stored spellings,
and read the capability under each of them, any-true, the same way it
aggregates over a cluster's proxies. Ownership gets the same treatment at
the source: hostnames are case-insensitive, so two spellings of one host
are one cluster and must conflict rather than being claimable side by
side, which also closes the same gap in the proxy-registration
availability check that shares the query.
The e2e's wait for a stopped cluster to leave the active list now allows
for the active window rather than 90s: a proxy that dies without closing
its stream is only dropped once its last heartbeat ages past
proxyActiveThreshold, so the old budget could fail the test on the slow
path alone.
A bad upstream or key saved cleanly and surfaced minutes later as a failed
request or an empty model picker, with nothing pointing back at the record.
CreateProvider now spends the credential once against the vendor's model
listing. UpdateProvider does the same when the upstream, the key, the catalog
provider or the skip-TLS flag changed — only then, so renames and price edits
neither wait on a vendor nor fail because one is down. Both run before the store
write, so a rejected rotation leaves the working key where it was.
What cannot be checked still saves: no listing endpoint, no derivable Bedrock
control-plane host, a private upstream, a record skipping TLS verification.
Everything else blocks, outages included — 5xx, 429 and timeouts leave the
record unverified just as a refusal does. Refusals return 422 and carry no
status code or echoed URL.
Discovery now reads as a partial edit, so a retyped URL can be listed against
without also rotating the credential. Entries with their own listing host
(Bedrock) get their configured upstream resolved separately, since a successful
listing said nothing about it.
After #7221, guardrail allowlists built from a path-style provider's declared
model ids (Bedrock, Vertex) stopped working: the raw region/version form was
compared against the parser's canonical id, so the agent config advertised an
empty model list and requests for the allowlisted model were refused.
Make every allowlist compare provider-aware, keyed on the destination
provider's catalog id: the agent config, the policy gate, and the synthesized
proxy allowlists match an entry on both its verbatim and canonical form —
Bedrock's strip only under bedrock_api, Vertex's only under vertex_ai_api,
verbatim everywhere else, so a plain provider's suffixed entries never widen.
The router's claim compare learns the Vertex @version strip.
New e2e, realstore, and unit tests reproduce both regressions and pin the fix.
The synthesised agent network gateway service is unconditionally private:
agents reach it over the WireGuard tunnel and are authorised by
ValidateTunnelPeer against the enabled policies' source groups, and its
only target is the cluster itself with DirectUpstream. Only a proxy
running embedded in a netbird client can serve that, which management
already reports per cluster as the `private` capability.
CreateSettings accepted any hostname as proxy_address, so a labeled
bootstrap could pin an account to a cluster that cannot serve its
gateway — another account's BYOP cluster, or one whose proxies are all
centralised. The endpoint assigned at bootstrap is immutable, so the
account is then stuck with a dead gateway until someone deletes and
re-bootstraps the settings row.
Validate the cluster before allocating an endpoint beneath it. Whether
management knows a cluster is decided on its proxy rows, never on how
fresh their heartbeats are: the rows outlive their proxies' liveness, so
a known cluster stays judged as one and has to prove with a live embedded
proxy that it can serve the gateway. Deciding on liveness instead would
let the same centralised cluster pass or fail depending on whether its
proxies had heartbeated in the last couple of minutes, turning "wait for
the proxy to go quiet" into a way to pin the endpoint to a cluster that
can never serve it. Ownership comes from the same time-independent
source, so a foreign cluster stays refused while it is offline.
Only a cluster no proxy has ever declared is still pinnable — that is the
address-first order the dedicated (self-addressed) path documents, and
the one self-hosted setups follow when they configure before deploying.
This introduces a disabled-by-default allow-remote-jobs setting that
controls whether the management server may run jobs (such as debug
bundles) on a peer. The flag propagates end to end: through client
configuration, the daemon SetConfig and Login requests, authentication,
and system info, up to management, where it is stored on the peer and
exposed on the peers API as remote_jobs_allowed. The client refuses any
management-requested job unless the peer has opted in. Because enabling
remote jobs crosses the user-to-root boundary, turning it on requires
privilege, mirroring the SSH-server gate. Administrators can enforce the
setting through MDM policy on both macOS and Windows, and MDM can also
override the debug-bundle upload URL. The change ships policy
documentation and generated profile templates, and adds configuration,
conflict, and enforcement tests covering the opt-in, privilege, and MDM
paths.
Delegating Agent Network today means handing out full account admin, and
regular users cannot see their own usage or how to connect a local tool.
Add two roles on top of the existing agent_network permission
submodules. agent_network_admin owns the whole area (providers,
policies, guardrails, budgets, usage, logs, settings) with read-only
users, groups, peers, and account info needed to build policies, and
nothing else in the account. usage_viewer is the regular User baseline
plus read on the aggregated usage and cost overview: no provider
configuration, no policies, no request-level logs, which can contain
captured prompts. billing_admin gets a proper permission-map entry with
the User baseline so role resolution stops failing with role-not-found;
its plan and invoice permissions stay enforced cloud-side.
Add the self-service endpoints behind the "My Agent Network" view,
available to every authenticated user because both answers are scoped
strictly to the caller. GET /api/agent-network/me/setup returns the
account endpoint plus the providers and models the caller's own groups
authorize, computed with the same rules the proxy enforces: policy
filtering as in policy selection, model allowlist union intersected
with declared models, orphan and disabled providers omitted. Not set up
and no access are deliberately indistinguishable, and the response
carries display metadata only. GET /api/agent-network/me/consumption
returns the caller's own user-dimension counters.
This extends the management-requested remote debug-bundle job with two
new, optional parameters. anonymize_level selects how aggressively the
bundle is scrubbed: "default" keeps internal (private) IP ranges
readable, while "strict" also anonymizes private, CGNAT and link-local
addresses; the value is trimmed and lowercased, and an unknown level is
rejected at creation. upload_url lets an operator point the peer at a
specific upload service instead of the default one; it must be a
well-formed https URL with a host, and an empty value falls back to the
default upload server. Both fields flow through the job workload API and
are surfaced in the create-debug-job modal on the dashboard. Validation
is shared so the client executor and the management boundary agree on
what a valid upload URL is, preventing drift between the two checks.
* Bump go version to 1.26 and go-quic to v0.62.0
* Replace deprecated ecdsa public key assembly and add tests for jwt
* Update goversioninfo
* Pin go toolchain to 1.26.7
* Support per-peer lazy connection state and default proxy peers to lazy
* Classify forward targets from incoming config in lazy exclusion
* Set IsUserspaceBind mock so lazy manager starts in engine test
* Skip lazy exclude reconciliation when the set is unchanged
* Keep cached lazy flag when a sync carries no peer config
* fix a nil-ptr error occuring in sendInitialSync when the peer being synced is deleted
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
* handle a nil ptr in GetPeerNetworkMapComponents
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
---------
Signed-off-by: Dmitri Dolguikh <dmitri.external@netbird.io>
[proxy,management] Serve Bedrock model discovery from the control plane
A Bedrock provider could never answer a model-discovery request. The router
sent GET /inference-profiles to the record's upstream, which has to be
bedrock-runtime.<region> for InvokeModel to work, and that host does not
implement the operation. ListInferenceProfiles is a control-plane operation on
bedrock.<region>.amazonaws.com, and one provider record carries one upstream,
so the two hosts genuinely differ.
The route now carries a discovery host, taken from the catalog's declaration
with the region read back out of the configured upstream, and the listing — and
only the listing — goes there. Inference is untouched. A proxied or self-hosted
Bedrock endpoint gets no discovery host at all rather than a guessed one, since
inventing a host would send the operator's credential somewhere they never
configured.
Two things had to follow for the listing to be usable once it arrives. The
response filter only understood OpenAI's {"data":[{"id":…}]}, so a Bedrock
listing fell through it untouched, offering every profile in the account
whatever the policy said. And discoverableModels intersected by exact string,
so a record registering the raw profile id while a guardrail names the catalog
key intersected to nothing — bounding a working provider's listing down to
empty.
Normalisation is the third. The geography in front of a cross-region profile
was matched against a hardcoded list of four, so every profile issued under jp,
au, ca, sa or us-gov carried its prefix into the pricing key, matched no
catalog entry and metered at zero. It is now recognised by either the geography
or the vendor that follows it, so an id has to be new on both axes at once to
slip through — a live eu-central-1 listing returned "global.xai.grok-4.6" days
after the vendor list was first written.
[management] Offer a provider's live model list in the config form
Adds POST /api/agent-network/catalog/providers/models, which asks a vendor
which models an operator's own credential can actually reach, so the provider
form can offer a live list instead of only the compiled-in catalog. The catalog
goes stale, and it cannot see an account: which OpenAI models an org is
entitled to, which Bedrock inference profiles an account and region hold, which
Vertex models a project has enabled.
The endpoints, auth headers and response shapes come from probing the live APIs
(#7244); each vendor invented its own envelope and none can be guessed from the
request. Bedrock shaped the design: its listing lives on the control plane
while inference must go to the runtime host, so Discovery carries its own host
rather than reusing the record's upstream, and profile ids are taken verbatim
because the region prefix is what AWS requires at invoke time.
A caller supplies either the key they are typing or the id of a saved record
whose stored credential is reused — never both, since accepting both would run
an arbitrary credential under the identity of a record the caller may only be
permitted to read. Gated on Create rather than Read, because this spends the
operator's credential against a third party.
Management has not made outbound calls on an operator's behalf before and it
holds a credential for every provider, so every resolved address must be public
— covering loopback, RFC1918, the cloud metadata address and NetBird's own
100.64/10 range — and redirects are not followed, since a redirect moves the
request to a host the check never saw.
The vendor is authoritative for the id; the catalog stays authoritative for
pricing. A discovered model the shipped table cannot price returns
pricing_known: false so the operator must set rates rather than being
registered at a silent zero.
[proxy,management] Bound model discovery to the caller's own policies
GET /v1/models was bounded by the provider record's enumerated models, which is
the right bound only while a single policy reaches a provider. Where two teams
share one provider under different allowlists, every caller was offered the
union — each model outside their own policy being a request the guardrail
refuses a moment later. A gateway record enumerating nothing was worse: it
offered the upstream's entire catalogue however narrow the policy was.
Each route now carries one rule per authorising policy — its source groups and
the models it permits — instead of a single flattened list. At request time the
router keeps the rules whose groups intersect the caller's, unions their
models, and intersects that with what the provider serves.
nil and [] stay distinct end to end: a policy setting no allowlist reaches the
router as nil and lifts the restriction for the groups it binds, while an
enabled allowlist with no models arrives as [] and permits nothing. Collapsing
them would let a listing that should offer nothing fall open to everything.
The guardrail's own per-provider allowlist is untouched. It is a fail-closed
backstop that cannot tell who is asking, so discovery is now narrower than the
backstop rather than wider.
