mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-12 17:59:06 +02:00
40b6a554c779d3faa1883db1dae6f7a3ffd51508
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Riccardo Manfrin
|
a419e770d9 |
[client, proxy] Make the buffer-pool retune reachable while a device is stalled (#7452)
* [client] Track the WireGuard device on the engine as a lock-free handle Add an atomic handle on the wg device next to wgInterface, stored once the interface is up and cleared when it is closed. Nothing reads it yet, so this is a pure addition with no behavior change; it exists so the next commit can reach the device without taking syncMsgMux. * [client] Retune the WireGuard buffer pool without the engine lock SetPerformance took syncMsgMux before reaching the device. That lock is held by handleSync while it adds and removes peers, and peer removal is exactly what blocks when a device's buffer pool is exhausted: Peer.Stop waits on a keepalive timer callback that is itself parked in WaitPool.Get. Raising the cap is the way out of that state, so the call must not queue behind the lock the stall is holding. Read the device through the atomic handle instead. Device.SetPreallocatedBuffersPerPool takes the pool's own lock and broadcasts, so the waiters wake up. * [proxy] Extract the buffer-cap apply loop out of the perf handler Pure move: the loop over the registered clients becomes applyBufferCap, with the same sequential behavior and the same return values. Split out so the next commit can change how it iterates without the diff also carrying the move. * [proxy] Bound the perf endpoint so one wedged client cannot hold it The apply loop was sequential and unbounded. embed.Client.SetPerformance goes through the client lock, which Start holds for the whole of a startup, so a single account that is busy or wedged delayed the new buffer cap for every other account on the node -- on the endpoint whose whole purpose is to un-wedge a node. Apply to all clients concurrently and give the whole call a 5s budget. Accounts that do not answer in time are reported in "failed" instead of blocking the response. * [client] Drop the device handle before closing the interface close() cleared the atomic handle only after wgInterface.Close() returned, so a concurrent SetPerformance could still load it, retune a device that is being torn down, and report the change as applied for an engine that has stopped. Clear it first, so the window closes before the teardown begins. Reported by cubic on PR #7452. * [proxy] Put the per-client retune behind a field Pure refactor: applyBufferCap calls h.setPerformance instead of the client method directly, and NewHandler wires it to setClientPerformance. Same call, same behavior; the seam is what lets the next two commits be tested without a live embedded client. * [proxy] Do not report a finished retune as timed out When the deadline fires, select chooses at random among the ready cases, so a result already sitting in the buffered channel could be skipped and its account reported as timed out even though the cap had been applied. Drain what is buffered before declaring the rest pending. Reported by cubic on PR #7452. * [proxy] Keep one retune per account in flight The 5s budget bounds how long the endpoint waits, not the work: SetPerformance goes through the embedded client's lock, and on a wedged account Stop holds that lock forever, so every retry left one more goroutine parked there. Route each account through a single worker. A request that finds one already running takes its result if it has landed, and otherwise reports the account under "in_flight" instead of starting a second attempt. One stuck account now costs one goroutine, no matter how often the endpoint is called. Reported by CodeRabbit and cubic on PR #7452. * [proxy] Make the retune budget a var Pure refactor: perfApplyTimeout becomes a var so a test can shorten it instead of waiting five seconds. Same value, same behavior in production. * [proxy] Extract the buffered-result drain Pure refactor: the loop that empties the results channel when the deadline fires becomes collectBuffered. Same behavior; split out so it can be tested on its own, which the inline version could not be without racing the deadline. * [proxy] Cover the retune single-flight and the deadline drain TestApplyBufferCapSingleFlightPerAccount fails without the worker registry: five calls against a client stuck in its own lock start five blocked workers instead of one. TestCollectBufferedCountsResultsReadyAtTheDeadline pins the drain helper's contract - buffered results counted, errors recorded, only unanswered accounts left pending. It drives collectBuffered directly: through applyBufferCap the two select cases race by construction, so an end-to-end version of it would pass on the unfixed code about half the time. * [proxy] Keep the worker alongside each pending account Pure refactor: the pending set becomes a map to the account's worker instead of an empty struct. Same membership and same behavior; the next commit needs the worker to resolve an account whose result has not reached the channel yet. * [proxy] Publish a retune result before releasing its slot The worker sent its result last, after taking perfMu to remove itself from the registry. That lock is taken once per account by every caller walking the fleet, so a worker that finished on time could queue behind an apply over thousands of accounts and land after the deadline. Send first, deregister after. Reported by cubic on PR #7452. * [proxy] Read the worker, not the clock, for a finished retune Publishing earlier only narrows the window: a client that answers just before the deadline can still be reported as timed out. At the deadline the workers themselves are authoritative - a closed done channel means the retune finished and w.err carries its outcome, ordered by the close. Consult them instead of declaring every pending account timed out, and keep the timeout label for the ones actually still running. Reported by cubic on PR #7452. * [proxy] Cover the finished-worker resolution at the deadline Fails on the previous behavior with "applied = 0, want 1": every pending account was labelled a timeout, including the one whose retune had already completed. |
||
|
Maycon Santos
|
fa1e241aea |
[management, client, proxy] Follow-up fixes for private reverse-proxy services (#6268)
* fix(proxy): gate tunnel-peer fast-path on inbound listener marker
forwardWithTunnelPeer previously accepted any RFC1918 / ULA / CGNAT
source IP, so a public client whose address happened to fall in those
ranges could bypass the configured operator auth scheme by colliding
with a known tunnel IP. The fast-path is now gated on
TunnelLookupFromContext(r.Context()) being present — that context value
is attached only by the per-account inbound (overlay) listener, so the
host-facing listener never enters this branch.
Tests updated to reflect the new requirement: requests that don't
carry the inbound marker now fall through to the regular auth flow.
* fix(proxy): harden inbound listener resource + startup-ctx handling
Three correctness fixes on the per-account inbound path, with tests:
- Close the logrus ErrorLog PipeWriter on tearDown. WriterLevel hands
back an *io.PipeWriter backed by a pipe + scanner goroutine that the
caller owns; the two writers per account (https + plain) were never
closed, leaking the pipe and goroutine on every teardown.
- Run the post-Start hooks on context.Background(). runClientStartup
is launched in a goroutine from AddPeer and was inheriting the
caller's request-scoped ctx, so a cancelled request could abort the
inbound bring-up or fail the management status notification. The
tail is split into notifyClientReady so the contract is testable.
Tests cover the PipeWriter close behaviour and assert the readyHandler
+ NotifyStatus calls receive a non-cancelled background context.
* feat(proxy): short-circuit peer-own-target loops with 421
When a peer that hosts the target of a private service dials its own
service URL the request was being looped through the proxy and back
over WireGuard to the same peer — twice the WG round-trip for no
benefit, with no signal to the caller that something was wrong.
Add isSelfTargetLoop to ReverseProxy.ServeHTTP: when the request
arrived on the per-account overlay listener (IsOverlayOrigin) and the
source tunnel IP matches the target host, refuse the request with 421
Misdirected Request and a body pointing the operator at the backend
directly.
The gate is scoped to overlay origin so requests on the public
listener that happen to share a source IP with the target host are
forwarded normally.
* fix(management): private-service validation + tunnel-IP lookup semantics
- Require an explicit port for L4 cluster targets. validateL4Target
exempted TargetTypeCluster from the port check, but buildPathMappings
serializes every L4 target via net.JoinHostPort(host, port) — port=0
shipped a ":0" upstream. Cluster targets use the same Host/Port
fields, so the same requirement applies.
- GetPeerByIP returns NotFound on a tunnel-IP miss instead of mapping
every error to Internal. The proxy's ValidateTunnelPeer probes IPs
that legitimately aren't in the roster; the miss is expected and now
distinguishable from a real store failure.
- Thread ctx into getClusterCapability's gorm query so a cancelled
request doesn't keep the store busy.
Tests updated for the L4-cluster port requirement and the GetPeerByIP
NotFound path.
* fix(client): include offlinePeers in PeerStateByIP lookup
ReplaceOfflinePeers moves peers into d.offlinePeers but PeerStateByIP
only scanned d.peers. Callers (the local DNS filter via
localPeerConnectivity, embed.Client.IdentityForIP used by the
proxy's tunnel-peer validator) were treating known-but-offline peers
as unknown, which:
- causes the DNS filter to keep returning records pointing at peers
that have no live tunnel, AND
- makes the proxy's local-roster check deny a request from such a
peer rather than letting the cached management RPC carry the
authorisation decision.
Search both slices in PeerStateByIP. Adds a unit test for the IPv4
and IPv6 offline-match paths.
* fix(rest): reject empty Delete path params in reverse-proxy clients
ReverseProxyClustersAPI.Delete and ReverseProxyTokensAPI.Delete passed
the path parameter into url.PathEscape without an empty check.
PathEscape("") returns "" which collapses the request onto the
collection endpoint ("/api/reverse-proxies/clusters/" /
"/api/reverse-proxies/proxy-tokens/"), so a caller bug delete with no
id reached a routable URL with surprising semantics (typically 405).
Short-circuit with a typed error before the request is built. Tests
mount a handler on the collection path that fails the test if hit, so
the regression is impossible to reintroduce silently.
* chore(api,ci,docs,test): private-service schema, proto-check, fixups
Non-functional cleanups and contract/CI hardening around the
private-service work:
API schema (openapi.yml):
- Require a non-empty access_groups and mode=http when private=true,
on both Service and ServiceRequest, mirroring
validatePrivateRequirements. mode stays optional-but-constrained
(empty defaults to http server-side), matching runtime.
CI (proto-version-check.yml):
- Cover renamed .pb.go files (read base via previous_filename).
- Match protoc-gen-go-grpc version headers (optional "- " prefix and
-gen-go-grpc suffix) so grpc-generated files are in scope.
Docs / comments:
- Reword Config field docs to say defaults are applied at Server.Start
(initDefaults), not New.
- Rename the obsolete --private-inbound flag to --private across
comments and the proto doc.
Pre-existing test fixups surfaced by review:
- Repair the integration-tagged validate_session_test.go (SignToken
signature growth + new Manager interface methods).
- Fix the CI-skip boolean precedence so Windows isn't skipped
unconditionally.
- Guard the router.HTTPListener type assertion with comma-ok.
* fix(proxy): background ctx for already-started AddPeer notification
The earlier ctx fix covered the async runClientStartup path but missed
the synchronous branch: when a service is added to an already-started
client, AddPeer called NotifyStatus with the caller's request-scoped
ctx. A cancelled request/stream could drop the connected notification
to management. Use context.Background() here too, matching
notifyClientReady.
Extends TestNetBird_AddPeer_ExistingStartedClient_NotifiesStatus to
pass a pre-cancelled caller ctx and assert the notification still ran
on a non-cancelled context.
* use the cmd context for roundtripper
|
||
|
Viktor Liu
|
e89b1e0596 | [proxy, client] Bound embed client WireGuard per-Device memory (#5962) | ||
|
Maycon Santos
|
7aebdd69dd |
[management, client, proxy] add expose NetBird-only services over tunnel peers (#6226)
Adds a new "private" service mode for the reverse proxy: services reachable exclusively over the embedded WireGuard tunnel, gated by per-peer group membership instead of operator auth schemes. Wire contract - ProxyMapping.private (field 13): the proxy MUST call ValidateTunnelPeer and fail closed; operator schemes are bypassed. - ProxyCapabilities.private (4) + supports_private_service (5): capability gate. Management never streams private mappings to proxies that don't claim the capability; the broadcast path applies the same filter via filterMappingsForProxy. - ValidateTunnelPeer RPC: resolves an inbound tunnel IP to a peer, checks the peer's groups against service.AccessGroups, and mints a session JWT on success. checkPeerGroupAccess fails closed when a private service has empty AccessGroups. - ValidateSession/ValidateTunnelPeer responses now carry peer_group_ids + peer_group_names so the proxy can authorise policy-aware middlewares without an extra management round-trip. - ProxyInboundListener + SendStatusUpdate.inbound_listener: per-account inbound listener state surfaced to dashboards. - PathTargetOptions.direct_upstream (11): bypass the embedded NetBird client and dial the target via the proxy host's network stack for upstreams reachable without WireGuard. Data model - Service.Private (bool) + Service.AccessGroups ([]string, JSON- serialised). Validate() rejects bearer auth on private services. Copy() deep-copies AccessGroups. pgx getServices loads the columns. - DomainConfig.Private threaded into the proxy auth middleware. Request handler routes private services through forwardWithTunnelPeer and returns 403 on validation failure. - Account-level SynthesizePrivateServiceZones (synthetic DNS) and injectPrivateServicePolicies (synthetic ACL) gate on len(svc.AccessGroups) > 0. Proxy - /netbird proxy --private (embedded mode) flag; Config.Private in proxy/lifecycle.go. - Per-account inbound listener (proxy/inbound.go) binding HTTP/HTTPS on the embedded NetBird client's WireGuard tunnel netstack. - proxy/internal/auth/tunnel_cache: ValidateTunnelPeer response cache with single-flight de-duplication and per-account eviction. - Local peerstore short-circuit: when the inbound IP isn't in the account roster, deny fast without an RPC. - proxy/server.go reports SupportsPrivateService=true and redacts the full ProxyMapping JSON from info logs (auth_token + header-auth hashed values now only at debug level). Identity forwarding - ValidateSessionJWT returns user_id, email, method, groups, group_names. sessionkey.Claims carries Email + Groups + GroupNames so the proxy can stamp identity onto upstream requests without an extra management round-trip on every cookie-bearing request. - CapturedData carries userEmail / userGroups / userGroupNames; the proxy stamps X-NetBird-User and X-NetBird-Groups on r.Out from the authenticated identity (strips client-supplied values first to prevent spoofing). - AccessLog.UserGroups: access-log enrichment captures the user's group memberships at write time so the dashboard can render group context without reverse-resolving stale memberships. OpenAPI/dashboard surface - ReverseProxyService gains private + access_groups; ReverseProxyCluster gains private + supports_private. ReverseProxyTarget target_type enum gains "cluster". ServiceTargetOptions gains direct_upstream. ProxyAccessLog gains user_groups. |
||
|
Viktor Liu
|
205ebcfda2 | [management, client] Add IPv6 overlay support (#5631) | ||
|
Viktor Liu
|
057d651d2e | [client, proxy] Add packet capture to debug bundle and CLI (#5891) | ||
|
Viktor Liu
|
82762280ee | [client] Add health check flag to status command and expose daemon status in output (#5650) | ||
|
Viktor Liu
|
3e6baea405 | [management,proxy,client] Add L4 capabilities (TLS/TCP/UDP) (#5530) | ||
|
|
f53155562f |
[management, reverse proxy] Add reverse proxy feature (#5291)
* implement reverse proxy --------- Co-authored-by: Alisdair MacLeod <git@alisdairmacleod.co.uk> Co-authored-by: mlsmaycon <mlsmaycon@gmail.com> Co-authored-by: Eduard Gert <kontakt@eduardgert.de> Co-authored-by: Viktor Liu <viktor@netbird.io> Co-authored-by: Diego Noguês <diego.sure@gmail.com> Co-authored-by: Diego Noguês <49420+diegocn@users.noreply.github.com> Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com> Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com> Co-authored-by: Ashley Mensah <ashleyamo982@gmail.com> |