mirror of
https://github.com/netbirdio/netbird.git
synced 2026-10-10 15:39:07 +02:00
f0a40e4395
* implement certificate posture check * log signal address * add keychain and cert store support * read the console user's keychain through a user session helper A root daemon cannot reach a login keychain: securityd is per session and a key ACL needs a session to prompt in, so dropping uid is not enough. The daemon now answers certificate challenges from the System keychain itself, where MDM installs device identities, and launches "netbird posture cert-proof" into the console user's desktop session with launchctl asuser for the login keychain. Only the signature and the chain cross back, never the private key. The console user comes from SCDynamicStoreCopyConsoleUser, bound with purego like the keychain calls. The login window reports no user, root, or "loginwindow", and all three are treated as no keychain to read, so a Mac at the lock screen sends device proofs alone. Adds info logging across the path: the keychain search list, per class query status and item counts, the chain built per candidate, and the verification error for every rejected candidate. A run that sends nothing now says why. README.md documents the trust model, the console user limitation and how to read the logs. * read the signed-in user's certificate store on Windows A service reads LocalMachine\MY, where AD and Intune enrol device certificates. CurrentUser\MY lives in the signed-in user's registry hive with keys protected against their profile, and a service that opens it does not fail: "current user" resolves to HKU\S-1-5-18, so it silently reads the service account's own empty store. The service therefore reads the machine store itself and launches "netbird posture cert-proof" with the session token for the rest, mirroring the macOS console user helper. Windows lets a privileged service assume a user identity, so the token goes straight into the child process and no external tooling is involved. CREATE_NO_WINDOW keeps a console window from flashing on the desktop every sync. In-process impersonation would also work but is per OS thread while goroutines migrate, so the child process avoids that class of bug. Session selection prefers the physical console and falls back to any active session, so remote desktop and VDI hosts are covered. WTSQueryUserToken needs SE_TCB_NAME, so a user-run client skips the helper and reads the machine store alone. SystemStore takes a store location, gaining NewUserStore alongside NewSystemStore and the per candidate logging macOS already had. The request building and proof merging move to helper_spawn.go, shared by both platforms, and helperStore picks what the helper reads per platform. * start TPM support * split goreleaser to support pkcs11 and exclude on docker * update goreleaser * go mod tidy * add tpm pin to netbird config * split cert and key location and allow key lookup on tpm * add unsupported flag for mobile devices * Isolate the cert proof helper from the service environment and cap its output * Read the PKCS#11 token PIN from NB_TPM_PIN instead of the profile config * Bound certificate proof collection so a stuck token or keychain cannot hold the sync loop * Stop retrying a PKCS#11 PIN the token rejected * Log certificate posture details at debug level * Sign only nonces and peer keys of the size management issues * Skip certificate files whose key belongs to another certificate * Bound PKCS#11 driver sizes, pin template values, and log out only a login the session owns * Never pass NULL to CFRelease and skip unreadable keychain identities * Keep the macOS keychain code out of iOS and the PKCS#11 driver out of Android * Find a chain to each challenge's CAs through every intermediate the store holds * Require a token label whenever a PKCS#11 PIN is set * Read user certificates only from the session of the active profile's owner * Collect certificate proofs again when the owner's session changes and report lost proofs * Test the PKCS#11 build against SoftHSM in CI and warn once where the build has no driver * Document where an inline PKCS#11 PIN is stored and how it is protected * Refuse PKCS#11 URIs that this client cannot honour instead of widening the match * Trust certificate and key files only when no other user can write or redirect them * Explain a Windows certificate whose key only a legacy CryptoAPI provider holds * Use platform absolute module paths in tests and add a real owner session test for Windows * Match the Windows profile owner by name instead of resolving it through the domain controller * Keep the certificate stores and TPM library out of the WebAssembly build * [client] Read TSS2 key files on go-tpm, checked against the library it replaces The TSS2 parser was the only reason this repository depended on a crypto suite whose own build tooling it inherits. The replacement sits on go-tpm, which was already a direct dependency and is in fact what that suite calls underneath, so this removes a wrapper rather than porting onto a different library: the load, the derived storage root key and the signing commands are the same calls. Swapping a parser on the one path a customer actually runs is not something to assert, so the two are held side by side for this commit. One test feeds the replacement bytes the old library wrote and requires the same key type, empty auth flag, parent handle, blobs and decoded public key; the other feeds both the fixtures the tests are built on, so those are the shape the format calls for and not merely the shape the new parser reads. The scaffolding goes away with the dependency in the commit that follows. The encoder behind the fixtures is written out separately from the parser under test, so an encoder bug and a decoder bug cannot cancel each other out. * [client] Drop go.step.sm/crypto and the repo-wide upgrades it imposed The TSS2 parser was the only thing in the repository that used this module, and it brought 302 modules into the graph to do it — 35 of them linters, along with Google Cloud KMS and IAM, the AWS SDK and a terminal styling library. Those are the module's own development dependencies, which minimal version selection turns into floors in ours, and they are the whole reason gRPC, protobuf, the AWS SDK, OpenTelemetry, logrus and five x/ packages had moved. Management, signal, relay and proxy inherited every one of them for a feature none of them runs. Removing the import is not enough, because tidy never downgrades: the raised floors stay written in go.mod. Each one is pinned back to the version main had, then tidy is left to raise again whatever something still genuinely needs. It raised nothing: all 43 are back where they were, and go-tpm was already in the graph at the same version, so the certificate feature now costs no new module at all. The differential tests go with it. They existed to check the swap against the library while both were present, and there is nothing left to compare against. * [client] Clear the lint findings only the macOS and Windows runners see golangci-lint analyses one build at a time, so running it on Linux says nothing about the two platforms CI also lints. Against those builds the feature's packages reported eight findings, and the structural one is Config.dir: it is dead on macOS and Windows because neither reads a directory at all, their collectors take the configuration and discard it. Moving the method beside its only callers makes that visible in the layout instead of in a linter, and leaves the gap itself — no file or token store on those platforms — where it belongs, as something to decide rather than something to silence. An absent key file beside a certificate was reported as a nil signer with a nil error, which the caller then had to recognise by its nilness. It is a sentinel now, so the meaning is in the error rather than in the absence of one. The rest follow the standard library: the elliptic coordinates and the private scalar come from the encoding helpers rather than the deprecated big.Int fields, and an error string loses its trailing colon. Lint is clean on linux, darwin and windows; the hardware TPM path was exercised separately against a real device and passes. * Accept the TSS2 emptyAuth boolean OpenSSL writes and persistent parents on 32-bit builds * Count the certificates field in the peer meta store test * Check the store directory before listing it, refuse group-writable files, and reject a URI with two PIN sources * Share a PKCS#11 login between sessions and send each PIN at most once at a time * Collect certificate proofs again when the meta sync carrying them failed * Use no Windows user store when a domainless owner matches accounts of several domains * Use no user certificate store when the active profile's owner cannot be read * Document the PIN sources on CertPKCS11URI and keep the README PIN example off the command line * Test that the PKCS#11 URI stays out of the debug bundle and run the wrong-PIN test only on a disposable token * Refuse a TPM PSS signature request for the maximum salt length * Add the certificate fields to the network map golden data * Retry posture checks whose meta sync timed out instead of dropping them * Start no system info gathering while a timed-out one is still running * Guard the applied posture checks across goroutines and keep refreshing proofs while a pending update times out * Log what a successful certificate proof helper wrote to stderr * Send recollected certificate proofs to management only when the proven chains changed * Explain a macOS keychain key whose access list does not allow netbird * Kill the whole macOS certificate helper process group when it times out * End sudo option parsing before the macOS certificate helper binary * Hold off system info gathering only while a timed-out one is still running * Collect certificate proofs on the posture watcher instead of under the sync lock * Read the certificate store directory and PKCS#11 URI from the daemon environment, not the profile config * Install the RPM sysconfig file readable by root only and show the certificate posture variables * Move the certificate posture README into the package doc and the docs site * Name NB_CERT_PKCS11_URI in the PIN-without-token error * Keep the file check results of the latest-started system info refresh * Give the full import command for a keychain key netbird may not use, and correct the package doc * Restrict the service environment file to root on every package install * Search only the System keychain in the macOS daemon and only the login keychain in the user helper * Let the certificate proof helper read the PKCS#11 token from the environment on Linux * Ask a macOS user's keychain again only after an hour when it proved nothing * Clear the lint findings in certificate posture * Hold off the keychain helper only after a completed or timed-out run, independent of CA order * Keep free functions out of the method lists of PKCS11Store, URI and Challenger * Name the post-install permission helper in snake case and shorten the sysconfig certificate block * Drop the certificate store directory from certproof.Config, which only NB_CERT_STORE_DIR sets * [management] Renew certificate challenge nonces on quiet accounts A certificate challenge nonce is accepted for its own window and the one before it, and it only reaches a peer attached to a network map. An account where nothing changes sends no map, so after a day the peer re-sends the nonce it still holds, verification rejects its whole proof set, and the certificates stored for it are dropped. It fails the certificate check and loses every policy gated on it until some unrelated change happens to push a map. The outage repairs itself in seconds, which is what makes it expensive: it is intermittent, it only hits stable networks, and it is not reproducible on demand. Push the account's peers an update often enough that the nonce they hold is never close to expiring. Only accounts whose posture checks actually ask for a certificate are tracked, so a deployment without the feature does no extra work. The refresh runs from one goroutine over a map of accounts rather than a timer per account: the period is hours, so one pass every few minutes costs nothing next to it, and there is no timer to re-arm when an account that falls due sooner appears. Each account's first run is offset by a hash of its ID, because the challenge window is global and an instance restart would otherwise arm every account in the same moment. The push carries no administrative change, so it is counted as a refresh rather than an update and stays out of the figures that track what was edited. (cherry picked from commit7ad4a0df37) * [management] Make the certificate challenge window one knob to turn Renewal was timed against the window in two different ways: the period derived from it, the sweep interval did not. Shortening the window to watch a renewal in an end-to-end run would have left the refresher still looking for due accounts every quarter of an hour, so nothing would have been renewed in time and the test would have reported the feature broken. Derive the sweep from the period, within bounds that keep a very short window from spinning and a normal one from checking less often than is useful, and allow the window itself to be set through the environment so a run can take seconds instead of half a day. A value that cannot be parsed or falls outside the bounds keeps the default, because a window nobody intended is a security property nobody chose, and an override is logged at warning level since it sets how long a device keeps passing the check after its key is gone. Every instance has to be given the same value: the window is part of the nonce, so instances that disagree reject each other's. (cherry picked from commit0e38fcf409) * [management] Pin the property that makes per-peer nonce state unnecessary A nonce carries the window it was minted in, not the instant, and is accepted for that window and the one before it. So a peer re-stamped at least once per window can never be left holding one outside the accepted pair, whenever it was last served and however much life its own nonce had left. That is the whole reason management tracks nothing per peer, and it was resting on an argument rather than a test. The phases are part of the property, not decoration: accounts are deliberately given a refresh phase of their own, so the guarantee has to hold off the window boundary too. The negative case shows why that matters — a cadence of exactly two windows lands inside the grace window when it is aligned to the boundary and leaves a gap when it is not. (cherry picked from commitdee68facfd) * [management] Renew challenges only for the peers that answer one The refresh pushed an update to every connected peer of the account, while only the peers a certificate check applies to carry a nonce. On an account where a handful of peers sit behind the check and the rest do not, everyone was woken several times a day to be handed a map that changed nothing for them. Push to the sources of the enabled policies whose posture checks include a certificate check, which is exactly the set that is sent a challenge. Resolving the set the other way round than the gRPC layer does is the risk here: a peer the refresh forgets stops being renewed and falls out of its policies silently, which is the failure this whole mechanism exists to prevent. So the selection is held against processPeerPostureChecks, the per-peer rule that decides who receives a challenge in the first place, by a test that asks both the same question and requires the same answer. (cherry picked from commitdc4d0e0274) * [management] Derive certificate challenge nonces from the stored encryption key The nonce secret came from the server's WireGuard key, which is generated afresh in every process and never persisted. A nonce carries no state, so the only thing that lets one instance verify what another issued is deriving the same secret — and that premise, written in the comment above the challenger, was not met: every instance had its own key. A peer reconnecting after a restart therefore presented a nonce minted under the previous secret, verification failed with a mismatch, its whole proof set was rejected and the certificates stored for it were dropped until it signed again. Reproduced three times on the lab, each one logging "nonce was not issued to this peer", which only a changed secret produces. On a single instance it costs seconds of lost policy access per restart; across instances it is not transient at all, because every reconnect that lands elsewhere is rejected the same way. Derive from the data store encryption key instead: it is generated once, written back to the configuration and read by every instance, so it survives restarts and is shared. Where none is configured the secret falls back to the WireGuard key with a warning — degraded but still unpredictable, which is the property that matters most: a peer able to guess it could mint the nonces of future windows, sign them while its key is present and keep passing after it is gone. The challenger is now built once and passed to the two places that need it, rather than re-derived per message. (cherry picked from commit278f2f3807) * [management] Register an account for renewal where its nonce is issued Renewal was armed when a peer connected or when a posture check was saved, both of which ask the store whether the account has a certificate check. That misses the case it most needs to catch: the check is created through one instance while the peers are connected to another, so the instance serving them never learns it has anything to renew and their nonce expires. It also charged a query to every peer connect in every account, including the ones that will never use the feature, which a fleet reconnecting after a restart pays all at once. Register where the nonce is actually stamped instead. A nonce is verified from a shared secret and so travels between instances, but the renewal that keeps it fresh cannot: only the instance holding a peer's stream can push to it. Issuing and renewing now line up by construction — an instance renews exactly the accounts it has issued nonces for — and an instance that never issues one has nothing to renew, so there is no case left to miss. The registration is a map insert with no store access, which is what lets it sit on a path taken by every login and every initial sync. Reported by Viktor Liu, who also proposed registering at the point of issue. (cherry picked from commit 2d16dd7d7cf54762f2e64c5630ea092f32ef63ab) * [management] Register for renewal on pushed updates, not only on connect Registering where the nonce is stamped only covered the login and the initial sync, which both happen when a peer opens a stream. That left out the path the mechanism exists for. On the cloud the network map controller is wrapped so that an update publishes to an event bus instead of pushing locally: an instance handling a REST change broadcasts, and every instance holding a peer of that account pushes to its own. Those pushes stamp a nonce through the update handler, and nothing there registered, so an instance learned about an account only when one of its peers happened to reconnect. For a quiet fleet that is the original bug: the check is created, the peers are told about it, and nobody renews what they were told. Registering on the pushed update closes it, and is the difference between stamping and marking a peer connected — one happens on every push, the other only when a stream opens. Reported by Viktor Liu; the broadcast that makes it work was pointed out by Pascal Fischer. (cherry picked from commit 59efe8d93e53bacdf57cb546f4ab2c19dc4eddab) * [management] Let the challenge refresh loop stop with the manager that owns it The loop was started on a context explicitly detached from the caller's, so nothing could ever stop it. Production is unaffected either way, since BuildManager is called with context.Background(), but a test that builds a manager leaked a sweeping goroutine for the rest of the run, and a shutdown path added later would have had no way to reach it. Take the manager's context as the request buffer built on the line above already does. The test pins the contract the loop offers, so a detached context cannot come back inside Start either. * [management] Bound one account's challenge refresh so it cannot starve the rest Resolving which peers answer a challenge reads the store three times, and the refresher sweeps accounts one after another on a single goroutine. A read that never returns held the sweep for the life of the process, so every other account on the instance stopped being renewed and its peers fell out of the policies gated on the check: one account's bad luck became an outage for all of them. Give each refresh the sweep interval it is allowed to occupy, capped at 30s so a 12-hour window does not grant minutes to a query that should take milliseconds. A refresh that runs out of time keeps its account tracked, since a deadline says nothing about whether that account still has a certificate check. * [management] Send challenge refreshes down the path the rest of management uses The refresh dispatched through UpdateAffectedPeers, the one variant that takes no reason, so it was missing from the update counters and coalesced with nothing. An administrator editing a policy while the sweep ran made the account's network map twice over, and UpdateOperationRefresh, added for exactly this caller, was never referenced. Buffer it with a posture_check/refresh reason instead. The periodic push is now visible in the metrics as what it is, distinct from an edit, and the send detaches from the sweep deadline on its own, so that deadline bounds the store reads it was meant for. * [management] Keep the certificate challenge comments to what the history does not say Four of these ran to three and four times the comment budget, the longest at 992 characters. Most of the excess argued against designs that were never written or explained a bug that no longer exists in the code, which is what the commit that fixed it is for. What is left is the part a reader cannot recover from the code: that the nonce secret has to be persisted and unpredictable, that stamping and renewing are decided together because only the serving instance can push, and that the target rule is the inverse of processPeerPostureChecks. * Keep the newest posture checks pending whatever made their meta sync fail * Report no lost certificate when the engine stops during a proof collection * Share the proof collection single-flight across engine restarts * Close a PKCS#11 module that loads but cannot be used * Fix the pending checks comments * Renew certificate challenges only for the peers streamed to this instance * Ignore a challenge stamp from an older sync stream of the same peer * Kill the Windows certificate proof helper with its whole process tree * Expect the challenge untrack in the session ownership test * Drop an invalid certificate proof without discarding the valid ones * Start a system info gathering beside one that has been stuck for ten timeouts --------- Co-authored-by: pascal <pascal@netbird.io> Co-authored-by: mlsmaycon <mlsmaycon@gmail.com> Co-authored-by: riccardom <riccardomanfrin@gmail.com>
1780 lines
59 KiB
Go
1780 lines
59 KiB
Go
package debug
|
|
|
|
import (
|
|
"archive/zip"
|
|
"bufio"
|
|
"bytes"
|
|
"compress/gzip"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"io/fs"
|
|
"net"
|
|
"net/netip"
|
|
"os"
|
|
"path/filepath"
|
|
"runtime"
|
|
"runtime/pprof"
|
|
"sort"
|
|
"strings"
|
|
"time"
|
|
|
|
log "github.com/sirupsen/logrus"
|
|
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
|
|
"google.golang.org/protobuf/encoding/protojson"
|
|
|
|
"github.com/netbirdio/netbird/client/anonymize"
|
|
"github.com/netbirdio/netbird/client/configs"
|
|
"github.com/netbirdio/netbird/client/internal/peer"
|
|
"github.com/netbirdio/netbird/client/internal/profilemanager"
|
|
"github.com/netbirdio/netbird/client/internal/updater/installer"
|
|
nbstatus "github.com/netbirdio/netbird/client/status"
|
|
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
|
|
"github.com/netbirdio/netbird/shared/netiputil"
|
|
)
|
|
|
|
const readmeContent = `This debug bundle contains the following files.
|
|
If anonymization is enabled (--anonymize / --anonymize-level), the files are anonymized to protect sensitive information.
|
|
|
|
status.txt: Anonymized status information of the NetBird client.
|
|
client.log: Most recent, anonymized client log file of the NetBird client.
|
|
netbird.err: Most recent, anonymized stderr log file of the NetBird client.
|
|
netbird.out: Most recent, anonymized stdout log file of the NetBird client.
|
|
routes.txt: Detailed system routing table in tabular format including destination, gateway, interface, metrics, and protocol information, if --system-info flag was provided.
|
|
interfaces.txt: Anonymized network interface information, if --system-info flag was provided.
|
|
ip_rules.txt: Detailed IP routing rules in tabular format including priority, source, destination, interfaces, table, and action information (Linux only), if --system-info flag was provided.
|
|
iptables.txt: Anonymized iptables (IPv4) rules with packet counters, if --system-info flag was provided.
|
|
ip6tables.txt: Anonymized ip6tables (IPv6) rules with packet counters, if --system-info flag was provided.
|
|
ipset.txt: Anonymized ipset list output, if --system-info flag was provided.
|
|
nftables.txt: Anonymized nftables rules with packet counters across all families (ip, ip6, inet, etc.), if --system-info flag was provided.
|
|
sysctls.txt: Forwarding, reverse-path filter, source-validation, and conntrack accounting sysctl values that the NetBird client may read or modify, if --system-info flag was provided (Linux only).
|
|
resolv.conf: DNS resolver configuration from /etc/resolv.conf (Unix systems only), if --system-info flag was provided.
|
|
scutil_dns.txt: DNS configuration from scutil --dns (macOS only), if --system-info flag was provided.
|
|
dns_windows.txt: Anonymized NRPT rules and policy table in effect, DNS client policy, and per-interface and per-adapter DNS configuration (Windows only), if --system-info flag was provided.
|
|
resolved_domains.txt: Anonymized resolved domain IP addresses from the status recorder.
|
|
config.txt: Anonymized configuration information of the NetBird client.
|
|
network_map.json: Anonymized sync response containing peer configurations, routes, DNS settings, and firewall rules.
|
|
state.json: Anonymized client state dump containing netbird states for the active profile.
|
|
service_params.json: Sanitized service install parameters (service.json). Sensitive environment variable values are masked. Only present when service.json exists.
|
|
metrics.txt: Buffered client metrics in InfluxDB line protocol format. Only present when metrics collection is enabled. Peer identifiers are anonymized.
|
|
mutex.prof: Mutex profiling information.
|
|
goroutine.prof: Goroutine profiling information.
|
|
block.prof: Block profiling information.
|
|
heap.prof: Heap profiling information (snapshot of memory allocations).
|
|
allocs.prof: Allocations profiling information.
|
|
threadcreate.prof: Thread creation profiling information.
|
|
cpu.prof: CPU profiling information.
|
|
stack_trace.txt: Complete stack traces of all goroutines at the time of bundle creation.
|
|
capture.pcap: Packet capture in pcap format. Only present when capture was running during bundle collection. Omitted from anonymized bundles because it contains raw decrypted packet data.
|
|
|
|
|
|
Anonymization Process
|
|
The files in this bundle have been anonymized to protect sensitive information. The level applied to this bundle is recorded at the top of this file. Here's how the anonymization was applied:
|
|
|
|
IP Addresses
|
|
|
|
Default level:
|
|
- Public IPv4 addresses are replaced with addresses starting from 198.51.100.0
|
|
- Public IPv6 addresses are replaced with addresses starting from 2001:db8:ffff::
|
|
- IPv6 unique local addresses (fc00::/7) are anonymized as well: their random global ID uniquely identifies the network.
|
|
- IP addresses from internal IPv4 ranges and well-known addresses are not anonymized (e.g. 8.8.8.8, 100.64.0.0/10, addresses starting with 192.168., 172.16., 10., 169.254., fe80::).
|
|
|
|
Strict level (--anonymize-level strict), in addition to the default level:
|
|
- Private (RFC 1918), CGNAT (100.64.0.0/10), and link-local (169.254.0.0/16, fe80::/10) addresses are anonymized too.
|
|
- Internal IPv4 addresses are replaced with addresses starting from 198.18.0.0 and internal IPv6 addresses with addresses starting from 2001:db8:1::, so internal addresses remain distinguishable from public ones.
|
|
- Addresses are mapped in order of first appearance: subnet structure, allocation scheme, and gateway conventions are not preserved. Prefix lengths of networks are preserved.
|
|
- Peer names in front of NetBird domains are replaced with numbered placeholders (e.g. peer-1.netbird.cloud), and subdomain labels of other domains with host-N placeholders.
|
|
- WireGuard public keys are replaced with consistent placeholder keys.
|
|
|
|
Reoccuring IP addresses are replaced with the same anonymized address.
|
|
|
|
Note: The anonymized IP addresses in the status file do not match those in the log and routes files. However, the anonymized IP addresses are consistent within the status file and across the routes and log files.
|
|
|
|
MAC Addresses
|
|
MAC addresses are replaced at every anonymization level with consistent placeholders counting up from 02:00:00:00:00:01. Broadcast, multicast, and all-zero addresses are kept. At the default level a preserved IPv6 link-local address may still embed a MAC address (EUI-64); the strict level anonymizes those addresses.
|
|
|
|
Domains
|
|
All domain names (except for the netbird domains) are replaced with randomly generated strings ending in ".domain". Anonymized domains are consistent across all files in the bundle.
|
|
Reoccuring domain names are replaced with the same anonymized domain.
|
|
At the strict level, the peer name labels in front of netbird domains are anonymized as well.
|
|
|
|
Sync Response
|
|
The network_map.json file contains the following anonymized information:
|
|
- Peer configurations (addresses, FQDNs, DNS settings)
|
|
- Remote and offline peer information (allowed IPs, FQDNs)
|
|
- Routes (network ranges, associated domains)
|
|
- DNS configuration (nameservers, domains, custom zones)
|
|
- Firewall rules (peer IPs, source/destination ranges)
|
|
|
|
SSH keys in the sync response are replaced with a placeholder value. All IP addresses and domains in the sync response follow the same anonymization rules as described above.
|
|
|
|
State File
|
|
The state.json file contains anonymized internal state information of the NetBird client, including:
|
|
- DNS settings and configuration
|
|
- Firewall rules
|
|
- Exclusion routes
|
|
- Route selection
|
|
- Other internal states that may be present
|
|
|
|
The state file follows the same anonymization rules as other files:
|
|
- IP addresses (both individual and CIDR ranges) are anonymized while preserving their structure
|
|
- Domain names are consistently anonymized
|
|
- Technical identifiers and non-sensitive data remain unchanged
|
|
|
|
Mutex, Goroutines, Block, and Heap Profiling Files
|
|
The goroutine, block, mutex, and heap profiling files contain process information that might help the NetBird team diagnose performance or memory issues. The information in these files doesn't contain personal data.
|
|
You can check each using the following go command:
|
|
|
|
go tool pprof -http=:8088 <profile_name>.prof
|
|
|
|
For example, to view the heap profile:
|
|
go tool pprof -http=:8088 heap.prof
|
|
|
|
This will open a web browser tab with the profiling information.
|
|
|
|
Stack Trace
|
|
The stack_trace.txt file contains a complete snapshot of all goroutine stack traces at the time the debug bundle was created.
|
|
|
|
Routes
|
|
The routes.txt file contains detailed routing table information in a tabular format:
|
|
|
|
- Destination: Network prefix (IP_ADDRESS/PREFIX_LENGTH)
|
|
- Gateway: Next hop IP address (or "-" if direct)
|
|
- Interface: Network interface name
|
|
- Metric: Route priority/metric (lower values preferred)
|
|
- Protocol: Routing protocol (kernel, static, dhcp, etc.)
|
|
- Scope: Route scope (global, link, host, etc.)
|
|
- Type: Route type (unicast, local, broadcast, etc.)
|
|
- Table: Routing table name (main, local, netbird, etc.)
|
|
|
|
The table format provides a comprehensive view of the system's routing configuration, including information from multiple routing tables on Linux systems. This is valuable for troubleshooting routing issues and understanding traffic flow.
|
|
|
|
For anonymized routes, IP addresses are replaced as described above. The prefix length remains unchanged. Note that for prefixes, the anonymized IP might not be a network address, but the prefix length is still correct. Interface names are anonymized using string anonymization.
|
|
|
|
Resolved Domains
|
|
The resolved_domains.txt file contains information about domain names that have been resolved to IP addresses by NetBird's DNS resolver. This includes:
|
|
- Original domain patterns that were configured for routing
|
|
- Resolved domain names that matched those patterns
|
|
- IP address prefixes that were resolved for each domain
|
|
- Parent domain associations showing which original pattern each resolved domain belongs to
|
|
|
|
All domain names and IP addresses in this file follow the same anonymization rules as described above. This information is valuable for troubleshooting DNS resolution and routing issues.
|
|
|
|
Network Interfaces
|
|
The interfaces.txt file contains information about network interfaces, including:
|
|
- Interface name
|
|
- Interface index
|
|
- MTU (Maximum Transmission Unit)
|
|
- Flags
|
|
- IP addresses associated with each interface
|
|
|
|
The IP addresses in the interfaces file are anonymized using the same process as described above. Interface names, indexes, MTUs, and flags are not anonymized.
|
|
|
|
Configuration
|
|
The config.txt file contains anonymized configuration information of the NetBird client. Sensitive information such as private keys and SSH keys are excluded. The following fields are anonymized:
|
|
- ManagementURL
|
|
- AdminURL
|
|
- NATExternalIPs
|
|
- CustomDNSAddress
|
|
|
|
Other non-sensitive configuration options are included without anonymization.
|
|
|
|
Firewall Rules (Linux only)
|
|
The bundle includes the following firewall-related files:
|
|
|
|
iptables.txt:
|
|
- IPv4 iptables ruleset with packet counters using 'iptables-save' and 'iptables -v -n -L'
|
|
- Includes all tables (filter, nat, mangle, raw, security)
|
|
- Shows packet and byte counters for each rule
|
|
- All IP addresses are anonymized
|
|
- Chain names, table names, and other non-sensitive information remain unchanged
|
|
|
|
ip6tables.txt:
|
|
- IPv6 ip6tables ruleset with packet counters using 'ip6tables-save' and 'ip6tables -v -n -L'
|
|
- Same table coverage and anonymization as iptables.txt
|
|
- Omitted when ip6tables is not installed or no IPv6 rules are present
|
|
|
|
ipset.txt:
|
|
- Output of 'ipset list' (family-agnostic)
|
|
- IP addresses are anonymized; set names and types remain unchanged
|
|
|
|
nftables.txt:
|
|
- Complete nftables ruleset across all families (ip, ip6, inet, arp, bridge, netdev) via 'nft -a list ruleset'
|
|
- Includes rule handle numbers and packet counters
|
|
- All IP addresses are anonymized; chain/table names remain unchanged
|
|
|
|
sysctls.txt:
|
|
- Forwarding (IPv4 + IPv6, global and per-interface), reverse-path filter, source-validation, conntrack accounting, and TCP-related sysctls that netbird may read or modify
|
|
- Per-interface keys are enumerated from /proc/sys/net/ipv{4,6}/conf
|
|
- Interface names anonymized when --anonymize is set
|
|
|
|
IP Rules (Linux only)
|
|
The ip_rules.txt file contains detailed IP routing rule information:
|
|
|
|
- Priority: Rule priority number (lower values processed first)
|
|
- From: Source IP prefix or "all" if unspecified
|
|
- To: Destination IP prefix or "all" if unspecified
|
|
- IIF: Input interface name or "-" if unspecified
|
|
- OIF: Output interface name or "-" if unspecified
|
|
- Table: Target routing table name (main, local, netbird, etc.)
|
|
- Action: Rule action (lookup, goto, blackhole, etc.)
|
|
- Mark: Firewall mark value in hex format or "-" if unspecified
|
|
|
|
The table format provides comprehensive visibility into the IP routing decision process, including how traffic is directed to different routing tables based on various criteria. This is valuable for troubleshooting advanced routing configurations and policy-based routing.
|
|
|
|
For anonymized rules, IP addresses and prefixes are replaced as described above. Interface names are anonymized using string anonymization. Table names, actions, and other non-sensitive information remain unchanged.
|
|
|
|
DNS Configuration
|
|
The debug bundle includes platform-specific DNS configuration files:
|
|
|
|
resolv.conf (Unix systems):
|
|
- Contains DNS resolver configuration from /etc/resolv.conf
|
|
- Includes nameserver entries, search domains, and resolver options
|
|
- All IP addresses and domain names are anonymized following the same rules as other files
|
|
|
|
scutil_dns.txt (macOS only):
|
|
- Contains detailed DNS configuration from scutil --dns
|
|
- Shows DNS configuration for all network interfaces
|
|
- Includes search domains, nameservers, and DNS resolver settings
|
|
- All IP addresses and domain names are anonymized
|
|
|
|
dns_windows.txt (Windows only):
|
|
- Lists the NRPT rules of both policy stores, the local one and the group policy one, marking the rules the client created
|
|
- Follows them with the policy table the resolver has loaded, which differs from the rules while a change has not been picked up yet
|
|
- Includes the DNS client group policy, the global TCP/IP and Dnscache parameters, and the DNS values of every interface that has any
|
|
- Ends with the resolver configuration in effect per adapter, from GetAdaptersAddresses
|
|
- All IP addresses and domain names are anonymized
|
|
`
|
|
|
|
const (
|
|
clientLogFile = "client.log"
|
|
errorLogFile = "netbird.err"
|
|
stdoutLogFile = "netbird.out"
|
|
|
|
// Rotated-log glob prefixes (base log name without extension) passed to
|
|
// addRotatedLogFiles. The daemon's own log and the GUI log live in the same
|
|
// dir, so the prefixes must be disjoint to keep their rotated siblings apart.
|
|
clientLogPrefix = "client"
|
|
uiLogPrefix = "gui-client"
|
|
|
|
darwinErrorLogPath = "/var/log/netbird.out.log"
|
|
darwinStdoutLogPath = "/var/log/netbird.err.log"
|
|
)
|
|
|
|
// MetricsExporter is an interface for exporting metrics
|
|
type MetricsExporter interface {
|
|
Export(w io.Writer) error
|
|
}
|
|
|
|
// LogOpener opens a log file for inclusion in the bundle. It exists so that log
|
|
// files whose path was supplied by an IPC caller can be opened under a check
|
|
// the daemon defines, instead of being opened with the daemon's privileges
|
|
// unconditionally.
|
|
type LogOpener func(path string) (*os.File, error)
|
|
|
|
func openLogFile(path string) (*os.File, error) {
|
|
f, err := os.Open(path)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("open %s: %w", path, err)
|
|
}
|
|
return f, nil
|
|
}
|
|
|
|
type BundleGenerator struct {
|
|
anonymizer *anonymize.Anonymizer
|
|
|
|
// deps
|
|
internalConfig *profilemanager.Config
|
|
statusRecorder *peer.Status
|
|
syncResponse *mgmProto.SyncResponse
|
|
logPath string
|
|
uiLogPath string
|
|
uiLogOpener LogOpener
|
|
tempDir string
|
|
statePath string
|
|
cpuProfile []byte
|
|
capturePath string
|
|
refreshStatus func() // Optional callback to refresh status before bundle generation
|
|
clientMetrics MetricsExporter
|
|
daemonVersion string
|
|
cliVersion string
|
|
|
|
anonymize bool
|
|
anonymizeLevel anonymize.Level
|
|
includeSystemInfo bool
|
|
logFileCount uint32
|
|
|
|
archive *zip.Writer
|
|
}
|
|
|
|
type BundleConfig struct {
|
|
Anonymize bool
|
|
// AnonymizeLevel selects how much the anonymizer redacts.
|
|
// anonymize.LevelStrict implies Anonymize.
|
|
AnonymizeLevel anonymize.Level
|
|
IncludeSystemInfo bool
|
|
LogFileCount uint32
|
|
}
|
|
|
|
type GeneratorDependencies struct {
|
|
InternalConfig *profilemanager.Config
|
|
StatusRecorder *peer.Status
|
|
SyncResponse *mgmProto.SyncResponse
|
|
LogPath string
|
|
UILogPath string // Absolute path to the desktop UI's gui-client.log, reported via RegisterUILog. Empty if no UI registered one.
|
|
// UILogOpener opens the UI log and its rotated siblings. The path comes from
|
|
// a local IPC caller, so the daemon must not open it with plain os.Open: the
|
|
// opener is where the caller's right to that file is enforced. Defaults to
|
|
// os.Open, which is only correct where the path is not caller-supplied
|
|
// (mobile).
|
|
UILogOpener LogOpener
|
|
TempDir string // Directory for temporary bundle zip files. If empty, os.TempDir() is used.
|
|
StatePath string // Path to the state file. If empty, the ServiceManager default path is used.
|
|
CPUProfile []byte
|
|
CapturePath string
|
|
RefreshStatus func()
|
|
ClientMetrics MetricsExporter
|
|
DaemonVersion string
|
|
CliVersion string
|
|
}
|
|
|
|
func NewBundleGenerator(deps GeneratorDependencies, cfg BundleConfig) *BundleGenerator {
|
|
// Default to 1 log file for backward compatibility when 0 is provided
|
|
logFileCount := cfg.LogFileCount
|
|
if logFileCount == 0 {
|
|
logFileCount = 1
|
|
}
|
|
|
|
uiLogOpener := deps.UILogOpener
|
|
if uiLogOpener == nil {
|
|
uiLogOpener = openLogFile
|
|
}
|
|
|
|
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
|
|
anonymizer.SetLevel(cfg.AnonymizeLevel)
|
|
|
|
return &BundleGenerator{
|
|
anonymizer: anonymizer,
|
|
|
|
internalConfig: deps.InternalConfig,
|
|
statusRecorder: deps.StatusRecorder,
|
|
syncResponse: deps.SyncResponse,
|
|
logPath: deps.LogPath,
|
|
uiLogPath: deps.UILogPath,
|
|
uiLogOpener: uiLogOpener,
|
|
tempDir: deps.TempDir,
|
|
statePath: deps.StatePath,
|
|
cpuProfile: deps.CPUProfile,
|
|
capturePath: deps.CapturePath,
|
|
refreshStatus: deps.RefreshStatus,
|
|
clientMetrics: deps.ClientMetrics,
|
|
daemonVersion: deps.DaemonVersion,
|
|
cliVersion: deps.CliVersion,
|
|
|
|
anonymize: cfg.Anonymize || cfg.AnonymizeLevel >= anonymize.LevelStrict,
|
|
anonymizeLevel: cfg.AnonymizeLevel,
|
|
includeSystemInfo: cfg.IncludeSystemInfo,
|
|
logFileCount: logFileCount,
|
|
}
|
|
}
|
|
|
|
// bundleFilePattern names the bundle zips Generate creates in tempDir; the
|
|
// asterisk is filled in by os.CreateTemp.
|
|
const bundleFilePattern = "netbird.debug.*.zip"
|
|
|
|
const exportedBundlePrefix = "netbird.debug-file."
|
|
|
|
const exportedBundleMaxAge = 24 * time.Hour
|
|
|
|
// RemoveStaleBundles deletes bundle zips that an interrupted generation or
|
|
// upload left behind in dir. Only files older than maxAge go, so a bundle that
|
|
// another caller is still writing or uploading in the same directory survives.
|
|
// Exported bundles are kept for exportedBundleMaxAge instead.
|
|
func RemoveStaleBundles(dir string, maxAge time.Duration) {
|
|
removeStaleFiles(dir, bundleFilePattern, maxAge)
|
|
removeStaleFiles(dir, exportedBundlePrefix+"*.zip", exportedBundleMaxAge)
|
|
}
|
|
|
|
// ExportBundle renames a generated bundle out of the RemoveStaleBundles pattern
|
|
// and returns the new path. The caller owns the file from then on; an export
|
|
// abandoned for longer than exportedBundleMaxAge is removed by RemoveStaleBundles.
|
|
func ExportBundle(path string) (string, error) {
|
|
base := strings.TrimPrefix(filepath.Base(path), strings.SplitN(bundleFilePattern, "*", 2)[0])
|
|
exported := filepath.Join(filepath.Dir(path), exportedBundlePrefix+base)
|
|
if err := os.Rename(path, exported); err != nil {
|
|
return "", fmt.Errorf("export debug bundle: %w", err)
|
|
}
|
|
return exported, nil
|
|
}
|
|
|
|
// Generate creates a debug bundle and returns the location.
|
|
func (g *BundleGenerator) Generate() (resp string, err error) {
|
|
bundlePath, err := os.CreateTemp(g.tempDir, bundleFilePattern)
|
|
if err != nil {
|
|
return "", fmt.Errorf("create zip file: %w", err)
|
|
}
|
|
defer func() {
|
|
if closeErr := bundlePath.Close(); closeErr != nil && err == nil {
|
|
err = fmt.Errorf("close zip file: %w", closeErr)
|
|
}
|
|
|
|
if err != nil {
|
|
if removeErr := os.Remove(bundlePath.Name()); removeErr != nil {
|
|
log.Errorf("Failed to remove zip file: %v", removeErr)
|
|
}
|
|
}
|
|
}()
|
|
|
|
g.archive = zip.NewWriter(bundlePath)
|
|
|
|
if err := g.createArchive(); err != nil {
|
|
return "", err
|
|
}
|
|
|
|
if err := g.archive.Close(); err != nil {
|
|
return "", fmt.Errorf("close archive writer: %w", err)
|
|
}
|
|
|
|
return bundlePath.Name(), nil
|
|
}
|
|
|
|
func (g *BundleGenerator) createArchive() error {
|
|
if err := g.addReadme(); err != nil {
|
|
return fmt.Errorf("add readme: %w", err)
|
|
}
|
|
|
|
if err := g.addStatus(); err != nil {
|
|
return fmt.Errorf("add status: %w", err)
|
|
}
|
|
|
|
if err := g.addConfig(); err != nil {
|
|
log.Errorf("failed to add config to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addResolvedDomains(); err != nil {
|
|
log.Errorf("failed to add resolved domains to debug bundle: %v", err)
|
|
}
|
|
|
|
if g.includeSystemInfo {
|
|
g.addSystemInfo()
|
|
}
|
|
|
|
if err := g.addProf(); err != nil {
|
|
log.Errorf("failed to add profiles to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addCPUProfile(); err != nil {
|
|
log.Errorf("failed to add CPU profile to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addCaptureFile(); err != nil {
|
|
log.Errorf("failed to add capture file to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addStackTrace(); err != nil {
|
|
log.Errorf("failed to add stack trace to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addSyncResponse(); err != nil {
|
|
return fmt.Errorf("add sync response: %w", err)
|
|
}
|
|
|
|
if err := g.addStateFile(); err != nil {
|
|
log.Errorf("failed to add state file to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addCorruptedStateFiles(); err != nil {
|
|
log.Errorf("failed to add corrupted state files to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addServiceParams(); err != nil {
|
|
log.Errorf("failed to add service params to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addMetrics(); err != nil {
|
|
log.Errorf("failed to add metrics to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addWgShow(); err != nil {
|
|
log.Errorf("failed to add wg show output: %v", err)
|
|
}
|
|
|
|
if err := g.addPlatformLog(); err != nil {
|
|
log.Errorf("failed to add logs to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addUILog(); err != nil {
|
|
log.Errorf("failed to add UI log to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addUpdateLogs(); err != nil {
|
|
log.Errorf("failed to add updater logs: %v", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addSystemInfo() {
|
|
if err := g.addRoutes(); err != nil {
|
|
log.Errorf("failed to add routes to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addInterfaces(); err != nil {
|
|
log.Errorf("failed to add interfaces to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addIPRules(); err != nil {
|
|
log.Errorf("failed to add IP rules to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addFirewallRules(); err != nil {
|
|
log.Errorf("failed to add firewall rules to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addSysctls(); err != nil {
|
|
log.Errorf("failed to add sysctls to debug bundle: %v", err)
|
|
}
|
|
|
|
if err := g.addDNSInfo(); err != nil {
|
|
log.Errorf("failed to add DNS info to debug bundle: %v", err)
|
|
}
|
|
}
|
|
|
|
func (g *BundleGenerator) addReadme() error {
|
|
level := "none (anonymization disabled)"
|
|
if g.anonymize {
|
|
level = g.anonymizeLevel.String()
|
|
}
|
|
header := fmt.Sprintf("Netbird debug bundle\nAnonymization level applied to this bundle: %s\n", level)
|
|
|
|
readmeReader := strings.NewReader(header + readmeContent)
|
|
if err := g.addFileToZip(readmeReader, "README.txt"); err != nil {
|
|
return fmt.Errorf("add README file to zip: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addStatus() error {
|
|
if g.statusRecorder != nil {
|
|
pm := profilemanager.NewProfileManager()
|
|
var profName string
|
|
if activeProf, err := pm.GetActiveProfile(); err == nil {
|
|
profName = activeProf.Name
|
|
}
|
|
|
|
if g.refreshStatus != nil {
|
|
g.refreshStatus()
|
|
}
|
|
|
|
fullStatus := g.statusRecorder.GetFullStatus()
|
|
protoFullStatus := nbstatus.ToProtoFullStatus(fullStatus)
|
|
overview := nbstatus.ConvertToStatusOutputOverview(protoFullStatus, nbstatus.ConvertOptions{
|
|
Anonymize: g.anonymize,
|
|
AnonymizeLevel: g.anonymizeLevel,
|
|
ProfileName: profName,
|
|
DaemonVersion: g.daemonVersion,
|
|
})
|
|
overview.CliVersion = g.cliVersion
|
|
statusOutput := overview.FullDetailSummary()
|
|
|
|
statusReader := strings.NewReader(statusOutput)
|
|
if err := g.addFileToZip(statusReader, "status.txt"); err != nil {
|
|
return fmt.Errorf("add status file to zip: %w", err)
|
|
}
|
|
seedFromStatus(g.anonymizer, &fullStatus)
|
|
} else {
|
|
log.Debugf("no status recorder available for seeding")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addConfig() error {
|
|
if g.internalConfig == nil {
|
|
log.Debug("skipping empty config in debug bundle")
|
|
return nil
|
|
}
|
|
|
|
var configContent strings.Builder
|
|
g.addCommonConfigFields(&configContent)
|
|
|
|
if g.anonymize {
|
|
if g.internalConfig.ManagementURL != nil {
|
|
configContent.WriteString(fmt.Sprintf("ManagementURL: %s\n", g.anonymizer.AnonymizeURI(g.internalConfig.ManagementURL.String())))
|
|
}
|
|
if g.internalConfig.AdminURL != nil {
|
|
configContent.WriteString(fmt.Sprintf("AdminURL: %s\n", g.anonymizer.AnonymizeURI(g.internalConfig.AdminURL.String())))
|
|
}
|
|
configContent.WriteString(fmt.Sprintf("NATExternalIPs: %v\n", anonymizeNATExternalIPs(g.internalConfig.NATExternalIPs, g.anonymizer)))
|
|
if g.internalConfig.CustomDNSAddress != "" {
|
|
configContent.WriteString(fmt.Sprintf("CustomDNSAddress: %s\n", g.anonymizer.AnonymizeString(g.internalConfig.CustomDNSAddress)))
|
|
}
|
|
} else {
|
|
if g.internalConfig.ManagementURL != nil {
|
|
configContent.WriteString(fmt.Sprintf("ManagementURL: %s\n", g.internalConfig.ManagementURL.String()))
|
|
}
|
|
if g.internalConfig.AdminURL != nil {
|
|
configContent.WriteString(fmt.Sprintf("AdminURL: %s\n", g.internalConfig.AdminURL.String()))
|
|
}
|
|
configContent.WriteString(fmt.Sprintf("NATExternalIPs: %v\n", g.internalConfig.NATExternalIPs))
|
|
if g.internalConfig.CustomDNSAddress != "" {
|
|
configContent.WriteString(fmt.Sprintf("CustomDNSAddress: %s\n", g.internalConfig.CustomDNSAddress))
|
|
}
|
|
}
|
|
|
|
// Surface the set of MDM-enforced keys so a support engineer reading
|
|
// the bundle can tell which field values are user-set vs MDM-overridden.
|
|
// Same semantics as the mDMManagedFields list returned by the
|
|
// GetConfig RPC consumed by `netbird debug config`.
|
|
if managed := g.internalConfig.Policy().ManagedKeys(); len(managed) > 0 {
|
|
configContent.WriteString(fmt.Sprintf("MDMManagedFields: %v\n", managed))
|
|
}
|
|
|
|
configReader := strings.NewReader(configContent.String())
|
|
if err := g.addFileToZip(configReader, "config.txt"); err != nil {
|
|
return fmt.Errorf("add config file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
const (
|
|
serviceParamsFile = "service.json"
|
|
serviceParamsBundle = "service_params.json"
|
|
maskedValue = "***"
|
|
envVarPrefix = "NB_"
|
|
jsonKeyManagementURL = "management_url"
|
|
jsonKeyServiceEnv = "service_env_vars"
|
|
)
|
|
|
|
var sensitiveEnvSubstrings = []string{"key", "token", "secret", "password", "credential", "pin", "pkcs11"}
|
|
|
|
// addServiceParams reads the service.json file and adds a sanitized version to the bundle.
|
|
// Non-NB_ env vars and vars with sensitive names are masked. Other NB_ values are anonymized.
|
|
func (g *BundleGenerator) addServiceParams() error {
|
|
path := filepath.Join(configs.StateDir, serviceParamsFile)
|
|
|
|
data, err := os.ReadFile(path)
|
|
if err != nil {
|
|
if os.IsNotExist(err) {
|
|
return nil
|
|
}
|
|
return fmt.Errorf("read service params: %w", err)
|
|
}
|
|
|
|
var params map[string]any
|
|
if err := json.Unmarshal(data, ¶ms); err != nil {
|
|
return fmt.Errorf("parse service params: %w", err)
|
|
}
|
|
|
|
if g.anonymize {
|
|
if mgmtURL, ok := params[jsonKeyManagementURL].(string); ok && mgmtURL != "" {
|
|
params[jsonKeyManagementURL] = g.anonymizer.AnonymizeURI(mgmtURL)
|
|
}
|
|
}
|
|
|
|
g.sanitizeServiceEnvVars(params)
|
|
|
|
sanitizedData, err := json.MarshalIndent(params, "", " ")
|
|
if err != nil {
|
|
return fmt.Errorf("marshal sanitized service params: %w", err)
|
|
}
|
|
|
|
if err := g.addFileToZip(bytes.NewReader(sanitizedData), serviceParamsBundle); err != nil {
|
|
return fmt.Errorf("add service params to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// sanitizeServiceEnvVars masks or anonymizes env var values in service params.
|
|
// Non-NB_ vars and vars with sensitive names (key, token, etc.) are fully masked.
|
|
// Other NB_ var values are passed through the anonymizer when anonymization is enabled.
|
|
func (g *BundleGenerator) sanitizeServiceEnvVars(params map[string]any) {
|
|
envVars, ok := params[jsonKeyServiceEnv].(map[string]any)
|
|
if !ok {
|
|
return
|
|
}
|
|
|
|
sanitized := make(map[string]any, len(envVars))
|
|
for k, v := range envVars {
|
|
val, _ := v.(string)
|
|
switch {
|
|
case !strings.HasPrefix(k, envVarPrefix) || isSensitiveEnvVar(k):
|
|
sanitized[k] = maskedValue
|
|
case g.anonymize:
|
|
sanitized[k] = g.anonymizer.AnonymizeString(val)
|
|
default:
|
|
sanitized[k] = val
|
|
}
|
|
}
|
|
params[jsonKeyServiceEnv] = sanitized
|
|
}
|
|
|
|
// isSensitiveEnvVar returns true for env var names that may contain secrets.
|
|
func isSensitiveEnvVar(key string) bool {
|
|
lower := strings.ToLower(key)
|
|
for _, s := range sensitiveEnvSubstrings {
|
|
if strings.Contains(lower, s) {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|
|
|
|
func (g *BundleGenerator) addCommonConfigFields(configContent *strings.Builder) {
|
|
configContent.WriteString("NetBird Client Configuration:\n\n")
|
|
|
|
if key, err := wgtypes.ParseKey(g.internalConfig.PrivateKey); err == nil {
|
|
configContent.WriteString(fmt.Sprintf("PublicKey: %s\n", g.anonymizer.AnonymizeWGKey(key.PublicKey().String())))
|
|
}
|
|
configContent.WriteString(fmt.Sprintf("WgIface: %s\n", g.internalConfig.WgIface))
|
|
configContent.WriteString(fmt.Sprintf("WgPort: %d\n", g.internalConfig.WgPort))
|
|
if g.internalConfig.NetworkMonitor != nil {
|
|
configContent.WriteString(fmt.Sprintf("NetworkMonitor: %v\n", *g.internalConfig.NetworkMonitor))
|
|
}
|
|
configContent.WriteString(fmt.Sprintf("IFaceBlackList: %v\n", g.internalConfig.IFaceBlackList))
|
|
configContent.WriteString(fmt.Sprintf("DisableIPv6Discovery: %v\n", g.internalConfig.DisableIPv6Discovery))
|
|
configContent.WriteString(fmt.Sprintf("RosenpassEnabled: %v\n", g.internalConfig.RosenpassEnabled))
|
|
configContent.WriteString(fmt.Sprintf("RosenpassPermissive: %v\n", g.internalConfig.RosenpassPermissive))
|
|
if g.internalConfig.ServerSSHAllowed != nil {
|
|
configContent.WriteString(fmt.Sprintf("ServerSSHAllowed: %v\n", *g.internalConfig.ServerSSHAllowed))
|
|
}
|
|
if g.internalConfig.RemoteJobsAllowed != nil {
|
|
configContent.WriteString(fmt.Sprintf("RemoteJobsAllowed: %v\n", *g.internalConfig.RemoteJobsAllowed))
|
|
}
|
|
if g.internalConfig.EnableSSHRoot != nil {
|
|
configContent.WriteString(fmt.Sprintf("EnableSSHRoot: %v\n", *g.internalConfig.EnableSSHRoot))
|
|
}
|
|
if g.internalConfig.EnableSSHSFTP != nil {
|
|
configContent.WriteString(fmt.Sprintf("EnableSSHSFTP: %v\n", *g.internalConfig.EnableSSHSFTP))
|
|
}
|
|
if g.internalConfig.EnableSSHLocalPortForwarding != nil {
|
|
configContent.WriteString(fmt.Sprintf("EnableSSHLocalPortForwarding: %v\n", *g.internalConfig.EnableSSHLocalPortForwarding))
|
|
}
|
|
if g.internalConfig.EnableSSHRemotePortForwarding != nil {
|
|
configContent.WriteString(fmt.Sprintf("EnableSSHRemotePortForwarding: %v\n", *g.internalConfig.EnableSSHRemotePortForwarding))
|
|
}
|
|
if g.internalConfig.DisableSSHAuth != nil {
|
|
configContent.WriteString(fmt.Sprintf("DisableSSHAuth: %v\n", *g.internalConfig.DisableSSHAuth))
|
|
}
|
|
if g.internalConfig.SSHJWTCacheTTL != nil {
|
|
configContent.WriteString(fmt.Sprintf("SSHJWTCacheTTL: %d\n", *g.internalConfig.SSHJWTCacheTTL))
|
|
}
|
|
|
|
configContent.WriteString(fmt.Sprintf("DisableClientRoutes: %v\n", g.internalConfig.DisableClientRoutes))
|
|
configContent.WriteString(fmt.Sprintf("DisableServerRoutes: %v\n", g.internalConfig.DisableServerRoutes))
|
|
configContent.WriteString(fmt.Sprintf("DisableDNS: %v\n", g.internalConfig.DisableDNS))
|
|
configContent.WriteString(fmt.Sprintf("DisableFirewall: %v\n", g.internalConfig.DisableFirewall))
|
|
configContent.WriteString(fmt.Sprintf("BlockLANAccess: %v\n", g.internalConfig.BlockLANAccess))
|
|
configContent.WriteString(fmt.Sprintf("BlockInbound: %v\n", g.internalConfig.BlockInbound))
|
|
configContent.WriteString(fmt.Sprintf("DisableIPv6: %v\n", g.internalConfig.DisableIPv6))
|
|
configContent.WriteString(fmt.Sprintf("LocalMetricsEnabled: %v\n", g.internalConfig.LocalMetricsEnabled))
|
|
configContent.WriteString(fmt.Sprintf("LocalMetricsAddress: %s\n", g.internalConfig.LocalMetricsAddress))
|
|
configContent.WriteString(fmt.Sprintf("SyncMessageVersion: %v\n", g.internalConfig.SyncMessageVersion))
|
|
|
|
if g.internalConfig.DisableNotifications != nil {
|
|
configContent.WriteString(fmt.Sprintf("DisableNotifications: %v\n", *g.internalConfig.DisableNotifications))
|
|
}
|
|
|
|
configContent.WriteString(fmt.Sprintf("DNSLabels: %v\n", g.internalConfig.DNSLabels))
|
|
|
|
configContent.WriteString(fmt.Sprintf("DisableAutoConnect: %v\n", g.internalConfig.DisableAutoConnect))
|
|
|
|
configContent.WriteString(fmt.Sprintf("DNSRouteInterval: %s\n", g.internalConfig.DNSRouteInterval))
|
|
|
|
if g.internalConfig.ClientCertPath != "" {
|
|
configContent.WriteString(fmt.Sprintf("ClientCertPath: %s\n", g.internalConfig.ClientCertPath))
|
|
}
|
|
if g.internalConfig.ClientCertKeyPath != "" {
|
|
configContent.WriteString(fmt.Sprintf("ClientCertKeyPath: %s\n", g.internalConfig.ClientCertKeyPath))
|
|
}
|
|
|
|
configContent.WriteString(fmt.Sprintf("LazyConnection: %q\n", g.internalConfig.LazyConnection))
|
|
configContent.WriteString(fmt.Sprintf("MTU: %d\n", g.internalConfig.MTU))
|
|
}
|
|
|
|
func (g *BundleGenerator) addProf() (err error) {
|
|
defer func() {
|
|
if r := recover(); r != nil {
|
|
err = fmt.Errorf("panic while profiling: %v", r)
|
|
}
|
|
}()
|
|
|
|
runtime.SetBlockProfileRate(1)
|
|
_ = runtime.SetMutexProfileFraction(1)
|
|
defer runtime.SetBlockProfileRate(0)
|
|
defer runtime.SetMutexProfileFraction(0)
|
|
|
|
time.Sleep(5 * time.Second)
|
|
|
|
for _, profile := range []string{"goroutine", "block", "mutex", "heap", "allocs", "threadcreate"} {
|
|
var buff []byte
|
|
myBuff := bytes.NewBuffer(buff)
|
|
err := pprof.Lookup(profile).WriteTo(myBuff, 0)
|
|
if err != nil {
|
|
return fmt.Errorf("write %s profile: %w", profile, err)
|
|
}
|
|
|
|
if err := g.addFileToZip(myBuff, profile+".prof"); err != nil {
|
|
return fmt.Errorf("add %s file to zip: %w", profile, err)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addCPUProfile() error {
|
|
if len(g.cpuProfile) == 0 {
|
|
return nil
|
|
}
|
|
|
|
reader := bytes.NewReader(g.cpuProfile)
|
|
if err := g.addFileToZip(reader, "cpu.prof"); err != nil {
|
|
return fmt.Errorf("add CPU profile to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addCaptureFile() error {
|
|
if g.capturePath == "" {
|
|
return nil
|
|
}
|
|
|
|
if g.anonymize {
|
|
log.Info("skipping capture file in anonymized bundle (contains raw packet data)")
|
|
return nil
|
|
}
|
|
|
|
f, err := os.Open(g.capturePath)
|
|
if err != nil {
|
|
return fmt.Errorf("open capture file: %w", err)
|
|
}
|
|
defer f.Close()
|
|
|
|
if err := g.addFileToZip(f, "capture.pcap"); err != nil {
|
|
return fmt.Errorf("add capture file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addStackTrace() error {
|
|
buf := make([]byte, 5242880) // 5 MB buffer
|
|
n := runtime.Stack(buf, true)
|
|
|
|
stackTrace := bytes.NewReader(buf[:n])
|
|
if err := g.addFileToZip(stackTrace, "stack_trace.txt"); err != nil {
|
|
return fmt.Errorf("add stack trace file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addInterfaces() error {
|
|
interfaces, err := net.Interfaces()
|
|
if err != nil {
|
|
return fmt.Errorf("get interfaces: %w", err)
|
|
}
|
|
|
|
interfacesContent := formatInterfaces(interfaces, g.anonymize, g.anonymizer)
|
|
interfacesReader := strings.NewReader(interfacesContent)
|
|
if err := g.addFileToZip(interfacesReader, "interfaces.txt"); err != nil {
|
|
return fmt.Errorf("add interfaces file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addResolvedDomains() error {
|
|
if g.statusRecorder == nil {
|
|
log.Debugf("skipping resolved domains in debug bundle: no status recorder")
|
|
return nil
|
|
}
|
|
|
|
resolvedDomains := g.statusRecorder.GetResolvedDomainsStates()
|
|
if len(resolvedDomains) == 0 {
|
|
log.Debugf("skipping resolved domains in debug bundle: no resolved domains")
|
|
return nil
|
|
}
|
|
|
|
resolvedDomainsContent := formatResolvedDomains(resolvedDomains, g.anonymize, g.anonymizer)
|
|
resolvedDomainsReader := strings.NewReader(resolvedDomainsContent)
|
|
if err := g.addFileToZip(resolvedDomainsReader, "resolved_domains.txt"); err != nil {
|
|
return fmt.Errorf("add resolved domains file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addSyncResponse() error {
|
|
if g.syncResponse == nil {
|
|
log.Debugf("skipping empty sync response in debug bundle")
|
|
return nil
|
|
}
|
|
|
|
if g.anonymize {
|
|
if err := anonymizeSyncResponse(g.syncResponse, g.anonymizer); err != nil {
|
|
return fmt.Errorf("anonymize sync response: %w", err)
|
|
}
|
|
}
|
|
|
|
options := protojson.MarshalOptions{
|
|
EmitUnpopulated: true,
|
|
UseProtoNames: true,
|
|
Indent: " ",
|
|
AllowPartial: true,
|
|
}
|
|
|
|
g.maskSecrets()
|
|
|
|
jsonBytes, err := options.Marshal(g.syncResponse)
|
|
if err != nil {
|
|
return fmt.Errorf("generate json: %w", err)
|
|
}
|
|
|
|
if err := g.addFileToZip(bytes.NewReader(jsonBytes), "network_map.json"); err != nil {
|
|
return fmt.Errorf("add sync response to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) maskSecrets() {
|
|
if g.syncResponse == nil || g.syncResponse.NetbirdConfig == nil {
|
|
return
|
|
}
|
|
|
|
if g.syncResponse.NetbirdConfig.Flow != nil {
|
|
g.syncResponse.NetbirdConfig.Flow.TokenPayload = maskedValue
|
|
|
|
}
|
|
|
|
if g.syncResponse.NetbirdConfig.Relay != nil {
|
|
g.syncResponse.NetbirdConfig.Relay.TokenPayload = maskedValue
|
|
}
|
|
|
|
for i := range g.syncResponse.NetbirdConfig.Turns {
|
|
if g.syncResponse.NetbirdConfig.Turns[i] != nil {
|
|
g.syncResponse.NetbirdConfig.Turns[i].Password = maskedValue
|
|
}
|
|
}
|
|
}
|
|
|
|
func (g *BundleGenerator) addStateFile() error {
|
|
path := g.statePath
|
|
if path == "" {
|
|
sm := profilemanager.NewServiceManager("")
|
|
path = sm.GetStatePath()
|
|
}
|
|
if path == "" {
|
|
return nil
|
|
}
|
|
|
|
log.Debugf("Adding state file from: %s", path)
|
|
|
|
data, err := os.ReadFile(path)
|
|
if err != nil {
|
|
if errors.Is(err, fs.ErrNotExist) {
|
|
return nil
|
|
}
|
|
return fmt.Errorf("read state file: %w", err)
|
|
}
|
|
|
|
if g.anonymize {
|
|
var rawStates map[string]json.RawMessage
|
|
if err := json.Unmarshal(data, &rawStates); err != nil {
|
|
return fmt.Errorf("unmarshal states: %w", err)
|
|
}
|
|
|
|
if err := anonymizeStateFile(&rawStates, g.anonymizer); err != nil {
|
|
return fmt.Errorf("anonymize state file: %w", err)
|
|
}
|
|
|
|
bs, err := json.MarshalIndent(rawStates, "", " ")
|
|
if err != nil {
|
|
return fmt.Errorf("marshal states: %w", err)
|
|
}
|
|
data = bs
|
|
}
|
|
|
|
if err := g.addFileToZip(bytes.NewReader(data), "state.json"); err != nil {
|
|
return fmt.Errorf("add state file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addUpdateLogs() error {
|
|
inst := installer.New()
|
|
logFiles := inst.LogFiles()
|
|
if len(logFiles) == 0 {
|
|
return nil
|
|
}
|
|
|
|
log.Infof("adding updater logs")
|
|
for _, logFile := range logFiles {
|
|
data, err := os.ReadFile(logFile)
|
|
if err != nil {
|
|
log.Warnf("failed to read update log file %s: %v", logFile, err)
|
|
continue
|
|
}
|
|
|
|
baseName := filepath.Base(logFile)
|
|
data, err = g.anonymizeBytes(data)
|
|
if err != nil {
|
|
log.Warnf("skipping update log file %s: %v", baseName, err)
|
|
continue
|
|
}
|
|
if err := g.addFileToZip(bytes.NewReader(data), filepath.Join("update-logs", baseName)); err != nil {
|
|
return fmt.Errorf("add update log file %s to zip: %w", baseName, err)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addCorruptedStateFiles() error {
|
|
sm := profilemanager.NewServiceManager("")
|
|
pattern := sm.GetStatePath()
|
|
if pattern == "" {
|
|
return nil
|
|
}
|
|
pattern += "*.corrupted.*"
|
|
matches, err := filepath.Glob(pattern)
|
|
if err != nil {
|
|
return fmt.Errorf("find corrupted state files: %w", err)
|
|
}
|
|
|
|
for _, match := range matches {
|
|
data, err := os.ReadFile(match)
|
|
if err != nil {
|
|
log.Warnf("Failed to read corrupted state file %s: %v", match, err)
|
|
continue
|
|
}
|
|
|
|
fileName := filepath.Base(match)
|
|
// Corrupted state files usually fail structured JSON anonymization,
|
|
// so run them through the string anonymizer instead.
|
|
data, err = g.anonymizeBytes(data)
|
|
if err != nil {
|
|
log.Warnf("skipping corrupted state file %s: %v", fileName, err)
|
|
continue
|
|
}
|
|
if err := g.addFileToZip(bytes.NewReader(data), "corrupted_states/"+fileName); err != nil {
|
|
log.Warnf("Failed to add corrupted state file %s to zip: %v", fileName, err)
|
|
continue
|
|
}
|
|
|
|
log.Debugf("Added corrupted state file to debug bundle: %s", fileName)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// anonymizeBytes runs raw file content through the string anonymizer line by
|
|
// line when anonymization is enabled. It errors instead of returning partial
|
|
// content, so a caller never adds an unanonymized fallback to the bundle.
|
|
func (g *BundleGenerator) anonymizeBytes(data []byte) ([]byte, error) {
|
|
if !g.anonymize {
|
|
return data, nil
|
|
}
|
|
|
|
var buf bytes.Buffer
|
|
scanner := bufio.NewScanner(bytes.NewReader(data))
|
|
scanner.Buffer(make([]byte, 1024*1024), 1024*1024)
|
|
for scanner.Scan() {
|
|
buf.WriteString(g.anonymizer.AnonymizeString(scanner.Text()))
|
|
buf.WriteByte('\n')
|
|
}
|
|
if err := scanner.Err(); err != nil {
|
|
return nil, fmt.Errorf("anonymize content: %w", err)
|
|
}
|
|
return buf.Bytes(), nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addMetrics() error {
|
|
if g.clientMetrics == nil {
|
|
log.Debugf("skipping metrics in debug bundle: no metrics collector")
|
|
return nil
|
|
}
|
|
|
|
var buf bytes.Buffer
|
|
if err := g.clientMetrics.Export(&buf); err != nil {
|
|
return fmt.Errorf("export metrics: %w", err)
|
|
}
|
|
|
|
if buf.Len() == 0 {
|
|
log.Debugf("skipping metrics.txt in debug bundle: no metrics data")
|
|
return nil
|
|
}
|
|
|
|
if err := g.addFileToZip(&buf, "metrics.txt"); err != nil {
|
|
return fmt.Errorf("add metrics file to zip: %w", err)
|
|
}
|
|
|
|
log.Debugf("added metrics to debug bundle")
|
|
return nil
|
|
}
|
|
|
|
func (g *BundleGenerator) addLogfile() error {
|
|
if g.logPath == "" {
|
|
log.Debugf("skipping empty log file in debug bundle")
|
|
return nil
|
|
}
|
|
|
|
logDir := filepath.Dir(g.logPath)
|
|
|
|
if err := g.addSingleLogfile(openLogFile, g.logPath, clientLogFile); err != nil {
|
|
return fmt.Errorf("add client log file to zip: %w", err)
|
|
}
|
|
|
|
g.addRotatedLogFiles(openLogFile, logDir, clientLogPrefix)
|
|
|
|
stdErrLogPath := filepath.Join(logDir, errorLogFile)
|
|
stdoutLogPath := filepath.Join(logDir, stdoutLogFile)
|
|
if runtime.GOOS == "darwin" {
|
|
stdErrLogPath = darwinErrorLogPath
|
|
stdoutLogPath = darwinStdoutLogPath
|
|
}
|
|
|
|
if err := g.addSingleLogfile(openLogFile, stdErrLogPath, errorLogFile); err != nil {
|
|
log.Warnf("Failed to add %s to zip: %v", errorLogFile, err)
|
|
}
|
|
|
|
if err := g.addSingleLogfile(openLogFile, stdoutLogPath, stdoutLogFile); err != nil {
|
|
log.Warnf("Failed to add %s to zip: %v", stdoutLogFile, err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// addUILog adds the desktop UI's gui-client.log (and its rotated siblings) to
|
|
// the bundle. The path is reported by the UI via RegisterUILog; empty when no
|
|
// UI registered one (e.g. headless / server). Missing file is non-fatal — the
|
|
// UI only writes it while the daemon is in debug, so it's often absent.
|
|
func (g *BundleGenerator) addUILog() error {
|
|
if g.uiLogPath == "" {
|
|
log.Debugf("no UI log path registered, skipping in debug bundle")
|
|
return nil
|
|
}
|
|
|
|
if err := g.addSingleLogfile(g.uiLogOpener, g.uiLogPath, configs.UILogFile); err != nil {
|
|
return fmt.Errorf("add UI log file to zip: %w", err)
|
|
}
|
|
|
|
g.addRotatedLogFiles(g.uiLogOpener, filepath.Dir(g.uiLogPath), uiLogPrefix)
|
|
|
|
return nil
|
|
}
|
|
|
|
// addSingleLogfile adds a single log file to the archive
|
|
func (g *BundleGenerator) addSingleLogfile(open LogOpener, logPath, targetName string) error {
|
|
logFile, err := open(logPath)
|
|
if err != nil {
|
|
return fmt.Errorf("open log file %s: %w", targetName, err)
|
|
}
|
|
defer func() {
|
|
if err := logFile.Close(); err != nil {
|
|
log.Errorf("failed to close log file %s: %v", targetName, err)
|
|
}
|
|
}()
|
|
|
|
var logReader io.Reader = logFile
|
|
if g.anonymize {
|
|
var writer *io.PipeWriter
|
|
logReader, writer = io.Pipe()
|
|
|
|
go anonymizeLog(logFile, writer, g.anonymizer)
|
|
}
|
|
if err := g.addFileToZip(logReader, targetName); err != nil {
|
|
return fmt.Errorf("add %s to zip: %w", targetName, err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// addSingleLogFileGz adds a single gzipped log file to the archive
|
|
func (g *BundleGenerator) addSingleLogFileGz(open LogOpener, logPath, targetName string) error {
|
|
f, err := open(logPath)
|
|
if err != nil {
|
|
return fmt.Errorf("open gz log file %s: %w", targetName, err)
|
|
}
|
|
defer func() {
|
|
if err := f.Close(); err != nil {
|
|
log.Errorf("failed to close gz file %s: %v", targetName, err)
|
|
}
|
|
}()
|
|
|
|
gzr, err := gzip.NewReader(f)
|
|
if err != nil {
|
|
return fmt.Errorf("create gzip reader: %w", err)
|
|
}
|
|
defer func() {
|
|
if err := gzr.Close(); err != nil {
|
|
log.Errorf("failed to close gzip reader %s: %v", targetName, err)
|
|
}
|
|
}()
|
|
|
|
var logReader io.Reader = gzr
|
|
if g.anonymize {
|
|
var pw *io.PipeWriter
|
|
logReader, pw = io.Pipe()
|
|
go anonymizeLog(gzr, pw, g.anonymizer)
|
|
}
|
|
|
|
var buf bytes.Buffer
|
|
gw := gzip.NewWriter(&buf)
|
|
if _, err := io.Copy(gw, logReader); err != nil {
|
|
return fmt.Errorf("re-gzip: %w", err)
|
|
}
|
|
|
|
if err := gw.Close(); err != nil {
|
|
return fmt.Errorf("close gzip writer: %w", err)
|
|
}
|
|
|
|
if err := g.addFileToZip(&buf, targetName); err != nil {
|
|
return fmt.Errorf("add anonymized gz: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// addRotatedLogFiles adds rotated log files to the bundle based on logFileCount.
|
|
// prefix is the base log name without extension (e.g. "client", "gui-client");
|
|
// the glob matches both files rotated by us and by logrotate on linux.
|
|
func (g *BundleGenerator) addRotatedLogFiles(open LogOpener, logDir, prefix string) {
|
|
if g.logFileCount == 0 {
|
|
return
|
|
}
|
|
|
|
// This pattern matches both logs rotated by us and logrotate on linux
|
|
pattern := filepath.Join(logDir, prefix+"*.log.*")
|
|
files, err := filepath.Glob(pattern)
|
|
if err != nil {
|
|
log.Warnf("failed to glob rotated logs: %v", err)
|
|
return
|
|
}
|
|
|
|
if len(files) == 0 {
|
|
return
|
|
}
|
|
|
|
// sort files by modification time (newest first)
|
|
sort.Slice(files, func(i, j int) bool {
|
|
fi, err := os.Stat(files[i])
|
|
if err != nil {
|
|
log.Warnf("failed to stat rotated log %s: %v", files[i], err)
|
|
return false
|
|
}
|
|
fj, err := os.Stat(files[j])
|
|
if err != nil {
|
|
log.Warnf("failed to stat rotated log %s: %v", files[j], err)
|
|
return false
|
|
}
|
|
return fi.ModTime().After(fj.ModTime())
|
|
})
|
|
|
|
maxFiles := int(g.logFileCount)
|
|
if maxFiles > len(files) {
|
|
maxFiles = len(files)
|
|
}
|
|
|
|
for i := 0; i < maxFiles; i++ {
|
|
name := filepath.Base(files[i])
|
|
if strings.HasSuffix(name, ".gz") {
|
|
err = g.addSingleLogFileGz(open, files[i], name)
|
|
} else {
|
|
err = g.addSingleLogfile(open, files[i], name)
|
|
}
|
|
if err != nil {
|
|
log.Warnf("failed to add rotated log %s: %v", name, err)
|
|
}
|
|
}
|
|
}
|
|
|
|
func (g *BundleGenerator) addFileToZip(reader io.Reader, filename string) error {
|
|
header := &zip.FileHeader{
|
|
Name: filename,
|
|
Method: zip.Deflate,
|
|
Modified: time.Now(),
|
|
|
|
CreatorVersion: 20, // Version 2.0
|
|
ReaderVersion: 20, // Version 2.0
|
|
Flags: 0x800, // UTF-8 filename
|
|
}
|
|
|
|
// If the reader is a file, we can get more accurate information
|
|
if f, ok := reader.(*os.File); ok {
|
|
if stat, err := f.Stat(); err != nil {
|
|
log.Tracef("failed to get file stat for %s: %v", filename, err)
|
|
} else {
|
|
header.Modified = stat.ModTime()
|
|
}
|
|
}
|
|
|
|
writer, err := g.archive.CreateHeader(header)
|
|
if err != nil {
|
|
return fmt.Errorf("create zip file header: %w", err)
|
|
}
|
|
|
|
if _, err := io.Copy(writer, reader); err != nil {
|
|
return fmt.Errorf("write file to zip: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func seedFromStatus(a *anonymize.Anonymizer, status *peer.FullStatus) {
|
|
status.ManagementState.URL = a.AnonymizeURI(status.ManagementState.URL)
|
|
status.SignalState.URL = a.AnonymizeURI(status.SignalState.URL)
|
|
|
|
status.LocalPeerState.FQDN = a.AnonymizeDomain(status.LocalPeerState.FQDN)
|
|
|
|
for _, p := range status.Peers {
|
|
a.AnonymizeDomain(p.FQDN)
|
|
for route := range p.GetRoutes() {
|
|
a.AnonymizeRoute(route)
|
|
}
|
|
}
|
|
|
|
for route := range status.LocalPeerState.Routes {
|
|
a.AnonymizeRoute(route)
|
|
}
|
|
|
|
for _, nsGroup := range status.NSGroupStates {
|
|
for _, domain := range nsGroup.Domains {
|
|
a.AnonymizeDomain(domain)
|
|
}
|
|
}
|
|
|
|
for _, relay := range status.Relays {
|
|
if relay.URI != "" {
|
|
a.AnonymizeURI(relay.URI)
|
|
}
|
|
}
|
|
}
|
|
|
|
func anonymizeLog(reader io.Reader, writer *io.PipeWriter, anonymizer *anonymize.Anonymizer) {
|
|
defer func() {
|
|
// always nil
|
|
_ = writer.Close()
|
|
}()
|
|
|
|
scanner := bufio.NewScanner(reader)
|
|
for scanner.Scan() {
|
|
line := anonymizer.AnonymizeString(scanner.Text())
|
|
if _, err := writer.Write([]byte(line + "\n")); err != nil {
|
|
if err := writer.CloseWithError(fmt.Errorf("anonymize write: %w", err)); err != nil {
|
|
log.Errorf("Failed to close writer: %v", err)
|
|
}
|
|
return
|
|
}
|
|
}
|
|
if err := scanner.Err(); err != nil {
|
|
if err := writer.CloseWithError(fmt.Errorf("anonymize scan: %w", err)); err != nil {
|
|
log.Errorf("Failed to close writer: %v", err)
|
|
}
|
|
return
|
|
}
|
|
}
|
|
|
|
func anonymizeNATExternalIPs(ips []string, anonymizer *anonymize.Anonymizer) []string {
|
|
anonymizedIPs := make([]string, len(ips))
|
|
for i, ip := range ips {
|
|
parts := strings.SplitN(ip, "/", 2)
|
|
|
|
ip1, err := netip.ParseAddr(parts[0])
|
|
if err != nil {
|
|
anonymizedIPs[i] = ip
|
|
continue
|
|
}
|
|
ip1anon := anonymizer.AnonymizeIP(ip1)
|
|
|
|
if len(parts) == 2 {
|
|
ip2, err := netip.ParseAddr(parts[1])
|
|
if err != nil {
|
|
anonymizedIPs[i] = fmt.Sprintf("%s/%s", ip1anon, parts[1])
|
|
} else {
|
|
ip2anon := anonymizer.AnonymizeIP(ip2)
|
|
anonymizedIPs[i] = fmt.Sprintf("%s/%s", ip1anon, ip2anon)
|
|
}
|
|
} else {
|
|
anonymizedIPs[i] = ip1anon.String()
|
|
}
|
|
}
|
|
return anonymizedIPs
|
|
}
|
|
|
|
func anonymizeNetworkMap(networkMap *mgmProto.NetworkMap, anonymizer *anonymize.Anonymizer) error {
|
|
if networkMap.PeerConfig != nil {
|
|
anonymizePeerConfig(networkMap.PeerConfig, anonymizer)
|
|
}
|
|
|
|
for _, p := range networkMap.RemotePeers {
|
|
anonymizeRemotePeer(p, anonymizer)
|
|
}
|
|
|
|
for _, p := range networkMap.OfflinePeers {
|
|
anonymizeRemotePeer(p, anonymizer)
|
|
}
|
|
|
|
for _, r := range networkMap.Routes {
|
|
anonymizeRoute(r, anonymizer)
|
|
}
|
|
|
|
if networkMap.DNSConfig != nil {
|
|
anonymizeDNSConfig(networkMap.DNSConfig, anonymizer)
|
|
}
|
|
|
|
for _, rule := range networkMap.FirewallRules {
|
|
anonymizeFirewallRule(rule, anonymizer)
|
|
}
|
|
|
|
for _, rule := range networkMap.RoutesFirewallRules {
|
|
anonymizeRouteFirewallRule(rule, anonymizer)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func anonymizeNetbirdConfig(config *mgmProto.NetbirdConfig, anonymizer *anonymize.Anonymizer) {
|
|
for _, stun := range config.Stuns {
|
|
if stun.Uri != "" {
|
|
stun.Uri = anonymizer.AnonymizeURI(stun.Uri)
|
|
}
|
|
}
|
|
|
|
for _, turn := range config.Turns {
|
|
if turn.HostConfig != nil && turn.HostConfig.Uri != "" {
|
|
turn.HostConfig.Uri = anonymizer.AnonymizeURI(turn.HostConfig.Uri)
|
|
}
|
|
if turn.User != "" {
|
|
turn.User = "turn-user-placeholder"
|
|
}
|
|
if turn.Password != "" {
|
|
turn.Password = "turn-password-placeholder"
|
|
}
|
|
}
|
|
|
|
if config.Signal != nil && config.Signal.Uri != "" {
|
|
config.Signal.Uri = anonymizer.AnonymizeURI(config.Signal.Uri)
|
|
}
|
|
|
|
if config.Relay != nil {
|
|
for i, url := range config.Relay.Urls {
|
|
config.Relay.Urls[i] = anonymizer.AnonymizeURI(url)
|
|
}
|
|
if config.Relay.TokenPayload != "" {
|
|
config.Relay.TokenPayload = "relay-token-payload-placeholder"
|
|
}
|
|
if config.Relay.TokenSignature != "" {
|
|
config.Relay.TokenSignature = "relay-token-signature-placeholder"
|
|
}
|
|
}
|
|
|
|
if config.Flow != nil {
|
|
if config.Flow.Url != "" {
|
|
config.Flow.Url = anonymizer.AnonymizeURI(config.Flow.Url)
|
|
}
|
|
if config.Flow.TokenPayload != "" {
|
|
config.Flow.TokenPayload = "flow-token-payload-placeholder"
|
|
}
|
|
if config.Flow.TokenSignature != "" {
|
|
config.Flow.TokenSignature = "flow-token-signature-placeholder"
|
|
}
|
|
}
|
|
}
|
|
|
|
func anonymizeSyncResponse(syncResponse *mgmProto.SyncResponse, anonymizer *anonymize.Anonymizer) error {
|
|
if syncResponse.NetbirdConfig != nil {
|
|
anonymizeNetbirdConfig(syncResponse.NetbirdConfig, anonymizer)
|
|
}
|
|
|
|
if syncResponse.PeerConfig != nil {
|
|
anonymizePeerConfig(syncResponse.PeerConfig, anonymizer)
|
|
}
|
|
|
|
for _, p := range syncResponse.RemotePeers {
|
|
anonymizeRemotePeer(p, anonymizer)
|
|
}
|
|
|
|
if syncResponse.NetworkMap != nil {
|
|
if err := anonymizeNetworkMap(syncResponse.NetworkMap, anonymizer); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
|
|
for _, check := range syncResponse.Checks {
|
|
for i, file := range check.Files {
|
|
check.Files[i] = anonymizer.AnonymizeString(file)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func anonymizeSSHConfig(sshConfig *mgmProto.SSHConfig) {
|
|
if sshConfig != nil && len(sshConfig.SshPubKey) > 0 {
|
|
sshConfig.SshPubKey = []byte("ssh-placeholder-key")
|
|
}
|
|
}
|
|
|
|
func anonymizePeerConfig(config *mgmProto.PeerConfig, anonymizer *anonymize.Anonymizer) {
|
|
if config == nil {
|
|
return
|
|
}
|
|
|
|
if addr, err := netip.ParseAddr(config.Address); err == nil {
|
|
config.Address = anonymizer.AnonymizeIP(addr).String()
|
|
}
|
|
|
|
if len(config.GetAddressV6()) > 0 {
|
|
v6Prefix, err := netiputil.DecodePrefix(config.GetAddressV6())
|
|
if err != nil {
|
|
config.AddressV6 = nil
|
|
} else {
|
|
anonV6 := anonymizer.AnonymizeIP(v6Prefix.Addr())
|
|
b, err := netiputil.EncodePrefix(netip.PrefixFrom(anonV6, v6Prefix.Bits()))
|
|
if err != nil {
|
|
config.AddressV6 = nil
|
|
} else {
|
|
config.AddressV6 = b
|
|
}
|
|
}
|
|
}
|
|
|
|
anonymizeSSHConfig(config.SshConfig)
|
|
|
|
config.Dns = anonymizer.AnonymizeString(config.Dns)
|
|
config.Fqdn = anonymizer.AnonymizeDomain(config.Fqdn)
|
|
}
|
|
|
|
func anonymizeRemotePeer(peer *mgmProto.RemotePeerConfig, anonymizer *anonymize.Anonymizer) {
|
|
if peer == nil {
|
|
return
|
|
}
|
|
|
|
for i, ip := range peer.AllowedIps {
|
|
if prefix, err := netip.ParsePrefix(ip); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
peer.AllowedIps[i] = fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
} else if addr, err := netip.ParseAddr(ip); err == nil {
|
|
peer.AllowedIps[i] = anonymizer.AnonymizeIP(addr).String()
|
|
}
|
|
}
|
|
|
|
peer.Fqdn = anonymizer.AnonymizeDomain(peer.Fqdn)
|
|
peer.WgPubKey = anonymizer.AnonymizeWGKey(peer.WgPubKey)
|
|
|
|
anonymizeSSHConfig(peer.SshConfig)
|
|
}
|
|
|
|
func anonymizeRoute(route *mgmProto.Route, anonymizer *anonymize.Anonymizer) {
|
|
if route == nil {
|
|
return
|
|
}
|
|
|
|
if prefix, err := netip.ParsePrefix(route.Network); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
route.Network = fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
}
|
|
|
|
for i, domain := range route.Domains {
|
|
route.Domains[i] = anonymizer.AnonymizeDomain(domain)
|
|
}
|
|
|
|
route.NetID = anonymizer.AnonymizeString(route.NetID)
|
|
}
|
|
|
|
func anonymizeDNSConfig(config *mgmProto.DNSConfig, anonymizer *anonymize.Anonymizer) {
|
|
if config == nil {
|
|
return
|
|
}
|
|
|
|
anonymizeNameBundleGeneratorGroups(config.NameServerGroups, anonymizer)
|
|
anonymizeCustomZones(config.CustomZones, anonymizer)
|
|
}
|
|
|
|
func anonymizeNameBundleGeneratorGroups(groups []*mgmProto.NameServerGroup, anonymizer *anonymize.Anonymizer) {
|
|
for _, group := range groups {
|
|
anonymizeBundleGenerators(group.NameServers, anonymizer)
|
|
anonymizeDomains(group.Domains, anonymizer)
|
|
}
|
|
}
|
|
|
|
func anonymizeBundleGenerators(servers []*mgmProto.NameServer, anonymizer *anonymize.Anonymizer) {
|
|
for _, server := range servers {
|
|
if addr, err := netip.ParseAddr(server.IP); err == nil {
|
|
server.IP = anonymizer.AnonymizeIP(addr).String()
|
|
}
|
|
}
|
|
}
|
|
|
|
func anonymizeDomains(domains []string, anonymizer *anonymize.Anonymizer) {
|
|
for i, domain := range domains {
|
|
domains[i] = anonymizer.AnonymizeDomain(domain)
|
|
}
|
|
}
|
|
|
|
func anonymizeCustomZones(zones []*mgmProto.CustomZone, anonymizer *anonymize.Anonymizer) {
|
|
for _, zone := range zones {
|
|
zone.Domain = anonymizer.AnonymizeDomain(zone.Domain)
|
|
anonymizeRecords(zone.Records, anonymizer)
|
|
}
|
|
}
|
|
|
|
func anonymizeRecords(records []*mgmProto.SimpleRecord, anonymizer *anonymize.Anonymizer) {
|
|
for _, record := range records {
|
|
record.Name = anonymizer.AnonymizeDomain(record.Name)
|
|
anonymizeRData(record, anonymizer)
|
|
}
|
|
}
|
|
|
|
func anonymizeRData(record *mgmProto.SimpleRecord, anonymizer *anonymize.Anonymizer) {
|
|
switch record.Type {
|
|
case 1, 28:
|
|
if addr, err := netip.ParseAddr(record.RData); err == nil {
|
|
record.RData = anonymizer.AnonymizeIP(addr).String()
|
|
}
|
|
default:
|
|
record.RData = anonymizer.AnonymizeString(record.RData)
|
|
}
|
|
}
|
|
|
|
func anonymizeFirewallRule(rule *mgmProto.FirewallRule, anonymizer *anonymize.Anonymizer) {
|
|
if rule == nil {
|
|
return
|
|
}
|
|
|
|
//nolint:staticcheck // PeerIP used for backward compatibility
|
|
if addr, err := netip.ParseAddr(rule.PeerIP); err == nil {
|
|
rule.PeerIP = anonymizer.AnonymizeIP(addr).String() //nolint:staticcheck
|
|
}
|
|
|
|
for i, raw := range rule.GetSourcePrefixes() {
|
|
p, err := netiputil.DecodePrefix(raw)
|
|
if err != nil {
|
|
continue
|
|
}
|
|
anonAddr := anonymizer.AnonymizeIP(p.Addr())
|
|
if b, err := netiputil.EncodePrefix(netip.PrefixFrom(anonAddr, p.Bits())); err == nil {
|
|
rule.SourcePrefixes[i] = b
|
|
}
|
|
}
|
|
}
|
|
|
|
func anonymizeRouteFirewallRule(rule *mgmProto.RouteFirewallRule, anonymizer *anonymize.Anonymizer) {
|
|
if rule == nil {
|
|
return
|
|
}
|
|
|
|
for i, sourceRange := range rule.SourceRanges {
|
|
if prefix, err := netip.ParsePrefix(sourceRange); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
rule.SourceRanges[i] = fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
}
|
|
}
|
|
|
|
if prefix, err := netip.ParsePrefix(rule.Destination); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
rule.Destination = fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
}
|
|
}
|
|
|
|
func anonymizeStateFile(rawStates *map[string]json.RawMessage, anonymizer *anonymize.Anonymizer) error {
|
|
for name, rawState := range *rawStates {
|
|
if string(rawState) == "null" {
|
|
continue
|
|
}
|
|
|
|
var state map[string]any
|
|
if err := json.Unmarshal(rawState, &state); err != nil {
|
|
return fmt.Errorf("unmarshal state %s: %w", name, err)
|
|
}
|
|
|
|
state = anonymizeValue(state, anonymizer).(map[string]any)
|
|
|
|
bs, err := json.Marshal(state)
|
|
if err != nil {
|
|
return fmt.Errorf("marshal state %s: %w", name, err)
|
|
}
|
|
|
|
(*rawStates)[name] = bs
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func anonymizeValue(value any, anonymizer *anonymize.Anonymizer) any {
|
|
switch v := value.(type) {
|
|
case string:
|
|
return anonymizeString(v, anonymizer)
|
|
case map[string]any:
|
|
return anonymizeMap(v, anonymizer)
|
|
case []any:
|
|
return anonymizeSlice(v, anonymizer)
|
|
}
|
|
return value
|
|
}
|
|
|
|
func anonymizeString(v string, anonymizer *anonymize.Anonymizer) string {
|
|
if prefix, err := netip.ParsePrefix(v); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
return fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
}
|
|
if ip, err := netip.ParseAddr(v); err == nil {
|
|
return anonymizer.AnonymizeIP(ip).String()
|
|
}
|
|
return anonymizer.AnonymizeString(v)
|
|
}
|
|
|
|
func anonymizeMap(v map[string]any, anonymizer *anonymize.Anonymizer) map[string]any {
|
|
result := make(map[string]any, len(v))
|
|
for key, val := range v {
|
|
newKey := anonymizeMapKey(key, anonymizer)
|
|
result[newKey] = anonymizeValue(val, anonymizer)
|
|
}
|
|
return result
|
|
}
|
|
|
|
func anonymizeMapKey(key string, anonymizer *anonymize.Anonymizer) string {
|
|
if prefix, err := netip.ParsePrefix(key); err == nil {
|
|
anonIP := anonymizer.AnonymizeIP(prefix.Addr())
|
|
return fmt.Sprintf("%s/%d", anonIP, prefix.Bits())
|
|
}
|
|
if ip, err := netip.ParseAddr(key); err == nil {
|
|
return anonymizer.AnonymizeIP(ip).String()
|
|
}
|
|
return key
|
|
}
|
|
|
|
func anonymizeSlice(v []any, anonymizer *anonymize.Anonymizer) []any {
|
|
for i, val := range v {
|
|
v[i] = anonymizeValue(val, anonymizer)
|
|
}
|
|
return v
|
|
}
|
|
|
|
func removeStaleFiles(dir, pattern string, maxAge time.Duration) {
|
|
matches, err := filepath.Glob(filepath.Join(dir, pattern))
|
|
if err != nil {
|
|
log.Debugf("glob stale debug bundles in %s: %v", dir, err)
|
|
return
|
|
}
|
|
|
|
cutoff := time.Now().Add(-maxAge)
|
|
for _, path := range matches {
|
|
info, err := os.Stat(path)
|
|
if err != nil || info.ModTime().After(cutoff) {
|
|
continue
|
|
}
|
|
if err := os.Remove(path); err != nil {
|
|
if !errors.Is(err, fs.ErrNotExist) {
|
|
log.Warnf("remove stale debug bundle %s: %v", path, err)
|
|
}
|
|
continue
|
|
}
|
|
log.Infof("removed stale debug bundle %s", path)
|
|
}
|
|
}
|