From dff289cb8b3e83fa789b25a1de38641b4e5af6bb Mon Sep 17 00:00:00 2001 From: Jack Carter <128555021+SunsetDrifter@users.noreply.github.com> Date: Wed, 30 Sep 2026 18:59:24 +0200 Subject: [PATCH] docs: BYOP run command and exposing L4 ports, with a host-networking option (#1008) * docs: BYOP run command and exposing L4 ports, with a host-networking option - Bring Your Own Proxy: update the docker run commands to the dashboard wizard's current form (NB_PROXY_ADDRESS=:443, named certificate volume); the image listens on 8443 by default and runs as uid 1000. - Bring Your Own Proxy: new section on exposing TCP and UDP services: publish and open each listen port, the recreate it takes, a pre-published range, or host networking and its port-1024 limit. - Enable Reverse Proxy: note that changing ports recreates the proxy, and document host networking with the two settings it needs. - Reverse Proxy overview: point BYOP users to the new section. * docs(reverse-proxy): UFW does not filter Docker-published ports; host mode needs Traefik's ProxyService route --- .../reverse-proxy/bring-your-own-proxy.mdx | 57 ++++++++++++++++++- src/pages/manage/reverse-proxy/index.mdx | 2 +- .../migration/enable-reverse-proxy.mdx | 26 +++++++++ 3 files changed, 82 insertions(+), 3 deletions(-) diff --git a/src/pages/manage/reverse-proxy/bring-your-own-proxy.mdx b/src/pages/manage/reverse-proxy/bring-your-own-proxy.mdx index e47b0a5b..2ab3a8d2 100644 --- a/src/pages/manage/reverse-proxy/bring-your-own-proxy.mdx +++ b/src/pages/manage/reverse-proxy/bring-your-own-proxy.mdx @@ -141,7 +141,7 @@ Switching to the **Run the Proxy** tab automatically generates a one-time, accou ```shell docker run -d \ - -v /var/lib/certs:/certs \ + -v proxy_certs:/certs \ -e NB_PROXY_CERTIFICATE_DIRECTORY=/certs \ -e NB_PROXY_ALLOW_INSECURE=true \ -e NB_PROXY_MANAGEMENT_ADDRESS=https://api.netbird.io \ @@ -149,6 +149,8 @@ docker run -d \ -e NB_PROXY_DOMAIN=proxy.company.com \ -e NB_PROXY_LOG_LEVEL=info \ -e NB_PROXY_TOKEN=nbx_... \ + -e NB_PROXY_PRIVATE=true \ + -e NB_PROXY_ADDRESS=:443 \ -p 80:80 -p 443:443 \ netbirdio/reverse-proxy:latest ``` @@ -167,6 +169,10 @@ The wizard substitutes `NB_PROXY_MANAGEMENT_ADDRESS` based on the dashboard's co The `-p 80:80` mapping is only needed if you switch to the `http-01` ACME challenge (see [TLS configuration](#tls-configuration)). With the default `tls-alpn-01` challenge you can drop it and publish only `-p 443:443`. + + Keep `NB_PROXY_ADDRESS=:443`. The `netbirdio/reverse-proxy` image listens on port `8443` by default, so without this variable nothing answers on the published port `443`. The certificates go in a named volume because the proxy runs as an unprivileged user (uid `1000`). If you bind-mount a host directory instead, make it writable by that user first (`sudo chown 1000:1000 /var/lib/certs`); otherwise the proxy cannot write to it and logs `permission denied`. + + Run the command on your server. The container will: 1. Connect to NetBird's management server (`https://api.netbird.io` for NetBird Cloud, or the management URL of your self-hosted deployment) over gRPC and authenticate with the token. The wizard fills this in based on the dashboard's configured management endpoint. @@ -201,6 +207,52 @@ Once the cluster is connected, your BYOP domain shows up in the service creation Traffic to `subdomain.proxy.company.com` is now received by your BYOP proxy, terminated locally with a Let's Encrypt certificate, and forwarded over WireGuard to the target peer or network resource. +## Expose TCP and UDP services + +HTTP services, and TLS services on port `443`, share the proxy's main port, so the command above is all they need. Every other listen port is different. When you create a TCP, UDP or TLS service on a port other than `443`, the proxy opens a listener on that port inside its container, and clients cannot reach it until Docker publishes the port: TCP connections to it are refused. + +For every such listen port: + +1. Allow the port through the firewall or security group in front of the server. +2. Publish it on the container: `-p 5432:5432` for TCP, `-p 5353:5353/udp` for UDP. + +Docker fixes a container's published ports when it creates the container, so publishing another port means recreating it (`docker rm -f` and `docker run` again, or `docker compose up -d` after editing `ports:`). A recreate briefly interrupts every service on the proxy, not only the new one. Publishing ports this way is the default, and it keeps Docker as a second filter in front of the proxy. If you add L4 services often, there are two ways to avoid a recreate for each one. + +**Publish a range up front.** For example, add `-p 20000-20099:20000-20099 -p 20000-20099:20000-20099/udp` and choose listen ports inside that range. Ranges cost memory: with Docker's default userland proxy, Docker starts helper processes for every published port and protocol, and a 100-port range for TCP and UDP uses about 500 MB. Keep the range as small as your plans allow. A host firewall such as UFW does not filter ports that Docker publishes ([Docker and ufw](https://docs.docker.com/engine/network/packet-filtering-firewalls/#docker-and-ufw)), so restrict the range with a firewall or security group in front of the server, or with rules in Docker's `DOCKER-USER` chain, and open only the ports you use. + +**Run the proxy on the host network.** This suits a server dedicated to the proxy, because it needs a host-wide setting (below) and leaves the firewall as the only filter. The proxy then binds each listen port on the server itself, so a new TCP or UDP service is reachable as soon as you create it, with no change to the container and no interruption to other services. The firewall becomes the only thing that decides which ports are reachable, so keep it closed by default and open each listen port as you add its service. + +The image runs as an unprivileged user, and on the host network an unprivileged process cannot bind ports below `1024`. `NB_PROXY_ADDRESS=:443` then fails with `bind: permission denied`, and `--cap-add NET_BIND_SERVICE` does not change that. Before you start the proxy, lower the host's unprivileged port floor to the lowest port you need: + +```shell +sudo sysctl -w net.ipv4.ip_unprivileged_port_start=443 +echo 'net.ipv4.ip_unprivileged_port_start=443' | sudo tee /etc/sysctl.d/90-netbird-proxy.conf +``` + +This setting applies to every process on the server, so give the proxy a host of its own. Then start it with the wizard's command, replacing the `-p` flags with `--network host`: + +```shell +docker run -d \ + --network host \ + -v proxy_certs:/certs \ + -e NB_PROXY_CERTIFICATE_DIRECTORY=/certs \ + -e NB_PROXY_ALLOW_INSECURE=true \ + -e NB_PROXY_MANAGEMENT_ADDRESS=https://api.netbird.io \ + -e NB_PROXY_ACME_CERTIFICATES=true \ + -e NB_PROXY_DOMAIN=proxy.company.com \ + -e NB_PROXY_LOG_LEVEL=info \ + -e NB_PROXY_TOKEN=nbx_... \ + -e NB_PROXY_PRIVATE=true \ + -e NB_PROXY_ADDRESS=:443 \ + netbirdio/reverse-proxy:latest +``` + +An L4 service on a port below the floor still fails to bind: its status becomes `error`, and the proxy logs `bind: permission denied` for that port. + + + Do not run the container as root (`--user 0:0`) to get around the port limit. As root, the proxy's embedded NetBird client does not connect to management, and HTTP services return `504`. + + ## Private services (NetBird-Only Access) A BYOP cluster whose proxy runs embedded as a NetBird peer (`netbird proxy` mode — the default for the `netbirdio/reverse-proxy` image) reports the `Private` capability if started with the flag `--private` or environment variable `NB_PROXY_PRIVATE=true`. With that capability set, the cluster unlocks two new options anywhere it is selected: @@ -283,12 +335,13 @@ curl -X POST /api/reverse-proxies/proxy-tokens \ ```shell docker run -d \ - -v /var/lib/certs:/certs \ + -v proxy_certs:/certs \ -e NB_PROXY_CERTIFICATE_DIRECTORY=/certs \ -e NB_PROXY_MANAGEMENT_ADDRESS= \ -e NB_PROXY_ACME_CERTIFICATES=true \ -e NB_PROXY_DOMAIN=proxy.company.com \ -e NB_PROXY_TOKEN= \ + -e NB_PROXY_ADDRESS=:443 \ -p 80:80 -p 443:443 \ netbirdio/reverse-proxy:latest ``` diff --git a/src/pages/manage/reverse-proxy/index.mdx b/src/pages/manage/reverse-proxy/index.mdx index d43da645..a16d1ea2 100644 --- a/src/pages/manage/reverse-proxy/index.mdx +++ b/src/pages/manage/reverse-proxy/index.mdx @@ -378,7 +378,7 @@ The dashboard indicates whether a proxy cluster supports custom ports when you s - **Self-hosted Docker deployments:** The default Docker Compose configuration only routes port 443 (via Traefik TLS passthrough) to the proxy container, where the proxy's main port is `8443` (`NB_PROXY_ADDRESS`). A TLS service given listen port `443` in such a deployment therefore listens on a custom port inside the container, not on the main port. L4 services that listen on additional TCP or UDP ports require you to manually expose those ports in your `docker-compose.yml`. See the [migration guide](/selfhosted/migration/enable-reverse-proxy#exposing-l4-ports) for instructions. + **Self-hosted Docker deployments:** The default Docker Compose configuration only routes port 443 (via Traefik TLS passthrough) to the proxy container, where the proxy's main port is `8443` (`NB_PROXY_ADDRESS`). A TLS service given listen port `443` in such a deployment therefore listens on a custom port inside the container, not on the main port. L4 services that listen on additional TCP or UDP ports require you to manually expose those ports in your `docker-compose.yml`. See the [migration guide](/selfhosted/migration/enable-reverse-proxy#exposing-l4-ports) for instructions. A Bring Your Own Proxy deployment has the same requirement; see [Expose TCP and UDP services](/manage/reverse-proxy/bring-your-own-proxy#expose-tcp-and-udp-services). In both cases, running the proxy on the host network removes the per-port Docker step. ### How services share ports diff --git a/src/pages/selfhosted/migration/enable-reverse-proxy.mdx b/src/pages/selfhosted/migration/enable-reverse-proxy.mdx index 544f739c..a9db50e3 100644 --- a/src/pages/selfhosted/migration/enable-reverse-proxy.mdx +++ b/src/pages/selfhosted/migration/enable-reverse-proxy.mdx @@ -271,6 +271,32 @@ docker compose up -d proxy You only need port mappings for TCP and UDP mode services. HTTP and TLS mode services are routed through port 443 via Traefik and do not require additional port entries. +Changing `ports` recreates the proxy container, which briefly interrupts every service it serves, not only the new one. To add L4 services without that step, run the proxy on the host network instead. + +#### Use host networking instead + +With host networking, the proxy binds each L4 listen port on the server itself, so a new TCP or UDP service is reachable as soon as you create it, with no compose change and no restart. The server's firewall then decides which ports are reachable, so open each listen port there as you add its service. The port mappings above remain the default: host networking gives up Docker's port isolation, and here the proxy shares the host's network with your NetBird server and Traefik. + +Two settings must change with it, because the proxy is no longer on the `netbird` Docker network: + +- **The management address.** `http://netbird-server:80` (or `http://management:33073` in a multi-container setup) is a Docker network name, which a container on the host network cannot resolve; the proxy logs `name resolver error: produced zero addresses` and never connects. In `proxy.env`, set `NB_PROXY_MANAGEMENT_ADDRESS` to your public NetBird URL, for example `https://netbird.example.com:443`. This connection goes through Traefik, so Traefik must route `/management.ProxyService/` and allow long-lived gRPC streams. Deployments from the current installer already do; for older ones, apply steps 1 and 3 of [Connecting through Traefik instead of Docker network](#connecting-through-traefik-instead-of-docker-network). +- **How Traefik reaches the proxy.** For a container on the host network, Traefik dials `127.0.0.1:8443`, which inside Traefik's own container is Traefik itself, so every HTTP service fails with `connection refused`. Mapping `host.docker.internal` to the host in Traefik's service makes it reach the proxy on the host instead. + +In `docker-compose.yml`, remove the `networks` and `ports` entries from the `proxy` service, then add: + +```yaml +proxy: + # ...existing configuration, without networks: and ports:... + network_mode: host + +traefik: + # ...existing configuration... + extra_hosts: + - host.docker.internal:host-gateway +``` + +Apply both with `docker compose up -d proxy traefik`. The proxy's main port stays `8443`, which an unprivileged process can bind on the host. An L4 listen port below `1024` needs the host's unprivileged port floor lowered first; see [Bring Your Own Proxy](/manage/reverse-proxy/bring-your-own-proxy#expose-tcp-and-udp-services). + ### Step 4: Set up DNS records Create DNS records pointing to the server running your NetBird stack: one for the base proxy domain and one wildcard for service subdomains.