pangolin_mirror/docs-v2
1
0
Fork 0
mirror of https://github.com/fosrl/docs-v2.git synced 2026-09-17 03:19:08 +02:00
Code Issues Packages Projects Releases Wiki Activity

HA docs pass 1

Browse Source
This commit is contained in:
Owen
2026-09-11 12:08:41 -04:00
parent aaab1bcaa1
commit 219d0ef2ed
7 changed files with 845 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files
about/pangolin-cloud-vs-self-hosted.mdx
+1 -1
View File
@@ -38,7 +38,7 @@ Public TCP/UDP listeners that bind a port on the Pangolin host are a self-hosted
| **Traffic path** | Cloud nodes, or your [remote nodes](/manage/remote-node/understanding-nodes) | Your server |
| **DNS and certificates** | Managed | You configure DNS and Let's Encrypt |
| **Updates and backups** | Included | You operate them |
| **High availability** | Included | You design it ([clustering](/self-host/advanced/clustering) on Enterprise) |
| **High availability** | Included | You design it ([clustering](/self-host/advanced/clustering/understanding-clustering) on Enterprise) |
| **Identity and logs** | Cloud plans | Community core; extra IdPs, streaming, and AI session logs on [Enterprise Edition](/self-host/enterprise-edition) |
| **Data residency** | Cloud regions, or traffic on your remote nodes | Entirely on your infrastructure |
| **Best for** | Fast start and less ops | Air-gapped networks, full control, compliance that requires self-hosting |
development/system-architecture.mdx
+2 -2
View File
@@ -34,7 +34,7 @@ You can run the control plane yourself ([self-hosted](/self-host/quick-install))
A **node** is the networking edge of your Pangolin deployment. It terminates inbound traffic from the public internet, manages WireGuard tunnels, and relays client traffic when a direct peer path is unavailable.
On a self-hosted deployment, your Pangolin server runs as a single node by default. [Enterprise clustering](/self-host/advanced/clustering) spreads multiple nodes behind a load balancer with shared database state. On [Pangolin Cloud](/manage/remote-node/understanding-nodes), you can add **remote nodes**: infrastructure you operate that still receives configuration from the cloud control plane.
On a self-hosted deployment, your Pangolin server runs as a single node by default. [Enterprise clustering](/self-host/advanced/clustering/understanding-clustering) spreads multiple nodes behind a load balancer with shared database state. On [Pangolin Cloud](/manage/remote-node/understanding-nodes), you can add **remote nodes**: infrastructure you operate that still receives configuration from the cloud control plane.
Each node runs several cooperating processes:
@@ -51,7 +51,7 @@ The control plane writes certificates, Traefik router definitions, and WireGuard
Required listener ports, firewall rules, and DNS records for a node.
</Card>
<Card title="Clustering for high availability" icon="server" href="/self-host/advanced/clustering">
<Card title="Clustering for high availability" icon="server" href="/self-host/advanced/clustering/understanding-clustering">
Multi-node Enterprise deployments with shared state and failover.
</Card>
docs.json
+14 -1
View File
@@ -318,7 +318,15 @@
"self-host/advanced/observability",
"self-host/advanced/enable-geolocation",
"self-host/advanced/enable-asn-lookup",
"self-host/advanced/clustering",
{
"group": "Clustering",
"icon": "server",
"pages": [
"self-host/advanced/clustering/understanding-clustering",
"self-host/advanced/clustering/requirements",
"self-host/advanced/clustering/deploy-a-cluster"
]
},
"self-host/advanced/traefik-log-rotation",
"self-host/telemetry"
]
@@ -504,6 +512,11 @@
]
},
"redirects": [
{
"source": "/self-host/advanced/clustering",
"destination": "/self-host/advanced/clustering/understanding-clustering",
"permanent": true
},
{
"source": "/self-host/advanced/enable-geoblocking",
"destination": "/self-host/advanced/enable-geolocation"
self-host/advanced/clustering.mdx
-157
View File
@@ -1,157 +0,0 @@
---
title: "Clustering for High Availability"
---
<Note>
Clustering is only available in [Enterprise Edition](/self-host/enterprise-edition). [Please reach out to us to deploy](https://pangolin.net/contact).
</Note>
Deploy multiple Pangolin servers for enterprise-grade high availability and performance in large deployments.
## Overview
For organizations requiring maximum uptime and performance, Pangolin supports clustered deployments where multiple server instances work together as a unified system. This architecture enables regional distribution, automatic failover, and horizontal scaling to handle demanding production workloads.
In a clustered configuration, multiple Pangolin server instances operate together, sharing state through a PostgreSQL database and Valkey server. Each instance can independently serve user requests, manage authentication, and coordinate with Gerbil instances to support thousands of sites across your organization.
<Frame caption="Pangolin clsuter diagram with two nodes">
<img src="/images/pangolin-cluster.png" alt="Pangolin Cluster"/>
</Frame>
## Architecture Components
A Pangolin cluster consists of several coordinated components that work together to provide high availability and seamless failover. Each component plays a specific role in ensuring your deployment remains online and performant.
### Pangolin Instances
**Purpose**: Serve the web UI, API, and manage cluster coordination.
**How It Works**:
- Multiple Pangolin instances run simultaneously across different nodes
- Each instance can independently handle user authentication and requests
- All instances share state through the PostgreSQL database and Valkey
- Instances coordinate to manage site configurations, resources, and access policies
**High Availability**: A load balancer sits in front of all Pangolin instances. If any instance goes down, the load balancer automatically routes traffic to healthy nodes, ensuring the UI and API remain accessible from the same domain without interruption.
### DNS Servers
**Purpose**: Provide dynamic DNS resolution for certificate generation and failover management.
**How It Works**:
- Each cluster node runs a DNS server in the same Docker stack as Pangolin
- DNS servers listen on port 53 UDP and serve DNS records for cluster resources
- You must create NS (name server) records in your domain provider pointing to each DNS instance
- You must delegate domains for resources to these name servers
- Multiple DNS instances prevent single points of failure
**Key Responsibilities**:
- Host ACME challenge tokens for Let's Encrypt certificate generation
- Automatically update DNS records when sites move between nodes
- Redirect traffic to healthy nodes when a node fails
- Enable failover by pointing resources to available instances
### PostgreSQL Database
**Purpose**: Store all persistent cluster state in a centralized, shared database.
**How It Works**:
- All Pangolin instances connect to a shared PostgreSQL database
- Stores user accounts, site configurations, resources, access policies, and organizational settings
- Certificates are stored encrypted in the database for security
- Changes made through any instance are immediately available cluster-wide
**High Availability**: Database replication and backup strategies ensure data persistence and availability across the cluster.
### Valkey (Redis)
**Purpose**: Provide real-time state synchronization between cluster nodes.
**How It Works**:
- Pub/sub messaging keeps nodes synchronized on ephemeral state
- Tracks active user sessions, WebSocket connections, and tunnel status
- When a user authenticates to one instance, all nodes become aware via Valkey
- Enables rapid failover by sharing connection state across the cluster
**High Availability**: Ensures that session and connection information remains available even when individual nodes fail.
### Certificate Generation Server
**Purpose**: Automate TLS certificate issuance and renewal for all cluster resources.
**How It Works**:
- Dedicated server communicates with Let's Encrypt for certificate issuance
- Uses DNS-01 challenge validation through the cluster's DNS servers
- Generates certificates and stores them encrypted in the PostgreSQL database
- Handles automatic certificate renewal across the entire cluster
**High Availability**: Centralized certificate management ensures all resources maintain valid TLS certificates regardless which node they are served from.
### Traefik Instances
**Purpose**: Route HTTP/HTTPS traffic to resources and terminate TLS connections.
**How It Works**:
- Each cluster node runs its own Traefik instance
- Pangolin writes configuration files and certificates to a shared volume with Traefik
- Each resource pulls its certificate from the database to files Traefik can read
- Traefik reads router configurations from local files to serve resources
- Sits behind Gerbil, which runs an SNI proxy for traffic routing
**High Availability**: Multiple Traefik instances ensure traffic routing continues even if individual nodes fail.
### Gerbil Instances
**Purpose**: Manage WireGuard tunnels to site connectors and route traffic between cluster nodes.
**How It Works**:
- Each Pangolin instance runs alongside its own Gerbil tunnel manager
- Handles WireGuard VPN connections from site connectors
- Site connectors can establish tunnels to any available Gerbil instance
- All Gerbil instances are aware of other nodes in the network
- Routes incoming requests to the correct Gerbil instance to exit through the appropriate site
- When DNS caching causes traffic to hit the wrong node, Gerbil routes requests to the correct node
**High Availability**: Distributed tunnel management ensures connectivity remains available even if individual Gerbil instances fail. The distributed architecture allows automatic failover between nodes.
### Load Balancer
**Purpose**: Distribute incoming traffic across healthy Pangolin instances.
**How It Works**:
- Sits in front of all Pangolin instances serving the UI and API
- Monitors instance health and routes traffic only to available nodes
- Ensures all traffic accesses the cluster through a single, consistent domain
- Provides seamless failover when instances become unavailable
**High Availability**: Essential for ensuring continuous access to the Pangolin UI and API regardless of individual instance failures. **You must provide your own HA load balancer** in front of the cluster.
## Traffic Flow
Understanding how requests flow through the cluster helps clarify how these components work together:
1. **User Access**: Users access the Pangolin UI/API through the load balancer, which routes to any healthy Pangolin instance
2. **Resource Requests**: When accessing a resource, DNS resolves to the appropriate Gerbil instance
3. **Cross-Node Routing**: If DNS caching points to the wrong node, Gerbil automatically routes to the correct node
4. **Tunnel Routing**: Gerbil receives the request and routes it to the local Traefik instance
5. **TLS Termination**: Traefik handles TLS termination using certificates pulled from the shared volume configuration to the right site connector tunnel
6. **Failover**: When nodes fail the load balancer and DNS automatically route traffic to healthy nodes
## Benefits of Clustering
**High Availability**: Eliminate single points of failure. If one server instance fails, traffic automatically routes to healthy nodes without user disruption.
**Regional Distribution**: Deploy servers closer to your users and sites across different geographic regions to minimize latency and improve performance.
**Horizontal Scaling**: Add more server instances to handle increased load as your organization grows, without architectural changes.
**Zero-Downtime Updates**: Perform rolling updates by taking nodes offline one at a time while others continue serving traffic.
**Resilient DNS**: Multiple DNS servers ensure certificate generation and failover continue operating even during outages.
**Dynamic Failover**: Automatic DNS updates and traffic routing ensure resources remain accessible when nodes fail.
## Enterprise Support
Clustered deployments require careful planning around database replication, Valkey configuration, network topology, DNS infrastructure, and monitoring. These advanced architectures are available as part of Pangolin's [Enterprise Edition](/self-host/enterprise-edition) with dedicated support for design, deployment, and ongoing operations.
For organizations interested in clustering for high availability or regional distribution, please [contact our enterprise team](https://pangolin.net/contact) to discuss your requirements and receive implementation guidance.
self-host/advanced/clustering/deploy-a-cluster.mdx
+555
View File
@@ -0,0 +1,555 @@
---
title: "Deploy a Cluster"
description: "Step-by-step walkthrough for deploying a two-node highly available Pangolin cluster"
---
<Note>
Clustering is only available in [Enterprise Edition](/self-host/enterprise-edition).
</Note>
This guide walks through deploying a minimal two-node Pangolin cluster: two Pangolin nodes behind a load balancer, sharing a PostgreSQL database and a Valkey (Redis) server. Read [Understanding Clustering](/self-host/advanced/clustering/understanding-clustering) for the architecture and [Requirements](/self-host/advanced/clustering/requirements) for the hosts, ports, and DNS records you need before starting.
<Card icon="github" arrow="true" cta="View reference configuration" href="https://github.com/fosrl/pangolin/tree/main/config/ha-reference">
The complete, working set of files used in this guide lives in the Pangolin repository at [`config/ha-reference`](https://github.com/fosrl/pangolin/tree/main/config/ha-reference). Clone it as a starting point instead of assembling files by hand.
</Card>
<Warning>
`server.secret` in `config.yml` must be **identical on every node in the cluster**. It's used to encrypt sensitive data, including the certificates stored in PostgreSQL - if nodes have different secrets, they won't be able to read each other's data.
</Warning>
Throughout this guide, replace the following placeholders with your own values:
| Placeholder | Description |
| --- | --- |
| `NODE1_EXTERNAL_IP` / `NODE2_EXTERNAL_IP` | Public static IP of each Pangolin node |
| `NODE1_INTERNAL_IP` / `NODE2_INTERNAL_IP` | Internal IP each node uses to address the other |
| `POSTGRES_INTERNAL_HOST` | Address of your shared PostgreSQL server |
| `POSTGRES_USERNAME` / `POSTGRES_PASSWORD` | Credentials for that PostgreSQL server |
| `REDIS_INTERNAL_HOST` | Address of your shared Redis-compatible server |
| `CONTACT_EMAIL` | Email address used for Let's Encrypt ACME registration |
| `SECRET` | Shared server secret - identical on every node |
| `LOAD_BALANCER_IP` | IP address of the load balancer in front of the cluster |
<Steps>
<Step title="Provision the shared database">
Stand up a PostgreSQL server and a Redis-compatible server that both nodes can reach. They don't need to run together, or even on a dedicated third host - use whatever you already run, including managed cloud offerings. The only hard requirement is that the Redis-compatible server supports **pub/sub**.
For a simple self-hosted starting point:
```yaml title="docker-compose.yml"
services:
postgres:
image: postgres:17
container_name: postgres
environment:
POSTGRES_DB: postgres # Default database name
POSTGRES_USER: postgres # Default user
POSTGRES_PASSWORD: password # Default password (change for production!)
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
restart: always
redis:
image: redis:latest
container_name: redis
ports:
- "6379:6379"
restart: always
volumes:
postgres_data:
```
<Warning>
Change the default PostgreSQL password before running this in production. See [Database Options](/self-host/advanced/database-options) for general PostgreSQL configuration.
</Warning>
</Step>
<Step title="Lay out each node's config directory">
On each of Node 1 and Node 2, create the following directory structure:
```
node/
├── docker-compose.yml
└── config/
├── config.yml
├── privateConfig.yml
├── certificates/ # empty, Pangolin/Traefik populate this
├── dynamic/
│ └── dynamic_config.yml
└── traefik/
└── traefik_config.yml
```
`config/certificates` and `config/dynamic` are shared volumes between the `pangolin` and `traefik` containers - Pangolin writes router configuration and certificates there for Traefik to read, since Traefik can only load certificates from files, not from the Pangolin API. This is `traefik.file_mode` in `config.yml`, covered below.
</Step>
<Step title="Write docker-compose.yml">
Both nodes run the same three containers: `pangolin`, `gerbil`, and `traefik`. Gerbil owns the host networking (WireGuard, relay, DNS, resource ports), and Traefik joins its network namespace so its ports appear alongside Gerbil's.
Each node's `--reachableAt` flag must point at **that node's own** internal address, and `--trusted-upstreams` lists the external IPs of every node in the cluster so Gerbil accepts proxied connections from them.
<CodeGroup>
```yaml Node 1
name: pangolin
services:
pangolin:
image: docker.io/fosrl/pangolin:ee-latest
container_name: pangolin
restart: unless-stopped
volumes:
- ./config:/app/config
- ./config/certificates:/var/certificates
- ./config/dynamic:/var/dynamic
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
interval: "10s"
timeout: "10s"
retries: 15
gerbil:
image: docker.io/fosrl/gerbil:latest
container_name: gerbil
restart: unless-stopped
depends_on:
pangolin:
condition: service_healthy
command:
- --reachableAt=http://<NODE1_INTERNAL_IP>:3004
- --generateAndSaveKeyTo=/var/config/key
- --remoteConfig=http://pangolin:3001/api/v1/
- --trusted-upstreams=<NODE1_EXTERNAL_IP>,<NODE2_EXTERNAL_IP> # All trusted nodes in the cluster
volumes:
- ./config/:/var/config
cap_add:
- NET_ADMIN
- SYS_MODULE
ports:
- 51820:51820/udp # wireguard
- 21820:21820/udp # relay
- 53:53/udp # DNS
- 443:8443 # resources
- 80:80 # web
- 3004:3004 # gerbil api
- 3000:3000 # Pangolin UI
traefik:
image: docker.io/traefik:v3.7.11
container_name: traefik
restart: unless-stopped
network_mode: service:gerbil # Ports appear on the gerbil service
depends_on:
pangolin:
condition: service_healthy
command:
- --configFile=/etc/traefik/traefik_config.yml
volumes:
- ./config/traefik:/etc/traefik:ro
- ./config/traefik/logs:/var/log/traefik
- ./config/certificates:/var/certificates:ro
- ./config/dynamic:/var/dynamic:ro
networks:
default:
driver: bridge
name: pangolin
```
```yaml Node 2
name: pangolin
services:
pangolin:
image: docker.io/fosrl/pangolin:ee-latest
container_name: pangolin
restart: unless-stopped
volumes:
- ./config:/app/config
- ./config/certificates:/var/certificates
- ./config/dynamic:/var/dynamic
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
interval: "10s"
timeout: "10s"
retries: 15
gerbil:
image: docker.io/fosrl/gerbil:latest
container_name: gerbil
restart: unless-stopped
depends_on:
pangolin:
condition: service_healthy
command:
- --reachableAt=http://<NODE2_INTERNAL_IP>:3004
- --generateAndSaveKeyTo=/var/config/key
- --remoteConfig=http://pangolin:3001/api/v1/
- --trusted-upstreams=<NODE1_EXTERNAL_IP>,<NODE2_EXTERNAL_IP> # All trusted nodes in the cluster
volumes:
- ./config/:/var/config
cap_add:
- NET_ADMIN
- SYS_MODULE
ports:
- 51820:51820/udp # wireguard
- 21820:21820/udp # relay
- 53:53/udp # DNS
- 443:8443 # resources
- 80:80 # web
- 3004:3004 # gerbil api
- 3000:3000 # Pangolin UI
traefik:
image: docker.io/traefik:v3.7.11
container_name: traefik
restart: unless-stopped
network_mode: service:gerbil # Ports appear on the gerbil service
depends_on:
pangolin:
condition: service_healthy
command:
- --configFile=/etc/traefik/traefik_config.yml
volumes:
- ./config/traefik:/etc/traefik:ro
- ./config/traefik/logs:/var/log/traefik
- ./config/certificates:/var/certificates:ro
- ./config/dynamic:/var/dynamic:ro
networks:
default:
driver: bridge
name: pangolin
```
</CodeGroup>
<Tip>
If Pangolin can't reach the local Gerbil at the address in `--reachableAt` (a loopback issue), override it in `privateConfig.yml` - see [Troubleshooting](#troubleshooting) below.
</Tip>
</Step>
<Step title="Write config.yml">
`config.yml` holds the settings that legitimately differ per node - `gerbil.base_endpoint` and `gerbil.exit_node_name` - alongside the shared PostgreSQL connection and site-type restrictions. In clustered deployments, only Newt sites are supported, so local and basic WireGuard sites are disabled.
<CodeGroup>
```yaml Node 1
# To see all available options, please visit the docs:
# https://docs.pangolin.net/
gerbil:
start_port: 51820
base_endpoint: "<NODE1_EXTERNAL_IP>"
exit_node_name: "node1"
app:
dashboard_url: "https://pangolin.example.com"
log_level: "info"
postgres:
connection_string: postgresql://<POSTGRES_USERNAME>:<POSTGRES_PASSWORD>@<POSTGRES_INTERNAL_HOST>:5432/postgres
traefik:
site_types: ["newt"] # Wireguard and local sites are not supported in clustering
file_mode: true # Pangolin will generate and save yaml files in a shared volume
server:
secret: "<SECRET>" # Must be identical on every node
cors:
origins: ["https://pangolin.example.com"]
methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
allowed_headers: ["X-CSRF-Token", "Content-Type"]
credentials: false
maxmind_db_path: "./config/GeoLite2-Country.mmdb" # Download and place into the config dir
maxmind_asn_path: "./config/GeoLite2-ASN.mmdb"
flags:
require_email_verification: false
disable_signup_without_invite: true
disable_user_create_org: false
allow_raw_resources: false
enable_acme_cert_sync: false
disable_local_sites: true
disable_basic_wireguard_sites: true
disable_config_managed_domains: true
```
```yaml Node 2
# To see all available options, please visit the docs:
# https://docs.pangolin.net/
gerbil:
start_port: 51820
base_endpoint: "<NODE2_EXTERNAL_IP>"
exit_node_name: "node2"
app:
dashboard_url: "https://pangolin.example.com"
log_level: "info"
postgres:
connection_string: postgresql://<POSTGRES_USERNAME>:<POSTGRES_PASSWORD>@<POSTGRES_INTERNAL_HOST>:5432/postgres
traefik:
site_types: ["newt"] # Wireguard and local sites are not supported in clustering
file_mode: true # Pangolin will generate and save yaml files in a shared volume
server:
secret: "<SECRET>" # Must be identical on every node
cors:
origins: ["https://pangolin.example.com"]
methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
allowed_headers: ["X-CSRF-Token", "Content-Type"]
credentials: false
maxmind_db_path: "./config/GeoLite2-Country.mmdb" # Download and place into the config dir
maxmind_asn_path: "./config/GeoLite2-ASN.mmdb"
flags:
require_email_verification: false
disable_signup_without_invite: true
disable_user_create_org: false
allow_raw_resources: false
enable_acme_cert_sync: false
disable_local_sites: true
disable_basic_wireguard_sites: true
disable_config_managed_domains: true
```
</CodeGroup>
See the [full configuration reference](/self-host/advanced/config-file) for every available option.
</Step>
<Step title="Write privateConfig.yml">
`privateConfig.yml` enables Redis-backed cluster sync and Pangolin's built-in DNS and ACME client. **Only one node** - Node 1 in this example - should have `acme.enable_acme_client` set to `true`. That node issues and renews certificates via DNS-01 challenges and stores them encrypted in PostgreSQL; every other node reads the same certificates from the database.
<Warning>
Do not enable `acme.enable_acme_client` on more than one node. Multiple nodes issuing certificates simultaneously will conflict with each other.
</Warning>
<CodeGroup>
```yaml Node 1 (ACME client enabled)
app:
region: "region1"
identity_provider_mode: "org"
redis:
host: "<REDIS_INTERNAL_HOST>"
port: 6379
flags:
enable_redis: true
use_pangolin_dns: true
acme:
cert_mode: "pangolin"
contact_email: "<CONTACT_EMAIL>"
enable_acme_client: true
dns:
enabled: true
nameserver_name: "ns.example.com"
cname_extension: "cname.example.com"
site_extension: "site.example.com" # Optional
```
```yaml Node 2 (ACME client disabled)
app:
region: "region1"
identity_provider_mode: "org"
redis:
host: "<REDIS_INTERNAL_HOST>"
port: 6379
flags:
enable_redis: true
use_pangolin_dns: true
acme:
cert_mode: "pangolin"
dns:
enabled: true
nameserver_name: "ns.example.com"
cname_extension: "cname.example.com"
site_extension: "site.example.com" # Optional
```
</CodeGroup>
See the [private configuration reference](/self-host/advanced/private-config-file) for every available option.
</Step>
<Step title="Write the Traefik configuration">
`traefik/traefik_config.yml` and `dynamic/dynamic_config.yml` are identical on every node - copy them as-is. Traefik loads router and certificate configuration from the shared `dynamic` volume (`file_mode`) instead of Pangolin's API, and exposes a `:53/udp` DNS entry point that forwards to Pangolin's built-in DNS server.
```yaml title="config/traefik/traefik_config.yml"
providers:
file:
directory: "/var/dynamic"
watch: true
experimental:
plugins:
badger:
moduleName: "github.com/fosrl/badger"
version: "v1.7.0"
log:
level: "INFO"
format: "common"
maxSize: 100
maxBackups: 3
maxAge: 3
compress: true
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
proxyProtocol: # We trust gerbil upstream
trustedIPs:
- 0.0.0.0/0
- ::1/128
transport:
respondingTimeouts:
readTimeout: "30m"
http:
encodedCharacters:
allowEncodedSlash: true
allowEncodedQuestionMark: true
dashboard:
address: ":3000"
dns:
address: ":53/udp"
serversTransport:
insecureSkipVerify: true
ping:
entryPoint: "web"
```
```yaml title="config/dynamic/dynamic_config.yml"
http:
middlewares:
badger:
plugin:
badger:
disableForwardAuth: true
routers:
# Next.js router (handles everything except API and WebSocket paths)
next-router:
rule: "!PathPrefix(`/api/v1`)"
service: next-service
entryPoints:
- dashboard
middlewares:
- badger
# API router (handles /api/v1 paths)
api-router:
rule: "PathPrefix(`/api/v1`)"
service: api-service
entryPoints:
- dashboard
middlewares:
- badger
# WebSocket router
ws-router:
rule: "PathPrefix(`/`)"
service: api-service
entryPoints:
- dashboard
middlewares:
- badger
services:
next-service:
loadBalancer:
servers:
- url: "http://pangolin:3002" # Next.js server
api-service:
loadBalancer:
servers:
- url: "http://pangolin:3000" # API/WebSocket server
tcp:
serversTransports:
pp-transport-v1:
proxyProtocol:
version: 1
pp-transport-v2:
proxyProtocol:
version: 2
udp:
routers:
dns-router:
entryPoints:
- dns
service: dns-service
services:
dns-service:
loadBalancer:
servers:
- address: "pangolin:53"
```
This is also where you place the MaxMind databases referenced in `config.yml` - download `GeoLite2-Country.mmdb` and `GeoLite2-ASN.mmdb` into each node's `config/` directory. See [Enable Geo-location](/self-host/advanced/enable-geolocation) and [Enable ASN Lookup](/self-host/advanced/enable-asn-lookup).
</Step>
<Step title="Point your load balancer at both nodes">
Configure your load balancer - a cloud load balancer or a self-hosted one such as Traefik - to:
- Route TCP `3000` to both nodes for your Pangolin domain. For example `pangolin.example.com` should resolve to the load balancer, which routes to either node's `:3000` port.
- Route UDP `53` to both nodes for DNS. For example `ns.example.com` should resolve to the load balancer, which routes to either node's `:53/udp` port.
- Health-check the `:80/ping` endpoint on each node, and stop routing to a node that fails it. `:3000/api/v1/` can also be monitored for the Pangolin UI and API
This is what makes the cluster appear as a single, consistent domain to users, and what drives failover when a node goes down.
</Step>
<Step title="Start the cluster">
Bring the database up first, then start **one** Pangolin node - it initializes the database and prints an init token to its logs.
```bash
# On the database host
docker compose up -d
# On Node 1
docker compose up -d
docker compose logs -f pangolin
```
Use the init token from the logs to visit the dashboard and create the first user. Once Node 1 is healthy and you've logged in, bring up Node 2:
```bash
# On Node 2
docker compose up -d
```
Repeat the Node 2 steps for any additional nodes, incrementing the `exit_node_name` and IP placeholders for each.
</Step>
<Step title="Verify the cluster">
- Confirm both nodes report healthy: `curl http://<NODE1_EXTERNAL_IP>/ping` and the same for Node 2
- Confirm DNS delegation resolves: `dig @ns.example.com ns.example.com`
- Confirm the dashboard is reachable at your `dashboard_url` through the load balancer
- Confirm you can create a site and it will report connected
- Create a resource, ensure the certificate generates, and is accessible
</Step>
</Steps>
## Troubleshooting
### Gerbil loopback addressing
If Pangolin can't reach the local Gerbil instance at the IP configured in `--reachableAt`, force it to address the Docker container directly instead:
```yaml title="privateConfig.yml"
gerbil:
local_exit_node_reachable_at: "http://gerbil:3004"
```
## Reference Configuration
<Card icon="github" arrow="true" cta="View reference configuration" href="https://github.com/fosrl/pangolin/tree/main/config/ha-reference">
A complete, working two-node reference configuration on GitHub - including both nodes' Docker Compose and config files - that you can clone and adapt: [github.com/fosrl/pangolin/tree/main/config/ha-reference](https://github.com/fosrl/pangolin/tree/main/config/ha-reference)
</Card>
self-host/advanced/clustering/requirements.mdx
+102
View File
@@ -0,0 +1,102 @@
---
title: "Clustering Requirements"
description: "Hosts, networking, and DNS delegation needed before deploying a Pangolin cluster"
---
<Note>
Clustering is only available in [Enterprise Edition](/self-host/enterprise-edition).
</Note>
Review these requirements before you start deploying. They cover the hosts you need, the ports that must be open, and the DNS records you need to delegate ahead of time. For background on why each piece exists, see [Understanding Clustering](/self-host/advanced/clustering/understanding-clustering).
## Hosts
You need a minimum of **three hosts**:
- **Node 1** -- runs Pangolin, Gerbil, and Traefik
- **Node 2** - runs Pangolin, Gerbil, and Traefik
- **A database host** - runs PostgreSQL and a Redis-compatible server (Valkey, Redis, etc.)
<Tip>
The database host doesn't need to be a dedicated instance. You can run PostgreSQL and Redis however you like - a managed cloud database, an existing cluster, etc. - as long as both nodes can reach them. The only hard requirement is that the Redis-compatible server supports **pub/sub**.
</Tip>
You can add more Pangolin nodes beyond two for additional capacity or regional distribution. The two-node topology in this guide is the minimum for high availability.
For sizing information, see [Choosing a VPS](/self-host/choosing-a-vps) - the same sizing from single pangolin node deployments applies to each node in a cluster.
## Networking
- **Node 1 and Node 2 each need a public, static IP address**, reachable from the internet
- **Node 1 and Node 2 need to be able to address each other** over an internal network
- **You must provide your own HA load balancer** in front of both nodes. It needs to route to both nodes and health-check the `/ping` endpoint on each, removing any node that fails the check
### Required Ports
Configure the following firewall rules on each Pangolin node.
**Inbound**
| Type | Protocol | Port range | Source | Description |
| --- | --- | --- | --- | --- |
| HTTP | TCP | 80 | 0.0.0.0/0 | Ping and redirects |
| HTTPS | TCP | 443 | 0.0.0.0/0 | Pangolin public resources |
| Custom UDP | UDP | 21820 | 0.0.0.0/0 | WireGuard relay port |
| Custom UDP | UDP | 51820 | 0.0.0.0/0 | WireGuard port |
| DNS (UDP) | UDP | 53 | Load balancer | DNS |
| HTTP | TCP | 3000 | Load balancer | Pangolin dashboard UI and API |
| Custom TCP | TCP | 3004 | Self + all other nodes | Gerbil node API |
**Outbound**
| Type | Protocol | Port range | Destination | Description |
| --- | --- | --- | --- | --- |
| All traffic | All | All | 0.0.0.0/0 | Allow all outbound |
## DNS Delegation
Pangolin's built-in DNS server needs to be delegated authority for a nameserver subdomain. Point an NS record at your load balancer, then optionally delegate additional subdomains through it.
<Steps>
<Step title="Create the nameserver record">
Create an A record pointing your chosen nameserver hostname at your load balancer's IP. The examples in this guide use `ns.example.com` - substitute your own domain or subdomain.
| Name | Type | Value |
| --- | --- | --- |
| `ns.example.com` | A | `<LOAD_BALANCER_IP>` |
</Step>
<Step title="Delegate CNAME-based domains (optional)">
If you want to support CNAME delegation for resource domains, delegate a subdomain to your nameserver.
| Name | Type | Value |
| --- | --- | --- |
| `cname.example.com` | NS | `ns.example.com` |
</Step>
<Step title="Delegate site-to-cloud resolution (optional)">
If you want to support site-to-cloud networking - resolving a site's tunnel address by DNS from within a cloud environment - delegate another subdomain the same way.
| Name | Type | Value |
| --- | --- | --- |
| `site.example.com` | NS | `ns.example.com` |
</Step>
</Steps>
These three hostnames map directly to the `dns` section of `privateConfig.yml`, covered in [Deploy a Cluster](/self-host/advanced/clustering/deploy-a-cluster):
```yaml
dns:
enabled: true
nameserver_name: "ns.example.com"
cname_extension: "cname.example.com"
site_extension: "site.example.com" # Optional
```
## Other Requirements
- **Contact email** for Let's Encrypt ACME registration
- **GeoIP databases** - download and keep up to date the MaxMind `GeoLite2-Country.mmdb` and `GeoLite2-ASN.mmdb` databases, placed in each node's `config/` directory. See [Enable Geo-location](/self-host/advanced/enable-geolocation) and [Enable ASN Lookup](/self-host/advanced/enable-asn-lookup)
- **Site type support** - in clustered deployments, only Newt sites are supported. Local sites and basic WireGuard sites are not supported
<Card title="Deploy a Cluster" href="/self-host/advanced/clustering/deploy-a-cluster" icon="server">
Once these requirements are met, follow the full deployment walkthrough.
</Card>
self-host/advanced/clustering/understanding-clustering.mdx
+171
View File
@@ -0,0 +1,171 @@
---
title: "Understanding Clustering"
description: "Architecture and concepts behind running Pangolin as a highly available cluster"
---
<Note>
Clustering is only available in [Enterprise Edition](/self-host/enterprise-edition).
</Note>
For organizations requiring maximum uptime and performance, Pangolin supports clustered deployments where multiple server instances work together as a unified system. This architecture enables regional distribution, automatic failover, and horizontal scaling to handle demanding production workloads.
In a clustered configuration, multiple Pangolin instances operate together, sharing state through a PostgreSQL database and a Valkey (Redis) server. Each instance independently serves user requests, resolves DNS, manages authentication, and coordinates with its own Gerbil instance to support thousands of sites across your organization.
## Architecture
A Pangolin cluster consists of several coordinated components that work together to provide high availability and seamless failover.
```mermaid
flowchart TB
Users(["Users / Browsers"])
Sites(["Site Connectors (Newt)"])
LB["Load Balancer<br/>(your own)<br/>TCP 80 · 443 · 3000 · UDP 53"]
subgraph Node1["Node 1"]
direction TB
G1["Gerbil<br/>WireGuard + SNI proxy"]
T1["Traefik<br/>TLS termination"]
P1["Pangolin<br/>UI · API · DNS · ACME client"]
G1 --- T1 --- P1
end
subgraph Node2["Node 2"]
direction TB
G2["Gerbil<br/>WireGuard + SNI proxy"]
T2["Traefik<br/>TLS termination"]
P2["Pangolin<br/>UI · API · DNS"]
G2 --- T2 --- P2
end
PG[("PostgreSQL<br/>shared state + certs")]
RD[("Valkey / Redis<br/>pub/sub + sessions")]
Users --> LB
LB --> G1
LB --> G2
Sites -. WireGuard tunnel .-> G1
Sites -. WireGuard tunnel .-> G2
G1 <-. cross-node request routing .-> G2
P1 <--> PG
P2 <--> PG
P1 <--> RD
P2 <--> RD
```
### Pangolin Instances
**Purpose**: Serve the web UI and API, resolve DNS, issue and renew TLS certificates, and coordinate cluster state.
**How It Works**:
- Multiple Pangolin instances run simultaneously across different nodes, one per node
- Each instance can independently handle user authentication and requests
- All instances share state through the PostgreSQL database and Valkey
- Each instance embeds a DNS server used for resource resolution and ACME DNS-01 challenges
- Only one instance in the cluster should be configured as the ACME client. It issues and renews certificates and stores them encrypted in PostgreSQL. Every other instance reads the same certificates from the database
**High Availability**: A load balancer sits in front of all Pangolin instances. If any instance goes down, the load balancer automatically routes traffic to healthy nodes, ensuring the UI, API, and DNS remain accessible from the same domain without interruption.
### PostgreSQL Database
**Purpose**: Store all persistent cluster state in a centralized, shared database.
**How It Works**:
- All Pangolin instances connect to the same shared PostgreSQL database
- Stores user accounts, site configurations, resources, access policies, and organizational settings
- Certificates are stored encrypted in the database for security
- Changes made through any instance are immediately available cluster-wide
**High Availability**: Database replication and backup strategies ensure data persistence and availability across the cluster. See [Database Options](/self-host/advanced/database-options) for general PostgreSQL configuration.
### Valkey (Redis)
**Purpose**: Provide real-time state synchronization between cluster nodes.
**How It Works**:
- Pub/sub messaging handles cross node messaging for websocket command control
- Handles caching for the cluster
- Any Redis-compatible server works, as long as it supports pub/sub
**High Availability**: Ensures that session and connection information remains available even when individual nodes fail.
### Traefik Instances
**Purpose**: Route HTTP/HTTPS traffic to resources and terminate TLS connections.
**How It Works**:
- Each cluster node runs its own Traefik instance
- Pangolin writes router configuration and certificates to a shared volume with Traefik (`file_mode`) instead of Traefik scraping the Pangolin API directly, since Traefik can only load certificates from files
- Each node's Pangolin instance pulls the certificate from the database to that shared volume so its local Traefik can read it
- Sits behind Gerbil, which runs an SNI proxy for traffic routing
**High Availability**: Multiple Traefik instances ensure traffic routing continues even if individual nodes fail.
### Gerbil Instances
**Purpose**: Manage WireGuard tunnels to site connectors and route traffic between cluster nodes.
**How It Works**:
- Each Pangolin instance runs alongside its own Gerbil tunnel manager
- Handles WireGuard VPN connections from site connectors (Newt)
- Site connectors can establish tunnels to any available Gerbil instance
- Every Gerbil instance is made aware of the other trusted nodes in the cluster
- When a request lands on the node that isn't holding the relevant tunnel, Gerbil routes it to the correct node instead of dropping it - this covers the case where DNS caching sends a client to the "wrong" node
**High Availability**: Distributed tunnel management ensures connectivity remains available even if individual Gerbil instances fail, with automatic cross-node failover.
### Load Balancer
**Purpose**: Distribute incoming traffic across healthy Pangolin instances.
**How It Works**:
- Sits in front of all cluster nodes, fronting the dashboard/API port, HTTP/HTTPS resource ports, and DNS
- Monitors instance health (the `/ping` endpoint) and routes traffic only to available nodes
- Ensures all traffic reaches the cluster through a single, consistent domain
- Does NOT handle resource routing - these are directed to the correct node directly by the DNS server
**High Availability**: Essential for continuous access to the Pangolin UI, API, and DNS regardless of individual instance failures. **You must provide your own HA load balancer** in front of the cluster - this can be a cloud load balancer or a self-hosted one.
## Traffic Flow
Understanding how requests flow through the cluster helps clarify how these components work together:
1. **User access**: Users access the Pangolin UI/API through the load balancer, which routes to any healthy Pangolin instance
2. **Resource requests**: When accessing a resource, DNS resolves to the node that the site is connected to, and the request is routed to that node's Gerbil instance
3. **Cross-node routing**: If DNS caching or the load balancer points to a node that isn't holding the relevant tunnel, Gerbil routes the request to the correct node
4. **Tunnel routing**: Gerbil receives the request and forwards it to the local Traefik instance
5. **TLS termination**: Traefik terminates TLS using the certificate synced to its shared volume, then proxies the request to the site connector's tunnel
## How Failover Works
When a node fails, the load balancer stops routing to it and traffic continues to flow through the remaining healthy nodes. This keeps the API, UI, and DNS running without interruption.
The sites connected to the failed node will being detecting ping failures and will initiate a reconnection requests to connect to a different online node. Once connected, the DNS will update the resource resolution to point to the new node, and traffic will continue without user disruption. There may be brief periods of downtime while the site connector detects the failure and reconnects to a healthy node, but this is typically only a few seconds.
## Benefits of Clustering
**High Availability**: Eliminate single points of failure. If one server instance fails, traffic automatically routes to healthy nodes without user disruption.
**Regional Distribution**: Deploy servers closer to your users and sites across different geographic regions to minimize latency and improve performance.
**Horizontal Scaling**: Add more server instances to handle increased load as your organization grows, without architectural changes.
**Zero-Downtime Updates**: Perform rolling updates by taking nodes offline one at a time while others continue serving traffic.
**Simplified Infrastructure**: DNS resolution and certificate management are built into Pangolin itself, so there's no separate DNS or certificate-issuing service to deploy, scale, or keep highly available on top of the cluster.
**Dynamic Failover**: Automatic traffic routing between nodes and the load balancer ensures resources remain accessible when nodes fail.
## Enterprise Support
Clustered deployments require careful planning around database replication, Valkey configuration, network topology, DNS delegation, and monitoring. For organizations interested in clustering for high availability or regional distribution, please [contact our enterprise team](https://pangolin.net/contact) to discuss your requirements and receive implementation guidance support. A support contract is not required for deployment.
<CardGroup cols={2}>
<Card title="Requirements" href="/self-host/advanced/clustering/requirements" icon="list-check">
Review the hosts, network, and DNS delegation a cluster needs before you deploy.
</Card>
<Card title="Deploy a Cluster" href="/self-host/advanced/clustering/deploy-a-cluster" icon="server">
Follow a complete walkthrough for standing up a two-node cluster.
</Card>
</CardGroup>