diff --git a/docs.json b/docs.json index 1b6084f..697182c 100644 --- a/docs.json +++ b/docs.json @@ -193,7 +193,7 @@ ] }, { - "group": "Newt", + "group": "Site (newt)", "pages": [ "self-host/manual/kubernetes/newt/helm", "self-host/manual/kubernetes/newt/kustomize", diff --git a/self-host/manual/kubernetes/choose-method.mdx b/self-host/manual/kubernetes/choose-method.mdx index ac18077..a736062 100644 --- a/self-host/manual/kubernetes/choose-method.mdx +++ b/self-host/manual/kubernetes/choose-method.mdx @@ -1,180 +1,53 @@ --- -title: "Choose a Method" -description: "Choose the right Kubernetes installation workflow for Pangolin and Newt." +title: "Choose an Installation Path" +description: "Choose the Kubernetes deployment workflow for Pangolin and Sites (Newt)." --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; +Use this page to pick the right Kubernetes guide for your deployment workflow. -This page helps you choose the right Kubernetes workflow for installing and managing Pangolin and related components. +These guides assume you are already familiar with Kubernetes and the deployment tools listed below. -## Quick decision table +If you are new to Kubernetes, start with the [official Kubernetes learning resources](https://kubernetes.io/docs/tutorials/kubernetes-basics/) first. Then review the [Prerequisites](/self-host/manual/kubernetes/prerequisites) guide to check your cluster, tools, and setup. -| If you... | Use | Why | +## Installation paths + +| Path | Use when | Start here | | --- | --- | --- | -| Want the recommended Kubernetes install path | **Helm** | Standard chart-based workflow for installing, upgrading, and uninstalling releases | -| Need environment-specific overlays or manifest customization | **Kustomize** | Patch and reuse Kubernetes manifests without a separate templating language | -| Already use Argo CD or want GitOps with a web UI | **Argo CD** | Git-driven reconciliation, sync status, drift detection, and optional auto-sync | -| Already use Flux or want GitOps defined through Kubernetes CRDs | **Flux** | Declarative reconciliation with resources such as `HelmRelease` and `Kustomization` | -| Need to manage several Helm releases together | **Helmfile** | Declarative orchestration for multiple Helm releases and shared values | +| Helm | You want the standard chart-based installation path for Pangolin or Sites (Newt). | [Helm Quick-Start](/self-host/manual/kubernetes/helm) | +| Kustomize | You want manifest overlays, for example for environment-specific configuration, patches, or rendered manifests that can be reviewed before applying. | [Kustomize Quick-Start](/self-host/manual/kubernetes/kustomize) | +| Argo CD | You already use Argo CD and want to deploy Pangolin or Sites (Newt) through a Kubernetes-native GitOps workflow. | [Argo CD Guide](/self-host/manual/kubernetes/gitops/argocd) | +| Flux | You already use Flux and want to manage Pangolin or Sites (Newt) through `HelmRelease` or `Kustomization` resources. | [Flux Guide](/self-host/manual/kubernetes/gitops/flux) | +| Helmfile | You want to manage multiple related Helm releases as one stack. | [Helmfile Guide](/self-host/manual/kubernetes/helmfile) | -## Detailed method descriptions +## Recommended starting point -### Use Helm if... +For most Kubernetes deployments, start with Helm. Use the GitOps guides only if Argo CD or Flux is already part of your deployment workflow. -- You want the recommended Kubernetes install workflow for Pangolin or Newt. -- You want a straightforward chart-based install. -- You manage releases manually or through CI/CD. -- You want normal Helm release operations such as install, upgrade, rollback, and uninstall. -- You are comfortable managing configuration through `values.yaml`. - -Helm is the default choice for most Kubernetes installations. It packages Kubernetes resources into versioned charts and manages releases in the cluster. - -**Get started**: [Helm Quick-Start](/self-host/manual/kubernetes/helm) - -### Use Kustomize if... - -- You need environment-specific overlays for dev, staging, or production. -- You want to patch Kubernetes manifests without using a templating language. -- You prefer a manifest-driven workflow. -- You want to keep rendered or curated manifests in Git. -- You are comfortable with `kubectl apply -k` or `kustomize build`. - -Kustomize works well when you want a shared base with small environment-specific changes. It can be used with curated manifests, generated manifests, or GitOps tools. - -**Common scenario**: Keep a base deployment and apply overlays for each environment. - -**Get started**: [Kustomize Quick-Start](/self-host/manual/kubernetes/kustomize) - -### Use Argo CD if... - -- Your Git repository should be the source of truth. -- You want a web UI for application status, sync state, and troubleshooting. -- You want drift detection when the live cluster state differs from the desired state. -- You want manual sync, automated sync, or self-healing behavior. -- You already use Argo CD for other applications. - -Argo CD reconciles applications from a declared source into the cluster. It can use Helm charts, Kustomize overlays, plain YAML, Jsonnet, or configured plugins as sources. - -When Argo CD deploys a Helm chart, Helm is used to render the manifests. The application lifecycle is then managed by Argo CD, not by the local `helm` CLI. - -**Argo CD can deploy**: - -- Helm charts -- Kustomize overlays -- Plain YAML manifests -- Jsonnet or custom config-management plugin output - -**Get started**: [Argo CD Guide](/self-host/manual/kubernetes/gitops/argocd) - -### Use Flux if... - -- You want GitOps managed through Kubernetes custom resources. -- You already use Flux for other workloads. -- You want Helm releases reconciled by a controller. -- You want Kustomize overlays reconciled from Git. -- You prefer a lightweight workflow without depending on a central web UI. - -Flux defines sources and desired state as Kubernetes resources. Typical resources include `GitRepository`, `HelmRepository`, `OCIRepository`, `Kustomization`, and `HelmRelease`. - -**Flux can deploy**: - -- Helm charts with `HelmRepository` and `HelmRelease` -- OCI-based Helm charts with `OCIRepository` and `HelmRelease` -- Kustomize overlays with `GitRepository` and `Kustomization` -- Plain manifests through a Flux `Kustomization` - -**Get started**: [Flux Guide](/self-host/manual/kubernetes/gitops/flux) - -### Use Helmfile if... - -- You need to manage multiple Helm releases as one deployment stack. -- You want to install Pangolin together with supporting components. -- You want one declarative file for releases, values files, and release ordering. -- You prefer running one controlled workflow instead of several manual `helm upgrade --install` commands. - -Helmfile is a declarative wrapper around Helm. It does not replace Helm; it calls Helm to apply the declared releases. - -**Common scenario**: One Helmfile manages supporting components such as an ingress controller, certificate management, database components, Pangolin, and Newt. - -**Get started**: [Helmfile Guide](/self-host/manual/kubernetes/helmfile) - -## Important clarifications - -### Argo CD and Flux are not Helm replacements - -Argo CD and Flux are delivery and reconciliation tools. They do not replace Helm or Kustomize. - -- **Helm** packages and renders Kubernetes resources from charts. -- **Kustomize** customizes Kubernetes manifests through bases, overlays, and patches. -- **Argo CD** reconciles applications from Git, Helm repositories, OCI registries, or other configured sources. -- **Flux** reconciles sources and workloads through Kubernetes custom resources such as `HelmRelease` and `Kustomization`. - -You can use Argo CD or Flux with Helm charts, Kustomize overlays, or plain manifests. - -### OCI is not a separate install method - -OCI (Open Container Initiative) describes a chart distribution format, not a separate deployment workflow. - -For Pangolin and Newt, OCI chart publishing is available in GHCR: - -- Newt: `oci://ghcr.io/fosrl/helm-charts/newt` (for example `1.4.0`) -- Pangolin: `oci://ghcr.io/fosrl/helm-charts/pangolin` (for example `0.1.0-alpha.0`) - -You still choose the same deployment method (Helm directly, or GitOps with Argo CD/Flux). OCI only changes where charts are pulled from. - -Classic Helm repository flow is still valid: - -```bash -helm repo add fossorial https://charts.fossorial.io -helm repo update fossorial -helm install my-newt fossorial/newt -helm install my-pangolin fossorial/pangolin -``` - -OCI workflow with Helm: - -```bash -helm pull oci://ghcr.io/fosrl/helm-charts/newt --version 1.4.0 -helm pull oci://ghcr.io/fosrl/helm-charts/pangolin --version 0.1.0-alpha.0 -``` - -```bash -helm install my-newt oci://ghcr.io/fosrl/helm-charts/newt \ - --version 1.4.0 - -helm install my-pangolin oci://ghcr.io/fosrl/helm-charts/pangolin \ - --version 0.1.0-alpha.0 -``` - -### Raw YAML is not a separate primary workflow - -This documentation does not provide a dedicated raw-YAML installation path. - -Raw manifests are still possible: - -- Render a Helm chart with `helm template`. -- Render Kustomize overlays with `kustomize build` or `kubectl kustomize`. -- Apply generated manifests with `kubectl apply -f`. -- Reconcile plain manifests with Argo CD or Flux. - -For most users, Helm, Kustomize, Argo CD, or Flux is easier to maintain than applying standalone YAML files manually. +Kustomize and Helmfile are useful when you need more control over manifests, overlays, or multiple coordinated releases. ## Next steps - Review cluster, tooling, ingress, DNS, storage, and secret requirements. + Review the required cluster, ingress, DNS, storage, and secret setup. - Install Pangolin or Newt with the recommended chart-based workflow. + Install Pangolin or Sites (Newt) with the standard chart-based workflow. - Use bases, overlays, and patches for manifest-driven deployments. + Use overlays and patches for manifest-based deployments. - - Deploy and reconcile Pangolin or Newt with Argo CD or Flux. + + Deploy Pangolin or Sites (Newt) with Argo CD. + + + Deploy Pangolin or Sites (Newt) with Flux. + + + Manage multiple Helm releases together. diff --git a/self-host/manual/kubernetes/gitops/argocd.mdx b/self-host/manual/kubernetes/gitops/argocd.mdx index 14a04b3..59a4867 100644 --- a/self-host/manual/kubernetes/gitops/argocd.mdx +++ b/self-host/manual/kubernetes/gitops/argocd.mdx @@ -10,50 +10,6 @@ import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; Argo CD is a declarative GitOps tool that continuously syncs your cluster state to your Git repository. This guide covers installing Pangolin and Newt using Argo CD. -## Argo CD overview - -Argo CD watches your Git repository (or Helm chart repository) and automatically reconciles Kubernetes resources to match the desired state defined in Git. - -**Key concepts**: - -- **Application**: Argo CD custom resource that defines what to deploy, where, and how -- **Helm source**: Argo CD uses Helm to render charts; you provide values -- **Kustomize source**: Argo CD uses Kustomize to build manifests -- **Sync**: Process of applying desired state to the cluster -- **Drift**: When cluster state diverges from Git (Argo CD can detect and correct) - -## Prerequisites - -- Argo CD installed in your cluster (in `argocd` namespace, typically) -- Helm repo configured: `helm repo add fossorial https://charts.fossorial.io` -- Git repository with Argo CD configuration (optional, can use chart repo as source) -- Newt auth secret (if installing Newt) - -## Install Argo CD - -If you don't have Argo CD yet: - -```bash -# Create namespace -kubectl create namespace argocd - -# Install Argo CD -helm repo add argo https://argoproj.github.io/argo-helm -helm repo update argo -helm install argocd argo/argo-cd -n argocd -``` - -Access the Argo CD UI: - -```bash -# Port-forward -kubectl port-forward -n argocd svc/argocd-server 8080:443 - -# Visit https://localhost:8080 -# Default username: admin -# Password: kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d -``` - ## Install Pangolin with Argo CD using Helm ### Step 1: Create Pangolin namespace diff --git a/self-host/manual/kubernetes/gitops/flux.mdx b/self-host/manual/kubernetes/gitops/flux.mdx index d4c823f..637659f 100644 --- a/self-host/manual/kubernetes/gitops/flux.mdx +++ b/self-host/manual/kubernetes/gitops/flux.mdx @@ -10,22 +10,6 @@ import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; Flux is a declarative GitOps tool that uses Kubernetes-native Custom Resources to manage deployments. This guide covers installing Pangolin and Newt using Flux. -## Flux overview - -Flux watches your Git repository and continuous reconciles cluster state using Kubernetes CRDs: - -- **HelmRepository**: Defines a Helm chart repository -- **HelmRelease**: Declaratively manages a Helm chart deployment -- **GitRepository**: References a Git repository -- **Kustomization**: Reconciles Kustomize overlays -- **OCIRepository**: References an OCI-based container registry (for Helm charts) - -**Key benefits**: - -- Native Kubernetes reconciliation (no separate UI needed, though one exists) -- Lightweight footprint -- Excellent for multi-cluster deployments -- Declarative everything: sources, releases, dependencies ## Flux prerequisites @@ -523,32 +507,6 @@ clusters/ Each environment's HelmRelease uses environment-specific values. -## Important notes - -### CRD management - -When using Flux with Helm charts that include CRDs: - -```yaml -spec: - install: - crds: Create # Create CRDs on first install - upgrade: - crds: CreateReplace # Update CRDs on upgrade -``` - -### Namespace creation - -Flux automatically creates namespaces if they don't exist. Ensure appropriate RBAC. - -### GitOps best practices - -- Use branches for different environments -- Protect production branches with review requirements -- Store secrets using sealed-secrets or external-secrets -- Track all changes in Git -- Use consistent naming conventions - ## Next steps diff --git a/self-host/manual/kubernetes/gitops/overview.mdx b/self-host/manual/kubernetes/gitops/overview.mdx index f5cbc57..6c3a547 100644 --- a/self-host/manual/kubernetes/gitops/overview.mdx +++ b/self-host/manual/kubernetes/gitops/overview.mdx @@ -1,351 +1,76 @@ --- title: "GitOps Overview" -description: "Git-driven Kubernetes deployments for Pangolin and Newt using Argo CD or Flux." +description: "Deploy Pangolin and Sites (Newt) with GitOps workflows such as Argo CD or Flux." --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; +Use GitOps when Pangolin and Sites (Newt) should be reconciled from Git instead of being installed manually from a local shell. +Can be used together with Blueprints — see [Blueprint config reference](/self-host/advanced/config-file) for details. -GitOps is a declarative approach to infrastructure management where your Git repository is the single source of truth for cluster state. Changes go through Git; the GitOps tool automatically reconciles the cluster to match. +These guides assume you already use, or plan to use, a GitOps controller such as Argo CD or Flux. +General GitOps concepts such as reconciliation, desired state, and Git-driven workflows are outside the scope of this documentation. Refer to your GitOps controller's documentation for those concepts. -## What is GitOps? +## Supported GitOps paths -GitOps reconciliation loop: + + + Deploy Pangolin or Sites (Newt) with Argo CD Applications. + + + Deploy Pangolin or Sites (Newt) with Flux HelmRelease or Kustomization resources. + + - - - **Git repository** contains desired state (manifests, Helm values, Kustomize overlays). - - - **GitOps tool** (Argo CD or Flux) watches the Git repository. - - - The controller detects new commits or drift between Git and the live cluster. - - - The controller syncs cluster state to match Git state. - - - **Result**: Cluster stays aligned with the declared Git configuration. - - +## What GitOps manages -## Benefits +A GitOps workflow can reconcile the same deployment inputs used by the other Kubernetes guides: -- **Version control**: All infrastructure changes tracked in Git -- **Audit trail**: See who changed what and when -- **Rollback**: Revert to previous state by reverting Git commits -- **Automation**: No manual `kubectl apply` commands needed -- **Drift detection**: Automatic alerts if cluster diverges from Git +| Input | Used for | +| --- | --- | +| Helm chart values | Configure Pangolin, controller mode, database mode, ingress, Sites, and related components. | +| Kustomize overlays | Patch or compose rendered manifests for environment-specific deployments. | +| Kubernetes Secrets | Provide credentials, TLS material, database connection details, or Site connector credentials. | +| Custom resources | Manage Argo CD Applications, Flux HelmReleases, Flux Kustomizations, or related controller resources. | -## GitOps tools for Kubernetes +## Recommended layout -### Argo CD +Keep the Pangolin and Site configuration close to the cluster or environment that owns it. -- **UI**: Web-based dashboard for monitoring and manual syncs -- **Approach**: External reconciler (watches Git, applies to cluster) -- **Supports**: Helm charts, Kustomize overlays, raw YAML -- **Best for**: Teams who want GitOps with a UI, hybrid manual/automated workflows - -**Use Argo CD if**: -- You want a visual dashboard -- You need frequent manual sync capabilities -- You're already using Argo CD for other workloads - -See: [Argo CD Install Guide](/self-host/manual/kubernetes/gitops/argocd) - -### Flux - -- **CRDs**: Kubernetes-native Custom Resources (HelmRelease, Kustomization, GitRepository) -- **Approach**: Declarative reconciliation using Kubernetes resources -- **Supports**: Helm charts, Kustomize overlays, raw YAML, OCI registries -- **Best for**: Teams who want declarative Kubernetes-way GitOps, lightweight controllers - -**Use Flux if**: -- You prefer Kubernetes-native CRDs -- You want a lightweight, modern GitOps tool -- You're already using Flux for other workloads - -See: [Flux Install Guide](/self-host/manual/kubernetes/gitops/flux) - -## Recommended repository structure - -For multi-environment Pangolin/Newt deployments, organize your Git repository like this: - -``` -my-org/infrastructure/ +```text +infrastructure/ ├── clusters/ │ ├── production/ │ │ ├── pangolin/ -│ │ │ ├── values.yaml -│ │ │ └── kustomization.yaml (if using Kustomize) -│ │ └── newt/ -│ │ ├── values.yaml -│ │ └── kustomization.yaml +│ │ └── sites/ │ ├── staging/ │ │ ├── pangolin/ -│ │ └── newt/ +│ │ └── sites/ │ └── dev/ │ ├── pangolin/ -│ └── newt/ -├── apps/ -│ ├── pangolin/ -│ │ ├── helm/ -│ │ │ ├── values-base.yaml -│ │ │ ├── values-prod.yaml -│ │ │ └── values-staging.yaml -│ │ └── kustomize/ -│ │ ├── base/ -│ │ └── overlays/ -│ └── newt/ -│ ├── helm/ -│ └── kustomize/ -└── .gitignore +│ └── sites/ +└── shared/ + ├── pangolin/ + └── sites/ ``` -**Pattern**: -- `clusters/` → environment-specific configuration -- `apps/` → shared, reusable application configuration -- Environment overlays layer on top of app definitions - -## Secrets in GitOps - - -**Never commit plaintext secrets to Git.** Use secret management tools instead. - - -Options for managing secrets in GitOps: - -### Sealed Secrets - -- **Tool**: [sealed-secrets](https://github.com/bitnami-labs/sealed-secrets) -- **How**: Encrypt secrets with cluster-specific key; safe to commit encrypted secrets -- **Decrypt**: Only the cluster can decrypt (uses private key) - -```bash -# Encrypt a secret -echo -n mypassword | kubeseal -f - > secret.yaml - -# Git tracks encrypted secret.yaml -# Cluster auto-decrypts on apply -``` - -### External Secrets Operator - -- **Tool**: [external-secrets](https://external-secrets.io/) -- **How**: Reference secrets stored in external vault (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault) -- **Git**: Stores reference only, not secret values - -```yaml -apiVersion: external-secrets.io/v1beta1 -kind: SecretStore -metadata: - name: vault-backend -spec: - provider: - vault: - server: "https://vault.example.com" -``` - -### SOPS (Secrets Operations) - -- **Tool**: [SOPS](https://github.com/mozilla/sops) -- **How**: Encrypt YAML files; decrypt at deploy time -- **Git**: Stores encrypted files - -```bash -sops --encrypt secrets.yaml > secrets.enc.yaml -# Commit secrets.enc.yaml; tool decrypts on apply -``` - -### Cloud Provider Secrets - -- **AWS**: Use AWS Secrets Manager or Parameter Store with IRSA (IAM Roles for Service Accounts) -- **Azure**: Use Azure Key Vault with pod identity -- **GCP**: Use Google Secret Manager with workload identity - - -Choose a secret management strategy **before** setting up GitOps. Seal secrets once; keep approach consistent. - - -## GitOps workflow example - -### 1. Set up Git repository - -```bash -git clone https://github.com/my-org/infrastructure.git -cd infrastructure -mkdir -p clusters/production/pangolin -cd clusters/production/pangolin -``` - -### 2. Create configuration - -```bash -# values.yaml with Pangolin config -cat > values.yaml <=0.1.0" # auto-upgrade to latest 0.1.x -``` - -### Drift detection and remediation - -Argo CD and Flux both support continuous drift detection: - -- **Argo CD**: Detects drift on demand or continuously; can auto-sync on drift -- **Flux**: Reconciles on interval; rolls back manual cluster changes +Use environment-specific directories for values, patches, and secrets that differ between clusters. Use shared directories only for reusable configuration that should stay the same across environments. ## Next steps - - - - + + Create Argo CD Applications for Pangolin and Sites (Newt). + + + Create Flux sources, HelmReleases, or Kustomizations for Pangolin and Sites (Newt). + + + Compare the supported Kubernetes deployment paths. + + + Review cluster, networking, storage, RBAC, and resource requirements. + diff --git a/self-host/manual/kubernetes/helm.mdx b/self-host/manual/kubernetes/helm.mdx index 8bc5069..cb16438 100644 --- a/self-host/manual/kubernetes/helm.mdx +++ b/self-host/manual/kubernetes/helm.mdx @@ -60,7 +60,7 @@ It is recommended to create the namespace explicitly before installation. This a For detailed installation steps, see: * [Pangolin Helm Quick-Start](/self-host/manual/kubernetes/pangolin/helm) — Install Pangolin -* [Newt Helm Quick-Start](/self-host/manual/kubernetes/newt/helm) — Install Newt +* [Site (Newt) Helm Quick-Start](/self-host/manual/kubernetes/newt/helm) — Install Site (Newt) ## Install command patterns @@ -340,15 +340,6 @@ helm uninstall newt --namespace pangolin Uninstalling a Helm release does not always remove persistent volumes, externally managed secrets, DNS records, certificates, or cloud load balancers. Review the namespace and related cluster resources before deleting data. -## Using Helm with GitOps - -Helm charts can also be installed and reconciled through GitOps tools. - -* [Argo CD](/self-host/manual/kubernetes/gitops/argocd) can deploy Helm charts from a Helm repository, Git repository, or OCI source. -* [Flux](/self-host/manual/kubernetes/gitops/flux) can reconcile Helm charts through `HelmRepository`, `OCIRepository`, and `HelmRelease`. - -In these workflows, Helm is the chart format. The GitOps controller manages reconciliation. - ## Troubleshooting For component-specific troubleshooting, see: @@ -381,8 +372,8 @@ kubectl logs -n pangolin Install Pangolin with the Helm chart. - - Install Newt with the Helm chart. + + Install Site (Newt) with the Helm chart. Configure Pangolin chart values for your cluster. diff --git a/self-host/manual/kubernetes/kustomize.mdx b/self-host/manual/kubernetes/kustomize.mdx index 8651e2d..f094e2b 100644 --- a/self-host/manual/kubernetes/kustomize.mdx +++ b/self-host/manual/kubernetes/kustomize.mdx @@ -19,18 +19,6 @@ Use Kustomize when you need: - a manifest-driven workflow for GitOps tools - small changes on top of a shared base without maintaining separate full manifests -## When to use Kustomize - -Use Kustomize if: - -- you want to manage rendered Pangolin or Newt manifests in Git -- you need different overlays for different environments -- your team prefers reviewing concrete Kubernetes manifests -- you use Argo CD or Flux with Kustomize sources -- you want to patch generated manifests without forking the Helm chart - -For a single environment or a first installation, [Helm](/self-host/manual/kubernetes/helm) is usually simpler. - ## Supported workflow The chart repository does not provide native Kustomize bases. Use this workflow instead: @@ -289,15 +277,6 @@ Apply after review: kubectl apply -k overlays/prod ``` -## Kustomize with GitOps - -Kustomize overlays work well with GitOps tools. - -* [Argo CD](/self-host/manual/kubernetes/gitops/argocd) can reconcile a Kustomize overlay path directly. -* [Flux](/self-host/manual/kubernetes/gitops/flux) can reconcile a Kustomize overlay with a `Kustomization` resource. - -In GitOps workflows, the controller applies the overlay. Do not also apply the same overlay manually unless you are debugging. - ## Important considerations ### Namespace handling @@ -400,4 +379,3 @@ kubectl get events -n pangolin --sort-by=.lastTimestamp Troubleshoot Pangolin deployments on Kubernetes. -``` diff --git a/self-host/manual/kubernetes/newt/configuration.mdx b/self-host/manual/kubernetes/newt/configuration.mdx index e376c69..ee5eb2a 100644 --- a/self-host/manual/kubernetes/newt/configuration.mdx +++ b/self-host/manual/kubernetes/newt/configuration.mdx @@ -1,5 +1,5 @@ --- -title: "Newt Configuration" +title: "Configuration" description: "Configuration reference for Newt Kubernetes deployments." --- @@ -189,7 +189,7 @@ Chart `1.4.0` also includes `auth.createSecret` and `auth.envVarsDirect` modes f -Newt 1.11+ supports provisioning-based installs. +Provisioning supports installs where Newt bootstraps credentials from a provisioning key. Use provisioning when Newt should bootstrap credentials from a provisioning key instead of using a static `NEWT_ID` and `NEWT_SECRET`. @@ -418,30 +418,6 @@ newtInstances: Use Secrets for certificates and sensitive script inputs. Avoid inline private keys or credentials in values files. - - - - -By default, Newt runs without native WireGuard mode. - -```yaml -global: - nativeMode: - enabled: false - -newtInstances: - - name: main-tunnel - useNativeInterface: false -``` - -Native mode requires elevated privileges. - -When native mode or `useNativeInterface` is enabled, Newt runs as root with privileged settings and capabilities such as `NET_ADMIN` and `SYS_MODULE`. - - -Only enable native WireGuard mode if your cluster policy allows privileged workloads and you understand the security impact. - - @@ -723,7 +699,7 @@ helm upgrade --install newt fossorial/newt \ --set 'newtInstances[0].auth.existingSecretName=newt-auth' ``` -See [Newt Helm](/self-host/manual/kubernetes/newt/helm) for the installation flow. +See [Site (newt) Helm](/self-host/manual/kubernetes/newt/helm) for the installation flow. ### Kustomize @@ -776,23 +752,6 @@ spec: See [GitOps](/self-host/manual/kubernetes/gitops/overview) for GitOps guidance. -## Production checklist - -Before deploying Newt to production: - -- [ ] Create the namespace before installation and apply required labels or annotations. -- [ ] Store credentials in Kubernetes Secrets. -- [ ] Avoid inline plaintext credentials. -- [ ] Use provisioning only with writable config persistence. -- [ ] Use a PVC for durable provisioning state. -- [ ] Keep native WireGuard mode disabled unless privileged workloads are allowed. -- [ ] Confirm the Pangolin endpoint is reachable from the Newt pod. -- [ ] Confirm TLS certificates are valid for the Pangolin endpoint. -- [ ] Set resources based on expected traffic. -- [ ] Configure NetworkPolicy rules if your cluster enforces network isolation. -- [ ] Enable metrics only when you have a scraping path. -- [ ] Review RBAC settings before upgrading from older chart versions. - ## Next steps diff --git a/self-host/manual/kubernetes/newt/helm.mdx b/self-host/manual/kubernetes/newt/helm.mdx index fa6197d..329d808 100644 --- a/self-host/manual/kubernetes/newt/helm.mdx +++ b/self-host/manual/kubernetes/newt/helm.mdx @@ -1,6 +1,6 @@ --- -title: "Newt Helm" -description: "Quick-start guide for installing Newt on Kubernetes using Helm." +title: "Helm" +description: "Quick-start guide for installing Site (newt) on Kubernetes using Helm." --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; @@ -8,20 +8,9 @@ import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; -Newt is the site connector used to expose private resources through Pangolin. It runs close to the resources you want to publish and connects back to Pangolin. +This guide installs and manages Site (newt) in Kubernetes using Helm. -Newt is a user-space WireGuard tunnel client and TCP/UDP proxy. It does not require users to manage WireGuard tunnels or NAT rules manually. - -## Version matrix - -| Item | Value | -| --- | --- | -| Chart version | `1.4.0` | -| App version | `1.12.3` | -| Kubernetes version | `>=1.30.14-0` | -| Default image tag | `1.12.3` | - -Newt chart `1.4.0` includes Newt `1.12.3` and supports Kubernetes `>=1.30.14-0`. +See [Version Matrix](https://github.com/fosrl/helm-charts/VERSION_MATRIX.md) for chart and default app version references. ## What the chart supports @@ -29,13 +18,12 @@ The Newt chart can deploy one or more Newt instances through `newtInstances[]`. Newt chart `1.4.0` includes support for: -- Newt 1.11+ provisioning with `NEWT_PROVISIONING_KEY` and `NEWT_NAME` +- provisioning with `NEWT_PROVISIONING_KEY` and `NEWT_NAME` - legacy credential installs with `NEWT_ID` and `NEWT_SECRET` - existing Kubernetes Secrets for production credentials - writable config persistence with `emptyDir` or an existing PVC - optional metrics, PodMonitor, ServiceMonitor, and PrometheusRule - optional NetworkPolicy -- optional native WireGuard mode - multi-instance deployments with per-instance overrides The chart README lists these features for version `1.4.0`. @@ -50,7 +38,7 @@ Before installing Newt, you need: - a reachable Pangolin instance - either: - Newt credentials from Pangolin: `NEWT_ID` and `NEWT_SECRET` - - or a provisioning key for Newt 1.11+ provisioning + - or a provisioning key for provisioning installs The chart quickstart lists Kubernetes `>=1.30.14`, Helm 3.x, configured `kubectl`, and Newt credentials from Pangolin as prerequisites. @@ -63,7 +51,7 @@ Newt chart `1.4.0` supports three credential patterns: | Method | Recommended for | Notes | | --- | --- | --- | | Existing Secret | Production | Credentials are stored in a Kubernetes Secret created outside Helm | -| Provisioning key | Newt 1.11+ provisioning | Requires writable config persistence | +| Provisioning key | Provisioning installs | Requires writable config persistence | | Inline values | Local testing only | Credentials may be stored in Helm release history | For production, use `auth.existingSecretName` or a GitOps-safe secret workflow. The chart values explicitly warn that inline credentials can be stored in Helm release history and recommend existing Secrets for production. @@ -204,7 +192,7 @@ kubectl wait --for=condition=ready pod \ ## Quick install with provisioning key -Newt 1.11+ supports provisioning-based installs. Use this when you want Newt to bootstrap credentials from a provisioning key. +Provisioning-based installs bootstrap credentials from a provisioning key. Provisioning requires writable config persistence so Newt can store the generated configuration. The chart quickstart explicitly notes that provisioning requires a writable `CONFIG_FILE` target and that the chart provides this through `newtInstances[x].configPersistence`. ([GitHub][3]) @@ -352,16 +340,6 @@ kubectl create secret generic newt-auth-site-b \ The chart values include `newtInstances[]`, per-instance namespace settings, and per-instance service account overrides. ([GitHub][2]) -### Security defaults - -By default, Newt runs without native WireGuard mode. - -The chart values describe the default non-native mode as non-root with privilege escalation disabled, read-only root filesystem, and dropped capabilities. Native WireGuard mode requires a privileged container with capabilities such as `NET_ADMIN` and `SYS_MODULE`. ([GitHub][2]) - - -Only enable native WireGuard mode if you understand the required privileges and your cluster policy allows them. - - ### RBAC Newt chart `1.4.0` defaults `rbac.create` to `false`. Enable RBAC only when your selected Newt configuration requires Kubernetes API permissions. diff --git a/self-host/manual/kubernetes/newt/kustomize.mdx b/self-host/manual/kubernetes/newt/kustomize.mdx index d031f14..92a9e0d 100644 --- a/self-host/manual/kubernetes/newt/kustomize.mdx +++ b/self-host/manual/kubernetes/newt/kustomize.mdx @@ -1,5 +1,5 @@ --- -title: "Newt Kustomize" +title: "Kustomize" description: "Deploy Newt on Kubernetes using Helm-rendered manifests and Kustomize overlays." --- @@ -559,15 +559,6 @@ kubectl get all --namespace pangolin kubectl get events --namespace pangolin --sort-by=.lastTimestamp ``` -## Kustomize with GitOps - -Kustomize overlays work well with GitOps tools. - -* [Argo CD](/self-host/manual/kubernetes/gitops/argocd) can reconcile a Kustomize overlay path directly. -* [Flux](/self-host/manual/kubernetes/gitops/flux) can reconcile a Kustomize overlay with a `Kustomization` resource. - -In GitOps workflows, the controller owns the apply operation. Do not also apply the same overlay manually unless you are debugging. - ## Troubleshooting ### The patch does not apply diff --git a/self-host/manual/kubernetes/newt/troubleshooting.mdx b/self-host/manual/kubernetes/newt/troubleshooting.mdx index 0d377e9..3f23f74 100644 --- a/self-host/manual/kubernetes/newt/troubleshooting.mdx +++ b/self-host/manual/kubernetes/newt/troubleshooting.mdx @@ -1,5 +1,5 @@ --- -title: "Newt Troubleshooting" +title: "Troubleshooting" description: "Diagnose and resolve common Newt Kubernetes deployment issues." --- @@ -130,7 +130,6 @@ kubectl logs "$NEWT_POD" --namespace "$NEWT_NAMESPACE" --previous --tail=100 | Authentication failure | Wrong `NEWT_ID`, `NEWT_SECRET`, or provisioning key | Check credentials in Pangolin | | Endpoint connection errors | `PANGOLIN_ENDPOINT` is wrong or unreachable | Test DNS and HTTPS from the pod | | Image pull failure | Registry or image settings are wrong | `kubectl describe pod` | -| Permission error with native mode | Native WireGuard mode requires privileged settings | Check `global.nativeMode.enabled` and `useNativeInterface` | ## Secret issues @@ -255,7 +254,7 @@ If you use provisioning, also verify: ## Provisioning issues -Newt 1.11+ provisioning requires writable config persistence. +Provisioning requires writable config persistence. ### Symptoms @@ -309,50 +308,6 @@ newtInstances: `emptyDir` is recreated when the pod is recreated. Use a PVC if the generated configuration must survive pod replacement. -## Native WireGuard permission issues - -By default, Newt does not require native WireGuard privileges. - -Only check this section if you enabled native WireGuard mode: - -```yaml -global: - nativeMode: - enabled: true - -newtInstances: - - name: main-tunnel - useNativeInterface: true -``` - -### Symptoms - -```text -operation not permitted -cannot create interface -permission denied -``` - -### Check security context - -```bash -kubectl get pod "$NEWT_POD" --namespace "$NEWT_NAMESPACE" -o yaml | grep -A30 securityContext -``` - -Native mode requires privileged workload settings and capabilities such as `NET_ADMIN` and `SYS_MODULE`. - - -Only enable native WireGuard mode if your cluster policy allows privileged workloads. Do not add `NET_ADMIN` to the default non-native deployment unless you know it is required. - - -### Check namespace policy - -```bash -kubectl get namespace "$NEWT_NAMESPACE" --show-labels -``` - -A namespace with a restricted Pod Security Admission profile may block native mode. - ## Service not created or not reachable ### Important behavior diff --git a/self-host/manual/kubernetes/overview.mdx b/self-host/manual/kubernetes/overview.mdx index 5cf2af3..ae76fd0 100644 --- a/self-host/manual/kubernetes/overview.mdx +++ b/self-host/manual/kubernetes/overview.mdx @@ -1,45 +1,25 @@ --- title: "Overview" -description: "Deploy Pangolin, Newt, and related components on Kubernetes with Helm, Kustomize, GitOps, or Helmfile." +description: "Deploy Pangolin, Sites (Newt), and related components on Kubernetes." --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; - -## Kubernetes deployment options - -Kubernetes is a good fit for running Pangolin-related components when you need repeatable deployments, workload isolation, rolling updates, and integration with existing cluster services such as ingress, storage, monitoring, and network policy. - -This section covers the main Kubernetes workflows: - -- **Helm** for the recommended chart-based installation and upgrade workflow. -- **Kustomize** for overlay-based customization and manifest-driven deployments. -- **GitOps** with Argo CD or Flux for reconciling Helm charts, Kustomize overlays, or manifests from Git. -- **Helmfile** for advanced setups that manage multiple Helm releases together. - -## What this section covers - -- Kubernetes prerequisites and cluster requirements. -- Installation workflows for Helm, Kustomize, GitOps, and Helmfile. -- Pangolin installation, configuration, and troubleshooting. -- Newt installation, configuration, and troubleshooting. -- How Pangolin, Gerbil, Traefik, Newt and Pangolin-Kube-Controller fit together in tunneled deployments. - ## Components | Component | Role | | --- | --- | -| Pangolin | Main application and control plane for the dashboard, API, authentication, configuration, and database-backed state. | -| Gerbil | WireGuard interface management service used as part of the Pangolin tunnel stack. | -| Newt | Site connector used to expose private resources through Pangolin. Newt runs as a user-space WireGuard tunnel client and TCP/UDP proxy. | +| Pangolin | Main application for the dashboard, API, authentication, configuration, and database-backed state. | +| Gerbil | Tunnel stack component used by Pangolin for site connectivity. | +| Site (Newt) | Site connector used to connect private resources to Pangolin. | | Traefik | Reverse proxy and router for ingress traffic. | -| PostgreSQL / SQLite | Database options for Pangolin deployments, depending on the selected installation workflow and chart configuration. | -| Controller | Kubernetes controller for integration with Traefik cluster resources, replacing single Traefik instances with Traefik ingress controllers. | +| PostgreSQL / SQLite | Database options for Pangolin deployments, depending on the selected chart configuration. | +| Pangolin Kube Controller | Kubernetes controller for integrating Pangolin with Kubernetes and Traefik resources. | -For local reverse proxy deployments, the full tunnel stack may not be required. Tunneled sites require the components needed for Newt and WireGuard-based connectivity. +Depending on your deployment mode, not every component is required. Local reverse proxy deployments and tunneled site deployments can have different component requirements. ```mermaid @@ -47,50 +27,37 @@ flowchart LR U[Users] --> T[Traefik] T --> P[Pangolin] P --> G[Gerbil] - N[Newt] --> G + S[Site connector
Newt] --> G P --> D[(Database)] ``` -## Method comparison - -Choose the workflow that matches how you already manage Kubernetes applications: - -| Method | Best for | Complexity | GitOps fit | -| --- | --- | --- | --- | -| **Helm** | Standard Kubernetes installs and upgrades | Low | Works with Argo CD and Flux | -| **Kustomize** | Environment-specific overlays and manifest customization | Medium | Works with Argo CD and Flux | -| **Argo CD** | Git-driven reconciliation with a web UI and sync status | Medium | Native GitOps workflow | -| **Flux** | Declarative GitOps using Kubernetes custom resources | Medium | Native GitOps workflow | -| **Helmfile** | Managing multiple Helm releases as one deployment stack | Medium | Usually used from CI/CD or a controlled automation workflow | - - -Argo CD and Flux are delivery and reconciliation tools. They do not replace Helm or Kustomize. They can deploy Helm charts, Kustomize overlays, and other Kubernetes manifests. - - -## Recommended starting points +## Installation paths - - Compare Helm, Kustomize, GitOps, and Helmfile before choosing a workflow. + + Pick the Kubernetes workflow that matches how you deploy applications. - Review the cluster, ingress, storage, DNS, and tooling requirements. + Review the required cluster, ingress, DNS, storage, and secret setup. - - Start with the recommended chart-based Kubernetes workflow. + + Install Pangolin or Sites (Newt) with the standard chart-based workflow. - + Use overlays and patches for manifest-based deployments. - - Deploy with Argo CD or Flux from Git. + + Deploy Pangolin or Sites (Newt) with Argo CD. - - Manage Pangolin, Newt, and supporting Helm releases together. + + Deploy Pangolin or Sites (Newt) with Flux. + + + Manage multiple Helm releases together. -## Component quick links +## Component guides @@ -99,10 +66,16 @@ Argo CD and Flux are delivery and reconciliation tools. They do not replace Helm Configure Pangolin for your Kubernetes environment. - - Install Newt in a Kubernetes cluster. + + Diagnose and resolve Pangolin deployment issues. - - Configure Newt credentials, endpoints, resources, and runtime settings. + + Install a Site connector with the Newt Helm chart. + + + Configure Site connector credentials and runtime settings. + + + Diagnose and resolve Site connector deployment issues. diff --git a/self-host/manual/kubernetes/pangolin/configuration.mdx b/self-host/manual/kubernetes/pangolin/configuration.mdx index 0e30081..7eed350 100644 --- a/self-host/manual/kubernetes/pangolin/configuration.mdx +++ b/self-host/manual/kubernetes/pangolin/configuration.mdx @@ -1,5 +1,5 @@ --- -title: "Pangolin Configuration" +title: "Configuration" description: "Configuration reference for Pangolin Kubernetes deployments." --- @@ -478,6 +478,10 @@ Important settings: | `gerbil.service.enabled` | Creates a Service for Gerbil UDP traffic. | | `gerbil.persistence.enabled` | Persists Gerbil key/config data. Recommended for production. | + +If Gerbil is exposed through a reverse proxy or UDP gateway, keep proxy protocol settings aligned end-to-end. Do not enable proxy protocol on the upstream hop unless Gerbil is configured to accept it. + + ### Startup mode ```yaml @@ -922,6 +926,11 @@ helm upgrade --install pangolin fossorial/pangolin \ See [Pangolin Helm](/self-host/manual/kubernetes/pangolin/helm) for the installation flow. +For complete application configuration keys and examples, see: + +- [Public config file reference](/self-host/advanced/config-file) +- [Private config file reference](/self-host/advanced/private-config-file) + ### Kustomize Render the chart with Helm, then apply Kustomize overlays: @@ -973,31 +982,6 @@ spec: See [GitOps](/self-host/manual/kubernetes/gitops/overview) for GitOps guidance. -## Production checklist - -Before deploying to production: - -* [ ] Use `deployment.type=controller`, unless you have a specific reason to use standalone mode. -* [ ] Use `deployment.mode=multi`. -* [ ] Use `database.mode=cloudnativepg` or `database.mode=external`. -* [ ] Avoid SQLite for production. -* [ ] Configure a real `pangolin.config.app.dashboard_url`. -* [ ] Replace the default `example.com` domain entry. -* [ ] Configure `pangolin.config.gerbil.base_endpoint`. -* [ ] Keep `pangolin.config.gerbil.start_port` aligned with `gerbil.ports.wg1`. -* [ ] Keep `pangolin.config.gerbil.clients_start_port` aligned with `gerbil.ports.wg2`. -* [ ] Configure TLS with either a Traefik cert resolver or an existing TLS Secret. -* [ ] Create or label the namespace so Gerbil can use `NET_ADMIN`. -* [ ] Keep Gerbil persistence enabled. -* [ ] Store the Pangolin app secret in a Kubernetes Secret. -* [ ] Use an existing database connection Secret for external PostgreSQL. -* [ ] Review NetworkPolicy egress requirements. -* [ ] Avoid broad `0.0.0.0/0` egress unless required. -* [ ] Enable monitoring resources only when the required CRDs exist. -* [ ] Set resource requests and limits based on expected traffic. -* [ ] Define a database backup strategy. -* [ ] Test upgrades in a staging environment before production. - ## Next steps diff --git a/self-host/manual/kubernetes/pangolin/helm.mdx b/self-host/manual/kubernetes/pangolin/helm.mdx index 185691f..2fb5ca8 100644 --- a/self-host/manual/kubernetes/pangolin/helm.mdx +++ b/self-host/manual/kubernetes/pangolin/helm.mdx @@ -1,5 +1,5 @@ --- -title: "Pangolin Helm" +title: "Helm" description: "Quick-start guide for installing Pangolin on Kubernetes using Helm." --- @@ -24,20 +24,15 @@ Depending on the selected values, the chart can deploy: - **Traefik integration**: Traefik CRD-based routing in controller mode, bundled Traefik controller when enabled, or standalone Traefik mode. - **Database backend**: CloudNativePG, external PostgreSQL, embedded PostgreSQL, or SQLite. -## Version matrix +See [Version Matrix](https://github.com/fosrl/helm-charts/VERSION_MATRIX.md) for chart and default app version references. -| Item | Value | -| --- | --- | -| Chart version | `0.1.0-alpha.0` | -| Kubernetes version | `>=1.30.14-0` | -| Pangolin appVersion | `1.18.2` | -| Pangolin default image tag | `1.18.2` | -| Pangolin PostgreSQL image tag | `postgresql-1.18.2` | -| pangolin-kube-controller tag | `0.1.0-alpha.1` | -| Gerbil tag | `1.3.1` | -| Traefik tag | `v3.6.15` | +## Gerbil setup in the Pangolin chart -The current chart metadata defines chart version `0.1.0-alpha.0`, app version `1.18.2`, Kubernetes `>=1.30.14-0`, and the component image metadata listed above. +This chart deploys Gerbil when `gerbil.enabled=true`. This is the default when using `deployment.type=controller` and recommended. + + +If Gerbil is exposed through a reverse proxy or UDP gateway, keep proxy protocol settings aligned end-to-end. Do not enable proxy protocol on the upstream hop unless Gerbil is configured to accept it. + ## Prerequisites diff --git a/self-host/manual/kubernetes/pangolin/kustomize.mdx b/self-host/manual/kubernetes/pangolin/kustomize.mdx index c9f1bf5..69410bd 100644 --- a/self-host/manual/kubernetes/pangolin/kustomize.mdx +++ b/self-host/manual/kubernetes/pangolin/kustomize.mdx @@ -1,5 +1,5 @@ --- -title: "Pangolin Kustomize" +title: "Kustomize" description: "Deploy Pangolin on Kubernetes using Helm-rendered manifests and Kustomize overlays." --- @@ -625,15 +625,6 @@ If a patch does not apply, inspect generated resource names: kustomize build base | grep -E "^(kind:| name:)" ``` -## Kustomize with GitOps - -Kustomize overlays work well with GitOps tools. - -* [Argo CD](/self-host/manual/kubernetes/gitops/argocd) can reconcile a Kustomize overlay path directly. -* [Flux](/self-host/manual/kubernetes/gitops/flux) can reconcile a Kustomize overlay with a `Kustomization` resource. - -In GitOps workflows, the controller owns the apply operation. Do not also apply the same overlay manually unless you are debugging. - ## Troubleshooting ### The patch does not apply diff --git a/self-host/manual/kubernetes/pangolin/troubleshooting.mdx b/self-host/manual/kubernetes/pangolin/troubleshooting.mdx index 91eb0be..54480f2 100644 --- a/self-host/manual/kubernetes/pangolin/troubleshooting.mdx +++ b/self-host/manual/kubernetes/pangolin/troubleshooting.mdx @@ -1,5 +1,5 @@ --- -title: "Pangolin Troubleshooting" +title: "Troubleshooting" description: "Diagnose and resolve Pangolin Kubernetes deployment issues." --- @@ -386,6 +386,47 @@ kubectl describe networkpolicy --namespace "$PANGOLIN_NAMESPACE" Verify external firewall rules for the configured UDP ports. + + + + +**Symptoms** + +* Newt peers do not establish stable handshakes. +* Tunnel traffic drops even though Gerbil pods are healthy. +* Logs show connection resets or malformed upstream traffic. + +**Cause** + +Proxy protocol handling is inconsistent between the upstream hop and Gerbil. + + +If Gerbil is exposed through a reverse proxy or UDP gateway, keep proxy protocol settings aligned end-to-end. Do not enable proxy protocol on the upstream hop unless Gerbil is configured to accept it. + + +**Checks** + +Check endpoint and port alignment: + +```bash +helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all | grep -A40 gerbil +``` + +Check Gerbil logs: + +```bash +kubectl logs --namespace "$PANGOLIN_NAMESPACE" \ + -l app.kubernetes.io/name=gerbil \ + --tail=200 +``` + +Check Service exposure: + +```bash +kubectl get svc --namespace "$PANGOLIN_NAMESPACE" \ + -l app.kubernetes.io/name=gerbil -o wide +``` + diff --git a/self-host/manual/kubernetes/prerequisites.mdx b/self-host/manual/kubernetes/prerequisites.mdx index 594a439..2c349ec 100644 --- a/self-host/manual/kubernetes/prerequisites.mdx +++ b/self-host/manual/kubernetes/prerequisites.mdx @@ -1,136 +1,67 @@ --- title: "Prerequisites" -description: "Cluster requirements, tools, and setup needed to deploy Pangolin and Newt on Kubernetes." +description: "Cluster, tooling, networking, and storage requirements for deploying Pangolin and Sites (Newt) on Kubernetes." --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; +Before installing Pangolin or Sites (Newt) on Kubernetes, check that your cluster, tools, networking, and storage setup match the deployment path you want to use. -Before installing Pangolin or Newt on Kubernetes, check that your cluster, local tools, networking, and storage setup are ready. +## Kubernetes cluster -## Cluster requirements - -### Kubernetes version - -The Pangolin and Newt Helm charts currently require Kubernetes **1.30.14 or newer**. - -Use a Kubernetes version that is both: - -- supported by your Kubernetes provider or distribution -- compatible with the chart requirement - -Kubernetes 1.30 satisfies the chart minimum, but it is no longer a supported upstream Kubernetes release. See the [Kubernetes version skew policy](https://kubernetes.io/releases/version-skew-policy/) for current support details. For production, use a currently supported Kubernetes minor release whenever possible. +Use a Kubernetes version that satisfies the Helm chart `kubeVersion` requirement and is supported by your Kubernetes provider or distribution. Check your cluster version: ```bash kubectl version -``` +``` -For the exact version support, refer to the Pangolin and Newt Helm chart READMEs before installing or upgrading. +See the [Version Matrix](https://github.com/fosrl/helm-charts/VERSION_MATRIX.md) for the supported Kubernetes versions of the Pangolin and Newt Helm charts. -### Cluster access +## Controller access and RBAC -You need `kubectl` access to the target cluster. +Controller mode is the default and recommended Kubernetes deployment mode for Pangolin. -The user or service account used for installation must be able to create and manage the resources required by the selected install method. +When controller mode is enabled, the Pangolin Kube Controller runs with its own ServiceAccount and needs permission to watch and manage the Kubernetes and Traefik resources it reconciles. The chart creates the required RBAC resources for you, unless RBAC creation is disabled. -For a standard Helm install, this usually includes: +By default, the controller is scoped to the namespace of a single Pangolin deployment. It can also be configured for a broader scope when one controller should reconcile resources for multiple Pangolin deployments. -* namespaces -* deployments and statefulsets -* services -* configmaps -* secrets -* persistent volume claims -* service accounts -* roles and role bindings -* network policies, if enabled +Depending on the configured controller scope, the controller needs namespace-scoped or cluster-scoped access to the resources it reconciles: -For Pangolin controller mode, additional permissions are required for the controller resources managed by the chart. +| API group | Resources | Verbs | +| --- | --- | --- | +| `""` | `events` | `create`, `patch`, `update` | +| `""` | `services`, `endpoints` | `get`, `list`, `watch`, `create`, `update`, `patch`, `delete` | +| `discovery.k8s.io` | `endpointslices` | `get`, `list`, `watch`, `create`, `update`, `patch`, `delete` | +| `traefik.io` | `ingressroutes`, `ingressroutetcps`, `ingressrouteudps`, `middlewares`, `middlewaretcps`, `traefikservices`, `serverstransports`, `serverstransporttcps`, `tlsoptions`, `tlsstores` | `get`, `list`, `watch`, `create`, `update`, `patch`, `delete` | -Check cluster access: +If leader election is enabled, the controller also needs access to: -```bash -kubectl cluster-info -kubectl auth can-i create namespace -kubectl auth can-i create deployments --namespace pangolin -kubectl auth can-i create secrets --namespace pangolin -``` +| API group | Resources | Verbs | +| --- | --- | --- | +| `coordination.k8s.io` | `leases` | `get`, `list`, `watch`, `create`, `update`, `patch` | - -The exact permissions depend on the selected chart options. If you install cluster-scoped components or CRDs, cluster-level permissions may be required. - +The controller also needs cluster-wide read access to Kubernetes discovery resources: -## Required tools +| API group | Resources | Verbs | +| --- | --- | --- | +| `networking.k8s.io` | `ingressclasses` | `get`, `list`, `watch` | +| `apiextensions.k8s.io` | `customresourcedefinitions` | `get`, `list`, `watch` | -### kubectl + +For namespace-scoped deployments, the chart creates namespaced RBAC for the controller namespace and, if configured, the target namespace. For broader controller scopes, the chart creates the required cluster-scoped RBAC. + -`kubectl` is required for checking cluster access, inspecting resources, and troubleshooting. +## Database and storage -Install a `kubectl` version that is compatible with your cluster version. As a general rule, keep `kubectl` close to the Kubernetes API server version used by your cluster. +Pangolin requires a database backend. The Helm chart supports multiple database modes, including CloudNativePG, external PostgreSQL, embedded PostgreSQL, and SQLite. -Verify: - -```bash -kubectl version --client -``` - -### Helm - -Helm is required for Helm-based installs and for rendering Helm charts. -- **Minimum version**: 3.10 or later -- **Installation**: [Helm official install guide](https://helm.sh/docs/intro/install/) - -Verify: - -```bash -helm version -``` - -Install Helm from the official Helm installation guide if it is not already available. - -### Kustomize - -Kustomize is required for Kustomize-based installs. - -You can use either: - -* `kubectl apply -k` -* `kubectl kustomize` -* the standalone `kustomize` CLI - -Verify: - -```bash -kubectl kustomize --help -``` - -Or, when using the standalone CLI: - -```bash -kustomize version -``` - -### Git - -Git is required for GitOps workflows with Argo CD or Flux. - -Verify: - -```bash -git --version -``` - -## Storage requirements - -### StorageClass - -Pangolin requires persistent storage depending on the selected database mode and chart configuration. +For persistent database-backed deployments, make sure your cluster has a usable StorageClass or configure the StorageClass explicitly in your chart values. Check available StorageClasses: @@ -138,66 +69,45 @@ Check available StorageClasses: kubectl get storageclasses ``` -If your cluster has a default StorageClass, Kubernetes can usually provision persistent volumes automatically. +For long-running/production deployments, prefer PostgreSQL-based modes such as CloudNativePG or external PostgreSQL. -If no default StorageClass exists, configure the StorageClass explicitly in your values file. + +SQLite can be useful for simple or test deployments, but PostgreSQL-based modes are the better fit for long-running/production Kubernetes deployments. + -### Pangolin storage +### Site connector storage -Pangolin deployments use persistent storage for database-backed setups, CloudNativePG, embedded PostgreSQL, or other chart-managed persistent components. +A Site (Newt) deployment does not require persistent storage by default. -For production, prefer one of these database modes: +Use writable configuration persistence only if your deployment needs runtime configuration to survive pod replacement, upgrades, node drains, or rescheduling. For simple deployments, no PVC is required. -* CloudNativePG -* external PostgreSQL +## Networking -Avoid SQLite for production deployments. - -### Newt storage - -Newt is lightweight, but persistent storage is still recommended for stable Kubernetes operation. -The Newt chart supports writable configuration persistence through either `emptyDir` or an existing PVC. While `emptyDir` can be used for simple test deployments, it is ephemeral and is recreated when the Pod is replaced. In that case, Newt may lose its writable runtime configuration and has to re-establish its connection state after the Pod starts again. During this reconnect and handshake phase, existing tunnels or proxied connections can be interrupted temporarily. - - -`emptyDir` is suitable for short-lived tests, but pod replacement can interrupt active Newt traffic while connection state is rebuilt. - - - -For production Newt deployments, use a PersistentVolumeClaim so writable runtime configuration survives restarts, upgrades, and rescheduling. - - -For production or any setup where short interruptions after Pod restarts should be avoided, use a PersistentVolumeClaim. This allows Newt to keep its writable configuration across Pod restarts, node drains, upgrades, and rescheduling events. - -## Networking requirements - -### Ingress or Traefik routing +### Ingress and routing Pangolin needs an external entrypoint for the dashboard, API, and site traffic. -The Pangolin chart supports different deployment modes: +Depending on your chart values, this can use: -* `controller` mode with a Traefik ingress controller and Traefik CRDs integration -* `standalone` mode with chart-managed Traefik workload -* optional bundled Traefik controller installation in controller mode +* controller mode with a Traefik ingress controller +* standalone mode with chart-managed Traefik components +* an existing ingress or load balancer setup -Controller mode is the recommended production mode. +If you use controller mode with Traefik CRDs, verify that the required Traefik API resources are available: - -In controller mode, Traefik CRDs must be available. And it may add additional version requirements based on the Pangolin/Controller version. Check the chart README for the exact requirements. - +```bash +kubectl api-resources --api-group=traefik.io +``` -Check ingress-related resources: +You can also check existing ingress resources: ```bash kubectl get ingress -A -kubectl get ingressroute.traefik.io -A ``` -The `IngressRoute.traefik.io` command only works when Traefik CRDs are installed. - ### DNS -Configure DNS records before exposing Pangolin publicly. +Configure DNS records for the domains used by Pangolin before exposing it publicly. At minimum, the Pangolin dashboard domain should resolve to the ingress controller, load balancer, or public endpoint used by your deployment. @@ -207,51 +117,30 @@ Example: nslookup pangolin.example.com ``` -For tunneled site deployments, also verify the DNS records used by the tunnel entrypoint and Newt connection settings. +For tunneled site deployments, also verify the DNS name used by the site connector endpoint. ### TLS Use HTTPS for the Pangolin dashboard and API. -Common TLS options are: +Common TLS options include: * Traefik ACME / Let's Encrypt -* cert-manager, Infisical or similar with an existing ingress or certificate workflow -* a pre-created Kubernetes TLS secret +* cert-manager +* a pre-created Kubernetes TLS Secret * TLS termination at an external load balancer or ingress controller -The Pangolin chart supports cert-manager, Infisical, Traefik-related TLS configuration and custom Kubernetes TLS secrets. Make sure the configured entrypoints, certificate resolver names, or TLS secret names match your actual Traefik setup. +Use the TLS method that matches your ingress and cluster setup. -If you use cert-manager, verify that cert-manager is already installed: +If you use cert-manager, verify that the certificate CRDs are available: - -cert-manager is useful for many Kubernetes TLS setups, but it is not required for every Pangolin deployment. Use the TLS method that matches your ingress controller and cluster. - +```bash +kubectl get crd certificates.cert-manager.io +``` -### Network policies +## Namespace and security -The Pangolin and Newt charts include NetworkPolicy configuration for the required application traffic. When enabled, the chart-managed policies are designed to allow the necessary communication between Pangolin, Newt, Gerbil, Traefik, the database, DNS, and other required components. - - -Keep chart-managed NetworkPolicies enabled by default. Add custom policies mainly when your security model requires stricter controls. - - -You usually do not need to create additional NetworkPolicies for a standard installation. However, if you disable the chart-managed policies or replace them with your own policies because of custom security or other requirements, make sure your policies still allow the required traffic between: - -* Traefik (IngressController or Standalone), Pangolin and Pangolin-Kube-Controller, if used -* Pangolin and its database -* Pangolin and Gerbil, when the tunnel stack is enabled -* Newt and the Pangolin endpoint -* workloads and DNS -* workloads and external identity providers or APIs, if used - -For most deployments, it is recommended to keep the chart-managed NetworkPolicies enabled and only customize them when your cluster has specific network security requirements. - -## Namespace and RBAC - -### Namespace - -Choose a namespace for the installation. +Choose the namespace where Pangolin and related components should run. Example: @@ -259,7 +148,7 @@ Example: kubectl create namespace pangolin ``` -When installing with Helm, you can also let Helm create the namespace: +When using Helm, you can also let Helm create the namespace: ```bash helm upgrade --install pangolin fossorial/pangolin \ @@ -267,158 +156,68 @@ helm upgrade --install pangolin fossorial/pangolin \ --create-namespace ``` -It's recommended to create the namespace explicitly before installation, so you can apply any required labels or annotations for Pod Security Admission or other cluster policies. +If your cluster enforces Pod Security Admission, make sure the namespace labels match the selected deployment mode. Deployments that include tunnel components may require permissions that are not compatible with a fully restricted namespace profile. -### Pod Security Admission +## NetworkPolicy -Some clusters enforce Pod Security Admission labels at the namespace level. +The Pangolin and Newt charts can manage NetworkPolicies for the required application traffic. -Pangolin deployments that include Gerbil may require permissions that are not compatible with a restricted namespace profile, because Gerbil manages WireGuard and requires capabilities such as `NET_ADMIN`. - -If the chart creates the namespace, it can apply the labels required by its configuration. If you manage the namespace yourself, apply the required labels manually based on the chart values and your selected deployment mode. - -### Service accounts and RBAC - -The Helm charts can create the required service accounts and RBAC resources. - -Controller mode requires Kubernetes API access for the Pangolin controller. The main Pangolin application and Gerbil do not need the same Kubernetes API permissions in the default multi-workload topology. - -## Secrets and configuration - -### Secret management - -Pangolin and Newt require secrets for different parts of the deployment. - -Common examples: - -* Pangolin application secrets -* database credentials or connection strings -* Newt credentials or provisioning keys -* TLS certificates, if not managed by the ingress layer -* identity provider client secrets, if used - -Do not commit plaintext secrets to Git. - - -Do not commit plaintext secrets to Git. Use encrypted or external secret backends for GitOps workflows. - - -If you deploy with GitOps tools such as Argo CD or Flux, use a secret management approach that allows secrets to be stored safely in Git or injected from an external secret backend. Common options include encrypted secrets with SOPS, Sealed Secrets, External Secrets Operator, HashiCorp Vault, Infisical, or cloud provider secret managers. These approaches allow you to keep the deployment declarative without storing plaintext credentials in the repository. - -Use your existing secret management workflow, for example: - -* SOPS-encrypted secrets for GitOps workflows -* Sealed Secrets -* External Secrets Operator -* HashiCorp Vault -* Infisical -* cloud provider secret managers -* manually created Kubernetes Secrets for small test environments - -### ConfigMaps - -Configuration that is not sensitive can be stored in ConfigMaps or provided through Helm values. - -Do not put passwords, private keys, API tokens, provisioning secrets, or other sensitive values into ConfigMaps. ConfigMaps are not designed for secret data and should only be used for non-sensitive configuration. -## Database requirements - -### Pangolin - -Pangolin supports several database modes through the Helm chart. - -For production, use: - -* CloudNativePG -* external PostgreSQL - -The default Pangolin chart database mode is `cloudnativepg`. - -When using CloudNativePG, make sure either: - -* the chart installs the required CloudNativePG components, or -* an existing CloudNativePG operator and cluster are already available, depending on your selected values - -When using an external PostgreSQL database, provide the connection details through a Kubernetes Secret or a chart-supported secret generation method. +If you enable chart-managed NetworkPolicies, review the generated policies before adding custom deny rules. If you replace them with your own policies, allow the required traffic between the components you deploy, such as Pangolin, Traefik, Gerbil, the database, DNS, and Site connectors. ## Resource planning -Set resource requests and limits according to your expected workload and cluster sizing. +Pangolin and Site (Newt) Kubernetes deployments include predefined resource profiles for the supported deployment methods. These profiles set CPU and memory requests and limits for the components used by the selected deployment mode. -At minimum, plan resources for: +The available profiles are: -* Pangolin -* Gerbil, when the tunnel stack is enabled -* pangolin-kube-controller, when controller mode is used -* Traefik, if installed or managed as part of the deployment -* PostgreSQL or CloudNativePG components, if used -* Newt instances +| Profile | Intended use | +| --- | --- | +| Small | Small deployments, or clusters with very limited available resources. | +| Standard | Default profile for most normal deployments. | +| Large | Larger environments with more Sites, more users, higher traffic, or stricter availability expectations. | -Resource usage depends on traffic volume, number of sites, number of users, database mode, enabled metrics, and ingress/tunnel configuration. +The selected profile applies to the workloads that are part of your deployment, for example: -Configure resources in your Helm values, for example: +| Component | Resource considerations | +| --- | --- | +| Pangolin | Main application workload. Size according to dashboard/API usage, users, and traffic. | +| Pangolin Kube Controller | Required in controller mode. Size according to the number of reconciled Kubernetes and Traefik resources. | +| Traefik | Size according to ingress and proxy traffic. | +| Gerbil | Required when the tunnel stack is enabled. Size according to tunnel traffic and number of connected Sites. | +| PostgreSQL / CloudNativePG | Size according to database mode, stored state, and expected write/read activity. | +| Site connectors (Newt) | Each Site connector adds its own resource usage. Size according to the traffic handled by that Site. | -```yaml -resources: - requests: - cpu: 500m - memory: 512Mi - limits: - memory: 1Gi -``` + +The Standard profile is intended to be enough for most standard deployments. Use Small for very limited lab or test environments, and Large for higher traffic, more Sites, more users, or larger production environments. + - -Use the chart defaults as the starting point, then adjust requests and limits based on actual usage in your cluster. - +After installation, monitor CPU and memory usage and adjust the selected profile or individual resource overrides if needed. -Avoid setting CPU limits unless you have a specific reason to enforce them. CPU limits can cause throttling when a workload temporarily needs more CPU, even if spare CPU capacity is available on the node. This can negatively affect latency-sensitive components such as ingress, tunnel, proxy, or controller workloads. -For most deployments, set CPU requests to reserve an appropriate baseline and set memory limits to protect the node from excessive memory usage. Add CPU limits only when your cluster policy requires them or when you intentionally want to cap a component's maximum CPU usage. +Avoid setting CPU limits on latency-sensitive Pangolin components unless your cluster policy requires them or you intentionally want to cap CPU usage. + +CPU limits can cause throttling when a workload temporarily needs more CPU, even if spare CPU capacity is available on the node. This can negatively affect ingress, tunnel, proxy, database, and controller workloads. + +For most deployments, use CPU requests to reserve baseline capacity and memory limits to protect the node from excessive memory usage. -## Optional tools - -### Argo CD - -Use Argo CD if you want GitOps reconciliation with a web UI, sync status, and drift detection. - -### Flux - -Use Flux if you want GitOps reconciliation through Kubernetes custom resources such as `HelmRelease` and `Kustomization`. - -### Helmfile - -Use Helmfile if you want to manage multiple Helm releases together, for example supporting components plus Pangolin and Newt. - -## Verification checklist - -Before proceeding with installation: - -* [ ] The cluster runs a supported Kubernetes version that satisfies the chart requirement. -* [ ] `kubectl` can access the cluster. -* [ ] Helm is installed, if using the Helm workflow. -* [ ] Kustomize is available, if using the Kustomize workflow. -* [ ] A namespace is created or planned. -* [ ] A StorageClass is available if persistent storage is required. -* [ ] The ingress or Traefik routing strategy is defined. -* [ ] DNS records are configured. -* [ ] The TLS strategy is defined. -* [ ] Database mode is selected for Pangolin. -* [ ] Secret management is planned. -* [ ] Pod Security Admission requirements are understood. -* [ ] NetworkPolicy requirements are understood, if policies are enabled. -* [ ] Resource requests and limits are reviewed. - ## Next steps + + Pick the Kubernetes workflow that matches how you deploy applications. + - Install Pangolin or Newt with Helm. + Install Pangolin or Sites (Newt) with Helm. Use Kustomize overlays and patches. - - Deploy with Argo CD or Flux. + + Deploy Pangolin or Sites (Newt) with Argo CD. + + + Deploy Pangolin or Sites (Newt) with Flux. Start with the Pangolin Helm installation guide.