guides for traefik logs and middleware

2026-02-08 05:56:45 +00:00 · 2025-08-20 10:17:38 +01:00
parent c9d6ba92c0
commit b52c52c738
2 changed files with 399 additions and 0 deletions
--- a/self-host/community-guides/middlewaremanager.mdx
+++ b/self-host/community-guides/middlewaremanager.mdx
@@ -0,0 +1,184 @@
+# Enhancing Your Pangolin Deployment with the Middleware Manager
+
+<Note>  
+This is a community guide and not officially supported. For issues, contributions, or bug reports, please use the [official GitHub repository](https://github.com/hhftechnology/middleware-manager).  
+</Note>  
+
+:warning: **Security Warning**  
+Middlewares can strengthen security but also create vulnerabilities if misconfigured.  
+* Test in staging before production.  
+* Misusing forward authentication can leak credentials.  
+* Bad rate limiter configs may be bypassed.  
+* Header misconfigurations can expose apps to XSS/CSRF.  
+* Stacking too many middlewares impacts performance.  
+* Always check provider references (`@http` vs `@file`).  
+
+---
+
+## What is Middleware Manager?
+
+The **Pangolin Middleware Manager** is a microservice that extends your existing Pangolin deployment.  
+It provides a **web UI** to attach Traefik middlewares to resources without editing Pangolin itself.  
+
+### Key Use Cases
+* External authentication (Authelia, Authentik, JWT)  
+* Security headers and CSP policies  
+* Geographic IP blocking  
+* Rate limiting / DDoS protection  
+* Redirects & path rewrites  
+* CrowdSec and other security tool integrations  
+
+---
+
+## Prerequisites
+* A running **Pangolin v1.0.0+**  
+* Docker + Docker Compose  
+* Basic Traefik knowledge  
+* Admin access to your Pangolin host  
+
+---
+
+## Step 1: Add Middleware Manager Service
+
+Update your `docker-compose.yml`:
+
+```yaml
+middleware-manager:
+  image: hhftechnology/middleware-manager:latest
+  container_name: middleware-manager
+  restart: unless-stopped
+  volumes:
+    - ./data:/data
+    - ./config/traefik/rules:/conf
+    - ./config/middleware-manager/templates.yaml:/app/config/templates.yaml  # Optional custom templates
+  environment:
+    - PANGOLIN_API_URL=http://pangolin:3001/api/v1
+    - TRAEFIK_CONF_DIR=/conf
+    - DB_PATH=/data/middleware.db
+    - PORT=3456
+  ports:
+    - "3456:3456"
+````
+
+---
+
+## Step 2: Create Required Directories
+
+```bash
+mkdir -p ./config/traefik/rules
+mkdir -p ./config/middleware-manager
+```
+
+Move any dynamic configs into `./config/traefik/rules`.
+
+---
+
+## Step 3: Update Traefik Volumes & Providers
+
+In your `traefik` service:
+
+```yaml
+volumes:
+  - ./config/traefik:/etc/traefik:ro
+  - ./config/letsencrypt:/letsencrypt
+  - ./config/traefik/logs:/var/log/traefik
+  - ./config/traefik/rules:/rules   # required
+```
+
+In `traefik_config.yml`:
+
+```yaml
+providers:
+  file:
+    directory: "/rules"
+    watch: true
+```
+
+---
+
+## Step 4: Enable Badger Plugin
+
+In `traefik_config.yml`:
+
+```yaml
+experimental:
+  plugins:
+    badger:
+      moduleName: "github.com/fosrl/badger"
+      version: "v1.0.0"
+```
+
+---
+
+## Step 5: Start Services
+
+```bash
+docker compose up -d
+```
+
+---
+
+## Step 6: Access the UI
+
+Middleware Manager runs at:
+👉 [http://localhost:3456](http://localhost:3456)
+
+---
+
+## Common Middleware Examples
+
+### Rate Limiting
+
+```yaml
+middlewares:
+  - id: "rate-limit"
+    type: "rateLimit"
+    config:
+      average: 100
+      burst: 50
+```
+
+### Security Headers
+
+```yaml
+middlewares:
+  - id: "security-headers"
+    type: "headers"
+    config:
+      customResponseHeaders:
+        Server: ""
+        X-Powered-By: ""
+      browserXSSFilter: true
+      contentTypeNosniff: true
+      forceSTSHeader: true
+      stsSeconds: 63072000
+```
+
+---
+
+## Troubleshooting
+
+* **Service does not exist** → Check `@http` or `@file` suffix in references
+* **Middleware does not exist** → Verify config and required plugins
+* **No changes applied** → Check Traefik logs, middleware priority, restart services
+* **UI not showing resources** → Confirm `PANGOLIN_API_URL` and network connectivity
+* **Database errors** → Check `./data` permissions, or reset `middleware.db`
+* **CrowdSec errors → Ensure the crowdsec container is running; middlewares fail if the service is down.
+* **Protecting Pangolin itself** → Apply middlewares (e.g. geoblock, headers) directly on the websecure entryPoint to cover all traffic.
+* **Applying to many services** → Attach middleware to entryPoints instead of individual resources to cover all subdomains at once.
+* **TCP / SMTP with STARTTLS** → Not supported. Traefik cannot handle STARTTLS negotiation (only implicit TLS like SMTPS on 465).
+
+---
+
+## Final Notes
+
+The Middleware Manager bridges Pangolin’s simplicity with Traefik’s powerful middleware ecosystem.
+
+* Start with simple configs → test thoroughly → expand gradually.
+* Use templates where possible.
+* Always validate in staging before production.
+
+Happy proxying 🚀
+
+```
+
--- a/self-host/community-guides/traefiklogsdashboard.mdx
+++ b/self-host/community-guides/traefiklogsdashboard.mdx
@@ -0,0 +1,215 @@
+## Enhanced Traefik Log Dashboard with Pangolin
+
+<Note>  
+This is a community guide and is not officially supported. For issues or advanced configuration, please visit the [official repository](https://github.com/hhftechnology/traefik-log-dashboard).  
+</Note>  
+
+If you’re already using the **Pangolin stack with Traefik as your reverse proxy**, you have powerful routing in place. But raw log files and scattered metrics make it difficult to truly understand traffic patterns.
+
+The **Enhanced Traefik Log Dashboard** adds **real-time monitoring, OpenTelemetry support, and geolocation analytics**, giving you a full picture of your traffic.
+
+## Features
+
+* **OpenTelemetry OTLP Support**: Real-time traces from Traefik v3+
+* **Hybrid Monitoring**: Combine OTLP traces with traditional log parsing
+* **Geolocation**: MaxMind GeoIP integration with automatic updates
+* **Analytics**: Live request rates, response times, error tracking
+* **Production Ready**: Resource limits, health checks, optimized GC/memory
+
+## Prerequisites
+
+* Docker + Docker Compose
+* Traefik v3.0+ (for OTLP) or v2.x (logs only)
+* A working Pangolin stack
+
+## Step 1: Configure Traefik
+
+### For OTLP + Logs (Recommended)
+
+Update `./config/traefik/traefik_config.yml`:
+
+```yaml
+log:
+  level: INFO
+  filePath: "/var/log/traefik/traefik.log"
+  format: json
+
+accessLog:
+  filePath: "/var/log/traefik/access.log"
+  format: json
+
+tracing:
+  otlp:
+    http:
+      endpoint: "http://log-dashboard-backend:4318/v1/traces"
+  sampleRate: 0.1 # Adjust as needed - 0.1 for 10% sampling
+  globalAttributes:
+    environment: "production"
+    service.version: "v3.0"
+    deployment.environment: "pangolin"
+```
+
+### For Logs Only
+
+```yaml
+log:
+  level: INFO
+  filePath: "/var/log/traefik/traefik.log"
+
+accessLog:
+  filePath: "/var/log/traefik/access.log"
+  format: json
+```
+
+## Step 2: Add Dashboard Services
+
+Extend your existing `docker-compose.yml` with:
+
+```yaml
+  log-dashboard-backend:
+    image: ghcr.io/hhftechnology/traefik-log-dashboard-backend:latest
+    container_name: log-dashboard-backend
+    restart: unless-stopped
+    ports:
+      - "4317:4317"   # OTLP gRPC
+      - "4318:4318"   # OTLP HTTP
+    volumes:
+      - ./config/traefik/logs:/logs:ro
+      - ./config/maxmind:/maxmind:ro
+    environment:
+      - PORT=3001
+      - TRAEFIK_LOG_FILE=/logs/access.log
+      - OTLP_ENABLED=true
+      - OTLP_GRPC_PORT=4317
+      - OTLP_HTTP_PORT=4318
+      - USE_MAXMIND=true
+      - MAXMIND_DB_PATH=/maxmind/GeoLite2-City.mmdb
+      - MAXMIND_FALLBACK_ONLINE=true
+      - GOGC=50
+      - GOMEMLIMIT=500MiB
+
+  log-dashboard-frontend:
+    image: ghcr.io/hhftechnology/traefik-log-dashboard-frontend:latest
+    container_name: log-dashboard-frontend
+    restart: unless-stopped
+    ports:
+      - "3000:80"
+    environment:
+      - BACKEND_SERVICE=log-dashboard-backend
+      - BACKEND_PORT=3001
+    depends_on:
+      - log-dashboard-backend
+```
+
+## Step 3: Setup MaxMind GeoIP (Optional but Recommended)
+
+1. Create a free MaxMind account → [GeoLite2 signup](https://www.maxmind.com/en/geolite2/signup)
+2. Generate a license key
+3. Create directory:
+
+```bash
+mkdir -p ./config/maxmind
+export MAXMIND_LICENSE_KEY=your_key_here
+```
+
+4. (Optional) Add updater service:
+
+```yaml
+  maxmind-updater:
+    image: alpine:latest
+    container_name: maxmind-db-updater
+    volumes:
+      - ./config/maxmind:/data
+    environment:
+      - MAXMIND_LICENSE_KEY=${MAXMIND_LICENSE_KEY}
+    command: >
+      sh -c "
+        apk add --no-cache wget tar &&
+        cd /data &&
+        wget -O GeoLite2-City.tar.gz 'https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=$MAXMIND_LICENSE_KEY&suffix=tar.gz' &&
+        tar -xzf GeoLite2-City.tar.gz --strip-components=1 '*/GeoLite2-City.mmdb' &&
+        rm -f GeoLite2-City.tar.gz
+      "
+```
+
+## Step 4: Launch Stack
+
+```bash
+docker compose up -d
+docker compose ps
+```
+
+## Step 5: Access the Dashboard
+
+* **Frontend UI** → [http://localhost:3000](http://localhost:3000)
+
+## Key Features
+
+* **Real-time statistics** (requests, response times, error rates)
+* **Interactive world map** (request origins via MaxMind)
+* **Service insights** (performance by router/service)
+* **Hybrid monitoring** (OTLP + logs together)
+
+## Performance Tuning
+
+For production or high-traffic environments, you may want to adjust settings to optimize resource usage and throughput.
+
+### Reduce OTLP Sampling
+Lower sampling to avoid overwhelming storage and dashboards:
+
+```yaml
+tracing:
+  sampleRate: 0.1  # 10% sampling in production
+````
+
+### Prefer gRPC over HTTP
+
+For lower latency and higher throughput, enable gRPC instead of HTTP:
+
+```yaml
+tracing:
+  otlp:
+    grpc:
+      endpoint: "log-dashboard-backend:4317"
+      insecure: true
+```
+
+### Tune Backend Memory
+
+Set environment variables in `log-dashboard-backend`:
+
+```yaml
+environment:
+  - GOGC=20        # More aggressive garbage collection
+  - GOMEMLIMIT=1GiB # Hard memory limit
+```
+
+### Resource Limits
+
+Add CPU and memory constraints to containers:
+
+```yaml
+deploy:
+  resources:
+    limits:
+      cpus: "1.0"
+      memory: 512M
+    reservations:
+      cpus: "0.2"
+      memory: 128M
+```
+
+These adjustments help keep the dashboard responsive while minimizing resource overhead.
+
+```
+
+## Troubleshooting
+
+* **OTLP not showing** → Check Traefik `tracing` config + ports `4317/4318`
+* **Logs not loading** → Ensure Traefik logs in JSON, volume mounted correctly
+* **GeoIP errors** → Verify `MAXMIND_LICENSE_KEY` and DB path
+* **Maxmind download errors** → Check license key validity by trying url with key directly in your browser
+* **Docker logs for Traefik not showing** →  Use `docker exec -it traefik tail -f /var/log/traefik/traefik.log` instead of `docker logs traefik -f` to see real-time logs on the command line.
+
+---
+